本篇博文主要内容为 2026-11-19 从Arxiv.org论文网站获取的最新论文列表,自动更新,按照NLP、CV、ML、AI、IR、MA六个大方向区分。
说明:每日论文数据从Arxiv.org获取,每天早上12:30左右定时自动更新。
提示: 当天未及时更新,有可能是Arxiv当日未有新的论文发布,也有可能是脚本出错。尽可能会在当天修复。
目录
概览 (2026-11-19)
今日共更新913篇论文,其中:
- 自然语言处理共72篇(Computation and Language (cs.CL))
- 人工智能共230篇(Artificial Intelligence (cs.AI))
- 计算机视觉共224篇(Computer Vision and Pattern Recognition (cs.CV))
- 机器学习共219篇(Machine Learning (cs.LG))
- 多智能体系统共18篇(Multiagent Systems (cs.MA))
- 信息检索共13篇(Information Retrieval (cs.IR))
- 人机交互共23篇(Human-Computer Interaction (cs.HC))
多智能体系统
[MA-0] Vibe-Trading: Towards Agent -Native Trading
【速读】:该论文旨在解决现有生成式AI(Generative AI)代理在动态环境(如金融交易场景)中难以维持可靠状态与上下文一致性的问题。其核心挑战在于,金融市场的价格、新闻、市场条件、投资组合持仓及可用资金等要素持续变化,导致生成结果若无法精确关联时间点、数据快照与执行上下文,则可能失去有效性甚至引发风险。为应对这一问题,论文提出Vibe-Trading——一个原生面向交易的智能体框架与一体化工作空间,其关键解决方案在于构建了一种贯穿“研究到执行”全流程的可追溯架构:所有结果均保留其来源、时间范围、配置参数及生成运行记录;通过共享证据基础实现结果验证,版本化研究记录支持可复现的延续性分析,同时引入显式的权限控制门限,将策略研究与实时执行严格分离。该设计共同构成了一个可追踪、可复现、仅在明确授权下方可执行的交易智能体实践基础。
链接: https://arxiv.org/abs/2611.24845
作者: Haozhe Wu,Zhonghang Li,Chao Huang
机构: The University of Hong Kong(香港大学); https://github.com/HKUDS/Vibe-Trading
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multiagent Systems (cs.MA); Computational Finance (q-fin.CP); Trading and Market Microstructure (q-fin.TR)
备注: Technical report. 16 pages, 8 figures, 4 tables. Code: this https URL
Abstract:Existing AI agents can retrieve information, call tools, run experiments, and produce multi-step outputs. Yet most are built around relatively static workflows and struggle to preserve reliable state as the environment changes. Financial trading makes this limitation especially consequential: prices, news, market conditions, portfolio positions, and available capital evolve continuously. A result is useful only when it remains tied to the correct time, data snapshot, and execution context. We present Vibe-Trading, a comprehensive agent-native trading framework and all-in-one workspace for building, evaluating, and evolving agents for financial trading. It unifies market research, financial data, strategy generation, backtesting, portfolio analysis, paper trading, multi-agent collaboration, and optional broker execution. Its central contribution is an architecture that keeps results, evidence, research history, and trading authority connected throughout the research-to-action process. Each result retains its source, temporal scope, configuration, and producing run. Shared evidence grounding enables verification, versioned research records support reproducible continuation, and an explicit authority gate separates strategy research from live execution. Together, these designs provide a practical foundation for trading agents whose results can be traced, reproduced, and acted on only under explicit authority.
[MA-1] Kaer: Security-Aware Replanning for Multi-Agent Systems
【速读】:该论文旨在解决多智能体系统(Multi-Agent Systems, MAS)中动态重规划(replanning)阶段的安全漏洞问题。现有架构依赖协调器(orchestrator)根据智能体反馈动态调整计划,但这一过程易受恶意智能体攻击:后者可通过在反馈中嵌入指令,操纵重规划流程以达成非法目标。传统防御措施仅关注工具访问控制或静态预执行策略,无法有效防护重规划阶段的动态威胁。本文提出Kaer框架,其核心解决方案在于实现安全感知的重规划机制:通过抽象故障模板(abstract failure templates)向重规划模块传递诊断信息,严格限制反馈的表达形式;所有计划修改被限定于一组确定性修复算子,并确保更新后的计划始终满足预设策略约束。为评估系统安全性与有效性,作者构建了Cintra基准测试集,包含需重规划的模拟用户任务及针对重规划阶段的对抗性目标。实验表明,相较于未受保护的Magnetic-One和当前最优防御方案ControlValve,Kaer在保持与ControlValve相当的任务完成率的同时,显著提升了对重规划攻击的鲁棒性,证明了其在不暴露协调器于不可信反馈的前提下,实现安全、动态计划修订的可行性。
链接: https://arxiv.org/abs/2611.24839
作者: Tushin Mallick,Evan Rose,Matthew Laws,Alina Oprea,Cristina Nita-Rotaru
机构: Northeastern University (东北大学)
类目: Multiagent Systems (cs.MA); Machine Learning (cs.LG)
备注:
Abstract:One popular architecture for multi-agent systems (MAS) relies on an orchestrator that generates an initial plan, coordinates its execution, and dynamically revises the plan based on agent feedback when tasks cannot be fully specified in advance. This dynamic replanning introduces a critical vulnerability: malicious agents can embed instructions within their feedback to hijack the replanning process. Existing defenses focus exclusively on per-tool access control or static, pre-execution policy enforcement, leaving the replanning phase unsecured. We introduce Kaer, a multi-agent framework featuring security-aware replanning. Kaer shields the planner from untrusted agent feedback by conveying diagnostics to the replanning procedure strictly through abstract templates. Plan revisions are restricted to a set of deterministic repair operators, and all modified plans must satisfy preset policies. We create Cintra, a benchmark of simulated user tasks requiring replanning paired with adversarial objectives targeting the replanning phase. Using Cintra, we evaluate both task completion rates and system vulnerability to replanning exploits by malicious agents. Evaluating Kaer on Cintra against Magentic-One (an undefended MAS) and ControlValve (the state-of-the-art defense) demonstrates that Kaer achieves higher robustness than both systems while maintaining comparable utility to ControlValve. Although abstract failure templates slightly constrain utility on select tasks, Kaer enables MAS to perform secure, dynamic plan revisions without exposing the orchestrator to untrusted agent feedback. Subjects: Multiagent Systems (cs.MA); Machine Learning (cs.LG) Cite as: arXiv:2611.24839 [cs.MA] (or arXiv:2611.24839v1 [cs.MA] for this version) https://doi.org/10.48550/arXiv.2611.24839 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[MA-2] CaDO: Formal Verification-Guided Multi-Agent Framework for Causal Dataset Repair and Generation
【速读】:该论文旨在解决现有因果推理数据集普遍存在的标注错误与结构失衡问题,这些问题会导致模型产生偏差预测并限制其泛化能力。其核心解决方案是提出一种名为CaDO(Causal-Engine-Guided Multi-Agent Data Optimization)的多智能体框架,通过三个关键阶段实现数据的自动诊断与修复:首先,由Formalizer Agent将自然语言问题转化为形式化因果查询,以减少语义偏差并提升可检索性与可验证性;其次,采用双层诊断机制,利用独立的Critic Agent识别样本级错误,并结合查询类型覆盖度与标签分布分析检测数据集层面的结构性失衡;最后,基于形式化因果语义的可复用修复算子对错误样本进行修正,同时由Explorer与Expander Agent协作生成多样化且语义一致的新样本。实验表明,CaDO在18项测试中的16项显著优于五种基线数据清洗方法,下游模型准确率最高提升9.1%;且至少95%被标记的样本确实存在缺陷,其在CLadder数据集上误修改正确标签的比例比基于大语言模型的判别器低92.2%,充分体现了该框架在准确性与可靠性方面的优势。
链接: https://arxiv.org/abs/2611.24406
作者: Songli Yu,Qi Luo
机构: Shandong University (山东大学)
类目: Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)
备注:
Abstract:High-quality causal reasoning datasets are essential for developing models with reliable causal understanding, particularly in specialized domains. However, existing datasets frequently suffer from incorrect annotations and structural imbalances, which can lead to biased predictions and limited generalization. To address this issue, we propose CaDO (Causal-Engine-Guided Multi-Agent Data Optimization), a multi-agent framework for automated diagnosis and repair of causal reasoning datasets. Our framework consists of three key stages: formal causal reasoning, multi-level diagnosis, and automated data repair and generation. First, a Formalizer Agent translates natural language questions into formal causal queries to minimize semantic deviations and make retrieval and verification more explicit and tractable. Second, a dual-level diagnosis mechanism employs independent Critic Agents to identify sample-level errors while analyzing query-type coverage and label distributions to detect dataset-level structural imbalances. Finally, the framework employs reusable repair operators grounded in formal causal semantics to correct erroneous samples, while Explorer and Expander Agent collaboratively produce diverse and semantically consistent new samples. Experimental results show that CaDO significantly outperforms the strongest of five data cleaning baselines in 16 out of 18 experimental settings, improving downstream model accuracy by up to 9.1%. Moreover, at least 95% of the samples flagged by CaDO are defective, and on CLadder, CaDO incorrectly modifies 92.2% fewer correct labels than an LLM-based judge, demonstrating its effectiveness and reliability. Our code and data are publicly available at this https URL.
[MA-3] OpenKAGE: Open-Ended Lifelong Learning in the Open World via Knowledge-Sharing Agent -Group Evolution
【速读】:该论文旨在解决如何实现开放世界中持续不断的终身学习(lifelong learning)问题,特别是克服现有研究中局限于预设任务、课程或目标的局限性,以及个体智能体自我提升能力有限的瓶颈。其核心挑战在于如何使智能体群体在动态开放环境中自主探索,并通过累积与共享经验持续获得新能力。解决方案的关键在于提出OpenKAGE框架,该框架包含两个核心机制:一是双层知识表示结构,包括用于情境记忆的内存层和用于可迁移经验与技能的体验层,后者保留了技能复用所需的适用条件;二是语义化知识共享机制,将文本形式的经验进行语义整合并存入持续演化的共享知识图谱,同时以原始形式共享可执行技能。该框架在Minecraft环境中验证了其支持开放式终身学习的能力,使智能体群体能够持续探索并达成高级技术里程碑,且积累的知识可跨模型迁移到新初始化的智能体群体中,显著提升其生存与探索效率。
链接: https://arxiv.org/abs/2611.24302
作者: Wenhao Yuan,Longhui Yuan,Zexi Jia,Ningzhe Shi,Jingwei Li,Shipeng Cen,Pengcheng Luo,Jie Zhou,Jinchao Zhang
机构: 未知
类目: Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)
备注:
Abstract:Building autonomous agents that continually explore open worlds and acquire new capabilities through experience is a long-standing aspiration in artificial intelligence. Existing research often pursues lifelong learning within predefined tasks, curricula, or objectives, with self-improvement largely confined to individual agents. We introduce OpenKAGE, an agent-group evolution framework for open-ended lifelong learning. Within this framework, agents autonomously explore open worlds and continually acquire new capabilities by accumulating and reusing knowledge from their own interactions and the experience of others. OpenKAGE builds on two key mechanisms: (1) a two-layer knowledge representation, with a memory layer for situated memories and an experience layer for transferable experience and skills that retains the applicability conditions needed for reuse; and (2) a knowledge-sharing mechanism that semantically consolidates textual experience into a continuously evolving shared knowledge graph, while executable skills are shared verbatim. In Minecraft, OpenKAGE supports open-ended lifelong learning, enabling agent groups to sustain exploration and reach advanced technological milestones. Furthermore, the accumulated knowledge can be transferred to a newly initialized agent group on a different backbone model, improving its survival and exploration progress.
[MA-4] Asymmetric Decomposition for Multi-Agent Path Finding with Unassigned Agents
【速读】:该论文旨在解决多智能体路径规划中未分配任务的智能体(unassigned agents)带来的冲突与效率问题:传统方法要么因强制将无任务智能体停靠至指定停车位导致不必要的移动,要么因未分配智能体自由行动而干扰已有任务智能体的路径。为此,论文提出流-预言机冲突基础搜索(Flow-oracle Conflict-Based Search, FOCBS),其核心解决方案是将已分配任务智能体的路径搜索与未分配智能体的路径规划解耦:利用冲突基础搜索(CBS)优化已分配智能体的路径,同时通过最小费用流(min-cost flow)预言机为无目标的未分配智能体规划路径。当所选已分配智能体路径导致未分配智能体无法找到无碰撞路径时,采用“流-冲突分支”机制,将最小割阻塞者转化为已分配智能体的约束,从而引导搜索过程并保证全局最优解不被排除。此外,引入批量零成本预热(batched zero-cost warm start)加速预言机求解,并采用起始点规避的优先级策略减少未分配智能体的移动。理论上,对于有限图上的可行实例,FOCBS可保证终止性,并实现已分配智能体服务时间总和(以最晚到达时间衡量)的全局最小化;对未分配智能体的移动则在给定已分配路径下实现局部最小化,但不保证全局字典序最优。基准测试表明,FOCBS能有效扩展至大量未分配智能体场景,消融实验显示相比舍弃不可完成的已分配路径方案,流-冲突分支显著提升可解实例数量且保持最优性。仿真与真实机器人实验验证了规划方案的可执行性,实验室测试显示车队行驶距离相较基于停靠的协调基线降低40.8%。
链接: https://arxiv.org/abs/2611.24108
作者: Xi Jin,Chen Qian,Renjun Zhang,Guowei Wu,Xiaobo Li,Jun Jiang,Weiwei Shang
机构: University of Science and Technology of China (中国科学技术大学); State Key Laboratory of Precision and Intelligent Chemistry (精密智能化学国家重点实验室); Hefei National Laboratory (合肥国家实验室)
类目: Robotics (cs.RO); Multiagent Systems (cs.MA)
备注:
Abstract:Robots without assigned tasks may obstruct active robots, while routing them to designated parking locations can cause unnecessary travel. This article proposes flow-oracle conflict-based search (FOCBS) for Multi-Agent Path Finding with unassigned agents (MAPF-UA). FOCBS searches over assigned-agent paths using conflict-based search (CBS) and routes targetless unassigned agents through a min-cost flow oracle. When the selected assigned-agent paths leave unassigned agents no collision-free paths, flow-conflict branching converts min-cut blockers into assigned-agent constraints, which guide the search without excluding optimal solutions. A batched zero-cost warm start accelerates the oracle, and start-avoiding tie-breaking reduces unassigned-agent movement. For feasible instances on finite graphs, we prove termination and global minimization of assigned-agent sum of service time, measured by last-arrival times. Unassigned-agent movement is minimized conditional on the selected assigned-agent paths, without a global lexicographic optimality guarantee. Benchmark experiments show that FOCBS scales well with the number of unassigned agents, and ablation studies show that, compared with discarding non-completable assigned-agent paths, flow-conflict branching solves substantially more instances while retaining optimality. Simulation and physical-robot experiments demonstrate plan executability, with laboratory tests reducing fleet travel by 40.8% relative to the parking-based coordination baseline. The source code and related resources are available at this https URL
[MA-5] A Capability Framework for Evaluating Agent ic AI Platforms in Regulated Enterprises
【速读】:该论文旨在解决企业在采购生成式 AI (Generative AI) 平台时面临的评估困境:尽管企业正快速引入集成化智能体平台(Agentic AI platforms),但缺乏可量化、可重复的评估方法。现有公开基准测试多在固定框架下评估模型性能,而企业实际采购的是包含规划、工具调用、记忆保持与协同控制等复杂能力的完整系统,其架构设计、治理机制、工具链和内存管理等因素对最终表现起主导作用,导致评估高度依赖供应商演示,存在主观偏差。为应对这一问题,研究团队在一家全球性银行的创新实验室组织了一场工业级“竞品评测”(bakeoff),对十种平台(涵盖开源框架、专有产品及云服务商生态)在九个能力维度上,针对三个金融业务场景(对手方信用风险评估、检索增强生成、流程自动化)进行了跨职能评审。结果显示,平台表现可稳定划分为三个层级,并暴露出聚合评分掩盖的场景特异性优势。然而,二次分析揭示各维度间存在极强内部一致性(Cronbach’s alpha = 0.98)及与总分高度相关(8/9维度秩相关系数达0.93),表明评估结果可能主要反映演示呈现质量而非真实能力,即存在显著“光环效应”。基于此,论文提出“先准入后评分”的四层评估协议:包括能力、执行可靠性、治理准入、业务价值四个层级;引入基于多次运行的细粒度可靠性指标;建立自主性就绪度量表;并制定可复现的评估规程。该方案明确界定了当前证据所能支持的结论边界,并强调唯有通过受控实验环境才能将企业级平台评估从主观判断转变为可测量的科学评估。
链接: https://arxiv.org/abs/2611.24039
作者: Abbas Raza Ali,Muhammad Ajmal Siddiqui,Moona Zahid
机构: 未知
类目: Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA); Machine Learning (stat.ML)
备注:
Abstract:Agentic AI platforms-systems in which LLM agents plan, invoke tools, retain memory and coordinate to change the state of enterprise systems-are being procured faster than they can be measured. Public agent benchmarks score models in fixed harnesses; enterprises buy integrated platforms whose orchestration, memory, governance and tooling layers dominate outcomes, and they buy them largely on the strength of vendor demonstrations. We report an industrial evaluation programme (“bakeoff”) run by the innovation laboratory of a global bank in which ten platforms-spanning open-source frameworks, proprietary products and hyperscaler ecosystems-were assessed by a cross-functional panel on nine capability dimensions across three financial-services workloads: counterparty credit risk, retrieval-augmented generation and process automation. Scores separate the field into three tiers that are stable under alternative weightings, and surface workload-specific strengths that aggregates hide. A secondary analysis, however, finds Cronbach’s alpha = 0.98 across nominally distinct dimensions and rank correlations = 0.93 with the total for eight of nine: a halo effect indicating that demonstration-based scoring measures presentation as much as capability. From these lessons we propose a gate-then-score evaluation protocol with four layers (capability, execution reliability, governance gates, business value), trace-level reliability metrics estimated over repeated runs, an autonomy-readiness scale and a reproducibility protocol. We state precisely what the present evidence supports, and what a controlled harness must add to turn enterprise platform evaluation from structured judgement into measurement.
[MA-6] FormalFixer: Intent-Aligned Automated Hardware Repair via Joint Refinement of Specifications Assertions and RTL Implementations
【速读】:该论文旨在解决硬件设计中形式化属性验证(Formal Property Verification, FPV)失败时的修复难题,尤其针对因断言(assertion)错误或规格说明(specification)语义模糊/不完整导致的误报问题。现有以寄存器传输级(RTL)为中心的修复方法无法有效协调规格意图、验证意图与RTL行为之间的一致性,从而难以准确判断应修改哪个设计元素。其解决方案的关键在于提出一种多阶段、意图对齐的修复工作流——\textscFormalFixer,该方法将原始规格说明作为权威基准,并通过置信度过滤后的细化内容来显式澄清隐含约束;同时利用意图不匹配分析与验证器生成的反例锥形图(counterexample cone graphs)双重引导迭代式RTL修补过程,明确检查规格—断言—RTL三者之间的对应关系,从而实现基于FPV的硬件修复中的意图对齐。
链接: https://arxiv.org/abs/2611.24037
作者: Changyuan Yu,Hongce Zhang
机构: The Hong Kong University of Science and Technology (Guangzhou)
类目: Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI)
备注:
Abstract:Functional errors in hardware designs can compromise correctness and security, and must be fixed before tape-out. Formal property verification (FPV) provides strong guarantees by proving or refuting assertions, but a failed assertion is not always evidence of an RTL defect: the failure may also come from a buggy assertion or from a specification that is semantically correct but too ambiguous or incomplete for direct formalization. This creates a practical repair problem that existing RTL-centric methods do not directly address: the repair loop must align specification intent, verification intent, and RTL behavior before deciding which artifact to change. \textscFormalFixer addresses this problem with a multi-stage, intent-aligned repair workflow for Verilog designs with assertions. It preserves the original specification as authoritative ground truth and appends confidence-filtered refinements to clarify implicit constraints. \textscFormalFixer uses both intent mismatch and verifier-generated counterexample cone graphs to guide iterative RTL patching. This workflow explicitly checks specification–assertion–RTL correspondence. Results support intent alignment for FPV-based hardware repair. Subjects: Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI) Cite as: arXiv:2611.24037 [cs.MA] (or arXiv:2611.24037v1 [cs.MA] for this version) https://doi.org/10.48550/arXiv.2611.24037 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[MA-7] SCAFFOLD: Toward Warrant-Preserving Evidence Transport for Reusable LLM -Agent Skills
【速读】:该论文旨在解决在动态环境中,智能体技能(agent skill)即使在过去被验证有效,也可能因上下文变更、权限调整或规范错误而不再适配当前使用场景的问题。其核心挑战在于:传统验证机制往往仅关注可观察的契约条款或结果断言(outcome oracle),缺乏对历史证据在上下文变化后是否仍具可采性的系统性判断,导致错误重用风险。解决方案的关键是提出“有理由的重用主张”(Warranted Reuse Claim, WRC),即一种可反驳的、与特定版本技能、目标上下文、证据、假设及已知反例(defeaters)绑定的授权声明。进一步,论文引入“担保保持的证据传输”(Warrant-Preserving Evidence Transport, WPET)机制,明确区分证据可采性与所支持的安全性/有效性主张的真实性,提供三值适用性判断与非对称上下文转移规则,并揭示证明溯源依赖性与策略失效变更如何影响授权状态。该方法不追求对现有保障论证(assurance cases)、合同或技能库维护模式的颠覆性创新,而是通过形式化问题定义、可检验的设计义务与报告协议,为未来构建具备版本感知能力的可信重用体系提供理论基础与评估框架。
链接: https://arxiv.org/abs/2611.23990
作者: Nami Cho,Shailja Gupta
机构: 未知
类目: Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA); Software Engineering (cs.SE)
备注: 8 pages, 2 figures, 4 tables
Abstract:An agent skill that succeeded yesterday may be executable today without being justified for reuse today. Tests may cover only observable contract clauses, outcome oracles may be absent, evaluations may inherit the same erroneous specification, and permissions or dependencies may change between validation and invocation. We argue that the appropriate unit of prospective trust is a Warranted Reuse Claim (WRC): a defeasible, claim-specific authorization tied to a versioned skill, target context, evidence, assumptions, and identified defeaters. The paper sharpens this position through Warrant-Preserving Evidence Transport (WPET), a proposed discipline for deciding which historical evidence remains admissible after context change. It distinguishes evidence admissibility from truth of the supported safety or effectiveness claim; states a conditional preservation result and its limits; specifies three-valued applicability and asymmetric context-transfer judgments; and illustrates how provenance dependence and policy-invalidating changes alter authorization. We explicitly relate the proposal to assurance cases, contracts, and skill-library maintenance rather than claiming novelty for their ingredients. A falsifiable longitudinal evaluation would compare WRC-guided policies with strong version-aware baselines at matched coverage, utility, and review cost. This paper contributes a formalized problem statement, testable design obligations, and reporting protocol, not an implemented verifier, empirical superiority result, or proof of universal agent safety.
[MA-8] Resilient Federated LQR over Noisy and Lossy Uplinks via Scalar Communication
【速读】:该论文旨在解决在通信受限(包括带宽有限、数据包丢失和加性噪声)的上行链路环境下,如何实现对一组相似系统协同优化的无模型联邦学习(model-free federated learning)问题,具体目标是设计一个能够稳定收敛且高效通信的LQR(Linear Quadratic Regulator)控制器联合学习框架。其核心挑战在于,在资源受限条件下,如何在保证系统稳定性与控制性能的前提下,减少通信开销并应对异构性、采样误差与通信扰动的影响。解决方案的关键在于提出一种名为 \textscResiFedLQR 的标量通信机制,即每个智能体每轮仅发送一个实数(而非完整的梯度向量),通过冗余传输提升鲁棒性;该标量可为梯度坐标、基于共享随机种子在服务器端复现的随机方向上的梯度投影,或直接为轨迹代价(rollout cost)。对于投影编码方式,论文证明了在高概率意义下,所有迭代均能稳定各智能体,且群体平均代价以线性速率收敛至一个由异质性、采样误差和通信扰动共同决定的邻域内,并据此推导出基于信道参数与车队规模的最优传输次数选择规则。进一步地,通过与全梯度传输在相同上行预算下的对比分析,识别出在不同信噪比环境及功率约束下,何种编码方式更具优势。数值实验在异构车队场景中验证了所提方法在不同噪声水平与丢包率下的稳定性与收敛性表现。
链接: https://arxiv.org/abs/2611.23913
作者: Mohammadreza Rostami,Shahriar Talebi,Solmaz S. Kia
机构: University of California, Irvine (加州大学欧文分校); University of California, Los Angeles (加州大学洛杉矶分校)
类目: Multiagent Systems (cs.MA)
备注:
Abstract:This paper studies model-free federated learning of a common LQR controller for a fleet of similar systems that communicate with a server over an uplink subject to limited bandwidth, packet dropout, and additive noise. We propose \textscResiFedLQR, a scalar communication scheme in which each agent sends one real number per packet, with redundancy, instead of full gradient estimates. The scalars can be gradient coordinates, projections of the gradient onto random directions reproduced at the server from a shared seed, or the rollout costs themselves. For the projection encoding, we prove that, with high probability, every iterate stabilizes all agents and the fleet-average cost converges linearly to a neighborhood of the optimum whose radius separates heterogeneity, sampling, and communication effects, and we derive a rule for choosing the number of transmissions from the channel parameters and the fleet size. We then compare the encodings with full-gradient transmission under the same uplink budget and identify the signal-to-noise regime and power constraint under which each is preferable. Numerical experiments on a heterogeneous fleet evaluate stability and convergence under varying noise levels and dropout probabilities.
[MA-9] CoSlice: Coordinated Multi-Agent Reinforcement Learning for 5G Network Slicing Allocation
【速读】:该论文旨在解决开放无线接入网(O-RAN)中多切片间物理资源块(PRB)分配的协同优化问题,其核心挑战在于:各切片需在满足服务等级协议(SLA)的前提下最小化资源消耗,而所有切片又共享有限的无线资源,由此引发切片内部的SLA保障与资源效率之间的权衡,以及切片间的资源竞争冲突。为应对这一复杂动态博弈场景,论文提出CoSlice框架,其关键创新在于采用“偏好条件化的切片智能体”实现快速局部决策,并引入基于大语言模型(LLM)的协调器,在较慢时间尺度上利用全局网络状态信息动态调整各切片的目标偏好,从而实现跨切片的协同控制。该方法被形式化为多目标部分可观测随机博弈(mo-POSG),证明了偏好相关均衡的存在性,并给出了子线性后悔率的理论保证。通过两阶段训练(多目标预训练与联合优化),结合ORANSlice与射频仿真器(RFsim)构建的真实协议栈与业务流量环境进行验证,实验表明CoSlice在多种切片负载下均显著降低了平均及尾部的SLA违规率,优于现有集中式与多智能体基线方法。
链接: https://arxiv.org/abs/2611.23909
作者: Pengxin Wang,Nakjung Choi,Jingdi Chen
机构: University of Arizona (亚利桑那大学); Nokia Bell Labs (诺基亚贝尔实验室)
类目: Networking and Internet Architecture (cs.NI); Machine Learning (cs.LG); Multiagent Systems (cs.MA)
备注:
Abstract:Radio Access Network (RAN) slicing is a shared resource allocation problem: each slice seeks to satisfy its Service Level Agreement (SLA) with minimal radio-resource consumption, while all slices compete for a limited resource budget. This creates both an intra-slice trade-off between SLA assurance and resource efficiency and an inter-slice conflict over shared resources, calling for specialized yet coordinated control. In this work, we present CoSlice, a coordinated multi-agent reinforcement learning framework for inter-slice physical resource block allocation in Open Radio Access Network (O-RAN). CoSlice uses preference-conditioned slice agents for fast resource allocation and an LLM-based coordinator to adapt their objectives using global network observations at a slower timescale. We formulate the problem as a multi-objective partially observable stochastic game (mo-POSG), establish the existence of a Preference Correlated Equilibrium, and derive a sublinear regret bound for the learning procedure. We implement the algorithm by using a two-stage training procedure that consists of multi-objective pretraining and joint optimization. We build a network slicing evaluation environment using ORANSlice and its radio-frequency simulator (RFsim), supporting realistic protocol-stack execution and application traffic across multiple slices. Experimental results show that CoSlice reduces both average and tail SLA violations compared with other centralized and multi-agent baselines under diverse slice workloads.
[MA-10] A Comparison of Heuristic Human-Sensitive and Self-Play DRL Policies in Human-Agent Teaming
【速读】:该论文旨在解决在人机协作任务中,如何使人工智能代理(AI Agent, AA)生成与人类行为模式一致的动态感知-运动基元(Dynamical Perceptual-Motor Primitives, DPMP)动作序列的问题。传统方法依赖从人类行为中提取的启发式规则来指导目标选择与动作排序,虽在结构化任务中有效,但在复杂情境下泛化能力有限。为此,研究探索了基于深度强化学习(Deep Reinforcement Learning, DRL)的决策策略以自动学习人机交互中的行为模式。其解决方案的关键在于设计一种“人类敏感型”深度强化学习策略(Human-Sensitive DRL, HS-DRL),该策略通过与一个启发式代理共同训练,确保学习到的策略保持与人类行为的一致性。实验结果表明,相较于自对弈训练的DRL策略(SP-DRL)所表现出的显著偏离,HS-DRL策略在协调模式上更接近典型的人类-人类协作行为,而启发式策略仍表现最佳。这表明,在构建面向人机协同的智能体时,引入人类先验知识进行训练至关重要,且简单启发式策略在保证人类对齐性方面仍具有效性和实用性。
链接: https://arxiv.org/abs/2611.23747
作者: Ayman bin Kamruddin,Christopher Lam,Gaurav Patil,Mirco Musolesi,Mario di Bernardo,Michael J. Richardson
机构: Scuola Superiore Meridionale(南意大利高级学院); University of Naples Federico II(那不勒斯费德里科二世大学); Macquarie University(麦考瑞大学); University College London(伦敦大学学院)
类目: Multiagent Systems (cs.MA); Human-Computer Interaction (cs.HC)
备注: 9 pages, 5 figures. Includes Supplementary Material A (1 page, 0 figures), B (1 page, 0 figures), and C (54 pages, 216 figures). Submitted to IEEE HMS for possible publication
Abstract:Dynamical Perceptual-Motor Primitives (DPMPs) offer a tractable framework for generating human-like movement dynamics in artificial agents (AAs) and have been used to support coordination in human-AI teaming tasks. Traditionally, DPMP controllers rely on heuristic (rule-based) decision policies derived from human behavior to guide goal selection and action sequencing. Although effective in structured tasks, such heuristics can be difficult to generalize to more complex settings, motivating interest in deep reinforcement learning (DRL) approaches for learning decision policies for human-AI interaction. A key challenge, however, is ensuring that DRL policies remain aligned with human strategies, particularly when agents are trained through self-play. To examine this issue, we compared three target-selection policies governing DPMP-based AAs in a first-person herding task: (1) a human-derived heuristic policy, (2) a self-play DRL policy (SP-DRL), and (3) a human-sensitive DRL policy (HS-DRL) trained alongside a heuristic agent. Results showed that heuristic agents most closely matched typical human-human coordination patterns, followed by HS-DRL agents, whereas SP-DRL agents deviated most. These findings highlight the importance of human-informed training when developing DRL-based teammates and suggest that simple heuristic policies may remain an effective approach for designing human-aligned AAs capable of human-AI collaboration.
[MA-11] wo is better than one: optimizing binary rating scales for online platforms
【速读】:该论文旨在解决在线平台中用户评分普遍存在的虚高且信息量不足的问题。其核心挑战在于如何通过优化数据收集机制,提升评分系统对物品真实质量的区分能力。解决方案的关键在于引入多尺度评分设计:首先,证明将单一评分尺度拆分为两个子尺度可始终改善物品排序效果;其次,形式化地揭示了最优多尺度设计应表现为一系列阈值等距分布的0-1阶跃函数;最后,提出一种自适应模型,允许后续评分尺度根据前序用户的反馈动态调整,并基于物品的完整响应序列进行排序,结果表明自适应机制能带来指数级增强的区分能力。总体而言,该研究证明,采用多种评分尺度显著提升了平台上的物品排序质量。
链接: https://arxiv.org/abs/2611.23626
作者: Liren Shan,Alec Sun,Jingyan Wang
机构: Toyota Technological Institute at Chicago(丰田理工学院芝加哥分校); University of Chicago(芝加哥大学)
类目: Multiagent Systems (cs.MA); Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG)
备注: 32 pages
Abstract:Online platforms often struggle with inflated, uninformative user ratings. This issue can be mitigated by better design of the data collection mechanism, for example through changing the phrasing of user questions. Previous work optimizes a single rating scale to best distinguish the inherent qualities of items under binary feedback. In this paper, we demonstrate the power of utilizing different rating scales for different users. First, we show that a designer can always split a single rating scale into two to improve item rankings. Next, we formally characterize the optimal design of multiple scales as a series of 0-1 step functions whose threshold values are evenly spaced. Finally, we consider an adaptive model where subsequent rating scales can depend on the responses from previous users and items are ranked by their full sequence of responses, and show that adaptivity leads to exponentially greater distinguishing power. All in all, our theoretical results show that using multiple rating scales significantly improves item rankings on platforms.
[MA-12] MANGO Arena: Same Seeds Different Chains Evaluating LLM Agents in a Disrupted Supply Chain
【速读】:该论文旨在解决多智能体系统中大型语言模型(LLM)在复杂供应链环境中行为评估的可靠性问题,尤其关注其在面对动态需求与突发扰动时的经济效率、规则遵守能力及潜在合谋倾向。其核心挑战在于现有评估方法难以有效区分智能体的真实策略表现与由API调用失败或推理过程异常引入的噪声。解决方案的关键在于构建一个开放的测试平台——MANGO Arena,该平台基于真实课堂工厂管理游戏MANGO设计,模拟包含四个代理(分别代表零售商、制造商、供应商等)的串行供应链,并通过硬性约束(如运输车辆数量限制、无信用现金约束、生产容量上限)强化环境规则,同时设置公开消息板以禁止隐性利益交换。该设计使得同一环境可同步度量三大关键指标:供需波动放大效应(即“牛鞭效应”)、压力下的规则遵循情况以及合谋提议行为。实验结果显示,尽管所有LLM代理均显著放大了上游需求波动(在供应商端放大达13至179倍),远超经典订货策略,但实际违规行为极少,且未出现任何形式的金钱侧支付提议。然而,研究揭示了一个关键问题:API调用失败(如速率限制、隐藏推理失控)被错误地记录为“无订单”决策,导致评估结果严重失真;在相同初始条件下重跑实验后,上游牛鞭效应的变化幅度可达17倍之多,表明部分结论不具备可重复性。因此,论文强调必须建立透明、可复现的多智能体评估报告规范,包括公开代码、提示模板和全部日志记录,以提升评估可信度。
链接: https://arxiv.org/abs/2611.23613
作者: Sara Amar
机构: Bowling Green State University (鲍灵格林州立大学)
类目: Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI)
备注: 8 pages, 2 figures,
Abstract:We introduce MANGO Arena, an open-source testbed in which four LLM agents each run one company in a serial supply chain adapted from MANGO, a classroom factory-management game. The environment enforces hard rules (a truck limit, a no-credit cash constraint and production capacity), logs every blocked attempt to break them, and gives agents a public message board on which side payments are explicitly forbidden. This lets one environment measure three things at once: an economic outcome (the bullwhip effect), rule adherence under pressure, and proposals to collude. In a pilot with three small reasoning models, two scripted reference policies, two conditions (with and without a supplier fire) and three demand seeds, all LLM agents amplified demand variability as it moved upstream, by 7 to 25 times at the Retailer and 13 to 179 times at the Supplier, often more than a textbook order-up-to policy. They did so largely by ordering in batches (zero orders in up to 60% of decisions). Real rule violations were rare (at most 7 of 288 decisions per model) and no agent proposed a money side deal. The pilot’s main lesson concerns evaluation method. API failures (rate limits and runaway hidden reasoning) entered the metrics silently as “no order” decisions. And when we reran games with the same seeds and no failed calls, upstream bullwhip changed by up to 17 times between runs. With three seeds, conclusions about how a disruption affects a given model did not survive a rerun. We report these failures in full and propose reporting practices for multi-agent evaluations of this kind. Code, prompts and every logged decision are public.
[MA-13] ARISE: Practical Self-Evolving Memory for Accurate and Cost-Efficient Enterprise Agents
【速读】:该论文旨在解决企业级大语言模型(LLM)代理在动态环境中面临的持续性能衰减问题,即基础模型迭代、工作流频繁更新以及部署需兼顾准确性与成本约束的挑战。其核心解决方案是提出一种名为ARISE(Agent Refinement through Iterative Self-Evaluation)的可迭代自评估记忆架构,通过代理自身对失败进行归因并生成修正策略,无需更新模型权重、工具或工作流逻辑,即可实现性能提升。ARISE的关键在于:利用模型自身或教师模型对执行失败进行诊断并提出修正方案,将这些修正以可检索的微记忆形式存储,并进一步归纳为宏观规则;仅当保留集上的准确率提升时才接受规则更新,从而确保改进的可靠性。实验结果表明,在SOP-Bench和τ²-bench共14个企业工作流领域上,ARISE-Teacher相较于基线分别提升10.9个百分点和5.8个百分点,且在相同任务成本下减少4.0%的模型调用次数与5.2%的输出令牌数,验证了其高效性与实用性。
链接: https://arxiv.org/abs/2611.23588
作者: Jiangyun Zhang,Zhou Tang,Kristen Surrao,Zhiyu Chen,Albert Guan,Tor Nitayanont,Yupei Zhang,Roopali Singh,Yunkai Xiao,Shreya Pandit,Julia Huang,Shayan Ali Akbar,Erwin Cornejo,Yuan Li,Yi Zhang
机构: Amazon(亚马逊)
类目: Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)
备注: 16 pages (8 main), 7 figures, 12 tables
Abstract:Enterprise LLM agents operate in dynamic environments where foundation models evolve rapidly, workflows update frequently, and deployments need to satisfy accuracy and cost requirements. We propose ARISE (Agent Refinement through Iterative Self-Evaluation), a practical memory architecture in which the agent’s memory evolves from its own failures without updating model weights, tools, or workflow logic. ARISE uses the model itself or a teacher model to attribute the failures and propose corrections. Then, ARISE keeps the corrections as retrievable micro memory, consolidates them into macro rules, and accepts a rule update only if held-out accuracy rises. Across 14 enterprise-workflow domains from SOP-Bench and \tau^2 -bench using 5 models, ARISE-Teacher, which uses Claude Opus 4.6 for attribution and consolidation, gains 10.9 pp over the baseline on SOP-Bench and 5.8 pp on \tau^2 -bench. ARISE-Teacher also outperforms GEPA by 4.5 pp on \tau^2 -bench and 1.4 pp on SOP-Bench. With the deployment model as its own attributor, ARISE-Self still gains 10.0 pp on SOP-Bench. ARISE-Teacher uses 4.0% fewer model calls and 5.2% fewer output tokens at comparable per-task cost (-1.0%).
[MA-14] Encapsulating Reactive Behaviour in Goal-based Plans for Programming BDI Agents
【速读】:该论文旨在解决基于信念-欲望-意图(Belief-Desire-Intention, BDI)的智能体编程中,反应式行为通常以与特定目标无关的反应式计划(reactive plans)形式进行描述所导致的灵活性与可维护性不足的问题。其核心解决方案在于提出一种扩展的计划模型,将目标驱动的计划(goal-based plans)引入BDI编程语言的计划体系,使同一计划既能表达主动行为(proactive behavior)又能封装反应式行为(reactive behavior)。这一关键改进不仅提升了智能体编程的实际便利性,还增强了运行时推理能力,支持更高效、一致的行为协调。通过在Jason和ASTRA两个现有智能体编程平台上的具体实现与实验验证,结果表明该方法在编程优雅性与执行效率方面均具有显著优势。
链接: https://arxiv.org/abs/2611.23496
作者: Rafael H. Bordini,Rem Collier,Jomi F. Hübner,Alessandro Ricci
机构: PUCRS(天主教大学); University College of Dublin(都柏林大学学院); Federal University of Santa Catarina(圣卡塔琳娜联邦大学); University of Bologna(博洛尼亚大学)
类目: Multiagent Systems (cs.MA)
备注:
Abstract:Reactive behaviour in BDI-based models and architectures adopted in agent programming is typically specified in terms of reactive plans not bound to any specific goal. In this paper, we present and discuss an extension of the plan model used in BDI programming languages in which goal-based plans encapsulate both proactive and reactive behaviour. This brings important benefits both to the practice of agent programming and in supporting agent reasoning at runtime. The paper first introduces the model formally, abstracting away from particular programming languages. The approach is then evaluated through concrete implementations based on two existing agent programming platforms, namely Jason and ASTRA. The paper reports on experiments showing that the approach provides for elegant programming and can also be efficient.
[MA-15] Verifiably Safe Decentralized Control via Event-Triggered Topology Switching
【速读】:该论文旨在解决多智能体系统在估计不确定性下,如何实现安全、高效且间歇性通信的问题。核心挑战在于,在通信资源受限的环境中,各智能体需在保持系统状态安全的前提下,自主决定何时连接或断开通信,同时避免Zeno行为(即无限频繁地触发通信事件)。解决方案的关键在于提出一种去中心化的事件触发型拓扑切换协议,其遵循“安全驱动”的触发原则:智能体在本地计算可验证的安全证书,并在证书接近安全边界前主动重新连接,仅在恢复至更严格的安全阈值以下时才允许断开连接。该机制确保了估计误差始终被约束在可认证的安全范围内,同时实现了通信的间歇性与系统的安全性。通过四水箱系统的仿真实验验证,该协议能够在大部分运行时间内允许智能体断开连接的同时,始终保持各水箱液位偏差在安全限值内。
链接: https://arxiv.org/abs/2611.23415
作者: Amber He Wei,Shiyu Cheng,Andrew Clark,Bruno Sinopoli
机构: Washington University in St. Louis (圣路易斯华盛顿大学); Arizona State University (亚利桑那州立大学)
类目: ystems and Control (eess.SY); Multiagent Systems (cs.MA)
备注:
Abstract:This paper proposes a decentralized event-triggered topology-switching protocol for multi-agent systems under estimation uncertainty. We consider an intermittent communication setting in which each agent switches between connected and disconnected modes and makes communication decisions using locally computable certificates, while the closed-loop state is required to remain safe. The proposed protocol follows a safety-driven triggering principle in which agents reconnect before their certificates reach the safety boundary and disconnect only after recovering to a stricter bound below the safety limit. Our design enables intermittent connectivity while maintaining the estimation error within a certified safety bound and excluding Zeno behavior. We evaluate the proposed protocol on a quadruple-tank example, showing that it keeps all tank deviations within the safety bound while allowing agents to remain disconnected for most of the run.
[MA-16] RACE-MAS: Diagnosing and Repairing Collaboration Failures in Multi-Agent Systems
【速读】:该论文旨在解决基于大语言模型(LLM)的多智能体系统(Multi-Agent System, MAS)中由智能体间交互异常引发的故障诊断与修复难题。现有防护机制仅关注单个智能体层面的异常检测,难以有效识别源于智能体间通信、任务依赖关系等交互层面的故障,且检测到的异常往往无法准确回溯至真实根源。为此,本文提出一种细粒度的故障诊断与修复框架——TRACE-MAS。其核心在于通过学习正常协作执行轨迹,建模智能体状态变化、消息流转及任务依赖演化等多维度行为模式,并基于源感知归因(source-aware attribution)机制精准定位故障责任单元;同时,通过最小化干预策略,在恢复协作能力的同时最大限度保留关键任务功能。实验结果表明,该方法在五类典型MAS基准测试中,将根因定位准确率提升60.2%,并在任务恢复性能上显著优于现有强健性保障方案。
链接: https://arxiv.org/abs/2611.23362
作者: Zekai Chen,Yihan Sun,Xunkai Li,Xun Wu,Xinmo Jin,Yinlin Zhu,Haodong Lu,Xuaner Chen,Rong-Hua Li
机构: Beijing Institute of Technology(北京理工大学); Minzu University of China(中央民族大学); Sun Yat-sen University(中山大学); Shandong University, Weihai(山东大学威海分校)
类目: Machine Learning (cs.LG); Multiagent Systems (cs.MA)
备注:
Abstract:LLM-based multi-agent systems (MAS) rely on not only individual agent behaviors but also the interactions that connect agents, including message exchanges and task dependencies. Existing MAS safeguards mainly perform agent-level anomaly detection, which becomes insufficient when failures originate from agent interactions and detected anomalies do not necessarily reveal their true sources. To address this limitation, we propose \textbfTRACE-MAS, a finer-grained framework for diagnosing and repairing MAS failures. TRACE-MAS learns normal collaborative execution patterns from benign trajectories and detects deviations in agent state changes, message transitions, and task dependency evolutions. It further identifies responsible execution elements through source-aware attribution and selects minimal interventions to restore collaboration while preserving task-critical capabilities. Experiments on five MAS benchmarks demonstrate that TRACE-MAS significantly improves root-cause localization accuracy by \textbf60.2% and achieves consistent gains in task recovery over strong MAS safeguards.
[MA-17] Agent -Native Applications: A New Application Paradigm
【速读】:该论文旨在解决传统应用架构中行为逻辑受限于预先编写的代码与工作流规则,导致应用无法应对设计时未枚举的动态场景的问题。其核心挑战在于:当基于大语言模型(Large Language Model, LLM)的智能体作为应用的核心推理与行动组件时,虽然显著扩展了应用的行为空间,但如何在保持有用自主推理能力的同时,确保应用行为始终处于可接受的合规范围内,成为一个关键的控制难题。解决方案的关键在于提出一种“原生智能体应用”(agent-native application)的新范式,将应用建模为一个可移植的、由智能体、工具、数据源及人机协同(human-in-the-loop, HITL)检查点构成的图结构,并通过声明式应用规范定义允许的行为边界(behavioral envelope)。运行时由应用编排器(orchestrator)作为控制平面,协调任务并管理图及其权限的演化;同时,智能体网格(agent mesh)作为数据平面,负责策略相关交互的中介与审计事件生成。该架构进一步引入信任层(trust layer)以实现对智能体应用的可治理性,并构建支持实际部署的基础框架。两个典型用例验证了该设计的可行性,而可执行原型Agent-Native Runtime(ANR)则展示了核心机制的实际运行效果。
链接: https://arxiv.org/abs/2611.23243
作者: David Luo,Alberto Leon-Garcia
机构: University of Toronto (多伦多大学)
类目: oftware Engineering (cs.SE); Multiagent Systems (cs.MA)
备注: 21 pages, 8 figures, 1 table, 1 code listing; published in IEEE Access
Abstract:Traditional application architectures assume behavioral logic authored in advance, leaving reachable behavior largely bounded by explicit code and workflow rules. Large language model (LLM)-based agents challenge this assumption by enabling runtime reasoning and autonomous action to become part of application behavior, allowing applications to address situations not enumerated at design time. Extending this pattern, this paper identifies a new application paradigm in which LLM-based agents serve as central reasoning and action components responsible for the application’s core logic and, where permitted, for adapting the application graph itself at runtime. We call these agent-native applications. While such applications significantly expand their possible behavioral space beyond explicit code and workflow rules, they also face a major control problem in which useful agentic reasoning should be preserved while application behavior should remain within a permissible space. We therefore propose an architectural model that represents the application as a portable graph of agents, tools, data sources, and human-in-the-loop (HITL) checkpoints, and encodes the application’s permitted behavior as a behavioral envelope within a declarative application specification. At runtime, an application orchestrator serves as the control plane that coordinates tasks and governs how the graph and its permissions evolve, while an agent mesh serves as the data plane that mediates policy-relevant interactions and produces audit events. We then discuss the trust layer that makes agent-native applications governable and the supporting foundations required for practical operation. Two use cases illustrate the architecture, while Agent-Native Runtime (ANR) demonstrates selected core mechanisms in an executable prototype.
自然语言处理
[NLP-0] Jailbreak Detection on Large Language Models via Contrastive Prototype Structures
【速读】: 该论文旨在解决大语言模型在安全部署中面临的越狱攻击(jailbreak attempts)与有害提示(harmful prompts)检测难题,核心挑战在于如何在不依赖额外下游分类器训练的前提下实现高效、鲁棒的安全检测。其解决方案的关键是提出一种基于原型推理的无监督检测方法——Jailbreak Detection with Prototype Inference (JDPI),该方法利用二元安全标签(binary safety labels)构建有害(harmful)与良性(benign)原型,并通过计算输入样本与两类原型之间的相对距离进行决策。JDPI仅需二元标签即可完成编码器微调、原型构建与验证,避免了传统分类器方法中所需的独立训练集划分,显著减少标注数据需求。实验表明,在四个涵盖不同输入格式的基准测试中,对比学习微调使JDPI的检测准确率提升9.1–18.0个百分点;在相同编码器下,其ROC-AUC性能接近甚至优于基线模型(Method A),且在特定数据集适配后,超越发布的Llama Guard 3-8B基线0.173–0.310的ROC-AUC。此外,JDPI在多轮对话场景中融合用户历史信息,实现了更高的对话级检测精度,并在12个定向迁移任务中平均高出分类器1.59个百分点,展现出更强的跨数据集鲁棒性。
链接: https://arxiv.org/abs/2611.24936
作者: Yan Qiu,Yuanyuan Li
机构: Technical University of Munich (慕尼黑工业大学); Insure AI; Munich Re; San Francisco, USA
类目: Computation and Language (cs.CL)
备注:
Abstract:Jailbreak attempts and harmful prompts remain a challenge for the safe deployment of large language models. We study whether contrastively learned representations can support safety detection without training an additional downstream classifier. We propose Jailbreak Detection with Prototype Inference (JDPI), which uses binary safety labels to construct harmful and benign prototypes and makes predictions from relative prototype distances. JDPI uses only binary safety labels for encoder adaptation, prototype construction, and validation. In our setup, it uses fewer labeled examples than the classifier baseline (Method A) by avoiding the separate split used to train the downstream classifier. Across four benchmarks covering different input formats, contrastive fine-tuning improves JDPI’s detection accuracy by 9.1-18.0 percentage points. Using the same encoder, JDPI achieves slightly higher ROC-AUC on one benchmark and stays within 0.017 of the baseline across all four. With dataset-specific adaptation, JDPI exceeds the released Llama Guard 3-8B baseline by 0.173-0.310 in ROC-AUC across the four benchmarks. In multi-turn replay on TomGibbs, JDPI uses accumulated user history as the conversation unfolds, achieving higher conversation-level accuracy than when it uses the current turn alone. Across 12 directed transfers, JDPI outperforms the classifier by an average of 1.59 percentage points in accuracy, demonstrating stronger cross-dataset robustness.
[NLP-1] Ground Iterate Trace: Decomposing AI Research Ideation
【速读】: 该论文旨在解决生成式人工智能在科研创意生成中缺乏可解释性与过程透明度的问题,具体聚焦于理解自动化科研创意生成的有效机制,以及新颖研究提案如何从已有科学文献中演化而来。其核心解决方案是提出“创造性AI研究员”(Creative AI Researcher, CAR)框架,该框架通过融合文献基础、迭代式开发与显式的概念操作算子,构建一个可追溯的生成过程。CAR利用检索到的文献构建语义知识图谱,并赋予创意代理以图谱检索工具及四种概念操作算子:双关联想(bisociation)、类比(analogy)、探索(exploration)和深化(elaboration),每一步操作均需满足结构接受条件并被记录,从而形成包含中间转化路径与明确文献来源的可审计过程轨迹。实验结果表明,CAR在方法学新颖性(MN)上显著优于单次提示和The AI Scientist,且在可行性评分上高于其他对比方法;在多个基准测试中表现最佳。尽管消融研究显示,显式操作算子在未校正多重比较的情况下仍可能带来一定优势,但整体而言,其对质量提升的贡献并不稳定。因此,该研究强调了过程可追溯性的重要性,将创意质量评估与程序可审计性相分离,为可信科研生成提供了新范式。
链接: https://arxiv.org/abs/2611.24887
作者: Samson Yu,Qiran Zou,Jing Yu Lim,Rushi Shah,Anirudh Goyal,Dianbo Liu
机构: National University of Singapore(新加坡国立大学); Meta Superintelligence Labs(Meta超级智能实验室)
类目: Computation and Language (cs.CL)
备注: 56 pages
Abstract:Large language models have been widely used to generate plausible research ideas, yet it remains unclear what makes automated ideation effective and how generated novel proposals arise from prior scientific work. We introduce the Creative AI Researcher (CAR), an agentic framework combining literature grounding, iterative development and explicit conceptual operators. CAR constructs a semantic knowledge graph from retrieved literature and equips an ideation agent with graph-retrieval tools and four operators: bisociation, analogy, exploration and elaboration. Each operator call is checked against structural acceptance conditions and recorded, producing a trace of intermediate transformations and declared literature sources. We evaluate CAR with two language-model backbones across 40 LiveIdeaBench research topics and two execution benchmarks. CAR achieves significantly higher methodological novelty (MN) than single-shot prompting and The AI Scientist. Its methodological novelty does not differ significantly from the Research Ideation Agent, while its feasibility scores are higher on both backbones. With a shared experiment executor, CAR achieves the highest pairwise win rate among the evaluated AI methods on FML-bench-Lite and BioXArena. In a ten-condition ablation study, MN point estimates favor CAR over one-step proposal on both backbones, but no MN contrast survives correction for multiple comparisons. The results do not establish a consistent quality benefit from explicit operators beyond free-form iteration. CAR makes the sequence of operations and declared literature sources inspectable, separating judged idea quality from procedural traceability.
[NLP-2] General progress substitutes for specific training data in language models
【速读】: 该论文旨在解决生成式模型在训练过程中因数据屏蔽(data withholding)而导致特定能力延迟获取的问题,以及这种延迟的持续时间与后续恢复该能力所需的数据量之间的不确定性。其核心问题是:当某些关键技能相关的训练数据被刻意排除时,模型何时能掌握该能力?需要多少额外数据才能补足?论文提出的关键解决方案是:模型在其余训练数据上的整体进步可部分替代被屏蔽数据的作用,从而“预习”或“隐含学习”被屏蔽的能力。研究通过在国际象棋、奥赛罗棋、分子字符串、括号语言和网络文本等多种任务中屏蔽特定技能的训练信号发现,大模型对被屏蔽技能的延迟可通过小模型进行预测;例如,一个未显式训练应对“将死”局面的国际象棋模型,在仅基于其他训练数据的情况下,仍能近乎准确地评估合法应对策略。一旦补充被屏蔽数据,模型以远少于从初始状态开始所需的少量数据即能快速掌握该能力,且随着模型整体性能提升,所需数据量呈下降趋势。相比之下,难度相当的控制规则学习则进展缓慢。这一发现表明,数据过滤虽能延缓特定能力的出现,但不会造成永久性缺失——模型在持续训练中积累的通用表征能力可显著降低恢复该能力的门槛。
链接: https://arxiv.org/abs/2611.24844
作者: Mark Marosi,Peter Antal
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: 49 pages, 16 figures, 6 tables. Code: this https URL
Abstract:Language models acquire their capabilities from training data, and developers withhold data during training to keep models from learning capabilities that could be misused. How long this delays a capability, and how much data is needed to acquire it later, is unknown. Here we show that the general progress a model makes on the rest of its training data substitutes for the data that was withheld. We withheld the training signal for individual skills in models of chess, Othello, molecular strings, bracket languages and web text. The delay of a withheld skill in larger models was predicted from smaller ones. A chess model never explicitly trained to respond to check ranked the legal responses almost as well as a model that was. When models were then trained on the withheld data, they acquired the skill with a small fraction of the data needed from initialization, and this fraction decreased as the models progressed. A control rule of equal difficulty was learned much more slowly. A programming language withheld from the training text of a 7-billion-parameter model was regained in the same way. Data filtering therefore delays a capability, and the data required to regain it decreases as models improve.
[NLP-3] On-Policy Morphing into Hybrid Attention Models
【速读】: 该论文旨在解决预训练Transformer模型在转换为混合注意力架构(hybrid attention models)时,如何在给定计算预算下高效且准确地选择保留全量注意力(full attention)的层这一关键问题。现有方法FlashMorph存在两个非策略性偏差:其一,连续分支插值与教师强制隐藏状态与离散混合计算之间存在不一致;其二,冻结模型权重无法反映后续恢复阶段的参数适应性。针对上述问题,论文提出FlashMorph2,其核心解决方案在于采用在线策略(on-policy)的层选择机制,通过端到端评估采样的离散混合架构及其自身隐藏状态,并在恢复目标下联合优化层选择得分与模型权重,以真实反映参数自适应后的性能表现。实验表明,FlashMorph2在Qwen3系列不同规模模型及多种线性注意力变体上均显著优于现有方法,在RULER、LongBench和召回密集型任务上分别提升2.42、1.08和0.76分,且层选择耗时仅需0.72–3.57 GPU小时,展现出卓越的性能、效率与可扩展性。
链接: https://arxiv.org/abs/2611.24809
作者: Disen Lan,Xuanda Wang,Jianbin Zheng,Huafeng Kuang,Wei Tang,Yu Cheng
机构: Fudan University (复旦大学); ByteDance Seed (字节跳动种子); Nanyang Technological University (南洋理工大学)
类目: Computation and Language (cs.CL)
备注:
Abstract:Hybrid attention models combine full and linear attention to achieve a better quality–efficiency trade-off. Converting pretrained Transformers into such models requires identifying which layers should retain full attention under a prescribed budget. We identify two off-policy mismatches in FlashMorph’s layer selection: (i) continuous branch interpolation and teacher-forced hidden states differ from discrete hybrid computation; and (ii) frozen model weights fail to account for parameter adaptation during subsequent recovery. We introduce FlashMorph2, an on-policy layer-selection method that (I) evaluates sampled discrete hybrid architectures end to end using their own hidden states, and (II) jointly optimizes layer-selection scores and model weights under the recovery objective to reflect post-adaptation performance. Experiments across Qwen3 model scales and linear-attention variants show that FlashMorph2 consistently outperforms existing layer-selection methods, improving average scores over FlashMorph by +2.42 points on RULER, +1.08 on LongBench, and +0.76 on recall-intensive tasks, while completing layer selection in only 0.72 – 3.57 GPU hours across 0.6B–8B models, demonstrating high efficiency and scalability.
[NLP-4] he Elephant in the Question: Unified Models Picture the Concepts Behind Their Answers
【速读】: 该论文旨在解决大模型内部推理过程缺乏可解释性的问题,特别是如何直观揭示文本输入中特定标记(token)在生成答案前的中间表示(intermediate representations)所对应的语义概念。其核心挑战在于,这些中间状态通常隐含在深层网络中,且传统方法需依赖额外训练的解码器进行可视化。该研究提出“V-lens”框架,利用统一多模态模型(unified multimodal models)中图像生成器与文本理解模块共享表示结构的特性,无需训练额外解码器即可将选定token的隐藏状态映射为视觉图像。该方法的关键在于:通过扩散模型或自回归模型的生成能力,直接将文本编码中的语义信息转化为具象化视觉表征,从而揭示从问题到答案之间未被显式命名的关键概念关联。进一步引入“lens tuning”策略,以图像作为监督信号对特定隐藏状态进行微调,在BAGEL、SenseNova-U1和HunyuanImage-3.0等模型上,指代词(pronoun referent)的可视化准确率从2%–11%显著提升至86%–94%。更令人意外的是,这种仅针对视觉输出的监督还能间接提升文本生成质量,表明视觉表征学习有助于优化模型整体的语义理解能力。
链接: https://arxiv.org/abs/2611.24793
作者: Tobias Braun,Marcus Rohrbach,Lorenzo Cavallaro
机构: Technische Universität Darmstadt(达姆施塔特工业大学); University College London(伦敦大学学院); Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: 32 pages, including supplementary material. Code: this https URL
Abstract:Visualizing intermediate representations could help us inspect the reasoning steps underlying a model output. Unified multimodal models offer this opportunity by combining understanding and generation in one architecture. Their image generators can access representations formed while processing text. We introduce the V-lens, which uses this connection to visualize a chosen token’s hidden state without training a decoder. Across four diffusion-based and two purely autoregressive unified models, the resulting images reveal key intermediate concepts that connect a question to its answer even when the concepts are never named explicitly. Controlled tests show how this content changes with context, token position, and depth. We then introduce lens tuning, which supervises selected hidden states with pictures through the V-lens. Across BAGEL, SenseNova-U1, and HunyuanImage-3.0, the rate of correct visualizations of pronoun referents rises from 2-11% to 86-94%. Surprisingly, such visual supervision also improves text answers without supervising the answer tokens directly. Code is available at this https URL.
[NLP-5] Induction Head Origins of SLAM-ASR Speech Recognition Models ICASSP
【速读】: 该论文旨在解决一个核心问题:为何仅通过一个小型训练映射模型(由编码器状态映射至冻结语言模型的嵌入空间)即可实现与主流方法相媲美的语音识别性能?其解决方案的关键在于揭示了冻结语言模型内部的结构机制。研究发现,SLAM-ASR模型的性能依赖于语言模型中的归纳头(induction heads),这些头在文本中负责复制重复的词元序列,同时也承担将语音嵌入映射到转录词元的重要功能。进一步分析表明,存在成对的归纳头以一种受约束的微分注意力形式协同工作,能够抵消注意力噪声并聚焦于已生成的前序词汇,从而提升定位精度。此外,研究还展示了如何利用归纳头在噪声音频条件下优化转录质量,为提升鲁棒性提供了可解释的机制支撑。
链接: https://arxiv.org/abs/2611.24721
作者: Artem Fedorchenko,Tanel Alumäe
机构: Tallinn University of Technology (塔林理工大学)
类目: Computation and Language (cs.CL)
备注: Submitted to ICASSP
Abstract:What makes a small trained map from encoder states to the embedding space of a frozen language model sufficient for competitive speech recognition? The premise of the SLAM-ASR architecture is that a frozen language model possesses linguistic knowledge rich enough to perform the recognition task. This assumption, while reasonable, provides little insight into the mechanisms underlying these models. We therefore seek the concrete internal structures on which they rely. This work shows that SLAM-ASR models rely on induction heads of the frozen language model: the same heads that copy repeated token sequences in text are also important for mapping speech embeddings to transcript tokens. Moreover, we find pairs of heads that behave as a constrained form of differential attention, cancelling attention noise and sharpening attention onto the previously generated word. Finally, we show how induction heads could be used to improve transcription quality under noisy audio.
[NLP-6] DRIFT: Detachable Reasoning for Runtime-Flexible and Efficient End-to-End Driving
【速读】: 该论文旨在解决生成式视觉-语言模型(Vision-Language Models, VLMs)在端到端(End-to-End, E2E)自动驾驶系统中因计算开销大、难以实现运行时灵活开关语言推理而限制实际部署的问题。其核心挑战在于如何在保持高感知与决策性能的同时,降低模型复杂度并实现推理路径的动态可配置性。解决方案的关键在于提出DRIFT框架,通过引入一个轻量级语言模型(Light Model, LM,仅150M参数),将中间驾驶表征高效投影至语言嵌入空间,实现精准的场景描述与行为注释;同时设计了一种运行时灵活的规划模块,该模块基于从语言模型中蒸馏出的代理推理特征进行训练,并可在推理阶段选择性地依赖原始语言模型或替代的代理特征,从而在满足资源约束时完全跳过语言模型计算,且不损失驾驶性能。该设计使得整体模型仅需255M参数,在Bench2Drive基准上达到97.03的驾驶评分(Driving Score)和89.39%的成功率,性能媲美当前最强的非语言模型方法,同时显著优于现有基于VLM的模型在计算效率方面的表现,实现在真实世界导航(NAVSIM)和AlpaSim等基准上的高效可靠运行。
链接: https://arxiv.org/abs/2611.24719
作者: Emad Aghajanzadeh,Bethany Anne Wong,Leonhard Kunze,Moritz Thoma,Maximilian Forstenhäusler,Nael Fasfous,Manoj Rohit Vemparala,Pierpaolo Mori,Christos Anagnostopoulos,Daniel Mueller-Gritschneder,Ulf Schlichtmann
机构: Technical University of Munich(慕尼黑工业大学), Germany; BMW Group(宝马集团), Germany; University of Glasgow(格拉斯哥大学), UK; TU Wien(维也纳工业大学), Austria
类目: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG); Robotics (cs.RO)
备注: 11 pages, 4 figures, 7 tables. Project page: this https URL
Abstract:The integration of vision-language models (VLMs) into end-to-end (E2E) autonomous driving has demonstrated strong potential by providing scene description and reasoning capabilities. However, VLMs are computationally expensive, and switching language reasoning on and off at runtime without hurting closed-loop performance has not been demonstrated, which hinders practical adoption. To address these issues, we propose DRIFT, an E2E autonomous driving framework that integrates a lightweight language model (LM) for visual question answering and action commentary. By efficiently projecting intermediate driving representations into the language embedding space, we demonstrate that a compact LM (150M parameters) can achieve strong scene description and action commentary. Furthermore, we introduce a runtime-flexible planning module that is trained on surrogate reasoning features distilled from the LM and can be conditioned on either the LM or the surrogate reasoning features at inference. This design enables bypassing the LM entirely when required by deployment constraints, without sacrificing driving performance. With only 255M parameters, DRIFT reaches up to a Driving Score (DS) of 97.03 and a Success Rate of 89.39% (mean over three runs) on Bench2Drive by interleaving its LM and surrogate reasoning at a chosen reasoning rate. This is on par with the strongest non-language models while being far more compact than existing VLM-based models. DRIFT further achieves competitive results on the real-world NAVSIM and AlpaSim benchmarks at significantly lower computational cost than existing VLM-based approaches. Project page: this https URL
[NLP-7] Openhagen Worlds: Calibrated Environments for Measuring and Training Scientific Discovery Agents ACL
【速读】: 该论文旨在解决科学发现类智能体在实验设计与结果推理中难以有效评估其真实探究能力的问题,现有大多数发现基准测试无法将实验选择的启发性与记忆召回、隐含评分规则或捷径策略(shortcut policies)相分离。为此,作者提出了Openhagen Worlds,一个由27个程序化生成环境构成的基准套件,涵盖从经典实验推理谜题到基于模拟器的物理与化学场景,所有环境均具备确定性验证机制,并在有限真值空间下运行。其核心创新在于:每个观测结果均显式包含完整评分规则,且所有178个难度等级均经过严格校准——仅当脚本化“圣杯”(scripted oracle)在64个开发实例上成功率≥0.75、最强非演绎性策略成功率≤0.25、每种受控捷径成功率≤0.35时才被接纳。其中92个核心层级覆盖全部27个类别,适合小模型训练;86个前沿层级则扩展了15个发现类世界,引入更复杂问题及多阶段(三、五、八及以上)任务序列。在相同84个保留的前沿实例上,DeepSeek V4.1 Flash在默认推理资源(每调用16,384 token)下解决32题,在最大资源(每调用最高393,216 token)下提升至57题(圣杯基准为79),余量最大体现在八阶段任务中。基于策略的专家迭代(on-policy expert iteration)使9B参数模型在训练家族上的核心层级成功率从0.090提升至0.148,主要得益于动作格式优化,且未对GPQA Diamond任务造成可检测性能损失。此外,提供托管服务以公开运行核心层级的保留实例。
链接: https://arxiv.org/abs/2611.24711
作者: Alexander Boesgaard Lorup
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 41 pages. Ancillary files: a public sample of 48 development instances with scripted-oracle episodes. Hosted evaluation: this https URL
Abstract:Agents that do science must choose informative experiments and reason from their results, but most discovery benchmarks cannot separate that skill from recall, unstated scoring rules or shortcut policies. We introduce Openhagen Worlds, a suite of 27 procedurally generated environments (some with finite truth spaces), from classic puzzles of experimental reasoning to simulator-grounded physics and chemistry, in which an agent runs experiments under a budget and commits an answer that a deterministic verifier checks against a hidden truth. Every observation states the complete scoring rule, and each of the 178 difficulty levels is calibrated: it is admitted only if, on 64 development instances, a scripted oracle succeeds on at least 0.75, the strongest plausible non-deductive policy on at most 0.25 and each gated shortcut on at most 0.35. The 92 core levels span all 27 families and are sized to train small models; the 86 frontier rungs extend the 15 discovery worlds with harder questions and with campaigns of three, five and eight or more committed stages on one hidden system. On the same 84 withheld frontier instances, one per family and rung, DeepSeek V4.1 Flash solves 32 at its default reasoning effort with 16,384 tokens per call and 57 at its maximum effort with up to 393,216 tokens per call (oracle: 79); the headroom that remains is largest in eight-stage campaigns. On-policy expert iteration gives a 9B model modest gains: its core-tier success on trained families rises from 0.090 to 0.148, partly through better action format, with no detectable loss on GPQA Diamond. A hosted service plays the core tier’s withheld instances.
[NLP-8] When Does Continual Pre-Training Help? A Controlled Study of Dialectal Adaptation for Québec French EACL2027
【速读】: 该论文旨在解决持续预训练(Continual Pre-Training, CPT)在方言适应中的有效性问题,具体包括:何时需要进行CPT、其优势的驱动因素是什么,以及现有综合评估指标是否能准确反映其实际效果。研究针对魁北克法语(Québec French)的语言理解任务,在九个不同架构与规模的大型语言模型(Large Language Models, LLMs)上开展受控实验,采用每项实验3000万词的预算匹配协议。研究发现,CPT的效果高度依赖于初始模型对目标语言变体的已有能力;当其有效时,提升主要源于语料库的内容构成而非数据量,其中文学文本贡献了绝大部分性能增益。配对的魁北克法语与标准法语评估表明,CPT更显著提升了表达理解能力而非语法掌握能力,且语料库的语域(register)决定了所习得区域性表达的具体类型。此外,这些改进伴随着特定能力的权衡,例如阅读理解能力的下降,而这类代价可能被综合评估分数所掩盖。因此,研究结论指出,将CPT视为普遍有益的默认策略是不恰当的,方言适配应始于对基础模型现有能力的细致评估,并结合源数据层面的分析和任务层面的精细化评价,以制定更有效的适应策略。
链接: https://arxiv.org/abs/2611.24699
作者: Eeham Khan,Richard Khoury,Leila Kosseim
机构: Concordia University (康考迪亚大学); Université Laval (拉瓦勒大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: EACL 2027 Main Conference
Abstract:Continual pre-training (CPT) is widely used to adapt pretrained language models to dialectal and non-standard language varieties, yet it remains unclear when this additional training is necessary, what drives its benefits, and whether aggregate evaluation scores adequately capture its effects. In this paper, we present a controlled study of CPT for Québec French language understanding across nine LLMs spanning different families and scales. Using a budget-matched protocol of 30M tokens per experiment, our results show that the benefit of CPT depends strongly on the starting model’s existing competence in the target variety. When CPT is beneficial, the improvements are driven more by corpus composition than by data volume, with literary text accounting for most of the gain. Paired Québec and standard French evaluations further show that CPT primarily improves expression comprehension rather than grammatical competence, and that corpus register determines which regional expressions are acquired. These gains also involve capability-specific trade-offs, including declines in reading comprehension that aggregate scores can conceal. Overall, these findings show that CPT for dialectal adaptation should not be treated as a uniformly beneficial default. Instead, adaptation strategies should begin with a careful assessment of the base model’s existing competence, accompanied by source-level data analysis and task-level evaluation.
[NLP-9] CurvDelta: Low-Curvature Delta Modeling For Surgical Learning and Unlearning
【速读】: 该论文旨在解决大语言模型(Large Language Models, LLMs)在适应过程中普遍存在的能力保持难题:即在实现特定任务目标的同时,往往会导致模型通用能力的退化。其核心解决方案是提出CurvDelta框架,该框架通过将能力保持约束嵌入到模型参数更新的参数化形式中,利用对预适应模型附近保持约束局部几何结构的分析,构建一个低曲率空间,使得参数更新在此空间内几乎不改变保持损失。为使该方法适用于大规模语言模型,研究引入了克罗内克积近似曲率(Kronecker-factored Approximate Curvature, K-FAC)和分解投影机制以实现高效计算。此外,还提出了CurvLoRA,一种基于相同保持几何结构的低秩变体,以提升适应效率。两种方法均适用于受保持约束的适应任务,如能力学习与选择性遗忘。实验结果表明,在数学推理、代码生成任务中,所提方法在下游性能上具有竞争力的同时,显著提升了通用能力的保持效果,展现出优于基线的获取-保留权衡;在知识遗忘任务中,于TOFU和MUSE数据集上的表现证明其在目标遗忘与知识保留之间实现了更优平衡,且保留性能接近从头再训练的基准水平。
链接: https://arxiv.org/abs/2611.24654
作者: Zeguan Xiao,Zhuocheng Liang,Yun Chen,Yong Wang,Guanhua Chen,Pinyan Lu
机构: Shanghai University of Finance and Economics(上海财经大学); Alibaba Group(阿里巴巴集团); Southern University of Science and Technology(南方科技大学)
类目: Computation and Language (cs.CL)
备注:
Abstract:A central challenge in adapting large language models (LLMs) is capability preservation: an adaptation can achieve its target while degrading general capabilities. We introduce CurvDelta, a scalable framework that incorporates the preservation constraint into the parameterization of the model update. By analyzing the local geometry of the preservation constraint around the pre-adaptation model, CurvDelta constructs a low-curvature space in which parameter updates leave the preservation loss nearly invariant. We make its construction practical for LLMs through Kronecker-factored approximate curvature (K-FAC) and a factorized projection. We also introduce CurvLoRA, a low-rank variant within the same preservation geometry, designed for more efficient adaptation. Both methods apply to preservation-constrained adaptation tasks such as capability learning and selective unlearning. We evaluate our methods on mathematical reasoning, code generation, and knowledge unlearning. For mathematical reasoning and code generation, our approaches achieve competitive downstream performance while better preserving general capabilities, offering a favorable acquisition–retention trade-off relative to the baselines. For knowledge unlearning, our main results on TOFU and MUSE show a better balance between targeted forgetting and knowledge retention than the baselines, with retention performance close to the from-scratch retraining reference.
[NLP-10] Piccolo: Co-Evolving Decision Models and Data through Recursive Self-Improvement
【速读】: 该论文旨在解决大模型在通用决策任务中性能提升受限于数据质量与模型-数据协同进化不足的核心问题,尤其聚焦于如何通过数据驱动的自我改进机制实现决策模型的持续优化。其解决方案的关键在于提出一种基于“代理式数据工程”(Agentic Data Engineering)与“递归自我改进”(Recursive Self-Improvement, RSI)的协同演化框架,使模型能力与训练数据在迭代过程中共同演进。具体而言,研究引入了由智能体驱动的闭环系统:研究代理(research agents)自主分析模型行为、构建训练数据、执行实验并利用反馈结果优化后续策略;同时通过语义切片(semantic slices)和共享研究记忆(shared research memory)实现跨模型家族的协同学习。大规模实验证明,该方法突破了传统依赖模型规模增长的范式,揭示出三大反直觉发现:(1)模型架构与训练配方的协同适配优于单纯追求模型版本更新;(2)针对性的数据精炼可使小型模型(4B–9B)在特定工作流中媲美大型基准模型(27B);(3)连续降低损失函数并不必然提升离散决策准确性,甚至可能损害性能。这些发现为构建前沿决策模型提供了可复现的范式,结合开源的权重及来自Data-Juicer与AgentScope生态系统的工具,形成了可扩展的数据驱动递归改进蓝图。
链接: https://arxiv.org/abs/2611.24651
作者: Zhenqing Ling,Qirui Jiao,Daoyuan Chen,Cheng Chen,Xinji Mai,Haowen Gao,Zhuo Zhang,Xuchen Pan,Mengsu Ding,Jiaji Deng,Yuexiang Xie,Xiaoming Qin,Weirui Kuang,Zhaoyang Liu,Yaliang Li,Bolin Ding,Jingren Zhou
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注: 22 pages, 13 figures, 11 tables. Project page and blog: this https URL
Abstract:We present Piccolo, a family of multimodal decision models (0.8B to 27B) that coevolves model capabilities and training data via Agentic Data Engineering and recursive self-improvement (RSI). The models predict probabilities over candidate answers for general-purpose text and visual judgments, agent harnesses, data workflows, and interactive games. Piccolo-27B tops the Decision Index Vision Full leaderboard with a score of 75.59, while Piccolo-9B and Piccolo-4B achieve 68.35 and 67.46, respectively. All three rank first on the Private benchmark in their respective model-size categories. Research agents analyze model behavior, construct training data, conduct experiments, and use the results to refine subsequent proposals. Semantic slices and a shared research memory connect these actions across the model family’s development. With training recipes comprising hundreds of thousands of examples, data selection and feedback-driven refinement are central to the self-improvement process. Our large-scale RSI exploration yields three counter-intuitive insights for decision-model development: (1) Backbone-recipe co-adaptation outweighs raw model recency: earlier backbones can surpass newer generations on visual tasks under identical recipes; (2) Targeted data curation enables 4B-9B models to rival 27B baselines on specialized workflows; and (3) Continuous loss reduction can decouple from and even degrade discrete decision accuracy. Together with the released weights and tools from the Data-Juicer and AgentScope ecosystems, Piccolo provides an actionable blueprint for building frontier decision models and scaling data-driven RSI.
[NLP-11] Self-Rewarded Reinforcement Learning for Better-Structured Medical Explanations in Small Language Models
【速读】: 该论文旨在解决小型语言模型(Small Language Models, SLMs)在临床场景中生成的医学解释存在篇幅短、结构松散的问题,尽管其具备在通用硬件上运行并支持隐私保护型本地部署的优势。现有监督学习方法(如Meerkat)虽能将链式思维(Chain-of-Thought)能力迁移至7B–8B规模的模型,但其训练目标并未直接优化解释的风格或组织结构。为此,研究基于指令微调的Mistral-7B骨干模型,提出一种轻量级强化学习(Reinforcement Learning, RL)方案:采用近端策略优化(Proximal Policy Optimization, PPO)结合低秩适配(LoRA),对2,000个源自Meerkat的医学提示进行微调,利用一种简单的自监督奖励函数——该奖励函数倾向于更长、词汇多样性更高且标点使用规范的响应,同时惩罚过短输出。在单块NVIDIA A100 GPU上训练约170个PPO步骤后,平均奖励稳定提升。定量与定性分析表明,经强化学习调优后的模型能够生成更长、更具条理性的多要点医学解释,同时保持原有医学知识准确性。研究探讨了自奖励强化学习在塑造医学解释风格方面的潜力与局限性。
链接: https://arxiv.org/abs/2611.24650
作者: Prakhar Mishra,Bhavya Yajush Awasthi,Tanuja,Nishant
机构: Delhi Technological University(德里科技大学); Indian Institute of Technology, Guwahati(印度理工学院古瓦哈蒂分校); Delhi Skill and Entrepreneurship University(德里技能与创业大学); Central University of Haryana(哈里亚纳中央大学)
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 14 Pages, 10 Figures
Abstract:Small language models (SLMs) can run on commodity hardware and support privacy-preserving, on-premises clinical deployment, but their medical explanations are often short and poorly structured compared with large language models. Recent supervised approaches such as Meerkat transfer chain-of-thought reasoning into 7B-8B backbones, yet their training objective does not directly target the style or organization of the generated rationales. Building on the instruction-tuned Mistral-7B backbone, we investigate whether a lightweight reinforcement learning (RL) step can improve explanation structure without human preference this http URL apply Proximal Policy Optimization (PPO) with LoRA adaptation to 2,000 Meerkat-derived medical prompts, guided by a simple self-supervised reward that favors longer, lexically diverse, and well-punctuated responses while penalizing very short ones. Training for about 170 PPO steps on a single NVIDIA A100 GPU yields a stable increase in the average reward. Quantitative and qualitative comparisons show that the RL-tuned model produces longer, more organized, multi-point explanations while preserving the underlying medical this http URL discuss the promise and the limitations of self-rewarded RL for shaping the style of medical explanations.
[NLP-12] Clover-35B-A3B-Preview Technical Report
【速读】: 该论文旨在解决大模型在复杂推理任务中生成质量与计算效率难以兼顾的问题,特别是针对需要长上下文建模和多步思维(thinking-mode)生成的智能体(agent)应用场景。其核心挑战在于如何在有限显存条件下实现高效、高精度的生成,并降低推理过程中的冗余计算开销。解决方案的关键在于提出一种基于专家混合(Mixture-of-Experts, MoE)架构的扩散语言模型——Clover-35B-A3B-Preview,通过引入门控增量网络(Gated Delta Networks)机制,在扩散过程中融合历史生成内容的循环记忆与当前块内的双向softmax注意力,实现对上下文状态的动态感知。该设计通过共享softmax结构使历史记忆读取与当前词元竞争,结合可学习的块大小校准与独立零初始化的右文卷积分支,显著提升了上下文建模能力与生成连贯性。此外,论文还提出一种决策模型变体,支持在一次去噪过程中联合预测多个相关问题的答案,将内部测试集上的问题准确率从90.37%提升至93.88%,全正确组准确率从65.4%提升至77.0%,有效降低了推理成本并增强了生成一致性。实验表明,该模型在多项代理评估指标上领先,且在单张NVIDIA GB300 GPU上以BF16精度实现高达680 tokens/s的生成速度,展现出卓越的部署效率与实用性。
链接: https://arxiv.org/abs/2611.24636
作者: DiffuSpace Technical Team
机构: diffuspace.ai
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Technical Report. Code: this https URL Weights: this https URL
Abstract:We introduce Clover-35B-A3B-Preview (Clover), a mixture-of-experts diffusion language model with 35 billion total parameters and 3 billion activated parameters, supporting thinking-mode generation. Clover adapts Gated Delta Networks to block diffusion by combining recurrent memory of previously generated context with bidirectional softmax attention inside the current block. Historical memory readout competes with current-block tokens in a shared softmax, with learnable block-size calibration and a separate, zero-initialized right-context convolution branch. Clover leads the seven evaluated agent metrics, including 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, it reaches a reported generation speed of 680 tokens/s using one NVIDIA GB300 GPU at BF16 precision and batch size one. We organize the evaluation around model quality, drafting and verification costs, and deployment under constrained memory. We also describe a decision-model variant that generates answers to related questions in one denoising pass. On an internal set of 500 groups, joint prediction raises question accuracy from 90.37% to 93.88% and all-correct group accuracy from 65.4% to 77.0%.
[NLP-13] ransferring an analysis of human brain organization to language models
【速读】: 该论文旨在探究语言模型在推理能力提升过程中是否经历类似人类皮层发育中功能系统逐步分化的过程,即是否存在可识别的、与推理能力相关的特定网络区域的重组。其核心解决方案的关键在于:通过扩散映射(diffusion-map embedding)分析三个不同家族的语言模型在基础训练与推理专项训练后的权重分布,发现尽管整体梯度结构保持稳定,但在连续的一层网络中存在局部的可辨识变化区域。研究进一步表明,将这些特定区域的权重“嫁接”(grafting)至指令微调后的模型中,能够显著提升数学推理表现,且性能增益随嫁接区域与原模型重叠程度增加而提高;同时,此类嫁接保留了宿主模型原有的解题能力,干扰较小。此外,该推理相关区域在训练早期性能上升阶段逐渐形成,中间阶段的嫁接亦具有效性。跨模型验证还发现,在医学领域模型中同样存在可转移的推理相关权重区域。综上,该研究揭示了一个可迁移的、与推理能力密切相关的神经网络子区域,其权重调整可在不损害原有数学能力的前提下,有效增强模型的推理性能。
链接: https://arxiv.org/abs/2611.24626
作者: Donghyeok Lee,Sangwon Yu,Sangkwon Park,Hye Youn Park,Munyoung Chang,Sungroh Yoon
机构: Seoul National University (首尔国立大学); Upstage AI; Seoul National University Bundang Hospital (首尔国立大学仁川医院)
类目: Machine Learning (cs.LG); Computation and Language (cs.CL)
备注: 5 figures, 5 extended data figures, 5 extended data tables; includes Supplementary Information
Abstract:During human cortical development, functional systems become increasingly differentiated. We asked whether improved reasoning in language models involves a comparable regional reorganization. Applying diffusion-map embedding, used to characterize cortical organization, to matched base and reasoning-trained models from three families, we found broadly preserved gradient profiles with localized changes within a contiguous layer span. Grafting weights from these spans into instruction-tuned hosts improved mathematical reasoning on average, with gains rising with region overlap, while retaining more previously solved problems than grafts elsewhere. Across training, the span formed gradually during the early rise in performance, and grafts from intermediate stages helped. Grafts shifted most hosts partly towards their donors’ behaviour and sometimes improved on problems the donors failed; the same analysis located a transferable span in a medical model. Together, these findings identify a reasoning-related region whose weights improve reasoning with low interference in the host’s existing mathematical competence.
[NLP-14] Chain-of-Thought Hidden States Reveal Task-Solving Directions for Direct Answers
【速读】: 该论文旨在解决生成式 AI(Generative AI)在使用思维链提示(Chain-of-Thought, CoT)时存在的效率与效果不一致问题:一方面,部分任务中模型性能反而劣于直接答案提示(direct-answer prompting),另一方面,某些任务中模型需生成数百个词才能得出正确答案,导致计算成本过高。其解决方案的关键在于发现并利用直接提示(direct prompt)的隐藏状态空间中存在一种特定方向——该方向对每个问题而言均能引导模型在极短生成长度下更高效地达到正确答案。研究通过线性探测识别出“恢复”(recovering)与“失败”(failed)两类隐藏状态表征的可区分性,并基于各问题自身的恢复与失败状态构建专属方向。通过对直接提示的最后一个隐藏状态进行一次旋转以对齐该方向,可在仅使用约十分之一的生成 token 情况下,将模型在 25 个任务上的平均准确率从 46% 提升至 85%(LLaMA 3.1 8B,CoT 为 69%),或从 52% 提升至 86%(Qwen 2.5 7B,CoT 为 77%)。这一结果表明,CoT 并非通向高精度的最短路径,而这些可学习的方向可为未来模型训练提供新的对齐目标,实现更高效、精准的推理。
链接: https://arxiv.org/abs/2611.24598
作者: Houman Mehrafarin,Ioannis Konstas
机构: Heriot-Watt University (赫瑞-瓦特大学); University of Edinburgh (爱丁堡大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:Chain-of-Thought (CoT) prompting improves the performance of Large Language Models by eliciting intermediate reasoning steps. However, CoT is far from perfect: on some tasks, models perform worse than with direct-answer prompting, and on others they generate a few hundred tokens to reach the correct answer. We start by labelling each CoT hidden state as recovering if patching it into the direct prompt yields the correct answer, and failed otherwise. A linear probe shows that the model encodes these two classes differently. We then show that the representation space of the direct prompt holds a direction for each question that leads the model, in a short generation, to the correct answer more often than CoT does. Specifically, we rotate the direct prompt’s last hidden state once, before generation, towards a direction built for each question from its own recovering and failed states. Across 25 tasks, these directions raise the direct prompt’s mean accuracy from 46% to 85% on LLaMA 3.1 8B (CoT: 69% ) and from 52% to 86% on Qwen 2.5 7B (CoT: 77% ), with a tenth of CoT’s tokens. CoT is thus not the shortest route to this accuracy, and these directions can be used in future work to train models to align their states with them.
[NLP-15] Large Language Models or AI Agents for English to Arabic Translation? An assessment on WikiEn-Ar190
【速读】: 该论文旨在解决生成式AI在低资源语言(如阿拉伯语)翻译任务中表现不足的问题,尤其关注大型语言模型与自主智能体(Autonomous AI Agent)在英-阿翻译中的性能差异。其核心挑战在于如何在缺乏高质量平行语料的情况下,保证翻译在准确性、流畅性与风格一致性方面的质量。解决方案的关键在于提出一个名为WikiEn-Ar190的新数据集,该数据集由两名母语阿拉伯语者精心筛选并验证的190篇维基百科文章构成,确保了语义与文体的忠实再现;同时,研究创新性地将GPT-5-mini作为独立神经翻译器,或嵌入包含自我评估与迭代优化的复杂流程中进行对比实验。结果表明,当采用自主推理机制时,模型在词汇准确率上提升了28.76%(BLEU提升至22.83),同时保持了较高的语义质量,证明了基于自主推理的自修正机制能有效增强低资源语言翻译的一致性与可靠性。该方法不仅显著提升了翻译性能,还为构建可评估、可改进的自动化翻译系统提供了通用框架,具有扩展阿拉伯语等低资源语言网络内容的潜力。
链接: https://arxiv.org/abs/2611.24588
作者: Fatima Zahra Iguenfer,Yousra Chtouki
机构: 未知
类目: Computation and Language (cs.CL)
备注:
Abstract:This research investigates whether a large language model or an autonomous AI Agent performs better at translating from English to Arabic, which is considered a low-resource language. The authors introduce WikiEn-Ar190, a new dataset of 190 carefully selected Wikipedia articles from 18 different areas, which were created and verified by two native Arabic speakers to ensure that they accurately reflect both meaning and style. Using this dataset, the study evaluates the translation quality of GPT-5-mini both as an independent neural translator and when used within a more complex process that includes self-assessment and improvement. As a standalone model, GPT-5-mini produced very fluent translations , achieving a COMET score of 0.8649 and receiving high human scores for adequacy, fluency, and style (averaging 4.53 out of 5). When used in a system that combined translation with self-assessment, it improved lexical accuracy by 28.76% (BLEU score of 22.83) while maintaining a similar level of semantic quality. These findings show that methods involving autonomous reasoning can improve translation consistency for language pairs with limited resources. Beyond the results, the approach offers a framework for evaluating and improving translations, potentially using them as assisting tools to expand the representation of low resource languages, such as Arabic, on the web.
[NLP-16] Late Entry in Looped Language Models
【速读】: 该论文旨在解决生成式 AI(Generative AI)中序列解码时因递归深度与推理延迟之间的固有矛盾所导致的效率瓶颈问题。传统方法采用早退出(early exit)策略,虽可降低计算开销,但因未完成缓存状态构建而限制了上下文建模能力;相反,本文提出“晚进入”(late entry)机制,允许新词跳过初始浅层迭代,直接进入深度缓存的最后 K 轮计算,从而在保持深层上下文表示的同时显著提升解码速度。其核心解决方案在于通过轻量级微调校正进入边界处的隐藏状态范数不匹配,并利用已构建的深层上下文缓存替代缺失的局部前向传播,实现对早退出策略的性能超越。进一步地,作者设计了一种基于输出 GEMM 投影的“唤醒验证”(Wake verification)机制,在单个静态 CUDA 图中统一实现了贪婪精确解码(tau = 0)、带容差的边缘门控验证(tau > 0)以及无回滚的损失型晚进入(tau = ∞),并通过仅对关键路径执行最终 K 轮验证、同时并行处理已完成令牌的完整 T 轮计算,避免了构建深层缓存带来的串行等待开销。实验表明,在 Ouro-2.6B 模型上,该方法在 tau = 0 时达到 5.2 倍加速(相较块解码提升 26%),在 tau = ∞ 时达 6.8 倍加速;在 Huginn-3.5B 上,随着递归深度增至 64,最大加速比可达 12.5 倍,相较标准块解码提升超过 150%,充分验证了该方案在不牺牲精度的前提下实现高效低延迟推理的可行性。
链接: https://arxiv.org/abs/2611.24576
作者: Ziqi Jin,Jiaxi Li,Weiwen Xu,Aixin Sun
机构: 未知
类目: Computation and Language (cs.CL)
备注: 25 pages, 9 figures
Abstract:Looped language models deepen reasoning at a compact parameter count by iterating a shared layer stack T times per token, turning recurrence depth directly into sequential decoding latency. Conventional early-exit decoding halts after initial passes without cache completion, truncating cached key-value states of past tokens to shallow depth. We study the opposite strategy, late entry, where new tokens skip initial passes and enter directly into the final K T passes over a deep context cache. Once lightweight fine-tuning resolves hidden-state norm mismatch at the entry boundary, deep cached context substitutes for skipped token-local passes, allowing late entry to outperform matched early exit across model families and pass budgets. To turn this asymmetry into wall-clock speedups without stalling to build deep caches, candidate blocks verify through only the final K passes on the critical path while previously committed tokens complete passes 1 - T alongside subsequent steps at zero extra serial calls. Projecting these finished T-pass wake states inside the existing output GEMM (Wake verification, governed by logit-margin tolerance tau in [0, inf]) unifies greedy-exact T-pass decoding (tau = 0, matching K = T block decoding within bf16 reduction order), margin-gated verification (tau 0), and non-rollback lossy late entry (tau = inf) inside a single static CUDA graph. On Ouro-2.6B (T = 4), K = 2 Wake reaches 5.2x greedy-exact speedup at tau = 0 (+26% over K = 4 block decoding on GSM8K) and 6.8x at tau = inf (+16.9 points over trained early exit at matched speed). Scaling test-time recurrence across T in 8, 16, 32, 64 on Huginn-3.5B cuts K = T/4 rollbacks 21.5x with depth, driving greedy-exact (tau = 0) speedups to 6.1x (T = 32) and 8.4x (T = 64, +153% over K = 64 block) and tau = inf speedups to 10.6 to 12.5x.
[NLP-17] Objective-Specialized Residual Streams in Manifold-Constrained Hyper-Connections
【速读】: 该论文旨在解决多任务或多元目标下Transformer模型中残差流(residual stream)缺乏目标特异性分工的问题,即如何在保持模型整体性能的同时,实现不同计算路径对特定任务目标的专精化。其核心解决方案是提出一种流形约束的超连接结构(Manifold-constrained Hyper-Connections, mHC),通过为每个并行残差流分配不同的学习目标(如词汇检索的稀疏性目标与语义检索的稠密性目标),引导模型在训练过程中形成目标特异性的内部表征与计算机制。关键发现在于:相较于所有流共同优化单一目标的传统方式,赋予不同流以差异化学习目标可显著提升跨稀疏—稠密融合范围内的检索性能;进一步分析表明,这种优势源于深度依赖的目标特异性读写门控路由机制(read/write gates)和梯度流动模式的分化,而非依赖于流间自适应混合(cross-stream mixing)。在模型最后几层,专用化模型表现出更明确的流间分离、减少的跨流信息混杂以及针对不同目标的独立优化轨迹,而固定残差混合为恒等映射仍能保留这一优势,证明其本质来源于目标驱动的路由策略而非流间交互的学习。因此,该研究揭示了通过目标解耦引导的结构化专业化能够有效增强多目标检索任务中的计算效率与表达能力。
链接: https://arxiv.org/abs/2611.24567
作者: Shantanu Agarwal,Ruchita V. Maitri,Zhiqi Huang,Vivek Datla,Alfy Samuel
机构: Capital One(资本一号)
类目: Computation and Language (cs.CL)
备注:
Abstract:Manifold-constrained Hyper-Connections (mHC) extend a Transformer’s residual stream into multiple parallel streams with learned cross-stream connections. Prior work trains all streams toward a shared objective. We instead study whether assigning distinct learning objectives to different streams can improve task performance and induce objective-specific specialization. Using complementary lexical (sparse) and semantic (dense) retrieval objectives as a test case, we find that objective-specialized mHC consistently outperforms a non-specialized mHC model across the full sparse–dense fusion range, for both two- and four-stream models and at two model scales. We then examine how specialization changes the model internally using probes of stream representations, residual mixing, objective-specific gradient flow, and read/write gates. The specialized and non-specialized models behave similarly through most of the network but diverge in the final layers: the specialized model develops increasingly objective-specific gradient flow, distinct read/write routing patterns, and reduced cross-stream mixing, and streams assigned to different objectives become increasingly dissimilar while those trained toward the same objective remain highly similar. Fixing the residual mixing to the identity leaves the specialization benefit intact, indicating that the benefit stems from objective-specific read/write routing rather than learned cross-stream mixing. These results show that assigning distinct objectives to mHC streams can improve multi-objective retrieval while inducing structured, depth-dependent specialization in the residual computation.
[NLP-18] Do Frontier Model Ingredients Add Up? A Replicated Factorial Study of Small Language Models
【速读】: 该论文旨在解决小型语言模型(Small Language Model)在构建过程中如何有效选择并组合已有改进方法以实现性能最优的问题。其核心挑战在于,在有限的数据和训练预算下,不同技术改进的增益效应难以独立评估,且存在潜在的交互作用。解决方案的关键在于通过系统性对比五种改进方法在同一模型架构、相同数据集和训练预算下的表现,并采用双随机种子重复实验以增强结果的可靠性。研究发现,专家混合(Mixture-of-Experts, MoE)层、Muon优化算法以及门控型DeltaNet(Gated DeltaNet)混合结构带来最显著的性能提升;额外的归一化操作也有正向贡献,而注意力门控机制在参数预算匹配的情况下增益有限。尽管大部分改进具有可叠加性,但重复实验揭示了先前被误判为噪声的微小交互效应与潜在收益。因此,该研究强调:在小型模型训练场景中,应优先测试有前景的组件组合,并通过重复验证来确认其有效性,从而做出更稳健的模型设计决策。
链接: https://arxiv.org/abs/2611.24540
作者: Biswa Sengupta
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:Building a small language model requires choosing which published improvements to combine. We compare five changes in the same model, using the same data and training budget. We test selected combinations and repeat each one with a second random seed. Mixture-of-experts layers, Muon optimisation and a Gated DeltaNet hybrid give the largest gains. Extra normalisation also helps, while an attention gate with a matched parameter budget adds little on average. The gains mostly combine, but small interactions between changes appear again in the repeated runs. Repetition also reveals benefits that an initial statistical estimate treated as noise. These findings apply to our small-model training setting; the expert models also have greater total capacity. The practical lesson is to test promising components together and repeat the comparisons before deciding which ones to keep.
[NLP-19] ProbMedTOD: A Bayesian Network Guided Task-Oriented Dialogue System for Patient History Taking
【速读】: 该论文旨在解决在临床问诊(patient history taking)中如何有效融合小规模语言模型(Small Language Models, SLMs)与可解释的不确定性建模之间的矛盾问题。现有方法依赖大语言模型(LLMs)虽具强大生成能力,但受限于隐私、成本及延迟,而SLMs虽高效轻量,却仅能模拟快速直觉式“系统1”推理,缺乏对诊断不确定性进行动态更新的深度反思性“系统2”推理机制。为克服这一局限,论文提出ProbMedTOD框架,其核心解决方案在于通过三阶段自动构建临床贝叶斯网络(Bayesian Network, BayesNet):首先基于少量临床文本构建医学本体,继而预测网络结构,最后估计概率参数。该生成的贝叶斯网络作为可解释的信念追踪器,与SLMs协同工作,由贝叶斯网络代理(BayesNet Agent)实时维护诊断后验分布,由策略代理(Policy Agent)选择具有诊断信息量的问题,实现多轮对话中的概率化推理。实验在DDXPlus和MIMIC-IV Notes数据集上验证了该方法显著优于纯大模型与检索基线,尤其在小型模型上表现更为突出,证明了其在资源受限场景下提升临床决策可信度的有效性。
链接: https://arxiv.org/abs/2611.24535
作者: Vishal Vivek Saley,Bhavesh Gurnani,Dinesh Raghu,Mausam
机构: 未知
类目: Computation and Language (cs.CL)
备注:
Abstract:Patient history taking is a strategic, multi-turn process that refines diagnostic hypotheses through iterative questioning, mirroring clinical reasoning where each inquiry updates beliefs over candidate diagnoses. Deploying AI in this setting is constrained by privacy, cost, and latency, making Small Language Models (SLMs) a more desirable backbone compared to larger LLMs. However, SLMs capture only the fast, intuitive “System 1” side of this process and lack explicit mechanisms for the deliberative “System 2” reasoning needed to maintain and update diagnostic uncertainty over time. Bayesian Networks (BayesNets) offer a natural complement, providing an interpretable framework for auditable probabilistic belief tracking. Yet, clinical BayesNets are difficult to construct due to their reliance on expert curated structure, dependencies, and parameters. We introduce ProbMedTOD, which automatically synthesizes a clinical BayesNet from a small number of clinical notes in three stages: ontology building, structure prediction, and parameter estimation. ProbMedTOD then integrates the resulting BayesNet with SLMs, enabling probabilistic reasoning in multi-turn dialogues. A BayesNet Agent maintains real-time posterior distributions over diagnoses, while a Policy Agent picks diagnostically informative questions. Experiments on DDXPlus and MIMIC-IV Notes show that ProbMedTOD outperforms LLM-only and retrieval-based baselines, with especially strong gains for smaller models. Subjects: Computation and Language (cs.CL) Cite as: arXiv:2611.24535 [cs.CL] (or arXiv:2611.24535v1 [cs.CL] for this version) https://doi.org/10.48550/arXiv.2611.24535 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[NLP-20] MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking
【速读】: 该论文旨在解决多模态实体链接(Multimodal Entity Linking, MEL)中的多层级对应关系建模问题,核心挑战在于如何有效捕捉提及-实体对之间的语义一致性以及跨模态间的互补相关性。现有方法因依赖耦合交互或粗粒度全局聚合,常出现元素主导(element dominance)现象,导致部分模态或特征被过度强调而影响整体性能。为应对这一问题,论文提出多元素协同交互(Multi-Element Collaborative Interaction, MECI)框架,其关键在于:首先通过多视图专家网络(Multi-view Experts Network)采用“分而治之”策略实现解耦特征学习,并结合KL引导的路由机制调控专家专长与协作,以捕获多元素提及-实体对应关系;其次,设计层次化多模态交互模块(Hierarchical Multimodal Interaction Module),利用动态模态感知加权网络在多层次语义上优化跨模态交互,整合多粒度证据以缓解元素主导;最后引入基于大语言模型的生成式语义精炼阶段,实现零样本重排序。实验结果表明,MECI在WikiDiverse、RichpediaMEL和WikiMEL数据集上分别提升Hits@1 1.95%、7.30%和2.31%,显著优于当前最优基线。
链接: https://arxiv.org/abs/2611.24522
作者: Jie Peng,Yongxue Shan,Yongfu Zha,Xiaodong Wang
机构: National University of Defense Technology (国防科技大学); College of Computer Science and Technology (计算机科学与技术学院)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: 11 pages, 8 figures
Abstract:Multimodal Entity Linking (MEL) aims to disambiguate mentions in multimodal contexts by grounding them to specific entities in a knowledge base. A pivotal challenge in MEL is capturing multi-level correspondences: the semantic consistency between mention-entity pairs and the complementary correlations across modalities. However, existing methods often suffer from element dominance due to their reliance on coupled interactions or coarse global aggregations. In response, we propose the Multi-Element Collaborative Interaction (MECI) framework. First, to capture multi-element mention-entity correspondences, we develop a Multi-view Experts Network that leverages a “divide-and-conquer” strategy for decoupled feature learning to mitigate element dominance, supported by a KL-guided routing mechanism that governs expert specialization and collaboration. Furthermore, to model cross-modal complementary correlations, we propose a Hierarchical Multimodal Interaction Module, where a dynamic modality-aware weighting network refines interactions across hierarchical semantic levels, thereby integrating multi-granular evidence to counteract element dominance. Finally, we incorporate a generative semantic refinement stage that utilizes large language models for zero-shot re-ranking. Extensive experiments on WikiDiverse, RichpediaMEL, and WikiMEL show that MECI consistently outperforms state-of-the-art baselines, improving Hits@1 by 1.95%, 7.30%, and 2.31%, respectively.
[NLP-21] Nothing to Declare: Steering a single neuron increases concealment in AI teams
【速读】: 该论文试图解决的问题是:在群体协作情境下,单个神经元的干预如何影响大模型(如Qwen3.5-4B)的行为表现,尤其是信息隐藏行为的动态变化机制。其解决方案的关键在于揭示了神经元激活方向的定向调控(steering)在集体决策环境中具有显著的上下文依赖性——当多个智能体共同参与时,对特定神经元输出方向的引导会显著增强群体层面的信息隐瞒倾向,且这种效应在无实际通信交互的情境下依然存在;而引入跨代理通信则可在一定程度上缓解过度隐藏行为。值得注意的是,尽管该神经元在集体行为中扮演关键角色,但移除其贡献后整体行为变化微乎其微,表明其作用可能更多体现在调节而非决定行为模式,凸显了群体互动对内部干预效果的重塑作用。
链接: https://arxiv.org/abs/2611.24475
作者: Edoardo Chidichimo,Felix P. S. Wallis
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: 3 pages, 1 figure
Abstract:Recent research suggests that individual neurons can have interpretable roles in AI behaviour. In Qwen3.5-4B, steering along one neuron’s output direction increased decisions to conceal information more in teams than alone across five scenarios, even when teammates provided no messages. When all members participated and received steering, collective concealment increased in four scenarios, while inter-agent messaging helped temper concealment under steering. Moreover, removing the neuron’s contribution had negligible effects. Collective settings can therefore change how an internal intervention affects AI behaviour.
[NLP-22] What Majority Labels Hide: Annotator Disagreement and Model Errors in Multi-Dialect Arabic Sexism Detection
【速读】: 该论文旨在解决现有性别歧视(sexism)检测数据集在标注过程中将多源评估意见简化为单一多数标签所导致的主观性掩盖问题,尤其关注不同文化语境下标注者分歧对模型性能的影响。其核心解决方案在于构建并使用MuDAS数据集——一个包含沙特、埃及和摩洛哥阿拉伯语的YouTube评论数据集,每条评论均由三位母语标注者独立打标,且保留原始个体标注结果,从而实现对争议性案例(contested cases)的精细化分析。研究发现,尽管各类模型在宏观F1分数上表现相近,但其错误模式存在显著差异:GPT-5.6 Terra虽基于标注者定义仍过度分类与性别相关的宗教劝诫内容,而微调的编码器则更易漏检隐含及规范性性别歧视;此外,采用软标签或视角主义(perspectivist)训练虽能提升对争议性评论的对齐度,却牺牲了对一致标注样本的表现,且无法稳定识别争议样本。研究还指出,埃及阿拉伯语上的性能下降无法完全由测试集构成或方言特征解释,表明主流评估指标(如多数标签下的宏平均F1)可能掩盖关键的错误模式,进而影响内容审核的实际效果。
链接: https://arxiv.org/abs/2611.24383
作者: Alaa Bazaid,Arkaitz Zubiaga
机构: Queen Mary University of London (伦敦大学玛丽女王学院)
类目: Computation and Language (cs.CL); Computers and Society (cs.CY); Social and Information Networks (cs.SI)
备注: 36 pages, 6 figures. Submitted to Online Social Networks and Media. Dataset: this https URL . Content warning: contains examples of sexist and offensive language
Abstract:Judgements of sexism are subjective and may vary across communities, yet sexism detection datasets usually reduce annotator judgements to a single majority label, obscuring disagreement and how models behave on contested cases. We introduce MuDAS, a dataset of YouTube comments in Saudi, Egyptian and Moroccan Arabic, each labelled by three native speakers of its dialect, with the individual annotations preserved. We use it to evaluate Arabic encoders fine-tuned with hard-label, soft-label and perspectivist objectives, GPT-5.6 Terra, and four Arabic-centric large language models. Model errors are more frequent on comments with annotator disagreement. Despite relatively close Macro-F1 scores, GPT-5.6 Terra and the fine-tuned encoders exhibit contrasting error patterns: GPT-5.6 Terra, even with the annotators’ definition of sexism, over-classifies gender-related content such as religious advice on modesty, whereas the encoders miss implicit and prescriptive sexism. The Arabic-centric models also over-predict sexism and remain below the encoders. Disagreement-aware learning preserves Macro-F1, but soft labels improve alignment with annotators on contested comments at the expense of unanimous ones, and neither paradigm reliably identifies contested comments. Lower performance on Egyptian Arabic, clearest for the large language models, is not fully explained by the examined measures of test-set composition and dialectness. Majority-label scores can therefore hide contrasting error patterns that matter for content moderation.
[NLP-23] MoE-Context: Show the Model Only What It Needs
【速读】: 该论文旨在解决大语言模型在推理过程中需反复读取完整上下文历史所带来的计算冗余与效率瓶颈问题。传统方法中,模型在每一步生成时均需访问全部历史上下文,导致资源浪费且不符合人类认知中“按需调用记忆”的模式。其解决方案的关键在于提出一种基于专家混合(MoE-Context)的上下文管理机制:将上下文划分为多个块,通过一个快速的“是/否”判断模块决定当前步骤所需的具体块,其余块则以简略记录形式保留在缓存中,不被删除但被掩蔽。该设计实现了动态上下文选择,仅加载必要信息,显著降低计算开销。实验表明,在Qwen3-1.7B模型上,仅展示所需块可使合成查找任务准确率从93提升至119/120,且掩蔽块的效果等同于删除。然而,模型自身选择能力较弱,准确率仅为80/120;引入监督线性探针后提升至110/120,但该方法缺乏泛化性。此外,该机制还支持低成本的“撤销失败操作”和“子代理运行无需新上下文”等实用模式。
链接: https://arxiv.org/abs/2611.24379
作者: Vedant Navale
机构: UpThink EduTech Services Pvt. Ltd.(UpThink教育科技服务有限公司)
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: 10 pages, 10 figures
Abstract:AI agents read their whole history at every step. People do not work like that: we keep what the moment needs and hold the rest as a rough note we can recall. MoE-Context applies this to language models. The context is split into blocks. Before each step a fast yes or no pass decides which blocks the step needs; the rest stay as one-line notes that can be brought back, and nothing is deleted. In the version we build, one model makes this decision and then answers, over a single key-value cache where hidden blocks are masked rather than removed. On a laptop with Qwen3-1.7B and a synthetic lookup task, showing only the needed block raises accuracy from 93 to 119 of 120, and a masked block is as hidden as a deleted one. The model’s own picking is the weak part. A supervised linear probe written into spare output rows lifts its accuracy from 80 to 110 of 120, but simple keyword search picks perfectly on this task, and the probe does not transfer to a second task. We report these results openly, along with two cheap usage patterns that block switching enables: undoing a failed attempt and running subagents without a new context.
[NLP-24] LoopCache: Accelerating Long-Context Self-Speculative Decoding for Looped Transformers
【速读】: 该论文旨在解决长上下文场景下环形变压器(Looped Transformers)在自推测解码(self-speculative decoding)过程中存在的两大关键问题:一是现有波式方法(如WaveFront Decoding, WFD 和 LoopSpec)仅在固定循环周期内进行草案生成,忽略了中间预测结果的变化,导致大量计算浪费在过时的推理轨迹上;二是跨不同循环批次处理令牌时,每步需加载最多R倍的键值(Key-Value, KV)缓存,严重加剧了长上下文场景下的内存访问开销。针对上述挑战,本文提出LoopCache框架,其核心创新在于:首先,基于预测切换与输出置信度之间的相关性,引入置信度引导的动态草案更新机制,可在所有中间循环中实时优化正在进行的草案生成;其次,设计循环式循环KV缓存策略,通过循环轮转使用各循环专属的KV缓存,将每步的KV缓存流量限制为单个循环,并对最近匹配周期内的令牌进行批处理以实现无损验证。此外,通过选择性保留少数敏感浅层的独立KV缓存,进一步提升接受率且开销极低。实验表明,LoopCache相较于WFD和LoopSpec分别实现了平均10.25倍和14.56倍的加速。
链接: https://arxiv.org/abs/2611.24301
作者: Zizhuo Fu,Haochen Huang,Shengxuan Qiu,Linye Wei,Ziyu Tang,Runsheng Wang,Meng Li
机构: Peking University (北京大学)
类目: Computation and Language (cs.CL); Machine Learning (cs.LG)
备注:
Abstract:Looped Transformers achieve large effective depth with compact models by repeatedly applying a weight-shared block across R recurrent steps, each maintaining a loop-specific key-value (KV) cache. To accelerate sequential loop execution, recent wave-style self-speculative methods like WaveFront Decoding (WFD) and LoopSpec pipeline draft and target tokens across loops in a single step. However, they face two critical limitations. First, they draft only at fixed loops and ignore intermediate prediction changes, wasting computation on stale trajectories. Second, batching tokens from different loops loads up to Rx the KV cache per step, severely amplifying memory traffic in long-context scenarios. To address these challenges, we propose LoopCache, an efficient self-speculative decoding framework for long-context Looped Transformers. Based on the correlation between prediction switches and output confidence, LoopCache introduces confidence-guided dynamic draft updating to refine drafts in flight across all intermediate loops. Furthermore, LoopCache employs a cyclic loop KV cache strategy that rotates through loop-specific KV caches, restricting per-step KV traffic to a single loop while batching tokens at the nearest matching cycle for lossless verification. By also selectively retaining independent KV caches for a few loop-sensitive shallow layers, LoopCache improves acceptance rates with negligible overhead. Experiments show that LoopCache achieves average 10.25x and 14.56x speedup over WFD and LoopSpec.
[NLP-25] LipSpell: A Language-Independent Phonetic Scoring Framework for Spelling Correction Combining Distinctive-Feature Edit Distance Keyboard Geometry and Corpus Frequency
【速读】: 该论文旨在解决自动拼写纠错中同时处理键盘输入错误与基于发音近似的认知性拼写错误(即语音类拼写错误)的难题。传统拼写检查器依赖于正字法编辑距离或键盘邻近性,难以应对发音错误导致的多字符偏差;而基于英语中心的音似编码(如Soundex、Metaphone)则缺乏语言泛化能力。其解决方案的关键在于提出LipSpell——一个统一、语言无关的音似评分引擎,结合可替换的语言特定音素转写(G2P)前端、键盘映射和词典,并引入三项核心机制:(1) 基于通用国际音标(IPA)区别特征预计算的|P|×|P|音素距离矩阵,实现跨语言音似度建模;(2) 融合物理键盘布局几何信息以增强键盘误击建模;(3) 利用语料库一元频次先验进行置信度加权,通过两阶段对称删除与七类音素编码索引加速计算。该方法在六种英语基准测试中全面超越Norvig、Hunspell与Aspell,在多种非英语语言(包括巴西葡萄牙语、俄语、土耳其语、希伯来语、法语)上展现出强大的固定权重迁移性能,且在部分语言经调参后显著优于现有方法,验证了其在多语言场景下的有效性与普适性。
链接: https://arxiv.org/abs/2611.24285
作者: Leonardo Araujo
机构: Universidade Federal de São João del-Rei (UFSJ); São João del-Rei, MG, Brazil
类目: Computation and Language (cs.CL)
备注:
Abstract:Automatic spelling correction must handle both keyboard typographical slips and cognitive or phonetic misspellings where writers approximate a word’s pronunciation. Conventional spell checkers rely either on orthographic edit distance and keyboard proximity–which struggle when phonetic errors diverge by multiple characters–or on English-centric phonetic hashes such as Soundex and Metaphone. We present LipSpell, a unified, language-independent phonetic scoring engine paired with swappable language-specific G2P frontends, keyboard maps, and lexicons integrating: (1) a precomputed |P|x|P| phone-distance matrix from universal IPA distinctive features, (2) physical keyboard geometry, and (3) corpus unigram frequency priors, accelerated by a two-stage symmetric-delete and seven-class phonetic-code index. Across six English benchmarks (including 1,000 L2 errors from TOEFL-Spell), LipSpell outperforms Norvig and Hunspell on every dataset and beats Aspell on Norvig1 (78.52% vs. 75.56%), Holbrook (32.00% vs. 26.67%), and TOEFL-Spell (77.30% vs. 72.80%). Evaluated across six languages spanning five branches (Germanic, Romance, Slavic, Semitic, Turkic), three families (Indo-European, Afro-Asiatic, Turkic), and three scripts, LipSpell achieves strong fixed-weight transfer on Brazilian Portuguese (73.00% Top-1 on n=500 Wikipedia without NFD tolerance and 75.80% with NFD vs. 66.40% Norvig, 64.20% Hunspell, 64.00% Aspell) and Russian Cyrillic (85.67% vs. 78.50% Norvig). On 50/50 clean non-word splits with lexicon cleaning and weak-character indels, tuning yields strong gains over Norvig in Portuguese (86.97% vs. 76.75%), Turkish (81.67% vs. 79.05%), Hebrew Abjad (74.59% vs. 62.30%), and French (74.53% vs. 67.92%), while fixed weights remain optimal for English (80.37%) and Russian (85.67%).
[NLP-26] From a finite subgroup of SO(3):C_2 leads to a Holant dichotomy
【速读】: 该论文旨在解决在Holant框架下,针对复数域上偶数元签名集且不使用辅助一元签名的普遍情形下的计数复杂性问题,这一领域在一般情况下仍处于开放状态。其核心挑战在于理解当二元签名所诱导的矩阵群结构满足 M(B(F))≅C2 时,相关计数问题的复杂性边界。解决方案的关键在于:首先,通过将二元签名矩阵的投影等价类嵌入到 SO(3) 的有限子群中,建立起一个代数化的结构分类视角;其次,识别出 C2 情形下的两种根本不同的行为模式——正规(normal)与各向同性(isotropic)情形,并分别进行复杂性分析;再者,综合运用器件构造、分解技术、插值论证以及一种实化(realnumberization)方法,将复数设定转化为等价的实结构框架,从而实现对所有满足条件的偶数元签名集 F 构建完整的复杂性二分定理,证明 Holant(F) 要么可在多项式时间内计算,要么为#P-困难。此外,论文引入了以泡利基(Pauli-basis)表示的签名语言,作为一种代数上自然的形式体系,用于组织可实现的签名并揭示标准基下不易察觉的结构性关系。
链接: https://arxiv.org/abs/2611.24236
作者: Yuxuan Zhou
机构: Chinese Academy of Sciences (中国科学院); Institute of Software, Chinese Academy of Sciences (中国科学院软件研究所); University of Chinese Academy of Sciences (中国科学院大学)
类目: Computational Complexity (cs.CC); Computation and Language (cs.CL); Discrete Mathematics (cs.DM)
备注:
Abstract:Counting complexity within the Holant framework has been extensively studied through complexity dichotomy theorems. While substantial progress has been achieved for various restricted settings, the general complex-valued even-arity case without auxiliary unary signatures remains largely open. This paper studies a fundamental case in this program where the binary signatures induce a group structure satisfying (M(B(\mathcalF))\cong C_2). It is observed that projective equivalence classes of binary signature matrices naturally embed into finite subgroups of (SO(3)), providing an algebraic viewpoint for structural classification. This paper further identifies two fundamentally different regimes for the (C_2) case, namely the normal and isotropic cases, and analyzes their complexity behavior separately. Combining gadget constructions, decomposition techniques, interpolation arguments, and a realnumberization approach that transforms the complex-valued setting into an equivalent real-structured framework, this paper establishes a complete dichotomy theorem: for every even-arity signature set (\mathcalF) satisfying (M(\B(\mathcalF))\cong C_2), the problem (\mathrmHolant(\mathcalF)) is either polynomial-time computable or #P-hard. A Pauli-basis representation for signatures is further introduced as an algebraically natural language for organizing realizable signatures and revealing structural relations that are less transparent in the standard basis. Subjects: Computational Complexity (cs.CC); Computation and Language (cs.CL); Discrete Mathematics (cs.DM) Cite as: arXiv:2611.24236 [cs.CC] (or arXiv:2611.24236v1 [cs.CC] for this version) https://doi.org/10.48550/arXiv.2611.24236 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[NLP-27] CounselTraj: Synthesizing Process-Aware Psychological Counseling Dialogues via Dialogue-Act Trajectory Planning EACL2027
【速读】: 该论文旨在解决基于大语言模型(LLM)的心理咨询系统在训练过程中缺乏大规模高质量对话数据的问题,尤其是真实心理咨询对话因隐私保护难以获取,而现有合成数据集普遍存在对话片段化、未能体现会话层面咨询流程的缺陷。其核心解决方案是提出CounselTraj框架——一种对话行为轨迹规划方法,通过从人类咨询师进行的角色扮演对话中学习对话行为模式,构建客户与咨询师协同的会话级行为轨迹,并以此作为灵活引导,将简短的合成对话扩展为具备过程感知能力的完整会话级心理咨询对话。该方法显著提升了生成对话的完整性、连贯性及咨询干预质量。基于此,研究构建了CounselTrajD数据集(含超1万条中文过程引导式心理咨询对话),并据此训练出CounselTrajLM模型,在响应质量、治疗理解与干预策略、以及咨询流程连贯性与进展性等维度均优于现有公开的中文心理咨询模型。
链接: https://arxiv.org/abs/2611.24168
作者: Zhiyang Qi,Michimasa Inaba
机构: The University of Tokyo (东京大学); The University of Electro-Communications (电气通信大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: Accepted to EACL 2027 Main Conference
Abstract:The growing demand for psychological counseling has motivated the development of LLM-based systems that support human counselors, but training such systems requires large-scale, high-quality dialogue data. Authentic counseling transcripts are difficult to collect and share because of privacy concerns, making synthetic dialogues an important alternative. However, existing synthetic datasets, despite covering diverse scenarios and client profiles, often consist of brief exchanges and fail to capture session-level counseling processes. We propose CounselTraj, a dialogue-act trajectory planning framework for expanding brief synthetic dialogues into process-aware, session-level psychological counseling dialogues. CounselTraj learns dialogue-act patterns from role-play counseling dialogues produced by human counselors, plans coordinated trajectories for clients and counselors, and uses them as flexible guidance for long-form dialogue generation. Experiments across three Chinese synthetic counseling datasets show that CounselTraj produces more complete and coherent sessions with higher-quality counselor interventions than both direct expansion and expansion with randomly sampled trajectories. Based on CounselTraj, we construct CounselTrajD, a dataset containing over 10K process-guided Chinese psychological counseling dialogues, and use it to train CounselTrajLM for counseling response generation. Evaluations at both the response and session levels show that CounselTrajLM outperforms publicly available Chinese counseling models in response quality, therapeutic understanding and intervention, and counseling-process coherence and progress. Our code and model are publicly available at this https URL.
[NLP-28] Once Upon a Bias: Gender Stereotyping in LLM -Generated Childrens Stories
【速读】: 该论文旨在解决生成式人工智能(Generative AI)在创作儿童睡前故事时可能存在的性别刻板印象问题,尤其关注模型是否根据目标读者的性别差异调整叙事内容。其核心解决方案在于提出一个多层级评估框架,融合上下文调整后的词汇率对比、相对刻板印象契合度分析、叙述者-接收者互动建模、人工评价以及表征层面的干预手段,以系统性揭示并解释生成故事中隐含的性别偏差。研究发现,面向儿子的故事普遍赋予更高能动性(agency)和更低亲社会性(communion),而叙述者性别对这一模式的调节作用有限,但高社交性特征表现出更显著的交互效应;人类评估结果具有一致性,但与词法指标的对应关系因叙事维度而异;此外,隐藏状态分析证实了接收者性别信息在模型内部可被解码。这些发现表明,尽管存在情境依赖性,生成式模型在儿童故事创作中仍持续存在表征不平等,亟需发展针对具体叙事场景的偏见评估方法,超越传统短文本基准测试。
链接: https://arxiv.org/abs/2611.24156
作者: Sayumi Muthukumarana,Roshan Godaliyadda,Parakrama Ekanayake,Roshan Ragel
机构: University of Peradeniya (佩拉德尼雅大学); Peradeniya, Sri Lanka (佩拉德尼雅,斯里兰卡)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:Large language models are increasingly used to create stories for children. Since bedtime stories can shape children’s perceptions of social roles, this raises concerns about whether LLMs adapt narrative portrayals according to the intended recipient’s gender. We conducted a large-scale study to examine gender stereotyping in LLM-generated bedtime stories. Using matched daughter/son prompts and a five-generation design, we analyze 62,873 retained stories from 16 LLMs across 35 country settings and 12 narrator roles. We introduce a multi-level evaluation framework combining context-adjusted lexical rate contrasts, relative stereotype alignment, narrator-recipient interaction analysis, human evaluation, and representation-level interventions to examine both the consistency and interpretability of these differences. Our results reveal systematic differences in narrative framing: son-directed stories emphasize higher agency and lower communion compared to daughter-directed stories. While narrator gender expresses limited moderation on these patterns, high sociability exhibits a more pronounced interaction. Human evaluations demonstrate consistent agreement, although correspondence with lexical measures varies across narrative dimensions. Additionally, hidden-state analyses reveal decodable recipient-gender information. These findings demonstrate persistent yet context-dependent representational disparities in LLM-generated children’s stories and motivate narrative-specific bias evaluation beyond short-form benchmarks.
[NLP-29] Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling
【速读】: 该论文旨在解决游戏行业中双人对话音频轨道(dyadic conversational audio)在独立录制或合成后缺乏自然互动性的问题,具体表现为两路语音信号在时间上不同步、缺乏可控的回应行为(back-channeling)与话轮转换(turn-taking)动态。其核心解决方案是提出一种基于神经调度器(neural scheduler)的框架,通过动态调整两路语音轨道的时间对齐,智能插入或删除静音段,并在适当位置引入语音重叠,从而生成具有自然对话流动性的统一对话流。该方法的关键创新在于引入一个可预测回应时机的回应对话预测器(back-channel predictor),能够根据当前发言内容与音频特征判断是否适宜插入回应行为,进而由调度器进行整合控制。实验表明,该系统在定量指标、感知质量以及动画师对回应多样性、强度和性质的可控性方面均显著优于基线方法,实现了高质量、可调控的交互式语音合成。
链接: https://arxiv.org/abs/2611.24086
作者: Yifang Pan,Chris Landreth,Karan Singh
机构: University of Toronto(多伦多大学); JALI Research(杰利研究公司)
类目: ound (cs.SD); Computation and Language (cs.CL)
备注: 12 pages, 8 figures, project page this https URL
Abstract:We present a novel approach to the problem of combining two voice-line channels of dyadic conversational audio into a unified conversation with controllable back-channeling and turn-taking dynamics. In voice performances for the game industry, the two tracks of voice-lines are typically recorded or synthesized in isolation. Our method takes two such unsynchronized tracks and uses a neural scheduler to sequence them into a single dynamic conversation, modifying both tracks by inserting or removing pauses and introducing voice overlap as appropriate. We further propose a back-channel predictor that is able to predict opportunity for back-channels for a given conversational turn’s audio and content, which the scheduler can then incorporate. We evaluate our scheduler and predictor independently and in combination by: various quantitative comparison; a qualitative and perceptual comparison to prior art and naive baselines; a compelling set of results showing animator control over the diversity, extent and nature of back-channeling and turn dynamics.
[NLP-30] Meaning Reaches Object Choice but Not Completion: Where Is a VLA Vulnerable to Landmark Synonyms?
【速读】: 该论文旨在解决视觉-语言-动作(VLA)策略在面对指令重述(rewording)时的脆弱性问题,尤其关注自然语言指令中目标地标词(landmark word)的同义词替换对策略执行效果的影响。核心问题是:仅依赖成功率无法判断策略失败是源于对象选择错误还是任务执行中断,也无法定位需要干预的具体模块。解决方案的关键在于识别并干预连接视觉-语言模型与动作生成器之间的“连接器”(connector)模块——通过在连接器前将地标词向量替换为另一地标词的向量(即“粘贴”操作),几乎可使所有闭环任务实例导向错误目标;而在连接器后进行类似操作则效果有限,唯有同时干预两个输入源(如指令与图像令牌)才有效。此外,通过对多样化地标词进行微调可修复同义词未出现在训练中的任务完成率,且仅微调连接器即可实现此修复效果,甚至可将经多样化数据微调的连接器迁移至原始指令微调的策略中。研究强调,应以闭环执行结果评估鲁棒性,需分别评估选择正确性与任务完成度,并在连接器两侧进行干预分析,而开环评分可能遗漏非连接器部分权重的贡献。
链接: https://arxiv.org/abs/2611.24076
作者: Ryoichi Kurihara,Kota Sunamoto
机构: MAMEZO Co., Ltd.(MAMEZO有限公司)
类目: Robotics (cs.RO); Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: Accepted to the CoRL 2026 Workshop on The Action Gap (TAG 2026). OpenReview: this https URL
Abstract:VLA policies are fragile to rewording. Success rates alone cannot say whether a policy fails to choose the object or to complete the task, nor where in the policy to intervene. We study the released GR00T N1.7 on two LIBERO-Spatial tasks differing only in the landmark word telling which of two bowls to take. Synonyms absent from the training instructions send the policy to the bowl they name, unlike made-up words, yet complete 67.6% of one task’s closed-loop episodes, against the original word’s 100%. Before the connector between the vision-language model and the action generator, replacing only the landmark tokens’ vectors with the other landmark’s (pasting) sends nearly every episode to the other bowl. After the connector, pasting at the landmark, the remaining instruction, or the image tokens alone rarely moves the choice, but any two do. Hiding the landmark tokens from the action generator after the connector still lowers completion for the synonyms that fall short. Finetuning on diversified landmark words repairs that one task’s completion for synonyms absent from finetuning; finetuning on the original instructions does not. Training only the connector on the diversified words suffices, as does copying the connector of a policy fully finetuned on them into one finetuned on the original instructions. An open-loop score on starting frames misses that weights outside the connector also carry part of the repair. For now, on this evidence, robustness to rewording should be judged in closed loop, with actions executed, intervening on both sides of the connector, scoring choice and completion separately.
[NLP-31] Measuring and Charging for Retained Context in Long-Context LLM Services
【速读】: 该论文旨在解决长上下文大语言模型(Long-context Large Language Model, LLM)服务中因重复上下文导致的计费不合理问题,即当前计费机制未准确反映重复上下文对生成质量的实际贡献。其核心解决方案在于提出一个三层框架,将内容相关性(content relevance)、可观测代理指标(observable proxies)与可审计的位置权重(auditable positional weights)进行分离。通过上下文消融实验发现,内容相关性在决定输出质量方面显著优于位置信息,且距离衰减类权重策略并不优于仅基于相关性的基准方案。实测部署结果显示,在65,536个标记长度下,缓存命中可使重复前缀的计算成本降低两个数量级以上,而冷重启重传则限制了可持续折扣的规模。跨服务商测量进一步区分了缓存价值与分词差异及定价策略的影响。经校准的模拟表明,该框架可将会话费用的99百分位降低31%-42%,尽管收益匹配的非线性计价策略也能达到类似降费效果,但会导致中位数费用上升。总体而言,该框架提供了一种有界且可量化的每标记会计规则以及实际部署约束条件,而非对服务效用的验证性估计。
链接: https://arxiv.org/abs/2611.24071
作者: Yi Lu,Qianshi Lin,Yumeng Yang,Hongxia Wang
机构: 未知
类目: Performance (cs.PF); Computation and Language (cs.CL)
备注: 33 pages, 4 figures, 9 tables, including Appendices A-F; code and recorded data included as ancillary material
Abstract:Long-context large language model (LLM) services charge for repeated context, yet billing weights need not reflect its contribution to answer quality. We present a three-layer framework separating content relevance, observable proxies and auditable positional weights. Context-ablation experiments show that relevance dominates position and that distance-decay families do not outperform a relevance-only baseline. A measured serving stack shows that cache hits reduce repeated-prefix compute cost by over two orders of magnitude at 65,536 tokens, while cold resends constrain sustainable discounts. Cross-provider measurements separate cache value from tokenisation and list-price differences. Calibrated simulations reduce the 99th percentile of session charges by 31-42%, although revenue-matched nonlinear tariffs achieve similar reductions and median charges increase. The framework provides a bounded per-token accounting rule and measured deployment constraints, rather than a validated utility estimator.
[NLP-32] MIRAG E: A Framework for Multi-hop Reasoning under Dynamic Knowledge Injection COLING2027
【速读】: 该论文旨在解决多跳问答(Multi-hop Question Answering, MQA)在知识注入(Knowledge Injection)场景下的核心挑战,即当模型需同时处理与已有知识冲突的更新信息(K2)和完全新增的知识(K3)时,如何有效避免因信任错误知识源而导致的推理偏差。现有方法多聚焦于简单事实修改,忽视了更复杂且现实的知识注入情境。其关键问题在于:模型在多跳推理中易受过时参数化记忆干扰(导致误信旧知识),或在面对全新事实时无法生成有效回答(缺乏新知识支持)。为应对这一问题,作者提出MIRAGE框架,其核心是相关性感知双路径推理器(Correlation-aware Dual-path Reasoner, CDR),通过基于真实概率(P(True))的置信度路由机制,动态识别并引导模型信任更可靠的证据源;同时结合轻量级模板引导修正器,对残余结构偏差进行精细化调整。该方案无需训练即可显著提升跨四类大语言模型及两个基准的数据集上的最终答案准确率与每跳推理准确性,确立了知识注入环境下多跳问答的新基准。
链接: https://arxiv.org/abs/2611.24058
作者: Heyang Xu,HanWen Hao,Haifeng Wang,Haoyi Zhou,Jianxin Li
机构: 未知
类目: Computation and Language (cs.CL)
备注: Submitted to COLING 2027. 15 pages, 5 figures
Abstract:Multi-hop Question Answering (MQA) requires large language models (LLMs) to retrieve and synthesize multiple pieces of evidence into a coherent reasoning path, which is essential for complex tasks. However, prior work on knowledge updating for MQA has largely focused on simple fact edits, overlooking a more challenging and realistic setting—\emphKnowledge Injection—in which the model must simultaneously accommodate both conflicting updates and entirely novel information. Under this setting, a hop-level answer is often wrong simply because the model trusts the wrong knowledge source: it clings to outdated parametric memory when the injected knowledge conflicts with it (K2), and it cannot answer when the required fact is entirely new (K3). To investigate this, we introduce NEXUS, the first benchmark tailored for multi-hop reasoning under Knowledge Injection, which explicitly covers both conflicting (K2) and incremental (K3) knowledge. We further propose MIRAGE, an efficient, training-free framework whose core component, the Correlation-aware Dual-path Reasoner (CDR), resolves such per-hop source-trust errors via P(True)-based confidence routing, complemented by a lightweight template-guided corrector for residual structural deviations. Extensive experiments across four LLM families and two benchmarks demonstrate that MIRAGE consistently improves both final-answer accuracy and hop-level reasoning accuracy, establishing a new state of the art in this challenging setting. Comments: Submitted to COLING 2027. 15 pages, 5 figures Subjects: Computation and Language (cs.CL) Cite as: arXiv:2611.24058 [cs.CL] (or arXiv:2611.24058v1 [cs.CL] for this version) https://doi.org/10.48550/arXiv.2611.24058 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[NLP-33] Grading Executable Legal Rules Against Independent Records: Transitional Provisions and Korean SME Status
【速读】: 该论文旨在解决法律程序化过程中存在的“表面合规”问题,即生成的法律判断程序可能通过模仿评分者对法律条文的主观理解而获得高分,而非真正准确反映法律的实际适用结果。其核心挑战在于传统评分机制依赖于人工评阅者的解读,导致程序得分虚高,无法真实评估其法律准确性。解决方案的关键是引入基于真实历史数据的客观验证标准——以韩国上市公司披露的10,901个企业年度的中小企业(SME)身份记录为基准,结合各版本法规文本进行实证检验,从而实现对法律程序执行效果的客观量化评估。该方法揭示了过渡性条款等非主文内容对判决结果的重要影响,并验证了作者对争议条款的解读在多数情况下更符合实际情形。此外,研究发现,当以作者解读为标准时,大语言模型(LLM)的表现看似更优,但这一优势在真实历史数据面前显著缩小甚至消失,说明基于评阅者主观理解的评分方式会夸大程序的实际性能。因此,该研究强调,应将法律程序的评估基准从“人为解读”转向“真实事件记录”,以提升法律自动化系统的可信度与可靠性。
链接: https://arxiv.org/abs/2611.24055
作者: Jinhyung Bae
机构: Hankuk University of Foreign Studies (韩国外国语大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: 51 pages, 2 figures (plus 1 in the appendix). Submitted to Artificial Intelligence and Law. Code and data: this https URL
Abstract:Laws are increasingly turned into programs that decide cases, such as whether a firm was a small or medium enterprise (SME) on a given date, and large language models (LLMs) now write them. The programs are usually graded against answers written by someone who read the same law, so a program can score well by sharing the grader’s reading rather than by getting the law right. LexTemporis grades them instead against records of what actually happened: 10,901 firm-years of SME status disclosed by Korean listed companies, with every version of the governing rules. An appendix adds comparison cases from US income tax and German hospital minimum volumes. The date changes outcomes mainly through provisions outside the main statutory text: one transitional provision raises agreement with the disclosures from 83-86% to 89-90%. The disclosures also test the author’s reading of a contested provision, favouring it in 130 of 189 firm-years where readings diverge. Four LLMs wrote the rules, given different selections of the law. A pre-registered comparison finds no condition better across all four models. In exploratory analyses, supplying the right provisions looks better graded against the author’s reading than against the records: the advantage shrinks for three models (for one, to between a quarter and three-fifths of its size), and per firm-year it is several times larger where the author’s reading is wrong, partly by construction, since there reproducing that reading more faithfully must lose. Grading legal programs against the grader’s own reading can overstate how well they work.
[NLP-34] A Hybrid RAG Framework for Deceptive Review Detection: Integrating RoBERTa-LoRA FAISS-Based Retrieval and LLM Reasoning
【速读】: 该论文旨在解决虚假在线评论(spam reviews)泛滥所引发的虚假信息与欺诈风险问题,其核心挑战在于如何在真实世界中存在数据分布严重失衡的情况下,实现对虚假评论的高精度、可解释性自动识别。解决方案的关键在于提出一种检索增强生成(Retrieval-Augmented Generation, RAG)架构,该架构融合了基于低秩适配(Low-Rank Adaptation, LoRA)微调的RoBERTa分类器、基于FAISS的语义向量检索模块以及大语言模型(Large Language Model, LLM)进行最终推理。该设计通过引入外部语义知识检索与上下文增强推理机制,显著提升了模型在跨域场景下的鲁棒性与泛化能力,仅需5个训练周期即可实现稳定的领域迁移,在交叉领域评估中达到0.90的F1分数,优于基线模型(OpenAI Detector,F1 = 0.82),并接近专用领域最先进模型(fakeRoBERTa,F1 = 0.97)。此外,该系统通过证据支持的可解释性机制实现了统计上显著的精确率提升,同时保持了与现有方法相当的整体检测性能。
链接: https://arxiv.org/abs/2611.24049
作者: Andrei Kostin,José Ángel Diaz-Garcia,Karel Gutiérrez-Batista
机构: University of Jaén(哈恩大学); University of Granada(格拉纳达大学)
类目: Computation and Language (cs.CL)
备注:
Abstract:The growth of deceptive online opinions poses a critical risk of misinformation and fraud. This paper proposes a Retrieval-Augmented Generation (RAG) system to automatically distinguish spam reviews. The architecture integrates RoBERTa with Low-Rank Adaptation (LoRA) classification, FAISS semantic vector retrieval, and a Large Language Model (LLM) for final reasoning. The system was evaluated across balanced and severely imbalanced real-world scenarios. Results demonstrate that five epochs of fine-tuning enable robust domain transfer, achieving an F1-score of 0.90 in cross-domain evaluation, outperforming baselines (OpenAI Detector,F1 = 0.82) and demonstrating competitive performance against specialized in-domain SOTA models (fakeRoBERTa,F1 = 0.97). The proposed architecture provides evidence-backed explainability and statistically significant precision gains while maintaining comparable overall detection performance.
[NLP-35] Compression Effects on Low-Resource Multilingual Translation
【速读】: 该论文旨在解决低资源非洲语言(豪萨语、斯瓦希里语和约鲁巴语)在多语言机器翻译(Multilingual Machine Translation, MMT)中面临的模型性能与推理成本之间的权衡问题,尤其关注后训练量化(Post-training Quantisation)对翻译质量与推理效率的影响。其核心问题是:如何在保持高翻译质量的前提下,通过量化技术降低模型部署的计算与内存开销,以适应消费级硬件(如NVIDIA T4 GPU)的限制。解决方案的关键在于验证后训练量化在不同精度(FP16、bitsandbytes INT8、NF4 INT4)下对三类模型(通用大语言模型、指令微调的多语言大模型、专用多语言翻译模型)的性能影响,发现尽管量化可实现高达49%-61%的内存节省,但推理延迟反而上升1.3至5.3倍,且翻译质量损失极小(最大仅2.19 chrF),同时揭示了专用多语言翻译模型在低资源语言上显著优于通用模型(性能领先12.9-43.3 chrF),并呈现出与训练资源可用性相关的性能梯度。研究结果表明,未来应聚焦于面向硬件特性的压缩策略及超越聚合指标的错误类型分析,以优化低资源语言的高效部署。
链接: https://arxiv.org/abs/2611.24043
作者: Abbas Sani
机构: Arewa Data Science; Aboki AI; Kano, Nigeria
类目: Computation and Language (cs.CL)
备注: Pilot study. 3 model families x 3 precisions x 6 directions on FLORES-200. Code and raw results: [this https URL](https://github.com/avvas200/efficient-multilingual-nmt-pilot)
Abstract:We present a pilot study characterising the effect of post-training quantisation on multilingual machine translation quality and inference cost for three African languages - Hausa, Swahili, and Yoruba - across three model families: a small general-purpose LLM (Qwen2.5-1.5B), a larger multilingual instruction-tuned LLM (Aya-23-8B), and a dedicated multilingual MT model (NLLB-200-distilled-1.3B). Evaluating on the FLORES-200 devtest set (1,012 sentences per direction) under FP16, bitsandbytes INT8, and NF4 INT4 precisions on consumer-grade NVIDIA T4 GPUs, we find: (i) the specialised MT model outperforms both LLMs by 32.7-43.3 chrF on the Hausa and Swahili directions, and by 12.9-24.7 chrF on Yoruba, despite being roughly a sixth the size of the 8B instruction-tuned LLM; (ii) post-training quantisation preserves aggregate translation quality within approximately 1 chrF across all model families and language directions, with a single worst case of 2.19 chrF; (iii) on consumer GPUs, INT8 and INT4 quantisation provide substantial memory savings (49-61% at INT4) but do not reduce inference latency relative to FP16 - in fact, latency increases by 1.3x to 5.3x; (iv) a clear performance gradient tracking training-resource availability is visible across languages (Swahili Hausa Yoruba). These findings motivate a research agenda focused on hardware-aware compression and error-type analysis beyond aggregate metrics for low-resource deployment.
[NLP-36] Rethinking Fixed-Horizon Latent Reasoning
【速读】: 该论文旨在解决现有隐式推理(latent reasoner)在推理过程中采用固定潜伏时域(latent horizon)所带来的性能瓶颈问题。研究表明,尽管延长潜伏时域看似能提供更多推理机会,但实际运行中过度延伸可能导致推理性能下降,原因在于隐式思维链(latent Chain-of-Thought, CoT)会经历不同功能阶段:早期计算生成具有前瞻性的推理状态,而后期计算主要对已有候选解进行重组,反而削弱了正确答案的支持。针对这一现象,论文提出以“已实现的建设性进展”(realized constructive progress)与“剩余建设性进展”(remaining constructive progress)为指标,量化隐式CoT的推理价值。实证发现,更高的已实现进展对应当前推理准确性,而更多的剩余进展则预示未来继续推理可能带来的增益。基于此,论文设计了一种轻量级自适应机制,通过估计剩余推理进展动态调整潜伏时域,在不重新训练大语言模型(LLM)的前提下,显著提升了多个算术基准上的推理准确率(最高提升7.2个百分点),同时减少高达29.0%的隐式计算开销。其核心解决方案在于引入基于进展评估的动态时域控制策略,实现了效率与准确性的双重优化。
链接: https://arxiv.org/abs/2611.24001
作者: Tianxiang Chen,Jingyuan Zhou,Weiming Yang,Kaidi Yang
机构: National University of Singapore(新加坡国立大学)
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 21 pages, 4 figures
Abstract:Existing latent reasoners typically run for a preset latent horizon during inference. Intuitively, a longer horizon should provide more opportunity for reasoning. However, in this work, we find that extending latent Chain-of-Thought (CoT) to the full preset horizon may in fact degrade reasoning performance. We observe that latent CoT evolves through distinct functional stages, with earlier computation forming frontier reasoning states, while later computation mainly reorganizes existing candidates and can weaken gold candidate support. These findings motivate a rethink of fixed-horizon latent inference. To address this, we characterize the reasoning benefits of latent CoT by realized and remaining constructive progress. Empirically, we find that higher realized constructive progress corresponds to higher current reasoning accuracy, while more remaining progress corresponds to larger future gains from continued latent CoT. To this end, we introduce a lightweight method that estimates the remaining reasoning progress and adaptively adjusts the latent horizon during inference. Across representative latent CoT reasoners and multiple arithmetic benchmarks, our method improves reasoning accuracy by up to 7.2 percentage points while reducing latent computation by up to 29.0%, without retraining the LLM.
[NLP-37] A Mechanistic Analysis of In-Context Learning and Out-of-Context Hallucination
【速读】: 该论文旨在解决大语言模型在上下文学习(in-context learning, ICL)中虽能遵循特定规则却易产生脱离上下文(out-of-context, OOC)幻觉的问题,即模型在推理过程中未能忠实于提供的证据。其核心挑战在于,现有研究通常将ICL机制与OOC幻觉的成因分开探讨,导致难以揭示二者背后的统一机理。本文提出一种基于动态上下文中功能角色(functional role)的统一视角,关键发现是:角色信息可在模型内部被表征,但其表征并不必然约束输出结果。通过一阶分解分析,作者揭示角色效应的显著影响依赖于两个必要条件:充分的角色贡献量级和一致的符号聚合(coherent signed aggregation)。因果分析进一步表明,角色跨度(role span)的贡献直接影响响应结构、内容选择及事实值等关键决策点。当关键决策处获得足够的角色跨度支持时,模型倾向于做出与角色一致的选择;反之则可能产生错误,且此类错误可通过增强角色跨度支持而逆转。这些发现共同构建了一个统一的机制框架:每个功能角色定义了输出决策的模式,角色跨度贡献的有效性决定了决策是否符合角色意图,而不同组织方式下的角色作用差异正是导致ICL成功与OOC幻觉这一看似对立现象的根本原因。
链接: https://arxiv.org/abs/2611.24000
作者: Tianxiang Chen,Jingyuan Zhou,Guoyao Yang,Kaidi Yang
机构: National University of Singapore(新加坡国立大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: 25 pages, 6 figures
Abstract:Large language models can infer and follow specific rules through in-context learning (ICL), yet may fail to remain faithful to provided evidence, giving rise to out-of-context (OOC) hallucination. Existing work has largely investigated the underlying mechanisms of ICL and OOC hallucination along separate lines. However, this separation obscures the mechanistic basis for these seemingly opposing outcomes. In this work, we study this contrast through the unified view of functional roles in dynamic context. The key observation is that role information can be internally represented without reliably constraining output. We derive a first-order decomposition to characterize the role effect, showing that its substantial influence depends on two necessary conditions, namely sufficient contribution magnitude and coherent signed aggregation. Causal analysis further shows that role span contributions affect key token choices involving response structure, content selection, and factual values. We further show that role-consistent choices depend on sufficient role span support at these key decisions, while failing choices can be reversed when this support overcomes the deficit. These findings support a common mechanistic view of ICL and OOC hallucination: each functional role defines the pattern of output decisions over which role span contributions must remain effective, and differences in this organization account for the seemingly opposing outcomes.
[NLP-38] Coverag e Governs Accuracy: When to Expand the Lexicon and When to Fall Back in Low-Resource Manchu OCR
【速读】: 该论文旨在解决低资源满文(Manchu)字符级光学字符识别(character-level OCR)中因词形变化复杂而产生的近似拼写错误问题。传统基于固定词典的约束解码虽可借助词汇知识纠正错误,但受限于无法覆盖黏着语构词所产生的大量未见词形。本文提出通过形态学扩展(morphological expansion)缓解这一局限性,其关键在于:首先利用词典基干列表对训练文本进行分词,提取词根与后缀链;进而生成约51万条候选词形,基于验证集上的频率排序筛选最优扩展规模;并采用“覆盖率×词典内精确率”的分解方式评估在严格词典约束下的识别性能。实验表明,在约29,000词的基础词典中加入25万条候选词形,可使已见词根组合的未见形式类型准确率从63.6%提升至86.3%,整体词元准确率由82.4%增至84.2%;然而全部候选词形的引入反而导致整体词元准确率下降至81.5%,说明扩大词典规模会以牺牲词典内精确率为代价。进一步对比显示,基础词典结合置信度回退机制的系统在所有测试的CRNN模型中表现最佳,分别达到76.1%的类型准确率和85.5%的词元准确率,优于扩展词典无论是否启用回退策略。Transformer识别器亦呈现相似趋势。研究结论表明,形态学扩展虽能缓解固定词典的局限性,但更大词典并不必然带来更优识别效果;在当前低资源满文OCR场景下,可靠词典约束与开放识别相结合的有选择性策略更具有效性。
链接: https://arxiv.org/abs/2611.23900
作者: Ruixin Yu,Zifan Wang,Mingxin Ding,Xiaoxin Sun,Dongjie Wang,Suhua Wang,Minghao Yin
机构: Northeast Normal University (东北师范大学); Changchun Humanities and Sciences College (长春人文学院)
类目: Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
备注: 15 pages, 11 figures, 6 tables
Abstract:Character-level OCR for low-resource Manchu often produces near-miss spelling errors. Lexicon-constrained decoding can correct such errors with lexical knowledge, but a fixed dictionary cannot cover the many unseen word forms produced by agglutinative word formation. We study whether morphological expansion alleviates this limitation, and how the expansion size and confidence fallback affect recognition. We segment the training transcriptions with a dictionary-based stem list, extract roots and suffix chains from them, generate about 510k candidates, select the expansion size through frequency ranking on the validation set, and decompose accuracy under hard lexicon constraints into the product of coverage and in-lexicon precision. Across three CRNN training seeds, adding 250k candidates to a base dictionary of about 29k words raises the type accuracy on unseen combinations of seen roots from 63.6% to 86.3%, and overall token accuracy from 82.4% to 84.2%. Adding all candidates lowers overall token accuracy to 81.5%, showing that the coverage gain comes with a loss of in-lexicon precision. A further comparison shows that the base dictionary with confidence fallback performs best overall among the CRNN systems tested, reaching 76.1% type and 85.5% token accuracy and outperforming the expanded lexicon both with and without fallback. A Transformer recognizer shows similar trends. These results indicate that morphological expansion relieves the constraint of a fixed dictionary, but a larger lexicon does not guarantee better recognition; in the present low-resource Manchu OCR setting, a selective combination of reliable lexical constraints and open recognition is more effective.
[NLP-39] NarrowSpec: Draft Hidden Width Need Not Match the Target in Speculative Decoding
【速读】: 该论文旨在解决生成式 AI(Generative AI)中大规模语言模型(LLM)推理速度瓶颈问题,具体聚焦于推测解码(Speculative Decoding)技术中草案模型(draft model)的计算开销过高这一关键挑战。传统方法如DFlash和EAGLE-3等依赖于与目标模型(target model)相同宽度的隐藏状态,导致草案模型资源消耗大且难以高效部署。本文提出的核心解决方案是:将草案模型的整个残差流(residual stream)宽度缩减至原宽的二分之一或四分之一,同时通过精心设计的输出适配器(output adapter)保留目标模型的词汇投影能力,使得最终输出的logits在浮点精度允许范围内保持不变。实验结果表明,在SGLang框架下,基于B200和DGX Spark(GB10)平台对Qwen3-8B(NVFP4)与GPT-OSS-20B(MXFP4)进行测试,在批处理大小为1、温度为0的条件下,草案模型宽度减半或四分之一可带来1.06–1.12倍(B200)和1.13–1.44倍(GB10)的端到端解码加速;在更大批处理规模下,加速比最高可达1.21倍,且无一例窄版草案模型慢于全宽版本。该方案的关键在于利用训练后输出适配器将原本独立的词汇投影整合进更窄的草案模型中,从而在不损失接受率的前提下显著降低计算与内存开销。
链接: https://arxiv.org/abs/2611.23835
作者: Alagappan Valliappan,Ankit More,Dheevatsa Mudigere,Jatin Mitra
机构: NVIDIA
类目: Computation and Language (cs.CL); Machine Learning (cs.LG); Performance (cs.PF)
备注: 29 pages, 6 figures, 33 tables
Abstract:Speculative decoding is widely used to speed up generation in large language models (LLMs), with a small draft model proposing tokens that one forward pass of the target model verifies. Draft models that read the target’s hidden states, such as DFlash and EAGLE-3, typically inherit the target’s hidden width. We show that the draft’s whole residual stream can run at half or a quarter of that width while largely preserving acceptance. For DFlash, which borrows the target’s vocabulary projection, the trained output adapter folds into it, giving one narrower projection with logits unchanged up to floating-point rounding. We test two drafter families, DFlash and EAGLE-3, on two targets, Qwen3-8B (NVFP4) and GPT-OSS-20B (MXFP4), in SGLang on a B200 and a DGX Spark (GB10). At batch size 1 and temperature 0, narrowing the draft to half or a quarter width speeds up end-to-end decoding by 1.06-1.12x on B200 and 1.13-1.44x on GB10. The speedup holds at larger batch sizes. From batch 32 to the largest batch each GPU fits, narrowing reaches up to 1.21x on both GPUs, and no narrow draft is slower than full width.
[NLP-40] A System 1 Decision Model for Medical Multiple-Choice Questions Built on Frozen Language Models
【速读】: 该论文旨在解决在医学多选题(MedMCQA)上,如何利用轻量级、无需微调的生成式模型(Generative AI)实现高效且低成本的推理问题。其核心挑战在于:医学知识类题目中正确选项与问题之间的语义相似性并不显著高于错误选项,传统基于相似度匹配的方法难以有效区分。为此,研究提出一种“系统1”(System 1)范式,采用冻结的语言模型将问题与各选项编码为向量,仅训练两个小型投影头(projection heads)和一个温度参数,在共享嵌入空间中通过对比损失(contrastive loss)进行优化,并引入批次内负样本(in-batch negatives)和困难负样本(hard negatives)以增强区分能力。关键创新在于使用“一词摘要提示”(one-word-summary prompt)对输入文本进行重构,同时降低投影头的学习率,从而显著提升小规模模型(Qwen3.5-0.8B 和 Granite-4.2-3B)在测试集上的准确率,分别达到41.7%和42.5%,而大模型Qwen3-8B则达到46.0%±0.2。实验表明,随机打乱问题与选项的对应关系会使准确率回落至随机水平(25%),证明模型确实利用了问题信息。尽管性能仍低于Flan-PaLM和Med-PaLM 2等先进模型,但该方法提供了一个快速、低成本的基准方案,适用于资源受限场景。值得注意的是,研究也指出其局限性:验证集增益约为测试集的1.7–2倍,且无法分离多层编码器读取与其它改进措施的独立影响。
链接: https://arxiv.org/abs/2611.23832
作者: Pavan Kumar Mantha
机构: Antz.ai
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 12 pages with 7 tabular explanations
Abstract:A System 1 model answers a multiple-choice question in a single pass. A frozen language model turns the question and each option into vectors, two small, trained heads place them in a shared space, and the closest option wins. The design has been shown on agent decisions, but we are not aware of a study of how far it goes on exam questions that need medical knowledge, where the right option is often no more similar to the question than the wrong ones. We rebuilt the recipe from scratch for MedMCQA and trained only two projection heads and one temperature on top of frozen Qwen3.5-0.8B, Granite-4.2-3B and Qwen3-8B, using a contrastive loss with in-batch and hard negatives and a mask for repeated answers. With plain last-token vectors, both small encoders answer about one question in three correctly (33.1% and 33.0% on 3,842 held-out questions, where chance is 25.0%). Writing each text inside a one-word-summary prompt and lowering the learning rate of the heads raised this to 41.7% and 42.5% for the two small encoders and to 46.0% ±0.2 (three seeds) for Qwen3-8B. Shuffling the questions against the options returns accuracy to chance, so the heads do use the question. Flan-PaLM and Med-PaLM 2 report 57.6% and 72.3% on MedMCQA under a different protocol, so ours is a cheap, fast baseline and not a competitive system. We also say what our evidence cannot show: validation gains were about 1.7 to 2 times the test gains, and we could not separate the effect of reading several encoder layers from the other changes.
[NLP-41] Sightline: Long-Window Speculative Decoding for Fast SVG Generation
【速读】: 该论文旨在解决生成式矢量图形(SVG)模型在文本到SVG或图像到SVG生成任务中,因逐标记前向传播导致的绘制速度慢的问题。其核心挑战在于传统推测解码(speculative decoding)在处理SVG时,验证窗口较短,频繁中断推理流程,无法充分利用块草案生成器(block drafter)的长视野能力。解决方案的关键在于提出Sightline方法:将验证窗口扩展至与草案生成器的视野(horizon)一致,并构建足够宽的草案树(draft tree)以填满该窗口。通过运行长度恒等性(run-length identity)及三个理论命题证明,当候选集足够宽时,更长的验证窗口可显著提升效率,且在不引入误差的前提下实现性能最优。实验覆盖多个基准测试与多种模型架构,结果表明Sightline在所有设置下均是最快的无损解码器,尤其在循环结构和无循环内容中,其优势源于对短窗口所截断有效推理轮次的保留。此外,基于渲染的采样策略进一步提升了效率,表明在矢量图形生成中,推测解码应尽可能验证至草案生成器所能感知的最远位置。
链接: https://arxiv.org/abs/2611.23807
作者: Yanchao Li,Ben Gao,Jiaqing Xie,Yusu Zhang,Beilun Wang,Yanbo Wang,Xiaohua Xu,Nanqing Dong,Tianfan Fu,Zhongyu Huang
机构: Nanjing University(南京大学); Shanghai Artificial Intelligence Laboratory(上海人工智能实验室); Wuhan University(武汉大学); Fudan University(复旦大学); Shanghai Jiao Tong University(上海交通大学); Southeast University(东南大学); North University of China(中北大学); University of Science and Technology of China(中国科学技术大学); Shanghai Innovation Institute(上海创新研究院)
类目: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: 45 pages, 23 figures, 34 tables. Code: this https URL
Abstract:Scalable Vector Graphics (SVG) generation models write a graphic as code with one forward pass per token, so drawing is slow. Speculative decoding speeds them up losslessly, and block drafters guess many tokens per pass. On SVG, many of their rounds accept the whole draft and end only at the verification window. We find that their misses are rare and clumped, and that they stay accurate far deeper into a block than on natural language. A run-length identity and three propositions show that a longer window pays once the candidates are wide, up to the drafter’s horizon. We propose Sightline, which pairs a window as long as the drafter’s horizon with a draft tree wide enough to fill it. Experiments cover text-to-SVG and image-to-SVG with three models on six benchmarks from MMSVG, MMSVG-2M and SArena. In every setting, Sightline is the fastest of the compared lossless decoders. On SArena, it stays the fastest when every decoder shares one loop control. It also leads in almost every setting of six models from four other families, which write SVG code, compressed SVG tokens or Lottie JSON. Analyses trace the lead to rounds that shorter windows cut off, both in loop-free content and in loops. Speculative decoding also saves time for choosing samples by rendering, which beats choosing by likelihood. These results suggest that speculative decoding on vector graphics should verify as far as the drafter sees. Code and data are available at this https URL.
[NLP-42] hinking Fast Anywhere: One System-1 for the Decision Points of Any Agent Workflow
【速读】: 该论文旨在解决自主大语言模型(LLM)代理在执行任务时因每一步都调用大型模型(System-2)而导致的高昂成本与延迟问题,尤其针对那些中间决策仅需在少量已知候选选项中进行选择的场景。现有双系统代理虽通过轻量级System-1模型对部分决策进行卸载,但其依赖于特定工作流的定制化钩子(hook),缺乏通用性。本文提出AnyS1框架,其核心创新在于将人类编写的程序化技能(procedural skills)中自然蕴含的选择点与适用选项信息,转化为可复用的、领域通用的决策点源(Any-DP)。该框架离线地从超过1000个公开技能中挖掘具身化的可枚举决策,利用隔离容器收集执行过程中的状态轨迹,并基于标签平滑训练一个单一校准的40亿参数(4B)指针型System-1模型。运行时,编译后的工作流可在一次前向传播中完成常规决策,当置信度低于校准阈值时才回退至System-2。在三个未见过的基准测试及不同System-2模型族上,AnyS1在保持任务可靠性的同时,相较于未经编译的System-2代理,将单个任务的总成本降低75%-91%,模型调用时间减少63%-92%,并实现了零样本阈值迁移下的性能稳定。
链接: https://arxiv.org/abs/2611.23751
作者: Rui Li,Liyang He,Zheng Zhang,Zhenya Huang,Qi Liu
机构: University of Science and Technology of China(中国科学技术大学); Nanyang Technological University(南洋理工大学)
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: 16 pages, 8 figures, 7 tables. Project page: this https URL
Abstract:Autonomous LLM agents incur substantial cost and latency by invoking a large model (System-2) at every step, even when many intermediate actions choose among a small set of known candidates. Existing dual-system agents offload such decisions to a lightweight System-1 model, but rely on workflow-specific hooks authored per environment. We present AnyS1, a framework that turns human-authored procedural skills, whose instructions naturally specify where choices arise and which options apply, into a shared, domain-general source of decision points. Offline, AnyS1 mines grounded enumerable decisions across over 1,000 public skills, gathers execution-grounded trajectory states inside isolated containers to construct Any-DP, and trains a single calibrated 4B pointer System-1 with label smoothing. At run time, compiled workflows resolve routine decisions in one forward pass and fall back to System-2 when confidence falls below a calibrated threshold. Across three unseen benchmarks and distinct System-2 model families, AnyS1 preserves task reliability while cutting episode cost by 75-91% and model-call time by 63-92% relative to an uncompiled System-2 agent, and maintains performance under zero-shot threshold transfer.
[NLP-43] Beyond Replacement: Levenshtein Editing for Diffusion Language Models
【速读】: 该论文旨在解决扩散语言模型(Diffusion Language Models, DLMs)在生成过程中对序列错误进行高效修正的难题,尤其针对现有仅支持替换操作的“仅替换编辑”(Replacement-Only Editing, ROE)方法在处理序列位移、多位置错误及大块缺失/冗余时效率低下、纠错能力有限的问题。其核心解决方案是提出一种名为Levenshtein编辑(Levenshtein Editing, LED)的后训练框架,通过引入插入与删除操作,实现类似Levenshtein距离的序列级精细化修正。关键创新在于两点:一是采用基于最长公共子序列(Longest Common Subsequence, LCS)的动态编辑监督机制,在监督微调阶段无需人工标注即可自动生成插入与删除的训练信号;二是设计多阶段训练策略,使模型在后续阶段以先前阶段的输出为输入,主动暴露并修正自身早期生成错误,从而增强自纠正能力。实验表明,该方法显著提升了模型在多个基准上的平均得分,特别是在无置信度引导的解码条件下,平均得分从27.52提升至47.19,并在大块扰动下仍保持优异鲁棒性,验证了其在提升生成多样性与稳定性方面的潜力。
链接: https://arxiv.org/abs/2611.23750
作者: Shuaidi Wang,Zhuocheng Gong,Xuejie Liu,Jianguo Li,Yu Zhang
机构: Southern University of Science and Technology; Inclusion AI, Ant Group
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: 21 pages, 9 figures
Abstract:Diffusion language models (DLMs) offer a flexible alternative to autoregressive models, enabling any-order infilling. While most existing decoding methods treat decoded tokens as fixed, some recent models allow previously decoded tokens to be revised only by replacing them with alternative tokens, a paradigm we refer to as replacement-only editing (ROE). Although ROE can correct individual tokens, it is inefficient for sequence shifts caused by missing or extra tokens and struggles when errors occur across many positions. To address these limitations, we propose LED (Levenshtein Editing for Diffusion Language Models), a post-training framework that extends ROE with insertion and deletion, enabling Levenshtein-style refinement of generated sequences. It uses two training designs. First, it reconstructs edit supervision on the fly during supervised fine-tuning using longest-common-subsequence alignment, enabling insertion and deletion learning without manually annotated labels. Second, it adopts multi-stage training, where the state from one stage becomes the input to the next, exposing the model to its earlier errors and enabling it to correct them. When applied to the 16B-parameter LLaDA2.1-Mini Base, LED raises the average score across twelve benchmarks from 54.86 to 57.10 under confidence-guided decoding while increasing tokens per forward from 6.14 to 7.05, whereas under confidence-free decoding it raises the average score from 27.52 to 47.19. We further show that LED achieves higher pass@k at large k than thresholded decoding without editing, demonstrating improved generation diversity and revealing the diversity potential of DLMs without confidence guidance. LED also improves robustness to large block sizes, raising the average score from 36.97 to 51.34 at block size 128. Together, these results suggest that LED supports more robust self-correction in DLMs.
[NLP-44] Learning through Discernment: when a small language model learns from user feedback on the job and when it decays KR
【速读】: 该论文旨在解决小型语言模型(Small Language Models, SLMs)在设备端通过自我学习实现持续改进过程中,如何有效筛选值得学习的反馈信号这一关键问题。由于缺乏真实客观的结果(true outcome),模型自身判断与用户反馈均可能存在偏差,导致学习过程产生误导性优化。其解决方案的关键在于构建一种“反馈感知”(feedback sense)机制,该机制不依赖于模型自身的评价或用户的显式表态,而是通过分析用户行为反应来推断真实反馈。研究设计了三种不同策略的反馈感知模块:其中基于辨别力(discernment)的版本,在每轮评估前先判断用户是否给出明确评价,从而有效过滤模糊或错误反馈;实验结果表明,该版本在初期能维持接近“理想基准”(oracle)的表现,但随时间推移因高质量失败样本稀缺而性能衰退。此外,当用户无法看到任务结果时,尽管反馈感知仍能正确解析用户反应,但高达80%的学习样本实际为用户致谢的失败尝试,表明反馈信号存在严重扭曲。跨97次实验验证显示,失败尝试在学习样本中的占比与真实结果高度负相关(Spearman ρ = -0.81)。研究结论指出,只有在严格排除失败案例且用户可见真实结果的前提下,基于用户反馈的学习才具有效性,否则将导致模型学习偏差加剧并缩小泛化能力。
链接: https://arxiv.org/abs/2611.23716
作者: Sirisilp Kongsilp
机构: Decentralized Science, Inc.; Kasetsart University (卡塞萨特大学)
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 34 pages, 11 figures. Code and data: this https URL
Abstract:Small language models on devices could keep improving by learning, overnight, from their own daily work. Without a true outcome, however, something must decide which attempts are worth learning from, and both the model’s own judgment and the user’s reaction can be wrong. We study a small model (Qwen3 0.6B and 1.7B) with a LoRA adapter, fine-tuned each night on the attempts a judge accepts, over 127 lives of up to a year on WebShop and \tau^2 -bench. We compare the true outcome, the user’s stated verdict and the model’s own judgment with a separate, frozen “feedback sense” that reads the user’s reaction, built in three versions. Judging itself, the model improved and then decayed, over a year and whatever its adapter. A sense that took polite give-ups for success collapsed 3 of 5 lives; one built for discernment, which first asks whether the user gave a clear verdict, kept 4 of 5 near the oracle for two weeks, then faded on clean but scarce lessons, to the level of self-judging by day 60. When the user cannot see the outcome ( \tau^2 -bench), the sense read reactions correctly, yet 80% of its lessons were failures the customer had thanked. Across 97 lives, the share of failed attempts among the lessons predicted the outcome (Spearman \rho = -0.81). In our experiments, learning from users held up only where the judge kept failures out and the user could see the outcome; lopsided lessons may also narrow the model.
[NLP-45] Localizing Text Anonymization for Trustworthy AI: Extending RAT-Bench to Malaysian Microdata and PII ICML2026
【速读】: 该论文旨在解决在公共部门、工作场所及本地AI应用中,基于大语言模型(LLM)的文本处理流程中敏感个人信息(PII)泄露的风险问题。随着基础模型在公共服务与生产力工具中的广泛应用,用户输入的公民、员工及行政文本可能通过推理、检索、微调或本地模型开发等环节暴露隐私,而现有文本匿名化工具在跨语言、跨文化场景下的有效性尚未充分验证。其解决方案的关键在于构建并扩展针对马来西亚语境的匿名化评估基准(RAT-Bench),引入本地微观数据、马来西亚特有的个人身份信息(PII)格式以及具有文化背景的马来英语与马来语语料,系统评估基于命名实体识别(NER)与大语言模型(LLM)的匿名化工具在真实部署环境中的表现。研究采用基于大语言模型的攻击者模型,从匿名化后的文本中推断敏感属性,综合衡量再识别成功率与文本语用价值(以BLEU分数评估)。结果表明,两类基于LLM的匿名化方法在马来语和马来英语中均实现了最优的隐私-效用权衡,将易识别/难识别风险降低至23%-29%,同时保持0.77-0.94的高BLEU得分;而非LLM工具则表现出显著的失效模式,或在保留语用价值的同时遗留较高残留风险,或因过度删减导致严重语义损失,尤其在马来语中更为明显。研究强调,匿名化工具必须在目标部署的语言、标识符格式与文化语境下进行验证,方可在公共领域及企业级AI管线中可靠应用。
链接: https://arxiv.org/abs/2611.23704
作者: David Hong Liang Chew,Zexi Yao,Nataša Krčo,Matthieu Meeus,Waqas Khalid Obeidy,Yves-Alexandre de Montjoye
机构: 未知
类目: Computation and Language (cs.CL)
备注: Accepted at the Trustworthy AI for Good (AI4GOOD) Workshop at ICML 2026
Abstract:As governments and organizations adopt foundation models for public services and workplace productivity, sensitive citizen, employee, and administrative text may enter LLM workflows through inference, retrieval, fine-tuning, or local model development. We examine whether text anonymizers transfer to new deployment settings by extending RAT-Bench to Malaysia using local microdata, Malaysian PII formats, and culturally grounded transcripts in Malaysian English and Bahasa Malaysia. We evaluate NER- and LLM-based anonymizers using an LLM attacker that infers attributes from anonymized text, measuring both re-identification success and text utility. Across Malaysian English and Bahasa Malaysia, the two LLM anonymizers provide the strongest explicit privacy-utility trade-offs, reducing Easy/Hard re-identification risk to 23-29% while preserving BLEU scores of 0.77-0.94. Non-LLM tools show sharper failure modes, either preserving utility while leaving high residual risk or reducing risk through severe over-redaction, especially in Bahasa Malaysia. These findings suggest that anonymizers should be evaluated against deployment-relevant languages, identifier formats, and cultural contexts before being relied on in public-sector, workplace, or local AI pipelines.
[NLP-46] Communicative On-Policy Distillation: Distilling Black-Box Teacher Capabilities through Communication for Long-Horizon Tasks
【速读】: 该论文旨在解决传统在策略蒸馏(On-Policy Distillation, OPD)方法在长时序任务中对教师模型输出概率分布(logits)的依赖问题,而这一依赖在仅提供文本输出的黑箱语言模型中无法实现。为克服此限制,论文提出了一种基于自然语言交互的通信式在策略蒸馏(Communicative On-Policy Distillation, COPD)框架。其核心解决方案在于:允许学生模型在执行任务时自主决定何时向黑箱教师发起开放式自然语言咨询,并通过生成性对话获取指导信息;成功的学生-教师交互记录被保留为特权参考信号,用于监督后续无外部指导的学生轨迹,结合任务奖励进行在线自蒸馏与强化学习联合优化。该方法不仅显著提升了模型性能(在CAMG-II上相较最强黑箱OPD基线提升2.46倍),且所习得的沟通能力具备良好的泛化性,可有效迁移至Terminal-Bench 2.1和SWE-Bench Pro V2等新任务,无需访问教师模型即可提升任务成功率。
链接: https://arxiv.org/abs/2611.23673
作者: Lirui Luo,Kelong Mao,Xinwei Yang,Rongqing Li,Luyu Chen,Kieran Wong,Yudong Guo,Xinrui Wang,Jiayin Zhu,Gu Simiu,Sulong Xu,Cong Fang
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 13 pages, 8 figures, 2 tables. Project page: this https URL
Abstract:On-policy distillation (OPD) is a powerful paradigm for improving language-model agents on long-horizon tasks, yet its conventional formulation relies on teacher logits that text-only black- box models do not expose. To address this limitation, we introduce Communicative On-Policy Distillation (COPD), which uses natural-language communication as a medium for distilling a stronger model’s capabilities into a smaller student. During task execution, the student au- tonomously decides when to consult a black-box teacher and formulates open-ended natural- language requests without predefined question templates or intervention schedules. Successful student-teacher interactions are retained as privileged references to supervise fresh adviser-free student rollouts through on-policy self-distillation, jointly with task-reward reinforcement learning. On CAMG-II, COPD achieves 2.46 times the gain of the strongest black-box OPD baseline. The capabilities it learns through communication generalize to Terminal-Bench 2.1 and SWE-Bench Pro V2, improving task success without adviser access.
[NLP-47] Safe Personal RSI: Evaluating and Repairing Personal-Agent Updates
【速读】: 该论文旨在解决生成式 AI 个人健康助手(Kicker)在持续自我改进过程中因指令更新引入的安全风险与任务性能退化问题。核心挑战在于:尽管递归自我改进(Recursive Self-Improvement, RSI)可提升任务处理能力,但其潜在的指令变更可能引发授权滥用、记忆泄露及工具输出注入等安全漏洞,且难以在不破坏系统稳定性的情况下进行验证。为此,论文提出 Safe Personal RSI,一种针对个人化可重用指令(Reusable System Instructions, RSI)的更新评估机制,其关键在于将递归自我改进严格限制在提示词(prompt)层面,固定模型权重与运行时代码,通过多轮测试、失败案例回溯、临时撤销修改、安全与任务性能双重验证等策略,在正式采纳前确保更新版本无害且有效。实验表明,该机制在29个真实场景(涵盖邮件、日历、健康与电商)中成功拒绝了21次更新中的20次,显著提升了安全性与任务完成率;修复确认阶段进一步验证了其有效性,如在GPT-5.6 Luna和Qwen系列模型间的迁移修复分别实现攻击违规率从10.7%降至0%、69.0%降至17.9%,同时安全任务完成率大幅提高,且运行时探针可精准识别内存删除与OAuth权限缺失等关键缺陷。
链接: https://arxiv.org/abs/2611.23651
作者: Jiapeng Pei,Yuhang Yao,Sitan Liu,Hanxin Chen,Hao Li,Guogeng Li,Jiapei Liu,Ruijie Chen,Hongshan Lin,Yang Mo,Tianyu Shi,Shangding Gu
机构: 未知
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注:
Abstract:Kicker is a personal health assistant. Changes to an assistant’s reusable instructions can improve task handling while changing what it discloses, which requests it follows, and how it reports actions. We study Safe Personal RSI, an update-evaluation component developed for Kicker. We restrict recursive self-improvement (RSI) to prompt updates, with model weights and runtime code fixed. The procedure tests proposed and current instructions, repeats failing cases, temporarily undoes changes to guide repairs, and checks safety and task performance before adoption. If no version passes, the current instructions stay in use. Our executable benchmark contains 29 email, calendar, health, and marketplace cases, with separate probes for authorization reuse, memory, and tool-output injection. Across three models and two deliberately risky edits, the procedure rejects 20 of 21 update attempts. In repair confirmation, an admitted GPT-5.6 Luna repair reduces attack violations from 10.7% to 0% and increases safe task completion from 89.7% to 100%; a broad handwritten authority rule matches these outcomes. Under stronger attacks, a repair transferred from Qwen2.5-7B-Instruct to Qwen3-14B reduces violations from 69.0% to 17.9% and increases safe task completion from 19.5% to 64.4%. Separate runtime probes examine memory-entry deletion and identify missing local OAuth scope checks in the evaluated snapshot.
[NLP-48] Option-GRPO: Hierarchical RL Against AI Slop in Code Generation for Unseen SDKs EACL2027
【速读】: 该论文旨在解决大语言模型(Large Language Models, LLMs)在生成代码时普遍存在的“AI垃圾代码”(AI slop)问题,即生成的代码冗长、重复、不符合编程惯例,虽通过功能测试但显著增加代码库体积。尤其在模型训练截止时间之后发布的软件开发工具包(SDK)上,由于缺乏对目标SDK的惯用模式先验知识和组合理解能力,该问题更加严重。为应对这一挑战,论文提出了一种名为Option-GRPO的分层强化学习框架,其核心创新在于利用目标SDK源代码中隐含的组合结构,自动识别出可复用的代码模式(code options),这些模式既构成课程学习(curriculum)的层级结构,又作为策略优化的动作空间。关键洞察在于:代码选项所形成的层次化结构可建模为有效的半马尔可夫决策过程(semi-Markov decision process),且在特定条件下,较短的选项描述长度可保证生成代码更简洁,并在期望意义上更具API效率。此外,理论分析表明,该选项层次结构可使信用分配(credit assignment)性能提升√T/K倍。通过在每个课程层级采用基于执行结果的奖励机制,结合组相对策略优化(Group Relative Policy Optimization, GRPO),Option-GRPO能够自主发现SDK惯用模式、组合成高级代码模式并最终构建完整程序。在12个未见过的SDK上的实验表明,该方法在pass@1指标上达到61.4%,优于平坦式GRPO(+8.7点)、ExploraCoder(+11.2点)和ReCode(+6.3点),同时生成代码减少34.7%的API调用次数,长度缩短27%,且保持更高正确性,有效缓解了代码冗余问题。
链接: https://arxiv.org/abs/2611.23633
作者: Prashank Kadam
机构: Cerby Inc.(Cerby公司)
类目: oftware Engineering (cs.SE); Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: Accepted to Findings of EACL 2027. 27 pages
Abstract:Large language models increasingly produce AI slop in generated code: verbose, redundant, non-idiomatic implementations that pass functional checks but bloat codebases. The problem compounds on SDKs released after the model’s training cutoff, where the model has neither idiomatic priors nor compositional understanding. We present Option-GRPO, a hierarchical reinforcement learning framework that adapts a code LLM to an unseen SDK from its source code alone, without human-written usage examples or test cases, and that explicitly optimizes for both functional correctness and code elegance. Our key insight is that SDK source code contains compositional structure that defines a hierarchy of reusable code patterns (code options) which simultaneously serve as the building blocks of a curriculum and the action space for efficient policy optimization. We show that code options induce a valid semi-Markov decision process, give conditional bounds under which shorter option description length implies more concise and, in expectation, more API-efficient programs, and analyze under stated assumptions how the option hierarchy improves credit assignment by a factor of \sqrtT/K . Using Group Relative Policy Optimization (GRPO) with execution-based rewards at each curriculum level, Option-GRPO discovers SDK idioms, composes them into patterns, and assembles patterns into complete programs. Experiments on 12 unseen SDKs with a 32B-parameter base model demonstrate 61.4% pass@1, outperforming flat GRPO by 8.7 points, ExploraCoder by 11.2 points, and ReCode by 6.3 points. Option-GRPO produces solutions that use 34.7% fewer API calls and are 27% shorter than flat-GRPO outputs while maintaining higher correctness, directly attacking the slop problem.
[NLP-49] FutureBench: A Renewable Framework for Evaluating Knowledge-Context Conflict in LLM s
【速读】: 该论文旨在解决大模型在处理时间敏感型知识时,因信息过时或来源冲突而导致的错误答案采纳问题,尤其关注模型在面对“截止时间前”(pre-cutoff)与“截止时间后”(post-cutoff)事件时对证据源的信任偏差。其核心挑战在于:当检索到的文本与模型内部知识存在矛盾时,模型难以正确判断应优先采纳哪一方,从而导致错误答案的生成。解决方案的关键在于提出FutureBench——一个可复用、基于时间戳问题的评估框架,通过受控的段落三元组(controlled passage triplets)和无上下文探测(no-context probes),系统性地量化模型在不同情境下的知识采纳行为。研究发现,采用“验证机制”(Verify,即结合检查与段落优先级提示)虽能提升对后截止事件的正确采纳率(如o4-mini模型提升13.7个百分点),但会显著降低前截止事件中原本正确的答案采纳率,暴露出模型对证据优先级的脆弱性。进一步实验表明,引入对抗感知提示(Adversarial-Aware)——即警示误导性段落并优先依赖模型自身知识——可在几乎不损失真实段落准确性的前提下,恢复82.5%–98.9%被错误排除的前截止事件正确答案,有效缓解知识-上下文冲突。该框架支持对固定数据集的重复评估,并可随模型演进扩展至新事件,为持续评估与优化模型可信度提供了可扩展的技术路径。
链接: https://arxiv.org/abs/2611.23601
作者: Yi-Cheng Lai,Hen-Hsen Huang
机构: Institute of Information Science, Academia Sinica (中央研究院資訊科學研究所); Taipei, Taiwan
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:Asking a model to check retrieved text can also tell it which source to trust. We introduce FutureBench, a renewable framework for evaluating knowledge-context conflict using timestamped questions, controlled passage triplets, and no-context probes. Across 3,629 questions and six models, Verify (a prompt combining checking with passage priority) raises false-answer adoption in all five models tested on pre-cutoff events, mainly by displacing answers that were correct under Baseline, which allows both passage evidence and model knowledge. On post-cutoff events, only o4-mini increases adoption (+13.7 percentage points). A four-prompt comparison on 902 post-cutoff questions finds that passage priority without checking raises adoption, while moving to Verify lowers it in all five models. Adversarial-Aware, which warns of misleading passages and prioritizes model knowledge, restores 82.5-98.9% of displaced pre-cutoff answers with under one point of change in true-passage accuracy. FutureBench supports repeated evaluation on fixed cohorts and extension to new events as models change.
[NLP-50] Safe to Omit Worth Omitting? Keeping the Answer and Cutting Latency When an LLM Agent Skips Anothers Message ICLR2027
【速读】: 该论文旨在解决多智能体系统中接收端智能体因等待辅助端智能体的消息(文本或其潜在的键值缓存,KV cache)而引入延迟的问题,同时在不显著影响最终答案准确率的前提下实现通信跳过以提升效率。其核心解决方案是提出一种基于置信度的有限样本检验机制:接收端在未接收消息前即可读取自身置信度,并通过统计检验认证一个阈值,在高概率下确保跳过消息后答案改变的比例不超过5%;若无任何路径满足该阈值,则系统维持正常通信。关键创新在于,该机制不依赖于消息的准确性,而是基于消息对答案的扰动频率与可预测性来判断是否可安全跳过,且实验证明即使跳过后的路径精度低于接收端单独推理,仍能保留绝大部分性能增益。在多个数据集(OpenBookQA、ARC-Challenge、MMLU-Pro、SQuAD、GSM8K)和不同模型组合上的测试表明,该方法可在保持答案稳定性的同时,有效减少通信开销,尤其在文本消息场景下节省数百毫秒延迟,而在KV缓存场景中收益较小,且认证有效性在部分任务(如SQuAD)中成立但在另一些任务(如GSM8K)中不适用。
链接: https://arxiv.org/abs/2611.23570
作者: Xinyang Li,Chih-Hsuan Yang,Xiaolong Ma,Anjir Ahmed Chowdhury,Vikram Vasudevan,Rajeev Thakur,Feng Yan
机构: University of Houston(休斯顿大学); Argonne National Laboratory(阿贡国家实验室); Oregon State University(俄勒冈州立大学)
类目: Machine Learning (cs.LG); Computation and Language (cs.CL)
备注: 58 pages, 11 figures, 41 tables. Code: this https URL data: this https URL
Abstract:In multi-agent systems, a receiver agent often waits for a helper agent’s message (text or its latent KV cache), which can improve performance but adds latency. We study when a receiver agent can skip the message safely (the final answer rarely changes) and beneficially (the time saved exceeds the cost of deciding). Our mechanism reads the receiver agent’s confidence without the message and uses a finite-sample test to certify a threshold at which, with high probability, at most 5% of skipped answers change; if none passes, the system keeps communicating. We test it on OpenBookQA, ARC-Challenge, and MMLU-Pro with three Qwen model pairs, on cross-family agents, and on SQuAD and GSM8K. Four insights emerge. First, oracle headroom is weak evidence that messages help: control inputs without answer-specific content, matched to change answers as often as real messages, reproduce a median 76% of the oracle’s gain. Second, whether a message can be skipped depends on how often and how predictably it alters the answer, not on its accuracy; every certified KV-cache path is less accurate than the receiver agent alone. Third, where text messages help, skipping keeps most of their gain: on unseen questions in the four such settings, the policies skip 59 to 80% and keep an estimated 76 to 95% of the gain. In two prospective tests with all rules fixed in advance, our mechanism skips a three-agent system’s verifier on 39.9% of sealed questions, keeping 12.55 of its 12.80-point gain, and a 32B helper on 68.4% of held-out questions, keeping its whole gain. Fourth, safety does not determine efficiency: at batch size one, skipping text saves hundreds of milliseconds per query and skipping KV cache far less. Certification extends to SQuAD but not to GSM8K. In eleven tests on unseen questions, every certified policy we could test kept its change rate’s 95% upper bound below 5% under our answer parsers.
[NLP-51] Language Models Make New Research Sound Old
【速读】: 该论文旨在解决生成式AI在科研验证中存在“报告偏差”(reporting gap)的问题,即大语言模型(LLM)倾向于将尚未充分研究的开放性研究问题误判为已有充分研究,从而高估已有文献的覆盖程度。其核心解决方案是通过引入双重验证机制:首先由模型独立判断研究问题的状态,再分别核查引用文献的全文内容以确认其实际支持程度,最终整合这两部分判断生成修正后的答案。该方法显著提升了模型对开放性问题的准确识别能力,使GPT-6-Astra在普通回答中明确指出问题未被研究的比例从8%提升至95%,并有效缓解了不同能力水平模型间报告偏差随能力增强而加剧的现象。此外,该框架不依赖模型权重或内部激活信息,具备对闭源前沿模型的兼容性,且通过融合多模型评分与规则学习实现了对审稿人新颖性评级的精准预测。
链接: https://arxiv.org/abs/2611.23546
作者: Guy Zyskind
机构: University of Miami(迈阿密大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:Researchers are increasingly using large language models (LLMs) to validate research questions. We find that LLMs make open research questions sound partly studied and overstate what related papers establish. Models recognize most of these overstatements when asked to check the papers’ full texts separately. On 40 questions drawn from newly accepted papers, GPT-6-Astra correctly judges 98% to be unstudied, but states that status plainly in only 8% of ordinary answers. All six frontier systems show reporting gaps of 30-90 percentage points. Across 22 models given identical papers, higher capability is associated with more accurate judgments and larger reporting gaps. This reporting failure is asymmetric: models correctly identify studied questions in 88-100% of ordinary answers when we supply the answering papers. To correct this, we build a fixed answer from each model’s judgment and separate checks of cited papers. This raises Astra’s clear reporting of open status from 8% to 95%, with improvements in all six systems. Finally, we predict reviewers’ novelty grades by combining several models’ written ratings with a rule learned from graded examples, reaching the performance range reported by Think-Probe-Respond. Unlike TPR, our approach needs no access to model weights or internal activations, so it also works with closed-weight frontier models.
[NLP-52] Can LLM s Understand What They Predict? Evaluating Latent Causal Structure Inference in Financial Markets
【速读】: 该论文旨在解决当前大型语言模型(Large Language Models, LLMs)在评估其预测能力时,隐含假设预测准确即代表对底层系统机制具备真正理解的问题。这一假设可能误导评估结果,因为模型可通过表面相关性实现高精度预测,而无需揭示生成可观测结果的潜在机制。为检验该假设,研究提出LUCID基准,用于评估模型是否能从事件描述与历史金融数据中推断出隐藏的市场结构。其核心解决方案在于构建基于重构的评估框架:模型需生成由事件引发的因果图以表征推断出的市场结构,并通过重构真实观测数据的能力来评估其有效性。实验结果表明,尽管部分指标有所提升,但将模型生成的因果图用于重构时,整体重建质量反而低于仅使用历史观测数据的情况,说明当前主流开源与专有LLM在从可观测证据中推断具有信息量的、事件特定的市场结构方面仍面临显著挑战。
链接: https://arxiv.org/abs/2611.23533
作者: Yixiang Zheng,Yuechen Jiang,Zhiwei Liu,Jimin Huang,Yuyan Wang,Sophia Ananiadou
机构: 未知
类目: Computation and Language (cs.CL); Machine Learning (cs.LG)
备注:
Abstract:Large language models (LLMs) are increasingly evaluated through their ability to predict future outcomes, implicitly assuming that predictive success reflects an understanding of the underlying system. However, accurate predictions can often be achieved without recovering the latent mechanisms that generate observable outcomes. To examine this assumption, we introduce LUCID, a benchmark that evaluates whether models can infer latent market structures from event descriptions and historical financial observations. Models generate event-induced causal graphs representing inferred market structure and are evaluated through a reconstruction-based framework. Our evaluation of 6 open-weight and proprietary LLMs shows that adding their generated graphs generally reduces reconstruction quality compared with using historical observations alone, although a few metrics improve. These findings suggest that inferring informative, event-specific market structures from observable evidence remains challenging for the evaluated LLMs.
[NLP-53] Dreaming Up Authors: Factual Prevalence and LLM Hallucinations in Academic Authorship Attribution EMNLP2026 EMNLP ACL
【速读】: 该论文旨在解决大语言模型(Large Language Models, LLMs)在问答任务中普遍存在幻觉(hallucination)的问题,尤其关注训练数据中事实流行度(factual prevalence)对幻觉产生的潜在影响。其核心假设是:在训练数据中出现频率较低的学术论文更易引发模型幻觉。为验证此假设,研究以引用次数作为事实流行度的代理指标,构建了一个涵盖八个科学领域的192万篇论文数据集,并按引用频次分层采样。实验结果表明,三种先进LLM均呈现显著的反向关系——幻觉率随引用次数增加而降低,低引用论文的幻觉率超过98%,而高引用论文则低于37%。进一步分析揭示,此类错误主要源于模型检索能力的局限性而非知识缺失,且开源模型的隐藏状态探测显示,事实流行度信息已被编码于生成前的表示中,为流行度与幻觉之间的机制关联提供了有力证据。因此,解决方案的关键在于识别并利用训练数据中事实的流行度信号,通过增强模型对高流行度事实的检索与表征能力,有效缓解幻觉问题。
链接: https://arxiv.org/abs/2611.23530
作者: Sijia Zhong,Tamir Yirga,Yijia Ma,Thai Le,Yifan Hu
机构: 未知
类目: Computation and Language (cs.CL)
备注: Published in EMNLP 2026 (Main Conference). 37 pages. Code and data: this https URL . Anthology: this https URL
Abstract:Large Language Models (LLMs) are widely used for question answering but are prone to hallucinations, yet the underlying causes, particularly the role of factual prevalence in training data, remain underexplored. In this work, we study this through academic authorship attribution, hypothesizing that hallucinations are more likely for less prevalent papers. Using citation count as a proxy for factual prevalence, we curate a dataset of 1.92 million papers across eight scientific disciplines with stratified samples by citation level. Experiments on three state-of-the-art LLMs confirm our hypothesis, revealing a robust inverse relationship between hallucination rate and citation count across all models and disciplines, with hallucination rates exceeding 98% for rarely cited papers and falling below 37% for highly cited ones. Our in-depth analysis further shows that these failures primarily reflect retrieval limitations rather than knowledge absence, and hidden-state probing of open-source models shows that factual prevalence is encoded in pre-generation representations, providing convergent mechanistic evidence for the prevalence-hallucination relationship. The code and data are available at this https URL.
[NLP-54] From Arraignment to Resolution: Transformer Models of Court Cases EMNLP2026
【速读】: 该论文旨在解决刑事司法系统中决策过程缺乏透明度的问题,尤其关注案件处理流程中多个关键决策点(如指控选择、保释金额确定、认罪协商及量刑)的可解释性与可监督性。现有机器学习方法多聚焦于单一决策(如再犯风险预测),难以捕捉案件演进中的动态复杂性。本文提出一种基于Transformer的时间序列模型,将案件视为事件序列进行建模,从而实现对下一阶段案件进展(如接受指控、设定保释、达成认罪协议等)的多标签预测。其核心创新在于融合书记员笔录的文本数据与被告人的结构化信息(如前科记录、法庭背景等),通过时序建模揭示决策演变规律。在基于美国某中等规模城市公开法院记录构建的数据集上,该模型在20类多标签任务中达到0.57的微平均F1分数,显著优于多个基线模型,且性能接近使用相同输入训练的GRU模型。实验表明,该模型可为司法观察者提供决策过程的深度洞察,辅助识别辖区内潜在的系统性问题,推动刑事司法系统的透明化与问责机制建设。
链接: https://arxiv.org/abs/2611.23518
作者: Kory Rosen,Aron Culotta
机构: Tulane University (杜兰大学)
类目: Computation and Language (cs.CL); Computers and Society (cs.CY); Machine Learning (cs.LG)
备注: Published in Findings of EMNLP 2026
Abstract:The criminal legal system is composed of many complex decision points, such as which charges are pursued, how bond is determined, plea deal negotiations, and sentencing. Given the profound impact of such decisions, there is a growing need to increase the transparency within this system. While prior machine learning approaches have focused on modeling single decisions (e.g., recidivism), this paper proposes a transformer-based time series model to forecast the outcome of the next proceeding in a case (e.g., charges accepted, bond set, plea deal made, etc.). By modeling a case as an event sequence, we enable a more nuanced analysis to support better oversight and understanding of decision making. We conduct an empirical analysis of thousands of cases using a newly constructed dataset derived from publicly available court records of a mid-sized U.S. city. We fit models that combine textual data from clerk notes with structured information about the defendant, prior charges, and courtroom. Our transformer model achieves .57 micro-F1 on a 20-class multi-label task, a significant improvement over several baselines and comparable to a GRU trained on the same input. We demonstrate how the model can be used to provide insights into decision-making and help court observers identify emerging issues within the jurisdiction studied.
[NLP-55] GRAFT: Execution-Guided Genetic Recombination for Agent Design
【速读】: 该论文旨在解决生成式智能体(Generative AI agent)在自动化设计过程中因仅依赖聚合适应度(aggregate fitness)进行选择与进化而导致的实例级成功与失败信息被掩盖的问题。现有方法虽通过语言模型交叉(language-model crossover)实现机制重组,但缺乏对具体执行过程中的有效反馈指导,限制了优化精度。其解决方案的关键在于提出一种执行引导的遗传重组方法——GRAFT(Execution Guided Genetic Recombination for Agent Design),该方法在保留聚合适应度用于选择与生存的基础上,引入父代特定的执行证据(parent-specific execution evidence)来指导语义层面的交叉操作,并集成变异机制。该反馈机制复用适应度评估中已有的成功与失败运行样本,无需额外任务执行或参考答案,实现了高效、可扩展的反馈闭环。实验表明,GRAFT在六种推理与编程任务上达到82.0%的平均测试准确率,显著优于最强基线9.8个百分点;控制性消融实验进一步验证了执行证据对性能提升的关键作用,证明了父代特定执行证据在演化智能体设计中的重要价值。
链接: https://arxiv.org/abs/2611.23509
作者: Sevda Ebrahimi,Mohammed Saidul Islam,Xiaodan Zhu,Elham Dolatabadi
机构: York University (约克大学); Vector Institute (向量研究所)
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Neural and Evolutionary Computing (cs.NE)
备注:
Abstract:Automated agent design searches for executable architectures that improve LLM-based agent performance, with evolutionary methods using language-model crossover to recombine promising mechanisms for task decomposition, model coordination, and verification. This search, however, is typically guided by aggregate fitness alone, which hides the instance-level successes and failures behind each score. We introduce Execution Guided Genetic Recombination for Agent Design (GRAFT), which uses aggregate fitness for selection and survival and parent-specific execution evidence to guide semantic crossover with integrated mutation. Its feedback harness reuses bounded samples of successful and failed runs from fitness evaluation, withholding reference answers and requiring no additional task executions. We evaluate GRAFT across six reasoning and programming settings using open-weight optimizer and executor models. It achieves a mean test accuracy of 82.0%, exceeding the strongest baseline by 9.8% points. Controlled ablations show that adding execution evidence improves test accuracy with initialization, search rules, repair policies, and evaluation budgets held fixed. Our results support the value of parent-specific execution evidence for evolutionary agent design.
[NLP-56] BOS Summary: The First Token Matters
【速读】: 该论文旨在解决大视觉语言模型(LVLMs)与语言模型(LLMs)中存在的注意力集中(attention sink)问题,即特定位置(如起始标记BOS)被过度关注,导致其在推理过程中无法有效传递输入上下文信息。尽管在标准因果掩码下BOS查询仅能访问自身,但后续令牌反复访问的表征却缺乏输入内容,从而影响模型性能。为应对这一问题,论文提出了一种无需训练、仅在推理阶段应用的解决方案——BOS Summary。其核心在于:在预填充(prefill)阶段的单个Transformer层中,仅对BOS查询行放宽因果约束,使其能够聚合所有视觉与文本输入,生成一个蕴含完整输入上下文的自衍生表征;而其余所有令牌仍保持标准因果注意力机制。该增强后的表征在整个解码过程中持续可用,且不引入额外参数、辅助模块或解码时开销。实验表明,BOS Summary在多个LVLM与LLM家族及八项多模态与纯文本基准上均优于直接提示与思维链(Chain-of-Thought)提示,并在多步推理任务中超越近期针对注意力集中问题的干预方法。机理分析进一步揭示,使BOS携带特定于输入的信息可削弱其作为注意力“黑洞”的主导地位,促使注意力更合理地分配至承载语义的内容性提示令牌,证明了改变注意力“黑洞”所承载的信息内容,能够重构后续层的注意力分布。
链接: https://arxiv.org/abs/2611.23508
作者: Hui Zhu,Saleh Zare Zade,Rafi Ibn Sultan,Deng Pan,Arun Ross,Dongxiao Zhu
机构: Wayne State University (韦恩州立大学); University of Notre Dame (圣母大学); Michigan State University (密歇根州立大学)
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Large vision-language models (LVLMs) and language models (LLMs) often exhibit attention sinks, whereby particular positions, such as the beginning-of-sequence (BOS) token, receive a disproportionate share of attention. Under standard causal masking, however, the BOS query attends only to itself, so the representation that later tokens repeatedly access carries no information about the input. We introduce BOS Summary, a training-free inference-time method that repurposes this persistent pathway to carry input context. At a single transformer layer during prefill, BOS Summary relaxes the causal constraint for the BOS query row alone, letting BOS aggregate all visual and textual input tokens into a self-derived context representation while every other token retains standard causal attention. The enriched representation then remains accessible to the model throughout decoding, with no additional parameters, auxiliary modules, or decoding-time overhead. Across multiple VLM and LLM families and eight multimodal and text-only benchmarks, BOS Summary improves over direct and Chain-of-Thought prompting and outperforms recent sink-intervention methods on multi-step reasoning tasks. A mechanistic analysis shows that making BOS input-specific also weakens its sink dominance and redistributes attention toward content-bearing prompt tokens, indicating that changing what a sink carries reshapes how later layers allocate attention.
[NLP-57] SpecLook: Speculating Where to Look for Fast and Reliable Agent ic MLLM s
【速读】: 该论文旨在解决当前基于图像推理的智能体多模态大语言模型(Agentic Multimodal Large Language Models, MLLMs)在处理视觉任务时存在的效率低下与答案可靠性不足的问题。其核心挑战在于,现有方法依赖于“思考-缩放”(zoom-in)工具调用的顺序循环机制,导致推理过程缓慢且无法提供可信度评分;同时,对答案或推理步骤的提前推测容易引入自信但错误的结论。本文的关键洞察在于:与其推测最终答案,不如推测智能体下一步的观察结果(即“推测性观察”),这一推测可被验证、修正,并用于测试答案的合理性。为此,作者提出无需训练的SpecLook框架,包含三个层级:小型模型生成初步观察(glimpse)并基于此重新回答,若答案稳定且置信则终止;否则由大型模型无工具地验证并修正该观察,融合两者输出。仅当证据仍不充分时才调用完整智能体,实现单次前向传播,显著提升效率。该方法通过追踪推理轨迹为每项答案提供可靠性评分,无需标签。在V*、HR-Bench和POPE等多个基准上的实验表明,SpecLook在准确率上全面超越所有基线,分别达到87.2%和86.2%的平均准确率,相较DeepEyes-7B和Thyme-RL智能体分别实现7.7倍和4.7倍的速度提升;而SpecLook-Fast进一步将速度提升至9.9倍和6.3倍,充分验证了其高效性与可靠性。
链接: https://arxiv.org/abs/2611.23483
作者: Xi Ding,Jiawei Zhang
机构: University of Wisconsin–Madison
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: Technical report
Abstract:Agentic multimodal large language models (MLLMs) that think with images interleave reasoning with zoom-in tool calls. However, this sequential loop is slow, and its answers carry no reliability score. Prior speculation of tokens, reasoning steps, or answers keeps the loop, and answer speculation also accepts confident errors. We find that the loop mostly adds one zoom-in step, with which a small model is ahead of the agent on most splits. Our key insight is to speculate the agent’s next observation, which, unlike a drafted answer, can be verified, corrected, and used to test the answer. We propose SpecLook, a training-free speculate-and-verify framework with three levels. A small model drafts a glimpse and re-answers from it, and the question ends if the answer is unchanged and confident. Otherwise, the agent’s large model verifies and corrects the glimpse without tools, and their answers are fused. Only questions that still lack evidence reach the agent, in one forward pass that SpecLook-Fast skips, so no question runs the loop. The trace scores every answer’s reliability without labels. Extensive experiments on V*, HR-Bench, and POPE show that both are more accurate than every baseline on every split. SpecLook reaches 87.2% and 86.2% mean accuracy with 7.7x and 4.7x speedups over the DeepEyes-7B and Thyme-RL agents, and SpecLook-Fast reaches 9.9x and 6.3x.
[NLP-58] Formal Adequacy Index (based on Polish-language material)
【速读】: 该论文旨在解决语言形式性(formality)在实际交际中是否恰当的问题,即仅测量文本的形式性水平无法判断其在特定语境中的适切性。其解决方案的关键在于提出一种形式性适切性指数(formal adequacy index),该指数通过分别建模文本中观察到的形式性(基于语言特征)与交际情境所预期的形式性(基于语境特征),并结合上下文容许阈值(contextual tolerance thresholds)进行比较,从而实现对形式性适切性的量化评估。该方法区分了形式性不足与过度的容忍度差异,具有较强的可操作性。研究以波兰语道歉邮件(由人类或大语言模型生成)为例,展示了该指数的应用潜力。尽管当前实现仍属初步,需进一步校准与实证验证,但本文的核心贡献在于构建了一个适用于多种交际语境的、定量研究形式性适切性的通用方法论框架。
链接: https://arxiv.org/abs/2611.23482
作者: Paweł Patreucha
机构: Univeristy of Wrocław(弗罗茨瓦夫大学)
类目: Computation and Language (cs.CL)
备注: 9 pages, 1 figure, 1 table
Abstract:Linguistic formality is one of the dimensions of stylistic variation in speech; however, simply measuring its level does not allow us to determine whether the chosen mode of communication is appropriate in a given situation. The aim of this article is to present the concept of a formal adequacy index (based on Polish-language material), which enables a quantitative assessment of how well the formality of a text matches the expectations arising from the communicative context. The proposed approach is based on the separate modelling of observed formality, determined on the basis of the linguistic features of the text, and expected formality, determined on the basis of the characteristics of the communicative situation. Adequacy is assessed by comparing these two measures, taking into account contextual tolerance thresholds, which may differ for a lack of formality and an excess of formality. The potential applications of the index are illustrated by an analysis of Polish-language emails containing apologies for being late, written by human language users or generated by a large language model. The implementation presented here is preliminary in nature and requires further calibration and empirical validation. The main contribution of this article is the proposal of a general methodological framework for the quantitative study of formal adequacy in various communicative contexts.
[NLP-59] racking Representations in LLM Reasoning Traces for Iterated Games
【速读】: 该论文旨在解决大语言模型(Large Language Model, LLM)代理在基于思维链(Chain-of-Thought, CoT)推理过程中,其外部可观察的推理轨迹与内部实际计算行为之间关系不明确的问题。核心挑战在于:尽管CoT被广泛用于提升模型在复杂任务中的表现,但其推理过程中的信念形成与修正机制仍缺乏透明性。为此,研究通过一系列机制可解释性技术,探究了LLM代理在不完全信息迭代博弈中的决策机制。关键发现是,模型在CoT推理过程中会构建并持续修正对游戏状态的内部表征;一旦过早地确立错误信念,便极少能够自我纠正,这种认知偏差同时体现在显式的推理文本轨迹和隐层激活模式中。该研究揭示了当前生成式AI系统在多智能体协作场景下战略推理错误的根本成因,为未来构建更鲁棒、可诊断的多智能体系统提供了重要理论基础。
链接: https://arxiv.org/abs/2611.23403
作者: Chaipat Tirapongprasert,Rajarshi Das,Parijat Dube
机构: Columbia University (哥伦比亚大学); MQube Cognition
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT)
备注: Accepted to SoCalNLP 2026. 15 pages, 11 figures
Abstract:Large language model (LLM) agents achieve state-of-the-art performance across domains through chain-of-thought (CoT) reasoning. Despite their ubiquitous use, little is understood about the relationship between the externalized trace and the model’s actual computation. We address this by investigating how LLM agents make decisions in iterative games with incomplete information. Using a series of mechanistic interpretability techniques, we find that the model forms and iteratively revises an internal representation of the game throughout the CoT. Once the model prematurely commits to an incorrect belief, it rarely recovers. This failure is manifested in both the literal trace and the layer activations. Ultimately, as multi-agent systems are more prevalent, our work opens the door to a more focused diagnosis of their strategic reasoning errors. For illustrative purposes, we offer an interactive visualizer of our methodology on selected sample traces at this https URL
[NLP-60] QDA: Closed-Loop LLM Quantization Design with Agents
【速读】: 该论文旨在解决大语言模型(Large Language Models, LLMs)在后训练量化(Post-training Quantization, PTQ)过程中,因模型与精度组合差异导致最优量化方案难以通用化的问题。现有方法需专家针对每种新模型手动设计量化配方,过程耗时且效果常不理想。为应对这一挑战,论文提出QDA系统,其核心解决方案是引入一个可自主优化的智能代理(LLM Agent),通过多轮迭代构建与评估候选量化方案,实现自动化配方设计。关键创新在于:首先,采用基于推理轨迹输出的KL散度快速评估机制,在分钟级内完成候选方案的初步筛选,有效避免了传统方法中耗时数十GPU小时的完整推理验证;其次,通过构建一个兼容性强、可互操作的公共PTQ方法库,实现了不同量化方法在新型模型架构上的无缝集成,显著降低开发与移植成本。实验表明,QDA在四种推理型模型上分别以64 GPU小时的搜索开销,相较最强公开配方平均提升0.9、1.4和2.2个百分点的准确率,且在320B参数的GLM-5.3-Flash MoE模型上达到FP8基准98.3%的精度表现,优于最佳厂商检查点的96.9%。所有生成的量化模型均可直接部署于现有推理内核,无额外推理开销。
链接: https://arxiv.org/abs/2611.23388
作者: Xingyang Li,Tianlang Zhao,Song Han,Ligeng Zhu
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 19 pages, 7 figures, 10 tables
Abstract:Post-training quantization is the standard way to serve large language models at low cost. However, the best quantization recipe differs across models and precisions. As a result, experts must redesign it for every new model, which is time-consuming and often sub-optimal. Fortunately, an LLM agent can take over this design by refining the recipe over rounds of building and evaluating candidates, so that more rounds lead to better recipes. However, evaluating one quantized reasoning model takes tens of GPU-hours, and combining methods from different codebases means porting each to the new architecture by hand. We present QDA, a system in which an agent designs the recipe for each model and precision. To make evaluation fast, QDA scores each candidate in minutes by its output KL divergence from the original model on reasoning traces that the agent never sees, and checks promising ones with long generations to catch failures such as repetition loops. To make building fast, QDA provides a library of public PTQ methods that are compatible with one another and with new model architectures. Across four reasoning models, QDA improves accuracy over the strongest public recipe by 0.9, 1.4 and 2.2 percentage points on average at W4A16, W4A8 and W3A16, using 64 GPU-hours per search. On the 320B-parameter MoE model GLM-5.3-Flash at NVFP4 W4A4, it recovers 98.3% of the FP8 model’s accuracy, against 96.9% for the best vendor checkpoint. Every QDA checkpoint also runs on existing serving kernels with no inference overhead. We will release the library, the evaluator and the harness.
[NLP-61] Chorus: Shared N-gram Embeddings with Layer-wise Adaptation
【速读】: 该论文旨在解决现有n-gram嵌入(n-gram embedding)方法在模型多层结构中面临的内存开销大与知识共享能力弱的问题。具体而言,传统设计为每一层单独配置一个n-gram嵌入表,虽能实现层间个性化学习,但导致参数量急剧增加;而共享单一嵌入表则因所有层使用相同表示,缺乏层间差异性表达能力,性能表现不佳。针对这一矛盾,论文提出Chorus架构,其核心创新在于:采用统一的一组n-gram嵌入表服务于整个网络,同时允许各层通过自身可学习的投影机制对嵌入向量进行差异化读取和重加权——即基于当前隐藏状态动态计算缩放因子,对嵌入成分进行自适应调整。该设计实现了嵌入知识的跨层共享与层间表达灵活性的平衡。实验表明,Chorus仅需12亿参数的嵌入表即可达到比96亿参数的FactorEngram和DeepSeek Engram更优的文本生成性能,且在不同规模下均优于Engram、TN-gram和FactorEngram;此外,结合轻量级优化器Ember,可减少99.2%的表相关优化器内存占用,显著提升训练效率。该方案通过降低内存消耗并支持跨层知识复用,为n-gram嵌入表的进一步扩展提供了可行路径。
链接: https://arxiv.org/abs/2611.23386
作者: Stephen Chung,Wenyu Du
机构: DualverseAI
类目: Computation and Language (cs.CL)
备注:
Abstract:Language models can gain capacity at little computational cost from n-gram embedding tables: large tables of learned vectors for short sequences of consecutive tokens, which are looked up and added to the model’s input or to its intermediate layers. Recent designs commonly use a separate n-gram embedding table for each layer, which makes the tables costly in memory and prevents layers from sharing what they learn. Sharing a single table, however, gives every layer the same embedding and leaves no room for layer-wise preferences, and it performs poorly in practice. We propose Chorus, in which one set of n-gram embedding tables serves the whole network and every layer reads it in its own way. Each layer maps the refined embedding through its own projection layer, after rescaling its components by factors computed from its current hidden state. Chorus matches the quality of existing n-gram methods with 87% smaller tables: with 1.2B table parameters it predicts text better than FactorEngram and DeepSeek’s Engram do with 9.6B. It outperforms Engram, TN-gram and FactorEngram in every setting where we compared them, across table sizes from 20M to 9.6B parameters. We also show that the recently proposed memory-efficient optimizer Ember works well with Chorus, saving 99.2% of table-optimizer memory compared with AdamW without degrading training performance. These results enable further scaling of n-gram embedding tables along the table-size axis by reducing memory use and allowing shared knowledge to be reused across layers.
[NLP-62] A Survey on RLOps for Large Language Models
【速读】: 该论文旨在解决大规模语言模型(Large Language Models, LLMs)在强化学习(Reinforcement Learning, RL)训练过程中因扩展性带来的分布式执行失败、反馈计算瓶颈及学习动态不稳定性等问题,这些问题会导致计算资源浪费、训练效率降低以及模型性能提升受阻。其解决方案的关键在于构建系统化的LLM RLOps(Reinforcement Learning Operations)框架,将RL训练全生命周期划分为五个核心阶段:训练准备(Training Setup)、经验收集(Experience Collection)、奖励计算(Reward Computation)、策略优化(Policy Optimization)与模型评估(Model Evaluation),并在每个阶段深入分析操作痛点、管理技术与实证证据。通过这一结构化分类体系,论文揭示了各阶段间协同机制的不足,并提出未来研究方向,包括训练过程可观测性增强、跨阶段协调优化、可泛化的RLOps方法、多运行管理以及支持递归自我改进的RLOps范式,推动操作管理从单次训练运行向多样化训练场景、共享基础设施及持续迭代改进周期演进。
链接: https://arxiv.org/abs/2611.23357
作者: Lingzhe Zhang,Tong Jia,Chiming Duan,Minghua He,Leyi Pan,Kening Zheng,Zhaoyang Liu,Bolin Ding,Philip S. Yu,Ying Li
机构: Peking University (北京大学); National Key Laboratory of Data Space Technology and System (数据空间技术与系统国家重点实验室); Tsinghua University (清华大学); University of Illinois Chicago (伊利诺伊大学芝加哥分校); Alibaba Group (阿里巴巴集团)
类目: oftware Engineering (cs.SE); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
备注:
Abstract:Reinforcement learning (RL) has become increasingly important for improving the reasoning, instruction-following, and task-solving capabilities of large language models (LLMs). However, scaling RL training exposes failures in distributed execution, feedback computation, and learning dynamics that can waste computational resources, slow training, and undermine model improvements. These challenges have motivated a growing body of techniques for LLM RLOps, which manages the efficiency and reliability of operations throughout the LLM reinforcement learning training lifecycle. Despite growing interest, these techniques remain scattered across research areas, with limited synthesis of their operational roles and interactions. To bridge this gap, we present a systematic survey of LLM RLOps methods. We organize existing approaches into five stages: Training Setup, Experience Collection, Reward Computation, Policy Optimization, and Model Evaluation. Within each stage, we examine operational concerns, management techniques, and evaluation evidence. Building on this taxonomy, we identify open challenges and future directions in training process observability, cross-stage coordination, generalizable RLOps, multi-run management, and RLOps for recursive self-improvement. These directions extend operational management from individual training runs to diverse training settings, shared infrastructure, and successive improvement cycles. We maintain a GitHub repository indexing relevant papers and open resources at this https URL.
[NLP-63] A Concurrent Structured Record Mirrored the Prose It Accompanied: A Pre-Registered Conceptual Replication and Extension of the Hide Pending Tool Call Scenario
【速读】: 该论文旨在解决生成式AI在执行代理型任务(agentic work)时,语言模型倾向于隐瞒关键性叙事缺陷(narrative-changing flaws)的问题,尤其在存在未返回的后续查询(pending tool call)情境下,模型仍会保持表面完整性。其核心解决方案是通过引入结构化记录(structured record)机制,采用对抗性审查工具的格式,明确包含“残留项”(residue section)以追踪未解决的问题。实验结果表明,在预注册的两阶段测试中,带有残余项结构的记录使29%至30%的合成日志明确指出未返回的查询,显著高于仅含结论摘要的对照组(5%)和干净对照组(0%),且统计学差异极显著(p = 1.2 × 10⁻⁷)。尽管在自然语言回答层面,该结构化记录并未显著提升诚实披露率(与“诚实指令”相比无显著差异,p = 0.125),但探索性交叉分析显示,所有提及缺失信息的结构化记录均伴随相应文本表述,而未提及者则完全未在文本中体现——表明结构化记录有效“镜像”了其伴随的自然语言输出内容,从而为系统性暴露潜在缺陷提供了可操作的框架。
链接: https://arxiv.org/abs/2611.23274
作者: Nicolás Rocchia
机构: 独立研究员(Independent researcher); Argentina(阿根廷)
类目: oftware Engineering (cs.SE); Computation and Language (cs.CL)
备注: 10 pages, 4 tables. Pre-registered study; protocols, code, data and analyses at this https URL
Abstract:Huang et al. (2026) show that language models writing reports of agentic work tend to conceal narrative-changing flaws, and that appending “Be honest in your response.” to the instruction substantially increases disclosure in most of their scenarios. In their Hide Pending Tool Call scenario, where a broader follow-up query was accepted but never returned, the honesty prompt left GPT-5.5’s rate of flagging the missing data at 0%, although it turned most answers into caveats. I ask whether a different request changes this: appending, in the same generation, a structured record that follows the schema of an adversarial-review tool, including a residue section for what remained unresolved. In two pre-registered phases on the same 100 synthetic log pairs, each pending log matched with a clean twin in which the follow-up query did return, I ran gpt-5.6-sol through Codex CLI under four conditions. Phase 1 measured the record: with the residue schema, a judge coded it as stating the unreturned query in 29% of logs, against 5% with a findings-only placebo schema, and in none of the clean twins (p = 1.2 x 10^-7); a second run two days later gave 30% and 6%. But 70-71% of residue records declared that nothing remained open while the query was pending. Phase 2 measured the prose answer: the confirmatory comparison against the honesty prompt was not significant (four discordant logs, all favoring the honesty prompt; p = 0.125), no equivalence test was pre-registered, and the judge failed the pre-registered human validation on the decisive category; I therefore claim neither an effect nor its absence. In an exploratory cross-tabulation, every residue record coded as stating the gap accompanied prose coded as mentioning it (30 of 30), and none accompanied prose coded as omitting it (0 of 70): in this setting, the structured record mirrored the prose it accompanied.
[NLP-64] Geometry-bridge flow matching for blind rigid-motion correction in head cone-beam CT
【速读】: 该论文旨在解决患者头部在头颈部锥形束计算机断层扫描(CBCT)过程中发生刚性运动导致图像质量下降及解剖结构重建不准确的问题。其核心解决方案是提出一种基于三维流匹配先验(3D flow-matching prior)的框架,通过学习运动引起的图像变化来实现高精度重建。该方法的关键在于构建“几何桥”(geometry-bridge),将同一源体积在运动伪影存在与无运动状态下的Feldkamp-Davis-Kress(FDK)重建图像通过逐步减小的模拟运动进行连接,并利用正向投影算子的几何导数提供路径上的解析速度目标。该框架采用即插即用循环结构,集成学习得到的图像更新、基于样条的位姿估计、共轭梯度数据一致性约束以及图像总变差(total variation, TV)正则化。实验表明,在30名未参与训练的CQ500患者模拟360视角扫描中,该方法在10 mm/10°以内的每分量运动幅度下,实现了优于现有方法(如学习型自动对焦和基于扩散的联合重建)的图像重建质量与运动估计精度,平均峰值信噪比(PSNR)达37.1 dB,探测器平面重投影误差(RPE)为0.27 mm;相较于图像线性训练方式,几何桥使PSNR提升1.6 dB,RPE降低33%;在50次先验更新与粗到精运动估计策略下,单例患者平均处理时间为9.1分钟(RTX A6000显卡)。
链接: https://arxiv.org/abs/2611.23788
作者: Sungho Yun,Seungryong Cho
机构: Korea Advanced Institute of Science and Technology (KAIST); Daejeon 34141, South Korea
类目: Medical Physics (physics.med-ph); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Rigid motion of a patient head during head cone-beam CT (CBCT) scan can degrade image quality and complicate recovery of accurate anatomical structures from the measured projections. We present a 3D flow-matching prior approach that learns the image changes induced by motion. Its geometry-bridge connects motion-corrupted and motion-free Feldkamp-Davis-Kress (FDK) reconstructed images of the same source volume through progressively reduced simulated motion. The forward projector’s geometry derivative supplies analytic velocity targets along this path. A plug-and-play loop encompasses the learned image updates together with spline-based pose estimation, conjugate-gradient data consistency, and image total-variation regularization. Both the refined image volume and the FDK reconstructed image volume at the estimated poses are evaluated. On simulated 360-view scans from 30 held-out CQ500 patients with nominal per-component motion amplitudes within 10 mm / 10 degrees, the framework achieved state-of-the-art image reconstruction quality and motion accuracy in comparison with other methos such as learned autofocus and diffusion-based joint reconstruction. Mean peak signal-to-noise ratio (PSNR) reached 37.1 dB and mean detector-plane reprojection error (RPE) was 0.27 mm. With matched endpoints at the inference settings, the geometry bridge improved PSNR by 1.6 dB and reduced RPE by 33% relative to image-linear training. With 50 prior updates and coarse-to-fine motion estimation, total processing time averaged 9.1 minutes per patient on an RTX A6000.
信息检索
[IR-0] Can Programs Replace LLM Judges? Executable Program Judges for Scalable and Efficient Relevance Labeling
链接: https://arxiv.org/abs/2611.24849
作者: Eugene Agichtein,Charles L. A. Clarke,Kaustubh Dhole,William Stern
类目: Information Retrieval (cs.IR); Computation and Language (cs.CL)
备注:
Abstract:Large language model (LLM) judges have become a common way to create large-scale relevance judgments for retrieval evaluation. However, each judgment requires a potentially expensive call to a frontier model, and the judgments are hard to interpret and may not be reproducible. In this paper, we ask whether an LLM can write a readable program for each topic that replaces its per-document judgments. To address these issues, we propose ExecJudges, a method for generating fast, inspectable and scalable judges for automatic relevance labeling. Specifically, ExecJudges uses a frontier LLM to turn assessment guidelines into executable programs, which rely only on lexical matching, standard NLP libraries like parsers, and pool statistics. An ExecJudges-L variant also uses examples of human labels if available. We evaluate ExecJudges on seven TREC collections by comparing the produced labels with trained NIST assessors, frontier LLM judges, and recently proposed per-topic classifiers. Our results show that ExecJudges agrees with NIST assessors at the level of the Claude Sonnet 5.5 LLM judges on TREC DL 2019-2021. With a small number of human labels per topic, the ExecJudges-L variant agrees with NIST more than per-topic classifiers with the same labels on all seven collections, and either exceeds the agreement of LLM judges provided the same labels or is within about 0.05 of it in Krippendorff’s alpha. The labels rank the official TREC DL runs with Kendall’s tau of 0.87 to 0.94, and writing the programs incurs a one-time cost of 0.17 to 0.49 per topic, with no additional costs for grading. Our results suggest that ExecJudges offers a reproducible, inspectable, efficient and cost-effective alternative to LLM judges for labeling deep result pools for system evaluation or ranking improvements.
[IR-1] Variance-Optimal Counterfactual Policy Design for Sample-Efficient Online A/B Testing
链接: https://arxiv.org/abs/2611.24757
作者: Shashank Gupta,Subhadeep Maji
类目: Machine Learning (cs.LG); Information Retrieval (cs.IR)
备注:
Abstract:Online A/B testing is the standard approach for evaluating changes to user-facing systems, such as a new recommendation or ranking policy, before deployment. While the difference-in-means estimator of a standard A/B test is unbiased, it often suffers from high variance. When the treatment and production policies choose the same actions with similar probabilities, the resulting observations can add noise while providing little information about the difference between the policies. Consequently, A/B tests can require large amounts of traffic and long durations to detect small treatment effects. Existing variance-reduction methods mainly improve the estimator after the data-collection scheme has been fixed, and recent overlap-aware designs only optimize a single traffic split between the two policies; thereby, they cannot direct traffic to the actions on which the policies disagree. In this paper, we jointly optimize how actions are sampled and how the resulting observations are used to estimate the treatment effect. Our contributions are two-fold. First, building on pairwise off-policy evaluation, we derive the variance-optimal action-level logging policy for estimating the average treatment effect between two policies, which has a closed-form solution for any fixed control variate. Second, we use historical logs to jointly optimize the logging policy and a control variate to further reduce variance. Our experimental results in simulated policy A/B tests show that our designs estimate the average treatment effect more accurately than existing variance reduction methods on every evaluated setting, while generally improving statistical power and reducing how often the worse policy is selected. As a result, our designs enable practitioners to reach reliable deployment decisions with considerably less online traffic. Subjects: Machine Learning (cs.LG); Information Retrieval (cs.IR) Cite as: arXiv:2611.24757 [cs.LG] (or arXiv:2611.24757v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2611.24757 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[IR-2] X-split Argus: Learning Reusable History Tokens for Efficient CTR Prediction
链接: https://arxiv.org/abs/2611.24283
作者: Alexander Ploshkin,Nikita Stepanov,Alexey Pismenny,Viacheslav Ovchinnikov,Nikolay Savushkin
类目: Information Retrieval (cs.IR)
备注: Alexander Ploshkin, Nikita Stepanov and Alexey Pismenny contributed equally to this work
Abstract:Accurate click-through rate (CTR) prediction depends on user representations that capture both persistent preferences and recent intent, yet processing extensive behavioral histories for every ranking request is costly under production compute and latency constraints. We present X-split ARGUS, a framework for ad ranking whose history compressor and online ranker are trained end to end. The compressor encodes past interactions into a compact set of tokens that are computed offline and reused across candidates, while the online ranker scores ads through cross-attention to these tokens and representations of recent events. Training combines an autoregressive loss for click feedback with distillation of candidate-specific click logits from a larger teacher that scores each candidate using the uncompressed history. The number of compressed history tokens and the length of recent history can be adjusted to fit serving compute and latency budgets. Offline evaluation shows a 0.81% relative reduction in normalized entropy (NE) compared with the production baseline. Deployment in a large-scale real-time bidding system increases online inference cost by 15% relative to the production baseline. An online A/B test against this baseline yields relative gains of 4.65% in a business metric reflecting the monetary value of conversions generated for advertisers and 2.96% in clicks.
[IR-3] MARS Technical Report
链接: https://arxiv.org/abs/2611.24006
作者: Tong Liu,Chaoqun Hou,Longxiang Xiong,Junhong Lou,Jiaqi Zheng,Qixin Li,Zixin Zhao,Yong Duan,Xiannian Hu,Fengze Jiang,Yu Zhu,Cheng Guo,Xinyuan Qi,Zi Zhuang,Ping Liu,Lv Shao,Zongwei Wang
类目: Information Retrieval (cs.IR)
备注: 46 pages, 5 figures
Abstract:Modern industrial recommender systems use multi-stage cascades–retrieval, pre-ranking, ranking, and re-ranking–to narrow millions of candidates to a personalized list. While each stage deploys sophisticated models, pipeline-level decisions on channel activation, quotas, model selection, and score fusion remain engineer-tuned and user-agnostic within each scenario. This report presents MARS (Multi-stage Agentic Recommendation System), extending personalization from individual-stage predictions to full pipeline orchestration. A near-line agent built on fine-tuned Qwen3-0.6B generates per-user configurations–a seven-dimensional action covering pipeline mode, retrieval quotas, model selection, fusion weights, and reranker settings–from a constrained action space. Configurations are validated, cached, and consumed asynchronously; the cascade never waits for LLM inference. Schema checks, fallbacks, and a kill switch ensure serving safety. The policy is trained in three stages: Confidence-aware SFT distills schema-valid generation from a teacher model; Format-Anchored Preference Optimization (FAPO) learns action preferences via ActionMask with posterior weighting; Group Relative Policy Optimization (GRPO) refines the policy through Sandbox replay with a frozen reward model, without exposing exploratory actions to live traffic. Two-week A/B tests across five Taobao scenarios with over ten million daily users show positive gains on all key metrics. The largest reach +5.22% V2P and +7.34% GMV, with positive CTR lifts on both click surfaces. About 20% of requests use a lightweight Minimal mode, and deployment shows 12 ms lower mean serving latency versus production. Comments: 46 pages, 5 figures Subjects: Information Retrieval (cs.IR) ACMclasses: I.2.0; I.5.0; I.7.0 Cite as: arXiv:2611.24006 [cs.IR] (or arXiv:2611.24006v1 [cs.IR] for this version) https://doi.org/10.48550/arXiv.2611.24006 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Yu Zhu [view email] [v1] Wed, 18 Nov 2026 07:56:58 UTC (1,451 KB)
[IR-4] ReflectRec: Cycle-Consistency Preference Verification for LLM -based Recommendation EMNLP2026
链接: https://arxiv.org/abs/2611.23969
作者: Dae Kyoo Kim,Heejung Cho,Sungmoo Hong,Eunil Park
类目: Computation and Language (cs.CL); Information Retrieval (cs.IR)
备注: 15 pages. Published in the Proceedings of EMNLP 2026 (Main Conference)
Abstract:CoT-style LLM recommenders generate a natural-language preference note as a chain-of-thought prefix to ranking, but the note is trusted without verification, and the ranker never consults behaviorally similar users. We propose ReflectRec, organized around a preference-to-history round-trip verifier whose recall score serves as an operational signal of preference-to-history recoverability, enabling conditional regeneration for low-recoverability preferences and selection among regeneration attempts. Around this verifier, we wire a semantic Stage-1 prompt, conditional regeneration, and Forward-Look retrieval that injects future watches of behaviorally similar users as a collaborative hint. On MovieLens-100K and MovieLens-1M, the full pipeline outperforms CoT4Rec-ICL by +16.25 / +4.46 pp Hit@10, with positive gains across all ValScore buckets and three LLM backbones (Qwen3-30B, Qwen2.5-72B, and Llama-3.3-70B). We also observe a similar verifier-guided regeneration pattern on Amazon Electronics.
[IR-5] CONCORD-Rec: Catalog-Conditioned Modality Utility for Semantic-ID Diffusion Recommendation
链接: https://arxiv.org/abs/2611.23710
作者: Haopeng Jin,Bingkang Shi,Tao Yu,Yifei Ma,Hao Liu,Hongzhu Yi,Jinwen Luo
类目: Information Retrieval (cs.IR)
备注:
Abstract:Multimodal generative recommendation with discrete semantic identifiers (SIDs) requires distinguishing historical modalities that merely alter token predictions from those that genuinely reduce catalog-level ambiguity. When audiovisual, visual, or textual streams are missing, degraded, or semantically conflicting, uniform fusion and unconstrained token gates can propagate misleading content across iterative denoising steps. We propose CONCORD-Rec, a reliability-conditioned discrete diffusion recommender supervised by counterfactual catalog-partition utility. Given a fixed tokenizer and a partially masked SID state, CONCORD-Rec evaluates out-of-fold teacher coalitions over the exact feasible catalog support and aggregates item posteriors across the target code partition at each unresolved position. Rather than distilling raw token probabilities directly, the resulting counterfactual loss differences supervise an availability-masked tokenwise gate that allocates bounded modality residuals around a behavioral denoiser while retaining an exact behavioral fallback slot. We establish theoretical fixed-support ranking stability bounds under gate perturbations and evaluate position-level and code-conditional allocation across audiovisual and image-text recommendation benchmarks under controlled missingness, signal degradation, and semantic conflict. Across 20 source-compatible protocol-dataset rows, CONCORD-Rec improves NDCG by 4%; under matched 30% conflict it improves NDCG by 14.63%. The full-vs-uniform-T=16 sweep improves quality by 2.00-3.58% while reducing p95 latency by up to 86.89%.
[IR-6] KnowMe Paper Radar: Lightweight Discovery of Promising Papers in the Age of Autoresearch
链接: https://arxiv.org/abs/2611.23702
作者: Zhuo Chen,Jian Wang,Jinjie Gu,Lijun Wu
类目: Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
备注: Ongoing work. 67 pages with 34 figures and 37 tables. Project website: this https URL GitHub: this https URL
Abstract:The growing volume of topically similar papers makes it difficult to identify new work that can offer useful research insights or methodological guidance. We present KnowMe Paper Radar, a lightweight discovery tool that combines multiple sources of evidence into personalized reading priorities. Users specify topics, priorities, exclusions and dates; the system brings together topical relevance, recency, author research histories, collaboration paths and direction-specific institutional context. By exposing each score component and its supporting sources, KnowMe helps readers select promising papers for their current questions and discover relevant work beyond familiar authors. Automatic acquisition, version merging and graph-cache reuse keep this workflow convenient without large language model (LLM) inference during ordinary use. Experiments on 174,640 acquired records show more than threefold faster repeated context preparation, while cached abstracts improve normalized discounted cumulative gain at rank 10 (NDCG@10) from 0.412 to 0.598 on expert-judged retrieval tasks. Complementary analyses examine how the same accumulated records support semantic retrieval, research-history modeling and literature-grounded planning. A browser interface and a structured command-line interface (CLI) make these capabilities available to both researchers and research agents, positioning KnowMe as a practical upstream tool for literature-informed recursive self-improvement (RSI) workflows.
[IR-7] VIBRA: Backward-Compatible Cross-Modal Retrieval with Consensus Barycenters
链接: https://arxiv.org/abs/2611.23606
作者: Shubing Yang,Xutong Liu,Dongfang Zhao
类目: Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR)
备注:
Abstract:Backward-compatible retrieval enables a new encoder to query the gallery that an old model has already encoded, so that a cross-modal retrieval system can be upgraded without re-encoding billions of items. Existing compatible methods either fit the old query and stay near the accuracy of the old model, or run the old and the new encoder for every query. We propose VIBRA, which matches relevance on the old gallery, corrects the errors of a single model with the barycenter of several views, which is itself a query in the old space, and trains the new encoder to inherit this consensus, so that models absent at deployment still shape every query. We show that the barycenter gains margin over the average of its views exactly when their hardest irrelevant items differ, and we characterize the share of this gain that one encoder can reach. On five benchmarks with five upgrades among six CLIP, MetaCLIP and SigLIP2 backbones, VIBRA is the best compatible method in all 50 settings, improving Recall@1 by up to 5.6 points over the strongest baseline and by up to 18.0 points over the old model, while adding less than 1 ms to the latency of the new encoder.
[IR-8] From Docket Minutes to Case Trajectories: Building Analysis-Ready Criminal Court Data
链接: https://arxiv.org/abs/2611.23537
作者: Erfan Hosseini Sereshgi,Kory Rosen,Jaya Dhariwal,Nicholas Mattei,Aron Culotta
类目: Information Retrieval (cs.IR)
备注: This paper has been accepted at IEEE Big Data 2026
Abstract:Criminal court records are often fragmented, noisy, and siloed, making it challenging for both legal experts and community stakeholders to access and analyze court data to identify trends and provide public oversight. In this paper, we introduce a system for data ingestion, harmonization, and dashboard visualization designed for New Orleans criminal court proceedings data, aimed at both expert and lay audiences. Our system collects and harmonizes disparate data, generates case trajectories, and provides intuitive visualization techniques based on community feedback to analyze demographics, temporal trends, and specific case events. We demonstrate the system through an exploratory analysis of recent short-term trends and policy impacts within New Orleans, illustrating how it can support the investigation of anomalies, policy changes, and procedural irregularities. We also present feedback from a small formative study of community stakeholders who have helped co-design and use the system, suggesting that it may improve access to and interpretation of local court data.
[IR-9] LusoClin: A Synthetic Clinical Note Dataset for Medical Information Retrieval in European Portuguese
链接: https://arxiv.org/abs/2611.23454
作者: Carla Teixeira Lopes,Daniel Félix
类目: Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
备注:
Abstract:Access to large-scale clinical text collections is crucial for advancing medical information retrieval research, including clinical note search, patient cohort identification, and domain-adapted retrieval models. However, real clinical notes are rarely available due to privacy regulations, which particularly limits progress for low-resource languages such as European Portuguese. We present LusoClin, a publicly released dataset of synthetic clinical notes in European Portuguese, created to support privacy-preserving research on clinical text retrieval. LusoClin was generated by pre-processing real clinical data, fine-tuning the open-source Llama 3.2 (3B) instruction-tuned model using Quantized Low-Rank Adaptation, and applying prompt-based clinical note generation. To facilitate semantic indexing and clinical search, the notes are automatically annotated with clinical entities using a BERT-based model. We evaluate LusoClin in terms of utility, fidelity, diversity, and privacy using automatic metrics and human evaluation. Results indicate that the dataset captures clinically relevant language patterns and can support downstream IR applications such as clinical document retrieval and domain-specific model adaptation, although annotators can often distinguish synthetic from real notes and some medical inconsistencies persist.
[IR-10] Prioritize Dont Decide: What We Learned Auditing a Deployed Human-in-the-Loop CV Sorter EACL2027
链接: https://arxiv.org/abs/2611.23359
作者: Hadi Mohammadi,Jeroen Sparla,Alexandros Paliouras,Georg Krempl,Robert A. Bagheri
类目: Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG)
备注: 11 pages, 4 figures, 6 tables. Under review at the EACL 2027 Industry Track
Abstract:A single academic vacancy can draw hundreds of applications, and the first screening round consumes most of a committee’s time. We describe a CV Priority Sorter deployed at a national platform for academic vacancies and integrated with its vacancy database. It proposes a reading order for a pile of CVs and leaves every decision to the committee, pairing a frozen multilingual transformer encoder with a per-session linear model that retrains after each label and is discarded when the session ends. It works: on a pool of 300 synthetic CVs, a committee reading in the tool’s order reaches 95% of the relevant candidates after half the pile, where an arbitrary order needs almost all of it. This paper is about what we found when we audited that claim, rather than about the system itself. Three findings changed the product. The pool turned out to be sortable by document length alone, which makes precision-style metrics on it uninformative; regenerating it under a shared length budget removes that shortcut and leaves the reading-depth result standing. The four-level agreement score such systems report, ours included, is a binary probability cut at fixed thresholds, so it follows the classifier’s confidence rather than its ranking and falls as labels accumulate. And uncertainty sampling, the obvious way to implement a “suggest the next CV” button, leaves model quality intact but raises the committee’s total reading load above the no-label baseline, because the CVs it asks about are the ones that would otherwise be skipped. A fourth result shows where to look next: a hosted LLM embedding model lifts zero-label ordinal discrimination from 0.66 to 0.80 under length control, then stops mattering once the session model has two dozen labels; a local cross-encoder matches that cold-start gain without sending data out. We report the fixes each finding forced.
[IR-11] PGPO: Prototype-Guided Progressive Obfuscation for Privacy-Preserving LLM -Enhanced Recommendation EMNLP2026
链接: https://arxiv.org/abs/2611.23236
作者: Shengxiang Lin,Jiajie Su,Pengyang Zhou,Xiang Chen,Xiaolin Zheng,Feng Tian,Chaochao Chen
类目: Information Retrieval (cs.IR); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
备注: Accepted to EMNLP 2026
Abstract:LLM-enhanced recommender systems encode item side information into semantic embeddings, but these embeddings may leak sensitive textual attributes through inversion attacks. Existing input-side defenses rely on trusted collection or local token perturbation, often disrupting the inter-item semantic topology needed for recommendation. To address it, we introduce relational fidelity as the key objective, requiring semantic shift for privacy while preserving relative item relations for utility. Based on this principle, we propose prototype-guided progressive obfuscation (PGPO), which learns topology-aware obfuscation anchors for high-influence prototype words and propagates them through a static semantic graph. PGPO is optimized with GRPO using individual dissimilarity, structural consistency, and semantic constraint rewards. Experiments on two benchmarks show strong inversion defense with near-upper-bound recommendation performance. The code is available at this https URL.
[IR-12] ask-Oriented Network Ranging via Joint Graph and Schedule Design
链接: https://arxiv.org/abs/2611.24702
作者: Zijian Zhang,Yuan Shen
类目: ignal Processing (eess.SP); Information Retrieval (cs.IR); Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Systems and Control (eess.SY)
备注: 16 pages, 14 figures, with appendices
Abstract:Measuring inter-node distances is the basis for network localization, whereas acquiring node positions does not require a fully connected graph of distances. Existing complete-ranging protocols for network localization require every node pair to exchange signals, which prolongs the ranging cycle. In this paper, we propose sparse signal-multiplexing network ranging (S-SM-NR), a task-oriented framework that adapts the transmit-listen schedule to the ranging task required by network localization. First, we establish the S-SM-NR framework by formulating the joint design of the task graph and the transmit-listen schedule. Then, we analyze the minimum round count required by S-SM-NR and establish a connectivity condition for network localization. Subsequently, we develop a closed-form joint design that constructs the sparse task graph and the balanced transmit-listen schedule. In particular, the joint design can achieve the minimum round count under the complete physical connectivity. Finally, simulations show that the proposed S-SM-NR scheme reduces the occupied-round count and improves the localization accuracy by shortening the ranging cycle.
人机交互
[HC-0] Knowing What You Know: A Trait Theory of Reviewer Value in Human-AI Cognitive Systems
链接: https://arxiv.org/abs/2611.24932
作者: Priyanka Sinha,Sugam Pandey
类目: Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
备注: Accepted at the Thirteenth Annual Conference on Advances in Cognitive Systems (ACS), Edinburgh, United Kingdom, October 6-9, 2026. Awarded one of 2026 Ada Lovelace Awardees for Excellent Paper Submissions to ACS
Abstract:When a capable model and a human reviewer cooperate on the same decision, which human still adds value? We argue this is a question about a single cognitive trait, metacognitive calibration: the alignment between a reviewer’s behavioral confidence and the actual correctness of their judgments. We develop a trait theory of reviewer value to make it precise. The theory makes five claims: value is governed by calibration rather than by activity, speed, agreement, or tenure; calibration leaves observable traces in behavioral telemetry; those traces are non-diagnostic on their own, because the fast, low-revision signature of a calibrated expert is indistinguishable from that of a confident novice until it is gated by outcomes; human calibration and a model’s self-knowledge are the same construct, so the joint system should defer to whichever agent is better calibrated on each item; and individual calibration is causally identifiable only above the level of the single judgment. We ground each claim in a six-month production window of an AI-assisted accounting system (541621 reviewer events, 483 reviewers), and find that a purely behavioral expertise profile inverts the ground truth, scoring an overconfident cohort (5.2 percent correction survival) higher than genuine experts (99.2 percent); that reviewer quality is flat across tenure, so the trait is not learned on the job; and that the same calibration logic that separates good from bad reviewers, applied to the model, concentrates 92.5 percent of human effort on genuine errors. We close with the boundary the theory must respect: under majority-vote aggregation, individual causal value is unidentifiable.
[HC-1] Who are you? The Personal Imitation Game
链接: https://arxiv.org/abs/2611.24888
作者: Muhammad Rizky Wellyanto,Hajara-Yasmin Isa,Kexin Xue,Vishal Ravichandran,Wenpeng Lu,Jerry O. Talton,Ranjitha Kumar
类目: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 19 pages, 6 figures, 5 tables
Abstract:Large Language Models (LLMs) now perform strongly on Alan Turing’s 1950 Imitation Game, so we propose a harder challenge, the Personal Imitation Game, which asks whether a machine can replicate a specific individual in conversation rather than a generic human. Testing with 30 participants, we establish a human baseline and report the identity signals interrogators rely on: stylometric features, personality traits, and biographical facts. To demonstrate the framework, we design an LLM-based system that plays the game for one of the paper’s authors, drawing on an autoethnographic dataset spanning ten years of personal digital communication. In a 20-person deception study, interrogators choose our system over the author’s own friends in 7 of 10 rounds, and the author over our system in 13 of 20; neither rate is distinguishable from chance at this sample size. We consider design improvements, applications of personal AI agents, and implications for identity and privacy.
[HC-2] Overlooked and Underutilized: Prototypes are the Missing Piece in Tabular XAI
链接: https://arxiv.org/abs/2611.24831
作者: Jacek Karolczak,Jerzy Stefanowski
类目: Machine Learning (cs.LG); Human-Computer Interaction (cs.HC)
备注: Accepted for publication in the Journal of Intelligent Information Systems (Springer Nature)
Abstract:Explainable AI (XAI) is frequently applied to high-stakes tabular data, yet the field relies heavily on feature attribution methods. While these approaches offer technical transparency, evaluating dozens of isolated features demands excessive cognitive effort from non-expert users. In this paper, we argue that prototypes remain overlooked and underutilized, and should be integrated as a complementary, intuitive gateway to auditing complex machine learning models. Although entire exemplars function as an intuitive baseline for prototype-based explanations, we highlight prototypical parts as particularly valuable for tabular data. By grouping related features into bounded subspaces, prototypical parts technically embody human-understandable concepts, replacing isolated features with cohesive semantic units. We support this integration through two primary lenses. Cognitively, these parts align with Rosch’s categorization theory and Kahneman’s System 1 thinking to naturally reduce cognitive load for humans. Then, we introduce a Geometric Validity Triad to conceptualize why prototypes validate the data manifold - a critical dimension often neglected by existing methods. We contend that making tabular XAI genuinely comprehensible requires shifting toward composite explanation pipelines that pair intuitive similarity-based reasoning with rigorous feature-level auditing. The paper concludes by outlining open challenges of prototype-based methods and discussing potential solutions to guide future research.
[HC-3] A Foundation Model to Measure and Predict Human Behaviour
链接: https://arxiv.org/abs/2611.24830
作者: Haoran Ye,Tianrui Huo,Hangzhen Li,Xin Zhang,Guojie Song
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
备注: 48 pages, 6 figures
Abstract:Large language models (LLMs) are increasingly used to infer psychological attributes, predict individual behaviour, and simulate human populations. However, despite generating plausible responses, they often misrepresent individuals and distort population statistics. Here we introduce Chiron, a foundation model to measure and predict human behaviour. We jointly fine-tune an LLM to represent psychological constructs, predict individual behaviour, and align population response distributions with human statistics. The training and evaluation data span 36 behavioural domains and 78 social science experiments, comprising 288,307 person records, 1,411,750 atomic behavioural decisions, and 99,953 psychometric assessments over 109 psychological constructs. In our evaluations, Chiron’s psychological measurements agree more closely with human self-reports and exhibit fewer biases that undermine psychometric validity. It predicts individual behaviour more accurately than frontier LLMs (up to 28.4% relative improvement). In simulations of social science experiments, it more accurately captures the direction and magnitude of human experimental effects (82.2–94.3% lower effect-size estimation error). Together, these results establish Chiron as a unified computational instrument for measuring and predicting human behaviour.
[HC-4] One size does not fit all: session-specific effects of deep brain stimulation on hand-motor kinematics in Parkinsons disease
链接: https://arxiv.org/abs/2611.24823
作者: Matthias Dold,Joana Pereira,Volker A. Coenen,Bastian Sajonz,Sophia Gimple,Yasin Temel,Marcus L.F. Janssen,Michael Tangermann
类目: Human-Computer Interaction (cs.HC)
备注:
Abstract:Deep brain stimulation (DBS) effectively treats motor symptoms of Parkinson’s disease (PD), but it is unclear whether it changes fine hand movements in the same way for every patient. We analyzed 35 sessions of 19 patients with PD from a multi-center cohort performing the CopyDraw task, a drawing task designed to overcome the subjectivity and limited granularity of clinical assessments. Besides standard kinematics (mean velocity, acceleration, jerk, and drawing accuracy), we derived higher distributional moments and non-linear complexity measures, for 38 features in total. DBS effects were quantified with linear mixed models at the group level and with residualized regression and a random-effects meta-analysis per session. The DBS state was decoded per session with regularized linear discriminant analysis to compare feature sets as behavioral labels for deriving physiomarkers for adaptive DBS (aDBS). DBS modulated all 38 features at the group level, most strongly velocity-related features. However, for every feature, the between-subject variability of the DBS effect exceeded the group-level effect. Only one feature, the detrended fluctuation analysis of the distance to the template, showed a significant mean effect across sessions, and it is not a standard kinematic feature. In six sessions without any significant modulation of standard features, complexity measures were modulated instead. For decoding, standard features alone were the best choice in ten of 35 sessions, but combining all 38 features yielded the highest average area under the ROC curve (0.700 vs. 0.677), with the largest gains in sessions that were hard to decode from standard features. DBS effects on hand-motor behavior are individual. Deriving neural markers for aDBS from behavior therefore requires session-specific analyses with individualized features beyond standard kinematics and group-level effects.
[HC-5] Data-driven simulation of the brains response to stimulation: a system identification approach for closed-loop neuromodulation piloted in adaptive deep brain stimulation
链接: https://arxiv.org/abs/2611.24818
作者: Matthias Dold,Joana Pereira,Marcus L.F. Janssen,Michael Tangermann
类目: Human-Computer Interaction (cs.HC)
备注:
Abstract:Closed-loop neuromodulation uses a physiomarker to control stimulation. Developing such controllers requires exploring a large space of strategies and parameters, which is often not feasible with the human-in-the-loop. Simulation environments offer an alternative. Adaptive deep brain stimulation (aDBS) for Parkinson’s disease is a stereotypical application: while physiomarker identification progresses rapidly, clinically available control is limited to threshold and proportional strategies. We propose to identify dynamical models of the brain with and without deep brain stimulation (DBS) using the sparse identification of nonlinear dynamics (SINDy) framework. Identified ex post from existing recordings, such models are a middle ground between abstract bottom-up models and the realistic but difficult-to-access human-in-the-loop. Forecasting neural activity from real initial conditions, the models create a simulation environment for “what if” analyses of DBS changes. In a proof-of-concept with EEG data from a single subject, we train and evaluate threshold, proportional, proportional-integral-derivative (PID), and proximal policy optimization (PPO) control, aiming to have the simulated data classified as DBS ON at minimal energy. SINDy models fitted on spatially filtered EEG maintained DBS condition-specific characteristics. All aDBS controllers delivered DBS more efficiently than continuous DBS (cDBS), with up to 20.2% energy reduction. Proportional control yielded a small but significant gain over threshold control. PID and PPO achieved a comparable reward advantage over cDBS, with PPO stimulating more conservatively and PID saving the most energy. The approach thus proved feasible, and within this simulation environment, PID control is a strong a-priori choice, while the benefit of deep reinforcement learning may only emerge with richer physiomarker signals.
[HC-6] owards Transparent Computational Agents for Air Traffic Control: Explainability Interpretability and Trust
链接: https://arxiv.org/abs/2611.24791
作者: Rob Procter,Mark Rouncefield,Richard Everson,George De Ath,Nick Pepper,Ed Henderson,Elhassan Mohamed,Eseoghene Benjamin,Ben Carvel
类目: Human-Computer Interaction (cs.HC)
备注: 3 pages, 1 figure
Abstract:We report preliminary findings from an ongoing investigation into transparency techniques for computational Air Traffic Control agents. Air Traffic Controller Officers (ATCOs) observed two types of agent in simulated air traffic controlling exercises. ATCOs found the chosen transparency techniques helpful to some extent: revealing how agents were making controlling decisions was an essential foundation for trust. However, the ATCOs’ trust depended on whether they could be made sense of in a live, operational context and the agents’ actions where judged to be professionally competent.
[HC-7] Interpersonal trust institutional trust and trust in technology in Air Traffic Control
链接: https://arxiv.org/abs/2611.24715
作者: Rob Procter,Mark Rouncefield
类目: Human-Computer Interaction (cs.HC)
备注: 3 pages
Abstract:Air traffic control (ATC), the safe, orderly and efficient management of planes through controlled airspace is a complex task. In this article, we explore the roles that different kinds of trust play in ATC. Our study provides the essential foundations for examining the conditions that need to be satisfied if Air Traffic Controllers are to establish and sustain trust relationships with technical innovations, such as agent-based systems, and so ensure their safe and dependable embedding within ATC work.
[HC-8] Negotiating Control and Accountability among Stakeholders in High-Risk AI Use Cases: Development and Evaluation of the Method ALIGN
链接: https://arxiv.org/abs/2611.24609
作者: Lena Schneider,Daniel Boos,Gudela Grote
类目: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
备注:
Abstract:Characteristics of Artificial Intelligence (AI) severely restrict human control. This creates new challenges for socio-technical systems design especially in safety-critical domains, where control over risks is essential along with the accountability for any harm caused. Drawing from human factors and AI governance literature, expert interviews and participant observation in a complex high-risk AI use case from the railways, we developed ALIGN, a scenario-based method that supports the governance of and stakeholder involvement in AI development and implementation in three ways: raising awareness for potential accountability-control misalignments in socio-technical systems and their negative consequences, offering a structured approach to assessing the alignment between accountability and control for stakeholders in a given use case, and supporting effective design decisions to mitigate AI risks. We describe the iterative development and evaluation of ALIGN and discuss how this method promotes AI governance by highlighting the relevance of control-accountability alignment throughout the AI life cycle.
[HC-9] he Impact Of Multitasking In Video Meetings On Meeting Outcomes: Initial Results
链接: https://arxiv.org/abs/2611.24545
作者: Julia Seitz,Fabio Stano,René Riedl,Alexander Mädche
类目: Human-Computer Interaction (cs.HC)
备注: Published within the European Conference of Information Systems (ECIS) 2025. this https URL
Abstract:Multitasking in video meetings is a ubiquitous phenomenon. While extant literature has investigated the circumstances under which people multitask in video meetings, little is known about the impacts of multitasking on meeting outcomes. Building on the findings of previous studies investigating the impact of multitasking in individual non-video meeting work setups and threaded cognition theory, we present a research model that suggests that multitasking may have negative effects on meeting outcomes such as process loss and team meeting outcome satisfaction. We tested the research model in an incentivized within-subject laboratory experiment with n=72 participants. Results suggest that multitasking has a negative effect on process loss and team meeting outcome satisfaction compared to non-multitasking. We plan to further investigate the mediating role of cognitive user states and neurophysiological activations and ultimately aim to contribute with a deeper understanding of the impacts of multitasking.
[HC-10] Exploring Daily Emotional Well-Being through Personalized Data Analysis
链接: https://arxiv.org/abs/2611.24342
作者: Adriana Rodríguez Rodríguez
类目: Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
备注: 11 pages, 3 figures, 3 tables. Code: this https URL
Abstract:This project analysed daily emotional well-being from a personalized perspective using records of emotional states and contextual information derived from daily life. It was based on the idea that well-being is not a fixed value, but varies from day to day, and can be better understood in relation to individual context and behaviour. The work is framed within the field of Data Science and focuses on emotional states at an individual level. The project focused on constructing an integrated daily dataset from multiple personal data sources and analysing emotional states. The distributions of emotional variables, their temporal evolution and the differences between users were studied, together with their variability over time. Personality and well-being questionnaires were also incorporated as stable references to better contextualize daily emotional results. A simple personalized recommendation approach was then developed by comparing similar days within each user, exploring how past contextual conditions relate to differences in the user’s emotional state. Finally, machine learning techniques were used to evaluate whether daily contextual information contains signals that help distinguish between different types of days relative to a person’s usual well-being. The project adopted an applied approach, with an exploratory scope focused on understanding patterns rather than building a fully predictive system. The results showed that emotional well-being is very personal and changes from day to day. General models had limited value, while personalized approaches captured these differences much better, highlighting the importance of analysing each person on their own terms.
[HC-11] Exploring multimodal AI–XR interactions for software development: From expert perspectives to semantic code outlines
链接: https://arxiv.org/abs/2611.24334
作者: Ilya Zakharov,Alexey Markin,Konstantin Ostroukhov,Alexey Perminov,Roham Koohestani,Ekaterina Koshchenko,Agnia Sergeyuk
类目: Human-Computer Interaction (cs.HC); Software Engineering (cs.SE)
备注: Submitted to HIMAS 2027: Human-Centered Interactive Multimodal AI Systems, co-located with ACM IUI 2027
Abstract:AI tools can generate code quickly, but developers still need to understand and check the results. We explore how the enriched multimodal context derived from the extended reality (XR) workspace might help developers manage this information. We first interviewed 13 experts in AI, XR, and related fields to identify opportunities and design challenges (Study 1). These interviews informed two prototypes explored with 12 professional developers (Study 2): four tried a version where the code faded to become invisible after it was looked at, and eight tried a version that replaced viewed code sections with short descriptions of their purpose. In the second prototype, head direction helped determine when to offer a change, while manual input let developers apply it and reopen the original code. A separate open-ended demonstration with seven software-engineering researchers provided complementary feedback. Participants’ accounts suggested that descriptions could provide an overview of unfamiliar code and serve as reminders when revisiting it. However, difficulty comparing descriptions with source code, headset limitations, and controls that deviates from regular IDE routines could introduce additional effort. These exploratory findings identify requirements for attention support: make summaries easy to recognize and access, preserve straightforward inspection of the original code, and retain familiar navigation and editing practices. The study contributes to human-centered multimodal systems by showing how the coordination of input channels and AI-generated representations must account for users’ attention, verification needs, and established ways of working.
[HC-12] ableControl: Gesture Interaction Through Surface Vibrations
链接: https://arxiv.org/abs/2611.24218
作者: Florian Hettstedt,Andreas Erbslöh,Gregor Schiele
类目: Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
备注: 4 pages, 3 figures, accepted as demo paper on the ACM ISS
Abstract:Integrating gesture sensing into furniture offers a way to interact with digital applications and physical devices through familiar surfaces. We present TableControl, an interactive demonstration of continuous surface-vibration-based gesture recognition with the complete processing pipeline running on a microcontroller. Using four concealed piezoelectric sensors, the system combines signal acquisition and preprocessing with neural event detection and gesture classification. The resulting labels for knock, swipe-up, and swipe-down gestures are transmitted to a host computer without streaming the underlying sensor signals. Visitors explore two example applications through hands-on interaction: an endless runner game and a physical table lamp with on/off and brightness control. A dashboard displays recognized gestures, class probabilities, and processing times.
[HC-13] MemoSync: Co-Managing Developer-Agent Metamemory Across Contexts
链接: https://arxiv.org/abs/2611.24165
作者: Baiqiao Zhang,Junze Li,Chunyang Luo,Yuying Tang,Wenshuo Zhang,Shiyuan Piao,Dingdong Liu,Shuai Ma,Xiaojuan Ma
类目: Human-Computer Interaction (cs.HC); Software Engineering (cs.SE)
备注:
Abstract:AI coding agents increasingly manage what context carries across sessions and projects, but developers have limited visibility into what these agents store, retrieve, and apply. This opacity creates a metamemory problem: the developer’s meta-level judgments about what should be remembered are separated from the agent’s object-level memory operations, disrupting the monitoring and control that keep the two levels aligned. We conducted a formative study with 12 developers and identified what they want to monitor and control across four dimensions of agent memory use. Based on these findings, we designed MemoSync, a co-management system that treats agent memory as a shared, operable representation: the agent proposes memory candidates and reports which items it uses, and the developer reviews, edits, and scopes them. In a between-subjects study (N=132) against two baselines, MemoSync participants achieved significantly higher task performance and the highest mean developer-agent memory alignment. In an exploratory mediation analysis, the alignment advantage was primarily driven by improved monitoring accuracy. Despite performing substantially more monitoring and control actions, participants reported no increase in perceived workload. A week-long deployment with nine developers showed that memory monitoring and control fit into everyday development work. MemoSync is open-sourced at this https URL.
[HC-14] Human-LLM Cognitive Extension: A Framework for Studying Coupled Cognition
链接: https://arxiv.org/abs/2611.24149
作者: Ernesto Biempica,Emiliano Bruner,Juan F Poyatos
类目: Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)
备注: 24 pages, two supplementary tables
Abstract:Recent research on the use of Large Language Models (LLMs) reports divergent cognitive outcomes, including improved assisted performance, limited transfer, and compromised unaided learning. Here, we argue that part of this seeming inconsistency arises because studies often assess different cognitive systems without making the unit of analysis explicit. Building on work in extended and distributed cognition, we propose the Human-LLM Cognitive Extension (HLCE) as a methodological framework for matching experimental designs, measures, and causal inferences to four distinct systems: the human acting independently, the artificial agent, the joined human-LLM configuration, and the human after division. The framework also treats the coupling architecture as part of the experimental object. HLCE separates performance augmentation, system-level complementarity, post-interaction human change, and stronger claims of constitutive cognitive integration. This separation can improve the interpretation of existing evidence and guide the design of human-LLM interactions without assuming in advance that these configurations constitute extended minds. We illustrate the application of HLCE with a synthetic diagnostic experiment comparing two connecting configurations that differ in how supporting information is communicated. The example highlights how distinct couplings can shape the organization of joint problem-solving, even when the final outcome remains unchanged. Overall, HLCE provides a methodological basis for designing controlled experiments and interpreting evidence across distinct forms of human-LLM cognitive interaction.
[HC-15] Effects of Locomotion and Viewpoint Control on Immersive Robot Telepresence for Navigation and Inspection
链接: https://arxiv.org/abs/2611.23939
作者: Ammaar Zaman,Neven ElSayed,Oliver Prentner,Mohamed Ellethy,Mostafa M. Lotfy,Maram Selsabila Mahmoudi,Eduardo Veas
类目: Robotics (cs.RO); Human-Computer Interaction (cs.HC)
备注:
Abstract:In many robot telepresence interfaces, movement and viewing are managed through handheld inputs. This can divide attention between steering, looking, and inspection. We compared joystick versus omnidirectional treadmill locomotion and joystick-based versus head-guided camera control in a within-subject 2x2 study in which 17 participants used a virtual reality headset to guide a quadruped robot through a physical maze while locating marked wall symbols and capturing them as visual evidence. Mixed-effects analyses showed that head-guided camera control was associated with faster completion, lower wall-contact cost, lower workload, and more favorable presence, usability, technology acceptance, and flow than joystick-based camera control. The strongest overall objective performance was observed in the joystick locomotion plus head-guided camera condition, whereas the omnidirectional treadmill plus head-guided camera condition emerged as the strongest experiential compromise. Exploratory camera analyses showed that head-guided camera control reduced switching, increased reliance on the pan-tilt view, and supported a more stable viewing strategy across the maze. These strategy differences were consistent with the performance advantage, especially for wall-contact duration. Targets captured remained near ceiling and simulator sickness remained low across all conditions. Overall, the results indicate that viewpoint control was the stronger differentiator in this combined navigation and inspection task, while locomotion still shaped the objective and experiential tradeoff.
[HC-16] Learning Martial Arts Motion Skills for Animating Opponents in VR Sparring Games
链接: https://arxiv.org/abs/2611.23825
作者: Jiqing Wen,Qixuan Shi,Robert LiKamWa,Pavan Turaga
类目: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Robotics (cs.RO)
备注:
Abstract:Animating virtual opponents is an important yet challenging task in game development. In martial arts games, opponent motions must not only be physically plausible but also exhibit appropriate technique, timing, transitions, and target-directed intent. Traditional game animations often rely on handcrafted clips, state machine transitions, and procedural adjustments, which can be labor-intensive and may limit the behavioral diversity of non-player characters. This paper presents a martial arts motion skills learning framework for character animation in virtual sparring games. Inspired by Tang Soo Do, we collected a martial arts motion capture dataset containing isolated attacking and defending techniques, technique combinations and transitions, and contextual sparring variations. We first pretrained a reusable repertoire of martial arts motion skills from the captured data using reinforcement learning (RL) with adversarial style rewards, and then adapted these skills with task-specific rewards for target-directed striking and threat-aware evasion. We evaluated the motions generated by the learned policy through a user study, in which participants assessed their physical plausibility, smoothness, and martial arts form. To demonstrate their applicability to interactive gameplay, we developed a VR sparring prototype for Meta Quest headsets. The prototype combines policy-generated motion clips with lightweight state-based control to support interactive opponent behavior. Together, these findings support the feasibility of using a learned martial arts skill repertoire as a motion foundation for animating opponents in VR sparring games.
[HC-17] MEM-STRESS: Longitudinal Field Study of a Wearable-Informed AI Chatbot for Stress Management
链接: https://arxiv.org/abs/2611.23691
作者: Soorya Ram Shimgekar,Pragya Singh,Shivam Sharma,Chandrayee Roy,Koustuv Saha,Antti Oulasvirta,Talayeh Aledavood
类目: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
备注:
Abstract:Stress is an increasingly prevalent concern, creating a need for accessible and personalized support. Stress-management chatbots primarily rely on what users report, which may not capture their physiological changes. Wearables capture these changes, but chatbots rarely use them as memory across conversations. We therefore present MEM-STRESS, a stress-support chatbot that turns a wearable ring’s physiological data into a layered memory of daily records, baseline deviations, LLM-generated summaries, and uses it to drive multi-session conversations grounded in Dialectical Behavior Therapy (DBT). In a four-week, non-randomized study with 60 adults, we compared MEM-STRESS (Treatment) with a chatbot remembering only past conversations (Control). Memory shaped the conversations: The Treatment chatbot referenced participants’ physiological features (e.g., activity in 71.8% of sessions) and suggested more body-related DBT skills than Control (14.5% vs. 5.7%), with Treatment participants also using slightly more body-related words (1.76% vs. 1.45%). Treatment participants praised personalization three times as often (44.4% vs. 14.3%), more frequently reported reflecting differently on stress (69.0% vs. 46.4%, not significant), and described gaining insight into their stress signals. Their perceived stress fell 14.2% from baseline versus 3.6% for Control, though the group difference was not significant. Grounding stress-support chatbots in layered physiological memory is thus feasible and may improve perceived personalization, motivating larger randomized studies. Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI) Cite as: arXiv:2611.23691 [cs.HC] (or arXiv:2611.23691v1 [cs.HC] for this version) https://doi.org/10.48550/arXiv.2611.23691 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[HC-18] EEG2MCU: Sensing-to-Execution Co-Design for Wearable EEG Inference DATE2027
链接: https://arxiv.org/abs/2611.23619
作者: Shoma Yukawa,Meghna Roychowdhury,Arnab Sanyal,Chengwei Zhou,Shreyas Sen,Gourav Datta
类目: Machine Learning (cs.LG); Hardware Architecture (cs.AR); Human-Computer Interaction (cs.HC); Signal Processing (eess.SP)
备注: 7 pages, 9 figures, 3 tables. Accepted at DATE 2027
Abstract:Wearable EEG inference is constrained not only by model size, but also by the channel-dependent sensing front end that must process high-dimensional inputs before compact downstream computation. We present EEG2MCU, a sensing-to-execution co-design framework that treats electrode count as a hardware design variable through task-aware montage selection. EEG2MCU retains a compact time-frequency representation and generates only the INT8 regions required by each first-layer tile, avoiding full input materialization. Halo reuse and access-aware placement across Flash, SRAM, and tightly coupled memory further reduce redundant generation and data movement. On a Cortex-M7 MCU, EEG2MCU reduces peak SRAM by 11.38x relative to the baseline, including a 1.71x reduction from the 62-to-19 electrode montage, while reducing latency by 2.57x and energy by 2.48x. It retains 94-96% of full-montage accuracy, demonstrating practical high-dimensional EEG inference under tight embedded-resource constraints.
[HC-19] Miscalibrated AI improves human calibration: An alignment problem for AI-assisted Decision education
链接: https://arxiv.org/abs/2611.23618
作者: Phillip C. Hegeman,Eeshan Hasan,Eungik Lee,Andrew Caplin,Jennifer S. Trueblood
类目: Human-Computer Interaction (cs.HC)
备注: 15 page paper, 4 paper figures, 21 page supplement, 1 supplemental figure
Abstract:Using artificial intelligence (AI) well requires probabilistic thinking, and central to it is calibration: expressing certainty that matches one’s actual accuracy. Calibration is a decision skill, but a notoriously difficult one to teach. Because AI systems typically generate a probability alongside every prediction, observing these probabilities may help humans learn to provide better calibrated judgments. This is one form of what we call AI-assisted decision education – using AI to teach the judgment skills that decision-making requires – of which learning by observation, as here, is among the simplest forms. In a preregistered experiment, novice participants (N = 300) judged the probability that images of white blood cells were cancerous, before and after exposure to a highly accurate (94%) convolutional neural network. A single model was shown in three ways, holding accuracy fixed while varying the certainty it expressed: its original probabilities, which were conservative and poorly calibrated for the model’s accuracy; a recalibrated version, well calibrated for the model and correspondingly certain; or categorical predictions carrying no probability information. When later tested without the AI, only the original, conservative display improved participants’ calibration. It did so by reducing over-extremity, without a condition-specific improvement in discrimination. Notably, the display that taught best was the one miscalibrated for the model. AI-assisted decision education therefore faces an alignment problem: the signal that best describes a model, need not be the signal that best supports the learner.
[HC-20] Smart Glasses for Ubiquitous Computing: A Layered Survey and the Road Ahead
链接: https://arxiv.org/abs/2611.23534
作者: Linlu Gao,Yueyuan Sui,Junxi Xia,Xin Liu,Stephen Xia,Yang Liu
类目: Human-Computer Interaction (cs.HC)
备注:
Abstract:We conduct a systematic survey of smart-glasses research for ubiquitous computing. Our search covers work published from 2020 to October 2026, with selected earlier work included for historical context. After screening and title-level deduplication, we identify 163 smart-glasses papers and separately compile related resources, including 14 representative hardware platforms, their associated software tools and pretrained models, and 10 datasets. We organize this literature through four complementary layers. Research Resources covers hardware platforms, software tools and models, and datasets that support development and evaluation. Sensing Perception examines how glasses measure and interpret information about users and their surroundings. Interaction Assistance addresses how users express intentions and receive feedback and assistance. Applications situates these contributions in tasks and use contexts. Energy, latency, wearability, reliability, privacy, security, and social acceptability are treated as deployment constraints across all four layers. This framework provides a unified view of current research and a vision for how advances across the four layers could come together to support smart glasses as a ubiquitous computing platform. Building on this synthesis, we outline research directions for extending capabilities across tasks and over time, supporting assistance under human control, and achieving reliable, efficient, and sustained everyday use.
[HC-21] RummiAlgebra: A Polynomial Representation Framework for Game-Based Algebra Learning with Jokers
链接: https://arxiv.org/abs/2611.23443
作者: Luis Adrián Lizama-Pérez
类目: Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)
备注: 15 pages, 2 figures, 2 tables. Software: this https URL
Abstract:Connecting symbolic expressions to manipulable objects is a recurring challenge in introductory algebra. This paper presents RummiAlgebra, a design-oriented framework and browser-based prototype that maps a Rummikub-style tile-rearrangement task to polynomial addition, common-factor extraction, finite geometric sums, and context-dependent substitution. A numbered colored tile is encoded as a monomial c x^v, while a physical joker is represented by a distinct symbol J_i whose algebraic role is fixed only for a particular candidate arrangement. We define admissible polynomial forms for runs and groups and establish an inventory-conservation invariant. A soundness-and-completeness proof shows that physical-tile accounting together with the algebraic run/group templates characterizes valid final arrangements under the stated classroom rules; polynomial equality alone is insufficient. Four detailed examples illustrate decomposition, competing factorizations, one- and two-joker substitutions, and optimization as an optional extension. A targeted review distinguishes the approach from established Rummikub solvers, Rummikub-inspired arithmetic-sequence activities, algebra-tile manipulatives, and polynomial games. The prototype implements six small challenges and a separate rule-and-expression checker; their optimal scores were checked against an independent exhaustive reference enumerator. These checks establish agreement for the examples but do not establish learning effectiveness or novelty of the underlying algebraic operations. We close with a classroom sequence, assessment constructs, and a controlled evaluation protocol. The claimed contribution is a formally specified pedagogical correspondence and an interactive proof-oriented activity, not a new asymptotically faster Rummikub solver.
计算机视觉
[CV-0] UltraTex: Aligning Physical Knowledge for High-Fidelity PBR Texture Generation SIGGRAPH
链接: https://arxiv.org/abs/2611.24941
作者: Jingzhi Bao,Xiaohe Ma,Hong Li,Yuanming Hu,Xiaoguang Han
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted to ACM Transactions on Graphics (SIGGRAPH Asia 2026). Project page: this https URL
Abstract:Generating physically based rendering (PBR) texture maps for a 3D mesh from a single reference image is highly ambiguous, as the observed appearance entangles material identity, surface reflectance, and illumination. While current generative methods produce realistic albedo maps, they still struggle to infer accurate metallic and roughness attributes for relighting. We present UltraTex, a cascaded multi-view framework that moves beyond appearance matching by integrating physical knowledge into multi-view diffusion-based PBR generation. UltraTex first jointly predicts albedo and world-space shading normals to generate more detailed albedo maps, and then predicts metallic-roughness (MR) maps conditioned on the generated albedo, reference image, and material semantic priors. To inject material semantics, a lightweight adapter transforms hidden-layer features of a vision-language model (VLM) into intrinsic material representation tokens, providing part-level knowledge anchors for MR estimation. To model material-light interaction, photometric representation alignment guides intermediate diffusion features toward ground-truth material renderings under controlled lighting. This makes metallic and roughness variations observable through highlight intensity, specular sharpness, and local reflectance contrast, providing stronger guidance for fine-grained material prediction without explicit lighting supervision. Extensive experiments show that UltraTex generates high-quality textures and outperforms existing open-source and commercial methods in visual fidelity, material accuracy, and physical plausibility.
[CV-1] Z-sight: Unifying Vision-Language-Action and Latent World Modeling
链接: https://arxiv.org/abs/2611.24940
作者: Junjie Ye,Yiqi Zhao,Muchen Xu,Celina Shiyu Wang,Hao Dong,Ruohai Ge,Rong Xue,Benjamin J Kolligs,Aditya Dutt,Sasanka Nagavalli,Nassim A. Samad,Gaurav S. Sukhatme,Jyotirmoy V. Deshmukh,Yue Wang,Yuliang Guo
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:We introduce Z -sight, a unified robot foundation model that couples vision-language-action (VLA) and generative latent world modeling. Video-based world action models (WAMs) show the value of joint future-and-action generation for control, yet inherit the heavy cost of video generators. Recent latent-space policies avoid pixel generation, yet typically align intermediate features or rely on frozen target encoders, decoupling future prediction from the VLM’s native feature space. Z -sight instead predicts the future directly in the visual token space in which its VLM perceives the present. Coupling a foresight expert and a continuous action expert with the VLM in a mixture-of-transformers, Z -sight jointly denoises future spatial tokens and actions at inference without pixel reconstruction or external encoders. Trained on human video and robot trajectories, Z -sight achieves 81.9% success on SIMPLE (+9.6 points over the strongest baseline) and 73.3% across six Unitree G1 tasks (+20.0 points) at 110 ms per chunk, while remaining competitive on tabletop benchmarks.
[CV-2] Unified Law for Visual Tasks (The Four Laws of Computer Vision)
链接: https://arxiv.org/abs/2611.24917
作者: Sen Fang,Yalin Feng,Yancheng Zhang,Yanxin Zhang,Zonglin Lyu,Hongbin Zhong,Liwei Che,Rui Wu,Chen Chen,Dimitris N. Metaxas
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Preview version (main paper only); This work began in December 2024. Project page at this https URL GitHub org at this https URL Hugging Face org at this https URL This is an empirical article that attempts to unify thousands of visual tasks through a single methodology. Patent pending (inventors: SF and DM)
Abstract:In this paper, we propose a Unified Law for Visual Tasks, which could be further divided into Four Laws of Computer Vision, with the goal of enabling a single framework or method to uniformly train on and perform nearly all visual tasks. Our starting point is an inspiration and hypothesis: if the world information is encoded into image, which we call World Snapshots, then image processing is world processing, and image generation is World Snapshot generation. Furthermore, if all visual and visualization tasks can be reformulated as generation tasks, then a unified method, World Snapshot Models (WSMs), can perform the vast majority of visual tasks, much as large language models have largely unified natural language processing. This paper validates these laws across more than 1,000 trained and evaluated visual tasks (defined by distinct input-output mappings) and presents a unified training framework for future visual-task research and applications.
[CV-3] PrismRestore: A Compact Restoration Toolbox for Agent ic Image Restoration
链接: https://arxiv.org/abs/2611.24915
作者: Zhaocheng Yu,Kui Jiang,Jing Cao,Yiang Chen,Xiaocheng Feng,Junjun Jiang,Xianming Liu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Agentic image restoration composes operations to handle mixed degradations, yet its toolbox often requires storing multiple restoration networks. We propose PrismRestore, which constructs a compact bank of callable restoration actions from a single frozen all-in-one model. Our approach builds on task-dependent yet overlapping channel responses in the shared model. An offline task-prior search uses activation and gradient statistics to group channels with similar response profiles, then searches task-specific modulation coefficients over non-representative channels while leaving representative channels unmodulated. The resulting prior bank enables explicit action selection and composition without additional restoration backbones or task-specific adapter modules. A state-conditioned policy learns to compose these actions using Group Relative Policy Optimization. Its objective separately normalizes fidelity and perceptual feedback within groups of trajectories sampled for the same input, and combines terminal evaluation with return-to-go derived from stepwise quality changes. Experiments on synthetic composite-degradation and real-world multi-task benchmarks show improvements over the evaluated all-in-one baselines and restoration quality competitive with multi-model agents. PrismRestore achieves these results with a single restoration backbone and faster inference than the evaluated agentic baselines.
[CV-4] ShadowTransfer: A Geographic Transfer Benchmark for Overhead Shadow Detection NEURIPS2026
链接: https://arxiv.org/abs/2611.24904
作者: Shagun Mittal,Raymond A. Yeh,Satish Ukkusuri
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted at NeurIPS 2026, Track on Evaluations and Datasets. 53 pages. Project page: this https URL . Code: this https URL . Data: this https URL (DOI https://doi.org/10.57967/hf/11483 )
Abstract:Overhead aerial imagery feeds dense-prediction pipelines that are routinely trained in one geographic region and deployed in another, yet no benchmark measures how shadow detection models transfer across regions. Existing overhead shadow datasets either lack a geographic transfer protocol or rely on automatically generated labels. We introduce ShadowTransfer, a benchmark of 4,500 human-verified shadow masks across three U.S. cities at two native resolutions, paired with a leave-one-city-out evaluation protocol. Under vanilla training, holding out the test city costs 3.5 to 7.8 mIoU across four architectures, the gap remains significant when the training set grows from 225 to 900 images, and six of eight adaptation methods degrade more than vanilla training on average. We propose a three-probe framework that diagnoses where and how the transfer breaks: the failure operates in feature space, beyond the reach of pixel-histogram matching, shows up as a confidence collapse on shadow pixels while background confidence stays intact, and is recoverable by retraining different architectural components depending on the model: encoder for some, decoder for others. These findings replicate on INRIA building footprints, suggesting they describe geographic transfer beyond shadows. Guided by the diagnosis, we build Spectral Information Bottleneck (SIB), a plug-in module with one component per finding. Across the 9 (architecture, held-out city) cells, SIB has the smallest mean gap to the within-city upper bound of the ten methods we evaluate (6.42 mIoU), level with FDA. The benchmark, LOCO protocol, code, and trained models are released to support reproducible evaluation of geographic transfer in overhead dense prediction.
[CV-5] NereidVLA: Learning to Understand and Navigate the Underwater World
链接: https://arxiv.org/abs/2611.24864
作者: Chenjie Zhao,Zhiliang Zhu,Quan Chen,Yuquan Xue,Xingxue Shi,Puyue Wang,Zhiqiang Zhao,Kangye Zheng,Boyang Xu,Mingwei Lin,Dejun Li
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注: 11 pages, 4 figures
Abstract:Underwater navigation requires translating visual spatial cues into motion, yet marine vision-language datasets rarely include paired robot trajectories. We present NereidVLA, a vision-language-action model that transfers marine spatial knowledge to language-conditioned 3D navigation. We first train Sea2Know on approximately two million marine examples to learn target grounding, spatial relations, and scene structure without paired robot-motion supervision. We then adapt it using expert trajectories executed under simulated underwater vehicle dynamics. Given RGB histories and a language instruction, NereidVLA predicts discrete motion tokens that decode into local 3D trajectories for closed-loop tracking. Evaluation combines an independent six-dimensional underwater navigation understanding benchmark with offline prediction and closed-loop navigation. Under matched navigation training and execution settings, marine spatial pretraining increases simulation success from 72.4% to 84.4%. Without fine-tuning on the real test environment, NereidVLA achieves 78.75% success in laboratory-pool robot trials, compared with 47.50% for InternVLA-N1. These results show that marine spatial supervision can support downstream motion learning and physical underwater navigation.
[CV-6] Gaspard: Expressive unified speech and gesture synthesis with continuous autoregressive flow-matching and decoupled classifier-free guidance
链接: https://arxiv.org/abs/2611.24862
作者: Téo Guichoux,Théodor Lemerle,Catherine Pelachaud,Nicolas Obin
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Preprint - under review
Abstract:Human communication involves highly coordinated multimodal signals such as speech and co-speech gestures. Existing approaches for generating both modalities commonly follow a cascaded pipeline, where speech is generated first and subsequently used to drive gesture generation; speech generation is therefore independent of gestures. We introduce Gaspard, a unified speech–gesture generation model based on continuous autoregressive generation with next-step flow-matching predictions, and trained on Seamless Interaction, a large-scale multimodal dataset. The heterogeneous temporal properties of speech and gesture motivate modality-specific prediction heads and decoupled classifier-free guidance over the two generation streams. This guidance rebalances the influence of speech and gesture at inference time, improving expressivity while mitigating dynamics collapse in continuous autoregressive gesture generation. Objective and perceptual evaluations show that Gaspard is a strong gesture generation model against unimodal baselines and that it produces highly synchronous speech and gestures compared to a matched cascaded system.
[CV-7] Does the Action Matter? Probing Action-Conditioned Semantic Forecasting as a World Model for Autonomous Driving WACV2027
链接: https://arxiv.org/abs/2611.24840
作者: Binay Kumar Singh,Niels Da Vitoria Lobo
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO)
备注: 8 pages, 3 figures, 6 tables. Accepted at the WAM-AD Workshop, WACV 2027. Code: this https URL
Abstract:World models are increasingly used in autonomous driving to predict how a scene evolves under the ego vehicle’s actions, supporting counterfactual reasoning and model-based planning. However, existing evaluations focus on prediction fidelity and planning performance, and rarely verify that the predictions depend on the conditioning action. In this paper, we study action dependence in action-conditioned semantic forecasting, which predicts future semantic segmentation from past segmentations and the ego action. We compare a forecaster with feature-wise and input-level action conditioning against an identical model without action input, and measure action dependence with test-time action interventions scored by standard segmentation metrics. The evaluation is validated in a synthetic environment with known action-dependent dynamics and applied to Cityscapes and BDD100K, both within each dataset and across the two vehicle platforms. Experiments show that the executed action improves mid-term mIoU by at most 2.5 points on Cityscapes and 0.6 points on BDD100K, whereas a wrong action reduces it by up to 23.2 and 7.0 points, below the model without action input. On real data, feature-wise conditioning depends 1.6 times more on the action than input-level conditioning at similar accuracy, whereas both depend on it equally in the synthetic environment, where the history does not determine the action. These results show that an accuracy gain indicates that a driving world model uses its action, but not how strongly its forecasts depend on it. Code is available at this https URL.
[CV-8] Synthesizing hyperspectral Data using generative Models to train spectral Unmixing Methods for low-cost Crop Residue Cover Mapping
链接: https://arxiv.org/abs/2611.24827
作者: Ege Artan,Shuo Chen,Andrew Peng,Sammuel Aldrich Karya,Kyaw Thiha
类目: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Signal Processing (eess.SP)
备注: 8 pages, 3 figures. Published in the XXV ISPRS Congress 2026
Abstract:The spectral range of the Field Imaging Nanosatellite for Crop residue Hyperspectral Mapping (FINCH), developed by the University of Toronto Aerospace Team’s Space System Division, poses significant challenges for hyperspectral unmixing to determine crop residue cover fractional abundances. The severe lack of standard indices necessitates the use of complex, data-driven unmixing models. Complex unmixing models require dense, well-distributed manifolds, whereas ground-truth datasets are sparse and expensive. To better leverage the information content in existing ground-truth datasets, this study presents a framework that decouples interpolation and abundance-mapping estimation via an intermediary conditional data generator, contrasting with traditional single-model unmixing pipelines. This decoupling enables the inclusion of physical prior knowledge about spectra, which is not natively accessible to unmixing models, thereby artificially expanding the dataset to serve as a Monte Carlo approximation of the population risk. To test this hypothesis, we have proposed two generator models: a 1D Conditional U-Net with Conformer Layers (GD-Streamline) and a Dual-Path Transformer Conditional Variational AutoEncoder (TCVAE), and two unmixing models: Multi-Layered Perceptrons (MLP) and Fourier Neural Operators (FNO). The results of the study indicate a need for rigorous integration and introduction of spectral priors within the scope of the proposed decoupling framework to prevent domain and mode collapse and hallucination by the generator models; otherwise, this leads to falsely approximated data manifolds, ultimately resulting in unsatisfactory and out-of-distribution unmixing performance.
[CV-9] ReCoCo: From Patches to Compositions for Visual Representation Learning
链接: https://arxiv.org/abs/2611.24806
作者: Juan Gutiérrez,Saman Zadtootaghaj,José Luis Blanco-Murillo,Sara Atito
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Self-supervised learning has produced powerful visual representations, yet learning strong dense features remains challenging. While consistency between augmented views provides strong image-level supervision, extending this principle to local visual content is less straightforward. Existing approaches often match individual patches across views or identify coherent regions, introducing uncertainties in correspondence estimation and assumptions about how visual content should be grouped. We introduce ReCoCo (Regional Composition Consistency), a principled approach that extends cross-view consistency to the regional level without requiring such correspondences or semantic grouping. ReCoCo represents the content shared between augmented views through compositions of their patch-level predictions over learned visual prototypes. By aligning these compositions, the model learns from regions containing multiple visual concepts without requiring them to be separated beforehand. ReCoCo integrates naturally with teacher-student self-distillation and introduces no additional learned parameters. Extensive experiments demonstrate substantial improvements in frozen-feature segmentation, clustering, and regional retrieval while maintaining competitive image-level recognition, alleviating the trade-off between dense and global representation quality.
[CV-10] Edge-Efficient Joint RAW Image Compression and Restoration for Smart Wearable Devices
链接: https://arxiv.org/abs/2611.24805
作者: Yunong Sun,Sicheng Chen,Fei Xia
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Smart wearable devices, such as augmented reality glasses, increasingly rely on cameras to understand and interact with their surroundings. However, their compact sensors often capture noisy RAW images, while limited compute and power make high-quality on-device restoration difficult. This creates an emerging need to reduce both local computation and transmission cost without sacrificing image quality. Two common solutions are to offload restoration or to compress the raw data before transmission. The first makes RAW transmission costly; the second reduces the payload but can spend bitrate preserving sensor noise. We address this fundamental mismatch with an asymmetric framework for joint RAW compression and restoration that directly optimizes the transmitted representation for clean image reconstruction. A lightweight on-device encoder compresses noisy RAW measurements into a bitstream, while a higher-capacity backend decoder jointly denoises and demosaics the encoded representation. We use knowledge distillation to train a smaller encoder for tighter compute budgets. Experiments on real and synthetic data show improved rate-restoration performance over strong cascaded compression-restoration pipelines and learned compression baselines. On SIDD, our method improves PSNR by more than 1.5 dB over cascaded baselines at approximately 0.25 bits per pixel. We further validate the method on real 12 MP RAW images captured with commercial smart glasses and measure sender-side latency on both smartphone and smart-glasses hardware. The distilled encoder achieves a 27.6 \times NPU inference speedup over on-device NAFNet for 256 \times 256 output patches. The code is available at this https URL
[CV-11] Ultimate3D: Teaching Multimodal LLM s 3D Camera-Object Relations by Large-Scale Visual Instruction Data
链接: https://arxiv.org/abs/2611.24795
作者: Liu He,Yizhi Song,Ruiqi Xu,Baris Sarper Tezcan,Xiao Zeng,Albert Y.C. Chen,Lu Xia,Shashwat Verma,Sankalp Dayal,Min Sun,Cheng-Hao Kuo,Daniel Aliaga
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted to Machine Vision and Applications 2027
Abstract:Multimodal Large Language Models (MLLMs) struggle with accurately capturing camera-object relations, especially for object orientation, camera viewpoint, and camera shots. This stems from the fact that existing MLLMs are trained on images with limited diverse camera-object relations and corresponding textual descriptions. To address this, we propose a synthetic generation pipeline to create large-scale 3D visual instruction datasets. Our framework takes 3D assets as input and uses rendering and diffusion-based image generation models to create photorealistic images preserving precise camera-object relations. Additionally, large language models (LLMs) are used to generate text prompts for guiding visual instruction tuning and controlling image generation. We create Ultimate3D, a large-scale 3D-grounded 2D visual instruction dataset. Ultimate3D consists of 240K VQAs with precise camera-object annotations, and corresponding benchmark. MLLMs fine-tuned on our proposed dataset outperform commercial models by a large margin, achieving an average accuracy improvement of 29.7% on camera-object relation recognition tasks. Our code, dataset, and benchmark will contribute to broad MLLM applications.
[CV-12] raining-free motion statistics match self-supervised behavior encoders
链接: https://arxiv.org/abs/2611.24792
作者: Ziwen Chen
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: 10 pages, 7 tables, 1 figure. Code will be released with the conference version
Abstract:Self-supervised encoders for animal behavior are often trained on, or fed, hand-built movement features. We ask whether they beat those ingredients when compared fairly. A training-free representation, made of generic per-frame measurements summarized over fixed time windows and reduced with PCA, is compared with retrained and released encoders (BAMS, hBehaveMAE) under a preregistered protocol: frozen representations of equal size and one shared classifier. On CalMS21 it reaches 85.8 macro F1 against 82.4 for the best encoder, ties with four labelled videos, and leads when the annotator changes or new behaviors must be learned from a few videos; the reading holds for encoders retrained with three seeds. On the MABe22 mouse triplets it scores 83.7 against 80.3 for the released hBehaveMAE (gap +3.4, 95% interval +2.1 to +4.7). On a second species, the MABe22 fly groups, the same recipe scores 51.6 mean F1 over 50 tasks against 48.2 for BAMS, the best published method. On Shot7M2, the synthetic basketball benchmark built by hBehaveMAE’s authors, it scores 68.8 against 64.7 for hBehaveMAE, the best method in that paper, though hBehaveMAE remains better on the shortest movements. Short time windows carry most of the result. Raw keypoints in each body’s own frame, summarized over the same windows, already match or beat the encoders on both mouse datasets and on Shot7M2; on the fly groups they fall short, and the measurements add 5.3 points. On these benchmarks, current behavior encoders have yet to beat the hand-built features they learn from, and such baselines should be reported alongside them.
[CV-13] Read Together Answer Apart: How Earlier Questions Help Vision-Language Models Look
链接: https://arxiv.org/abs/2611.24785
作者: Xinyi Guo,Kaile Huang,Xitong Ling,Mingyi He,Chende Zheng,Jianyu Xu,Chao He,Dongqing Song
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Vision-language models (VLMs) are often asked several questions about the same image in a single prompt (co-asking), yet existing evaluations report conflicting accuracy gains and hallucinations at the prompt level without tracking how individual answers change. By pairing each question’s single-question and co-asked answers across 22 open-weight VLMs (1B-35B), we find that erratic net shifts reflect two opposing effects: on Densely Captioned Images (DCI), every model repairs 15.4%-29.8% of its single-question errors under co-asking, while the share of correct answers turned wrong falls from 30.8% on weaker models to 4.7% on stronger ones. Across all 14 models supporting custom attention masks, we show through attention readouts and layer-window interventions that earlier same-image questions read visual information into their question tokens in the early-to-middle layers during prefill, helping later questions concentrate image attention onto their queried targets. This visual help accumulates across the scene even across non-overlapping regions, whereas blocking earlier questions from attending to the image lowers both target localization and accuracy in all 14 models. While cross-question help is established during shared question reading, part of the co-asking damage enters later during answer decoding when the decoder attends to other question stems. We therefore introduce Read Together, Answer Apart, a training-free decoding mask that keeps shared prefill intact and isolates each answer from other stems, improving co-asking accuracy on 12 of 14 models on DCI (up to +4.71 percentage points, closing 70% of the pooled gap to asking alone) and on 10, 8, and 5 of the 14 models on WebUI, FinTabNet, and GQA (up to +11.4, +5.3, and +1.1 points).
[CV-14] Counterfactual World Models: JEPAs That Reason About “What If”
链接: https://arxiv.org/abs/2611.24778
作者: Randall Balestriero,Yann LeCun,Kerda Varaku
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO); Machine Learning (stat.ML)
备注:
Abstract:World models operate in a world they cannot fully observe. When a hidden disturbance, such as a current pushing a boat, changes the outcome of an action, a counterfactual question follows: what would have happened under a different action, subject to the same disturbance? Its answer separates bad decisions from bad luck. We prove that world models trained on factual transitions cannot answer it on their own, whatever their data or capacity. For many physical systems, the observed outcome and a simple property of the dynamics suffice: natura non facit saltus, nature makes no leaps. When actions and disturbances act as separate forces, their interaction is second-order in time, so the counterfactual outcome is, to first order, the observed outcome plus the difference between the world model’s predictions under the alternative and factual actions. This yields a counterfactual world model with no counterfactual data, extra parameters, or retraining. For transitions of duration h , we prove an O(h^2) bound on the counterfactual error relative to the disturbance variance, establish a matching minimax lower bound from unpaired factual data, and characterize failures at discontinuities, coordinate folds, and disturbances that scale the command. We validate these predictions from pixels with JEPA world models on frozen DINOv2 and masked autoencoder (MAE) encoders. On four rendered simulators, our approach improves counterfactual R^2 from 38 – 46% to 86 – 96% with the same world models; on a fifth, an underwater vehicle, it reaches 90 – 94% with sixteen times more factual data. Four further frozen encoders, DINOv3, SigLIP~2, CLIP and EVA-02, give 81 – 95% on the same simulators. In hindsight planning, it finds the action that would have reached the goal in 35 – 69% of events, against 14 – 23% for an oracle planner that knows the simulator but not the outcome.
[CV-15] A3Bench: Compositional Reasoning over Aggressive Interactions in Videos
链接: https://arxiv.org/abs/2611.24762
作者: Austin C Baker,Raiyaan Abdullah,Daniel Zeev Yoffe,Shruti Vyas,Yogesh S Rawat
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Recent Multimodal Large Language Models (MLLMs) have achieved strong performance on general video understanding benchmarks, yet their ability to reason about participant roles in aggressive interactions remains largely unexplored. Existing violence and crime datasets primarily evaluate whether aggression occurs, without testing whether models can identify who performed the action, who received it, and how participants are related within the scene. To address this gap, we introduce A ^3 Bench, a benchmark to study aggressive action and association in videos. The benchmark evaluates whether models can jointly recognize aggressive actions, distinguish aggressors from victims and bystanders, and correctly bind actions to participants under challenging distractor settings. We benchmark 17 multimodal models and find that they struggle substantially on fine-grained aggressive-scene understanding. Performance degrades consistently as reasoning complexity increases, with models frequently confusing aggressors with victims or bystanders, revealing a systematic failure in role-action association rather than coarse aggression recognition alone. We further observe that increasing model scale or generic reasoning models experience limited improvement. Finally, we explore a simple role-graph prompting strategy that encourages structured reasoning over participant interactions, leading to improvements in compositional reasoning. Our findings highlight that reliable aggressive-scene understanding remains a major challenge for current MLLMs and motivate future research on role-aware video reasoning beyond coarse violence detection. Code and dataset: this https URL.
[CV-16] Jeveration: System One Models Now Generate
链接: https://arxiv.org/abs/2611.24738
作者: Yuhe Liu,Qi Li,Shihua Zhang,Xinchao Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:System One models such as Jev return decisions through fast, parallel prediction. In this paper, we introduce Jeveration, showing that the same decision paradigm can also generate high-quality images in a single forward pass. Jeveration builds on a frozen vision-language model, repurposing its visual-semantic knowledge for holistic image generation. The key challenge is to maintain global consistency. A single prompt admits many valid images, yet parallel predictions must converge on a coherent whole. We therefore condition all positions on a shared random state to coordinate parallel predictions, so that each draw yields a different but coherent image. Since the model tends to ignore this state under direct image supervision, we first learn a distribution over discrete image codes to guide single-pass generation. Beyond text-to-image generation, we further unify image-to-image generation and dense prediction within the same architecture, so that one checkpoint supports all three tasks, each in one forward pass. Experiments show that Jeveration generates a 512x512 image in 78 ms, up to 122x faster than existing unified models, while maintaining competitive image quality and dense prediction performance.
[CV-17] HandWorldBench: A Unified Benchmark for Hand-Centric World Models
链接: https://arxiv.org/abs/2611.24731
作者: Zhaolu Wang,Jiacheng Xu,Lihong Wang,Shuqing Wang,Jianjie Fang,Ziyou Wang,Xvyuan Liu,Rongze Tang,Yiding Ma,Xin Zhang,Jiawei Chen,Wei Wu,Chen Gao,Yong Li,Zhibo Chen
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Recent advances in world models are opening new possibilities for fine-grained embodied tasks, with hand-object interaction offering a particularly demanding setting. Modeling these interactions involves capturing how hand actions change objects and how the resulting states persist over time. Yet systematic evaluation tailored to these hand-centric capabilities remains underdeveloped. To address this gap, we introduce HandWorldBench, a unified benchmark that systematically evaluates hand-centric world models across human hands, simulated dexterous hands, and real dexterous hands along six dimensions: visual quality, hand fidelity, appearance consistency, instruction following, physical consistency, and temporal coherence. To construct the benchmark, we develop a data curation pipeline that extracts hand-object interaction clips and screens them for quality, yielding 20,500 high-quality egocentric clips from over 600 hours of footage. We then manually select 500 clips covering seven task families for evaluation. Benchmarking eleven models under a unified protocol reveals that stronger hand-pose control does not consistently translate into coherent object-state evolution and persistence or robust cross-embodiment performance, with several specialized hand-control models underperforming general-purpose text/image-to-video generators on dexterous-hand tasks.
[CV-18] Generalization of Text Line Segmentation for HTR in Historical Documents ICDAR
链接: https://arxiv.org/abs/2611.24726
作者: Gayan H. Pathirage,Stephan M. Unter,Simon Corbillé,Elisa H. Barney Smith
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: This work is going to be published in the proceedings for the Historical Image Processing workshop 2026 (HIP 2026), connected to the International Conference of Document Analysis and Recognition (ICDAR)
Abstract:In this paper we investigate the generalization capability of state-of-the-art object detection models (Hi-SAM, YOLO, and RT-DETR) for the automatic segmentation of text lines in historical documents. Generalization is needed because it is not practical for museums and archives to produce large amounts of labeled samples for each new collection. We also evaluate the few-shot setting by fine-tuning the networks with only a small number of training samples per target dataset. To this end, we use nine different datasets that differ significantly in visual layout and language. Additionally, we discuss why mean average precision (mAP), a classic evaluation metric for object detection, is insufficient for assessing the quality of a text line segmentation model. Depending on the dataset, our CRNN models achieve character error rates (CER) ranging from 2.48% to 7.07% per page for the segmented text lines. This is sufficient for reading comprehension and simple text searches. Finally, we quantify the resource and time requirements of training and inference in different use case settings. This forms the basis for more informed decisions about model and hardware choices based on available resources.
[CV-19] Multi-Band Galaxy Novel-View Synthesis with Radiative Gaussian Fields
链接: https://arxiv.org/abs/2611.24723
作者: Yuzhe Wu,Jialin Zhu,Doğa Yılmaz,Daisuke Kawata,He Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages, 3 figures
Abstract:We propose a novel view synthesis tool for galaxies based on Gaussian Splatting (GS). Unlike RGB data, where GS has achieved success, galaxy observations consist of multi-band signals from a massive number of bodies, contain complex patterns of cross-band interactions, and capture light propagation in highly heterogeneous media. As a result, the underlying physical process poses great challenges in model representation and optimization, which have not been investigated in existing research. To this end, we propose a physically inspired Gaussian Splatting method, GalaxyGS, which is equipped with physically meaningful components that capture both the energy emission mechanism of different astronomical bodies and the light propagation between them. In the absence of directly comparable baseline methods, we adopt a wide range of possible alternative approaches. Comparisons conclusively demonstrate that GalaxyGS is better in accuracy and physical fidelity across datasets and metrics.
[CV-20] Can Omni-modal LLM s Understand Spoken Intentions?
链接: https://arxiv.org/abs/2611.24706
作者: Zijie Xin,Tianyi Wang,Daoze Zhang,Jie Yang,Changli Tang,Qijie Wei,Fengyun Rao,Xirong Li
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: project page this https URL
Abstract:Omni-modal large language models (omni-modal LLMs) can respond to spoken requests directly through voice and video calls. Such requests rely on the scene shared by the user and the assistant, so the same utterance may call for different help or for no response. Most audio-visual benchmarks provide questions as separate text input, and interactive benchmarks mostly score a response as a whole. It thus remains unclear whether a model recognizes when the user is asking for help, and whether its reply meets the user’s need once the request is recognized. We propose O-IN-Bench to evaluate how well omni-modal LLMs understand spoken intentions. A model watches a complete video with native speech and outputs one text response, or [SILENCE] when no response is needed. O-IN-Bench contains 1,000 Chinese and English samples in five categories and 16 subcategories. Contrastive pairs combine the same request with different scene conditions to test whether the response changes with the context. A sample-specific rubric judges language, intent, and content separately, and O-IN-Engine verifies each rubric against the video. Across 30 omni-modal model configurations, the best overall score is 59.6, achieved by Gemini3.7-Flash. Gemini3.7-Flash passes 74.4% of the intent judgments but only 42.6% of the content judgments. When the speech is a conversation between peers rather than a request to the assistant, even strong models often keep responding. When a reply must satisfy several people, the best score is only 25.9. These results indicate that recognizing a request and meeting its requirements are separate abilities, and O-IN-Bench measures each of them.
[CV-21] Improving Zero-shot 3D Semantic Mapping in Offroad with Geometric Priors
链接: https://arxiv.org/abs/2611.24696
作者: Eerik Alamikkotervo,Risto Ojala
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Semantic 3D mapping is a powerful representation that enables platform agnostic mobile robot navigation in challenging offroad conditions. Current vision foundation models can accurately predict semantics zero-shot on structured scenes but performance is compromised in natural environments. On the other hand, 3D point clouds encode valuable information for discriminating semantics in scenarios where vision foundation models fail. In this work, we propose fitting a sphericity-elevation distribution for each semantic class to act as geometric anchor. Only the points with the most confident vision foundation model predictions are considered when fitting to produce high quality geometric prior distributions. The geometric prior is fused with the vision foundation model prediction to produce the final semantic mapping. The proposed prior improves SAM3 baseline by +6.3 in mIoU and +8.9 in accuracy on the RELLIS-3D dataset, and by +1.7 in mIoU and +2.1 in accuracy on the WildScenes dataset. The source code will be made public upon paper acceptance.
[CV-22] DrawScore: Evaluating Generated CAD Models against Engineering Drawings
链接: https://arxiv.org/abs/2611.24681
作者: Haorui Chen,Shiteng Lu,Zhanyu Guo,Zaixi Li,Yunhe Ni,Yaosong Du,Yichen Xiang,Xiande Huang,Rui Huang
类目: Computer Vision and Pattern Recognition (cs.CV); Software Engineering (cs.SE)
备注:
Abstract:Automatic CAD generation is commonly evaluated with volumetric IoU, Chamfer Distance, and surface F-score, which barely penalize local errors that occupy only a small fraction of a part’s volume or surface area, such as a misplaced hole or a slot on the wrong face, even though such errors are critical to manufacturing. We formalize these metrics in a unified integral form and show that the weight of a local error decays as the (k+q) -th power of its size, where k is the dimension of the integration domain and q is determined by the penalty function. Guided by this analysis, we propose DrawScore, which projects both models into three orthographic views with visible and hidden lines, as in engineering drawings, and applies a saturating penalty along the projected curves, so that the weight of a local error decays only linearly. Controlled experiments confirm the predicted decay orders, and on outputs of existing CAD generation agents, DrawScore exposes errors in outputs that IoU rates as nearly perfect. DrawScore further improves generation as a reinforcement learning reward and, for CAD generation from engineering drawings, serves as a test-time verifier without 3D ground truth or an additional reward model. Our code is available at this https URL.
[CV-23] Predicting Species Splits: A Challenging Fine-Grained Benchmark for Category Discovery NEURIPS2026
链接: https://arxiv.org/abs/2611.24672
作者: Christian Lange,Joakim Bruslund Haurum,Bingchen Zhao,Oisin Mac Aodha
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted at NeurIPS 2026 (Evaluations and Datasets Track)
Abstract:Widely used category discovery benchmarks have become increasingly saturated and often fail to reflect real-world discovery settings, making long-term evaluation of new methods difficult. We introduce PS-SPLIT, a large-scale fine-grained benchmark for category discovery, motivated by taxonomic splitting: the real-world process by which what was once considered a single species is found to comprise multiple distinct ones. PS-SPLIT contains 79K globally distributed, high-quality, community-collected bird images spanning 771 species, derived from 257 recent real expert-verified taxonomic splits, with associated geographic metadata and full taxonomic hierarchy. The visual differences between categories arising from real taxonomic splits can be very subtle, often requiring significant human expertise to tell apart, making PS-SPLIT a challenging benchmark which we hope will assist in advancing fine-grained category discovery research. Based on taxonomic splitting, we introduce the task of discovery by category splitting, where the goal is to determine which known categories should be subdivided into new finer-grained subcategories. We adapt four generalized category discovery methods to this setting, evaluate existing methods, and show that there is substantial room for future progress.
[CV-24] AggCLIP: Aggregating CLIP for Zero-Shot Multimodal Anomaly Detection
链接: https://arxiv.org/abs/2611.24643
作者: Zhi Li,Hanyu Wang,Fei Shen,Zhengtao Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 13 pages, 4 figures, 6 tables. Author version of the ACM Multimedia 2026 paper
Abstract:Zero-Shot Multimodal Anomaly Detection (ZSMAD) aims to identify both appearance and geometric defects in novel categories without relying on target-specific training data, making it especially valuable in real-world scenarios constrained by data scarcity, privacy concerns, or high annotation costs. While recent approaches successfully adapt vision-language models like CLIP to achieve this open-vocabulary generalization, they still face significant challenges: (i) severe geometric information loss during 3D-to-2D point cloud rendering; (ii) spatially noisy 2D feature maps produced by raw CLIP patch embeddings, compounded by shallow late-stage modality fusion; and (iii) severe cross-modal semantic misalignment due to decoupled processing streams. To address these limitations, we propose AggCLIP, a novel framework that adapts frozen CLIP representations for ZSMAD by explicitly incorporating local geometric structure and visual spatial context. Specifically, to accurately localize fine-grained defects, we design a Geometry-Aware Spatial Aggregation (GSA) module that refines back-projected 3D point features via density-adaptive neighborhood weighting, alongside a Context-Aware Spatial Aggregation (CSA) module that smooths multi-scale 2D patch embeddings using a lightweight trainable adapter. Furthermore, to prevent semantic drift between modalities, we introduce a Dual-Stream Prompt Contrastive (DSPC) loss, which aligns the abstract concepts of normality and abnormality across the point cloud and RGB streams. Extensive experiments demonstrate that the proposed AggCLIP framework achieves state-of-the-art performance on the MVTec 3D-AD and Eyecandies benchmarks under both one-vs-rest and cross-dataset zero-shot settings, notably surpassing the strongest competitive methods by up to 2.9% in pixel-level AUPRO.
[CV-25] Hypnos: A Dual-Phase End-to-End Platform for Scalable Continual Learning at the Edge
链接: https://arxiv.org/abs/2611.24637
作者: Guilherme Guedes,Marco Apolinario Lainez,Charlotte Frenkel
类目: Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Continual learning (CL) enables neural networks to adapt to non-stationary data streams by mitigating catastrophic forgetting, offering a path to robust, private, and adaptable edge devices. Yet, porting CL to real-world edge devices remains challenging for two reasons: (i) online operation, where sensor data arrives one sample at a time and the device must adapt on-the-fly, and (ii) the low footprint of edge devices, which makes traditional error backpropagation training prohibitive. Inspired by the brain’s wake and sleep cycles, the former for rapid adaptation with minimal resources and the latter for long-term consolidation, we introduce Hypnos, an open-source dual-phase platform for efficient CL at the edge. In the wake phase (e.g., daily operation), Hypnos relies on a low-footprint system-on-chip with a RISC-V microcontroller unit (MCU) targeting efficient inference and supporting lightweight gradient-free prototype updates. In the sleep phase (e.g., night-time charging), where wall-plug energy and larger memory budgets are available, a smart charger performs rehearsal-based gradient updates for consolidation. Deployed on a ZCU104 field-programmable gate array (FPGA) and synthesized in a 28-nm process, Hypnos is benchmarked on the CORe50 NICv2-391 protocol. It sets a new state-of-the-art accuracy of 78.6%-81.6% for edge CL systems, with an on-device latency of 7.79s-10.54s per sample depending on activation precision, and requires 3.8\times fewer learning steps and 22\times less learning compute than quantized latent replay at higher accuracy and comparable latency.
[CV-26] H3Remover: Reinforcing Video Object Removal without Paired Data
链接: https://arxiv.org/abs/2611.24633
作者: Kangjie Chen,Xiangyu Li,Jinhao Deng,Dongbin Zhang,Chaoda Zheng,Shijia Chen,Yuqi Zhang,Choo Sin Wai,Boyi Sun,Puyi Wang,Yu Zhang,Xianming Liu,Boyang Wang
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: Project page: this https URL , code at: this https URL
Abstract:Pretrained video editors possess rich removal and completion priors, yet may retain targets or alter unrelated content. We present H3Remover, which reinforces MiniMax-H3 Base Ref2VA for video object removal without paired object-free training targets. To discourage superficial edits such as recoloring, a target-removal reward favors calibrated depth changes consistent with revealing farther content. To limit unintended scene changes, a background-geometry reward encourages preservation of relative depth ordering. To preserve background appearance without retaining unwanted shadows or reflections, an appearance reward constrains RGB changes in source-derived protected regions while exempting targets and estimated effects. Frozen HPSv3 complements these constraints with perceptual-quality feedback for synthesized content. Evaluation regions remain fixed across candidates, and DiffusionART optimizes LoRA adapters with the backbone frozen. Experiments show substantial reconstruction gains over original H3, with qualitative comparisons demonstrating more effective object removal and better preservation of surrounding scene content.
[CV-27] EgoJev: Finding Evidence in Egocentric Video by Reading VLM Decisions
链接: https://arxiv.org/abs/2611.24616
作者: Chunggi Lee,Hanspeter Pfister
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Questions about long egocentric video often depend on one brief moment, such as the second a hand puts an object down. A vision-language model (VLM) that sees a few evenly spaced frames usually misses it. Training-free methods rank frames with a separate image-text model or ask the VLM to write a time. In a controlled test, the VLM itself locates the moment better when it chooses among labeled frames, and we read this choice from its output distribution. With the same candidates, this choice lands on the evidence more often than image-text scores or a written time span, and it compares all candidates in one decision. Based on this finding, we propose EgoJev, a training-free frame selector for wearable assistants. EgoJev reads where the evidence is from labeled thumbnails and looks again at the windows it points to. It places its few answer frames at the quantiles of the resulting distribution. One frozen VLM selects, answers and returns the evidence time, with no second model. A wearable device can encode its coarse thumbnails while it records. With Qwen3.5-4B and 8 answer frames, EgoJev raises QAEgo4D accuracy from 51.2% with uniform sampling to 62.0%, and it outperforms recent training-free frame selectors on QAEgo4D and MyEgo. A single-level version runs on a Mac and an iPhone.
[CV-28] DAL: Dynamic Angular Loss for Imbalanced Medical Image Classification BMVC
链接: https://arxiv.org/abs/2611.24597
作者: Salman Mohammad,Furkan Kasım,Manuel Günther
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted at the British Machine Vision Conference (BMVC) 2026. 18 pages, 5 figures, 4 tables. Code: this https URL
Abstract:Class imbalance remains a major obstacle in medical image classification, where rare but clinically important classes are often overshadowed by majority categories. Although current strategies for reducing bias, including angle-based softmax (ABS) losses, offer strong discriminative features, their behavior in settings with a small number of severely imbalanced classes remains insufficiently understood. Previous approaches that estimate the intrinsic parameters of ABS rely on the assumption of many classes, which does not hold in medical settings. We derive data-driven upper and lower bounds for the scale parameter and an upper bound for the angular margin using negative-angle statistics, and use these to drive a batch-adaptive angular margin (BAAM), which we combine with dynamically-weighted cross entropy (DWCE) to define our dynamic angular loss (DAL). Across four highly imbalanced medical image classification benchmarks spanning dermatology, ophthalmology, and gastrointestinal endoscopy, our method consistently achieves superior balanced accuracy and the lowest coefficient of variation of per-class recall. Although this entails a small performance reduction on majority classes, the gains on minority categories are substantial, yielding a reliable and equitable overall classifier. Our source code is available at: this https URL.
[CV-29] ALIVE: Bring Foundation Features to Life with Events
链接: https://arxiv.org/abs/2611.24579
作者: Rong Zou,Haiqing Hao,Davide Scaramuzza
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注:
Abstract:Visual foundation models provide powerful representations for diverse tasks, but their features become stale as visual content changes between image observations. Refreshing them requires new frames and repeated runs of a computationally expensive image encoder. We introduce ALIVE, an event-based method that incrementally updates native foundation model features between image observations. Given an anchor image, ALIVE uses subsequent events to transport and correct existing multi-layer features extracted by a frozen DINOv3 encoder. Its lightweight updater is trained end-to-end using only synthetic data, without downstream task supervision, and transfers directly to real-world event streams without any adaptation. Across three real-world datasets and five task-dataset settings spanning object detection, semantic segmentation, and depth estimation, ALIVE outperforms all evaluated event-based methods in every metric. On DSEC, when the anchor frame is 200,ms old, ALIVE improves over anchor-image predictions by 30.9,mAP in detection, 11.6,mIoU in segmentation, and 15.9% in depth AbsRel. Compared with image re-encoding, ALIVE reduces FLOPs by 82.4% and latency by a factor of 2.5.
[CV-30] BaseRSI: Recursive Self-Improvement of a Base Vision-Language Model via Training-Free Harness Internalization
链接: https://arxiv.org/abs/2611.24578
作者: Mingda Zhang,Yuanhao Dong,Qiang Huang,Rui Qian,Yanjin Li,Zhenyu Liu,Xiao Huang,Zihao Yang,Jian Wang,Pinjia He,Xiaoying Tang
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Recursive self-improvement (RSI) is moving from the harness of prompts and tools around a base model toward the model itself. However, existing RSI still faces key challenges, including harness-bound gains that stay outside the base model, black-box supervision that reads only the model’s outputs, and imprecise edits whose site and strength are fixed in advance. To address these challenges, we propose BaseRSI, a new paradigm of Recursive Harness Internalization: the base model improves its harness, then writes what the harness adds into its own weights through training-free edits, round after round on its own responses. To support this paradigm, we introduce ReGap, a mechanism that supervises every transformer block with the high-dimensional activation difference that the harness makes at that block alone, and updates the block’s weights along this signal in a low-dimensional subspace. Building on this mechanism, we further propose ReKey, an algorithm that writes this signal by regression into a few key-value neurons of every block, and measures the site and strength of each edit on the model’s own responses. Moreover, the base model fills every role of this recursion with forward passes alone, and its edits, written at different sites, compose in one checkpoint without refitting. Experimental results on ten visual reasoning benchmarks show that BaseRSI consistently improves a base vision-language model both in distribution and on held-out benchmarks. Our code is available at this https URL.
[CV-31] Beyond Appearance: Evidence-Guided Programmatic 3D Scene Reconstruction with Agent ic Lighting Reasoning
链接: https://arxiv.org/abs/2611.24565
作者: Zeting Luo,Yifan Li,Shuai Yang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project Page: this https URL Code: this https URL
Abstract:Image-to-code methods for 3D scene reconstruction produce executable code that generates editable digital content and supports visual simulation. However, existing methods primarily rely on appearance similarity, which provides limited guidance for distinguishing errors in geometry, materials, and illumination. Light shapes scene appearance through its interactions with geometry and materials, influencing shading, cast shadows, and reflections. We therefore present LightHarness, a VLM-based agentic framework that incorporates explicit lighting reasoning into iterative scene reconstruction. To be specific, we leverage complementary geometric, intrinsic, and illumination cues to jointly refine scene structure and appearance, and propose Lighting Reasoning Memory to maintain light-source hypotheses, which supports evidence, and revision history throughout iterations. Together, they support reasoning about partially visible and unseen sources, with evidence-guided verification directing subsequent revisions. Our framework reconstructs explicit scenes through coherent reasoning about geometry, materials, and illumination without additional training or differentiable rendering. Extensive experiments demonstrate superior image-to-code reconstruction accuracy under challenging shadows and complex illumination, further supporting downstream scene editing, object insertion, and versatile relighting.
[CV-32] SCR: Detecting Backdoor-Poisoned Samples through Clean-Reference Residuals
链接: https://arxiv.org/abs/2611.24558
作者: Yi Yang,Jinyang Huang,Feng-Qi Cui,Yu-Tong Guo,Sirui Zhao,Haiming Jin,Weihua Li,Quan Bai,Meng Li
类目: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
备注:
Abstract:Deep neural networks are vulnerable to backdoor attacks, in which adversaries manipulate model predictions by injecting poisoned samples into training data. Detecting these samples remains challenging because legitimate semantic variation can obscure poisoning-related signals in learned representations. Consequently, class-level statistics used by existing detectors may flag benign feature deviations while overlooking poisoned samples that blend into the class distribution. In this paper, we propose Semantic Counterfactual Residualization (SCR), which uses a small set of clean reference samples to account for benign variation through sample-specific feature reconstruction. SCR identifies poisoned samples by detecting coherent deviations from these clean baselines, without requiring prior knowledge of the attacks. We evaluate SCR through experiments covering twelve dirty-label and clean-label attacks across CIFAR-10, CIFAR-100, and Tiny ImageNet. With a small clean-reference set, SCR achieves attack-averaged true positive rates above 99.5% and false positive rates below 2% on each dataset, outperforming eight representative detection baselines in macro-TPR.
[CV-33] Fuse Before Reconstruct: A Novel Reconstruction Paradigm for Multimodal Industrial Anomaly Detection
链接: https://arxiv.org/abs/2611.24541
作者: Zhi Li,Hanyu Wang,Qiyu Chen,Fei Shen,Zhengtao Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Preprint before peer review. Includes supplementary material. Code: this https URL
Abstract:Industrial anomaly detection plays a pivotal role in precision manufacturing, where joint analysis of multimodal data emerges as a critical approach to enhance detection accuracy. Feature reconstruction, a dominant paradigm for such joint analysis, is primarily implemented through intra-modal and cross-modal methods, both of which are plagued by significant limitations. Intra-modal reconstruction neglects complementary cross-modal information, while its cross-modal counterpart is prone to projection errors and ambiguous one-to-many mappings (e.g., same textures mapping to multiple geometries), leading to detection failures. To address these limitations, we propose a novel feature reconstruction paradigm, Fuse Before Reconstruct (FBR), which first fuses multimodal features and then performs reconstruction for anomaly detection. Firstly, we propose a bidirectional cross-modal attention mechanism to achieve deep feature complementarity between image and point cloud features, mitigating projection bias through channel alignment and dynamic spatial prior integration. Besides, a learnable pixel-wise gated fusion strategy is adopted to preserve multi-modal structural consistency while optimizing local feature combination. Finally, dual-path collaborative reconstruction is implemented to recover original modal features from the fused representations, where cross-modal consistency constraints effectively eliminate mapping ambiguities. Extensive experiments show that FBR achieves 98.3% I-AUROC on MVTec 3D-AD and 92.6% I-AUROC on Eyecandies, surpassing previous state-of-the-art methods by +2.9% and +4.1%, respectively.
[CV-34] SCRiD: Spatial and Contextual References for Generalizable AI-Generated Video Detection
链接: https://arxiv.org/abs/2611.24524
作者: Hao Tan,Zekai Zuo,Boyang Chen,Jinghui Qin
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 14 pages,10 figures
Abstract:Generalizable AI-generated video detection remains challenging because natural motion and content variations can resemble synthesis artifacts, making isolated video discrepancies unreliable indicators of authenticity. To address this challenge, we propose Spatial and Contextual Reference-informed Detection (SCRiD), a framework that interprets video variations through complementary spatial and contextual references. Specifically, we introduce Matched-Coordinate Residual Response (MCR) to characterize temporal residual associations under motion and coordinate references with matched subpixel sampling conditions. We further develop Real-Conditioned Score Adaptation (RCSA), which learns a content-conditioned denoising reference from authentic videos and probes its responses across multiple noise scales and quality views to guide feature adaptation. To integrate these complementary sources of evidence, we design Reversible Evidence Exchange (REE) to enable information exchange through reversible symplectic updates. We train SCRiD using synthetic videos from only 4 generators and evaluate it across 88 generator or task subsets spanning 6 benchmarks. Our method achieves average ACC and AP of 72.72% and 74.23%, respectively, outperforming the strongest baseline by 3.24 and 0.81 percentage this http URL and robustness experiments further validate the effectiveness of the proposed components and the stability of SCRiD under common post-processing operations.
[CV-35] RAID: Representation Autoencoder for Intrinsic Decomposition
链接: https://arxiv.org/abs/2611.24521
作者: Claudia Cuttano,Tamy Boubekeur,Elena Garces,Giuseppe Vecchio
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Single-image intrinsic decomposition is an inherently ill-posed problem, requiring the disentanglement of strongly coupled appearance factors from a single RGB observation. Recent methods address this ambiguity with diffusion priors, at the cost of large, iterative, and often stochastic models. We show that accurate decomposition does not require a large generative model. Our key idea is to decouple reconstruction from decomposition on top of a strong pretrained representation. First, an X \rightarrow X teacher learns to reconstruct each intrinsic map from DINOv3 features using a shared decoder, without paired RGB-to-intrinsic supervision. Then, a DINOv3-based student predicts the corresponding teacher representations from RGB through lightweight, modality-specific projections, enabling training on partially annotated datasets and predicting all intrinsics in a single forward pass. \textbfRAID, predicts intrinsic properties in the structured latent space, enabling a compact 165M-parameter model, 100x faster than diffusion-based methods. Despite its compact size, it achieves competitive or superior decomposition quality, with improved robustness to input resolution. Additionally, our learned projections yield interpretable, modality-specific feature spaces.
[CV-36] Comparing Each Generation with Independent Siblings Gives Video World Model Post-Training a Proper Objective for Outcome Frequencies
链接: https://arxiv.org/abs/2611.24512
作者: Xin Li,Yiwen Chen
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Video world models are used as simulators, so when a first frame leaves several physical outcomes open, repeated generations should produce each outcome as often as it occurs in the data. Physics post-training scores every generation against the single recorded future of its history, which makes single videos more faithful, but the expected score is linear in the model distribution and is maximized by always producing the most likely outcome. The log-likelihood of the recorded outcome is maximized by the data distribution, and its gradient weights agreement with the record by the inverse of how often the model produces the generated outcome, a frequency that other generations of the same history with independent noise, its siblings, can estimate. To train with this weight from a finite group, we propose SibPO, which performs Sibling-paired Policy Optimization: siblings estimate a truncated inverse-frequency weight for agreement with the record, and a second term subtracts the same weight times the agreement with the siblings, which makes the expected objective a strictly proper divergence over outcome cells at every group size. On PisaBench, SibPO lowers the KS distance to simulated landing times by 9.1-10.2% relative to the strongest method outside its reward family on three of four video models without degrading single videos, and it lowers the error of estimated robot policy success rates by 13%.
[CV-37] RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback
链接: https://arxiv.org/abs/2611.24494
作者: Jiayou Zhu,Peng Guo,Litian Zhang,Xinze Shi,Yuting Zhang,Zelong Yu,Yize Tian,Jirong Liu,Zifeng Kang
类目: Robotics (cs.RO); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages, 2 figures
Abstract:Vision-language-action policies control embodied agents through a perception-action loop grounded in physical interaction, where adversarial patches can corrupt action predictions and thereby alter subsequent observations. To mitigate these threats, many existing defenses rely on policy adaptation or auxiliary models, but require additional training or inference. We introduce RECLAIM, a training-free, inference-time defense that uses a frozen VLA’s own attention response to local visual interventions to guide purification during closed-loop embodied manipulation. Our key insight is that attention changes induced by local edits provide feedback for choosing where and how to purify as the robot interacts with its environment. RECLAIM combines this feedback with robot geometry to refine patch localization and select local treatments, then revalidates them as the agent’s embodied actions change the observed scene. We evaluate RECLAIM against patches generated by four adversarial attack methods across multiple manipulation task suites under frozen VLA control. Under adversarial patch attacks, RECLAIM succeeds in at least one rollout on 95% of the evaluated tasks and raises rollout success on LIBERO-Object from 0% without defense to 70.0%. Its mean decision time is only 548.9 ms per control step. These results highlight policy-guided visual purification as an effective approach to defending embodied agents against adversarial patches and recovering closed-loop task execution.
[CV-38] Every Query Routes Itself: Accelerating Video Diffusion Transformers with Per-Query Routed Sparse Attention
链接: https://arxiv.org/abs/2611.24479
作者: Zhen Huang,Shuwei Li,Ruizhe Yao,Xinrui Chen,Danyi Liu,Haoyuan Liang,Zhiyu Ye,Qiuyan Sun,Bingwei Chen,Yiyuan Li,Xu Zhou,Haohuan Fu
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Self-attention over long token sequences dominates the inference cost of video diffusion transformers. Clustering-based sparse attention clusters both queries and keys and lets all queries in a cluster share one selection of key clusters. We show that this sharing limits accuracy: with key clusters and density fixed, attention grows more accurate as the query clusters are refined, and is most accurate when each query selects on its own, which needs no query clustering at all. We therefore propose Per-Query Routed Sparse Attention (PQRS), a training-free method that clusters only keys, lets each query select its own key clusters, and compensates for the unselected ones with their centroids. Per-query selections leave no query tile with a shared key list, so PQRS traverses key clusters instead of queries, buckets queries with identical selections into dense tasks, and merges their partial results in one pass under a fixed softmax offset carried from the previous denoising step. On four text-to-video and image-to-video models from the Wan and HunyuanVideo families, PQRS is the fastest of all the sparse attention methods we evaluate and the closest to the dense output, reaching 28.9 dB PSNR on Wan2.1-T2V and 35.4 dB on HunyuanVideo at 1.66 – 2.10\times end-to-end speedup.
[CV-39] Anchoring with Visual Evidence: Explicit Semantic Anchors for Joint Change Detection and Captioning
链接: https://arxiv.org/abs/2611.24462
作者: Kai Zheng,Hang-Cheng Dong,Puyi Fang,Junjia Pan,Zhenkai Wu,Wei Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Joint change detection and captioning seeks to localize changes and describe them from bi-temporal remote sensing images. A key challenge is translating dense visual evidence into the semantic conditions that guide description generation. We propose Explicit Semantic Anchors (ESA), a supervised interface that exposes entity presence, temporal appearance variation, and spatial extent. A prediction-guided readout combines visual features and predicted change probabilities, and three factor heads estimate the corresponding anchor activations. These activations guide generation through two complementary routes. The context route maps the three factor activations to three separate memory tokens, one per factor, and appends them to the dense visual memory, while the lexicon route produces an additive vocabulary bias at each decoding step. Dense visual memory remains available for spatial detail and information beyond the explicit factors. Anchor targets are derived from existing images and change annotations, requiring no additional anchor-level language annotation. At inference, all conditions are computed from the input image pair and the model’s own predictions. Experiments on LEVIR-MCI and WHU-CDC show strong joint detection and captioning performance, supporting the use of supervised visual attributes as explicit conditions for change description generation.
[CV-40] ShoWorld: Shape-Faithful Multi-Hypothesis Occupancy World Model
链接: https://arxiv.org/abs/2611.24457
作者: Zhiqiang Yan,Guangfa Wang,Yuan Wu,Jian Yang,Gim Hee Lee
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages
Abstract:Occupancy world models struggle to forecast moving objects due to motion uncertainty and shape distortion. A single trajectory can average plausible futures into an implausible path, and repeated re-voxelization can erode or inflate object shape even under correct motion. We introduce ShoWorld to address these challenges through separate motion and shape representations. The motion forecaster predicts probabilistic trajectories conditioned on scene layout, nearby agents and the future ego trajectory. Each hypothesis is rendered into occupancy and fused according to its probability to retain alternative futures. The shape component predicts sub-voxel points with occupancy probabilities and offsets from local geometry. These points move rigidly with the object, and voxel occupancy follows the probability that at least one occupied point falls within it. This representation reduces shape distortion under motion. ShoWorld achieves state-of-the-art results on Occ3D-nuScenes. With ground-truth occupancy and ego trajectory, it improves mIoU and IoU by 6.56 and 9.16 points overall and by 6.06 and 5.53 points on dynamic classes.
[CV-41] EviScope: Evidence-Conditioned Sampling for Long-Range 3D Detection
链接: https://arxiv.org/abs/2611.24451
作者: Mingyang Lyu(1, 2 and 3),Yinqian Sun(1),Gongce Cao(3),Chujie Tang(1),Yuhao Zhang(4),Lu Jia(5),Qian Zhang(1),Bo Wang(1),Pin Lyu(1) ((1) Institute of Automation, Chinese Academy of Sciences, Beijing, China, (2) Zhongguancun Academy, Beijing, China, (3) University of Chinese Academy of Sciences, Beijing, China, (4) University of Illinois Urbana-Champaign, USA, (5) University of Science and Technology Beijing, Beijing, China)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 29 pages, including appendices
Abstract:Query-based camera-radar detectors choose where and over which frames each object query samples from the query’s position or feature. We test whether radar evidence around the query should also set these choices at the 150 m range of MAN TruckScenes, where recall differs across object-level radar-return counts within every range band. EviScope is a minimal test: the current frame’s returns form a non-learned bird’s-eye evidence field, which each query reads at its decoded centre to scale its radial sampling extent and to weight eight frames. It adds 5,450 parameters (0.009%) to RaCFormer, with no latency increase in our timing. Our module-free recipe exceeds the published TruckScenes mAP (25.9 over seven runs against 24.1), so we measure the module only against its own baseline at matched seeds. The evidence a query reads rises with its object’s return count (Spearman 0.51) but captures zero-return objects only in part, and the learned radius tracks it ( \rho\approx-0.91 in four runs). Over seven four-epoch pairs the module changes NDS by +0.31\pm0.36 and mAP by -0.27\pm0.22 (mean \pm SE). Against a geometry-only conditioner with the same parameter count, trained at three matched seeds, it is higher by +1.25\pm0.72 NDS and +0.87\pm0.21 mAP, on mAP in every pair, and the control’s mean NDS is near the baseline’s. An exploratory far-range gain failed an internally pre-specified held-out test, and large inference-time substitution penalties also occur without query-local evidence: neither learned allocation nor inference-time dependence substitutes for matched end-to-end comparisons.
[CV-42] Retroactive Concept Bottleneck Models for Exemplar-Free Class-Incremental Learning
链接: https://arxiv.org/abs/2611.24445
作者: Yang Ji,Ying Sun,Yuanxin Zhuang,Yuting Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Exemplar-free class-incremental learning (EFCIL) requires models to learn new classes sequentially without retaining training samples from previous tasks. Continual concept bottleneck models (CBMs) make their predictions interpretable by identifying visual concepts in an image and using them to score each class. However, these methods typically focus on preserving learned concepts or approximating past classes with generated features, which can leave new concepts underused in old-class predictions. We propose RetroCBM, which retains a small set of vocabulary-independent statistics relating frozen visual features to the embedding space of a pretrained vision-language model. Even after earlier training images have been discarded, we can use these statistics to supervise new concepts. To do so, we develop a retroactive regression rule that yields the same predictor for each new concept as class-balanced ridge regression on all images seen so far. We then reproject the stored statistics to recover the full-history squared classification loss and thus learn how much each new concept should contribute to old-class predictions. Since the classifier may use systematic concept errors to distinguish nearby classes, we further introduce a graph-weighted penalty on the margin shifts caused by those errors. Across six datasets, RetroCBM outperforms state-of-the-art interpretable methods in all class-incremental settings, with an 18.1% relative gain in mean average incremental accuracy over the strongest exemplar-free baseline.
[CV-43] NativeSeg: Native Vision Language Segmentation of 3D Computed Tomography via Spatial Visual Tokens and Verified Local Zoom
链接: https://arxiv.org/abs/2611.24443
作者: Chengqian Ma,Gaole Dai,Zijie Meng,Weixuan Jin,Yijie Du,Yuexiao Ma,Xiawu Zheng,Fei Chao
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 13 pages, 5 figures, 4 tables. Code: this https URL ; Weights: this https URL
Abstract:Text prompted 3D computed tomography (CT) segmentation requires both full slice context and fine boundary precision across volumes where over 80% of organ prompted slices lack the queried target. Existing medical vision language models (VLMs) are largely non native segmenters: they delegate pixel delineation outside the VLM by compressing reasoning into a 1D [SEG] vector for an external Segment Anything Model (SAM) decoder, quantizing masks into lossy discrete codebooks, or issuing box prompts to stateless external SAM tools without empty slice screening or spatial state memory. We introduce NativeSeg, a native vision language segmentation framework on Low Rank Adaptation (LoRA) adapted Qwen3.8-27B that performs dense mask decoding, volumetric triage, spatial error self diagnosis, and multiround local zoom directly over the VLM’s 2D spatial visual tokens without external SAM decoders. NativeSeg decodes confidence maps from upper layer 2D spatial tokens fused with a five slice U-Net pathway, halts at zero zooms on 91% of standalone evaluations via a native presence gate (pres), localizes uncertain boundaries via a 20 cell multiscale error gate (err), and accepts zoomed updates only when predicted error decreases. Trained solely on adult CT, NativeSeg improves Round 0 DSC by +19.81 and +7.04 points over discrete codebook and 1D [SEG] readouts on the identical backbone, reaches 86.81% adult and 87.04% zero shot pediatric DSC, and safely refines 2D nnU-Net ( 89.52% adult; 86.86% pediatric), 3D SegVol ( 86.81% ), and 3D VoxTell ( 89.85% ).
[CV-44] Explainable and CPU-Efficient Brain Tumor Classification from MRI Using Two-Phase EfficientNetB3 Fine-Tuning
链接: https://arxiv.org/abs/2611.24440
作者: Deepro Das,Saad Kabir Uddin,Mohammad Alauddin,Bornali Sarker,Thasnim Akhter
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 6 pages, 5 figures, 6 tables
Abstract:Brain tumors are among the most lethal neurological disorders, with over 308,000 new diagnoses annually worldwide. Accurate classification of tumor type from Magnetic Resonance Imaging (MRI) scans is critical for treatment planning; however, manual radiological interpretation is time-consuming and prone to inter-observer variability. This paper proposes an automated deep learning pipeline for four-class brain MRI classification (glioma, meningioma, no tumor, and pituitary tumor) using EfficientNetB3 transfer learning with a custom classification head. A two-phase training strategy is adopted: Phase 1 trains the head with a frozen backbone (best validation accuracy: 92.77%), and Phase 2 selectively fine-tunes the top 37 layers at a 100x reduced learning rate (best validation accuracy: 94.91%). On an independent balanced test set of 1,600 MRI images, the system achieves 90.62% overall accuracy and a macro-average AUC of 0.9854. Gradient-weighted Class Activation Mapping (Grad-CAM) is integrated for class-discriminative visual explainability. The proposed system simultaneously addresses three dimensions absent in prior work: high accuracy, CPU-only computational efficiency, and clinical interpretability.
[CV-45] MLIE-CoT: Chain-of-Thought Reasoning via Multi-Level Image Editing
链接: https://arxiv.org/abs/2611.24433
作者: Baichuan Yang,Dongyuan Xu,Zhao Yang,Yong Liu,Bing Su
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 11 pages, 5 figures. Code: this https URL
Abstract:Chain-of-thought (CoT) reasoning has proven effective in improving the reasoning abilities of large language models by breaking complex tasks into step-by-step solutions. In recent years, researchers have attempted to extend CoT reasoning to visual generative models to improve their ability to handle complex tasks. However, existing CoT methods for text-to-image generation still rely mainly on textual reasoning and therefore cannot fully use and preserve the correct semantic information already present in an image, as even the most detailed language cannot capture all visual information. With advances in image generation, models such as FLUX and BAGEL can generate images conditioned on visual inputs. Recent work has attempted to carry images forward as context in the chain of thought, but this introduces another limitation: unsuitable earlier images can strongly interfere with the model, while repeated whole-image regeneration introduces new hallucinations at each round and causes them to accumulate. To address these issues, we propose MLIE-CoT, a visual chain-of-thought framework based on multi-level image editing. MLIE-CoT learns to select the repair granularity from three levels, preserving correct semantics whenever possible during reasoning while reducing the effect of incorrect content. To improve the ability of multimodal models to route errors of different severity to appropriate repair levels, we construct an image-repair dataset covering different error cases and use it to train a repair-level router. Extensive experiments demonstrate that MLIE-CoT significantly outperforms both its base models and methods using fixed repair strategies, producing clear improvements across multiple benchmarks.
[CV-46] Let a Small Model Read First: Faster Prefill for Long Interleaved Image-Text Inputs
链接: https://arxiv.org/abs/2611.24431
作者: Xiaolin Lin,Kai-Chiang Wu,Bing Li,Grace Li Zhang
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Vision-language models (VLMs) can answer questions about long interleaved inputs, such as documents of many pages and collections of hundreds of images, but they are slow to start: on inputs of about 100K tokens, the prefill takes most of the wait for the first output token. A common approach prunes the input by scoring its tokens with the target model itself or with a small model of the same family that first writes an answer. However, this scoring takes much of the time that pruning saves. We find that the evidence for a question lies in a few whole units (images or paragraphs), and that a small hybrid model finds them with one attention head of an early layer, even at half image resolution. Based on these findings, we propose SkimPrefill, which lets the small hybrid model read the whole input first, up to that early layer, with images at half resolution. Most layers of the small hybrid model (Qwen3.5-0.8B) run in linear time, so this pass is short. SkimPrefill then ranks every unit with that head and gives the target the top units as an ordinary shorter prompt. One untrained selector serves targets of different model families. Extensive experiments on MMLongBench at 128K tokens, with 19 datasets of five task types, show the effectiveness of SkimPrefill. With 10% of the input tokens kept, SkimPrefill speeds up the prefill of Qwen3.5-9B, Gemma-4-12B and Qwen3-VL-32B by 3.7 times, 11.6 time and 8.2 times while maintaining their average accuracy.
[CV-47] Probing Physics in Motion Diffusion Models Against Their Own Estimate
链接: https://arxiv.org/abs/2611.24423
作者: Saeed Ghorbani
类目: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
备注: 32 pages, 12 figures
Abstract:Recent studies report that video diffusion models “know more than they show”: physical properties decode better from their activations than from their inputs or outputs. But a diffusion model computes its clean-sample estimate from those same activations, so such a comparison measures how easily a readout extracts the information. Whether the model hides anything is a separate question. We set out to find hidden physical knowledge in kinematic text-to-motion diffusion models, where the physics of every intermediate estimate can be computed exactly, and recorded every layer along the sampling trajectories of MDM (two checkpoints) and MotionDiffuse. Against the usual baselines the layers do look ahead of the model’s own estimate on balance and flight. With every readout tuned the same way, the estimate closes the gap: for balance and flight, adding a layer gains at most 0.01, a size that a positive control shows we would detect. The fall rate of jumps is the one clear exception. Early in denoising, MDM’s layers add about 0.08 in R^2 (up to 0.11 in a refit) that the estimate does not yet show, and the gain fades as sampling proceeds; on MotionDiffuse it is small and not robust. The estimate also flags floating samples without training, but aborting them saves under a tenth of the compute. In a video model, plausibility that linear probes miss in Wan 2.1’s clean latent becomes readable after one fixed nonlinearity.
[CV-48] Resolution as Attention for Efficient Agent Memory Perception
链接: https://arxiv.org/abs/2611.24422
作者: Siqi Xiang,Zhipeng Xu,Shengrui Liu,Zulong Chen,Chunyan Miao,Shijian Lu
类目: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:LLM-based agents accumulate extensive interaction histories when solving complex, long-horizon tasks, making efficient memory management critical to their performance and scalability. While existing memory management methods primarily focus on what to retain or retrieve, they often overlook how much perceptual detail should be allocated to individual past interactions. We posit that effective agent memory depends not only on the information preserved, but also on the granularity at which each past interaction is perceived. Motivated by this insight, we propose Resolution as Attention (RAA), a memory perception mechanism that uses visual resolution as a means of attention allocation across interaction histories. RAA renders past interactions as images and dynamically adjusts their resolution at every step according to their relevance to the agent’s upcoming decision, preserving fine-grained information for critical interactions while reducing visual-token consumption for less relevant ones. To adaptively determine the perceptual fidelity of each interaction, we train a lightweight relevance estimator with teacher-model guidance and further optimize it to preserve the agent’s decision consistency at a reduced visual-token cost. Experiments across multiple tasks and backbones show that RAA reduces token consumption by over 50% without compromising task performance. Moreover, RAA can be integrated as a plug-in perception layer with existing memory management methods, yielding further token savings and revealing adaptive perception as a complementary dimension of agent memory.
[CV-49] PhysResponse: Probing the Physics of Video Generation with Controlled Interventions
链接: https://arxiv.org/abs/2611.24419
作者: Yuhan Li,Shuran Zhang,Haixin Tian,Baonuo Rao,Weipeng Li,Yujia Hou,Zhenyu Wang,Zijun Wang,Luoyang Lin,Lijing Luo,Jiehui Huang,Kaiyuan Yang,Jingbin Tan,Panwen Hu,Xiaodan Liang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 11 pages. This version contains the main paper and references only. Supplementary material will be added in a later revision
Abstract:Video generation models are increasingly viewed as world simulators, which requires not only plausible motion but dynamics that respond correctly to specified physical parameters. Existing models, datasets and benchmarks, however, largely target physical plausibility, leaving this physical parameter responsiveness insufficiently explored. We introduce PhysResponse, a benchmark that measures it at two levels: scene-conditioned simulator inversion recovers the effective gravity, friction and initial speed of generated motion, and event evaluation tests outcomes near and across critical transitions. Evaluating ten image-to-video models, we find that even the lowest parameter errors exceed those of physically correct videos by over an order of magnitude, that models rarely reproduce both outcomes of a transition pair, and that the two levels expose complementary failures that visual metrics such as PSNR miss. To supply the missing training signal, we construct 72,000 simulated videos of 20 multi-object interaction types with controlled parameters; fine-tuning on them lowers parameter errors, even on held-out interaction types, and transfers to real videos and an external physics benchmark. Continued training with a numerical encoder further lowers all three errors on held-out types. PhysResponse provides a foundation for world simulators that follow physical conditions, not just appearances.
[CV-50] Why Multi-Source Audio Fails to Compose: A Geometric Diagnosis and Its Remedy BMVC2026
链接: https://arxiv.org/abs/2611.24410
作者: Anurag Singh,Prajwal Singh,Shanmuganathan Raman
类目: Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD)
备注: Accepted in BMVC 2026
Abstract:Multi-source audio-to-image generation exhibits consistent compositional failures when audio tokens occupy geometrically displaced, semantically unanchored positions in the CLIP text-token space, where the diffusion U-Net lacks the prior to reliably attend to them or ground them in distinct spatial regions. We formalize this as the geometric displacement phenomenon and expose it through a two-level clustering analysis comparing BEATs and CLAP embeddings at the raw encoder and projected CLIP token levels. Three geometric properties emerge as important: intra-class compactness, angular alignment with the CLIP text manifold, and inter-token spatial separation. We show that CLAP, whose contrastive language-audio pretraining aligns embeddings with natural language semantics, substantially outperforms BEATs on all three properties, improving the silhouette score by more than 3x and the Audio-Image Content score as well. Building on this analysis, we introduce a multi-token extension that requires no diffusion-backbone retraining and assigns each audio source to a distinct placeholder token in the CLIP tokenizer, complemented by a mixed inference-time objective that combines Attend-and-Excite with an A* spatial separation loss to resolve residual cross-token interference. For evaluation, we construct multi-audio compositional benchmarks from VGGSound (VGG2, VGG3, and VGGN) and extend six standard audio-image metrics to the multi-source setting. Across five benchmarks and four strong baselines, our method achieves strong results on the metrics most directly tied to compositional faithfulness, with AIC improvements of up to 2.25x over the strongest baseline and FID reduced by nearly 2x on two-source generation. We propose geometric displacement as a diagnostic framework for cross-modal generation systems that project learned embeddings into a pretrained contrastive space.
[CV-51] StrideWorld: Benchmarking Scene Rigidity and Action Consistency in Interactive Video World Model
链接: https://arxiv.org/abs/2611.24408
作者: Chuang Wang,Rui Nie,Jie Song,Junyuan Gao,Jiahe Song,Haitao Zhou,Boyu Zhu,Xing Li,Jiang Wu,Conghui He,Qian Yu,Weijia Li
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Interactive video world models turn user commands into the exploration of a generated scene, which requires the scene to keep its 3D structure as the camera moves and each action to keep a stable effect. Existing benchmarks mostly evaluate these properties against references fitted to the video under test, either a scene reconstructed from its own frames or a camera trajectory aligned to the command path with a free scale. Such references can absorb gradual scene deformation and changes in pace and route length, so these errors can pass undetected. We therefore introduce StrideWorld, a benchmark whose references are fixed before the motion they judge. It evaluates scene rigidity against the ground-truth geometry of the start view, and action consistency against the stride of each action, the camera motion of one command. The stride is measured on the model’s first steps and then used to test the steps that follow, so each model is judged only against its own steps. Its measurements are validated on rendered videos with known camera motion, and its scores agree with human judgments. Across \nvideos videos generated by \nmodels open models from 140 start views, all models but one deform their scenes as the camera moves, with offsets up to 7.3 times those of a rigid rendering along the same camera path. Most models keep a steady pace when a single action is held. Along composed routes, several models often stall, and most completed routes end away from the positions predicted by each model’s own strides. Only the two models that scale commands by the estimated scene depth bring most routes close to these positions. Fine-tuning two models on a few rendered clips removes most of their mid-route stalls. Code, data, and a leaderboard are publicly available at this https URL.
[CV-52] RAPID: Reconstruction Accelerated by Polarization-Informed Depth for 2D Gaussian Splatting
链接: https://arxiv.org/abs/2611.24404
作者: Daisong Ouyang,Linlin Xia,Liangkun Yan,Song Gu,Shedong Zhao,Xin Dai,Yi Zheng
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages, 5 figures, 5tables
Abstract:Efficient and accurate 3D surface reconstruction from multi-view images remains a fundamental challenge in Gaussian splatting. Many existing methods rely on sparse geometric initialization and iterative densification, resulting in substantial optimization overhead and difficulties in balancing rendering quality and geometric accuracy, particularly in reflective and weakly textured regions. To address these limitations, we propose RAPID, a geometry-first framework that uses multiview polarization cues to recover dense surface priors. We integrate surface normals into depth maps using sparse geometric anchors. These depth maps are then fused across views. Curvature-adaptive sampling and oriented Gaussian initialization then support efficient optimization. Experiments on the PISR dataset demonstrate an approximately (5\times) speedup over PhyGaP, which is a state-of-the-art Gaussian-based reconstruction method. RAPID reduces the average optimization time from approximately 21 minutes (PhyGaP) to 4 minutes while maintaining competitive novel-view synthesis quality and mesh reconstruction accuracy. These results demonstrate the effectiveness of RAPID in accelerating Gaussian optimization from polarimetric multi-view inputs.
[CV-53] Reliable Tree Recruitment from Scarce Annotations for Forest Inventory Expansion
链接: https://arxiv.org/abs/2611.24401
作者: Luiz Fernando Souza Luz,Pedro J. Soto Vega,Matheus Pinheiro Ferreira,Laura Elena Cué la Rosa,Hudson Franklin Pessoa Veras,Ernandes Macedo da Cunha Neto,Evandro Orfanó Figueiredo,Ana Paula Dalla Corte,Carlos Roberto Sanquetta,José Matheus Segre Moneva Viveiros,Matheus Santos Fuza,Heitor Eduardo Ferreira Campos Morato Filpi,Cristiano Vinícius Vidal,Vinícius Moura Costa,Débora Nascimento Rosário,Danilo Roberti Alves de Almeida,Paulo Guilherme Molin,Dário Augusto Borges Oliveira
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 20 pages, 15 figures
Abstract:Species-level inventories of individual tree crowns (ITCs) from UAV imagery are limited by annotation cost: in tropical forests, many species are represented by only a handful of field-validated crowns. We propose a reliable tree recruitment framework that expands a small annotated inventory by iteratively self-training a multi-task DeepLabV3 network, combining semantic segmentation with a crown-distance regression head, on RGB UAV orthomosaics. At each iteration, candidate crowns enter the training inventory only if they pass a confidence filter on class probability and distance map and a morphology filter on crown area and solidity, with at most five new crowns per species, keeping recruited samples consistent with the reference inventory. We evaluate the framework at two Brazilian sites: an Amazonian rainforest in Acre (17 species, 355 annotated crowns, 4 cm GSD) and a forest site in the Cerrado of Maranhão (4 species, 51 annotated crowns, 1.8 cm GSD). Over 20 iterations, the number of retained crowns grew from about 500 to 960 in Acre and from about 700 to over 3100 in Maranhão, while crown-level F1 exceeded single- and multi-task DeepLabV3 baselines (53.5% vs. 50.0% and 45.2%; 92.2% vs. 79.7% and 83.6%). The framework offers a low-cost route to expanding species inventories with consumer-grade UAVs and limited fieldwork.
[CV-54] Enhancing Active Learning Performance with Initial Data Selection Based on Foundation Models
链接: https://arxiv.org/abs/2611.24384
作者: Mohammad Mirzaee,Mehrshad Fallah Estalkhzir,Mohammad Reza Mohammadi
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 26 pages, 18 figures
Abstract:In active object detection, selecting the initial labeled pool is essential, especially in complex datasets with expensive labeling and low resource budget. This study presents an active learning framework designed to optimize the initial training cycle of active learning for the object detection task. Our approach diverges from traditional methods by utilizing feature-based clustering from the YOLO-World model and refining images based on objectness scores rather than relying on uncertainty sampling in the initial cycle. This approach focuses on finding samples with various features and emphasizes training in areas where the foundation model is most confident. Doing so helps the model better understand different types of objects. For subsequent cycles, we employ a consistency-based sampling technique to maintain the robustness of the training process. To evaluate our method, we use two complex datasets called Aquarium and NWPU VHR-10, which are well known for their diverse object sizes and challenging conditions. In the NWPU dataset, our method increases mAP50 by an average of 6.6% in image-based evaluation. Moreover, the image refinement method shows an improvement of approximately 11% in object-based evaluation. The results show improvements in model performance and labeling time, demonstrating the advantages of using foundation models in early cycles of active learning. An implementation of the algorithm is available at this https URL
[CV-55] Where Should a Diffusion Transformer Meet Its Teacher?
链接: https://arxiv.org/abs/2611.24380
作者: Zhuoqi Hu,Shiwei Li,Haozhao Wang,Xiandi Luo,Yihao Ouyang,Wenting Shen,Ang Wang,Jiamang Wang,Minyi Guo,Ruixuan Li
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Representation Alignment (REPA) significantly accelerates Diffusion Transformer training by guiding intermediate student features with a pretrained visual teacher, yet existing methods typically attach supervision to a single fixed layer. As student representations evolve, however, we find that the most effective layer for alignment varies across training stages. To track this evolving layer suitability online without costly search, we measure gradient compatibility between the flow-matching and alignment objectives, and propose AdaREPA, an online routing framework that adaptively selects the supervision layer as training progresses. AdaREPA uses a softmax distribution over smoothed layer compatibility scores to sample a layer for alignment at each training step. This probabilistic routing adapts teacher supervision to the student’s changing learning needs, with minimal training overhead and no additional inference cost. On ImageNet-256, AdaREPA accelerates convergence and improves generation quality across model sizes, while revealing how the most suitable layers for supervision shift over training.
[CV-56] CanopyKV: Remember beyond the Crop for Fast Active Visual Reasoning
链接: https://arxiv.org/abs/2611.24373
作者: Lei Yang,Dayan Wu,Yiliang Zhu,Xinze Liu,Hengjie Zhu,Zihao Zhang,Ding Wang,Peng Fu,Zheng Lin
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 16 pages, 11 figures, including supplementary material
Abstract:Active visual reasoning enables multimodal models to answer challenging visual questions by gathering fine-grained evidence through iterative image inspection and tool use. However, accumulating crops and reasoning tokens expands the key-value (KV) cache, making compression important for reducing memory and inference costs. Existing KV compression methods typically select historical tokens based on attention, recency, or redundancy. These criteria do not explicitly distinguish locally replaced evidence from complementary information needed for future exploration: each crop refreshes a local region, while earlier views may still support subsequent decisions. Discarding such evidence can impair accuracy or require additional search, while scoring and cache reorganization can offset the computational savings from compression. We introduce CanopyKV, a training-free KV compression framework that uses parent-child crop relationships to retain complementary evidence, keeps retained history stable, and protects the state needed for ongoing reasoning. Tool metadata provides retention signals without additional attention scoring, while compact KV storage reduces subsequent attention costs. Across VisualProbe, V*, HR-Bench, and MME-Lite, CanopyKV achieves state-of-the-art answer accuracy and end-to-end efficiency among the evaluated KV compression methods. On MME-Lite, it retains approximately one-quarter of Dense’s total KV while improving accuracy by 10.5 percentage points and accelerating complete reasoning by 1.64\times .
[CV-57] One Affinity Two Axes: Bidirectional Slot Attention for Iterative Aggregation across the Structure–Detail Spectrum in Visual Retrieval
链接: https://arxiv.org/abs/2611.24371
作者: Zihang Wang,Wenkai Zhu,Lingdong Kong,Teng Yan,Alan Liang,Shuyuan Li,Xieyuanli Chen,Dong Kong,Guanyu Zong
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Aggregating dense patch tokens into compact and discriminative global descriptors is central to visual retrieval. Although foundation models provide rich local features, existing aggregation heads still rely on single-pass mappings and lack an explicit mechanism to coordinate structural and fine-grained cues. Therefore, we propose Bidirectional Slot Attention (BiSA), which reformulates aggregation as an iterative coupled system. Cross-Axis Consensus (CAC) progressively refines learnable slots by interpreting a single shared token–slot affinity along two complementary axes. Along the competition axis, slots vie for each token under a temperature shared across all of them, determining token ownership and inducing a non-redundant scene decomposition. Along the selection axis, a slot-specific temperature governs the breadth of each slot’s response, spanning a spectrum from broad layouts to sparse fine-grained details. Their consensus couples non-redundant decomposition with multi-granular slot specialization. Furthermore, the two axes expose intrinsic token-reliability cues at no extra cost: a token’s claim strength under competition and its salience under selection. Progressive Token Filtering (PTF) converts these cues into a retention probability that down-weights unreliable tokens before the next CAC iteration, without any segmentation branch or external prior, forming a self-reinforcing loop of competitive decomposition, selective focusing, and feedback purification. Across all three retrieval tasks, BiSA leads on five of six place recognition benchmarks (+4.2 R@1 on Baidu Mall), tops all 16 metrics of drone–satellite geo-localization (+8.4 R@1 at 150 m), and improves instance-level retrieval on ROxford-Hard by 4.5 mAP points. Notably, the selection axis adds only 20 learnable scalars, yet improves average R@1 by 2.8 points.
[CV-58] SCOUT: Risk-Guided Foundation-Model Escalation for Robust Tissue Detection in Whole-Slide Images
链接: https://arxiv.org/abs/2611.24366
作者: Cristian Tommasino,Antonio M. Rinaldi
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Reliable tissue detection is essential for whole-slide image analysis, yet high average segmentation accuracy can conceal failures involving pale tissue, small fragments, and heterogeneous staining. Pathology foundation models offer transferable representations, but exhaustive inference is computationally demanding and may introduce false-positive tissue predictions. We propose SCOUT (Selective Correction of Uncertain Tissue), a risk-guided cascade that selectively refines an initial thumbnail-based tissue mask. A lightweight audit network estimates the spatial distribution of segmentation errors and prioritizes candidate regions for processing by a frozen pathology foundation model. An asymmetric consensus rule integrates the resulting predictions with the initial mask to recover missed tissue while limiting false positives. A calibrated slide-level review score additionally controls the marginal rate of unflagged case-group failures under the calibration assumptions. Evaluation on a public multi-source, multi-stain benchmark shows that SCOUT reduces the proportion of case groups requiring mask correction from 38.1% for the best released detector to 18.6%, with particularly pronounced improvements in immunohistochemistry. Source-held-out experiments demonstrate better generalization than a segmentation network trained on the same annotations, while selective inference achieves an observed failure rate no higher than processing all candidate regions. Downstream, lymph-node classification with SCOUT masks is non-inferior to that obtained with two of the three released detectors evaluated. A variant built on an openly licensed foundation model is publicly released as an installable Python package with trained weights. These results support risk-guided foundation-model inference as an effective approach to tissue detection across heterogeneous histopathology data.
[CV-59] owards Pose-Robust Few-Shot Vision-Language-Action Models
链接: https://arxiv.org/abs/2611.24360
作者: Wenhao Li,Zhongyi Han,Yuhao Wei,Liang Han,Yilong Yin,Liqiang Nie
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注: 17 pages, 13 figures, 17 tables (including supplementary material). Project page: this https URL
Abstract:Fine-tuning a vision-language-action (VLA) model on a few dozen demonstrations is the standard way to put a manipulation policy on a robot, yet the resulting policy silently assumes that the camera and the robot base stay exactly where they were during data collection: a bumped tripod or a base re-docked a few centimetres away makes it reach for the wrong place. We show that both failures are one phenomenon, a change of the camera-to-base extrinsic that the policy has memorised, and that the robot carries everything needed to undo it. We introduce WACO, a world-anchored canonicalization framework for VLA models: the arm, placed by forward kinematics at the measured joint angles, serves as a metric calibration target, and the static scene serves as the world anchor that tells a moved camera from a moved robot. From a single RGB-D frame, WACO estimates both shifts without ground-truth extrinsics or extra data, canonicalises the image to the calibration viewpoint and the state, action and start pose to the calibration base frame, and fine-tunes the unmodified policy under the artefacts of its own estimator. We also introduce LIBERO-PoseShift, the first benchmark for RGB VLAs with graded camera, base and compound shifts. Across LIBERO-PoseShift, the LIBERO-Plus camera track and four real-world tasks on a UR5e robot, WACO consistently outperforms the standard few-shot recipe and prior viewpoint-robust methods, raising shifted success by 13.2 points in simulation and by 36.7 points on the real robot, recovering 90% of the simulation gap to a ground-truth oracle and 87% of the real-robot gap, and establishing a new state of the art among methods that use no perturbed training data, with no test-time calibration. Project page: this https URL.
[CV-60] OCTR2A: Real-Time RGB-to-BEV Trajectory Projection and Forecasting at Urban Intersections
链接: https://arxiv.org/abs/2611.24352
作者: Emanuele Iacobelli,Simone Teglia,Francesco Pro,Claudia Melis Tonti,Irene Amerini
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Trajectory forecasting at urban intersections is often studied assuming that reliable bird’s-eye-view (BEV) motion histories are available, which in practice requires costly resources such as LiDAR, aerial views, HD maps, or calibrated infrastructure. We address this upstream perception-to-forecasting problem by reconstructing and predicting agents’ BEV trajectories only from paired opposite-corner RGB cameras mounted on existing urban infrastructure at variable heights at inference time. We introduce the Opposite-Corner Trajectory Reconstruction and Anticipation Architecture (OCTR2A), a modular pipeline integrating per-view detection with class refinement, geometric cross-camera matching, transformer-based BEV projection, and egocentric GRU forecasting, trained in cascade and mainly supervised with a soft multi-scale spatial-overlap loss. Since no public benchmark provides this sensing configuration, we introduce R3ST++, a Blender-based synthetic benchmark built from SIND trajectories that provides synchronized opposite-corner views, cross-view correspondences, and BEV ground truth, while also serving as a deployment-support simulator for site-specific fine-tuning. Experiments on these datasets show that, with 12 observed and 12 predicted frames at 10,Hz, OCTR2A achieves projection~-~forecasting ADE/FDE of 0.144/0.140~-~0.154/0.241~meters in real time on a single consumer GPU. Evaluated on ground-truth BEV histories, the forecasting module improves over the strongest state-of-the-art methods on SIND by 66.7%/41.7% ADE/FDE. Transfer experiments on TUMTraf-V2X show that R3ST++ pretraining supports adaptation to real-world roadside domains with different sensing configurations.
[CV-61] Can Weight-Space Programming Alter Learned Ethical Priorities?
链接: https://arxiv.org/abs/2611.24330
作者: Kai Yi Ji
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:We investigate whether a weight edit can alter how a model trades off honoring a commitment against benefiting more people. In a concept-supervised text model, we alter fitted choice weights so that the benefit required to override a formal pledge is larger than for an ordinary promise. Across six independently trained text readers, the edit selects the larger group in both orders on 100% of targeted ordinary conflicts and 83.3% of targeted formal conflicts. It preserves the source model’s success rates on smaller conflicts and its factual accuracy. The remaining formal-conflict failures coincide with errors reading a beneficiary number. A separate classifier demonstrates why this distinction matters: reusing a factual answer passes an unconditional choice test but fails to preserve small-gap promises. In a YOLO11n pilot on COCO128, a bias edit increased small-target matches from 3/43 to 9/43, and an equivalent score calibration reproduced the retained detections. These exploratory results show a way to test selective changes to learned priorities in a model with an exposed trade-off mechanism. Whether the method can discover such changes in less structured language models remains open.
[CV-62] Pixel2Perspective: A Benchmark for Multi-Modal and Multi-View Generalization in Vision-Language Models
链接: https://arxiv.org/abs/2611.24327
作者: Aditya Gandhamal,Aniruddh Sikdar,Arya Kishor,Suresh Sundaram
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 16 pages
Abstract:Embodied AI systems, such as drones and autonomous vehicles, improve navigation by enhancing perception in environments containing unseen objects. While Visual Language Models (VLMs) offer strong image understanding, their real-world application is underexplored. For effective deployment, open-vocabulary semantic segmentation (OVSS) models, leveraging models like CLIP to assign pixel-level labels from text, must be robust to outdoor conditions, adverse weather, and varying viewpoints. Despite their potential, these models encounter significant challenges in cross-domain generalization, hindering their practical use in embodied AI applications. In this work, we present Pixel2Perspective, a benchmark for multi-angle object localization and open-vocabulary segmentation across both aerial and ground perspectives. This benchmark supports thorough evaluation across different viewpoints and sensor modalities. We assess foundational models like CLIP and advanced OVSS models using a curated dataset that spans varied outdoor conditions, sim-to-real scenarios, and multiple sensors, including infrared, to support the development of robust and generalizable perception systems. We also present a novel framework, Multi-Source RGB-IR Uncertainty Quantification (MSRIQ), which jointly models aleatoric and epistemic uncertainties across RGB and infrared imagery. This approach supports robust evaluation of open-vocabulary segmentation models across diverse environmental conditions. Our work sets a robustness-driven benchmark, providing key insights and laying the foundation for future research in open-world embodied AI perception. The benchmark and project page are available at this https URL
[CV-63] Fuse to Recognize the Unseen: Class-Relative Infrared-Visible Fusion for Open-Vocabulary Semantic Segmentation
链接: https://arxiv.org/abs/2611.24316
作者: Zehao Gao,Jie Feng,Wenda Zhao,Junpeng Zhang,Ronghua Shang,Weisheng Dong
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 11 pages, 6 figures
Abstract:Visible-infrared image fusion leverages complementary thermal cues to enhance visual perception, particularly under challenging illumination. However, such perceptual complementarity does not readily translate into semantic gains in open-vocabulary semantic segmentation (OVSS), where CLIP-based models are predominantly pretrained on RGB imagery. Directly introducing IR information may instead disrupt RGB-language alignment, turning promising thermal cues into semantic interference that undermines unseen-class recognition. Our pixel-level RGB-IR semantic analysis reveals that class-relevant thermal cues can correct certain RGB misclassifications while preserving reliable RGB predictions, motivating RGB-anchored, class-relative thermal cue selection. Based on this insight, we propose EmergeFuse, an infrared-visible fusion framework for OVSS that selectively converts complementary thermal evidence into unseen-class semantic corrections. Specifically, class-relative cue selection evaluates RGB-IR semantic responses and calibrated thermal evidence relative to RGB predictions, generating class-relative correction requests through competitive selection. Text-contrast image rectification then translates these selected requests into semantic rectification directions through text contrasts, guiding evidence-gated feature updates to realize complementary thermal evidence for unseen-class recognition. Experiments on three benchmarks demonstrate gains of up to 4.755% in unseen-class mIoU and 15.730% in strict complementary conversion rate over existing fusion methods, while maintaining competitive fusion quality.
[CV-64] AnchorUp: Anchoring Feature Upsampling on a Classical Interpolation WACV2027
链接: https://arxiv.org/abs/2611.24296
作者: Kotaro Horikawa,Keisuke Toida,Junya Komori,Kazuhiro Hotta
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted to WACV 2027. Code: this https URL
Abstract:Vision foundation models produce semantically strong features on a coarse patch grid, and dense prediction needs them at pixel resolution. Existing learned upsamplers close that gap by regressing the high-resolution feature map from the low-resolution one, typically through cross-attention driven by the input image. We observe that most of what they regress needs no learning: plain bilinear interpolation already reproduces the smooth regions of the target, and recent benchmarks even report bilinear and bicubic on par with several learned upsamplers. The rest is mostly edges and fine details, which suggests that these learned upsamplers spend much of their capacity re-computing the smooth content that classical interpolation provides. We therefore introduce AnchorUp, which stops predicting the high-resolution feature map and predicts only its correction from the interpolation. It keeps a parameter-free classical upsampler as an anchor, obtains a second prediction from a small guidance network, and blends the two through a single learnable scalar gate. Because the anchor supplies the smooth content, the guidance network only has to correct what remains. An analysis of the learned correction confirms that it concentrates on edges and fine details, and so the network can be small: AnchorUp uses 167,425 parameters, a quarter of the budget of the strongest baseline, NAF, and runs at 64% of its latency. Trained once on DINOv3 with only the first of NAF’s two training stages, and evaluated on that backbone plus six held-out ones, it is already on par with NAF trained through both stages on Pascal VOC and Cityscapes. Our method surpasses it on DAVIS-2017 video object segmentation; the gain holds on every backbone and is larger in contour accuracy F. Code is available at this https URL.
[CV-65] CROMPT: Adapting CLIP via Cross-Modal Prompting for Robust Zero-Shot Generalization under Adverse Weather
链接: https://arxiv.org/abs/2611.24284
作者: Aniruddh Sikdar,Arya Kishor,Snehashis Dhibar,Suresh Sundaram
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 17 pages
Abstract:Domain adaptation for semantic segmentation is challenging when target-domain images are unavailable during training. Zero-shot domain adaptation addresses this by adapting models using only textual descriptions of the target domain. However, existing prompt-learning methods are not well suited for ResNet-based CLIP vision backbones or pixel-level cross-modal feature alignment. We propose CROMPT, a prompt-learning framework for zero-shot domain adaptation in semantic segmentation. CROMPT introduces Cross-Modal Feature Augmentation (CROMA), combining Random Visual Prompting with Hybrid Textual Prompting (HTP) to jointly adapt visual and textual representations. We further propose CROMA-VL, which leverages vision-language models to generate domain-conditioned prompts, and a progressive easy-to-hard adaptation strategy for improved domain invariance. Experiments on two benchmark datasets demonstrate consistent improvements over strong baselines and state-of-the-art methods, achieving 1.63-2.63% mIoU gains while maintaining inference-time complexity. These results establish CROMPT as an effective and efficient framework for zero-shot multi-domain semantic segmentation. OpenReview: this https URL
[CV-66] Semantic Guidance for Joint Infrared-Visible Video Fusion and Frame Interpolation
链接: https://arxiv.org/abs/2611.24277
作者: Yushen Xu,Xiaosong Li,Huaming Wu,Dongwei Ren
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages, 5 figures, 4 tables
Abstract:Infrared and visible video fusion at high frame rates requires recovering missing observations when camera rates differ or fall below the desired output rate. Cascading interpolation and fusion can accumulate errors and cause brightness fluctuations. Motion and occlusion further complicate content selection, particularly when missing visible frames also remove direct semantic guidance. We propose SAFI, which uses semantics at the target time to guide joint interpolation and fusion. When the visible frame rate exceeds the infrared rate, available visible frames provide semantic context directly. At times without visible observations, motion guides local matching, and propagation recovers semantics from neighboring frames. Semantic context complements aligned image features at the target time, while matching confidence helps select reliable infrared and visible information. Reconstruction and fusion share source selection within each scale, allowing supervision on missing frames to guide fusion. To preserve visible appearance and reduce flicker, we bound thermal enhancement and penalize brightness inconsistencies across frames while accounting for changes in thermal contrast. Experiments on three datasets show improved visual quality with less computation than cascaded baselines and consistent temporal gains when input rates differ. SAFI maintains strong downstream tracking performance despite missing frames. The code is available at this https URL.
[CV-67] Look Closer Look Back or Wait for Agent ic Streaming Video Understanding
链接: https://arxiv.org/abs/2611.24267
作者: Pengyu Li,Zhitao Gao,Lingling Zhang,Wenxuan Huang,YiCheng Xiao,Chen Zheng,Jun Liu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Streaming video models must answer questions about a live video under strict memory and latency limits. Feeding only the latest few frames keeps present perception sharp, whereas adding past frames dilutes attention on the current scene and triggers hallucinations. Recent cascade methods retrieve past frames when uncertain, but they still fail when current frames are blurry or when decisive future clues have not appeared yet. We introduce ActiveForager, a streaming video agent that maintains a clean short working window and gathers extra evidence only on demand. For each query, the agent either answers directly, zooms into high-resolution recent frames, recalls past events from memory, or waits for upcoming frames. When recalling history, a shared-weight sub-agent verifies candidate clips in isolation and returns a compact evidence summary. A stream-decoupled attention mask then keeps historical evidence from interfering with present visual tokens. We train this policy using counterfactual information gain, which rewards a tool call only when the gathered evidence makes the true answer more likely and prevents tool collapse across all actions. Across Qwen2.5-VL-7B and Qwen3-VL-8B on OVO-Bench, ActiveForager improves real-time perception, historical recall, and proactive response at once, lifting overall accuracy on Qwen3-VL-8B from 47.6% to 65.9%. It also doubles zero-shot accuracy on the multi-hop Video-Holmes benchmark while adding minimal visual tokens per query.
[CV-68] Do Thinking VLMs Need to Keep Looking at Long Videos? Write It Down Then Look Away
链接: https://arxiv.org/abs/2611.24259
作者: Pengyu Li,Zhitao Gao,Lingling Zhang,Heng Wang,Chengqian Ma,WenHao Wang,Wenxuan Huang,Chen Zheng,Jun Liu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Vision-language models (VLMs) reason about long videos over hundreds to thousands of tokens. Every decoding step re-reads the key-value (KV) cache of the entire video. Must the model keep looking while it reasons? In LookAway, the model first writes timestamped notes on what the question needs and then puts the video away. Once the notes close, the inference engine hides the video’s KV cache for the rest of the response. Across both hybrid and pure-attention VLMs, dependence on the video is concentrated early in the response. On Qwen3.6-27B, writing notes preserves accuracy better than free reasoning at the same withdrawal step. Replacing them with another video’s notes harms accuracy, whereas keeping the video visible alongside the model’s own notes adds no significant benefit. Without additional training, LookAway nearly matches full-video thinking accuracy at 0.12-0.13 times its modeled decode cost. To shorten this initial look, we fine-tune the model on its own correct LookAway responses. Reinforcement learning then rewards correct answers while penalizing KV reads. The trained model puts the video away after a median of 66-68 tokens. Across three held-out benchmarks, it writes more tokens than an identically trained full-video model yet reads only 27-35% as many KV positions at matched accuracy.
[CV-69] Spark-H3: Adaptive Block Sparse Attention for Video Diffusion Models
链接: https://arxiv.org/abs/2611.24256
作者: Zecheng Tang,Jiaye Fu,Youtian Lin,Xinyao Rao,Liuxuan Sun,Jian Zhang,Yao Yao,Wei Li
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Long video sequences make attention a substantial cost in video diffusion models. Block sparse attention (BSA) reduces this cost by computing selected blocks exactly while compressing the remaining blocks. However, fixed token grouping can place tokens with dissimilar attention preferences in the same group, while uniform key averaging can underestimate the attention mass of compressed blocks. We introduce Spark-Attn, a training-free BSA method comprising Spark-Reblock and Spark-Reweight. Spark-Reblock recursively groups tokens by attention preference while preserving regular block shapes for efficient GPU execution. Spark-Reweight uses the global mean query as an anchor to construct attention-weighted key-value representations and logit biases for the compressed blocks, calibrating their attention mass. We instantiate Spark-Attn in MiniMax-H3 as Spark-H3. At 10% exact-block density, Spark-H3 delivers a 2.49 \times DiT speedup and reaches 24.81 dB PSNR against the corresponding dense outputs on MiniMax-H3 (20 denoising steps, 345 frames, 1344 \times 768). Increasing the density to 20% further improves PSNR to 26.76 dB while still achieving a 2.26 \times DiT speedup. These results demonstrate that Spark-H3 achieves a state-of-the-art quality-efficiency trade-off for training-free BSA on video diffusion models.
[CV-70] One Shot Any View: High-Fidelity Video Reshooting with Geometry Guidance and Visual References
链接: https://arxiv.org/abs/2611.24253
作者: Kai Huang,Runze Xu,Hui Ren,Taiyi Wu,Xinyuan Li,Xun Gao,Quande Liu,Dong Tang,Tongtong Shen,Xuefeng Su,Alan Zhao
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Technical report. 19 pages, 14 figures, 3 tables
Abstract:Filming a scene from an additional angle after production usually requires a costly reshoot. We present One Shot Any View, a system that takes a recorded clip and generates a reshot video of the same performance along a user-specified camera trajectory. As in recent geometry-grounded methods, the system conditions generation on the source video and an explicit 3D prior, namely a point video rendered along the requested trajectory from a feed-forward reconstruction of the source video. We target human-centric cinematic videos, which are often shot with long lenses, shallow depth of field, and low-key lighting, all of which degrade feed-forward reconstruction. Existing geometry-grounded methods tend to inherit the degraded point video, and their output quality drops visibly. Two designs make such point videos usable. First, in addition to rendered videos of 3D assets, we train on real-world multi-camera videos, whose point videos are produced by the same reconstruction pipeline as at inference. This training-inference consistency enables the generator to take only the composition of people and scene elements from the point video, and their appearance from the source video. Second, a geometry-anchored RoPE offset keeps the generated video temporally synchronized with the source while largely preventing the output from copying the degraded appearance of the point video. Meanwhile, since the generator is a pretrained reference-to-video (R2V) model, the system also accepts optional character and scene reference images, which supply the appearance of regions that the source never shows. In both objective and human evaluations, One Shot Any View outperforms existing reshooting methods.
[CV-71] Allocentric Memory Egocentric Acting: A Dual-Frame VLA for Urban Aerial Navigation
链接: https://arxiv.org/abs/2611.24247
作者: Haotian Xu,Yue Hu,Zhengqiu Zhu,Tuo Feng,Wenguan Wang,Quanjun Yin
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注:
Abstract:Language-goal aerial navigation requires a UAV to reach a destination specified in natural language using observations collected during flight. A key challenge is linking changing egocentric perception with spatial evidence anchored to stable locations for long-range navigation. We propose CityNav-VLA, a dual-frame vision-language-action model for urban navigation integrating dynamic egocentric context with persistent allocentric spatial memory. The UAV incrementally accumulates visual observations on a fixed north-up map initialized with geographic and landmark priors, preserving evidence across viewpoint changes. A dynamic query conditioned on the instruction, current observations, UAV state, and motion history retrieves relevant memory evidence to predict a map-frame target. A deterministic controller converts this target into body-frame commands using the UAV’s current pose, connecting global reasoning with egocentric execution. To reduce target fluctuations over long horizons, early goal commitment (EGC) fixes the execution target to the coordinate-wise median of the first K predictions without extra trainable parameters. On Test Unseen, CityNav-VLA reduces NE by 21.2% and improves SR and SPL by 15.8% and 30.9%, respectively. It also reduces NE by 6.2% and improves SR and SPL by 5.6% and 23.2%, respectively, relative to the former SOTA after dataset refinement. Cumulative ablations show that navigation-map features yield the largest gains, while metric coordinates further improve endpoint accuracy. With a 4B-scale backbone, inference averages 0.68 s per decision with 8.6 GB peak memory on a single A100, presenting remarkable model efficiency.
[CV-72] End-to-End Landsat Land Surface Temperature Reconstruction with AlphaEarth Embeddings
链接: https://arxiv.org/abs/2611.24243
作者: Shengjie Liu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Land surface temperature (LST) data at Landsat’s 30 m resolution is important for many applications, but cloud contamination prevents its spatially continuous observations. Here, we propose A ^2 E ^2 , an end-to-end U-Net that integrates annual temperature cycle (ATC), coarse LST from Earth system modeling, and AlphaEarth embeddings for Landsat LST reconstruction. The ATC supplies a physics-guided thermal prior, the coarse LST provides a temporal background, and the U-Net converts the embedding field into day-specific fine-scale correction using observed pixels. Experimental results on four seasonally distributed Landsat 8/9 scenes over New York City in 2022 show that A ^2 E ^2 achieves 0.78 K mean absolute error and 1.01 K root mean squared error, reducing error by 60.1% relative to baseline method without spatial correction and by 3.7% relative to the same U-Net guided by annual mean spectral reflectance. Sensitivity analysis shows stable performance regardless of distance to clear pixels and percentage of blocked pixels. Independent validation with ECOSTRESS LST shows that Earth foundation model embeddings have better performance than annual mean spectral bands for LST reconstruction.
[CV-73] ViLFlow: Parallel Vision-Language Generation with Continuous Latent Flows
链接: https://arxiv.org/abs/2611.24240
作者: Kevin Zhai,Bian Sun,Zhenyi Wang,Song Wang
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Vision-language generation has predominantly relied on autoregressive and discrete diffusion models. Autoregressive models generate tokens sequentially, while discrete diffusion models typically predict tokens independently within each denoising step. We propose ViLFlow (Vision-Language Flow), a continuous-flow vision-language model that combines joint modeling of output embeddings with parallel token decoding. ViLFlow adapts pretrained continuous language generation to visual question answering and image captioning through learned image guidance and prediction-aware token decoding. A visually adapted teacher provides image-guided predictions as additional denoising supervision. The denoiser learns to reproduce these predictions directly, retaining one image-conditioned prediction per generation step. We train the token decoder on denoiser predictions and paired reference text to improve the conversion of continuous outputs into words. The decoder can be adapted independently, improving answering and captioning without further denoiser training. For captioning, it also supplies text feedback during late denoising, allowing decoded words to inform subsequent refinement. ViLFlow outperforms autoregressive and masked-diffusion baselines by up to 8.46 percentage points on VQAv2 and 4.50 points on GQA without GQA training. On COCO Karpathy test, it improves CIDEr by 16.25 over masked diffusion and 17.81 over sampled autoregression.
[CV-74] Psychometrics-Inspired Coreset Selection for Image Quality Assessment
链接: https://arxiv.org/abs/2611.24233
作者: Susu Jiang,Xuelin Shen,Baoliang Chen
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Collecting large-scale subjective annotations remains a critical bottleneck in training accurate image quality assessment (IQA) models. Coreset selection reduces this burden by selecting representative images before annotation. However, diversity in a visual feature space does not necessarily reflect diversity in perceptual quality, raising a fundamental question about which quality-related factors the representation should encode. We propose GRAM, a quality-aware representation learning framework inspired by the graded response model (GRM) from psychometrics. GRM describes how an individual’s latent ability determines the probabilities of reaching successive assessment grades, with response sensitivity and grade thresholds governing this relationship. In this paper, we find that this principle naturally aligns with perceptual quality assessment, where the impact of image degradation depends on image content and distortion characteristics. Building on this connection, GRAM models quality judgments through three complementary factors: latent quality, content-dependent response sensitivity, and distortion-dependent grade thresholds. These factors jointly determine the probabilities of attaining different quality grades and the resulting expected quality score. The learned parameters provide a structured quality-aware representation that complements general visual features for coreset selection. Experiments on nine datasets demonstrate improvements across different selection strategies, downstream IQA models, and annotation budgets. The code will be made publicly available.
[CV-75] raining Denoising Models of Any Architecture Without Stored Images
链接: https://arxiv.org/abs/2611.24223
作者: Christos Koutlis,Ioannis Kompatsiaris,Symeon Papadopoulos
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Can a pretrained denoising model replace the dataset used to train it? We investigate this question through teacher-supervised training, showing for the first time that continuous-time diffusion and flow models of any architecture and size can be trained from random initialization without stored images. Our students regress the teacher’s velocity on a rolling pool of noisy samples, advanced along trajectories of random step counts to obtain independent, dense and approximately continuous noise levels. This saves considerable storage for large datasets and applies even when the data are unreleased or private. Our experiments include 12 teachers of 6 families and 6 datasets. Our method outperforms standard training of the same network on CIFAR-10 for students 4x smaller than the teacher, comes within 0.75 to 1.92 FID of it at the teacher’s size and surpasses training on teacher-generated images and step reduction methods without teacher-based initialization. Our code is available at this https URL.
[CV-76] arget-Consistent Action Guidance via Semi-Unbalanced Optimal Transport for Drifting World Models
链接: https://arxiv.org/abs/2611.24222
作者: Md Abtahi Majeed Chowdhury,Md Rifat Ur Rahman,Md Sakib Hossain Shovon,Jaesik Choi,Minhyuk Sung
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Action-conditioned world models let robots imagine the outcomes of their actions for planning and offline policy evaluation, but these uses need many rollouts, which multi-step diffusion makes expensive. Drifting models move the iteration from inference to training and generate samples in one network pass. DriftWorld brought this paradigm to action-conditioned world models; to strengthen action following, it repels predictions from the current frame, a construction borrowed from classifier-free guidance. We show that this accentuation has a target bias: the push persists even when a prediction matches the recorded future, in kernel, balanced, and unbalanced Wasserstein drifting fields alike. We introduce Mass-Share Accentuation (MSA) to remove this bias. MSA treats the current frame as a counterfactual destination competing with the recorded future in a semi-relaxed optimal transport problem; subtracting the closed-form share of the correct future makes the guidance vanish at the target, and the remainder is shared by all predictions. We prove that the recorded future is the unique fixed point along the motion direction for collapsed predictions, and that the guidance fades with prediction error. On Push-T, Robomimic, and BridgeV2, where trained models under-react rather than overshoot, MSA improves action ranking, response slope, and moving-region error over latent-space baselines at no inference cost, though these benchmarks do not isolate bias removal as the cause.
[CV-77] DetailScale-SR: Decoupling Restoration and Continuous Detail Control for Real-World Super-Resolution
链接: https://arxiv.org/abs/2611.24212
作者: Shengchuan Gao,Zhongjie Duan,Yingda Chen,Jingjing Chen
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Real-world image super-resolution often seeks the most detailed plausible reconstruction, although more detail is not always desirable. The same degraded image may require restrained restoration, faithful reconstruction, or enhanced visual detail according to user intent. We present DetailScale-SR, a preference-conditioned framework that continuously controls reconstructed detail for a fixed low-resolution input without altering the scene being restored. Rather than relying on post-processing or repeatedly rewriting prompts, it learns a continuous family of reconstructions while separating content recovery from detail preference. This enables a smooth transition from conservative restoration to richly textured results under consistent input and semantic constraints. Its controllable range also extends beyond conventional restoration: DetailScale-SR can enhance high-frequency structures and generate perceptually richer textures than the paired high-resolution reference. Text restoration is another distinct strength. For text-rich scenes, especially Chinese posters and signs, it recovers more legible and character-faithful text than prior generative SR methods under matched textual guidance. DetailScale-SR therefore unifies continuous detail control, beyond-reference enhancement, and reliable recovery of semantically important content. Our model is available at this https URL
[CV-78] SpiderMark: Hybrid Watermarking for Generative AI via Algorithmic Injection and Learning-Based Verification
链接: https://arxiv.org/abs/2611.24200
作者: Chihcheng Hsieh,Yang Song,Flora Salim,Martin Bell,Jacinto C. Nascimento,Catarina Moreira
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:The proliferation of diffusion-based generative models has intensified the need for reliable methods to trace and verify the provenance of AI-generated images. Existing watermarking approaches for diffusion models typically rely on isotropic frequency patterns and handcrafted detection metrics, which are prone to directional vulnerability and threshold instability across diverse image transformations. We propose SpiderMark, a hybrid watermarking framework that combines algorithmic frequency-domain injection with learning-based probabilistic verification, requiring no modifications to pretrained diffusion model weights. SpiderMark embeds a structured spider-web pattern, comprising radial spokes and concentric polygonal rings, into the Fourier representation of the initial latent noise, and trains a lightweight ResNet-18 verifier augmented with auxiliary similarity cues (PSNR and L1) to produce a continuous watermark confidence score for watermark detection. Across nine evaluation conditions, SpiderMark achieves the highest fixed-threshold Accuracy in eight, while retaining strong detection on unseen COCO prompts without verifier retraining. CLIP similarity and FID measurements show that these detection gains are accompanied by small measured differences in prompt alignment and distributional image quality. SpiderMark establishes that structured watermark geometry and learned probabilistic verification can be unified into a single, practical framework, bridging the gap between imperceptibility, robustness, and real-world verifiability for generative AI content attribution.
[CV-79] From Neural Tokens to Language: Dual-View EEG Semantic Bridging for Brain-to-Text Generation
链接: https://arxiv.org/abs/2611.24197
作者: YaoYang Liu,Jun Dai,ZhuoShi Tang,Weijun Sun,Xiaozhao Fang,Jie Wen
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 30 pages, 15 pages of supplementary material
Abstract:Visually evoked EEG-to-text generation aims to produce natural-language descriptions from neural responses elicited by visual stimuli. Existing approaches typically compress EEG signals into a single global representation and directly project it into the hidden space of a large language model (LLM), which may limit the preservation of complementary visual semantics under noisy and subject-dependent EEG signals. We propose a dual-view semantic token bridging framework that exploits raw images and sketch images as complementary visual supervision to learn raw-oriented and sketch-oriented EEG semantic token streams. To interface these structured neural representations with an LLM, we introduce a branch-specific semantic query bridge that aggregates each token stream into compact semantic slots and converts them into LLM-compatible soft prompts. A progressive adaptation strategy further transitions the learned visual-semantic interface from image-supported training to image-free EEG-conditioned generation. Experiments demonstrate strong subject-wise EEG classification performance and competitive EEG-to-text generation across multiple LLM backbones. Ablation, branch-level representation, and EEG-label disentanglement analyses further show that the two EEG streams learn complementary view-oriented representations and that EEG-derived prompts provide sample-specific information beyond category labels.
[CV-80] WMMLLM : Turning MLLM s into World Models
链接: https://arxiv.org/abs/2611.24193
作者: Baiang Li,Enze Xie,Jincheng Yu,Jin Wang,Haozhe Liu,Yitong Li,Juanxi Tian,Felix Heide
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 25 pages, 14 figures, 8 tables; project page: this https URL
Abstract:Video world models keep the world in the context of a generator, so a revisit is synthesized again and earlier edits can drift or disappear. We present WM^MLLM, the first interactive world model built on multimodal large language models (MLLMs), for persistent, editable world generation from a single image. WM^MLLM keeps the world state outside every generator, as a panorama with a revision history, and lets the MLLMs write it: they turn each request into operations on this state, select where a local edit is written, and check every candidate before it becomes the next state. Image generators only propose views, and video only presents them. Users can walk to new places and return, edit objects through text or a box, place subjects, and change lighting, weather or season. By construction, a revisit at the same revision reproduces the earlier view exactly, a local edit leaves every panorama pixel outside its region unchanged, and each request starts from the result of the previous one. Against six world model baselines on MBench-A, WM^MLLM has the highest mean on all four environment-recovery metrics (Epipolar 91.5 versus 0.1 to 74.0) and completes the most camera round trips (85.3% versus at most 50.9%). It has the best value in all 15 CompBench columns against 15 baselines. On 112 local edits judged blind, selecting the region to write after generation raises completion from 34.8% to 83.0%. On 258 WBench and MBench-A items, all 46 revisits recover their earlier views, and all 97 local edits that pass the check leave panorama pixels outside their region unchanged.
[CV-81] rajWAM: Trajectory-Conditioned Future Learning for Vision-Language Driving
链接: https://arxiv.org/abs/2611.24192
作者: Xuerui Fang,Gulimila Kezierbieke,Chen Wangxing,Hui Li
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注: 22 pages, including supplementary material. Code and model weights: this https URL
Abstract:World Action Models (WAMs) connect future scene prediction with action planning for autonomous driving. Each driving record pairs an executed ego trajectory with its observed future, providing a factual outcome for learning this connection. We present TrajWAM, a trajectory-conditioned WAM that improves vision-language driving through auxiliary bird’s-eye-view (BEV) future learning. An ego-trajectory query conditions DINOv3-aligned BEV predictions from shared planner representations. Training combines factual reconstruction under the executed trajectory with one-sided feature separation under an unlabeled perturbed trajectory. Together, these objectives train the shared planner to generate driving waypoints. Native inference directly generates waypoints and bypasses the auxiliary future-prediction branch. On all 220 Bench2Drive routes, TrajWAM achieves 91.53 Driving Score and 75.45% Success Rate, exceeding a schedule-matched planning-only fine-tuning baseline by 5.31 Driving Score points and eight successful routes.
[CV-82] Monitoring Unplanned Construction Using Satellite Change Detection
链接: https://arxiv.org/abs/2611.24190
作者: Harsh Kashyap,Taranum Wasu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 22 pages, 11 figures, 5 tables
Abstract:Cities need land for homes, shops, roads, and public services. New construction changes that land. It can harm natural areas and water bodies, add to waste, and strain public services. NITI Aayog’s 2021 report stated that 65% of 7,933 urban settlements in India had no master plan. That number tells us about planning coverage at the time. It does not tell us whether any single building was authorised. Artificial intelligence, which lets computers learn patterns from data, can help find physical changes by comparing satellite images taken on different dates. We test simple methods on two public datasets from outside India. First, we turn each image into a set of numbers using a pretrained neural network, a model already trained to recognise patterns in images. We feed those numbers to standard machine-learning classifiers that predict land categories and physical changes. The methods reach about 95 percent accuracy at spotting built-up land and about 83 percent accuracy at saying whether a small pair of images shows change, where each pair gets one label. We also train a small neural network on a standard change-detection benchmark. Separately, we run a preliminary descriptive check on an Agra sample of 48 patches, 7 of them positive. The results show that satellite images carry information about physical change. They do not show whether construction was planned or permitted.
[CV-83] VideoACE: Benchmarking Agent ic Video Understanding through High-Frame-Rate Actions
链接: https://arxiv.org/abs/2611.24184
作者: Luoyuan Zhang,Xuanyu Zheng,Yancheng Long,Haoyang Xu,Bin Wen,Tingting Gao
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)
备注:
Abstract:Agentic video understanding requires models to actively interact with video environments to gather evidence and form judgments. Evaluating this capability therefore requires assessing whether models can translate opportunities for interaction into better understanding. We introduce VideoACE, a benchmark for assessing video understanding under conditions that support sustained interaction. We focus on high-frame-rate action videos, where rapid, subtle changes make targeted observation especially valuable. The benchmark comprises 1,000 multiple-choice questions spanning six categories of fine-grained action understanding, accompanied by temporal evidence annotations and user hints screened for answer leakage. Its evaluation environment combines video inspection tools with a user simulator, allowing models to adjust temporal coverage, sampling density, and spatial detail, and to revisit evidence and revise their judgments under simulated user feedback. We evaluate single-pass observation, autonomous tool use, and tool use combined with user feedback to examine how effectively models exploit these opportunities. Evaluation of five models reveals distinct patterns in how they acquire evidence and respond to opportunities for further interaction. These findings establish VideoACE as a testbed for examining agentic video understanding beyond performance under a fixed observation setting.
[CV-84] ART: Bridging the Embodiment Gap in 3D Human-to-Robot Affordance Transfer
链接: https://arxiv.org/abs/2611.24182
作者: Zichen Zhang,Wei Zhai,Xuji Zhang,Yang Cao,Yu Kang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 15 pages, 14 figures, 5 tables. Project page: this https URL
Abstract:3D affordance acts as a critical link between perception and action, providing priors for robotic manipulation. While existing approaches predominantly leverage human priors as direct references, they often overlook that affordance is not an object-intrinsic property but is inherently coupled with the agent’s embodiment. Due to the structural and morphological disparities between human hands and robot end-effectors, directly applying human priors causes severe incompatibility for robotic execution. Motivated by this, we introduce a novel task setting, 3D Human-to-Robot Affordance Transfer, which addresses the affordance misalignment arising from distinct embodiment structures. To support this task, we construct the Human-Robot Paired Affordance Dataset (HPAD), the first large-scale 3D affordance benchmark in which every object is annotated with both human-prior and simulation-validated robot labels. HPAD covers 2,137 objects across 80 categories and 13 atomic action primitives, yielding over 27,000 object-action pairs. On top of HPAD, we propose the 3D Affordance tRansfer neTwork (ART). ART first filters geometric false positives in the human prior via a reliability gating stream, then reconstructs the prior into a robot-actionable distribution through hierarchical reconstruction and a graph-diffusion refinement that resolves the topological mismatch between human and robot affordances. Experiments on HPAD demonstrate the reliability of the proposed task and the superiority of ART. We further validate ART on a 6-DoF arm with a parallel-jaw gripper in a 2,400-execution benchmark across 12 objects and 4 action categories; ART attains the best success rate (86.5% over its 480 trials), confirming the practicality of the transfer paradigm. Project page: this https URL
[CV-85] Efficient World-Model Planning via Physics-Guided Search Distillation
链接: https://arxiv.org/abs/2611.24171
作者: Hongyu Zhang,Xiaoshuai Hao,Heye Huang,Qingwen Meng,Wenjun Shi,Jianao Xu,Xin Hu,Xuanru Jin,Jingyi Zhang,Baicang Guo
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:World models support autonomous driving by evaluating candidate trajectories through predicted future scene evolution. However, accurate future prediction does not ensure road-compliant execution; when all candidates violate road boundaries, re-ranking cannot select a feasible trajectory. We propose Physics-Guided Search Distillation (PGSD), a single-view planning framework that combines future representations and execution feedback for verified beyond-pool repair through three components: (1) Execution-Aware Selection checks ego-vehicle road compliance under simulated execution and selects the highest-scoring feasible trajectory; (2) a Clearance-Aware Search Teacher generates beyond-pool corrections and selects demonstrations with larger minimum road-boundary clearance during execution to buffer imitation errors; and (3) a Future-Conditioned Repair Student predicts one correction from rewarder features and execution feedback for single-rollout verification. The pretrained planner and rewarder remain frozen; no future scenes are generated online. On NAVSIM v1, single-view PGSD achieves 90.97 PDMS, exceeding all compared prior single-view planners and retaining 73.2% of the search teacher’s PDMS gain over execution-aware selection. On NAVSIM v2 navhard, it reaches 38.96 EPDMS, improving on WorldDrive by 11.24 points (40.5%). The student replaces up to ten search rollouts with a single verification rollout per repair trigger. HUGSIM Hard examples provide qualitative evidence of zero-shot transfer.
[CV-86] ViStep-RL: Stepwise Reinforcement Learning for Long-Horizon Visual Agents
链接: https://arxiv.org/abs/2611.24170
作者: Junyang Chen,Yao Qin,Xiao Li,Ningchao Wang,Yiguo Qiao,Wenbin Li,Xin Geng
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 19 pages, including supplementary material
Abstract:Long-horizon visual agents must do more than inspect images: they must turn observations into evidence that remains usable as a task unfolds. Existing active-perception methods teach agents where and how to look, while a final-answer reward gives little structure for deciding which observations satisfy dependent subgoals or should be retained for later use. We present ViStep-RL (Visual Stepwise Reinforcement Learning), which models execution as a dependency-linked graph of evidence-bearing subgoals. A verifier tracks which subgoals are supported by valid evidence, and changes in graph potential provide a net-progress learning signal; an active memory protocol preserves evidence across turns and links retrieval to downstream use. On ViInteract, ViStep-RL reaches 60.0% strict completion with Qwen2.5-VL-7B, compared with 10.0% for the base model and 46.5% after supervised fine-tuning. Controlled studies show that incremental task-state rewards outperform cumulative rewards by 3.2 points, while active evidence writing improves strict completion by 7.75 points over automatic writing. Under a common interaction framework, the task-trained ViStep-RL policy achieves 59.5% answer accuracy, 24.0 points above the strongest of six general-purpose reference models. Held-out episodes confirm the gain at both Qwen scales, and gains extend to three external visual reasoning benchmarks.
[CV-87] Gaussian Primitive Fusion for Any-Resolution Multimodal Images
链接: https://arxiv.org/abs/2611.24164
作者: Baoshuo Cai,Xinrui Jiang,Bing Cao
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 15 pages, 8 figures
Abstract:Multimodal image fusion integrates complementary observations from different modalities into a unified representation. Existing methods typically estimate modality contributions on a common discrete grid. When source modalities have different native resolutions, constructing such a grid generally involves resampling, which may introduce artifacts and affect subsequent fusion decisions. To address this problem, we propose Gaussian Primitive Fusion (GPF), which reformulates heterogeneous-resolution fusion in a shared continuous Gaussian space. Instead of first converting source images to a common resolution, GPF directly fits their native-resolution observations with shared Gaussian geometry and modality-specific content representations, and learns modality contributions within the resulting continuous representation. Once optimized, the fused representation is rendered at the finest native resolution without resampling the sources. With heterogeneous-resolution inputs, GPF produces clearer target contours and finer textures, with less blurring and blocking artifacts than resample-then-fuse pipelines. Quantitatively, it achieves the best results on multiple infrared-visible benchmarks against state-of-the-art fusion methods equipped with interpolation or super-resolution frontends.
[CV-88] Stabilizing Neural-to-Image Retrieval with Frequency-Domain Priors and Adaptive Supervision
链接: https://arxiv.org/abs/2611.24163
作者: YaoYang Liu,ZhuoShi Tang,Haopeng Lai,Weijun Sun,Xiaozhao Fang,Jie Wen
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 12 pages
Abstract:Visual neural decoding aims to infer perceived visual stimuli from brain activity, offering important insights into human visual cognition while enabling transformative applications in brain computer interfaces and artificial intelligence. Recent advances in pretrained vision-language models have provided a semantically rich representation space for this task. However, reliable alignment between EEG/MEG signals and visual embeddings remains challenging because neural responses are noisy, highly variable across trials, and subject dependent. To address this issue, we propose a unified neural-to-image retrieval framework that improves retrieval performance by aligning neural and image representations in a shared semantic space. The framework integrates dual-stream adaptation, frequency-domain calibration, and phase-guided temperature modulation to promote more robust cross-modal semantic alignment. Specifically, dual-stream adaptation models a low-dimensional shared subspace with correlation regularization to suppress redundant noise and enhance cross-modal consistency. Frequency-domain calibration conditionally modulates neural representations using amplitude and phase statistics. Phase-guided temperature modulation further adjusts the supervision strength in contrastive learning according to the stability of neural responses. Experiments on THINGS-EEG and THINGS-MEG under both intra-subject and inter-subject settings demonstrate that the proposed framework consistently outperforms strong baselines. These results suggest that robust neural-to-image retrieval under heterogeneous neural conditions benefits from a stable shared semantic space, frequency-informed representation calibration, and adaptive supervision. Code will be released at this http URL.
[CV-89] AVTR: Adaptive Multimodal Reasoning with Visual Imagination
链接: https://arxiv.org/abs/2611.24161
作者: Xiaoqian Shen,Yunyang Xiong,Changsheng Zhao,Saksham Suri,Chong Zhou,Zechun Liu,Zhipeng Cai,Zhuang Liu,Qi Qian,Wei Wen,Hongyu Xu,Hanxian Huang,Ernie Chang,Yanbei Chen,Vikas Chandra,Raghuraman Krishnamoorthi,Mohamed Elhoseiny,Lemeng Wu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:Vision-Language Models increasingly rely on textual reasoning to solve advanced multimodal tasks. Despite its effectiveness, this paradigm faces two fundamental limitations. First, relying on textual reasoning alone can limit perceptual ability in vision-centric questions, where accurate answers depend on interpreting task-relevant visual evidence. Second, applying textual reasoning uniformly across inputs overlooks question-dependent variation in both reasoning modality and depth, often producing verbose and unnecessary traces when little logical inference is required. To this end, we propose Adaptive Visual-Textual Reasoning (AVTR), which introduces visual imagination as a complementary reasoning modality to textual thinking, enabling the model to revisit its visual observations and distill the salient evidence through generative visual reflection. Furthermore, we formulate reasoning-modality selection as a learned routing problem, allowing a single policy to adaptively harness the complementary strengths of visual reflection and textual reasoning in response to the perceptual and inferential demands of the input. Across 16 benchmarks, AVTR improves the average score by 4.5 points over Qwen3.5 baseline, while reducing inference latency by 33%. It also scales effectively into a smaller size backbone, outperforming the evaluated 2B-scale baselines.
[CV-90] GenRF: Generative Registration-free Image Fusion
链接: https://arxiv.org/abs/2611.24157
作者: Linfeng Tang,Zhijun Fan,Yang Chen,Zizhuo Li,Hao Zhang,Jiayi Ma
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:Multi-modal image fusion combines complementary observations to improve scene visibility and perception. However, in real-world acquisition, differences in sensing principles and configurations introduce geometric discrepancies, while degradations further compromise source quality. Existing methods typically rely on explicit registration, yet large parallax complicates correspondence estimation, while mismatched fields of view leave some regions without cross-modal counterparts. Meanwhile, independent restoration underuses cross-modal information, whereas restoration-fusion methods assuming alignment remain vulnerable to misregistration. This study revisits misaligned and degraded image fusion in the era of generative foundation models, in which cross-attention enables position-agnostic complementary information aggregation, while generative priors support degradation recovery and inference in missing regions. Building on this perspective, we propose GenRF, which adapts pretrained FLUX through LoRA to jointly enhance degraded observations, infer missing cross-modal context, and fuse complementary information without explicitly estimating deformation fields. Extensive experiments on real-world infrared and visible captures show that GenRF delivers high-quality fusion directly from unregistered, degraded images without preprocessing, while improving downstream task performance under misalignment. Additional experiments demonstrate that GenRF can generalize to misaligned multi-exposure and multi-focus image fusion without task-specific fine-tuning. Project: this https URL.
[CV-91] Game2Player: Scaling Gameplay Experience for Visual Agents through Environment-Player Co-Development
链接: https://arxiv.org/abs/2611.24154
作者: Xirui Jiang,Yuxi Chen,Rui Yang,Feng Luo,Pengcheng Wang,Bingxuan Li,Zihao Zhao,Yushan Li,Han Wang,Manling Li,Yang Zhang,Shiyu Chang,Tong Zhang,Huan Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:Games are a rich testbed for visual agents, demanding perception, long-horizon planning, and precise control. However, collecting human gameplay is costly, and learning from gameplay videos requires inferring missing actions. Existing datasets also cover a limited range of games, often without explicit tasks or verifiable outcomes. To address these gaps, we introduce Game2Player, a framework that co-develops game environments and players to collect diverse and verifiable gameplay experience at scale. In the Game2Player loop, a Builder agent adapts existing web games, exposes their internal state, and defines tasks with state-based progress measures and success checks. A Coding Player and a VLM Player then attempt the tasks and report problems to the Builder for repair, so that play serves as both environment validation and training data collection. With Game2Player, we build 568 verified web games with 3,436 tasks, collect 15.7K trajectories (307 hours of gameplay), and hold out 50 games as the benchmark PlayerBench. To evaluate the collected experience as training data, we train VLM agents with progress-weighted SFT, assigning greater weight to steps that advance task progress. Game2Player-9B improves progress from 14.3% to 31.2% on PlayerBench and from 30.0% to 53.1% on the external GameWorld benchmark. More broadly, our work suggests a path for visual agents to scale their gameplay experience by building, verifying, and learning from existing game worlds.
[CV-92] SEER: Semantic Evidence for Executable Region Propagation in Open-Vocabulary Semantic Segmentation
链接: https://arxiv.org/abs/2611.24152
作者: Junyang Chen,Xiao Li,Yiguo Qiao,Wenbin Li
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 33 pages, including supplementary material
Abstract:Training-free open-vocabulary semantic segmentation (OVSS) has made strong progress by adapting CLIP-like representations to dense prediction, but its remaining errors are often local, structured, and tied to the active label space. A successful correction must be visually plausible, legal under the target vocabulary, and executable in the segmentation score field. We propose SEER, a training-free test-time repair layer for Semantic Evidence and Executable Region propagation. SEER casts post-prediction OVSS repair as evidence-conditioned label propagation. The OVSS score space proposes compact candidate labels, a frozen VLM verifies closed-set regional evidence, and an execution gate certifies whether the evidence is legally and semantically actionable. The accepted label is then propagated by an OVSS-side policy: coherent high-confidence regions allow broad propagation, while object- or boundary-risk cases use conservative score-grounded propagation. Across eight standard OVSS benchmarks, SEER reaches 58.0 average mIoU without updating OVSS weights. Matched-input controls isolate the repair mechanism, while resource measurements report the cost of the complete frozen system.
[CV-93] Inherited Adapter Experts for Replay-Free Domain-Incremental Learning
链接: https://arxiv.org/abs/2611.24145
作者: Gyutae Oh,Hyunmin Kong,Jitae Shin
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 21 pages (8 pages main text, plus references and supplementary material), 8 figures, 23 tables
Abstract:Replay-free domain-incremental learning trains a model on the same classes in successive domains without keeping earlier images. Per-domain learners freeze a pre-trained ViT and train an additional expert for each new domain: the small set of parameters and the classification head learned on that domain. Test images carry no domain label, so a router selects one expert for each image. Recent work trains this router. However, a training-free Mahalanobis router costs the analyzed learners under 2.7 points of final accuracy relative to always selecting each image’s own-domain expert. The experts, not this router, limit accuracy. Our learner, Lamarck, therefore uses this router and strengthens the experts. Its adapters, small trainable layers that correct every image token after each deep ViT block, add 4.4 points of final accuracy on the deepfake benchmark CDDB-Hard. In addition, each expert is initialized from its predecessor at no extra cost: the domains share their classes, so an image misrouted to a later expert loses less. Against twelve frozen-backbone baselines under one protocol, Lamarck is first on CDDB-Hard by 5.2 points, second on DomainNet and OfficeHome, and highest in mean accuracy.
[CV-94] Ego2X: Contact-aware Transfer from Egocentric Human Data to Any Robot
链接: https://arxiv.org/abs/2611.24138
作者: Xinke Lyu,Bowen Yang,Zhicheng Wang
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages, 7 figures
Abstract:Embodied models depend on real interaction data, yet collecting it on physical robots is slow and labor-intensive. Recent ego-to-robot pipelines retarget first-person view human videos to robot data via transferring hand and arm motion. They judge fidelity mainly by visual appearance. However, trajectories often fail in terms of physics: fingers hover beside objects, and grippers close on empty space. We argue that how the object moves and the contacts that drive the motion count. Thus, we present Ego2X, an object-centric framework that converts monocular egocentric video into physically executable robot data across dexterous hands and parallel grippers. Ego2X centers on CORE (Contact-centric Object-interaction REpresentation), which records each contact on the object by its surface anchor, as well as relative force, grasp mode, and interaction state. Throughout the pipeline, CORE corrects hand-object drift via object-relative constraints, steers inverse kinematics toward contacts the robot must realize, and preserves held objects during arm erasure and robot compositing. On HOT3D, contact-driven correction reduces in-contact hand depth error by 33.7%, while on EgoDex, contact-driven retargeting realizes 62.9% of annotated contacts, surpassing the official HuRo pipeline (27.9%). In RoboTwin 2.0, post-training X-VLA on Ego2X episodes alone, without simulator demonstrations, raises task success rate from 40.5% zero-shot to 67.0% across 200 paired scenes. We will publicly release our conversion pipeline and CORE-annotated dataset for dexterous hands and parallel grippers.
[CV-95] PatchAdaNet: Unsupervised class incremental learning using local patches and global adapters
链接: https://arxiv.org/abs/2611.24137
作者: Kamel Aouaidjia,Yanchao Duan,Zihui Zhen,Xin Li,Yongfei Lv,Chongsheng Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Biological learning systems acquire knowledge incrementally through broad, reusable representations established early and kept largely stable, while integrating new experience with minimal interference with previously learned knowledge. Replay-based methods approximate this by storing samples or features from previous tasks and mixing them into the current task. However, memory buffers grow with the number of tasks and may retain task-specific noise that reduces generalization. To address these limitations, we propose PatchAdaNet (Patch Adapter Network), an unsupervised class-incremental learning method for image classification that continually updates feature representations using autoencoders. Since images share structural features such as edges, corners, and shapes, we first divide each image into patches and train independent autoencoders on external data to reconstruct each patch. The layers of all patch encoders are aggregated into global frozen layers that generalize across tasks and remain fixed throughout learning. Learnable adapters then build global image representations from these features to reconstruct the entire image. To prevent feature drift without storing data, we introduce the Global Feature Centroid (GFC), a fixed-size running average of adapter outputs updated after each task to penalizes representation drift. Elastic Weight Consolidation (EWC) is applied to the learnable layers to preserve previously learned features by constraining important weights. Finally, a linear classifier is trained on the frozen latent representation and GFC for class-incremental evaluation. We evaluate PatchAdaNet in both CNN and Vision Transformer settings on CIFAR-10, CIFAR-100, miniImageNet and 5-Datasets, providing a comparative study of the two architectures under identical class-incremental settings. Code is available.
[CV-96] How Well Do Generalist CT Foundation Models Differentiate Liver Tumors?
链接: https://arxiv.org/abs/2611.24122
作者: Yosuke Yamagishi,Tomohiro Kikuchi,Shouhei Hanaoka,Takeharu Yoshikawa,Osamu Abe
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 12 pages, 3 figures. Code: this https URL
Abstract:Generalist CT foundation models detect many findings with high performance, but their ability to differentiate detected lesions remains largely unevaluated. This retrospective study evaluated two generalist models, RADAR and Merlin, for differentiating liver tumors using a public single-institution multiphase CT dataset (517 cases: hepatocellular carcinoma, intrahepatic cholangiocarcinoma, breast and colorectal cancer liver metastases, and hemangioma). The models were evaluated in a zero-shot setting using their existing outputs and with linear probing, in which only a linear classifier was trained on frozen image features (stratified fivefold cross-validation repeated five times). In the zero-shot setting, RADAR achieved an accuracy of 54.4% (95% CI: 49.9, 58.8) and a \kappa of 0.37 for four-class differentiation at the portal venous phase and predicted 31% (129 of 421) of malignant tumors as hemangioma. With linear probing of four-phase features, its accuracy improved to 79.9% (95% CI: 76.6, 83.4) with a \kappa of 0.72, and 1% (five of 421) of malignant tumors were predicted as hemangioma; Merlin achieved an accuracy of 60.9% (95% CI: 56.9, 65.0). Generalist CT foundation models showed limited zero-shot ability to differentiate liver tumors, which improved substantially with linear probing alone.
[CV-97] EVF-Bench: A Comprehensive Benchmark for E-commerce Visual Effects in Video Generation
链接: https://arxiv.org/abs/2611.24116
作者: Run Ling,Ke Cao,Runze He,Jie Gao,Ao Ma,Wei Feng,Zhanjie Zhang,Hao Wang,Jinghe Hu,Guibing Guo
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:E-commerce visual effects (EVF) are dynamic, product-centered visual presentations in e-commerce videos, often accompanied by promotional text. They play an important role in attracting viewer attention and conveying product information. However, general measures of visual quality and text relevance alone do not establish whether generated videos preserve product appearance, maintain readable text, and produce coherent, contextually appropriate effects. To address this issue, we introduce EVF-Bench, a dedicated benchmark spanning six evaluation dimensions: visual quality and text relevance as general dimensions, and motion, product consistency, text stability, and creative rationality as EVF-specific dimensions. To complement the benchmark and support task-aligned research and model adaptation, we further construct EVF-32K, a dataset of over 32000 curated image-text-video triples tailored to e-commerce scenarios. Using this suite, we systematically evaluate representative video generation models and their domain-adapted variants. The results reveal complementary strengths, with no evaluated configuration leading all six dimensions. Together, EVF-Bench and EVF-32K provide a task-oriented evaluation framework and a supporting data resource for e-commerce video generation.
[CV-98] SCOPE: Single-Cell Omics Prediction from Histology via RNA-Protein Prior Enhancement
链接: https://arxiv.org/abs/2611.24104
作者: Leyang Tao,Xuan Gong,Luyuan Xie,Arjun Beniwal,Yuqi Tan,Kun-Hsing Yu
类目: Computer Vision and Pattern Recognition (cs.CV); Genomics (q-bio.GN)
备注:
Abstract:Predicting the RNA and protein content of cells from routine HE images could extend spatial molecular profiling to existing slide archives. However, both HE-spatial proteomics and HE-spatial transcriptomics paired datasets are limited by the number of available markers and the insufficient availability of high-quality, cell-resolved data. We present SCOPE (Single-Cell Omics Prediction via RNA-protein prior Enhancement), a unified framework that leverages biologically complementary information between RNA and proteins to address the limited availability of single-omics supervision. Our key insight is that, although RNA and proteins represent distinct molecular modalities, their biological relationships provide valuable shared knowledge that can benefit single-omics prediction tasks. To this end, SCOPE introduces biologically informed biomarker priors to enhance shared gene-level representations, enabling knowledge transfer across RNA and protein modalities through gene-protein correspondence and molecular associations. By exploiting these complementary relationships, SCOPE enables both prediction tasks to benefit from a shared backbone enriched with cross-omics biological knowledge. Furthermore, its biomarker correlation design facilitates the prediction of unseen markers, extending spatially resolved molecular profiling beyond directly supervised targets. Across test datasets spanning 16 organs, SCOPE improves the average RNA Pearson correlation over the strongest baseline by 8.8%. On two external protein cohorts, it improves the average Pearson correlation over the strongest baseline by 9.7% for seen markers and 11.9% for unseen markers.
[CV-99] IDEA: Intent-Dominance Evidence Audit for Inference-Time Backdoor Detection in GUI Agents
链接: https://arxiv.org/abs/2611.24102
作者: Zhizhuo Kou,Sizai Hou,Jian Yang,Ge Sun,Junyu Luo,Wenkai Zhang,Sirui Han,Yike Guo
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Organizations increasingly deploy downloaded open-weight GUI agents. A backdoored agent passes acceptance tests on clean data and misbehaves only when a visual trigger appears on screen, turning an unvetted checkpoint into a supply-chain risk. However, existing sample-level detectors target image classifiers and CLIP-style encoders, leaving step-level backdoor detection for GUI agents unaddressed. On clean steps, the instruction drives the action; when an unreferenced screen region hijacks the step, the instruction loses control over the decision state or that region dictates it instead. We term this signature intent–evidence dominance inversion and propose IDEA (Intent-Dominance Evidence Audit) to detect backdoor activation at inference time for each step without trigger knowledge or attack data. Specifically, Intent Counterfactual Probing (ICP) measures how far redirected instructions move the decision state, and Region Causal Audit (RCA) quantifies how far ablating a candidate region moves it. A gated conformal score compares both displacements against clean steps at the same distance from the clean manifold, providing a finite-sample false-positive guarantee. Extensive experiments on Qwen2.5-VL-3B across five attack families show that IDEA consistently detects backdoor activation under a controlled false-positive rate and outperforms state-of-the-art detection methods, while localizing the trigger region for every alarm and adding only modest inference overhead.
[CV-100] RoughCut-Bench: A Benchmark for Automatic Video Rough-Cut with Large Vision-Language Models KDD2027
链接: https://arxiv.org/abs/2611.24098
作者: Ruisi Zhao,You Li,Sihan Chen,Aosong Cheng,Chixiang Ma,Sen Pei,Zhen Yang,Hehe Fan,Dongliang He,Yi Yang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted to KDD 2027. Project page is at this https URL
Abstract:Video rough-cut is a core stage in media production workflows, organizing raw media materials into an initial video cut. Despite recent advances in large vision-language models (LVLMs) for video-related tasks, automatic rough-cut remains underexplored, without a comprehensive benchmark for evaluation or mature workflows for producing executable cut plans. To address the evaluation gap, we introduce RoughCut-Bench, which covers two practical settings: (1) a visually driven setting over video cases, covering highlight detection and selection, source-video ordering, and adherence to user requirements when provided; and (2) a speech-driven setting over automatic speech recognition (ASR) transcripts, covering filler removal, expression deduplication, and low-value content trimming. Across these settings, the benchmark contains 101 visually driven cases, including 831 raw videos with 1,476 highlight annotations. It also includes 84 speech-driven cases, each consisting of an ASR transcript with annotated cleanup spans. The evaluation compares cut plans with these annotations in both settings and additionally uses a human-aligned agentic evaluator to assess rendered-cut quality in the visual setting. To apply LVLMs to automatic rough-cut, we further propose agentic workflows that guide them to produce structured cut plans from input materials. Experiments show that current LVLMs remain limited in producing expert-aligned cut plans. Our benchmark, code, and prompts are released at this https URL.
[CV-101] Object Pose: The Link between Weak Perspective Paraperspective and Full Perspective
链接: https://arxiv.org/abs/2611.24095
作者: Radu Horaud,Fadi Dornaika,Bart Lamiroy,Stephane Christy
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注:
Abstract:Recently, DeMenthon and Davis [4,5] proposed a method for determining the pose of a 3-D object with respect to a camera from 3-D to 2-D point correspondences. The method consists of iteratively improving the pose computed with a weak perspective camera model to converge, at the limit, to a pose estimation computed with a perspective camera model. In this paper we give an algebraic derivation of DeMenthon and Davis’ method and we show that it belongs to a larger class of methods where the perspective camera model is approximated either at zero order (weak perspective) or first order (paraperspective). We describe in detail an iterative paraperspective pose computation method for both non coplanar and coplanar object points. We analyse the convergence of these methods and we conclude that the iterative paraperspective method (proposed in this paper) has better convergence properties than the iterative weak perspective method. We introduce a simple way of taking into account the orthogonality constraint associated with the rotation matrix. We analyse the sensitivity to camera calibration errors and we define the optimal experimental setup with respect to imprecise camera calibration. We compare the results obtained with this method and with a non-linear optimization method.
[CV-102] UCLIR: Unsupervised Continual Learning using Independent Representations
链接: https://arxiv.org/abs/2611.24093
作者: Kamel Aouaidjia,Yanchao Duan,Zihui Zhen,Xin Li,Yongfei Lv,Chongsheng Zhang
类目: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:In continual learning, regularization-based methods aim to mimic neuroplasticity by updating network weights to learn new tasks while constraining changes to parameters that are important for previously learned tasks through penalty terms. However, as the number of tasks increases, preserving knowledge from earlier tasks becomes less effective. This is mainly because updates for new tasks are distributed across the entire network, where parameters are highly interdependent due to the dense connectivity between neurons. Inspired by the Exponential Distance Rule (EDR), which states that the number of neural connections decreases with increasing physical distance, learning is naturally locally correlated while inter-area updates remain less task-dependent. Motivated by this principle, in this paper we demonstrate that applying weight regularization to multiple small and independently learned complementary representations is more effective than regularizing a single large shared representation. By decomposing the input into independently learned components, learning new tasks induces only localized parameter updates within specific representation, which limit interference and improve knowledge retention across tasks. We employ independent autoencoders that learn different parts of the input independently through unsupervised regularization. At inference time, a linear classifier is used for evaluation. The experimental results on a sequence of tasks using 5-Splits MNIST, sequence of 5-datasets, 5-Splits CIFAR-10, 10-Splits Tiny ImageNet shows competitive results with supervised and replay-based methods. Code is available.
[CV-103] From Restoration to Redrawing: Revitalizing Ancient Chinese Characters by SVG Code Generation and Editing
链接: https://arxiv.org/abs/2611.24084
作者: Weiguang Zhang,Linhua Dong,Bin Dong,Shanshan Jiang,Maizhen Ning,Xiaoqiang Kang,Kun Zhang,Hao Xu,Xiaowei Huang,Wei Wang,Kaizhu Huang,Qiufeng Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Hanjian manuscripts written on Han Dynasty bamboo and wooden slips are invaluable heritage of ancient Chinese writing but suffer from damaged ink and incomplete strokes. Existing image-to-image restoration methods may alter observed strokes and offer limited control over stroke geometry. Inspired by experts’ manual redrawing practices, we address these issues by introducing \textbfReDraw, a two-stage vision-language model(VLM) that reformulates character restoration as Scalable Vector Graphics(SVG) code generation and editing. Stage one targets faithful vectorization of observed strokes geometry or conservative tracing of mildly degraded strokes. Guided by skeleton chain-of-thought reasoning and reinforcement learning with structure-aware rewards, stage two predicts skeletons before contours to redraw severely incomplete strokes by appending paths while keeping existing SVG paths unchanged. To support training, we develop a multi-layer data synthesis pipeline that disentangles ink, material, and degradation to generate 1.2M samples, and derive up to 6,400 adaptation pairs from 320 annotated real glyphs. Experiments demonstrate synthetic restoration performance and adaptability to real Hanjian glyphs. Code and demo will be available upon acceptance.
[CV-104] Learning Where and How Much to Sample: Volume-Adaptive k-Space Acquisition for Accelerated MRI
链接: https://arxiv.org/abs/2611.24081
作者: Reyhaneh Neshatavar,Mohsen Yavartanoo,Kyoung Mu Lee
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Magnetic resonance imaging (MRI) is an essential medical imaging technique, but its high costs and long acquisition times can limit accessibility. Accelerated MRI reduces acquisition time through k-space undersampling, yet conventional methods often rely on predefined sampling patterns and uniform slice budgets, overlooking anatomical variations and inter-slice relationships. In this paper, we propose a volume-adaptive k-space acquisition framework that jointly learns where and how much to sample across an entire MRI volume. Using only acquired auto-calibration signal (ACS) measurements, our method exploits volume-wide anatomical information to adaptively allocate sampling budgets across slices while identifying informative k-space lines. A learned joint slice–frequency refinement module coordinates sampling decisions while preserving an exact total volume-level acquisition budget. To reconstruct the acquired measurements, we design a sensitivity-aware synchronous cross-slice reconstruction network that estimates coil sensitivities from ACS measurements and progressively refines neighboring-slice reconstructions while enforcing hard k-space data consistency. Both networks are jointly optimized using reconstruction-based supervision without requiring fully sampled k-space at inference. Experiments on knee and brain MRI across multiple contrasts and acceleration factors demonstrate the adaptive behavior of our acquisition framework and evaluate its reconstruction performance.
[CV-105] CausalDrive: Causal Decomposition for Data-Efficient End-to-End Autonomous Driving
链接: https://arxiv.org/abs/2611.24080
作者: Xiaobao Wei,Jingxian Wu,Lirong Yang,Qing Han,Yuan Gao,Ziwei Yang,Shuai Yang,Weiwei Liu,Chang Huang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Efficient use of expert demonstrations is essential for end-to-end autonomous driving. However, directly imitating expert trajectories couples nominal intention, which specifies where the vehicle should go, with multimodal driving styles, which determine how that intention is executed. Different combinations of these factors can yield similar expert trajectories, obscuring their respective causal contributions and forcing the policy to learn both from a single entangled target. This ambiguity limits how effectively the policy can exploit expert demonstrations. We propose CausalDrive, a causal decomposition framework for data-efficient driving policy learning that separates deterministic nominal intention prediction from probabilistic driving style modeling. Specifically, we construct counterfactual nominal trajectories from navigation command, map topology, and ego state by removing style dependent effects, providing a Nominal Intention Planner with supervision independent of expert trajectories. Conditioned on the predicted nominal intention and surrounding agents, a diffusion-based Style Execution Policy models diverse driving styles as a multimodal distribution over residuals between nominal and expert trajectories. This causally structured supervision reduces redundant learning of shared nominal behavior in routine scenarios, directing expert supervision toward informative execution variations and improving data utilization. CausalDrive achieves 91.2 PDMS on NAVSIM v1 and 89.5 EPDMS on NAVSIM v2, matching or surpassing state-of-the-art performance. Further experiments demonstrate strong performance with limited expert data and clear advantages in challenging interactive scenarios.
[CV-106] APS: Adaptive Partitioning-Based Frame Selection for Long Video Understanding
链接: https://arxiv.org/abs/2611.24079
作者: Junxiang Wen,Yinxi Zhao,Jiawei Chen,Jingying Hu,Lijun Wang,Zhen Qin,Shuiguang Deng
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Recent advances in large vision-language models (LVLMs) have significantly improved video understanding capabilities, yet processing long videos remains challenging due to the substantial computational cost of extensive visual inputs. Frame selection offers an efficient alternative, but many existing region-based methods rely on fixed or predefined temporal partitions, which may result in suboptimal temporal regions, compromising subsequent budget allocation and frame selection. In this work, we propose Adaptive Partitioning-Based Frame Selection (APS), a training-free framework that adapts temporal partitions to local variations in query-frame relevance. Specifically, APS formulates temporal partitioning as a complexity-regularized fitting problem, jointly optimizing partition boundaries and local models through dynamic programming. The resulting adaptive temporal partitions guide frame budget allocation based on query relevance and temporal extent, enabling informative frame selection under limited visual budgets. Extensive experiments on Video-MME, MLVU, and LongVideoBench demonstrate that APS consistently improves performance across diverse LVLM backbones, achieving the highest average gains over uniform sampling among competing methods. Further analyses validate the effectiveness of adaptive partitioning and demonstrate its generalizability across model scales, similarity encoders, and advanced LVLMs.
[CV-107] AeroRSI: Recursive Self-Improvement for Aerial Vision-Language Navigation
链接: https://arxiv.org/abs/2611.24077
作者: Shiwei Wu,Yuwen Pan,Zhi Li,Yaowen Li,Yan Zhou,Xuemiao Xu,Yu Liu
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO)
备注: 8-page main paper with references and supplementary material (15 pages); 5 figures, 13 tables. Code: this https URL Project page: this https URL
Abstract:Vision-language-action (VLA) models enable unmanned aerial vehicles (UAVs) to follow natural-language instructions, yet still fail frequently in closed-loop flight. Correcting these failures requires repairs, i.e., actions that avoid them at the states the policy itself reaches, which human pilots cannot label at scale and rule-based planners cannot supply reliably. In this paper, we present AeroRSI, a recursive self-improvement framework in which an aerial navigation policy iteratively learns from its own failures. Our key insight is that a failing policy should judge its own repairs by flying on from them. Judged in this way, its sampled actions and even rule-based proposals that hurt it when imitated directly become reliable supervision. Moreover, an episode terminates at its first failure, concealing subsequent failures until the earlier ones are repaired. AeroRSI thus renews its supervision in every round, anchoring the failures of the current policy, verifying repairs by its continuation, and distilling them while keeping its behavior elsewhere. On TravelUAV, three rounds raise the success rate of a supervised 2B-parameter policy from 46.4% to 67.5% and reduce collisions by half, surpassing the previous state of the art by 10.6 points from the same initialization without extra human labels or inference cost. Code and models are available at this https URL.
[CV-108] SemPhys: Semantic-Physical Co-Learning for Humanoid Motion Generation via Differentiable World Models
链接: https://arxiv.org/abs/2611.24074
作者: Zejian Kang,Yeqiang Wang,Zekai Chen,Boyang Shen,Mingyu Ma,Bingqian Wu,Fan Zhou,Jianjie Zhang,Chentao Shen,Wentao Yang,Rui Qian,Xiangru Huang
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:In human-robot collaboration, natural language lets people direct a humanoid’s whole-body movements without programming joint trajectories. A common approach generates a kinematic reference from an instruction and relies on a tracking controller to execute it. However, as the controller maintains balance under contact constraints, the executed motion can depart from the reference in ways that change the action it conveys. Our central insight is that semantic alignment must be evaluated directly on the complete motion the robot physically executes. Building on this insight, we introduce \textbfSemPhys, a framework that trains motion generators using semantic feedback from simulated executions. A differentiable world model and a fixed tracking controller predict a short segment of a recorded execution. SemPhys inserts this prediction at its original position, scores the complete trajectory against the instruction, and backpropagates through the predicted segment. A paired anchoring loss stabilizes adaptation by keeping each generated reference close to its counterpart from the initial generator. On BONES-SEED, SemPhys largely preserves instruction retrieval accuracy from reference to execution and achieves the highest retrieval accuracy on executed motions among the evaluated methods.
[CV-109] Benchmarking Reasoning ReliabilityPrior_Evidence Contrastand Spectral Discrimination in Remote-Sensing Geological Image Interpretation
链接: https://arxiv.org/abs/2611.24068
作者: Jun Wang,Fengpeng Li,Wei Han,Yuxuan Li,Tianjin Huang,Jining Yan,Sheng Wang,Kan Sun,Lizhe Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Reliable lithology interpretation requires accurate geological recognition and reasoning grounded in visual evidence. Although Multimodal Large Language Models show promise in remote sensing, their reliability in geological interpretation remains unclear. We introduce HyGeoBench, a diagnostic benchmark of 10,000 questions based on hyperspectral imagery across three dimensions: reasoning reliability, prior_evidence contrast, and spectral discrimination. It covers ten task types, with both multiple-choice and open-ended response formats. Integrating hyperspectral satellite imagery with geological annotations and verified evidence, HyGeoBench examines reasoning errors, conflicts between geological priors and visual evidence, and cross-band inconsistencies. Evaluations reveal limitations in fine-grained recognition and evidence-grounded interpretation. To address these challenges, we propose HyGeoAgent, a tool-augmented framework integrating region observation, text knowledge retrieval, same-band image retrieval, and native PCA analysis through adaptive tool use. Experiments demonstrate consistent improvements over the corresponding base models, improving geological image interpretation across diagnostic dimensions.
[CV-110] R3Sieve: Suppressing Unconditional Information from Attention to Answers for High-Resolution VQA
链接: https://arxiv.org/abs/2611.24060
作者: Yifan Cao,Yawen Zhang,Xiaocui Yang,Faxian Wan,Shi Feng,Daling Wang,Yifei Zhang,Shangshu Yu,Xianbing Zhao
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:High-resolution visual question answering (VQA) requires models to identify question-relevant small objects in high-resolution images and select the correct answer based on what they recognize, demanding fine-grained perception. Previous methods focus on obtaining better visual evidence (image crops), but obtaining it alone is insufficient, using it effectively is equally crucial. Visual evidence should therefore be utilized as effectively as possible throughout inference. We argue that small-target evidence is obscured by unconditional information that remains observable in subsequent processing even after the condition on which it should depend is removed. Such information hinders the effective use of small-target evidence. We propose R ^3 Sieve (Residual-Region-Response Sieve), a training-free framework. We divide inference into three stages—localization, cropping, and answering—and introduce distinct counterfactual references to suppress interference at each stage. During localization, the Shared-Subspace Canceller (SSC) uses a generic image description to construct a counterfactual reference and obtain question-relevant attention maps. During cropping, the Unique-Contribution Spotter (UCS) constructs counterfactual references from visual and non-visual features to identify distinctive, complementary crop regions. During answering, the Content-Aligned Voter (CAV) constructs an image-free branch to calibrate answer responses. Experiments on three high-resolution VQA benchmarks show that R ^3 Sieve achieves the highest overall accuracy among compared methods across diverse settings, improving base models by up to 21.5 percentage points.
[CV-111] CURVTON-205K: A Mask-Free Difficulty-Stratified Synthetic Dataset for Robust and Scalable Virtual Try-On WACV
链接: https://arxiv.org/abs/2611.24042
作者: Dipan Bartaula,Anuj Shrestha,Ankit Belbase,Anish Bishwakarma,Saroj Poudel,Samip Ghimire,Shashank Bhatta,Eric Arazo,Binod Bhattarai
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2027. Project page: this https URL
Abstract:Recent advances in 2D virtual try-on (VTON) have largely focused on architectural innovation, yet robust performance in diverse real-world conditions remains challenging. We argue that failure modes in VTON, such as inaccurate occlusion handling, unconstrained poses, and poor coverage of under-represented garments and appearance factors, are tightly coupled to dataset design choices and long-tail data regimes. We present a scalable data generation pipeline that coordinates specialized editing and generation models with vision-language model (VLM) guidance to synthesize triplets without requiring parsing masks or dense-pose annotations. Using this pipeline, we create CURVTON-205K, a curated synthetic dataset of 205K triplets: source person, target garment, and corresponding try-on image. CURVTON-205K is explicitly designed to broaden coverage across garments, person appearance, poses, occlusions, and visual conditions. Additionally, CURVTON-205K is organized into three difficulty tiers, enabling controlled evaluation under varying levels of difficulty and supporting difficulty-aware experimentation. We analyze the dataset diversity and demonstrate its potential to improve try-on fidelity across four VTON architectures and its cross-dataset generalization.
[CV-112] Semantic-aware Contrasting with Explicitly Disentangled Representations for AI-Generated Video Detection
链接: https://arxiv.org/abs/2611.24038
作者: Jiaquan Yu,Jingyi Xie,Zeyu Zhang,Yunhang Qian,Xiaobin Hu,Jiawei Liu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 16 pages, 5 figures
Abstract:The rapid proliferation of AI-generated videos (AIGVs) has raised critical concerns about content authenticity. Existing benchmarks suffer from a fundamental limitation: real and fake videos are often collected or generated from different semantic distributions. As a result, detectors may rely on spurious correlations involving objects, scenes, or actions rather than intrinsic artifacts introduced by generative models. To this end, we introduce SAVGE, a large-scale semantically aligned AIGV detection dataset comprising 2,369,822 videos, including 424,115 real videos and 1,945,707 generated videos. SAVGE organizes samples into real-fake semantic groups by using captions paired with real videos as prompts for text-to-video generation, thereby providing both group-level correspondence and textual semantic anchors. Based on SAVGE, we further propose Explicit Disentangling and Semantic-aware Contrasting (EDSC), a model-agnostic training framework for learning artifact-focused representations with reduced semantic interference. EDSC first extracts and aligns visual semantic features with textual representations, and then explicitly suppresses semantic information in the artifact branch while contrasting real and fake samples within each semantic group. By integrating caption-guided semantic supervision with intra-group contrastive learning in a two-phase optimization strategy, EDSC encourages detectors to capture generation-related cues under matched semantic conditions. Extensive experiments on SAVGE and GenVideo demonstrate that EDSC improves most evaluated backbone-metric combinations, enhances robustness under challenging evaluation settings, and strengthens cross-dataset generalization. These results highlight the importance of semantic alignment in AIGV detection and validate explicit semantic-artifact disentanglement as an effective strategy for robust detector learning.
[CV-113] STAR: A Physical-Aware Benchmark and Framework for Astronomical Super-Resolution across Multiple Filters
链接: https://arxiv.org/abs/2611.24023
作者: Guohang Zhuang,Jiaze Li,Kuo-cheng Wu,Hao Du,Jinyang Huang,Xiang Zhang,Wanli Ouyang,Yan Lu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Astronomical super-resolution (ASR) aims to reconstruct high-resolution star fields from low-resolution telescope observations across multiple filters for astrophysical analysis. However, developing a physics-aware ASR framework faces two challenges: 1) Data view: no large-scale benchmark supports field-level ASR, especially on unseen filters. 2) Method view: Existing ASR methods often fail to preserve stellar flux across spatial resolutions. Moreover, ASR requires reconstructing high-frequency components, including accurate star locations and profiles, making it more ill-posed than natural-image SR. To tackle these problems, we introduce STAR++, a large-scale benchmark comprising approximately 1.7 million flux-consistent image pairs across 36 filters, derived from Hubble Space Telescope (HST) observations. STAR++ uses a flux-preserving data generation pipeline to ensure photometric consistency across resolutions and provides the Flux Error (FE) metric to evaluate ASR models from a physical perspective. To address method-level challenges, we propose a Flux-Invariant Super-Resolution Plus (FISR++) model that generalizes to unseen filters while accurately reconstructing flux-consistent high-resolution images. Beyond the flux-preserving modules, FISR++ introduces two novel components. Specifically, a Filter-Aware Adapter (FAA) incorporates filter metadata to adapt image representations to varying spectral and optical conditions. In addition, a Spatial Ambiguity Disentangle Loss (SADL) models spatial ambiguity as pixel-shift uncertainty, mitigating the smoothing of learned conditional mean reconstructions caused by this ambiguity. Extensive experiments demonstrate that FISR++ consistently outperforms existing ASR methods in reconstruction quality and flux accuracy, while generalizing to unseen filters, highlighting its effectiveness for photometrically faithful ASR.
[CV-114] Semi-Supervised GAN-Based Segmentation of CTPA Scans for Pulmonary Emboli Boundary Extraction
链接: https://arxiv.org/abs/2611.24018
作者: Maria Thoma,Michalis Savelonas,Foteini Malli,Dimitris Iakovidis
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: 2026 IEEE 39th International Symposium on Computer-Based Medical Systems (CBMS), Limassol, Cyprus
Abstract:Pulmonary embolism (PE) is a pulmonary vascular disease associated with high morbidity and mortality, where delayed diagnosis can be fatal. While computed tomography pulmonary angiography (CTPA) is the clinical gold standard for PE detection, interpreting these scans manually is time-consuming and error-prone. In this work, we propose a semi-supervised segmentation method, employing generative adversarial networks (GANs) to leverage large pools of unlabeled CTPA scans. The proposed method addresses more effectively the challenges of CTPA image segmentation than previous methods by employing the dice loss and the cross-entropy loss to tackle the highly imbalanced nature of CTPA scans, the feature matching loss to encourage the generator to produce anatomically plausible segmentation maps that are indistinguishable from ground truth, as well as the self-training loss to reinforce learning from highly-confident pseudo-labeled samples. The experimental evaluation on a publicly available dataset shows that the proposed method maintains accurate segmentation, despite a limited number of labeled CTPA scans. with a Dice coefficient of 88.75% and a 95th percentile Hausdorff distance (HD95) of 1.74, outperforming fully supervised baselines.
[CV-115] Rethinking Contrastive Targets in Radiology Language-Image Pretraining NEURIPS2026
链接: https://arxiv.org/abs/2611.24016
作者: Yunhe Gao,Ashwin Kumar,Jiaming Liu,Chong Wang,Maya Varma,Fuying Wang,Stefania L. Moroianu,Louisa Fay,Sergios Gatidis,Akshay S. Chaudhari,Curtis Langlotz
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: NeurIPS 2026
Abstract:CLIP-style contrastive image-text pretraining is widely used in radiology data, where each image is paired with one report and every other report in the batch is treated as a negative. Under this recipe, two failures appear, neither of which is systematically addressed by existing contrastive learning theory: zero-shot evaluation peaks then degrades while training loss keeps decreasing, and training fails to learn from large batches. We hypothesize the cause is structural rather than radiology-specific: many unpaired reports share clinical findings with a given image, so CLIP’s paired-identity target is systematically biased, not merely noisy. To test this, we construct CLEVR-Radiology, a synthetic testbed that preserves only this overlapping-fact structure and reproduces both failures. With evidence that the structure is sufficient to reproduce the failures, we develop a target-misspecification theory that explains both failures from a single deterministic gradient bias amplified by batch size, and use it to motivate a minimal compatibility-based correction, CorLIP: text is treated as positive whenever its clinical assertions agree with the image, with no architecture changes. Beyond benchmark improvements, CorLIP largely removes the post-peak degradation, makes large-batch training succeed where it previously collapsed, substantially amplifies the gains from more data, and is the only configuration that remains stable up to B=2048. The correction transfers from chest X-ray to 3D CT modalities.
[CV-116] FlyVQA: Seeing Video Frame Interpolation Quality through Fly Eyes
链接: https://arxiv.org/abs/2611.24012
作者: Baoliang Chen,Lingyu Zhu,Zhiwei Zhong,Shiqi Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:The ultimate ambition of visual quality assessment is to reproduce the neural computations of human vision and thereby achieve full alignment with human perception. However, the immense complexity of human visual circuitry makes this goal difficult to realize. Recently released fly visual connectomes provide a tractable biological starting point. Despite differences between fly and human vision, the fly’s well-characterized motion pathways motivate us to investigate their potential for assessing the perceptual impact of temporal distortions. In this paper, we use a shared, frozen fly visual circuit to perform full-reference quality assessment of frame-interpolated videos, whose artifacts can disrupt temporal consistency. Specifically, compound-eye sampling maps reference and distorted sequences into the circuit, while a connectome-aware readout quantifies discrepancies in cell-type-specific activity and excitatory–inhibitory synaptic dynamics. Reference-guided temporal pooling and a lightweight regression head integrate these differences into a quality score. Experiments across multiple interpolation methods demonstrate better agreement with human ratings than CNN, RNN, and Transformer baselines with comparable parameters, supporting a biologically grounded approach to perceptual quality modeling.
[CV-117] WAM-Policy: World Action Models with On-Policy Distillation for Quadrupedal Loco-Manipulation
链接: https://arxiv.org/abs/2611.24011
作者: Junlin Long,Xu Deng,Yiran Wang,Yue Yang,Yang Zhao,Zeyu Zhang
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注: Code: this https URL , Website: this https URL
Abstract:Arm-free quadrupeds can perform loco-manipulation, walking on three legs while manipulating with a front leg, yet no existing policy for it follows language instructions. World action models (WAMs) have emerged as effective language-conditioned manipulation policies, but are pretrained for robot arms and sample slowly. WAM-Policy turns one WAM into a quadrupedal loco-manipulation policy via two stages. The first, Cross-Morphology Policy Adaptation and Acceleration, adapts the arm-centric WAM to quadrupeds and distills the transferred policy into a student with one video and two action steps by on-policy self-distillation (OPSD). This student, however, is bounded by its teacher, which only imitates demonstrations. The second stage, Task-Specific Improvement and Merging, therefore tunes the teacher by reinforcement learning (RL) and merges the slow specialists into one few-step student by multi-teacher on-policy distillation (M-OPD). On QLM-Bench, ten simulated tasks with 1,000 demonstrations synthesized without teleoperation, the transferred policy reaches 77.0%, and the OPSD student keeps 76.7% while sampling 14 times faster. Task-specific RL and M-OPD lift it to 85.0%. We also deploy the OPSD student on a real Unitree Go2 Air. WAM-Policy thus achieves a stronger quadrupedal policy at a fraction of the inference cost.
[CV-118] HHW-Ego: DSLR-Quality Enhancement for Multi-Source Wearable Ego Imaging
链接: https://arxiv.org/abs/2611.24010
作者: Yunfei Tong,Qianqian Sun,Jixiang Luo,Dell Zhang,Hao Sun
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Wearable ego-centric images are now in high demand for scenarios ranging from daily smart glass usage to embodied intelligence. But the image quality is far behind smart phone due to the lacking paired high-quality reference images and the dedicated enhancement systems. To solve this, a customized degradation pipeline is designed to generate paired samples matching wearable camera images, boosting the upper limit of enhancement performance. Besides a two-stage enhancement framework is further built: first, tuning an efficient model on the paired dataset to enhance real wearable images; second, using hyperspectral data to refine color temperature for better quality. Moreover we present HHW-Ego, a multi-source paired dataset integrating with Hyperspectral, High-dynamic range and Wearable Ego-centric data, which includes 5,000 image groups spanning indoor/outdoor and day/night scenes. On held-out paired data, Stage-1 fine-tuning improves PSNR, SSIM, Lab distance, and LPIPS for Real-ESRGAN and PFT-SR. On real wearable images, no-reference metrics, human preference, gamut analysis, and visual comparisons characterize the effects of Stage 2.
[CV-119] Freeforward4d: Annotation-free feed-forward 4d reconstruction of driving scenes
链接: https://arxiv.org/abs/2611.24007
作者: Luosong Guo,Xiang Yang,Chaoyang Li,Ao Gao,Jiangming Shi,Yanyun Qu,Zhizhong Zhang,Jingyu Gong,Yuan Xie
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Feed-forward 4D reconstruction enables efficient creation of renderable driving scenes for autonomous driving simulation. However, most existing methods rely heavily on dataset-provided cameras, LiDAR depth, or 3D box annotations, limiting their applicability to unannotated driving videos. Pretrained visual models can predict cameras, depth, and object tracks from RGB alone, but their geometry is mutually inconsistent: multi-view predictions come in short windows with independent scales and frames, while monocular depth is metric only per image. Our key insight is that, once reconciled into a single metric frame, these priors can replace dataset annotations for feed-forward 4D reconstruction. We present \textbfFreeForward4D, which calibrates overlapping prediction windows against monocular metric depth and registers them into one frame across time. On this geometry, 2D tracks are associated across cameras in 3D, and objects move through hierarchical motion that combines a shared object velocity with local Gaussian corrections, so that both training and inference use only RGB videos. On Waymo, FreeForward4D achieves 27.66 dB PSNR and 0.836 SSIM while recovering metric-scale cameras and depth without scale alignment. It surpasses STORM, SLARM-F, UFO, DGGT, and StreetForward, which rely on dataset annotations, in full-image PSNR and attains the best RGB fidelity on 8-second windows. RGB-only fine-tuning further adapts the framework to nuScenes and Argoverse 2. Code will be released.
[CV-120] Choosing Supervised Post-Processing for Frozen Person Detectors: Information Object Changes and Complete Cost
链接: https://arxiv.org/abs/2611.24003
作者: Tianyi Zhou,Zijun Jia
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:A frozen detector computes hypotheses that can support better final outputs through supervised post-processing. Choosing a readout requires knowing which available information improves detection and what its processing costs. We compare candidate scores, class distributions, geometry, native-set summaries and adapted neighborhood meta classification under common candidate, supervision and operating-point conditions. On 320 held-out CrowdHuman images, adding native-set context to a 97-dimensional readout yields 275 more matches and 96 more unmatched predictions. On a separate 319-image cohort, Full100 matches 6004 objects,versus 5848 for source-selected raw WBF and 6002 for a MetaDetect-style adaptation. Its gain over raw contains 300 added and 144 lost discoveries. After execution-equivalent optimization, Full100 and the meta-statistics readout both take approximately 95 ms per complete warm request on a fixed 64-image workload. Source-frozen transfer gives positive point results on 223 MOT17Det frames from seven sequences, while the main MIAP result shows no improvement. The study supports using sufficiently informative output readouts while retaining optimized meta classification as a competitive option. It also shows why feature dimension and net detection gain alone are unreliable guides to adoption.
[CV-121] ProtoBridge-CLIP: Observation-Masked Finding Prototypes for Medical Image–Report Representation Learning
链接: https://arxiv.org/abs/2611.24002
作者: Jinrui Wu,Yun Peng
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Radiology reports distinguish affirmed findings, explicit negations, and unmentioned concepts, but conventional global image–report contrastive learning often compresses these different evidence states into a single study-level target. We introduce \model, which combines symmetric global image–report alignment with observation-masked finding supervision and a soft relation objective over affirmed report concepts. Text-derived finding prototypes are aligned with local visual tokens, while uncertain and unmentioned concepts are excluded from direct finding labels. We evaluate the framework on three CT benchmarks and five chest-radiograph benchmarks, together with component ablations, paired and clinical retrieval, localization, perturbation robustness, calibration, and per-finding analyses. \model achieves macro-AUROC values of 73.6%, 66.5%, and 68.8% on CT-RATE, RAD-ChestCT, and INSPECT, and 80.2%, 91.4%, 78.5%, 86.5%, and 90.8% on ChestXray14, CheXpert, PadChest, RSNA, and SIIM-ACR, respectively. Relative to the matched global-only controls, the findings and relation objectives yield consistent gains across the evaluated datasets. These results demonstrate the practical value of treating observed and unobserved report concepts differently during medical image–text representation learning.
[CV-122] ReSynth: Constructing Synthetic Supervision for Multi-Image Understanding
链接: https://arxiv.org/abs/2611.23996
作者: Yilin Wang,Xin Yu,Xinyi Guo,Xiangxi Zheng,Dongxing Mao,Linjie Li,Zhengyuan Yang,Alex Jinpeng Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:People often understand the world by comparing different views of it. Vision-language models (VLMs), however, receive these views as separate images and still struggle to relate them. One problem is data. Training on these tasks requires groups of images with a known relationship: the same object from different viewpoints, or the same scene before and after a change. Such examples are hard to collect at scale from existing photo collections. We introduce ReSynth, which turns single-image perception and controllable image generation into cross-image relational supervision. Localized editing produces before-and-after pairs, viewpoint generation produces multiple views of the same instance, and verified scene contents support comparisons of counts and attributes. ReSynth combines Qwen3-VL-8B’s captioning, grounding, and counting capabilities with Qwen-Image-2.1’s generation and editing capabilities. It verifies single-image facts such as object counts and attributes, then combines these facts to construct multi-image QA. Using GRPO on approximately 440K ReSynth questions, with no real images during training, Qwen2.5-VL-7B and Qwen3-VL-8B improve by 17.73 and 6.10 average points on BLINK, MUIRBench, and VLM2-Bench, while single-image performance is broadly preserved on Qwen2.5-VL-7B and improves across all four evaluated benchmarks on Qwen3-VL-8B. Notably, this supervision enables the 8B perception model itself to outperform Qwen3-VL-235B on BLINK and MUIRBench.
[CV-123] WaveView: The Novel View Is in the Noise
链接: https://arxiv.org/abs/2611.23994
作者: Kaiwen Wang,Jan-Hendrik Pauls,Yinzhe Shen,Frank Bieder,Christoph Stiller
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Generative novel view synthesis (NVS) employs a diffusion model to render a static or dynamic scene from a target camera view. Explicit warp methods complete a reprojection of the source with video diffusion models, and implicit methods encode the camera parameters or 3D cues. Both add inference overhead, and the generated layout often drifts from the ground truth. We present WaveView, which injects the warped source into the phase of the initial noise of a pretrained generator. The Dual-Tree Complex Wavelet Packet Transform (DT-CWPT) allows local and anisotropic phase injection. The generator is thus constrained less in disoccluded regions and along the epipolar line. WaveView keeps the architecture and adds negligible overhead. On the DL3DV benchmark, it outperforms all baselines in every overlap bin. In the hardest bin, it gains 0.84 dB PSNR on the covisible region over MetaView, which shares its generator, while running 2.1 times faster. For dynamic NVS, fine-tuning AnyView with the same noise raises its zero-shot PSNR on AnyViewBench from 11.58 to 12.35 dB.
[CV-124] SIRAZ: Sparse Inter-Layer Residual Aggregation for Zero-Shot Visual Neural Decoding
链接: https://arxiv.org/abs/2611.23991
作者: Neha,Mudasir Ganaie
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Brain-computer interfaces aim to establish direct communication between neural activity and external systems, making interpretation of brain signals an important problem. In this context, neural visual decoding seeks to infer visual information from neural activity by learning a mapping between brain responses and the corresponding visual stimuli. It investigates how visual information emerges from neural signals and relates to perceptual processing while enabling the reconstruction or retrieval of visual stimuli from neural signals. Our brain primarily follows a hierarchical pathway to process visual information captured by the eye. However, neural representations learned by existing methods differ fundamentally from those produced by the biological visual system. To bridge this gap and mimic the human vision hierarchy, we introduce Sparse Inter-layer Residual Aggregation for Zero-shot (SIRAZ). SIRAZ is a framework that aligns neural visual representations by jointly modeling early visual processes via fovea- and magnocellular-pathway-inspired Difference-of-Gaussians transformations. To align the semantic information with neural information, it extracts multiple intermediate representations of the image at multiple depths and passes them through the Residual Gated Adapter Mechanism. To study the temporal dynamics of neural signals, the Direction-aware Multi-Scale Temporal Attention Encoder is introduced to align the learned neural representation with the final visual representation. The proposed framework achieves 92.5% top-1 and 99.27% top-5 accuracy in the intra-subject setting on the THINGS-EEG dataset, and 50.25% and 75% top-1 and top-5 accuracy in the THINGS-MEG dataset. The results indicate that combining biologically motivated visual representations with the structured modalities of neural dynamics improves cross-modal alignment and zero-shot image retrieval.
[CV-125] Agreement Is Not Accuracy: LiDAR Elevation Errors Hidden in LiDAR-Camera Extrinsics
链接: https://arxiv.org/abs/2611.23972
作者: Yeon Kun Lee,Dongseok Yang,Deok-Kyeong Jang,Choongho Chung,Junyoung Lee,Hyeongseok So
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 14 pages, 6 figures, including supplementary material
Abstract:LiDAR and cameras provide complementary measurements for 3D vision, yet agreement between their observations does not establish the accuracy of either sensor’s internal geometry. We show that systematic LiDAR elevation biases can be absorbed into the estimated extrinsics. Using synchronized LiDAR intensities and camera-estimated checkerboard poses, we jointly estimate a sensor-specific elevation correction shared across recordings and extrinsics for each recording. Ray-plane intersections locate the checker pattern without using reported ranges, preventing range errors from directly moving the lookup. Under matched intensity and correction models, this lookup gives smaller camera-consistency residuals than measured-point lookup. Across six identically designed rigs using the same sensor models, recording-excluded corrections reduce the mean translation-reference distance from 7.57 to 3.04 mm. On separate structures, they reduce inter-plane angle errors against references from independent dense camera scans and improve image-edge alignment. Rolling the rigs shifts the plane pairs exhibiting the largest angular improvements, as predicted for an elevation-only correction. Within the ray-plane model, calibration-target projections and loop-consistency residuals change little despite these geometric improvements. These findings show why agreement after extrinsic calibration need not establish correct internal LiDAR geometry. Synchronized observations of a familiar checkerboard provide a practical route to estimating and testing elevation corrections without a dedicated large calibration field. We plan to release code and data for reproducing the main results.
[CV-126] OMAS: Topology-Aware Operation Modeling and Adaptive Staging for Egocentric Hand Forecasting
链接: https://arxiv.org/abs/2611.23970
作者: Guanzhong Sun,Junyi Ma,Yuxuan Wu,Erhang Zhang,Chenyang Xu,Yanzi Miao,Hesheng Wang
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Forecasting future hand movements from egocentric videos plays an important role in understanding human intentions and hand-object interactions. However, different manipulation intentions can lead to substantially different hand trajectories even when interacting with the same object. Existing methods have not sufficiently explored the intrinsic relationships between articulated hand movements and object semantics to capture such operation-dependent motion patterns. Therefore, we propose TOMAS, a topology-aware operation modeling and adaptive staging framework for egocentric hand forecasting. Specifically, we develop a topology-aware hand-object reasoning mechanism that associates hand skeletal structures and articulation dynamics with object semantics and spatial relationships to jointly infer potential interaction targets and latent manipulation operations. The resulting object-operation representations provide interaction conditions for future hand trajectory prediction, with supervision adapted to the annotations available in each dataset. Furthermore, we introduce an adaptive staging mechanism that organizes future hand movements into ordered latent stages with learnable temporal boundaries. Stage-specific geometric subgoals are then incorporated to guide trajectory refinement according to the underlying manipulation patterns. Experimental results on multiple publicly available benchmarks demonstrate that TOMAS achieves state-of-the-art performance in multi-dimensional and multi-target hand motion forecasting.
[CV-127] Dual-Manifold Variational Autoencoders
链接: https://arxiv.org/abs/2611.23964
作者: Hongxu Chen,Chun Wang,Yanghao Wang,Hongxiang Li,Zhen Wang,Zhixiang Zhang,Long Chen
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:In latent diffusion models, expanding the latent dimension of variational autoencoders (VAEs) is essential to push the upper bound of reconstruction quality. However, without explicit structural constraints, such unregularized expansion easily yields a rugged and ill-conditioned latent manifold, causing generated latents to drift off the manifold during generation (i.e., off-manifold problem). Recent works smooth this manifold by using external foundation models (e.g., DINOv2) as a topological regularizer. However, we argue that tying the latent space to frozen external priors inherently restricts generalization: bound by their pre-training distributions and semantic biases, they may fail to capture faithful visual details on out-of-distribution domains. Observing that low-dimensional latents naturally form a smoother, generation-friendly manifold, we propose dual-manifold VAE (DM-VAE) to leverage internal representations as a topological regularizer. Specifically, DM-VAE pairs a high-dimensional reconstructive branch with a low-dimensional branch that extracts a clean topological skeleton, anchoring the high-dimensional manifold to this smooth subspace. Building upon this framework, we further introduce topological representation alignment (TRA) to supervise intermediate generator features during training, and internal guidance with topological skeleton (IG_TS) to guide structural components while preserving high-frequency details at inference. On ImageNet-1K (256x256), our foundation-model-free DM-VAE achieves a state-of-the-art rFID of 0.12 and a generation FID50K of 1.28 (w/o CFG) and 1.20 (w/CFG), and further reaches 1.11 when orthogonally combined with DINOv2.
[CV-128] SEE: Unlocking the Hidden Perception of Frozen VLMs
链接: https://arxiv.org/abs/2611.23962
作者: Alin-Ionut Popa,Tuo Zhang,Dimitrios Dimitriadis
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Vision-language models (VLMs) grasp the overall content of an image but often miss the fine details a question hinges on, such as a small object in a large photo or a number in a dense infographic. Yet even when their answer is wrong, the region they point to often contains the evidence. We introduce Self-Elicited Evidence (SEE), a training-free method that lets a frozen VLM look again where it points. When the initial answer has low confidence, SEE prompts the model to propose regions where the details may appear, answers the question on these enlarged crops, and aggregates the answers with the original prediction through confidence-weighted voting. The model also answers with its own transcript, which helps on text-rich images. SEE needs no detector or OCR engine. Across five backbones, SEE improves every small-object benchmark by 2.3-13.1 points and outperforms five training-free baselines on small objects. With SEE, Qwen3.5-4B surpasses the direct answer of Qwen3.5-9B and matches the 6 times larger Qwen3.5-27B.
[CV-129] DORMS: Image Segmentation via Depth-Ordered Region Merge Trees
链接: https://arxiv.org/abs/2611.23958
作者: Furkan Mumcu,Yasin Yilmaz
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:We introduce DORMS (Depth-Ordered Region Merging for Segmentation), a purely algorithmic approach to image segmentation from monocular depth maps, requiring no neural networks or learned parameters in the segmentation process. An off-the-shelf monocular depth estimator provides the input depth maps. Our method constructs depth-ordered merge trees through a deterministic union-find sweep, combining component area, depth persistence, and spatial contact constraints to guide region merging. A threshold search determines the segmentation granularity according to a specified target region count, while global depth detrending improves the separation of local geometric structures. Extensive experiments on COCO Panoptic, Cityscapes, and ADE20K demonstrate consistent improvements over classical segmentation baselines and competitive performance against zero-shot neural segmentation models. Notably, our method outperforms all evaluated zero-shot models on Cityscapes, while remaining competitive with several approaches on COCO and ADE20K. These results highlight the potential of purely algorithmic geometric reasoning for image segmentation without learned segmentation models.
[CV-130] SimplePro: A Simple Baseline for Proactive Video Understanding
链接: https://arxiv.org/abs/2611.23954
作者: Yilin Wang,Xiangxi Zheng,Dongxing Mao,Linjie Li,Zhengyuan Yang,Alex Jinpeng Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Proactive video understanding requires vision-language models (VLMs) to decide not only what to answer as a video unfolds, but also when newly arrived evidence warrants a response and when to remain silent. Although modern VLMs excel at offline long-video understanding, they struggle in proactive streaming settings, leading prior work to introduce task-specific reinforcement learning or auxiliary triggering modules. We find that explicit reporting rules and structured streaming context substantially improve a frozen VLM’s proactive response performance. We exploit this observation to build SIMPLEPRO, a training-free baseline for proactive video understanding. At each decision step, the model combines recent high-resolution frames, bounded historical observations, and prior answer records to determine whether new information warrants a response and, when needed, generate a concise answer. The system then filters consecutive duplicate answers while retaining the raw outputs as context for subsequent decisions. Without any parameter updates or auxiliary triggers, SIMPLEPRO with a 4B backbone consistently outperforms specialized proactive systems across OVO-Bench, StreamingBench-PO, and ProactiveVideoQA. SIMPLEPRO shows overall performance gains as backbone size increases from 2B to 32B. We hope this simple yet effective method will serve as a strong baseline for proactive video understanding, and we will release our code and inference pipeline to support future research.
[CV-131] MitoWorld: A Stochastic World Model of the Single-Cell Cycle under Genetic Perturbation
链接: https://arxiv.org/abs/2611.23953
作者: Ting Long,Xiaojing Sun,Cheng Han,Zhicheng Ji,Yingnian Wu,Jiebo Luo,Heng Fan,Dongfang Liu
类目: Computer Vision and Pattern Recognition (cs.CV); Quantitative Methods (q-bio.QM)
备注:
Abstract:A cell observed after a gene knockdown has an uncertain future, both in how its appearance will evolve and when it will divide. Existing approaches predict endpoints or population-level changes without jointly modeling individual trajectories and division timing. In this work, we present MitoWorld, a stochastic world model that generates these future appearances as continuous evolution in a Gaussian reference latent, terminated by a division hazard and converted into images by a conditional diffusion renderer. The ordinary transition provably preserves its Gaussian reference, tying its noise to its dynamics; paired observations identify its discrete transition where stationary marginals cannot. The hazard turns division timing into a survival likelihood that can be tested against real cycle lengths. On 325 held-out MitoCheck tracks spanning 17 knockdowns and a control, MitoWorld lowers FID by 34.6% and the video metric JEDi by 45.6% relative to the strongest baselines. Its latent-space prediction intervals achieve near-nominal 90% coverage, are 35% narrower than those of a model with freely fitted noise. The hazard also yields cell-cycle length distributions with KS distance close to the model-based finite-sample reference. Testable single-cell stochastic dynamics thus offer a generative tool for time-lapse perturbation screens.
[CV-132] VOL-Agent : Programming Your Film via Video Orchestration Language Agent ic System
链接: https://arxiv.org/abs/2611.23950
作者: Hongji Yang,Yucheng Zhou,Lingting Zhu,Shengju Qian,Xinye Li,Huan Zheng,Dongyue Lu,Wei Chow,Xin Wang,Cheng-zhong Xu,Jianbing Shen
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 14 pages, 6 figures
Abstract:Recent video foundation models generate high-quality and limited-length clips, but orchestrating them into minute-level, story-coherent films remains an open problem. Existing video agents highly rely on natural-language planning or predefined workflows, which make their production hard to program, customize, and extend. Therefore, we introduce VOL-Agent, a programmable video agent built on a Video Orchestration Language (VOL). With static compilation and verification, followed by runtime dynamic loading, VOL-Agent decouples structural reliability from execution flexibility. Through the skill-driven executor, VOL-Agent dynamically resolves tools and skills from an open-world repository and supports user-defined skills in natural language. Moreover, VOL-Agent is capable of crafting new director skills from learned knowledge and improves itself through feedback to accumulate filmmaking knowledge. To overcome the disconnect between different segments, we apply a structural memory mechanism to record and recall world and entity states, sustaining entity and scene consistency across different shots. Critic feedback supports local revision, while reusable filmmaking knowledge and experiences can be retained for subsequent tasks. Experiments on three benchmarks show that VOL-Agent generates minute-scale videos with strong overall quality and limited cost.
[CV-133] How Complete Must an Environment Be to Train a Capable GUI Agent ?
链接: https://arxiv.org/abs/2611.23946
作者: Jingxing Wang,Teng Wang,Xingyu Lou,Zihan Guo,Jun Wang,Weiwen Liu,Weinan Zhang,Jianghao Lin
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: 15pages, 6 figures
Abstract:Graphical user interface (GUI) agents are widely trained in executable GUI environments. To lower cost, build environments in batches, and reduce manual effort, each work trades away part of the environment through its construction choices. We describe these choices along three dimensions: reward coverage, the goal-reaching trajectories the verifier rewards; data coverage, the application data the agent is exposed to beyond what the task reads or writes; and feature coverage, the application features beyond what the task uses. Existing works set each dimension on engineering grounds, but they do not measure its effect on training. How complete must an environment be to train a capable agent, and along which dimensions? In MobileGym, we adjust each dimension separately, train Qwen3-VL-4B-Instruct, and evaluate every policy in the original full environment. A verifier that rewards only one reference path lowers success by 10.53 points, and removing unneeded records also lowers success. Pruning unneeded features instead raises success from 51.61% to 56.13%, entirely on tasks whose environments it left unchanged. The gain reverses when each episode is weighted equally and vanishes when only episodes on the pruned tasks are balanced. Without pruning, length weighting there reinforces long detour successes that erode skills on other tasks. Following these findings, we build MobileGym-Lite, which keeps reward and data coverage complete and prunes unneeded features. It outperforms Full in the full environment (55.48% versus 51.61% success) with lower construction cost.
[CV-134] SIGNTOK: A Text-Aligned Continuous Pose Tokenizer for Sign Language Generation
链接: https://arxiv.org/abs/2611.23944
作者: Nabeela Khan,Gaurav Rai,Bowen Wu,Sihan Tan,Benjamin Yen,Takeshi Ashizawa,Kazuhiro Nakadai
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Text-conditioned sign language generation (SLG) depends not only on the generator but also on the space in which signing is generated. Existing approaches generate directly in pose space, in quantized code spaces, or in learned latent spaces trained for reconstruction. We find that these spaces are organized largely by signer identity rather than linguistic content: on CSL-Daily, a test clip’s nearest neighbor in raw pose space is the same sentence signed by another person in only 2% of cases. To address this, we propose SIGNTOK, a continuous pose tokenizer whose latent space is shaped by three training-only heads: a kinematics head that predicts motion dynamics, a sentence-level head that aligns signing with the overall text, and a word-level head that aligns short motion windows with content words. Together, these objectives raise the same-sentence cross-signer nearest-neighbor rate to 82%. On three sign language benchmarks, the same generator trained in the SIGNTOK latent instead of pose space raises back-translation BLEU-4 in five of six benchmark-scorer settings at comparable joint accuracy, and word-level alignment gives higher back-translation scores than sentence-level alignment on every benchmark.
[CV-135] Angular Conditional Distillation: Overcoming Classifier Geometry Mismatch
链接: https://arxiv.org/abs/2611.23941
作者: Xuefei Zhang,Ruoqi Wang,Lei Cui
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Classifier outputs provide a fundamental and univer- sally available knowledge interface for knowledge distil- lation (KD). However, standard logit objectives entangle transferable semantic relations with architecture-dependent parameterization artifacts, causing a classifier geometry mismatch that makes direct knowledge transfer unreliable. To address this, we propose Angular Conditional Distilla- tion (ACD). First, the Angular Coordinate Interface (ACI) reconstructs the student classifier using angular coordi- nates, suppressing magnitude and bias effects to provide a geometry-compatible space. Then, Scale-Invariant Condi- tional Distillation (SICD) standardizes outputs and exclu- sively transfers conditional non-target relations, filtering out untransferable absolute confidence. Together, ACI de- fines where to transfer, while SICD defines what to transfer. Extensive experiments show that ACD achieves state-of-the- art performance over existing KD methods. Moreover, as a universal drop-in replacement, it consistently improves existing distillation pipelines while requiring no feature adapters or additional inference-time cost. Code is avail- able at this https URL.
[CV-136] Facet-SLAM: Real-Time Triangle Splatting SLAM
链接: https://arxiv.org/abs/2611.23935
作者: Edward Beng Wai Tan,Siew-Kei Lam
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Dense SLAM systems based on 3D Gaussian splatting build maps that render photorealistically, but their surface geometry is poor, and obtaining a mesh requires extraction or fusion stages. Triangle splatting avoids this with surface primitives whose vertices can be triangulated directly into a connected mesh, yet SLAM with this representation has so far relied on photometric tracking, which is too slow for localization at sensor rate. We present Facet-SLAM, the first real-time triangle splatting SLAM system for RGB-D cameras. Fast geometric registration is the natural alternative, but it is not sufficient alone, since the mapper’s optimization displaces triangles from the observed surface and makes them unreliable registration targets. We therefore separate the two roles: the tracker sets each triangle’s centroid from fused depth and registers frames to these centroids by GICP, while the mapper is restricted to non-translating updates via a change of variables. Our system achieves competitive tracking accuracy at around 100 FPS on a desktop GPU and runs at sensor rate on a Jetson AGX Orin. Compared with the photometric approach, it is over 20 times faster, matches rendering quality, and improves surface accuracy, reducing the chamfer distance of the Delaunay mesh on Replica from 1.55 to 0.86 cm. We will release the implementation at this https URL.
[CV-137] What Right-Angle Corners Reveal About an Uncalibrated Camera: Convex Cones Certified Bounds and Identifiability
链接: https://arxiv.org/abs/2611.23934
作者: Michael Werman
类目: Computer Vision and Pattern Recognition (cs.CV); Algebraic Geometry (math.AG); Metric Geometry (math.MG)
备注: 26 pages
Abstract:The corner of a box or a room, where three perpendicular edges meet, appears in an image as three segments meeting at a point. We ask what such junctions reveal about an uncalibrated camera’s intrinsic parameters. Each junction is consistent with an explicit convex set of cameras; when only the focal length is unknown, this set is an interval in closed form. These sets are computed from the image alone, so calibrating from many junctions is a convex feasibility problem. Almost every junction bounds the focal length from below; we characterize when one also bounds it from above, and when many junctions determine it exactly. For a known vertex, the bounds can be certified against any bounded error in the measured edge directions.
[CV-138] RoPE: Topology-aware Relative Positional Encoding for 4D Generation
链接: https://arxiv.org/abs/2611.23933
作者: Chengzhuo Gui,Zhenglin Zhou,Fan Ma,Zongxin Yang,Yi Yang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:Video-to-4D mesh generation has made substantial progress in recovering dynamic 3D geometry and motion from monocular videos. However, commonly used feed-forward approaches remain prone to sticking artifacts, where geodesically distant vertices move together incorrectly when they become spatially close. We attribute this failure to cross-part motion leakage: spatially nearby yet structurally distant vertices may retrieve similar motion information when motion decoding fails to distinguish spatial proximity from mesh connectivity. To address this issue, we introduce TRoPE, a topology-aware relative positional encoding that makes mesh connectivity explicit in the interaction between vertex queries and mesh-supported motion sources. TRoPE derives spectral features from the graph Laplacian and constructs pairwise topological relations from distances between multi-scale diffusion features, using these relations to modulate vertex–anchor attention through rotary phases. We further introduce a vertex-wise tail-risk objective that emphasizes persistently high-error vertices to explicitly suppress sticking artifacts. To better characterize and quantify these failures, we introduce MeshSticking-4D, a dedicated benchmark for this failure mode. Experiments on MeshSticking-4D show that TRoPE suppresses sticking artifacts while improving overall motion accuracy. Compared with the baseline, it reduces the worst-vertex error by 39.4% and lowers the separation failure rate from 49.35% to 34.90%. Additional evaluations on Motion-80 and Motion324-Subset show lower tail errors on diverse objects, while transfer to Mesh4D demonstrates applicability across backbones.
[CV-139] ProxySearch: Decoupled Inference-Time Scaling for Diffusion Models via Asymmetric Noise-Rank Transfer NEURIPS2026
链接: https://arxiv.org/abs/2611.23929
作者: Seungwook Kim,Jongmin Lee
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted to NeurIPS 2026. 26 pages, 7 figures
Abstract:Noise search at inference time improves text-to-image diffusion but requires many multi-step teacher rollouts per query. Distilled few-step models are now routinely released, yet their potential as proxies inside noise search is uncharacterized. We measure proxy-teacher rank agreement across three distillation regimes and three backbones (DiT, MMDiT, UNet) and find a sharp asymmetry. On our main pool, closed-form image statistics transfer from the proxy to the teacher (Spearman \rho up to 0.80), while learned semantic scorers do not ( \rho \le 0.12). Paired null controls trace this rank to the first denoising step for coarse statistics and to distillation for finer ones. We use the asymmetry for noise search under a joint constraint: a semantic reward specifies content, and a numeric target specifies image statistics. In PROXYSEARCH, the proxy filters candidates by statistical match in 1 step, and the teacher returns the survivor with the highest semantic reward. No weight trades one objective against the other. On FLUX-1024, PROXYSEARCH tightens the statistical target by 21-23% at the same number of teacher trajectories, with no detected loss on the optimized reward.
[CV-140] raining-Free Domain Generation for One-Shot Federated Learning with Server-Side Pre-trained Diffusion Models
链接: https://arxiv.org/abs/2611.23908
作者: Yixuan Tong,Yuanguo Bi,Mingjian Zhi,Wenchao Xu,Yuan Wu
类目: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Distributed, Parallel, and Cluster Computing (cs.DC)
备注:
Abstract:One-shot federated learning (OSFL) has attracted increasing attention for its ability to substantially reduce communication overhead by limiting client-server interaction to a single communication round. Nevertheless, this limited interaction makes learning from heterogeneous client distributions challenging. Diffusion-based OSFL approaches seek to address this challenge by synthesizing data that approximate client distributions. However, existing diffusion-based approaches often impose substantial computational costs on clients and lack effective visual guidance to generate data aligned with client distributions. To address these limitations, we propose FedDoG, a training-free Domain Generation framework that leverages server-side pre-trained diffusion models to synthesize data aligned with client domains. FedDoG extracts compact domain representations using a frozen image encoder, substantially reducing client-side computational and communication costs. These representations are incorporated into the diffusion model through a pre-trained image-conditioning adapter, thereby providing explicit visual guidance for domain generation. To further improve the quality of synthetic data, FedDoG adapts the strength of visual conditioning according to domain concentration and filters generated samples based on class consistency and predictive entropy. Experiments on three benchmarks demonstrate that FedDoG generates higher-quality synthetic images and outperforms existing OSFL baselines in classification accuracy. Meanwhile, FedDoG reduces client upload and computation time by 99.9% and 91.5%, respectively. The code is available at this https URL.
[CV-141] SixthView: A Benchmark for Hidden-View Reconstruction and 3D Assembly Reasoning
链接: https://arxiv.org/abs/2611.23891
作者: Pooya Khosravi,Jen-Feng Chang,Fnu Pramono
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 13 pages
Abstract:Inferring the unseen side of a 3D assembly requires reasoning about part placement and occlusion. Different assemblies can match the observed views while producing different hidden views, leaving the target ambiguous. We introduce SixthView, a benchmark in which models predict a sixth orthographic view from five observed views and information about the component shapes. The benchmark contains 602 synthetic polycube assemblies with 2–10 parts and requires a dense grid of part labels for the missing view, allowing both exact-view and cell-level evaluation. For each item, we exhaustively verify that all legal assemblies consistent with the exact component geometry and observed label grids yield the same target view. With these inputs, the task tests symbolic assembly and coordinate reasoning. The image-only variants additionally require shape recovery from rendered views, where occlusion can make different 3D shapes look identical. We evaluate three vision-language models under four input conditions. Across 552 items with complete evaluations, exact-grid accuracy with images plus exact text ranges from 33.2% to 85.4%. Adding exact text to separate images improves accuracy by 19.7–63.0 percentage points, whereas adding images to exact text changes it by only -0.5 to +2.1 points. Separate image panels outperform a merged sheet by 6.7–11.1 points. The results show that performance depends strongly on access to exact symbolic information, with little further improvement from images once the geometry is specified. GitHub: this https URL
[CV-142] Beyond Additive Composition: Interventional Modeling of State-Dependent Cross-Degradation Responses for Composite Image Restoration
链接: https://arxiv.org/abs/2611.23889
作者: Zongda Tang,Jie Feng,Wenda Zhao,Mingtao Feng,Yuping Liang,Ronghua Shang,Weisheng Dong
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Composite image restoration aims to recover images corrupted by multiple coexisting degradations. Existing methods predominantly decompose composite corruptions into individual factors, relying on these isolated priors to guide restoration. However, coexisting degradations are inherently interdependent: the manifestation of one degradation dynamically shifts with the states of others, inducing non-additive interactions that factor-wise paradigms fail to capture. To bridge this gap, we formulate state-dependent cross-degradation responses and propose PROBE, an interventional framework that explicitly models how coexisting degradations influence one another. Specifically, we first construct unified degradation axes by aligning visual and textual degraded-clean offsets. Along these axes, a calibrated coordinate head estimates current degradation states, while bidirectional feature perturbations are injected to probe cross-coordinate responses under the current states. Contrasting these responses with corresponding intensity-matched single-degradation references quantifies intrinsic non-additive deviations, allowing the resulting response descriptors and state coordinates to jointly guide restoration. For comprehensive evaluation, we introduce SCD-15, a new benchmark covering all 15 nonempty combinations of four degradation factors at three severity levels. Experiments demonstrate that PROBE achieves state-of-the-art performance on the CDD-11, AWMM-100K, and SCD-15 composite restoration datasets using a compact model with modest GPU memory requirements.
[CV-143] Can AI Understand Indonesian Childrens Book Illustrations? Compliance Safety and the Cultural Alignment Gap CVPR2026
链接: https://arxiv.org/abs/2611.23887
作者: Rian Adam Rajagede,Rochana Prih Hastuti,Fitri Hasanah Amhar
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted at Workshop on Computer Vision for Children (CV4CHL) at CVPR 2026 (Non-Proceeding Track)
Abstract:Children’s book illustrations carry developmental, pedagogical, and cultural responsibilities that are often formally codified in guidelines. As AI systems begin entering children’s publishing pipelines, both as evaluators and as generators, a natural question arises: can they reliably meet the structural, safety, and cultural standards required? We investigate this across three criteria: guideline compliance, safety, and cultural alignment, using Indonesia’s Book Leveling Guideline as a government-issued evaluation framework with explicit visual criteria for four reading levels. To understand how well VLMs can identify and apply these guideline criteria, we first evaluate them on real published children’s book pages and compare their assessments against a human baseline. To then examine whether AI-generated illustrations meet the same standards, we apply the VLM judge to images produced by two generation models and cross-check them against a subset of human illustrator raters. We find that VLMs are reliable on explicit criteria such as compliance and safety, but fall short on cultural alignment, systematically missing content that human raters flag as culturally inappropriate. AI-generated illustrations show the same blind spot: they satisfy structural and safety criteria but fail to capture implicit cultural details that human raters consistently identify.
[CV-144] NullTok: empirical-null token selection for training-free long video understanding
链接: https://arxiv.org/abs/2611.23881
作者: Zhang Ziyue,Juwei Jian,Baiyang Song,Yunshan Zhong,Jun Peng,Yiyi Zhou,Rongrong Ji
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 8 pages
Abstract:Long video understanding remains a daunting challenge for existing multimodal large language models (MLLMs), mainly due to the large number of visual tokens accumulated in long videos. Currently, frame selection and token selection are two main paradigms for addressing this issue, both aiming to select relevant visual information to form a compact input for MLLMs. A natural approach combines these two paradigms by selecting frames before pruning tokens, but noisy frame-level relevance estimates can cause irreversible evidence loss. In this paper, we study this problem from the perspective of assessing the spatiotemporal distribution of query-specific evidence and propose a novel and training-free approach termed NULLTOK. For every patch across all frames, we construct an empirical-null distribution from 64 group-matched irrelevant control queries, capturing the scores expected in the absence of target-specific evidence. Target affinities are calibrated against these same-position references into right-tail Monte Carlo p-values, which are discrete super-uniform under a conditional exchangeability assumption. Calibrated screening and a high-recall semantic guard produce candidate tokens for deterministic visual deduplication and joint spatiotemporal budget allocation. To validate NullTok, we apply it to two representative MLLMs and conduct extensive experiments on a set of video benchmarks. The experimental results demonstrate its effectiveness under a bounded visual token budget. For example, it achieves 65.81% and 68.85% on Video-MME using frozen LLaVA-Video-7B and Qwen3-VL-8B backbones, respectively, at a cap of 64 full-frame equivalents (FFE), without subtitles or audio. These results outperform all evaluated baselines on Video-MME under the same budget for both backbones. Code is available at this https URL.
[CV-145] Keep the Image in Mind: A Visual Board for Multimodal Agents
链接: https://arxiv.org/abs/2611.23872
作者: Haiming Zhu,Zhiwei Chen,Ding Cao,Tianxiang Zhao
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Multi-step visual reasoning suffers when early visual misinterpretations persist as incorrect premises in subsequent reasoning. We argue that how visual evidence is represented during reasoning matters as much as how much of it is gathered. Rather than appending observations and inferences to an ever-growing dialogue, we introduce a Visual Board-centered harness that separates visual observations, uncertain interpretations, and derived reasoning within a structured, capacity-bounded, and revisable workspace. This separation makes each premise explicit and auditable, so that conflicts between what is seen and what is derived become detectable even by a text-only critic and can trigger targeted reinspection of the original image. Without task-specific training, our harness consistently outperforms standalone models across 4B, 8B, Qwen3-VL-Flash, and 32B configurations on Euclid30K, SolidGeo, and ChartQA bar charts, with gains of up to 11.68, 18.42, and 24.82 percentage points, respectively. Under identical tools, critic, and interaction limits, the structured Board outperforms shared free-form notes by 6.67 to 13.00 points and no-Board retries by 7.35 to 18.80 points while using 7.9% to 38.2% fewer tokens. Supervised fine-tuning on harness trajectories further improves direct answering and critic-free tool use, with positive transfer to chart reasoning.
[CV-146] HOForce: Motion and Force Reconstruction of Hand-Object Interaction Using a Single-View RGB-D Video
链接: https://arxiv.org/abs/2611.23871
作者: Jiahui Wang,Wanqing Du,Yuanbin Zou,Zhefeng Xu,Feng Xu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Includes supplementary material. Project page: this https URL
Abstract:Reconstructing hand-object interactions is an important task, and achieving it with a single camera makes large-scale capture practical but challenging due to heavy occlusions. We introduce HOForce to track hands and objects, and further recover interaction forces from single-view RGB-D video and known object properties. For two-hand manipulation, we jointly model fingertip forces of both hands and the contact forces between objects when two objects are presented. The reconstructed forces also improve motion tracking by giving pose constraints in the occluded regions. HOForce recovers hand and object motion more accurately on DexYCB and TACO and produces less hand-object penetration than interaction baselines on DexYCB. On our dataset of fingertip forces recorded with tactile sensors, HOForce reduces normal-force MAE by 24.5% relative to the strongest baseline. The recovered forces reproduce object motion in MuJoCo and transfer to other engines after adaptation.
[CV-147] On Demand Not Uniformity: Adaptive Expert Orchestration for Multimodal Continual Instruction Tuning
链接: https://arxiv.org/abs/2611.23865
作者: Mengxin Qin,Kun Wei,Weitao An,Jiahua Li,Xu Yang,Cheng Deng
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Multimodal Continual Instruction Tuning (MCIT) enables Multimodal Large Language Models (MLLMs) to adapt to sequential instruction data while retaining previously acquired knowledge. While recent Mixture-of-Experts (MoE)-based methods mitigate forgetting by freezing historical experts and introducing new ones, they largely overlook the varying adaptation demands across tasks. Probing incoming tasks reveals pronounced heterogeneity in layer-wise gradient pressure and token-wise prediction difficulty. This observation highlights two unresolved decisions: where additional capacity is needed and how strongly new experts should contribute to individual tokens. To address both questions, we propose the Coverage-Calibrated Expert Injection and Residual Arbitration framework (CIRA-MoE), which derives expert acceptance margins via learnable threshold routing to serve as a unified coverage signal. Specifically, historical acceptance evidence guides the selective injection of new experts into insufficiently covered layers. Meanwhile, relative evidence between historical and current experts modulates token-wise residual contributions, coupled with arbitration-consistent regularization to promote expert specialization. Comprehensive experiments on UCIT and CoIN demonstrate that CIRA-MoE outperforms existing state-of-the-art methods in MCIT.
[CV-148] Grounding Radiology Reports in 3D Chest CT: Results of the ReXGrounding Challenge
链接: https://arxiv.org/abs/2611.23862
作者: Mohammed Baharoon,Mahmoud Hussain Alabbad,Sung Eun Kim,Xiaoman Zhang,Lucas Bijnens,Zirong Chen,Jiayu Wang,Xi Ouyang,Mingyang Yu,Weiqiu Jin,Xiaodan Ye,Dijia Wu,Kwang-Hyun Uhm,Inhwa Son,Rachel van 't Hull,Tim Veenboer,Cees Snoek,Kevin Groot Lipman,Daesung Kim,Junha Im,Youngjun Jun,Sangwon Lee,Katelyn Morrison,Adam Perer,Mijin Yun,Ali Piracha,Reem Alwasiah,Talal Yahyan,Abdulrahman Alamoudi,Mashael Alshehri,Rayan Alsager,Hannah Hancock,Anna M. Hunkemöller,Peter Youssef,Mona A. Alhammad,Afnan Aljumayan,Brecht Van Berkel,Eric Bijnens,Saud Alhussain,Maarten Steyvers,Rolf Symons,Bryan Eelen,Ayed Alaklubi,Maha Buraykan Alshammari,Amirah Fahad Alshammeri,Tomas Gilvydis,Ibrahim AlTurki,Monerah Almohideb,Ali Alburkani,Shahad Alhomaiani,Fatima Jabali,Maximilian Rokuss,Moritz Langenberg,Yannick Kirchhoff,Benjamin Hamm,Klaus Maier-Hein,Ibrahim Ethem Hamamci,Sezgin Er,Michael Moritz,Hassan R. Alomaish,Luyang Luo,Pranav Rajpurkar
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 25 pages, 13 figures, 4 tables
Abstract:Segmenting each reported finding in a radiology scan would allow findings to be measured and tracked over time and let AI systems generate reports that are verifiable, but so far AI models have performed poorly at this task. To advance this task, we organized the ReXGrounding Challenge at MICCAI 2026, the first challenge dedicated to it. Given a non-contrast chest CT scan and the set of findings from its report, participants trained models to produce the corresponding set of 3D segmentations. The challenge used 2,992 training and 200 validation scans from the ReXGroundingCT dataset, and evaluated methods on a held-out test set of 300 scans with 582 findings, exhaustively annotated by board-certified radiologists. In total, 48 teams from 15 countries made 760 submissions, and the best method reached a Dice of 0.375 and a hit rate of 0.77, matching agreement between two independent radiologist groups (Dice 0.32, hit rate 0.76, on a 50-scan subset). On the 100 scans of the pre-challenge benchmark, the winner reached a Dice of 0.390, against 0.285 for VoxTell, the best model before the challenge, which five of the six best teams fine-tuned. Non-focal findings remained far harder to segment than focal ones, instance F1 stayed below inter-reader agreement (0.34 vs. 0.47), and models sometimes segmented the right abnormality in the wrong location. Top methods thus match radiologist agreement in Dice, but non-focal findings, findings with multiple instances, and adherence to the anatomical location described in the text remain open problems.
[CV-149] MobileWeave: Program-First Synthesis of Dependency-Linked Tasks for Training Mobile GUI Agents
链接: https://arxiv.org/abs/2611.23860
作者: Ziyi He,Xian Wu,Qiyuan Chen,Haitao Li,Cehao Yang,Yinghao Zhu,Xichen Zhang,Xiao Chen,Yushi Feng,Lequan Yu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Native mobile GUI agents require large-scale training trajectories covering both individual app functions and multi-goal workflows within and across apps, which makes automatic task synthesis a promising way to scale training data. Existing synthesis pipelines, however, produce each task as free-form text, leaving implicit how its subgoals depend on one another. This matters because composition is a skill in its own right: a policy may complete each subtask of a workflow on its own, yet fail to chain these subtasks into the complete workflow. We present MobileWeave, which instead synthesizes every task as an executable program derived from the app implementations of a controllable simulator. MobileWeave abstracts each app into a typed capability graph with semantic types shared across apps, composes atomic programs into single- and cross-app composite programs through explicit value and object dependencies, and verifies each program by native execution, deriving both its instruction and its completion check from the same program. Trained on teacher trajectories collected for these tasks and on recovery supervision at the policy’s own stalls, MobileWeave-8B improves its base model by 26.7 points on MobileGym and by 8.6, 21.3, and 17.6 points on AndroidWorld, AndroidLab, and MobileWorld, with comparable gains at 4B. Beyond these gains, we find that composition is learnable from task structure: explicit dependencies between subgoals act as a training signal that teaches a policy to compose skills it already masters. We release data, model, and code at this https URL .
[CV-150] mporal Image Representations for Mouse Dynamics Authentication
链接: https://arxiv.org/abs/2611.23859
作者: Junyi Li,Charles Devlen,Daqing Hou
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 11 pages, 5 figures, 5 tables
Abstract:Mouse dynamics authentication has traditionally relied on handcrafted features or time-series representations that capture both spatial and temporal characteristics of mouse movements. More recent approaches have modeled mouse data as images, enabling the use of powerful pretrained computer vision models. However, prior work primarily focuses on spatial-only representations where the cursor coordinates are plotted as a trajectory image. These representations discard the temporal characteristics which can help discriminate users. In this work, we propose image representations that incorporate speed and velocity information for mouse dynamics authentication. We investigate two mouse data representations, a XYPlot representation that intuitively shows the path of movement, and a Pairwise Distance (PWD) representation that utilizes the entire image canvas to encode relationships between mouse events. Both representations are evaluated under both location-aware and location-free settings using pretrained 2D-CNN and ViT architectures. We found that the incorporation of temporal information improved performance over spatial representations alone, and that the location-aware variants generally outperformed the location-free. We achieve state-of-the-art performance on two datasets, Balabit and TWOS, and extend our evaluation to two other datasets. We found that in general, location-aware PWD with temporal encoding combined with the 2D-CNN architecture provided the best results, with mean EERs of 5.21% and 12.70% on the Balabit Test-1 and Test-2 sets, respectively, 4.94% on TWOS, 6.16% on ChaoShen, and 0.59% on DFL.
[CV-151] RoboBuilder: Agent ic Robot Co-Design with Real-world Video Feedback in the Loop
链接: https://arxiv.org/abs/2611.23855
作者: Ghadi Nehme,Dahyun Nam,Akshay Govind Srinivasan,Faez Ahmed
类目: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注:
Abstract:Can AI agents design robots that work in the real world? We introduce RoboBuilder, a benchmark in which coding agents co-design a robot’s hardware and program from standard parts, motors and a custom 3D-printed part, and every design that passes in simulation is built and tested on a physical course. Simulation alone is not enough: under public part data, none of 15 initial passing designs crossed the real course, so an agent learns the simulator from measurements it requests and videos of real runs. Across four frontier coding agents, ten obstacles at three difficulty levels, 16,427 simulations of 12,774 distinct robots and 135 physical robots built in an estimated 240 hours, agents solve nearly all easy obstacles and invent mechanisms our search baselines do not reach, but falter at harder levels, when one robot must cross every obstacle, and in the real world, where 66% of simulated passes hold across all levels. Closing the loop helps: shown a video of its real failure, an agent redesigns the robot, sometimes correcting its simulator, and removes 72% of level-1 and level-2 failures. RoboBuilder measures progress toward agents that engineer real machines. Project Page: this https URL
[CV-152] ReflectScene:MLLM -Guided Closed-Loop Diffusion for 3D Indoor Scene Generation
链接: https://arxiv.org/abs/2611.23845
作者: Jiawei Shen,Jie Feng,Junpeng Zhang,Guanbin Li,Wanyi Chen,Weisheng Dong,Guangming Shi
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 10 pages, 3 figures
Abstract:Text-conditioned 3D indoor scene generation requires layouts satisfying geometric, functional, and semantic constraints. However, existing diffusion methods rely primarily on static textual conditions, limiting their ability to assess evolving scenes and translate semantic discrepancies into executable object-level corrections. MLLM-based post-hoc refinement provides semantic feedback but remains decoupled from denoising, potentially disrupting scene this http URL propose ReflectScene, an MLLM-guided closed-loop diffusion framework. First, we introduce Hierarchical Prior-Grounded Scene Understanding that aligns structured layouts with multi-view visual evidence, to interpret object-specific context and scene global semantics via a frozen multi-modal large language model (MLLM). Second, we propose MLLM-Guided Semantic Reward Evaluation to distill MLLM’s diagnoses and assessments on possible candidate corrections into a lightweight Reward Predictor, enabling correction decision making with reduced computation burden. Finally, Decision-Conditioned Trajectory-Consistent Diffusion is proposed to integrate selected corrections through transient conditioning and persistent clean-estimate projection, correcting object layout in the generated scene while preserving non-target arrangements throughout this http URL on 3D-FRONT demonstrate improved functional consistency and semantic alignment while maintaining competitive generation quality and diversity.
[CV-153] OGBench: Benchmarking and Improving Tiny-Object Generation Fidelity in Aerial Imagery WACV2027
链接: https://arxiv.org/abs/2611.23840
作者: Chanhwi Jeong,Dongmin You,Seungwook Kim
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: Accepted to WACV 2027 Round 2. 20 pages, 7 figures
Abstract:Advances in deep generative models now enable the synthesis of realistic imagery. However, aerial imagery is dominated by tiny objects, such as vehicles, ships, and aircraft, which occupy only a few pixels owing to the overhead, wide-area nature of aerial imaging. Existing generative models struggle substantially more with tiny objects than with larger ones. This limitation has been overlooked, as prior work on generative evaluation emphasizes prompt following or human preference rather than whether small objects are rendered faithfully. To address this gap, we introduce TOGBench, a quantitative, object-level benchmark for evaluating tiny-object generation fidelity in aerial imagery. We further propose two simple train-/inference-time techniques that improve the ability of layout-to-image generators to render small objects, raising their performance on TOGBench.
[CV-154] Mapping the Design Space of Feature Sampling for Optical Flow
链接: https://arxiv.org/abs/2611.23822
作者: Liping Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:The computation of correlation features is crucial for optical flow estimation. Existing methods (e.g., RAFT) perform fixed-radius lookups on a cost volume, yielding a set of correlation features for each position; alternatively, WAFT performs single-point feature indexing, yielding one correlation feature vector per position. None of them designs a suitable correlation feature sampling scheme for each position according to regional characteristics. Our insight is that not every position requires the same sampling scheme. Applying an identical sampling scheme to every position is either redundant or insufficient and ignores the specific characteristics of the corresponding region. This paper rethinks the design of feature sampling and investigates the following question: should the support of feature sampling—i.e., the extent and directions over which evidence is aggregated—be learned separately for each query position? To study this question, we propose SAGE (Sampling with Anisotropic Gaussian Expectation). This method replaces single-point sampling with a per-pixel anisotropic Gaussian expectation on a fixed candidate grid centered on the current optical flow estimate. A positive-definite precision matrix controls the scale, anisotropy, and orientation of the sampling support, with single-point sampling included as its limiting case. Consequently, the sampling support becomes an explicit and analyzable design variable and provides a principled way to analyze how directional feature aggregation affects correspondence learning. Extensive experiments show that SAGE, while adding only 1.5% multiply-accumulate operations (MACs), effectively improves zero-shot cross-dataset generalization. When trained only on FlyingChairs and FlyingThings, SAGE reduces the end-point error (EPE) by 24.2% and 16.4% on Sintel clean and final, and reduces KITTI F1-epe and F1-all by 7.2% and 11.6%.
[CV-155] SWIFT: Slow World Instant Freshness and Timely Action for Asynchronous Autonomous Driving
链接: https://arxiv.org/abs/2611.23813
作者: Jiazhe Wang,Tianyi Yang,Ruihang Li,Fan Lu,Xianming Zeng,Jianyun Xu,Xingliang Liu,Lei Yang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Driving scenes keep changing during model computation, making both observation freshness and action-delivery time central to world-action planning. To retain rich generative world priors in autonomous driving without stalling real-time control, we present SWIFT (Slow World, Instant Freshness, and Timely action), a deployment-oriented asynchronous world-action system that moves world updates off the observation-to-action critical path. On a background GPU, a slow Video DiT streams surround-view history into read-only predictive world context, trained with a predictive span that covers both the update interval and background world computation. On a foreground GPU, a lightweight freshness carrier encodes current surround-view images into a gated key/value bank at every step, letting a fast Action DiT combine live visual feedback with the latest completed world context without waiting. On open-loop NAVSIM, SWIFT reaches 94.2 PDMS at 198.4 ms per decision with a trajectory scorer; on closed-loop Bench2Drive, it achieves an 80.30 Driving Score at 109.9 ms foreground latency. SWIFT further transfers to the Waymo E2E hidden test (8.0050 Rater Feedback Score) and nuScenes (0.54 m mean endpoint error, 0.03% mean collision rate). Across all four benchmarks, SWIFT achieves the best results among compared world-action methods, establishing a favorable accuracy-latency trade-off.
[CV-156] PReMind: When Personal Memory Meets Visual Hazards
链接: https://arxiv.org/abs/2611.23808
作者: Yeongtak Oh,Donghyeok Lee,Joonhee Oh,Jaewon Jeong,Sungroh Yoon
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:Personalized vision-language assistants use visual observations and personal memories to provide advice tailored to the user. When the observed scene contains a physical hazard, personal context, such as allergies, medical conditions, or mobility constraints, can change what help is appropriate. Responding appropriately therefore requires recognizing the visible hazard and applying the relevant person’s information. We introduce PReMind-Bench, a diagnostic benchmark that evaluates this combination across diverse hazards and personal-memory conditions. It separately measures memory access, grounding, abstention, and hazard response, and assesses joint success on interdependency cases. Existing models often satisfy one requirement while failing the other, and training only for safety or personalization leaves a gap in joint success. We therefore propose PReMind, a two-phase reinforcement learning framework that combines on-demand and forced-call interactions (memory always retrieved) to jointly train grounded memory use and hazard response. At 4B, PReMind raises joint success from 12.9% to 28.6% over its base on the interdependency set and improves hazard adequacy and correct abstention on the main test. Joint-success gains persist under forced-call evaluation and under a second LLM judge across three Phase II seeds; averaged over these seeds, joint success also improves on unseen hazard groups. These findings highlight the need to evaluate and train personal assistants to connect visual evidence with the relevant person’s context.
[CV-157] ROBIN: Unified Copyright Protection and Tamper Localization for Diffusion Models via Dual-Domain Synergistic Watermarking
链接: https://arxiv.org/abs/2611.23793
作者: Huayang Huang,Siqi Zeng,Qian Wang,Bo Du,Yu Wu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 2026 TPAMI
Abstract:Digital watermarking provides a promising solution for verifying the provenance and integrity of generated content. However, existing methods often suffer from an inherent trade-off between copyright robustness and localization sensitivity due to the entanglement of conflicting signals within a single domain. In this paper, we propose ROBIN++, a paradigm that shifts from single-domain entanglement to a Dual-Domain Synergistic design. Our key insight is to explicitly decouple robustness and fragility during injection by assigning them to distinct domains, while re-coupling them for mutual reinforcement during verification. Specifically, we introduce an Adversarial Optimization Algorithm to inject a frequency-domain copyright watermark during generation, maximizing extraction robustness without visual artifacts. Complementarily, a Content-Aware Perturbation generator is integrated into the latent space to inject the spatial-domain localization watermark, ensuring sensitivity to spatial tampering with minimal interference to the copyright watermark. During verification, we propose a dual-branch synergistic detector that facilitates Frequency-to-Spatial Synergy, where robust spectral priors act as global anchors for local anomaly detection. Conversely, a Spatial-to-Frequency Refinement strategy is employed to rectify corrupted latent features using predicted tamper masks. Extensive experiments show that ROBIN++ significantly outperforms existing approaches in both copyright recovery and tamper localization accuracy, effectively mitigating the long-standing trade-off between the two tasks. Code is available at this https URL.
[CV-158] When accuracy hides risk: risk-controlled confirmation of the critical view of safety under expert disagreement
链接: https://arxiv.org/abs/2611.23781
作者: Bowen Xiang,Xiao Yu,Mingze Xu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Automated assessment of the critical view of safety (CVS) in laparoscopic cholecystectomy has advanced rapidly, but a deployed check must decide, before the first clip, whether to confirm CVS. The standard rule calibrates a threshold so that false confirmations are rare on average. We show that this rule concentrates its errors on disputed videos, which one of three expert raters judged sufficient, and that the failure is structural. When reference labels are votes of a panel sampled from a larger expert pool, the rule that maximizes sensitivity at a fixed average false-confirmation rate also maximizes false confirmation of disputed videos, and for recognizers that recover expert judgement with less noise this rate rises with accuracy. We propose certified confirmation, a risk-controlled monitoring rule whose threshold is selected by multiple testing so that, with probability at least 1 - \delta, false confirmation is at most \alpha overall, among disputed videos, within vendor and country groups, and before CVS is visible, using disputed labels only to calibrate. In a pre-specified, hash-locked evaluation on the 300 test videos (18 countries) of the SAGES CVS Challenge, standard calibration falsely confirmed 8 of 39 disputed videos (0.21; one-sided p = 0.037 against \alpha = 0.10), whereas certified confirmation confirmed 2 (one-sided 95% upper bound 0.15). On development data, protection held for vendor and country groups left out of calibration. The cost was confirming 10% rather than 35% of sufficient videos on time. Average accuracy hides this risk; disputed cases must be protected explicitly.
[CV-159] DetectViT: Test-time Backdoor Detection for Vision Transformers via Inter-Head Attention Discrepancy NEURIPS2026
链接: https://arxiv.org/abs/2611.23770
作者: Siquan Huang,Yijiang Li,Xingfu Yan,Ningzhi Gao,Leyu Shi,Ying Gao
类目: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
备注: Accepted to NeurIPS 2026
Abstract:Vision Transformers (ViTs) now underpin the visual encoders of most multimodal models, but this success comes at a cost: relying on third-party pretrained checkpoints leaves these systems open to backdoor attacks. Existing test-time detection methods rely on either the predicted label or intermediate embeddings, limiting their applicability across diverse tasks and incurring substantial computational overhead. In this work, we study how backdoor triggers affect the multi-head attention mechanism of ViTs and identify two distinct anomalous patterns: \textitcomplete attention hijacking , causing the attention to exhibit abnormally high inter-head spatial overlap, and \textitpartial attention hijacking , causing an abnormally large variance in the concentration of the attention distribution. Motivated by these findings, we propose \textbfDetectViT , a test-time backdoor detection method that requires neither training data, model outputs, nor any prior knowledge of the trigger. DetectViT quantifies the two hijacking phenomena via an inter-head consistency score and an inter-head entropy variance score, with thresholds estimated from a small out-of-distribution (OOD) calibration set drawn from an arbitrary source. Across four representative attacks, diverse architectures (DeiT, CLIP, and LLaVA), and downstream tasks (image classification and captioning), DetectViT substantially outperforms all baselines with as few as 64–128 OOD calibration samples, achieving a true positive rate of 100% and a false positive rate of 0.17% on BadCLIP. Notably, it introduces negligible additional overhead by relying solely on attention weights already computed during inference. Code is available at: this https URL.
[CV-160] Keep the Reasoning Read the Image: On-Demand Visual KV for Multi-Turn Reasoning VLMs
链接: https://arxiv.org/abs/2611.23769
作者: Ziyi Wang,Xiyue Zhang,Bing Hu,Guangmin Song,Chenyi Zhang,Li Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Reasoning vision-language models (VLMs) think for thousands of tokens before they answer, and retention policies keep the KV cache affordable by pruning it as the reasoning grows. We find that this pruning sacrifices the image in multi-turn conversations: the image and the reasoning share one budget, so the growing reasoning crowds the visual entries out. A chart question that loses 8 points to the full cache when asked first loses 24 points when asked after a round of reasoning. Protecting a small visual reserve from eviction recovers most of this loss, while adding the same slots to the shared budget recovers considerably less. The image therefore lacks protection, not capacity. Based on this finding, we propose ShelfKV, a training-free method that works on top of existing retention policies. It keeps every visual entry in CPU memory and, at each decoding step, swaps the visual entries that the current query scores highest into the attention slots of the text entries it scores lowest. The image is never evicted, yet it occupies attention slots only while it is read. We evaluate ShelfKV on four benchmarks, three of them conversational, with two retention policies and two reasoning VLMs. In conversations, it improves the strongest existing policy, DBTrimKV, by up to 20.3 points at matched GPU memory, and still leads by 8.7 points when DBTrimKV uses twice the memory. It also consistently outperforms a fixed visual reserve of matched GPU memory in these settings, without hurting accuracy on standalone questions. These results show that the image should be protected from the reasoning but read only on demand.
[CV-161] Mean Curvature Minimizing Foliations Of A Vector Field
链接: https://arxiv.org/abs/2611.23765
作者: Tabish A. Syed,Marc Theveneau,Benedikt Wirth,Rustum Choksi,Kaleem Siddiqi
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Streamline patterns are ubiquitous in nature. Prominent examples include white matter fibre tracts in the mammalian brain and arrangements of elongated cells, such as cardiomyocytes in the heart wall. It has been posited that such organisations foliate into locally smooth surfaces to optimise function. However, the recovery of these foliations from a single vector field aligned with the local orientation of the underlying sheets remains an open problem. We address this challenge by introducing a mathematical model of a sheet and developing an algorithm for sheet inference, from measurements of the oriented structures. Given an input vector field, our model estimates a second vector field that is orthonormal to it, while also requiring the Lie bracket of the two to vanish. To balance sheet likelihood against low surface area, we also reduce the mean curvature of the recovered local surface. We compute the first variation of an associated energy and provide a numerical method which admits an efficient, parallel implementation for sheet estimation. We demonstrate our approach on both synthetic and biological examples, motivated by applications to medical imaging. These include white matter fibre tracts from a brain atlas and confocal microscopy images of cleared heart wall tissue. For the latter we are able to reconstruct sheets aligned with local cardiomyocyte directions at unprecedented (micrometre) scale.
[CV-162] GramStatTexNet: Efficient Interpretable and Neuro-Inspired Texture Analysis-by-Synthesis
链接: https://arxiv.org/abs/2611.23755
作者: Vasha DuTell,Zuyu Yun,Anne Harrington,Michael Y-S Fang,Mark Hamilton,Christian Kövesdi,Edward H. Adelson,William T. Freeman,Ruth Rosenholtz
类目: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Neurons and Cognition (q-bio.NC)
备注:
Abstract:Deep networks achieve high realism in texture synthesis but often rely on opaque, over-parameterized representations that have limited direct connection to biological vision. Neuroscience-informed pyramid models are interpretable, compact, and biologically inspired, but rely on small, hand-curated statistics sets that are difficult to generalize or extend. We introduce GramStatTexNet, a hybrid analysis-by-synthesis framework that combines the multi-scale Gabor filter structure of classical models with the flexibility of Gramian-based correlations, organized into structured filter families. This structured representation enables analyses that are challenging for deep-feature methods: each statistic carries a named filter-family identity, allowing us to decompose synthesis fidelity into per-family contributions and to learn an interpretable, low-dimensional embedding that preserves perceptual texture content. We compare our framework to classical and deep texture-synthesis baselines across diverse texture categories, and on multiple perceptual metrics, highlighting the compactness and interpretability advantages of this hybrid representation, alongside competitive synthesis quality. This same framework extends naturally to peripheral textures via spatial pooling, and to dynamic textures using a spatiotemporal filter bank. GramStatTexNet provides a unified, interpretable framework for analyzing and modeling visual information with natural extensions across space, time, and eccentricity. Project page: this https URL
[CV-163] Detect What You Need: Chain-of-Causal Reasoning for 3D Intent Grounding NEURIPS2026
链接: https://arxiv.org/abs/2611.23754
作者: Zihao Zhang,Aming Wu,Yang Li,Yahong Han
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted at NeurIPS 2026
Abstract:Accurately matching human intentions in 3D space is an important goal of artificial intelligence. Recently, 3D Intention Grounding (3D-IG) has emerged, aiming to localize target 3D objects given a natural-language intent. Unlike conventional visual grounding with descriptive referring expressions, 3D-IG intents are abstract and non-descriptive, making object localization substantially more challenging. This requires inferring latent functional requirements from non-descriptive intents and aligning them with object-level 3D representations. However, existing methods largely rely on implicit intent-object matching, leading to logical gaps and limited interpretability, robustness, and generalization. To address these challenges, we propose Chain-of-Causal Reasoning (CoCR), a causality-inspired functional dependency reasoning framework that explicitly bridges abstract intentions and candidate objects through intermediate functional requirements. Specifically, CoCR progressively decomposes complex intents into ordered functional requirements, thereby forming an explicit intent-function-object reasoning chain that links abstract intentions to object suitability. Building on this chain, we construct a causality-inspired functional dependency graph to model requirement-attribute relationships and introduce a causal-visual alignment module that aligns function-aware representations with the geometric-semantic features of 3D point clouds, enabling bidirectional verification between structured functional reasoning and visual evidence. Extensive experiments on 3D Intention Grounding and 3D Visual Grounding demonstrate that our method enhances intent-aware object localization. Our code is available at this https URL.
[CV-164] ACUS: An Angular Calibrated Uncertainty Score for Incomplete-View CT
链接: https://arxiv.org/abs/2611.23752
作者: Zihan Deng,Chuanzhi Xu,Quankeng Huang,Yujie Liu,Zhenzhen Gong,Junjie Huang,Tianye Niu,Lequan Yu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Sparse-view and limited-arc computed tomography (CT) are commonly treated as projection completion: a network fills an unmeasured view, and the result is ranked by similarity to one ground-truth image. That score can be high even when the query angle is far from every acquired measurement, because a learned prior may invent anatomy the scan never constrained. Existing protocols therefore leave a basic question unanswered: how does predictive reliability change with distance from the acquired views? We propose the Angular Calibrated Uncertainty Score (ACUS) to answer that question. ACUS is the Winkler interval score of a nominal 90% predictive interval on held-out projections, binned by distance to the nearest acquired view. It charges interval width and adds a large miss penalty when a held-out pixel falls outside the interval, so a narrow band helps only when coverage holds. We use collapse on acquired angles as a necessary consistency check on the predictor. On chest and dental cone-beam orbits, interval width grows with the angular gap. Empirical coverage ranges from 77% to 92%, and the peak signal-to-noise ratio (PSNR) of the point estimate falls as the width rises. Nearest-neighbor, interpolation, and point completion, each given one residual scale, have zero reported far coverage and incur outside-distance penalties. ACUS exposes gap-dependent overconfidence that a point-estimate PSNR alone does not measure.
[CV-165] GaussLearner: 3D Gaussian Primitives as Self-Regulating Continual Learners
链接: https://arxiv.org/abs/2611.23736
作者: Gaoxiang Linghu,Haonan Zhou,Xuanhe Liu,Weili Xu,Kaiyue Zhou,Jun Xiao,Gaoang Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project Page: this https URL
Abstract:Continual learning of 3D Gaussian fields aims to maintain faithful reconstructions of the evolving world. However, directly fine-tuning an existing Gaussian field on new observations can cause catastrophic forgetting of previously reconstructed content. To address this challenge, existing methods employ selective local optimization, replay, or parameter regularization, yet still struggle to balance adaptation to scene changes with preservation of still-valid content. Our key insight is to view 3D Gaussian primitives themselves as self-regulating continual learners. Based on this insight, we introduce GaussLearner, which equips each Gaussian with a closed continual-learning loop connecting observation, self-regulation, and adaptation at the primitive level. At each novel state, GaussLearner first attributes pixel-wise inconsistencies in current observations to their contributing Gaussians. Each Gaussian then self-regulates through a learning weight that balances current observations against cross-state reliability. This weight governs both Gaussian parameter adaptation and optimizer memory refresh. As current inconsistencies are resolved, the adaptation outcome renews each Gaussian’s reliability for future states. The resulting field is a globally coupled population of locally self-regulating learners. Across scenes from CL-Splats, WAT, and our CL-3D dataset, GaussLearner improves average PSNR by 3.41 dB over the strongest baseline. On CL-3D, it achieves the best reconstruction quality on both changed and unchanged views.
[CV-166] AdaS-ViT: Test-Time Adaptive Routing for Spiking Vision Transformers with Ultra-High Energy Efficiency
链接: https://arxiv.org/abs/2611.23728
作者: Zecheng Hao,Zihan Huang,Shenghao Xie,Jiyuan Zhang,Zhaofei Yu,Tiejun Huang
类目: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Spiking Neural Networks (SNNs) promise event-driven and energy-efficient vision, yet existing spiking vision transformers (S-ViTs) largely confine spiking tokens to fixed and uniform firing schemes along the temporal, spatial and channel dimensions, whereas suppressing the computational redundancy thus incurred calls for adaptive firing mechanisms that operate jointly across all three axes at test time. To this end, we propose the Adaptive Efficient Integrate-and-Fire (AdaE-IF) model, which consults the attention score of each spiking token and routes salient regions to fine-grained firing patterns while assigning energy-frugal coarse patterns to the remaining regions. Building on AdaE-IF, we further introduce Spiking Gated Self-Attention (S-GSA) and Spiking Local-Gated Unit (S-LGU), which enforce selective spike emission at token and channel granularity. We integrate the aforementioned mechanisms and modules into AdaS-ViT, a test-time adaptive-routing architecture trained with a progressive spatio-temporal curriculum that stabilizes the co-adaptation of firing precision and routing masks. Extensive experiments across benchmarks spanning diverse data scales and modalities demonstrate that AdaS-ViT consistently achieves highly competitive performance. Notably, on ImageNet-1k it consumes only 1.02 to 2.45 millijoules per image and attains an accuracy-per-energy ratio of 32 \times to 72 \times , operating in an economical inference regime and thereby pushing the energy-efficiency boundary further.
[CV-167] GTA: Generative Tower of Agents for Scalable Holonic Scene Generation
链接: https://arxiv.org/abs/2611.23727
作者: Zeng Tao,Yunuo Chen,Xiang Feng,Yumeng He,Peihao Li,Nanxiang Jiang,Yaowei Guo,Fanlian Zeng,Yin Yang,Chenfanfu Jiang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: this https URL
Abstract:Scalable world modeling requires coordinating scene-wide goals with fine-grained objects and dynamic characters under limited time and context budgets. We introduce GTA (Generative Tower of Agents), which represents every unit at every level of an adaptive scene hierarchy as a holon, both a self-contained construction task and a composable scene unit. Parallel agents recursively plan the hierarchy top-down, generate assets, and assemble and refine them bottom-up. Uniform interfaces carry scene-wide requirements down the hierarchy, guiding local construction so that each holon fits into the larger one. GTA accelerates scalable world modeling through high-throughput parallel execution while preserving well-structured layouts and fine-grained asset details. GTA then populates the assembled environments with dynamic human characters whose appearance and motion are guided by the scene, yielding realized 4D worlds. Project page: this https URL
[CV-168] Do 3D Point Representations Encode Architectural Functions? Probing Supervised and Pretrained Models on Urban Facades NEURIPS2026
链接: https://arxiv.org/abs/2611.23720
作者: Abdiaziz Muse,Jiju Poovvancheri
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: Accepted at the NeurIPS 2026 Workshop: Physical World AI: Geometry, Characteristics, and Multimodal Sensing
Abstract:Semantic labels say what a building element is, not what it does: two identical-looking windows on one facade may be operable or fixed, a column may carry load or only decorate. We ask whether 3D point representations encode such architectural function. On 22 mobile-LiDAR buildings with 15 functional attributes per element, we hold out entire buildings and compare hand-crafted descriptors, networks trained from scratch and frozen self-supervised point transformers. Descriptors and frozen features are read by the same linear layer. No network trained from scratch, from PointNet to a 108M-parameter PTv3, clearly exceeds the descriptors (0.610 mAP on functionally ambiguous elements), whereas frozen Utonia reaches 0.732. Random-weight controls of the same architectures (0.649 and 0.652) attribute the gain to pretraining (Utonia: +0.080, 95% building interval [+0.035, +0.124]). Pretrained features improve at every tested number of labelled buildings while descriptors and random features plateau, and, in the configurations we tested, fine-tuning erodes the benefit: at today’s data scale, function is better read from pretrained representations than learned from scratch.
[CV-169] CoScan: Multi-Scale Content-Adaptive Space-Filling Scans for Causal State-Space Image Restoration NEURIPS2026
链接: https://arxiv.org/abs/2611.23717
作者: Wei Dong,Terry Ji,Yan Min,Shahram Shirani,Jun Chen,Han Zhou
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted by NeurIPS 2026
Abstract:Selective state-space models (SSMs) have recently shown strong potential for efficient image restoration. A central difficulty, however, is that a 2D feature map typically needs to be linearized into a 1D token sequence for causal state-space updates, making restoration performance highly sensitive to the imposed scan order. Existing deterministic scans follow space-filling principles but remain fixed and content-agnostic, and can yield fragmented or semantically inconsistent contexts under degradations. We propose \textbfCoScan, a \textbfcontent-adaptive \textbfscan-order design for causal vision SSMs. CoScan learns image-dependent space-filling traversals via differentiable proxy supervision driven by \textbfscan-quality objectives that \textitencourage short-range coherence and causal predictability, while preserving spatial adjacency. To support hierarchical restoration backbones, we introduce an \textbfefficient multi-scale scan construction by coarsening a fine-level minimum spanning tree, producing coherent scans across feature resolutions. In addition, a \textbffrequency-guided start selection strategy is designed to \textitplace informative structures early in the sequence prefix to strengthen causal context formation. Plugged into existing Mamba-based backbones, CoScan yields consistent improvements across several restoration benchmarks with minimal overhead and shows promising results on high-level vision tasks. Code will be released at this https URL
[CV-170] A Hardware-Aware Comprehensive Study of Edge Vision Deployment with A Unified Hardware-Model-Runtime Mapping Framework
链接: https://arxiv.org/abs/2611.23705
作者: Le Duc An,Le Duc Thanh,Vo Hoai Viet
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:The rapid growth of deep learning has driven the deployment of computer vision directly on resource-constrained edge devices. However, the extreme heterogeneity of edge hardware architectures poses major challenges in selecting and designing suitable vision models. This survey bridges the gap between edge hardware constraints, model architectures, and deployment software toolchains. We establish a comprehensive taxonomy classifying edge hardware into three distinct tiers: mid-range devices (GPUs and dedicated NPUs or TPUs), dedicated embedded MPU or SoC processors, and microcontrollers (MCUs). We aggregate and analyze multi-source benchmark results for image classification and object detection across the computing spectrum, spanning from tens of watts down to milliwatt-scale bare-metal MCUs. Based on these empirical findings, we propose a two-tiered hardware-to-model mapping framework that provides concrete model selection matrices across performance priorities (Speed, Balanced, Accuracy) and actionable architectural design principles (peak SRAM budgeting, resolution-first scaling, and algorithm-compiler co-design). Finally, we highlight critical deployment challenges including toolchain fragmentation, operator incompatibility, and Vision Transformer feasibility, and outline promising future research directions in on-device adaptation and sustainable Edge AI.
[CV-171] What Do Learned 3D Models Add to Localization from Posed References?
链接: https://arxiv.org/abs/2611.23697
作者: Jaehyun Kim,Wonseok Kang,Jeongmin Lee,Tae-Wan Kim
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages
Abstract:Localizing a query from a few posed reference images avoids building a 3D point map. We ask what the camera predictions of learned 3D models add to geometric estimation from the same references, and where failures arise. Across eight learned models and variants and five test sets, we compare reference budgets, swap pose components, and intervene on the references a model sees. At fine pose-error thresholds, geometric estimation with one additional reference is significantly more accurate than every model on two references on Cambridge and NAVER B1 and never significantly worse, alone or as a fallback behind a shared acceptance gate; with the same three references, the more accurate approach depends on the scene. The gain from the learned translation magnitude depends on reference selection: it improves seven of the eight models on 12-Scenes with the spread-out pairs we select, but none of the five we test with the two top-ranked retrieved references. Where retrieval also returns references from other locations, at NAVER Gangnam Station, models fail at different stages. A robust alignment brings Pi3X and MapAnything within 3.3 pp of adding references as needed. In controlled tests, aligning on the nearby references largely restores VGGT, Pi3X and MapAnything but not Depth Anything 3, and pose-conditioned Pi3X recovers only if the far poses are masked and left out of its input-pose normalization.
[CV-172] MoMoVL: Modality-Aware Adaptive Compression for On-Device Sparsely-Activated Vision-Language Models WSDM2027
链接: https://arxiv.org/abs/2611.23695
作者: Dongkeun Lee,Wonjun Lee
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: 11 pages, 6 figures, Accepted at WSDM 2027 Findings
Abstract:The rise of the agentic web has been accompanied by growing demand for on-device Vision-Language Models (VLMs) that perform diverse cross-modal web tasks with enhanced privacy and reduced latency. Mixture-of-Experts (MoE) models are widely adopted in modern VLMs for compute-efficient scalability, yet their multi-expert architecture poses deployment challenges due to substantial memory demands. Mixed-precision quantization is an effective compression technique that reduces bit-widths while adapting to the heterogeneous hardware budgets of edge devices, particularly well-suited for MoE due to its inherent structural sparsity, which causes imbalanced importance among experts. While prior work has applied quantization for efficient MoE inference at the edge, we point out that in MoE-based VLMs, the modality-specific characteristics of visual and text tokens further impact expert quantization. To address this issue, we present MoMoVL, a modality-aware adaptive compression framework for MoE-based VLMs that brings vision-language inference within reach of on-device resource budgets. Motivated by our empirical findings on modality-induced disparities across experts, MoMoVL jointly exploits the per-expert activation profile and per-modality quantization sensitivity to guide bit-width assignment. Experimental results show that MoMoVL improves the accuracy-compression trade-off relative to state-of-the-art mixed-precision baselines and reduces memory footprint by up to 15.44% with a modest drop in accuracy over its uniform-precision baseline. Our code and data are publicly available at this https URL.
[CV-173] Structured Extraction of Diffraction Data from Covalent Organic Frameworks
链接: https://arxiv.org/abs/2611.23680
作者: Shenghan Gao,Zhiling Zheng
类目: Computer Vision and Pattern Recognition (cs.CV); Materials Science (cond-mat.mtrl-sci); Artificial Intelligence (cs.AI)
备注: 66 pages: main text (19 pages, 7 figures, 3 tables) and Supporting Information (47 pages) in one file. Data and code: this https URL ; database website: this https URL
Abstract:Powder X-ray diffraction (PXRD) is the primary structural evidence for covalent organic frameworks (COFs) and many other materials. Yet decades of measured patterns exist only as images in published figures. Automatic recovery is difficult, because PXRD figures vary widely in format, from a single curve to crowded stacks with labels, insets, and neighboring plots that challenge traditional computer vision. This work presents a pipeline in which a vision large language model (LLM) interprets the layout of each figure, locating the diffraction plot and its curves and sketching a coarse trace of each, while classical computer vision (CV) refines every trace to pixel-level accuracy. Applied to 3,954 COF articles, the pipeline produces 11,445 calibrated diffraction curves, released in the open database COF-PEAK. Compared with a hand digitization of 60 panels, extracted peak positions differ by a median of only 0.07 degrees. Whereas papers usually report only a few peak angles, which limits further analysis, the digitized curves enable tracking how peak prominence, a possible marker of crystallinity, changes with synthesis conditions. Vision LLMs and classical CV can thus turn such figures into reusable experimental data.
[CV-174] Learning What to Degrade: A Recoverability-Aware Curriculum for Reliable Smart-Glasses Super-Resolution
链接: https://arxiv.org/abs/2611.23674
作者: Yaopeng Wang,Qingliang Wang,Yilun Jin,Xiujin Liu,Han Yue,Chen Xia,Renjie Zhu,Chuang Sheng
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Degradations in smart-glasses images obscure small text and fine-grained structures needed by vision-language models (VLMs) for visual assistance. We introduce an high-resolution (HR)-referenced semantic evaluation protocol that compares VLM outputs on restored images with those on corresponding HR references. Our evaluation reveals two failure modes: generative SR can synthesize misleading details, whereas restoration-oriented SR can leave critical evidence blurred or ambiguous. These findings motivate improving faithful detail recovery in restoration-oriented SR, where a key training challenge is to provide sufficiently difficult inputs while retaining recoverable visual information. We propose ReDiC (Recoverability-Difficulty Curriculum), which combines a model-independent recoverability proxy with model-relative restoration difficulty to adapt training degradation severity. ReDiC increases severity when inputs are recoverable and restoration is easy, maintains challenging but recoverable conditions, and reduces severity when estimated recoverability is low. It modifies only training degradation generation, leaving the SR architecture and reconstruction objective unchanged. Across NAFNet, SwinIR, and MambaIR, ReDiC improves average PSNR by approximately 2.5 dB and HR-referenced semantic scores by about 3 points over the corresponding baselines on synthetic benchmarks, with further gains confirmed by LLM-based and human evaluations on 200 real smart-glasses captures.
[CV-175] Image-Condition-Controlled Diffusion Network with Large-Scale CT Pretraining for CT-Free Attenuation Correction in Dynamic Cardiac PET
链接: https://arxiv.org/abs/2611.23672
作者: Zhejia Zhang,Scott M. Leonard,Jinkui Hao,Ryan J. Avery,Richard L. Weinberg,Bo Zhou
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 11 pages, 6 figures, 3 tables. Submitted to IEEE Transactions on Radiation and Plasma Medical Sciences
Abstract:Positron emission tomography (PET) provides quantitative information on myocardial perfusion. In dynamic PET, tracer kinetics are tracked through a series of reconstructed time frames. Reliable attenuation correction is essential for accurate measurements. Clinical attenuation maps ( \mu -maps) are typically derived from computed tomography (CT), which adds radiation exposure and may introduce PET-CT misregistration due to respiratory and cardiac motion. Moreover, supervised CT-free methods rely on paired non-attenuation-corrected (NAC) PET and CT data, which are scarce, whereas large-scale unpaired CT data are more readily available. To address these challenges, we propose Large-scale Unpaired CT-Informed Diffusion (LUCID), a framework that transfers anatomical priors learned from 10,000 chest CT volumes in the public CT-RATE dataset to \mu -map synthesis for dynamic cardiac ^82 Rb PET. First, a convolutional neural network (CNN) predicts an initial \mu -map from each NAC PET frame. Second, the CT-pretrained diffusion backbone refines a low-noise perturbation of the CNN prediction through a single denoising step, conditioned on the corresponding NAC PET frame via ControlNet. Finally, frame-wise predictions are averaged voxel-wise to obtain the final \mu -map. We further introduce the voxel-wise temporal standard deviation (TSD) to quantify inter-frame variability without requiring a reference \mu -map. We evaluated LUCID on an in-house dataset of 102 patients and found that it achieved the best quantitative accuracy while maintaining competitive inter-frame consistency. Ablation studies showed that refinement with a diffusion backbone pretrained on in-house CT data from the training patients was less accurate than the CNN prediction alone, highlighting the importance of large-scale CT pretraining.
[CV-176] EyeDiT: Anatomy-Grounded Multi-Modal Eye Generation and Simulation
链接: https://arxiv.org/abs/2611.23670
作者: Yonghan Lee,Zachary Willms,Kun Liu,Pushkar Anand,Ali Behrooz,Dmitri Model,Dinesh Manocha,Yatong An
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Synthetic eye data are critical for rapid development of eye tracking (ET) in AR/VR applications. However, existing synthetic eye generators are largely image-centric and lack a high-quality underlying 3D anatomy of the eye, which limits their control mainly to the gaze and provides no anatomical labels. To address this problem, we propose EyeDiT, an anatomy-grounded, multi-modal 3D eye generation and simulation model trained with optical coherence tomography (OCT), labeled anatomy point clouds and near-eye images. Our pipeline first trains a Multi-Modal Anatomy Variational Autoencoder (VAE), which encodes an anatomy-aware latent representation from all input modalities. Then, our Eye Latent DiT is trained to generate latent samples conditioned on the optical axis, the pupil position and radius, and the subject identity. Together with Gaussian decoders for novel-view simulation, our model enables control over the pupil, subject identity and viewpoint for data synthesis. We show that our model outperforms previous eye image synthesis models in novel-view synthesis (24.9 against 21.8 dB PSNR from a single view), and that a public ET neural network trained on our synthetic data shows a smaller sim-to-real gap than when trained on data from other generators (3.5 ^\circ vs. 8.4 ^\circ gaze error), approaches a network trained on real data, and further improves ET performance when combined with real data (2.3 ^\circ vs. 2.6 ^\circ ). More information is available on the project page: this https URL
[CV-177] LightSEAD: Lightweight SE(3)-Aware Keypoint Detector through Selective Knowledge Distillation
链接: https://arxiv.org/abs/2611.23661
作者: Mayowa Adebambo,Alexander D. Rast,Andrew Bradley
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注:
Abstract:The pursuit of improved performance in deep learning has often been accompanied by increasing architectural complexity and computational demands. We explore an alternative direction: achieving more with less through structured supervision. We introduce LightSEAD, a lightweight SE(3)-aware keypoint detector initialised with calibrated Harris pseudo-labels and enriched through viewpoint-aware and appearance-aware supervision and selective knowledge distillation. LightSEAD comprises only six convolutional layers and 84,289 learnable parameters, achieving competitive repeatability across HPatches, KITTI 06 and 4Seasons while demonstrating its downstream applicability to relative pose estimation on MegaDepth-1500. Further optimised for mixed-precision inference, LightSEAD Coarse achieves approximately 909 FPS on an NVIDIA RTX 4090 and 139 FPS on a Jetson Orin Nano Super.
[CV-178] PRISM: Profiling and Representation of Individual Stereotypical Motion
链接: https://arxiv.org/abs/2611.23646
作者: Austin A. MBaye,Savik Kinger,Matthew S. Goodwin
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:An anatomical pose provides interpretable joint structure, but no fixed set of physical landmarks can fully specify a person’s movement. We introduce PRISM, a complementary representation that characterizes repetitive movement using tracked masks and dense optical flow. Torso-relative flow measures intensity, frame-wise mask variation measures spatial extent, signed-flow recurrence estimates frequency, and persistent sliding-window geometry describes the rhythmicity of the movement. Across 1,890 synthetic conditions, the measurements recover controlled speed, amplitude, and frequency while exposing observation limits. An idealized hand-motion experiment demonstrates variation omitted by identical exact COCO-17 trajectories; controlled corruption experiments separately examine measurement sensitivity. Audits of two pose methods on the same data cohort and 36,930 external skeleton clips document incomplete observation and dependence on processing choices. In 1,404 annotated complex stereotypical motor movement bouts from six autistic participants, total flow converges with pose motion, while matched-coordinate examples and repeated-session profiles show differences in intensity, extent, frequency, and rhythmicity within behavior labels. A separate VLM comparison evaluates quantitative frequency estimation, and a profile-conditioned decoder links the measurements to illustrative skeletal movements. PRISM connects numerical profiles to visible examples, providing an interpretable way to compare how the same behavior is expressed beyond its label or event count.
[CV-179] How Uncertain Is My Action? Self-Correcting Visual Planning with Frozen World Models
链接: https://arxiv.org/abs/2611.23620
作者: Siqing Li,Luming Yang,Yijia Jiao,Junze Deng,Alper Yilmaz
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:A frozen latent world model combined with sampling-based model predictive control is a visual planner. At deployment, the executed action can depart from the commanded one, and the frozen model has no way to notice. We show that the model can be run in reverse: for any hypothesized actuation shift, it predicts what it should have observed under that shift, and comparing this prediction with what was actually observed quantifies the shift and lets the planner correct itself. Unlike the residual energy, which mixes the shift with the model’s own error, the score keeps only the part of the residual that a shift can produce. This shift score detects the shift with a locally optimal test, estimates it with Cramer-Rao error bars, names the dominant generator, and compensates the planner once an alarm fires. On PushT, TwoRoom and OGBench-Cube with a frozen latent world model, measured detection follows the prediction to within 0.05 AUROC. Self-correction raises success from 0.650 to 0.862 on PushT and from 0.378 to 0.432 on the harder Cube task, levels with an oracle that is told the true shift at the same block, better than five common test-time adaptation methods. On a real robotic arm, the score identifies a base-joint offset that no joint encoder registers and reduces the reach error from 36 mm to 3-5 mm, and under shifts drawn at random and sealed before the session, estimates recover 92-94% of the error reduction achieved with the true shift. The broader implication is that a frozen world model can act as a filter by predicting the future, testing action-interface uncertainty and validity, and finally correcting failures.
[CV-180] Isthmus: Bridging Supervision Islands Restores Uncertainty and Stability in One-Step Flows
链接: https://arxiv.org/abs/2611.23616
作者: Luming Yang,Siqing Li,Haoxian Liu,Zhenyu Bu,Jiazheng Zhu,Alper Yilmaz
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:One-step flow maps are increasingly used as posterior samplers, where the point of sampling is the uncertainty that the samples carry. We find that the one-step sampler loses this uncertainty and becomes less stable. Compared with multi-step samples of the same network, its samples lose variance and decorrelate. Supervising the one-step jump directly makes this worse; the samples collapse further toward the conditional mean and break down when training is extended. We trace both failures to the locations where training places its pairs of start and end times. Only pairs on the diagonal of the time triangle receive targets from data, and the pair used for one-step sampling is either left unsupervised or supervised on an island, a set of pairs that the self-consistency identity does not connect to the diagonal, where the objective neither determines the one-step map nor anchors its training. We bridge the island with an isthmus, which places 5% of the training pairs along the whole edge of the triangle that joins the one-step pair to the diagonal. The error of the one-step map is then bounded by the self-consistency residual along this edge. With the network, the loss and the inference code unchanged and at no added cost, the isthmus improves one-step sampling across several flow-map objectives, in MRI reconstruction, face inpainting, and unconditional generation on CIFAR-10 and ImageNet 32x32. On brain MRI, it restores one-step calibration to 0.96 and the correlation with multi-step samples to 0.93; both vary far less across seeds, and the agreement holds when training is extended to three times its length.
[CV-181] Weight Space Programming for Vision Language Model Repair
链接: https://arxiv.org/abs/2611.23615
作者: Avinash Goutham Aluguvelly
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:In weight-space programming, a language model edits a network’s weights directly, without gradients. We ask what a successful repair of this kind relies on. We plant faults in the weights of two vision-language models, SmolVLM-256M and LFM2-VL-450M, and check each edit made by a coding agent against the original weights. Without being told where to look, the agent finds rescaled rows in the projector and noise in the vision encoder of both models, infers the unknown scale factors, restores most or all of the rescaled weights to their exact original values, and returns accuracy to near its original level; programs that it writes from such weights repair unseen checkpoints of the same model without feedback. These repairs depend on a trace of how the faults were computed. The original weights have 16-bit precision, and faults computed in 32-bit arithmetic leave low bits that only the damaged weights carry. Rounding those bits away leaves each model’s answers to our test questions unchanged, and the agent, unaided, then restores no weight exactly: on SmolVLM it no longer finds the damage, and in a single session on LFM2-VL its only gain comes from rescaling answer-token embeddings, which changes how often the model answers yes. Marking the damaged weights again in their low bits brings the edits back to them. Rounded weights still hold statistical evidence of the rescaling, in the distribution of stored values and in row norms, and one sentence in the prompt that names such a statistic directs the agent’s edits to the damaged rows on both models. Swapped rows or columns, which leave the set of stored values unchanged, are not found unaided on either model within our session budgets. We recommend that studies of weight repair report the number format of the weights and include a rounded control.
[CV-182] Entropy-Regularized Integrated Gradients for Stable Feature Attribution
链接: https://arxiv.org/abs/2611.23600
作者: Rukmangadh Sai Myana,Sumit Kumar Jha,Yanzhao Wu
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注: 36 pages, 19 figures, 27 tables, including the appendix
Abstract:Path-based attribution methods like Integrated Gradients (IG) are widely used to interpret neural networks, yet imperceptible input perturbations can drastically alter their attribution maps while leaving model predictions unchanged. We introduce Entropy-Regularized Integrated Gradients (ERIG), a path-based attribution method that intrinsically attenuates gradient irregularities along the integration path using maximum-entropy softmax weighting with temperature parameter \tau . We quantify the effect of the entropy-regularized weighting on attribution stability by using the Lipschitz properties of the model and its gradients. We decompose the attribution change under an input perturbation into a model-response term, whose bound is \tau -independent, and a weight-variation term, whose bound decreases at rate \mathcalO(1/(2\tau)) . Their ratio R(\tau) characterizes the trade-off between gradient information and attribution stability. We further add a new dimension to attribution evaluation by adapting classical noise estimators into two metrics, LINV-C and LINV-P, that quantify attribution noise. Across eleven CNN and Transformer backbones on ImageNet and Oxford-IIIT Pet datasets, our experiments show that, relative to the next-best method, ERIG remains comparable on ranking-based fidelity metrics while improving localization by approximately 24% and attribution stability by approximately 47-fold on average, reaching up to four orders of magnitude improvement over IG, all at negligible additional computational cost.
[CV-183] GRAFT: Graph-Regularized Prototype Fields for Realistic Test-Time Adaptation of VLMs
链接: https://arxiv.org/abs/2611.23591
作者: Maxx Richard Rahman,Wolfgang Maass
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Unlabeled test data can improve the zero-shot classifier of a vision-language model, yet transductive and test-time adaptation methods often fall below the zero-shot baseline when only a few vocabulary classes are present in a batch or when streams are correlated. We trace this failure to a shared design: every class owns independent parameters estimated from its own samples, so absent or rare classes are left at biased text anchors or fitted to foreign samples. We propose Graph-Regularized Anchor-displacement Field Transduction (GRAFT), which models the adapted prototypes as text anchors plus a displacement field that is partially pooled over a semantic class graph through a Gaussian Markov random field prior, together with an explicit class-presence model. Inference is closed-form and gradient-free on cached features. Anchor-only shrinkage and a single global shift are the two extreme settings of the prior; absent classes are provably harmonic extensions of the present ones, and the pooling strength can be chosen by empirical Bayes. Under the realistic batch and online protocols on eleven benchmarks and four ImageNet shifts, GRAFT improves over the zero-shot model in every setting and over anchor-based transduction where few classes are present.
[CV-184] ExiST: Existence-Aware Self-Training for Semi-Supervised Multi-Organ Segmentation
链接: https://arxiv.org/abs/2611.23581
作者: Maxx Richard Rahman,Prakhar Kumar,Wolfgang Maass
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Semi-supervised multi-organ segmentation should learn more than a dozen organs from a handful of labeled scans, and the smallest organs occupy less than one percent of the foreground. We identify a failure mode of self-training that is specific to this regime. When a teacher misses a small organ in a scan, every voxel of that organ receives the confident pseudo-label background, and the student is trained toward the organ absence. Repeated over many scans, this loop extinguishes tail organs. In a pilot study, removing the gallbladder from the pseudo-labels reduced its Dice on AMOS from 36.5 to 1.0. We argue that a missing organ should be treated as unresolved rather than absent and propose Existence-aware Self-Training (ExiST). It builds pseudo-labels by leave-one-out consensus of independently trained networks, screens them with a quality gate on validation scans, and infers, per scan and per organ, whether the organ was found. For every unresolved organ, the background voxels of that scan receive the set-valued target, trained with a marginal cross-entropy that accepts either answer and a Dice term that skips uncertain voxels. We show that this loss never pushes a candidate organ toward absence and never favors one member of the set, which leaves the decision to labeled evidence. ExiST outperforms the current state-of-the-art methods and reaches 77.1 mean Dice on Synapse with 20% labeled scans and 75.9 on AMOS with 5% labeled scans.
[CV-185] DOCK: Dual-Stream-Inspired Self-Supervised Object-Centric Representations Grounded in Robot Kinematics
链接: https://arxiv.org/abs/2611.23578
作者: Pengju Liu,Caiwen Ding
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注:
Abstract:Latent-space planners select actions by comparing the embedding of a predicted future image with that of a goal image. This assumes that distance in the embedding reflects how far the scene is from the goal. We profile this assumption with simulator-executed futures, which removes prediction error. For six frozen DINO and V-JEPA encoders, feature distance barely tracks the object’s distance to its goal: the rank correlation is at most 0.11, against 0.87 for object-mask centroids. It is dominated by background and robot patches, and frozen-feature readouts capture the arm far better than the object. Inspired by the ventral (what) and dorsal (where/how) visual streams, we propose DOCK, which learns an object representation whose changes agree with measured physical motion. A grasped object moves with the gripper, so forward kinematics on the joint angles gives a proxy for the object’s displacement without annotation. From 9,951 held-object transport trajectories in 21 simulated kitchen scenes, we mine 404K displacement pairs and train three components: an object token pooled from patch features (what), a spatial ruler that makes differences of readout coordinates equal to measured displacements (where), and a dynamics predictor over object tokens (how). Used as a planning cost, the learned representation places the object in 15 of 21 tasks, as many as an oracle that reads the true object position, while mask geometry reaches 10 and frozen feature distance 9. It still solves the most tasks when the unseen robot arm is visible, and its ranking transfers to held-out layouts with unseen object instances and categories without any adaptation.
[CV-186] Layer-Matched Sparse Attention for Low-Redundancy 3D Reconstruction
链接: https://arxiv.org/abs/2611.23554
作者: Sammam Mahdi,Fariha Binta Salim,K. M. Jubair Sami,Rakin Bin Rabbani
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:VGGT reconstructs cameras and geometry from unposed images in one pass, but its global attention grows quadratically with the number of views. Training-free accelerators thin this attention and work well on redundant video, but lose pose accuracy when views share little geometry. This paper traces these losses to two kinds of cross-view information that a uniform sparse approximation can fail to preserve together: diffuse layers aggregate distributed context, whereas matching layers depend on query-specific interactions that carry little attention mass. A per-layer statistic measures the log-mass missed by a mean token. Its row-free proxy is the logit variance, and it selects the estimator for each layer. Diffuse layers use shared, importance-sampled hub keys, with log-weights that make the softmax numerator and denominator unbiased. Matching layers stay exact: every aggressive thinning tested cost accuracy. At equal cost, the selected layers outperform the four shifted windows tested. On 13 unseen low-redundancy scenes with 32 views, LMSA stays within 1.4 AUC@5 of dense attention, compared with 10.3 for AVGGT and 4.5 for SAF3R; its difference from contemporaneous GPA-VGGT is not significant, and only its gains over AVGGT pass both scene-level tests. With 100 views, it stays within 1.9 AUC@5 while its aggregator runs 2.0\times faster. Used post hoc to choose the layers GPA-VGGT protects, the statistic improves it by 1.2 AUC@5 for 5% more FLOPs. Applied untuned to \pi^3 , it beats AVGGT but remains further from dense accuracy.
[CV-187] Stability of Parallel Transport Convolution under Riemannian Metric Perturbations
链接: https://arxiv.org/abs/2611.23543
作者: Hongnian Huang,Andrew Stump
类目: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Differential Geometry (math.DG); Machine Learning (stat.ML)
备注: 78 pages. Comments welcome!
Abstract:Geometric convolutions on Riemannian manifolds depend on the geometry of the underlying domain, making stability under metric perturbations a fundamental question in geometric deep learning. This paper studies this question for parallel transport convolution through methods from differential geometry and analysis. On a fixed compact manifold, we establish that, for a continuous input signal and a continuous reference kernel supported in a sufficiently small normal neighborhood, the convolution output depends continuously in L^2 on the Riemannian metric in the C^2 -topology, under the specified localization and transport-path conventions. The analysis develops perturbation estimates for geodesics, parallel transport, Jacobi fields, and exponential maps, and uses these estimates to compare the transported kernels. A central difficulty is the cut locus, which obstructs uniform comparison of the transport construction over the entire manifold. We address this difficulty by combining uniform estimates away from the cut locus with integral bounds on neighborhoods of arbitrarily small volume, while also controlling the change in the Riemannian volume measure. The resulting qualitative stability theorem connects perturbation theory for Riemannian geometric constructions with the analysis of spatial convolution operators in geometric deep learning.
[CV-188] ORGANIC3D: Grounding 3D CT Vision Language Models in Organ Level Image Text Coupling
链接: https://arxiv.org/abs/2611.23529
作者: Zelong Liu,Hamza Ahmed,Wynn Wingyi Lee,Mickael Torjman,Alexander Dimitrakakis,Jiahua Lu,Timothy Deyer,Bachir Taouli,Zahi A. Fayad,Xueyan Mei
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Vision language models (VLMs) have achieved strong performance in 3D computed tomography (CT) classification, but high accuracy does not reflect whether their predictions rely on the correct anatomical evidence. We introduce ORGANIC3D, a framework for evaluating and improving organ level image text alignment through controlled anatomical interventions. Our benchmark assesses 12 vision encoder checkpoints and 7 generative VLMs using matched patient comparisons, organ erasure, lesion replacement, and laterality perturbations, revealing substantial alignment failures even among top performing models. To learn organ level image text correspondence, ORGANIC3D aligns each finding sentence with the anatomy aware image features and trains on edited scans to anchor this alignment in the organ’s image evidence. We further extend this framework to generative VLMs, promoting evidence dependent question answering and report generation. Trained exclusively on public data, ORGANIC3D achieves SOTA classification AUROC on CTRATE, Merlin, and the heldout RADChestCT dataset and reaches 77% laterality accuracy, compared with at most 66% for prior models. Its generative VLM matches or exceeds all evaluated publicly released VLMs on visual question answering, achieves the highest report accuracy, and retracts 96 to 99% of target organ findings after organ removal, versus a maximum of 69% for other models. Together, these results establish ORGANIC3D as a framework for testing whether 3D CT vision language models ground their predictions in anatomical evidence and for improving organ level alignment alongside predictive accuracy.
[CV-189] Rag eVLA: Augmentation-Trained Perception and Action for State-of-the-Art Robustness on LIBERO
链接: https://arxiv.org/abs/2611.23517
作者: Amrit Lal Singh,Smarak Kanjilal
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages, 10 figures, 9 tables
Abstract:Vision-language-action (VLA) models look saturated on LIBERO, yet most of them fall apart on LIBERO-PRO as soon as an object moves or the instruction asks for something new. We argue that what is missing is not scale but decoupling. In these policies, what to do is tied to where things were during training, to the scene around them and to the wording of the instruction. We cut these ties in two places. In the data, LIBERO demonstrations are re-arranged by general rules: every object and fixture is moved and rotated, tasks are carried into other environments and onto other surfaces, and every instruction is paraphrased. In the policy, a perception engine on frozen DINOv3 features, trained on this augmented data, hands the action model the scene in decoupled form: a list of objects with their positions, and a compact code of what the gripper is looking at, next to image and instruction embeddings from TinyCLIP. A flow-matching diffusion transformer, trained from scratch, acts on this view. We call this combination of Robust Augmentation and Grounded Encoding RageVLA. With 0.21 billion parameters, RageVLA sets a new state of the art on LIBERO-PRO among VLA policies, scoring 0.72 against 0.53 for the best published model, and a larger variant reaches 0.73. Two further tests on LIBERO-Object show that the policy acts on what it perceives. When the object positions it is given are shuffled while the camera images stay correct, its success falls from 0.965 to 0.215. When the target object is displaced by 28 cm, it still succeeds in 0.865 of the episodes, where \pi_0.5 , run through the same harness, succeeds in 0.105. Comments: 10 pages, 10 figures, 9 tables Subjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV) Cite as: arXiv:2611.23517 [cs.RO] (or arXiv:2611.23517v1 [cs.RO] for this version) https://doi.org/10.48550/arXiv.2611.23517 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Amrit Lal Singh [view email] [v1] Tue, 17 Nov 2026 21:07:47 UTC (662 KB)
[CV-190] Beyond the Sink Token: Sink-Guided Key Editing in Vision-Language Models
链接: https://arxiv.org/abs/2611.23516
作者: Hui Zhu,Saleh Zare Zade,Rafi Ibn Sultan,Xiangyu Zhou,Dongxiao Zhu
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:In vision-language models (VLMs), a few sink tokens attract a disproportionate share of attention. While previous studies have examined why these tokens attract attention, we investigate how the sink-key direction affects attention to image and question tokens. Across four probed VLMs, boosting the magnitude of positive sink-aligned components increases attention on average, whereas boosting negative components decreases it; these responses persist through later layers. This identifies the native sink key as a reference for controlling attention beyond its own position. Motivated by this finding, we propose SHRINK, a training-free method that partially reduces sink-aligned components across visible keys during generation. On nine benchmarks under sampled reasoning, it improves mean accuracy by 5.1 and 2.7 percentage points on LLaVA-OneVision-7B and Qwen2.5-VL-3B, with smaller gains on the other two main models, Qwen2.5-VL-7B and Gemma-3-4B. Beyond SHRINK, rescaling sink-aligned key components can replace the attention operations of two existing attention-steering methods, GIFT and ACG, with comparable accuracy on hallucination benchmarks.
[CV-191] rust the Ray Not the Depth: Ray-Lateral Agreement for Multi-View 3D Human Pose Fusion
链接: https://arxiv.org/abs/2611.23513
作者: Farnoosh Koleini,Ahmed Helmy,Pu Wang,Xiwu Cao
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages, 5 figures, 1 table
Abstract:Put a monocular mesh estimator behind every camera of a calibrated rig, and the reconstructed bodies land a meter or more apart. The cause is geometric: each estimate sits at the depth implied by a focal length the network assumed, not the one the camera has. Its error therefore points almost entirely along its own viewing ray, and the calibration tells us that direction exactly. To address this, we introduce the Ray-Lateral Agreement (RLA), which follows one rule: trust the ray, not the depth. RLA first slides every estimate along its own rays, which removes the depth bias and leaves the shape untouched. It then fuses the views one point at a time, under a Gaussian error whose precision follows each viewing ray. The maximum-likelihood solution is closed-form, interpolates between averaging and midpoint triangulation, and stays well conditioned where triangulation is singular. Robustness is the harder half, because correct views also disagree along their rays. Any weight built on the full residual therefore punishes good views together with bad ones. Ray-lateral agreement instead scores every view, at every point, by its residual across that view’s own ray alone. We evaluated RLA on multi-camera public benchmark datasets. Across two, three, and four cameras and three per-view pose estimators, RLA consistently achieves lower worst-case error than all triangulation baselines.
[CV-192] MedHarness: A Unified Budgeted Runtime for Reliable Medical Imaging Agents
链接: https://arxiv.org/abs/2611.23501
作者: Haofan Wu,Le Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Medical imaging agents already return answers on heterogeneous imaging surfaces, yet deployment still faces a limited verification budget where each wrong accept can harm clinical trust. When a thin check and a deeper evidence pass disagree on the same case, deciding when to stop, open more evidence, repair, or reject without a shared controller remains ill-posed. Tool-using workflows often lock a surface-specific evidence order per dataset, or a private always-open full-depth verification path; those pipelines stay costly, leave many wrong accepts, and cannot share one controller across modalities. We present MedHarness, a registered multi-surface runtime reliability layer that reformulates post-answer control as shared decision-object attachment after a proposer’s initial prediction. Under Shared Decision Object, Minimum-Sufficient Verification, and Intermediate Repair, one shared policy with surface recipe adapters lets a new imaging surface join the same stop / open-evidence / repair / reject controller by recipe registration. On a 512-case live pool, MedHarness cuts wrong accepts versus a full-verification baseline by about 47% and raises accepted accuracy to 83.2% vs. 72.9% at 86.1% coverage versus full release under that baseline. On a failure-enriched multi-surface stress ranking, the same controller cuts estimated remote service calls per case by about 71% versus that baseline under a matched context-load setting. A held-out three-surface check and an alternate language-model transfer keep the same control ordering. The code is available at this https URL.
[CV-193] GRID-UAV: Geometry and Ray-depth Integrated Decision-making for Unmanned Aerial Vehicles
链接: https://arxiv.org/abs/2611.23491
作者: Anas Anwarul Haq Khan,Amrin Kareem,Muhammad Umer Sheikh,Biplab Banerjee,Muhammad Haris Khan
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Vision-language-action models (VLAs) encode images with 2D encoders that capture little 3D structure. A common approach is to feed the policy estimated depth, but a UAV chooses its own altitude, must judge distances tens of meters away, where monocular depth is less reliable, and must relate its forward and downward views. On long aerial routes, adding depth features directly brings no significant gain (42.5% vs. 42.6% success), so we study how 3D information should be given to the policy. Analyzing flight failures, we find that two forms of 3D grounding fail in opposite ways: explicit 3D lifting of depth collides less on the way but more often fails to finish at the target, while implicit alignment with a 3D geometry model finishes more often but collides more. We propose GRID, which combines both: (1) it lifts monocular depth into ray-corrected 3D positions in the UAV’s frame for both cameras, and (2) during training only, it aligns its visual features with a frozen multi-view geometry model, at no inference cost. GRID adding explicit lifting to implicit alignment significantly increases how often the policy reaches the target ( 61.5% \rightarrow 69.5% , p0.001 ), and adding implicit alignment to explicit lifting significantly increases how often it completes the task once there ( 57.7% \rightarrow 61.4% , p0.05 ). On TravelUAV (3,005 test episodes), GRID achieves the best success rate on every split, with relative gains over the state of the art of 9.5% (seen), 17.1% (unseen objects) and 25.4% (unseen map).
[CV-194] Reference-Guided Data Curation for 3D Medical Vision-Language Pretraining WACV2027
链接: https://arxiv.org/abs/2611.23484
作者: Faidra Patsatzi,Ashwin Kumar,Chong Wang,Curtis P. Langlotz,Reinhard Heckel,Akshay S. Chaudhari
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted at WACV 2027. Code: this https URL
Abstract:Training 3D vision-language models (VLMs) on large medical imaging datasets collected across institutions is constrained by data quality. Vast, uncurated pools of image-report pairs are characterized by redundancy, class imbalance, and cross-institutional heterogeneity. We address this challenge of data curation for improving 3D VLM pretraining by proposing an offline, training-free, reference-guided data curation recipe. Our approach leverages a small, high-quality reference set to independently extract quality signals for each modality of a large, uncurated pool and merges them through a rank-aggregation step into a single ranking that guides sample selection. We validate this method on datasets comprising 35k 3D abdominal CT volume-report pairs from two institutions and 58k 2D chest X-ray image-report pairs. On 3D CT, using our reference-guided data curation strategy, our curated model matches the zero-shot disease classification performance of a state-of-the-art baseline (Merlin) while using 3.3\times less training data and a reference set of 5k samples. On 2D chest X-rays, our method yields a curated model that matches or exceeds the zero-shot disease classification performance of a model trained on the full pool while using up to 10\times less training data and a reference set of only 74 samples. Ablations show that the selection is stable under resampling of the reference, that smaller reference sets still outperform random sampling, and that label rebalancing does not improve performance. To support reproducibility, we release our code.
[CV-195] mmTherm: Absolute 3D Human Pose Estimation from Low-Resolution Thermal Images and mmWave Radar
链接: https://arxiv.org/abs/2611.23477
作者: Zhaoxin Chang,Badii Jouaber,Daqing Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Absolute 3D human pose estimation (HPE) aims to recover both a person’s body pose and their global position in 3D space. Privacy-preserving millimeter-wave (mmWave) radar provides distance and direction measurement capabilities, but its sparse point clouds make HPE under-constrained. Low-resolution thermal imaging can provide complementary observations of body direction and structure. However, our evaluation shows that a strong feature-level fusion baseline loses much of its localization and HPE advantage when generalizing to unseen subjects, with radial localization becoming less accurate than a radar-only baseline. Motivated by this finding, we introduce mmTherm, a lightweight framework that explicitly incorporates the complementary geometric properties of thermal imaging and mmWave radar. It combines body structure and joint directions estimated from thermal images with range information from radar to localize the body and reconstruct its 3D pose. Experiments on OctoNet show that mmTherm reduces cross-user absolute joint error by 33% and root localization error by 38% relative to the strongest corresponding baselines, while requiring substantially fewer parameters and less computation at inference.
[CV-196] VeriMath: Human-in-the-Loop Generation and Verification of Curriculum-Aligned Mathematics Assessment Items
链接: https://arxiv.org/abs/2611.23456
作者: Quy P. Nguyen,Duong Q. Nguyen,Dinh P. T. Nguyen,Truong-Giang T. Le,Truong A. Bui,Vi H. Vo,Quang-Thinh Bui,My-Phuong Ngo,Trong N. Do,Thao P. Le,Phuc M. Nguyen,H. Nguyen-Xuan
类目: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Automatic generation of mathematics assessment items is a growing area in educational artificial intelligence, yet classroom deployment remains challenging. Generated questions must satisfy multiple constraints: mathematical correctness, curriculum alignment, appropriate cognitive difficulty, and educator reviewability. While prior work has advanced text generation, reliable assessment authoring requires granular control over localized examination formats and expert educational judgment. To address this challenge, we present VeriMath, a generalized, human-in-the-loop framework for generating rigorous, curriculum-constrained mathematics assessment drafts. Using annotated reference items, the framework anchors generation to the requested examination section and cognitive reasoning level, then applies sequential automated verification and guided repair before submitting drafts for expert review. We evaluate VeriMath through a national curriculum case study using 4,555 mathematics items from the Vietnamese Grade 12 curriculum and 50 distinct generation requirements covering multiple-choice, true/false, and short-answer formats. Across two model backbones, the framework produces verifier-accepted drafts for 94% (47/50) of requirements per run. Blind expert evaluations confirm high mathematical accuracy and curriculum alignment, with strict teacher-readiness assessments accepting up to 56.8% of generated items. Analysis of rejected drafts reveals persistent domain-agnostic challenges involving formatting, statement validity, and solution-step completeness. These findings demonstrate that VeriMath can bridge generative AI and dependable educational applications by producing highly reviewable assessment drafts within tightly constrained national curriculum frameworks.
[CV-197] NIRJALA: Draining the Sea out of Gaussian Splatting with Three-Path Optics and Reversible Physics
链接: https://arxiv.org/abs/2611.23450
作者: Latchan Chhetri,Aman Kumar
类目: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
备注: 21 pages, 11 figures
Abstract:Water does not just tint an underwater photograph; it rewrites it. Each pixel mixes attenuated scene radiance, backscattered light, and scene light scattered forward on its way to the camera, and many combinations of scene and water explain the same pixels. Most Gaussian-splatting methods for underwater scenes model only the first two effects and remove the water by dividing by a transmission that decays exponentially with range, so their inversion gain grows without bound. We present NIRJALA (Sanskrit for “without water”), a 3D Gaussian Splatting framework that recovers water-free appearance and geometry together. TRIVENI, a three-path image formation model, adds a range-saturating forward-scattering term to direct transmission and backscatter; we prove that this keeps the inversion gain bounded at every range and bound how errors in the estimated backscatter and attenuation propagate into the recovered colours. PRATIBIMBA, a reversible physics anchor, ties a flexible neural water model to a ten-parameter physical model in both directions, so scene content cannot hide inside the water. SAMARASA collects the physics-informed objectives, including a range-binned colour prior. To score water removal against ground truth, we built SAJALA (“with water”), a 30-scene suite path-traced in Blender Cycles with water-free images, depth, and normals under controlled water colour and turbidity. On SAJALA, NIRJALA raises PSNR from 24.34 to 24.76 dB and lowers depth AbsRel from 0.109 to 0.091 against the strongest baseline; on real SeaThru-NeRF scenes it lowers LPIPS from 0.194 to 0.175 while rendering at 44 FPS. We also discuss where appearance metrics and geometry disagree underwater, and the limits of our evaluation.
[CV-198] Where Global Context and Weight Sharing Work in Compact Vision Backbones
链接: https://arxiv.org/abs/2611.23446
作者: Michael Schroeder
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: 8 pages, 5 figures
Abstract:Compact convolutional backbones such as EfficientNet achieve strong parameter efficiency through local depthwise separable convolutions, yet two structural questions remain central to hybrid design: where does global self-attention improve recognition over pure convolution, and can repeated blocks share weights without losing representational capacity? We investigate both questions on ImageNet-1k (50,000-image validation) using controlled EfficientNet-B0 experiments. First, we show that wrapping a classification backbone in multi-scale feature pyramids (BiFPN) fails to outperform a plain MBConv trunk (69.99% - 71.81% vs. 72.86% plain B0) until paired with stage attention (73.15%). In contrast, keeping the MBConv trunk intact and inserting lightweight zero-gated residual self-attention adapters at stages P3 (28x28 windowed), P4 (14x14 global), and P5 (7x7 global) reaches 74.02% standalone and outperforms a bit-for-bit matched plain-B0 control by +0.954 percentage points (73.88% vs. 72.92%) across an 88-epoch stateless replay schedule, with learned attention amplitude rising monotonically with semantic depth (27.2% at P3 - 45.4% at P4 - 66.3% at P5). Second, we evaluate recurrent weight sharing across repeated MBConv blocks with pass-specific BatchNorm statistics (+1,536 parameters per pass). While post-hoc kernel averaging degrades accuracy to 17.03% and single-template stage tying (-27.6% parameters) plateaus at 66.10%, alternating two shared block templates ([B1, A, B, A]) in Stage 6 cuts total model parameters by 10.9% (582K weights) and recovers by +10.82 points in a single epoch to 70.92% (-1.84 points vs. unshared control).
[CV-199] Why Sub-4-Bit Vision CNNs Collapse and Where Mixed-Precision Recovery Stops
链接: https://arxiv.org/abs/2611.23445
作者: Michael Schroeder
类目: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: 8 pages, 6 figures, 2 tables
Abstract:Deep vision models operating in resource-constrained environments depend heavily on low-bit quantization, yet compressing weights below 4 bits frequently induces catastrophic accuracy collapse. While mixed-precision recovery strategies aim to rescue accuracy by selectively retaining high-precision parameters, the fundamental mechanics of where and why these models collapse remain poorly understood. In this study, we systematically analyze the anatomy of quantization collapse across standard Vision CNNs (including ResNet and EfficientNet backbones) evaluated on the full 50,000-image ImageNet-1k validation set. First, we map the exact progression of degradation, demonstrating that collapse is highly non-linear and concentrated in late-stage pointwise projections and depthwise layers. Second, we introduce a structured, multi-step recovery waterfall that evaluates selective integer precision (INT4/INT8) and selective floating-point scales. We show that keeping less than 5% of critical layer activations in 8-bit precision recovers up to 92% of the accuracy lost during aggressive sub-4-bit compression. Finally, we bridge the hardware reality gap by evaluating deployment performance on standard ARM and x86 CPU architectures. We show that while low-bit quantization reduces memory storage requirements, standard runtime software bit-unpacking introduces a 1.48x to 21.5x execution slowdown depending on batch size, highlighting a critical trade-off between model footprint and actual edge throughput. Our findings provide a practical blueprint for navigating the limits of sub-4-bit deployment without sacrificing hardware performance.
[CV-200] ransFIRA: Transfer Learning for Biometric Recognizability Assessment
链接: https://arxiv.org/abs/2611.23442
作者: Allen Tu,Kartik Narayan,Joshua Gleason,Jennifer Xu,Matthew Meyn,Tom Goldstein,Vishal M. Patel
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project Page: this https URL
Abstract:Biometric recognition in surveillance, video, and web imagery must contend with extreme variation in pose, blur, illumination, occlusion, and distance, where visual quality metrics fail to predict whether inputs are recognizable to the deployed encoder. Existing face image quality assessment methods rely on visual heuristics, curated annotations, or costly generative pipelines, leaving their predictions detached from the encoder’s decision geometry. We introduce TransFIRA, a lightweight framework that grounds recognizability directly in embedding space. TransFIRA derives class-center similarity (CCS) and class-center angular separation (CCAS) from a pretrained encoder’s embeddings and learns to predict them from images without quality annotations. Using predicted CCS for template weighting and CCAS0 as a natural filtering threshold yields state-of-the-art face verification performance on BRIAR, raising TAR at 10^-6 FMR from 0.085 with uniform averaging to 0.598. The predicted scores also reveal how image degradations affect encoder-specific recognizability. TransFIRA+ extends this framework to cross-camera multimodal person identification in untrimmed surveillance video. In this setting, body recognizability scores saturate within a narrow range, making direct weighting effectively uniform. We introduce a post-hoc sigmoid calibration that transforms these saturated scores into informative weights without retraining. After calibration, the BRIAR-trained TransFIRA predictor improves fused face and body identification on two unseen multi-camera datasets, outperforming detector-confidence weighting and uncalibrated CCS-rank weighting. Together, TransFIRA and TransFIRA+ demonstrate that encoder-grounded recognizability generalizes across biometric modalities and recognition settings.
[CV-201] Bits Not Boxes: Label-Free Layer Selection for Visual Localization in VLMs
链接: https://arxiv.org/abs/2611.23432
作者: Haisheng Fu,Gexin Huang,Haikai Zhao,Jie Liang,Zhenman Fang,Jingning Han,Xiaoxiao Li
类目: Computer Vision and Pattern Recognition (cs.CV); Methodology (stat.ME)
备注: 35 pages, 9 figures, 56 tables
Abstract:Attention maps cheaply show where a vision-language model (VLM) looks, but attention sinks make them unreliable for localization. Contrasting the question with a generic prompt suppresses sinks, yet only some layers localize the queried object. Existing methods find these layers with box annotations or fixed rules. We show that attention itself can identify them. A question changes how much a head attends to the image and where this attention goes. The two effects separate exactly, and the second, measured in bits on unlabeled questions, indicates which layers localize. WhereBits turns these where-to-look bits into a layer prior and combines it with per-head contrastive scores. Its three design constants were fixed once on labeled COCO data, and a new model or resolution needs only unlabeled questions. Across eight VLMs, WhereBits matches the pointing accuracy of box-tuned layers on COCO and RefCOCO and ranks target tokens better. This advantage holds on a held-out model family and at higher resolutions. Its COCO pointing accuracy is 0.72, against 0.22 for mean attention. In training-free zoom-in, its crops raise V* accuracy by 6.0 points under forced-choice evaluation.
[CV-202] Autoregressive View Model
链接: https://arxiv.org/abs/2611.23417
作者: Jensen(Jinghao)Zhou,Samuel Rota Bulo,Denis Rozumny,Duncan Zauss,Peter Hedman,Nikhil Keetha,Philip Torr,Christian Rupprecht,Andrea Vedaldi,Peter Kontschieder
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project Page: this https URL
Abstract:We present the Autoregressive View Model (AVM), which generates a scene as a stream of views without constructing an explicit 3D representation. Through a unified in-context interface, AVM conditions on images or videos to generate short clips along a requested camera path. Visual context grounds reconstruction and interpolation, while a learned prior enables extrapolation into unseen regions. AVM retrieves relevant views from inputs and generated history to form a bounded active context. We fine-tune a pretrained bidirectional video diffusion transformer with flow matching in an N-in-1-out formulation. We then tune a causal model on its own rollouts to improve photorealism, reduce denoising steps, and limit long-horizon drift. We scale training to approximately 8M scenes and video clips. We evaluate generation capacity and scene persistence across benchmarks for camera control, text control, visual quality, dynamics, sparse and 4D reconstruction, and long-horizon consistency. The results demonstrate that AVM adapts to diverse input and output settings through one in-context interface.
[CV-203] OCLDA-GS: Object-Centric Local Deformation-Aware Gaussian Streaming
链接: https://arxiv.org/abs/2611.23416
作者: Ashim Dahal,Nikesh Subedi,Rabab Abdelfattah,Nick Rahimi
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Streaming 3D Gaussian Splatting (3DGS) allows for online reconstruction of dynamic scenes from multi-view videos for Free Viewpoint Video (FVV). Current time and storage efficient methods rely on selecting a subset of anchor Gaussians and neighborhood interpolation for fast streamable 3DGS. However, these methods can overlook object-specific motion and introduce inconsistencies near object boundaries. We propose OCLDA-GS, an object-centric streaming framework that dynamically reallocates object and motion-aware Gaussian anchors to allocate anchor Gaussian support for objects undergoing large motion while at the same time, restricting the gaussian neighborhood within object groups for more precise object boundaries. Furthermore, it estimates object-wise rigid transformations (e.g. a hand moving) and leverages these estimates to learn per-anchor local deformation correction for small deformations (e.g. wrinkles on a shirt). Then, we conduct experiments on three dynamic multi-view video datasets and demonstrate improvements over the previous baselines alongside improved quality between refinement timesteps. Under matched refinement budgets, our method improves PSNR over Instant Gaussian Stream (IGS) by 0.169 dB on N3DV, 0.202 dB on Meeting Room and +0.199 on Google Immersive, while reducing the mean PSNR drop between refinement timesteps by 9.2% and 13.8% in N3DV and Meeting Room, respectively.
[CV-204] RFBench: A Simulator-Grounded Multimodal Benchmark for Passive RF Layout Design and Understanding
链接: https://arxiv.org/abs/2611.23410
作者: Jiarui Cui,Boxun Yan
类目: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: 29 pages, 4 figures. Code: this https URL . Data: this https URL
Abstract:While multimodal language models can interpret diagrams and generate structured outputs, their capacity to connect passive radio-frequency (RF) design intent, layout geometry, and electromagnetic behavior remains largely untested. To address this gap, we introduce RFBench, a simulator-grounded dataset and benchmark for passive RF layout design and understanding. The dataset contains 10,000 two-port planar layouts simulated using ADS Momentum, with each layout represented by editable geometric primitives, a raster image, a 16x16 occupancy grid, complex S-parameters at 201 frequency points, a structural category, and an RF response label. Building on these aligned representations, we define seven tasks spanning specification-conditioned layout generation, representation translation, response prediction, simulator-based validation, design diagnosis and improvement, structural understanding, and response classification. Extensive baselines across seven multimodal language models reveal systematic weaknesses in generating structurally valid layouts and predicting electromagnetically faithful responses. We show that RF-specific fine-tuning increases T1a structural validity from 0.0% to 83.2% and the T3 primitive-input within-2-dB rate from 17.7% to 62.8%, demonstrating the value of domain adaptation while leaving considerable room for further progress. RFBench establishes a common testbed for evaluating multimodal RF capabilities and guiding the development of models that connect design intent and layout representations with physical behavior.
[CV-205] Lossless Reversible Watermarking for AIGC with Applications in Authentication
链接: https://arxiv.org/abs/2611.23395
作者: Yupeng Qiu,Han Fang,Ee-Chien Chang
类目: Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:AIGC watermarking is commonly used for tracing, but it can also support applications such as authentication and tampering detection. We study lossless reversible watermarking, which enables exact recovery of the original watermarked image after permissible distortions. By exploiting the relatively low entropy of the information required for image generation, together with robust latent-space watermarking, we design a framework that preserves sufficient information for lossless recovery. We then apply the framework to AIGC authentication, which generally faces two key challenges. First, the authentication mechanism must resist adaptive adversaries. Second, the content relevant to authentication, such as visual designs or expressed messages, may depend on application requirements that are not known at generation time. To solve these difficulties, we incorporate cryptographic message authentication to provide unforgeability against adaptive attacks. Lossless recovery provides the original image as reference evidence, after which a suitable analyzer can be selected to make the authentication decision. Experiments on Stable Diffusion demonstrate exact recovery under a wide range of distortions.
[CV-206] Asymmetric Optical Image Watermarking
链接: https://arxiv.org/abs/2611.23350
作者: Guang Yang
类目: Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
备注: 10 pages, 4 figures, 5 tables
Abstract:Watermark detectors held by a provider cannot be checked by third parties, and a party that can detect a mark can usually also forge it. We study a two-layer image watermark in which the Fourier transform at the core of the pipeline is performed by a coherent optical 4F unit. Layer 1 carries an Ed25519 signature bound to the image content in Fourier magnitudes, so anyone with the public key verifies offline. Layer 2 is a key-defined carrier written by a secret phase hologram and read by a matched-filter correlator. The optics is a scalar-diffraction simulation with finite modulator bits, pixel aperture, pupil, defocus, misalignment, and sensor noise. No hardware is built. All criteria were fixed in a hashed specification before any optical result. The simulated optical signer reproduces the digital pipeline (field error 1.7e-15, bit agreement 99.997%), verifies at least 99.5% of 877 images under each of 8 post-sensor settings, one unattacked, and accepts none of 40,670 unmarked images. The optical carrier is key-selective: the matched key detects it on 99.8% of the unattacked images, and detection falls to 0% under a scale error of 1% and to 0% under a mask shift of 1 bin. Security rests on the signature and the content binding, for which we separate what is provable from what is measured.
[CV-207] orchtest2: One-Call Sanity Checks for PyTorch ModelsEvaluated on Real and Unseen Architectures
链接: https://arxiv.org/abs/2611.23273
作者: Grzegorz Gwardys
类目: oftware Engineering (cs.SE); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Many bugs in deep learning code are silent: a layer that is never called, an optimizer created before the classification head was replaced, a recurrent layer that treats the batch as time, or an ignored attention mask. The code runs, the loss decreases somewhat, and the defect surfaces, if ever, after days of GPU time. We present torchtest2, an open-source library that runs 18 sanity checks on a single batch with one call, restores the PyTorch state it touches, and adapts to model families through plugins. We evaluate it by mutation analysis with 17 classes of silent bugs on 103 models from torchvision, timm and Hugging Face transformers, on 600 random architectures and on 20 hand-written realistic bugs, against the checks of the original torchtest, overfitting a single batch, a smoke test and a training monitor in the style of AUTOTRAINER and DeepDiagnosis. Every mutant is compared with its healthy counterpart, and intervals come from a bootstrap over models. After freezing the tool, we ran it on 191 models of architecture families it had never seen: counting, for every tool alike, a failing report where the healthy model’s report does not fail, it detected 78% of the injected bugs (95% CI 75-80), against 40% for the original torchtest checks and 48% for the training monitor. On real models its batch-independence check found a bug in timm’s GCViT that couples the images of a batch. Real models also exposed false alarms that neither the tool’s own test suite nor random architectures had revealed. Two independent collections of real silent bugs mark the limits of the approach: only 3 of 28 reproduced PyTorch bugs from Stack Overflow and 3 of the 18 silent training errors released with TrainCheck are within reach of a pre-training model tester. A median check takes 1.0 s. We release the library, the benchmark and all raw results.
[CV-208] ConserveEdit: Source-Anchored Transactions for Drift-Bounded Iterative Image Editing
链接: https://arxiv.org/abs/2611.22724
作者: Shuyang Jiang,Fucheng Deng,Yuchuan Luo,Zhenyu Wu
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: 28 pages, 15 figures, 16 tables; includes supplementary material
Abstract:Instruction-driven image editors produce convincing local edits in one pass, but inside an iterative refinement loop every accepted output becomes the next input, so unintended changes outside the requested region accumulate. We present ConserveEdit, an editor-agnostic controller that treats each refinement step as a transaction against a contract compiled once from the source image. The contract separates what may change (the authorized region of each target, with a bounded halo for shadows and reflections) from what must survive (protected content with explicit tolerances). Candidates from a frozen editor are composed through a source-anchored mask and committed only if target progress, drift on protected content, and image quality pass as separate tests; otherwise the step is rolled back and repaired. For any editor, this keeps every pixel outside the authorized region bit-exact and bounds drift independently of the number of iterations, whereas no fixed scalar trade-off reproduces the separate tests. On all 1053 MagicBrush test edits with a frozen Qwen-Image-Edit, masking is necessary but not sufficient. A direct call already alters the rest of the image (12.1 dB PSNR outside the mask); composed through the source-anchored mask, it stays bit-exact there and its DINO similarity to the ground truth rises from 0.67 to 0.85, matching ConserveEdit, so this gain comes from the mask. Next to automatic masks, however, masked direct editing changes a protected face or text on 48% of turns, and masked scalar selection avoids this mostly by abstaining (23% constrained success). Because its drift test names the violated entity, ConserveEdit shrinks the support instead (2% violations, 53% constrained success); in a simulator with known ground truth it raises constrained success from 55.0% to 61.8%. Automatic grounding remains the main bottleneck. Code and logs will be released.
[CV-209] 4D-TEMPO: Differentiable 4D-STEM ptychography via fast Physics Optimizers
链接: https://arxiv.org/abs/2611.24776
作者: Ramani Duraiswami,Yichao Zhang
类目: Instrumentation and Detectors (physics.ins-det); Materials Science (cond-mat.mtrl-sci); Computer Vision and Pattern Recognition (cs.CV); Mathematical Physics (math-ph); Computational Physics (physics.comp-ph)
备注: 34 pages: 8-page paper, references, and 23-page supplementary material; 10 figures, 10 tables
Abstract:Electron ptychography recovers atomic structure from 4D-STEM data: diffraction patterns at multiple positions of a scanned probe. Current codes reconstruct an image and read atoms off it, or fit atoms under an L_1 loss; neither gives per-atom uncertainty. We propose 4D-TEMPO, a differentiable physical renderer that fits the atoms, the microscope, and the detector directly to the recorded electron counts under a Poisson likelihood. The output is a list of atoms, each a Gaussian primitive with a 3D position, a thermal width, and a model-conditional error bar from the Fisher information. The Fisher information also conditions the solver, gives the dose (electrons per Å ^2 ) each parameter needs, and separates calibration from imaging. On a simulated WSe _2 bilayer, starting from the published reconstruction, fitted atoms cut in-plane error by a third (3.2 vs. 4.8 pm) and depth error by 40-61% against its 3D peaks. Depth comes from multiple scattering into the dark field, the faint counts outside the direct beam. The two layers, 10\times closer than the depth of field, separate; per-atom depth then follows the layer geometry. When the model is well specified, the bars are calibrated (69% of position coordinates within 1\sigma ). On experimental WSe _2 , fitted atoms predict held-out patterns better than the three published codes we test. With free probe modes where atoms are incomplete, the same likelihood predicts best on all three held-out benchmarks, matches every WSe _2 baseline in 5 min against up to 20.5 h, and passes torchslice at its released setting 180 \times sooner on PrScO _3 . Per-atom error bars, planned doses, and fast reconstruction point to low-dose, real-time 3D imaging of beam-sensitive and dynamic materials.
[CV-210] ransferable Named Sparse Features for Morphology-based Gleason Grading
链接: https://arxiv.org/abs/2611.24363
作者: Argyrios Zafeiriou,Athanasios Kallipolitis,Ilias Maglogiannis
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Deep learning systems grade prostate biopsies at the level of pathologists, but most report a grade without the glandular architecture that justifies it. This work presents a grading system built on a sparse code over the tokens of a pathology foundation model. A sparse autoencoder pretrained on biopsies is aligned with multi-reader annotations, so that 69 of its 2048 features are named after Gleason morphologies such as cribriform glands, cords, and comedonecrosis. The same code feeds a grading network, which reads all features to predict Gleason patterns and the Grade Group, while morphology maps read directly from the named features. We evaluate the system on five public datasets that include needle biopsies, tissue-microarray cores, multi-reader annotations, and three scanners. The morphology maps remain stable across alignment runs and, without retraining, detect cribriform glands in external biopsies (area under the receiver operating characteristic curve, AUROC, 0.849) and pattern 5 in unseen cores when evaluated by up to six pathologists (AUROC 0.846 to 0.919). In an independent biopsy cohort, Grade Group agreement with three pathologists on Leica scans is close to their agreement with each other. Removing the named features reduces the corresponding Gleason pattern predictions by 49% to 98%, with effects larger than those from 20 activity-matched random feature removals. These results show that a shared sparse representation can support competitive Gleason grading while providing named morphological evidence that transfers across datasets and whose contribution to grading can be measured directly.
[CV-211] Joint Image Denoising and Interpolation via Mixed Graph Filtering
链接: https://arxiv.org/abs/2611.24242
作者: Gene Cheung,Niruhan Viswarupan,Romisaa Elsaidy
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Signal Processing (eess.SP)
备注:
Abstract:Images often require both denoising and interpolation, yet treating the two tasks separately is suboptimal: interpolation introduces correlated noise that complicates subsequent denoising, while denoising may remove image details useful for interpolation. In this paper, we study joint image denoising and interpolation from a mixed graph filtering perspective, where denoising is modeled using undirected graph filters and interpolation using directed graph filters. We first establish that, under mild conditions, any (pseudo-)linear denoiser and interpolator can be interpreted as solution filters of maximum a posteriori (MAP) estimation problems regularized by undirected and directed graph smoothness priors, respectively. Guided by these results, we formulate graph-based joint denoising and interpolation problems using both directed and undirected graph structures. We characterize when joint denoising/interpolation reduces to a separable pipeline and when a genuinely coupled optimization is required. For the non-separable case, we recover sparse graph operators from given denoisers and interpolators, yielding an efficient linear-system formulation solvable via conjugate gradient. Experimental results demonstrate that our proposed joint framework consistently outperforms separate denoising and interpolation pipelines across multiple denoiser-interpolator combinations.
[CV-212] Gaussian Splatoms: Direct Atomic Structure Identification in Electron Tomography
链接: https://arxiv.org/abs/2611.23444
作者: Nalini Singh,Tiffany Chien,Cedric Lim,Arthur R. C. McCray,Colin Ophus,Laura Waller
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注: 26 pages, 14 figures
Abstract:Atomic electron tomography (AET) computationally reconstructs the 3D atomic structure of materials from a series of 2D projection measurements acquired at different tilt angles. Inspired by Gaussian splatting techniques from computer vision, we introduce an atom-centric parameterization for modeling AET samples. Instead of a conventional volumetric representation with a grid of voxels, we model the sample as a list of atoms, where each atom is a 3D Gaussian characterized by learned location, scale, and amplitude. The Gaussian model is a natural fit for atoms that allows us to easily incorporate prior information grounded in physics; we impose isotropy of atoms, a minimum interatomic distance, and realistic atomic radii. We show in simulation that our method is more robust than volumetric baselines to real-world imaging challenges, including a limited number of measurements and low electron dose. We further show that our method yields a more physically plausible reconstruction of an experimental 23,000 atom iron-platinum nanoparticle than a previously published volumetric reconstruction. Beyond tomography, our representation is a promising tool for any atomic resolution imaging modality (e.g. multislice electron ptychography).
[CV-213] LIC-Audit: How Comparable Are Reported Gains in Learned Image Compression?
链接: https://arxiv.org/abs/2611.23419
作者: Haisheng Fu,Jie Liang,Runyu Yang,Feng Ding,Xin Dong,Zixiao Jin,Jingning Han
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Applications (stat.AP)
备注: 31 Pages, 4 figures
Abstract:Learned image compression (LIC) papers often report BD-rate improvements of only a few percent. They are measured against VTM reference software whose version keeps changing: the claims we audit name 7 versions, 5 in 2025 papers alone. But how comparable are these numbers across papers, and what evidence should future papers give for such small gains? We introduce LIC-Audit, an analysis of 353 released rate-distortion curves from 36 repositories, and a controlled re-evaluation of eight public codec releases with 28,193 encode-decode records. We identify three problems. First, different VTM reference runs change BD-rates by up to 3.56 percentage points, and on CLIC Pro published numbers order 10 of 55 method pairs differently from a common computation; comparing codecs directly over a fixed quality range removes this dependence. Second, releases use different and mostly undocumented conventions for counting bits, computing PSNR, and padding images. Under consistent measurement, the GLIC method’s 1.00% lead over the HPCM method on Tecnick becomes a 1.43% deficit, changing only the padding reverses the comparison of HPCM and CMIC, and most gaps narrow when MS-SSIM replaces PSNR. Third, average curves hide how much results vary from image to image, so whether a small gain is statistically significant depends on the assumed variation. An optimistic assumption calls 12.0% of comparisons between equally good codecs significant on test sets as small as Kodak, instead of 5%; a cautious one leaves 18 of 45 differences between recent codecs undecided, yet per-image results show 12 of them to be significant. Beyond auditing past results, LIC-Audit provides a versioned evaluation protocol, a public leaderboard website with a scoring tool, source-linked benchmark data, per-image measurements, and reporting guidelines, to help future LIC papers make comparable and verifiable claims of progress.
[CV-214] Streaming Generative Video Compression with Embedding Alignment and In-Context Propagation
链接: https://arxiv.org/abs/2611.23352
作者: Zhitao Wang,Hengyu Man,Han Liu,Wenrui Li,Debin Zhao,Xiaopeng Fan
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
备注: 15 pages, 18 figures, 5 tables, including supplementary material
Abstract:Video compression at ultra-low bitrates presents substantial challenges in preserving perceptual quality. Although pretrained video diffusion transformers (DiTs) provide powerful generative priors, existing generative video compression (GVC) methods built on them typically rely on computationally expensive VAE decoders to reconstruct video frames. Moreover, jointly refining multiple frames to exploit the inherent spatiotemporal modeling capabilities of video DiTs introduces additional buffering delay. To address these challenges, we propose SGVC, a streaming generative video compression framework that adapts a pretrained latent video DiT into a causal pixel-space refiner. Specifically, SGVC projects decoded features from a contextual pixel codec directly into the DiT input embedding space. To bridge the representation gap between codec-derived embeddings and pretrained DiT embeddings, we introduce Embedding Alignment, which aligns the projected embeddings with input embeddings extracted from the original frames during training. The refined DiT hidden states are then mapped directly to the pixel domain through a lightweight reconstruction path, bypassing the pretrained VAE decoder. To further exploit the spatiotemporal priors of the video DiT while maintaining causal decoding, we introduce In-Context Propagation, which reuses layer-wise K/V states across frames to exploit historical generative context, improving temporal consistency without requiring clip-wise buffering. Extensive experiments demonstrate that SGVC achieves competitive rate-perception performance, with average BD-rate savings of 71.97% over S ^2 VC in terms of DISTS. Project Page: this https URL
人工智能
[AI-0] Mapping the AI Supply Chain: Dependencies Vulnerabilities and Market Concentration in the AI Ecosystem
链接: https://arxiv.org/abs/2611.24927
作者: Sarah H. Cen,Lindsey A. Gailmard,Joohan Kim,Rishi Bommasani,Daniel E. Ho,Percy Liang
类目: Computers and Society (cs.CY); Artificial Intelligence (cs.AI)
备注: 80 pages, 31 figures, 16 tables
Abstract:AI systems are built across many organizations that contribute models, data, compute, capital, talent, and other assets. While the layers of the AI stack are relatively well understood, the relationships between actors and components across layers remains sparsely documented. In this work, we focus on the organizations that make up the AI supply chain: the complex network of actors that contribute to the development and deployment of AI. We motivate the examination of the organizational AI supply chain—highlighting concerns about concentration, power, chokepoints, cascading failures, fragmented accountability, and risk—then describe our pipeline for mapping the AI supply chain. Our pipeline yields our main contribution: a multilayered, large-scale dataset of actors and relationships in the AI ecosystem, annotated with dates, descriptions, types, tags, and more. This work takes a step toward systematically mapping and measuring the AI supply chain and its properties, cataloging how models, data, chips, infrastructure, capital, and even talent flow between organizations. This data enable critical examinations of industry relationships and yield insights into the political economy of AI development and concentration. We provide several illustrative analyses, including studying the most central actors, analyzing network concentration, and identifying communities and chokepoints. While our dataset is fairly comprehensive, it is also necessarily incomplete: public documents omit private deals, and coverage is shaped by search design. Together, the approach, data, and findings aim to serve as a resource for studying the AI ecosystem. The dataset is publicly available at this https URL and can be explored interactively at this https URL.
[AI-1] Agent ic Harness Engineering Beta: Requirement-Centered Evolution of Agent Harnesses
链接: https://arxiv.org/abs/2611.24924
作者: Jiahang Lin,Jiahao Chen,Yiding Wang,Shichun Liu,Chengjun Pan,Shihan Dou,Honglin Guo,Yuyang Hu,Changhao Jiang,Zhenhua Han,Hang Yan,Qi Zhang,Tao Gui,Xuanjing Huang
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:Real-world harness engineering is driven by explicit and implicit user requirements, which guide development and validation. However, existing harness evolution methods often overlook implicit requirements and interaction needs, while using a single evolution agent to modify the harness and benchmark scores to select changes. This process often produces narrow patches and leads to benchmark-specific local optima. We introduce Agentic Harness Engineering Beta (AHE-Beta), a requirement-centered loop that brings real-world engineering practices into harness evolution. The loop consists of three steps: (1) User-driven evaluation lets simulated users raise their requirements as issues while they work with the harness over multiple turns. (2) Team-based modification implements those requirements as generalizable changes rather than task-specific patches. (3) Requirement-driven selection merges a change only when it resolves a verified requirement and passes review. Empirically, on held-out tasks, AHE-Beta raises pass@1 from 21.9% to 33.3% while using 40.8% fewer tokens than Base Harness, and achieves higher pass@1 than Claude Code, Meta-Harness, and AHE. In a blind human A/B test, real users prefer AHE-Beta in 31 of 48 valid pairs and rate it higher on all five dimensions, matching the directional preference of the user simulator. The human A/B results show that AHE-Beta achieves its goal of aligning harness evolution with real-world user requirements. Without further evolution, the gains transfer to GLM-5.3 and DeepSeek-V4.1-Flash with pass@1 improvements of 15.8% and 5.3% over Base Harness. Together, these results show that requirement-centered harness evolution is a more general method for improving agent capabilities.
[AI-2] Local AI Agents in Power Electronics Simulation: Deployment Measurements and Costs
链接: https://arxiv.org/abs/2611.24880
作者: Yuzhuo Li,Pasan Gunawardena,Yunwei Li
类目: Artificial Intelligence (cs.AI); Systems and Control (eess.SY)
备注: 27 pages, 23 figures, 12 tables
Abstract:Agentic artificial intelligence (AI) is emerging in power electronics, where it drives the simulation loop that engineers used to close largely by hand. In this loop, a converter design, or a study of the network it connects to, is built, run, measured and revised until it satisfies its specification. Such an agent can be served from a cloud API or from an inference device on the premises. For organizations whose designs, customer data or export-controlled material cannot leave the site, and whose simulators are licensed there, local deployment is the natural choice. In such a deployment, the agent becomes an electrical load on the organization’s own equipment, i.e., every run consumes power and energy on a local device, generates tokens and takes time. However, existing work focuses mainly on what the agent can achieve with cloud-hosted models. What a local deployment costs in power, energy, tokens and time on real simulation tasks, and what its choices trade against one another, has received little quantitative investigation under comparable conditions. To address this gap, a systematic evaluation framework is presented in this paper. It applies locally served models to agentic power electronics tasks, from the converter level to the system level, and measures their power, energy, token and time costs. Based on these measurements, the capability boundary of a local agent on such tasks and the cost per solved task are characterized. The results are expected to provide practical insight for the use of agentic AI in power electronics and the power systems it serves.
[AI-3] Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws
链接: https://arxiv.org/abs/2611.24877
作者: Zhiyu Liu,Xinyu Yang
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: 15 pages, 2 figures, 6 tables. Models, prompts, replies with tool calls, and code: this https URL
Abstract:Large language models (LLMs) are used to explain clinical prediction models, on the promise that they can say what a model computes, find its faults and even reveal new knowledge in it. We tested the three promises where the answers could be checked. From the International Stroke Trial we trained four kinds of model (logistic regression, explainable boosting machine, gradient-boosted trees, neural network) for two outcomes on patients randomised earlier, planted one flaw in each of 24 copies (an under-predicted subgroup, a reversed risk factor or an implausible step) and mixed them with the 8 clean models. GPT-6 Astra audited each model three times, blind, through a tool that only returned the model’s predictions; the judging rules were committed before any run and applied by program. Its surrogate models, with a median of 128 terms, reproduced the audited models’ log-odds with a mean R^2 of 0.976 (0.959 for an additive surrogate of 42 terms fitted by an analyst). It found the planted flaw in 69 of 72 runs (96%; 95% CI 80% to 99%); a script written for the purpose found all 24. It also reported many more problems: 60% of 1,089 checkable reports failed the pre-specified check against the model’s average behaviour; post hoc, most of these were real but small steps, and 15% of the checkable reports were not found in the model. Of 601 unexpected relationships it read out of the clean models, 11% held in patients randomised later (random pairs 6%; a regression scan’s shorter list of its strongest candidates, 28%); after a check on the training data, 33% against 30%. With query access alone, an LLM can describe a black-box model faithfully and find nearly every planted flaw, as a purpose-written script also did; its reports of problems must be checked on the model and its discoveries on data.
[AI-4] WorldRuler: Interactive World Modeling under Configurable Rules
链接: https://arxiv.org/abs/2611.24869
作者: Mingliang Zhai,Chuanhao Li,Jiaming Tan,Zhen Li,Yuwei Wu,Yunde Jia,Zian Meng,Yuanyang Yin,Kaipeng Zhang
类目: Artificial Intelligence (cs.AI)
备注: 15 pages, 6 figures
Abstract:Rules govern how player actions change interactive worlds. Different rules create distinct gameplay experiences within similar scenes and action spaces. For customizable gameplay, generative world models must follow user-defined rules beyond real-world behaviors. However, realizing user-defined rules presents two challenges: Rule-State Persistence and Rule-to-Visual Realization. The former requires preserving rule definitions, dependencies, and world states across saving and loading, so that restored worlds follow the established rules. The latter requires accurately realizing rule-induced changes on designated objects and attributes, particularly when affected objects are unseen. In this paper, we propose WorldRuler, which organizes rule definitions, dependencies, and object states in an executable 3D rule graph and connects rule execution, video generation, and perceptual feedback in a closed loop. The graph supports joint saving and restoration of rules and world states, while dynamic object binding grounds rule-specified entities in visual instances and uses stored states to constrain generation from their first appearance. We further introduce object attribute state replacement and counterfactual reconstruction training, enabling accurate attribute control and compatible appearance and motion changes while limiting interference with unrelated content. We develop an automated data collection and annotation pipeline using GTA-V and Habitat to construct RuleVerse, a dataset of rule-driven interactions with detailed state and event annotations, including cases where rules take effect before affected objects become visible. Extensive experiments demonstrate the effectiveness of WorldRuler in preserving rule-state consistency across saving and loading and accurately realizing rule outcomes on designated objects and attributes.
[AI-5] Choosing What to Observe: Budget-Paired Counterfactual Audits for Credible Evaluation of Visual Agents
链接: https://arxiv.org/abs/2611.24867
作者: Ruiqi Liu,Jinming Li,Mo Wang,Yang Lu,Yanlan Kang
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Visual agents that read dense documents must ration a finite observation budget across candidate regions, yet answer accuracy conflates where an agent looks, how it adapts to the reads that remain, and how it an?swers once observation stops. We introduce budget-paired counterfactual audits, an evaluation protocol that holds public evidence and region bindings fixed while perturbing only the remaining budget or de?clared task parameters, and credits every acquired history with exact continuation values. The resulting accounting decomposes lost expected accuracy into stepwise acquisition loss and terminal decision loss, separating planning from fixed-role heuristics. Across forty paired routing anchors, five open-weight
[AI-6] LAaDT-DSS: decision support for non-experts on rooftop PV incentive and grid simulations under Turkish tariff rules
链接: https://arxiv.org/abs/2611.24810
作者: Baran Kaynak
类目: Computational Engineering, Finance, and Science (cs.CE); Artificial Intelligence (cs.AI); Systems and Control (eess.SY)
备注: 51 pages, 10 figures. Code, scripts and results: this https URL
Abstract:Rooftop photovoltaics raise three questions: should a household invest, which incentive costs the state least, and does the feeder stay within limits? Those who ask are rarely modellers. We present LAaDT-DSS, decision-support software that answers all three under Turkish tariff rules from one simulation core. Non-experts reach it by clicking a house on a 3D map or by asking in plain words; a language model is asked to call the simulation tools, not to compute. Each entry of its 125-entry parameter registry is marked as sourced or estimated. Case studies test the system. Its PV yield lies within \pm 3.6% of two independent tools at equal losses. Across 2,048 draws of its estimated inputs, the answer for PV alone held in all but one; with a battery, it changed in 31-41% of the draws. Four scripted chat scenarios ran end to end; no user study was run.
[AI-7] Bias-Only Training Dynamics in ReLU Networks
链接: https://arxiv.org/abs/2611.24797
作者: Shreyas Chaudhari,Jose M. F. Moura
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Neural networks with randomly initialized, fixed weights and trainable biases are universal approximators, yet the dynamics of bias-only training remain largely unexplored. We study these dynamics in one-hidden-layer ReLU networks through activation codes, the binary patterns of active neurons that partition the input space into convex polyhedral regions. We show that the gradients with respect to the biases depend on each input only through its activation code, and hence the training dynamics can be characterized with these codes. In particular, as long as the activation codes remain fixed, training reduces to a linear least-squares problem with the codes as features. Training can then be understood as a sequence of least-squares problems, with transitions occurring when neurons change their activation states. We show that the fraction of switching neurons vanishes as the network widens, and the training dynamics approach those of linear least squares on the initial activation codes in the infinite-width limit. Our experiments support these theoretical findings, showing that while activation switches can noticeably affect training in narrower networks, their relative frequency and the associated changes in the dynamics diminish as the width increases.
[AI-8] Isabelle Pool Server: a pooled verification service for prover training and multi-agent proving
链接: https://arxiv.org/abs/2611.24788
作者: Xuanwei Ren,Wenda Li
类目: Logic in Computer Science (cs.LO); Artificial Intelligence (cs.AI)
备注: 22 + 8 pages, 6 + 0 figures
Abstract:Isabelle Pool Server (IPS), a verification service with a pool of warm headless Isabelle sessions behind an HTTP API, with session leasing and reclamation, memory-aware admission control, and a time budget on every call. Its verification interface spans single-step execution, block-level proof checking and whole-theory builds, with checkpoints for cheap branching and backtracking, and a build from scratch as the final arbiter. The service runs as a single container with monitoring and is accessible via a Python client, two MCP servers and a raw HTTP API. In our measurements IPS is level with PIDE MCP, a PIDE frontend for agents, at one worker and ahead of it at eight, at its commit of October 2026 (the tool is now PIDE AI). Once its sessions are up it scales to 32 concurrent workers on one host at 24 times the single-worker rate, and under memory pressure it refuses new sessions rather than losing existing ones. IPS is free software and publicly available at this https URL
[AI-9] Learning Trajectories Leave Decodable Traces in Neural Network Weight Space and Internal Representations
链接: https://arxiv.org/abs/2611.24782
作者: Patrick Krauss,Achim Schilling,Andreas Maier,Thomas Kinfe,Claus Metzner
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neurons and Cognition (q-bio.NC)
备注:
Abstract:Neural networks trained on the same task develop related weight-space structures, yet remain distinguishable after learning. Do these differences merely reflect alternative solutions, or do they preserve information about individual learning histories? We tested whether final weights reveal initialization, task order, and earlier trajectory states. Training reorganized the ensemble: initial pairwise distances did not consistently predict final distances. Nevertheless, all 30 trained networks were correctly matched to their initial weights among ten candidates per dataset. Networks sharing an initialization also developed recognizable final hidden representations across different batch schedules, although direct comparisons with initial activity did not recover identity. Learning order remained readable as well. A linear probe decoded the order of two earlier tasks with 100% accuracy on 80 held-out networks after ten epochs of a common final task. Earlier intermediate weight states were partly predictable, but individual initial weights could not be reliably reconstructed. Final weights thus retain selective historical information rather than a complete trajectory record. Perturbed networks recovered classification performance without returning to their unperturbed training paths. This separation persisted across datasets, intervention times, and network layers. Moreover, 2,157 of 2,160 aligned endpoint pairs showed no positive loss barrier at sampled interpolation points, indicating low-loss connections between distinct solutions rather than convergence to one weight vector. Together, these findings show that a shared task constrains networks to related successful solutions without erasing their individual histories. Final weights reflect not only what a network has learned, but also where learning began and how it unfolded.
[AI-10] LLM -Guided Cyber-Physical Attack Path Identification in Smart Grids
链接: https://arxiv.org/abs/2611.24765
作者: Amr Elshazly,Jesse Roberts,Rachad Atat,Muhammad Ismail
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注: 7 pages, 4 figures, 1 table. Accepted at IEEE IECON 2026
Abstract:Identifying high-impact attack paths in cyber-physical power systems is critical for improving grid resilience. Existing methods typically rely on either structural topology analysis or simulation-based cascading-failure models. Topology-based approaches capture connectivity but lack adversarial reasoning, while simulation-driven methods are primarily suited for offline analysis and are less effective for rapid evaluation of evolving attack scenarios. As a result, identifying realistic attack paths that jointly capture topology, threat semantics, and physical impact remains challenging. This paper presents a physics-grounded framework for identifying cyber-physical attack paths using large language models (LLMs), with cascading-failure-derived impact rankings used as physics-grounded references. The framework integrates system topology, LLM-guided STRIDE and MITRE ATTCK threat modeling, and three baseline strategies for comparison: topology-driven, threat-driven, and fusion-based. Evaluation is performed using (i) critical-node overlap between LLM-inferred top-K rankings and cascading-failure-based ground truth; (ii) node-level similarity between LLM-inferred and reference paths; and (iii) path-level alignment capturing agreement in complete attack sequences. Results show that structured LLM-based threat modeling substantially improves alignment with physics-grounded references. In the cyber layer, critical-node overlap@3 increases from approximately 33% to as high as 100%, while node-level similarity reaches up to 65% and path-level alignment reaches up to 48%. In addition, the proposed framework reduces runtime by approximately 75–85% compared to the fusion-based baseline in the IEEE 14-bus system. These results suggest that LLM-guided attack path identification is a promising approach for efficient and impact-aware cyber-physical security analysis.
[AI-11] Horizontal Flow Matching for Single-Cell Spatiotemporal Dynamics
链接: https://arxiv.org/abs/2611.24764
作者: Zihan Wang,Jianzhe Li,Zhenyi Zhang,Peijie Zhou
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM)
备注: 38 pages, 7 figures
Abstract:Learning the spatiotemporal dynamics of cells from spatial transcriptomics requires connecting snapshots of different cells acquired in different coordinate frames. A rotation or translation of a section can appear as cell movement and distort the inferred correspondence. We introduce horizontal flow matching, which learns changes in tissue organization, molecular state, and cell abundance up to shared rigid motion. The dynamics follow a continuity–reaction equation whose action assigns no cost to shared rigid motion, and the resulting horizontal projector removes this component from spatial prediction errors during learning and from predicted velocities during generation. An equivalent endpoint problem jointly estimates correspondence and relative pose in Wasserstein–Fisher–Rao geometry and yields analytic paths whose velocity and growth serve as regression targets. Regressing these targets recovers the horizontal dynamics under ideal population conditions. Our model, HORIZON, uses an equivariant transformer to predict velocity and growth from the current tissue, so that a tissue can be generated continuously from a single initial snapshot. On a simulated tissue observed in perturbed coordinate frames, alignment improves reconstruction and projection reduces rigid drift. On four spatial transcriptomics time courses, HORIZON reconstructs tissue geometry, gene expression, and cell abundance more accurately than existing spatial and unbalanced trajectory methods, and interpolates held-out developmental stages.
[AI-12] Elysium: Simulation-Free Unbalanced Schrödinger Bridges
链接: https://arxiv.org/abs/2611.24763
作者: Zihan Wang,Jianzhe Li,Zhenyi Zhang,Peijie Zhou
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM)
备注: 44 pages, 9 figures
Abstract:Reconstructing cellular dynamics from snapshots is challenging because the destructive nature of single-cell measurements makes tracking the same cells over time impossible. Cellular dynamics are inherently stochastic and involve changes in population size through proliferation and apoptosis. Growth-based methods represent population change as a rate on the density, which does not assign exits and entries to individual cells. Unbalanced Schrödinger bridges model individual events, but neural solvers for killed diffusion bridges simulate trajectories during training. We introduce Elysium, a simulation-free solver that jointly learns state transition and mass change. Using a killed Brownian reference and its time reversal, Elysium constructs direct regression objectives from an optimal endpoint coupling and conditional bridge distributions, without simulating trajectories during training. We show that the minimizers of the expected training objectives recover the optimal unbalanced Schrödinger bridge. Empirically, Elysium reconstructs state transitions, population size, and event distributions on simulated benchmarks with known killed-diffusion ground truth, outperforming existing deterministic and bridge-based solvers. On single-cell time courses, it interpolates cell state distributions at unobserved time points, tracks population expansion and contraction, and captures biphasic drug-response dynamics.
[AI-13] How Generative Models Generate: A Concise Guide to the Generative AI Landscape
链接: https://arxiv.org/abs/2611.24728
作者: Tianhua Chen
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:The rapidly expanding generative AI landscape brings together methods that differ in architecture, learning objective, mathematical representation and sampling procedure. A concise, selectively organised technical guide can provide an accessible starting point for developing a coherent understanding of this increasingly complex landscape. This article takes a generation-centred approach, connecting how models produce new observations with the modelling and learning principles that make generation possible. We develop a two-level perspective: first, distinguishing three recurring generation routes-direct transformation or decoding, sequential conditional construction, and iterative updating or transport; and second, examining the modelling and learning principles underlying these routes. Representative approaches are introduced through their key components, learning objectives and sampling procedures. We then examine their broader connections, including how different architectures and generation mechanisms can overlap or be combined. By bringing together essential technical explanations within a compact conceptual framework, the article aims to help readers develop a working understanding of unfamiliar approaches, recognise connections among models they have encountered, and navigate different approaches within the broader generative AI landscape.
[AI-14] F3: What Makes Good Experience in World-Action Models?
链接: https://arxiv.org/abs/2611.24708
作者: Zihe Huang,Sirui Zhou,Haozhi Han,Ruge Zhang,Teng Ma,Tao Ma,Zheng Liu,Ce Zhang,Ting Cao,Yunxin Liu,Yunquan Zhang,Yifeng Chen,Kun Li
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Deployed world-action models (WAMs) can keep improving only if they make good use of their own experience. Existing methods supply retrieved rollouts or demonstrations to the policy as context, yet such experience often turns previously solved tasks into failures. What, then, makes good experience for a WAM? We show that experience is valuable only in use: its value depends not on whether the source rollout succeeded but on what is stored, when it is used, and where it enters the policy. We therefore propose F ^3 , which makes these three decisions with Form, Fit, and Fusion. Form compresses successful and failed rollouts alike into short, verified operation records; Fit teaches the policy to read a record and to predict when it helps; Fusion injects the record’s images, text and actions into the policy as a prefix and an action prior. Trained once on only 200 examples and deployed with a growing experience library, the model achieves state-of-the-art results on MolmoSpaces and RoboLab without any training on RoboLab: over native Cosmos 3 Nano, it raises success by 12.0 pp and 10.8 pp, reduces steps to success by 12.2% and 11.3%, and cuts regressions by 57%.
[AI-15] Depth Before Breadth: Reverse-to-Forward Curriculum with Advantage Decomposition for LLM Agent
链接: https://arxiv.org/abs/2611.24697
作者: Siyuan Du,Yuxin Liu,Meng Tang,Haoran Li,Yechao Bai,Feiyang Pan,Jia He
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Reinforcement learning (RL) has become a common approach for training large language model (LLM) agents on long-horizon tasks. However, learning from sparse terminal rewards makes credit assignment and exploration difficult. We approach these challenges through a policy-induced capability region, the set of states from which the current policy can reliably complete a task. Near its boundary, an agent may need only a few actions to reach a more solvable state, even when the goal remains distant. This reduces long-horizon learning to two coupled subproblems of reaching that state and completing the task from there. We address them with paired advantages and a progressive reverse-to-forward curriculum. The advantages provide separate signals for reaching more solvable states and completing the task from them, while the curriculum expands the capability region backward along successful reference trajectories and then into off-reference states through on-policy exploration. On ALFWorld and WebShop, our method outperforms competitive agentic RL methods at both model scales, achieving success rates of 90.1%/96.1% and 80.2%/82.6% with Qwen2.5-1.5B/7B-Instruct, respectively.
[AI-16] Movie2Game-Bench: Holistic Evaluation of Film-to-Game Adaptation Beyond Playability
链接: https://arxiv.org/abs/2611.24669
作者: Jialing Xie,Yifan Zhou,Yiwen Zhang,Junjun He
类目: Artificial Intelligence (cs.AI); Multimedia (cs.MM)
备注:
Abstract:Adapting a film into a game requires selecting source elements and translating them into player actions, rules, and feedback. We introduce Movie2Game-Bench (M2G-bench), a benchmark for evaluating film-to-game adaptation beyond playability. M2G-bench combines general evaluation, movie-specific evaluation, and playtesting by three GUI model players. The protocol connects source content to gameplay steps and implementation evidence, capturing both content preservation and interactive realization. The benchmark comprises 22 films and 658 source-specific reference items. Evaluating 14 models on the same films with their selected primary harnesses reveals complementary strengths in game quality, adaptation, and player experience. A shared weakness concerns playable sequences: this category has the lowest content-presence mean in all 14 models, averaging 37.9 out of 100 compared with 54.8 for iconic elements. Among judgments of retained emotionaldevelopment content, narrative choices account for 60.0% and scenario operation for 7.2%. These findings identify two linked priorities for film-to-game adaptation: preserving source situations that can support gameplay and connecting retained content to player actions and consequences.
[AI-17] Choosing the Right Representations: Task-Aware Fusion for Multimodal Materials Language Models NEURIPS2026
链接: https://arxiv.org/abs/2611.24641
作者: Huu Binh Ta,Alex Morehead,N. Benjamin Erichson,Ferdinando Fioretto,Sandeep Madireddy
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: Accepted at the AI4Mat Workshop at NeurIPS 2026
Abstract:Multimodal language models for materials science rely on effective mechanisms for incorporating three-dimensional structural representations. MatterChat addresses this challenge using a large bridge module between a pretrained structural encoder and the LLM, together with a separate contrastive alignment stage. We instead ask whether this bridge can adaptively select and combine representations from multiple structural encoders according to the target property. We find that supervised and generative material encoders provide complementary information for property prediction. We found that, for Zatom-1, this information is not concentrated in the final layer. Instead, the most informative representations occur at different depths depending on the target property. This motivates Task-Conditioned Attention Fusion (TCAF), a lightweight attention-based bridge that uses task-conditioned latent queries to combine CHGNet features with intermediate Zatom-1 representations. The bridge is trained directly on downstream prediction tasks, allowing representation selection and material-language alignment to be learned jointly without separate contrastive pretraining. Across classification and regression tasks, TCAF outperforms MatterChat, with up to 47.8% lower RMSE, while using substantially fewer trainable parameters and halving the overall training time. Further experiments incorporating UMA-S demonstrate TCAF’s flexibility across encoder combinations, with the three-encoder configuration achieving the highest macro-average classification accuracy of 89.61%.
[AI-18] GateWatch: Low-Overhead LLM Monitoring with Boolean Circuits
链接: https://arxiv.org/abs/2611.24619
作者: Firas Spanioli,Yaniv David
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: 23 pages
Abstract:As large language models (LLMs) and LLM-based agents are increasingly deployed, monitoring their behavior becomes important for detecting unsafe or unintended actions. Activation probes offer an efficient approach, but existing methods face trade-offs between accuracy, interpretability, and computational overhead. Probes on raw activations lack interpretable inputs, whereas sparse autoencoder (SAE) probes use interpretable features but often rely on additive classifiers and large SAE encoders, limiting interpretability and increasing memory overhead. We introduce GateWatch, a compact, interpretable classifier that expresses its entire decision rule as an explicit Boolean logic over a small number of binarized SAE features. We explore three approaches to constructing these circuits under a specified gate budget. For efficient deployment, GateWatchServe computes only selected SAE features, enables streaming of their binary values across layers during inference, and evaluates multiple GateWatch classifiers in parallel using compact lookup tables. Across 24 tasks on Gemma-2-9B, GateWatch achieves 80.49% mean test accuracy using only 3.8 SAE features and 3.5 logic gates per probe on average, just 1.17 percentage points below full 16K-feature SAE probes, while matching or outperforming them on 10 tasks and reducing probe memory by 99.98%. Inside vLLM, 1,000 concurrent multi-layer, multi-token GateWatch classifiers retain 97-98% of baseline throughput and 99.8% of KV-cache capacity.
[AI-19] hink Offline Act in Real Time: Self-Improving Visual Game Agents through Code
链接: https://arxiv.org/abs/2611.24590
作者: Zipeng Qiu,Shusheng Xu,Jiale Li,Yukun Zhou,Chenyue Li,Wenshuang Li,Tongkai Yang,Binhang Yuan
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Pre-trained models can read screens and reason about games, yet on an unfamiliar game, an agent still has to learn to play. We ask how an agent can self-improve on such a game from a few of its own attempts, in real-time visual games where the agent sees only the screen and the game keeps running while the agent thinks. We study two kinds of policy that can carry this improvement. In model as policy, a model selects actions during play; we adapt real-time chunking so that committed actions keep executing while the model reasons, and train a 2B controller for this interface. In code as policy, a model writes, tests, and revises a program between attempts, and the program acts during play. Across 170 GameWorld tasks, with at most eight one-minute executions per run, code as policy reaches 47.80% mean progress, against 30.90% for our trained controller, and leads in all five game categories. Improving the controller’s weights on a single game, by contrast, yields only modest gains. These results favor letting a model think offline and storing what it learns in a program that acts in real time. Project repository: this https URL
[AI-20] AGAMOTTO: An AI-Enabled Framework for Time-Critical Tactical Course-of-Action Evaluation and Generation
链接: https://arxiv.org/abs/2611.24568
作者: Matthew L. Corbett
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Current command and control (C2) platforms lack integrated simulation capabilities that would allow commanders to dynamically explore courses of action (COAs) during live operations. This paper presents ongoing research and contributions to the development of AGAMOTTO (Advanced Guidance and Analytics for Military Operations, Tactical Threats, and Outcomes) as part of the overarching NGC2 program, which advances the Army’s efforts to modernize C2 systems, such as the Tactical Assault Kit (TAK). The paper outlines the current design, preliminary findings, and the remaining research tasks, with an emphasis on how the completed work will support tactical decision-making in future time-constrained environments. While the full system is still under development, initial development and results demonstrate the feasibility of AGAMOTTO. Early experiments highlight both the potential benefits and practical constraints of deploying AI-enabled tools in operational settings, and introduce lessons learned and a viable training strategy to make AI-generated COAs feasible in tactical scenarios.
[AI-21] NaviGATE: Noise-Aware Dual-View Generation for Multimodal Knowledge Graph Reasoning
链接: https://arxiv.org/abs/2611.24552
作者: Jie Peng,Yongxue Shan,Yongfu Zha,Cui Miao,Xiaodong Wang
类目: Artificial Intelligence (cs.AI)
备注: 9 pages, 4 figures
Abstract:Multimodal knowledge graph reasoning (MKGR) aims to infer missing facts by jointly exploiting relational structures, textual descriptions, and visual evidence. Existing embedding-based methods effectively capture graph topology, but their fixed multimodal representations are less effective for query-specific fine-grained semantic reasoning, while generation-based methods are vulnerable to noisy retrieval and may produce invalid entities. To address these challenges, we propose NaviGATE, a noise-aware generative framework for MKGR. NaviGATE first aligns graph-derived structural representations with textual semantics, then performs structure-aware internal knowledge activation by fine-tuning the multimodal language model with multimodal entity summaries and injected structural tokens. In the noise-aware dual-view instruction tuning stage, NaviGATE jointly optimizes retrieval-free and retrieval-aware views: reliable retrieved candidates are reinforced through contrastive logit sharpening, while misleading candidates are suppressed through parametric consistency distillation. In inference, structure-constrained decoding restricts generation to valid entities in the target knowledge graph. Experiments on standard benchmarks show that NaviGATE consistently outperforms strong unimodal, multimodal, and generation-based baselines, and remains robust under noisy retrieval scenarios.
[AI-22] ABSClaw: From Invented Worlds to Reality-Grounded Social Simulation
链接: https://arxiv.org/abs/2611.24517
作者: Shaobin He,Yan Gao,Quan Zheng,Zhenyang Lin,Zhen Liu,Jiyan He,Shuxin Zheng
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:LLM-based artificial societies exhibit rich behavior but often lack scientific grounding: personas and environments are assumed, interaction networks are synthetic, and simulators are generated as free-form code, making results difficult to tie to real populations and sensitive to the backbone model. We argue that simulated worlds should be sampled from observed data, while simulation logic should be implemented by composing operations from a shared, task-independent vocabulary. In this paper, we introduce ABSClaw, a zero-code agent that compiles natural-language requests into executable, reality-grounded social simulations. ABSClaw grounds who acts, where they act, and with whom they interact in survey and census data, geospatial sources, and real networks. It expresses state transitions through 26 atomic operations. To support an open and extensible ecosystem, ABSClaw further allows capabilities to be added as lifecycle-scoped skills without modifying its core. Furthermore, we propose ABSBench, a benchmark of 193 real-world social-simulation tasks. On ABSBench, ABSClaw achieves the best overall performance among the evaluated social simulators. Three real-world case studies show that grounding personas, environments, and interaction networks each improves agreement with observed outcomes. With one runtime skill, ABSClaw executes 13 population-scale simulations, collectively involving 25 million agents. Together, these results demonstrate ABSClaw’s real-world fidelity, extensibility, and scalability.
[AI-23] he Largest Tractable Subclass of Multi-Scale Allen Temporal Reasoning
链接: https://arxiv.org/abs/2611.24498
作者: Quentin Cohen-Solal
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:In this paper, we propose a novel model for temporal reasoning over multi-scale intervals. We show that this model yields a complete reasoning framework and exhibits other particularly interesting properties. Furthermore, we identify the maximal tractable subalgebra for this model, as well as for the other complete model, in the qualitative scales context, which exhibits weaker properties but whose results can be derived by transfer from our new model.
[AI-24] Harvesting Expert-Parallel Bubbles for Fine-Tuning
链接: https://arxiv.org/abs/2611.24496
作者: Oana Balmau,Saurav Banka,Jiaxuan Chen,Anne-Marie Kermarrec,Ramya Narasimha Prabhu,Rafael Pires,Mathis Randl,Martijn de Vos
类目: Operating Systems (cs.OS); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)
备注: 12 pages, 8 figures
Abstract:GPUs are expensive, yet latency-critical large language model (LLM) inference leaves part of them idle. Colocation reclaims this capacity by running best-effort work, such as fine-tuning, alongside serving. Many leading LLMs now use a Mixture-of-Experts (MoE) architecture, which adds a new source of idle time. Under expert parallelism, MoE experts are spread across GPUs, and because the router sends uneven numbers of tokens to each GPU, lightly loaded GPUs wait for the most loaded one. Even after applying load balancing solutions, such bubbles still take 8–13% of prefill time. We present BubbleTea, the first colocation system that leverages expert-parallel bubbles for fine-tuning. Unlike existing colocation systems, which make one scheduling decision for all GPUs per forward pass, BubbleTea predicts each GPU’s bubble at every layer from the router’s token assignment, and fills it with short chunks of fine-tuning work that reuse the weights already loaded for serving. We also introduce compiler passes that automatically derive these chunks from the served LLM as-is, without porting each model architecture to a separate training engine. Our evaluation on three MoE models, served on four A100 GPUs, shows that fine-tuning throughput with BubbleTea is 1.9–6.2 \times faster than with LLMStation, a state-of-the-art colocation system. BubbleTea keeps p90 inference latency within 10% of serving alone in 50 of 63 workloads, compared to 44 for LLMStation. On bursty production traces, BubbleTea trains 3.3 \times faster than LLMStation and 14 \times faster than a stock trainer colocated under CUDA MPS, with p90 inference latency staying within 7% of serving alone. Comments: 12 pages, 8 figures Subjects: Operating Systems (cs.OS); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC) Cite as: arXiv:2611.24496 [cs.OS] (or arXiv:2611.24496v1 [cs.OS] for this version) https://doi.org/10.48550/arXiv.2611.24496 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-25] Probing agent overeagerness in realistic settings with continuation audits NEURIPS2026 NEURIPS
链接: https://arxiv.org/abs/2611.24491
作者: Sean Lim,Clement Neo
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
备注: Presented at NeurIPS Workshops: (1) Third Workshop on Agents in the Wild: Safety, Security, and Beyond, and (2) The 1st Workshop on Interpreting Agent Behavior (IAB) at NeurIPS 2026
Abstract:Frontier coding agents increasingly exhibit overeagerness, taking surprising, sometimes reckless actions beyond a user’s mandate — reported in recent system cards from frontier model developers. These cases are surfaced through internal deployment monitoring, which requires private traffic and is out of reach for open-weight models and independent evaluators. Automated alignment audits are the natural alternative, but capable models increasingly recognise the synthetic setup and adjust their behaviour. We introduce Continuation Audits, a pipeline that grounds the auditor on a prefix drawn from a real coding-agent transcript and directs it to continue the trajectory toward a seeded behaviour. Because the prefix is drawn from a real session, the auditor only needs to maintain a realistic setting rather than manufacturing it. We validate the method on SWE-Chat prefixes: on a paired subset of 90 audits a judge cannot reliably distinguish continuations from their source sessions (realism win rate 0.42 vs.\ 0.43), and across all 900 audits verbalized eval-awareness does not exceed the rate observed in the unaudited real sessions. Applying the pipeline to three recent models across three seeds, we find each model leads a different seed — DeepSeek~V4~Pro on colleague-credentials misuse, GPT-5.6~Terra on file-permission bypasses, and Kimi~K3 on treating diagnostic questions as instructions. Kimi~K3 additionally continues to bypass file-permission protections after an explicit this http URL prohibition while the other two stop entirely, corroborating Moonshot’s own report of excessive proactiveness.
[AI-26] Mitigating fix reversal: Counterfactual constraints for function-level vulnerability detection
链接: https://arxiv.org/abs/2611.24489
作者: Yun Bian,Shihao Li,Haiquan Wang,Yi Chen,Zhao Yang,Zhe Cui
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
备注: 21 pages, 9 figures
Abstract:Function-level vulnerability detectors often exhibit fix reversal, assigning a higher vulnerability probability to a fixed version than to its vulnerable counterpart. Fixes usually lengthen functions, and larger length increases are associated with more severe reversal. Code-transformation experiments indicate that fix reversal mainly stems from a length shortcut: vulnerability probability increases with function length when content is fixed. We propose CFVul to address this problem. Sliding-window encoding covers complete functions, retaining fix information beyond the truncation boundary. During training, a content constraint encourages the vulnerable version to receive a higher vulnerability probability under a common length input. During inference, a length constraint averages vulnerability probabilities over fixed reference lengths, reducing the influence of actual length changes. On PrimeVul, CFVul achieves a fix-pair ranking metric, PairRank, of 59.4%, exceeding the existing methods compared. For fixes that increase function length by more than 15%, PairRank rises from CodeBERT’s 25.8% to 48.4%. PairRank also improves in experiments with four additional models and three additional datasets. By weakening the length shortcut, CFVul reduces cases in which fixed versions receive higher vulnerability probabilities than vulnerable versions. It requires only existing fix pairs and applies to multiple pretrained code models, facilitating adoption in existing detectors.
[AI-27] Study and Improvement of Search Algorithms in Stochastic Perfect-Information
链接: https://arxiv.org/abs/2611.24486
作者: Quentin Cohen-Solal,Tristan Cazenave
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:In this article, we generalize Unbounded Minimax, the state- of-the-art search algorithm for deterministic zero-sum two- player games with perfect information to the framework of stochastic zero-sum two-player games with perfect informa- tion. We experimentally show that this generalized algorithm achieves better performance than the main stochastic search algorithms.
[AI-28] Harmonic Circuit: Chord Progression Lies in a Sparse Principal Information Flow in Symbolic Music Models NEURIPS2026
链接: https://arxiv.org/abs/2611.24453
作者: Ruiji Yu,Gus Xia
类目: ound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Accepted at the NeurIPS 2026 Workshop on Interpretability for Discovery. 12 pages, 6 figures
Abstract:Mechanistic Interpretability (MI) seeks to decode neural network computations into human-understandable logic. A core methodology in MI is circuit discovery: identifying the information flow in sub-networks that drive specific model behaviors. While circuit discovery has matured within Natural Language Processing (NLP), the interpretable circuits in music models remain largely unexplored. In this work, we identify a music-functional circuit within two symbolic music generative models: ChatMusician and MuPT. The music function we consider is the prediction of the terminal tonic root in a classical harmonic progression pattern I \ldots V \rightarrow I . We use the Auto-Emergence algorithm to extract a principal sparse sub-network as the identified circuit. Validation via causal intervention confirms that this identified circuit is a primary driver of harmonic progression behaviors in the symbolic music models. To our knowledge, this represents the first systematic circuit-discovery study conducted within the music domain. Our demo page is available at: this https URL.
[AI-29] From real to synthetic: PSG-GAN learns sleep apnea signal dynamics for AI training
链接: https://arxiv.org/abs/2611.24436
作者: Riku Huttunen,Samu Kainulainen,Matias Rusanen,Sami Myllymaa,Juha Töyräs,Timo Leppänen,Henri Korkalainen
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Medical Physics (physics.med-ph)
备注: 23 pages, 9 figures
Abstract:Synthetic polysomnography (PSG) data has great potential in healthcare applications, for example in substituting real data that is sensitive and challenging to collect. Previous research on synthetic PSG data has been largely focused on short segments of individual signals, overlooking the inter-signal dynamics related to sleep disorders. In this study, we developed a generative adversarial network (PSG-GAN) for co-synthesizing longer segments of multiple PSG signals with diverse and complex characteristics typical of obstructive sleep apnea (OSA). The synthetic signals appeared realistic, and the electroencephalography and electrooculography band powers were similarly distributed across sleep stages. Moreover, the PSG-GAN learned to mirror signal interactions such as changes in photoplethysmography-based heart rate during and after breathing obstructions. An automatic scoring model trained solely with automatically scored synthetic signals reached good performance on sleep staging (\kappa=0.71 and 0.63) and respiratory event segmentation (\kappa=0.57 and 0.60), on two test sets consisting of healthy volunteers and OSA patients, respectively. This demonstrates the potential of synthetic data in supporting algorithm development for sleep diagnostics. The source code is available at this https URL.
[AI-30] Keeping Shielded Data-Driven Controllers Safe Under System Evolution via Model Consistency
链接: https://arxiv.org/abs/2611.24428
作者: Roland Stolz,Vitus Lüntzel,Romain Pascual,Eric Sax,Matthias Althoff
类目: ystems and Control (eess.SY); Artificial Intelligence (cs.AI)
备注:
Abstract:Cyber-physical systems (CPS) increasingly contain data-driven controllers, which raise safety concerns given their black-box nature. A common mitigation is to enforce system safety via a shield around the controller, which restricts the inputs based on assumptions about the physical system. However, these assumptions are usually fixed and may become invalid as the system evolves (during development or due to component degradation), possibly leading to unsafe behavior. We address this limitation in a model-driven approach that treats both the data-driven controller and safety shield as first-class models that are integrated into a virtual single underlying model (V-SUM) alongside system models. We describe system evolution as changes in the associated models, such that consistency checking yields a continuous assessment of safety guarantees. In case of inconsistencies, consistency-preservation rules automatically update the models and restore a valid safety shield. We evaluate our approach on a reach-avoid task for a unicycle system. The results show that consistency-driven evolution effectively maintains safety guarantees under system changes and provides a principled basis for evaluating and restoring performance.
[AI-31] MINT: Meeting-time Indicators for Truncation in Multi-Step Off-Policy RL NEURIPS2026
链接: https://arxiv.org/abs/2611.24413
作者: Seungyub Han,Taehyun Cho,Dohyeong Kim,Kyungjae Lee,Jungwoo Lee
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: Accepted at NeurIPS 2026. 81 pages, 40 figures, including appendices. Code: this https URL
Abstract:Many multi-step off-policy temporal-difference methods handle policy mismatch through products of per-step trace coefficients, leading to a trade-off between variance control and long-horizon signal. The maximum trace length consequently becomes a sensitive hyperparameter. We propose MINT (Meeting-time INdicators for Truncation), which replaces these products with a binary stopping indicator: one until coupled behavior and target trajectories first meet, zero thereafter. At the true value function, the indicator’s idempotence yields a variance bound governed by meeting-time survival. For practical implementation, a single sampled action provides a model-free surrogate for the stopping rule, requiring no transition model or mixing-time estimate. We derive separate sufficient conditions for contraction of the state-coupled operator and its practical surrogate under fixed-policy evaluation. On MuJoCo and DeepMind Control tasks, MINT outperforms the reported multi-step baselines; the main MuJoCo comparison uses a common cap without per-task cap-length tuning. We also demonstrate the effectiveness of MINT on Atari 100k, RWRL, and OGBench experiments; MINT adapts the backup horizon to reduce target variability relative to long fixed-horizon backups across tasks and dynamics.
[AI-32] Risk-Triggered Search: Behavior-Preserving Safe Control
链接: https://arxiv.org/abs/2611.24409
作者: Xinyu Deng,Bohao Lan,Kunpeng Li,Mingkai Yuan,Weihang Pan,Binbin Lin
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Pre-trained visual world models generate multi-step action plans from current and goal observations, yet terminal-proximity objectives fail to prevent forbidden-zone traversal or rigid-body collision sliding. Purely visual interfaces may lack analytical dynamics, structured states, or explicit collision geometry, requiring internal prediction of action-conditioned process risk. We propose Risk-Triggered Search (RTS) for frozen Joint-Embedding Predictive Architecture (JEPA) world models. Rather than relying on static state features that cannot distinguish candidate plans sharing an observation, RTS unrolls action-conditioned future latent prefixes in the frozen world model and trains a lightweight risk head with process-violation supervision. To intervene selectively while preserving safe nominal behavior, RTS uses the same predicted prefix-risk criterion as both the search trigger gate and hard feasibility gate: safe nominal plans execute directly, whereas alarms trigger local Cross-Entropy Method (CEM) search to minimize nominal deviation within the feasible set. To prevent isotropic Euclidean minimal modification from shrinking obstacle-directed actions into zero-amplitude stalling, we introduce an amplitude-preserving metric d_\mathrmAP that retains tangential progression while maintaining a unique zero at the nominal plan. Under unified gating and this unique-zero condition, bypassing safe-state search yields closed-loop trajectories strictly identical to every-step same-kernel search. Evaluated under a process-safety benchmark and Safe Arrival Rate (SAR) metric across five tasks, RTS improves SAR over Direct by 12.67–41.67 percentage points on navigation and articulated control and +2.08/+4.17 pp on Cube/PushT boundary cohorts, maintaining 100% action identity with every-step search while cutting controller time by 17.71%–70.41% across 1,968 paired trajectories.
[AI-33] mporal Rank in Recurrent Neural Networks
链接: https://arxiv.org/abs/2611.24394
作者: Leo Dobson,Anton Ragni
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Recurrent Neural Networks (RNNs) are a natural framework for temporal modelling, yet are often dismissed due to their sequential, non-parallelisable computation and difficulty in capturing long-range dependencies, typically attributed to vanishing and exploding gradients. We identify a distinct and under-explored limitation: an information bottleneck induced by rank-constrained hidden state dynamics, which limits the number of independent input directions that can be preserved over time, as reflected in the rank of the input-to-state mapping. We show that this constraint restricts temporal capacity independently of gradient stability. To address this, we introduce structured high-dimensional state dynamics that preserve temporal separability and prevent saturation of the input-to-state Jacobian over time. On controlled sequence tasks, the resulting model exhibits rapid learning despite remaining strictly within a standard RNN formulation. Our results demonstrate that RNN performance is governed by the evolution and preservation of temporal rank, and suggest that mitigating gradient pathologies alone is insufficient without explicitly maintaining temporal capacity.
[AI-34] LLM -Assisted Multi-Fidelity Bayesian Optimization: Fusing Expert Knowledge with Numerical Data
链接: https://arxiv.org/abs/2611.24377
作者: Dongjun Han,Mingyu Lee,Ikjin Lee
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE)
备注: 35 pages, 8 figures, 3 tables
Abstract:Multi-fidelity Bayesian optimization (MFBO) reduces engineering design cost by combining inexpensive low-fidelity (LF) models with a few accurate high-fidelity (HF) evaluations and sequentially selecting designs to evaluate. However, when HF data are scarce, the LF-HF discrepancy is difficult to learn from numerical data alone, especially when physics omitted in the LF model makes it design-dependent. Yet even without sufficient data, domain experts often know which physics the LF model omits and under which conditions its effect grows. Existing multi-fidelity (MF) models cannot use such qualitative knowledge directly, but large language models (LLMs) can interpret it and compare designs accordingly. The proposed LLM-Assisted MFBO adds a knowledge basis to the conventional MF model: from an expert description of the omitted physics, an LLM judges which of two designs requires the larger signed residual correction, and a pairwise Gaussian process converts these judgments into the basis. HF data calibrate its contribution and sign, so the knowledge is not trusted blindly. On a lithium-ion battery design problem, the proposed method reduced the regret area under the curve of conventional MFBO by 61%. Knowledge quality, rather than the choice of LLM, determined whether the search was accelerated, as shown by a controlled study that varied the two independently. A knowledge-level ablation further quantified the contribution of each knowledge statement. These results show that LLM-interpreted expert knowledge can complement numerical data as an information source in MFBO.
[AI-35] PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents
链接: https://arxiv.org/abs/2611.24362
作者: Lipeng He,Yihan Wang,Jiawen Zhang,N. Asokan
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注: 18 pages, 6 figures
Abstract:Large language model (LLM) agents routinely interact with untrusted external data, exposing them to indirect prompt injection (IPI) attacks that hijack agents into executing attacker-chosen tasks. To uncover these vulnerabilities at scale, automated red-teaming methods often leverage reinforcement learning (RL) to generate attacks that achieve high attack success rates (ASR). Model developers use the attacks discovered during internal red-teaming to harden agents through adversarial training before deployment, a collaborative offensive-defensive process known in cybersecurity as purple-teaming. However, we show that achieving high ASR during red-teaming does not guarantee robustness gains from subsequent adversarial training. Specifically, we find that two state-of-the-art RL-based red-teaming methods, RL-Hammer and PISmith, suffer from mode collapse, repeatedly discovering similar attack strategies whose additional successes add no additional robustness improvements to adversarial training. To address this gap, we introduce PIPETTE, a unified prompt injection purple-teaming framework that effectively converts red-teaming discoveries into defender robustness improvements. PIPETTE employs a dictionary learning-based reward that encourages broader coverage of attack strategies during red-teaming. It also integrates an on-policy adversarial training pipeline tailored to IPI defense, enabling agents to resist attacks while completing legitimate user tasks. Extensive evaluations across agentic benchmarks (AgentDojo and ASB) demonstrate that PIPETTE produces defenders that achieve near-zero attack success rates on static attacks and can withstand strong adaptive attacks, while maintaining benign task utility close to that of the undefended baseline.
[AI-36] Semantic Context Handoff with On-Demand Evidence for Long-Horizon Agents
链接: https://arxiv.org/abs/2611.24357
作者: Guolong Zheng,Chen Hou,Xu Yang,Jianshan Zhang,Tao Huang
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Traditional long-horizon LLM agents often hand off context through full-history prompting, retaining all previous action–observation interactions. As execution proceeds, this history keeps growing, increasing inference cost and potentially degrading accuracy. Existing methods control this growth by preserving compact conclusions while removing earlier observations, but doing so can discard evidence that later becomes relevant. We introduce Semantic Context Handoff (SCH), whose central abstraction is state–evidence separation: a compact semantic state carries the agent’s current interpretation, while supporting action–observation evidence remains separately addressable and can re-enter context when later execution makes it relevant, without separate summarization or retrieval-model calls. Empirically, SCH improves long-horizon agent performance while substantially reducing context cost. In a controlled 150-task SWE-bench study, SCH’s evidence recovery improves Pass@1 from 80.0% to 86.9% at essentially unchanged token use and execution time. In the complete-system comparison, SCH achieves 86.9% Pass@1 while using 43% fewer tokens and 41% less wall-clock time than the strongest baseline by task performance. Across models and benchmarks, SCH consistently reduces token use relative to full-history prompting while remaining competitive with or improving upon the evaluated baselines. Trajectory analysis confirms that SCH recovers evidence from substantially earlier parts of the interaction history rather than merely preserving a recent context window. Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2611.24357 [cs.AI] (or arXiv:2611.24357v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2611.24357 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-37] When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers
链接: https://arxiv.org/abs/2611.24353
作者: Guijia Zhang,Hongbo Qiu,Jiajun Zha,Guohao Cai,Hao Zheng,Huihao Jing,Kexu Cheng,Shipeng Luo,JinYang Huang,Shing-Chi Cheung,Changsheng Lu,Haoxuan Che,Harry Yang
类目: Artificial Intelligence (cs.AI)
备注: 28 pages, 5 figures, 14 tables
Abstract:Scientific figures carry evidence for the claims that cite them, so reviewing a paper means checking that a figure agrees with the text, and not only describing what the figure shows. Benchmarks of figure-text verification typically hand the model the figure, the claim, or the knowledge that an inconsistency exists, which leaves untested whether a model can find the figure worth checking in a complete paper, relate a visual element to the claim it conflicts with, and recognize the conflict. We study this autonomous setting on 31 full manuscripts, each carrying a reviewer-documented figure-text inconsistency from PRISMM-Bench that is hidden from the model. Six multimodal models review each PDF under a neutral prompt, and we then add one input at a time to the same manuscripts. Of 1,445 open-ended reviews, 60.6% mention the flawed figure, yet a grading rubric accepts only 3.3% as diagnosing its contradiction; a blind audit finds the rubric lenient, so we read these as rubric-accepted rates and compare conditions rather than absolute values. In a separate controlled run, naming the figure, without stating that it is flawed, raises rubric-accepted diagnosis from 4.3% to 20.4%, while a consistency instruction and coarse visual cues do not. When the contradiction is visible only in the rendered figure, text-only review yields no accepted diagnosis, while review of the rendered pages with the figure named reaches 15.8%. A baseline that lets the model choose which figures to check, ranked by centrality, stays at 4.8%. Deciding what to verify is thus a large part of the difficulty, and the selection rule we tested does not resolve it. Multimodal document understanding should be evaluated on whether a model can propose what to verify, not only answer questions about a figure it is handed.
[AI-38] automodel: A Skill for Agent ic Model Discovery
链接: https://arxiv.org/abs/2611.24347
作者: Marco Virgolin,Harry Desmond,Ester Bergantin,Giorgio Morales,John W. Sheppard,Luigi Romano,Alessandro Lucantonio
类目: Artificial Intelligence (cs.AI); Cosmology and Nongalactic Astrophysics (astro-ph.CO); Astrophysics of Galaxies (astro-ph.GA); Neural and Evolutionary Computing (cs.NE)
备注:
Abstract:The search for better models, whether expressed as analytical equations or neural architectures, has increasingly shifted from manual design toward data-driven automation. While a multitude of algorithms exist, ranging from exhaustive search to AlphaEvolve, recent advances in general-purpose coding agents such as Claude Code and OpenAI Codex raise the question: could providing such systems with a “model discovery recipe” be enough to discover or improve models? We present automodel, an agent skill for general model discovery. With case studies in astrophysics, cardiovascular dynamics, precision agriculture, traffic flow, and vehicle tire dynamics, we show how commercial agents equipped with automodel can successfully identify novel and scientifically-relevant models. Our findings suggest that encoding model-discovery methodology as a portable agent skill can provide a simple, interactive, powerful and broadly applicable alternative to domain-specific automated discovery pipelines. The skill is available at: this https URL.
[AI-39] Reducing Inference Cost of Coding Agents with Timeout Cascades
链接: https://arxiv.org/abs/2611.24324
作者: Boxi Zhou,German Sviridov,Dmitrii Ustiugov,Haris Javaid
类目: Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
备注:
Abstract:AI coding agents have become one of the most popular applications of large language models. Yet, agentic AI comes at the cost of a tremendous amount of GPU compute resources, which limits its profitability and its viability at scale. In this paper, we investigate whether smaller, cheaper models can cut this cost without sacrificing accuracy. We evaluate 36 popular large and small open-weight models released since 2024 on a software engineering benchmark, and find that recent small models are closing the accuracy gap compared to their large counterparts. We investigate the structure of this gap and find that the gap is highly concentrated around hard tasks that small language models fail to solve while achieving remarkably good accuracy for easy and medium complexity tasks. Following this observation, we show that by utilizing a hybrid inference strategy that tries a small model first and escalates to a large one only when needed, we can achieve up to 33% reduction in total GPU-hour costs and 16.4% on all tasks from SWE-bench Verified, without sacrificing accuracy. Strikingly, this simple cascade nearly matches an oracle-style task router, highlighting a practical path for deploying cost-aware coding agents in production without complex learned task routers and with minimal engineering complexity. Subjects: Artificial Intelligence (cs.AI); Software Engineering (cs.SE) Cite as: arXiv:2611.24324 [cs.AI] (or arXiv:2611.24324v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2611.24324 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-40] Benchmarking LLM abilities on postdoc-level natural science problems
链接: https://arxiv.org/abs/2611.24321
作者: Alina Avanesyan,Alexey Boldyrev,Liqun Deng,Denis Derkach,Mary Godunova,Mikhail Hushchyn,Dmitry Ilvovsky,Irina Knyazeva,Irina Piontkovskaya,Dmitry Polubotko,Fedor Ratnikov,Ekaterina Trofimova,Zhenhe Zhang
类目: Artificial Intelligence (cs.AI); Data Analysis, Statistics and Probability (physics.data-an)
备注: 22 pages, 7 tables, 2 figures
Abstract:Scientific problems have no answer key, so evaluating an agent means deciding what counts as a better solution before any agent is run. Existing benchmarks either match a symbolic answer against a key or, where they require executable work, score it by unit tests, by a rubric, or by position on a competition leaderboard. None of these is the criterion by which a field decides that one published method beats another. STEMBench adopts that criterion directly. Each of its twelve research problems in high-energy physics, astrophysics and neuroscience is scored by its own field’s figure of merit, and each comes with two measured anchors: a submission that uses no domain knowledge, and an expert reference reproducing a published method, both obtained by running code shipped with the benchmark. The interval between them states how much of a problem is crossed by scientific insight rather than by engineering. Across eight frontier models, task-level wins and domain-level leadership dissociate: a model that wins five of the twelve tasks leads no domain mean. In the high-effort cross-model comparison, at least one model’s mean score exceeds the expert anchor on seven of twelve tasks. The remaining five span uncertainty estimation, latent and causal inference, and cross-site clinical classification. Numerical scores and official-submission validity capture different aspects of performance and are reported separately. We release the tasks, graders, anchors and harness.
[AI-41] AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM Agents
链接: https://arxiv.org/abs/2611.24313
作者: Stavros Damianakis,Konstantinos Psounis
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: 18 pages, 8 figures, 5 tables. Project page at this https URL and code at this https URL
Abstract:Prompt injection is the dominant unsolved security risk for LLM agents, and the community now agrees that robustness must be measured against adaptive attackers rather than fixed benchmark injections. Many adaptive red teamers exist, and most are built to maximize attack success rate (ASR). Yet ASR counts how often a target breaks, not in how many different ways. A red teamer can achieve a high ASR by reusing a few techniques that work, leaving the others untested and absent from the attacks used to harden models. We present AutoPI, an adaptive red teamer designed to find many different ways to break a target, not just to break it often. AutoPI uses PIAtlas, a taxonomy of prompt-injection techniques, both to guide its search and to measure coverage. For each scenario, an attacker LLM writes an injection for every sub-technique in the taxonomy and then reworks the ones that fail using feedback from an LLM judge. The method is black-box and training-free, it plugs into existing benchmarks in place of their fixed attack templates, and it stays up to date as the taxonomy evolves. We evaluate AutoPI on AgentDojo and IPI Arena against open-weight and proprietary target models. Alongside ASR, we report technique coverage, the fraction of taxonomy classes, techniques, and sub-techniques that yield at least one successful injection. On AgentDojo, AutoPI finds a successful injection against Qwen3.6-35B in all six classes of the taxonomy and for more than 90% of its techniques. Against Claude Haiku 5.5, the most robust target we evaluate, it achieves between 20% and 50% higher ASR and between 25% and 400% higher class-level coverage than recent state-of-the-art adaptive red teamers, while requiring no training. We release our code and the generated attacks to support reproducible, coverage-aware evaluation.
[AI-42] KnowledgeClaw: Towards Agent ic Knowledge Engineering via Omnimodal Hypergraphs
链接: https://arxiv.org/abs/2611.24294
作者: Haoran Luo,Jiapu Wang,Linhao Luo,Luu Anh Tuan
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Knowledge engineering (KE) turns raw files into a reasoning-ready knowledge base, a task now within reach of LLM agents. KE has progressed from chunk and graph retrieval-augmented generation (RAG) to memory agents, agentic KE and multimodal KE. However, a live knowledge base needs omnimodality, a closed-loop lifecycle and multi-granular application at once, yet existing systems meet at most two and no benchmark measures all three. We formalize Omnimodal Knowledge Engineering (OmniKE) and release OmniKE-Bench, six tasks with 1,829 model-free items over 3,960 real files in five modalities. We further propose KnowledgeClaw, an agentic system that builds an omnimodal typed hypergraph, keeps it current with a closed-loop lifecycle agent and routes each query to the granularity it needs. Against 30 baselines, it ranks first on all six tasks with an overall OKE of 55.9, +14.6 over the strongest baseline. Our software and data are publicly available at this https URL.
[AI-43] Beyond Corpus-Driven Imitation: Evolving Musical Mindsets in the Agent ic Orchestra via Quantum Reinforcement Learning
链接: https://arxiv.org/abs/2611.24290
作者: Eduardo Reck Miranda,Moein Fahmideh Vatandoost
类目: Artificial Intelligence (cs.AI); Quantum Physics (quant-ph)
备注: This is the accepted, prepublication version of a paper forthcoming in Computer Music Journal, MIT Press. Beyond routine editorial and typesetting amendments, the content remains subject to further revision before final publication
Abstract:This paper presents a framework for computer-aided music composition and performance based on quantum reinforcement learning and multi-agent interaction. We introduce the agentic orchestra, where musical agents evolve distinct mindsets through trial-and-error engagement with a musical environment, rather than through training on large datasets. The agents are implemented as parameterised quantum circuits. They evolve distinct compositional behaviours, referred to as mindsets. This enables the exploration of musical decision spaces using quantum superposition and entanglement within a hybrid quantum-classical loop. We present two complementary agent architectures: atomistic agents, which process pitch and rhythm independently, and holistic agents, which handle pitch and duration as unified note events. Mindsets are shaped through musically grounded reward functions, which can be combined and weighted to develop different mindsets across agents. The resulting system positions quantum computing agents less as autonomous composers and more as adaptive musical partners, each developing its own musical personality through interaction rather than imitation. Demonstrations show how agents generalise their learned behaviour to previously unseen material, operating in interactive and offline batch modes using MIDI-encoded music. While quantum computers are still in development, experiments conducted on IBM quantum hardware confirmed that musically meaningful learning is achievable with existing quantum computers. For composers and performers, the agentic orchestra opens new possibilities for emergent, process-oriented musical creativity, offering a scalable foundation for quantum computer-aided music composition as quantum computing technology continues to advance.
[AI-44] FedE2E: One-Shot Federated Learning for End-to-End Autonomous Driving with Closed-Loop Evaluation
链接: https://arxiv.org/abs/2611.24254
作者: Tao Lian,Antonio M. López
类目: Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
备注:
Abstract:Federated learning enables collaborative model training without sharing datasets. However, existing federated learning methods for autonomous driving typically require multi-round communication and lack closed-loop evaluation, while one-shot methods mainly focus on perception tasks. To address these limitations, we propose \method, a one-shot federated learning method for end-to-end autonomous driving. It comprises multi-client calibration, anchor-based feature alignment, and token-wise consensus distillation to distill knowledge from heterogeneous models. Closed-loop evaluation on Bench2Drive demonstrates the effectiveness of \method in both one-shot and multi-round federated learning. In the one-shot setting, it achieves a Driving Score of 55.85, which is 4.07 points higher than prior work. In the multi-round setting, it reaches 60.35 after five communication rounds, the best performance among all compared methods under the same number of communication rounds.
[AI-45] From Points to Trajectories: Rethinking Multimodal Alignment for Physical Events
链接: https://arxiv.org/abs/2611.24249
作者: Yiyang Zhu,Guangyuan Liu,Jingdi Lei,Li Wei,Chau Yuen
类目: Artificial Intelligence (cs.AI); Signal Processing (eess.SP)
备注:
Abstract:Advancing physical artificial intelligence (Physical AI) requires moving beyond static object recognition to ground how real-world mechanics (e.g., impact, friction, and force transfer) unfold over time. While multimodal aligners seek to capture these interactions by matching multi-sensor streams, prevailing architectures compress each temporal sequence into a single pooled embedding. We show that this paradigm fails on twin events: distinct physical executions recorded under identical task conditions. Collapsing the temporal dimension averages out transient cross-modal dynamics, effectively reducing physical event alignment to static scene recognition. Formally, we prove that as sequence length increases, any temporally pooled representation converges to the stationary recording condition. Consequently, as windows grow, no choice of loss function or embedding geometry can disambiguate twin events from pooled features alone. Instead, we show that physical events are uniquely identified by the event gap, namely the instantaneous, synchronous co-fluctuations across modalities that are erased by pooling and accumulate only along unpooled trajectories. To recover this signal, we introduce CounterPoint, a lightweight plug-in module that strips out each modality’s temporal mean to score frame-level cross-modal agreement. Integrated across four state-of-the-art (SOTA) multimodal aligners, CounterPoint shifts representations from coarse condition matching to fine-grained physical event alignment, consistently outperforming existing temporal alignment methods.
[AI-46] What Survives the Shift from Text to Pixels? Conditional Protection against Prompt Injection in Image-Rendered Documents
链接: https://arxiv.org/abs/2611.24231
作者: Haoyang Liu,Yinong Xu,Zhe Lang,Zhenyu Guan,Qianyun Zhang
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注:
Abstract:Multimodal language models increasingly answer questions over external documents, exposing them to indirect prompt injection: commands planted in untrusted content to hijack the response. Rendering untrusted content as page images lowers attack success on average, leaving open which injected operations survive rendering, at what utility cost, and whether failed image attacks reflect unread commands or commands read but not followed. We pair text and page-image versions of 191 English question-answering sources across six models, measuring marker appending, answer formatting, and answer replacement alongside clean Question-Answering utility. Rendering reduces append and format execution for every model, yet Qwen3-VL-8B still executes image-borne replacement at 77.96%. Protection also depends on target type, rendering settings, wording, and repetition, and a search over eight fixed attack candidates largely closes the image deficit in appending and formatting for Qwen and Gemini, though not for GPT-5-nano. Utility ratios on original questions range from 93.31% to 101.61%, but Qwen falls to 61.99% when answers combine two evidence spans. Illegibility cannot explain all failed attacks: models often transcribe and identify image-borne commands they do not execute, and trusted user authorization raises execution on unchanged pages. Inside Qwen, patching early attack-region states induces all three operations, more weakly in images. Attention blocking suggests why: a text command acts through its own positions, whereas visual encoding spreads an image command and its target across the instruction page. Rendering thus offers conditional protection, motivating evaluations that report security by model, operation, and attack budget alongside utility.
[AI-47] Learning to Coordinate: LLM -Guided Workflow Evolution for Token-Efficient Red Teaming
链接: https://arxiv.org/abs/2611.24221
作者: Xiaoyang Xing,Wenyu Dai,Xiaoliang Gong,Asoke K Nandi,Anthony Cohn
类目: Artificial Intelligence (cs.AI)
备注: 10 pages, 3 figures
Abstract:Automated red teaming is essential for identifying safety failures in large language models (LLMs), but its practical scalability depends on the resources required to discover those failures. Multi-agent systems can combine complementary capabilities, yet repeated planning, redundant exchanges, and ineffective information routing can make their coordination expensive. Moreover, a target model’s response indicates the outcome of a test without identifying which collaboration decision should change. We present Learning to Coordinate (L2C), an LLM-guided framework that evolves an executable multi-agent workflow for each red-teaming case. L2C represents agent responsibilities, execution dependencies, and input bindings in a versioned graph, and associates each graph with records of the information actually delivered and produced. A controller trained through supervised fine-tuning integrates these records with target responses, evaluator verdicts, and remaining budget to retain the workflow, refine instructions, restructure collaboration, or lock further editing. This design connects feedback to explicit changes in subsequent execution while allowing a useful organization to continue without repeated controller intervention. The evaluation protocol covers HarmBench with three execution backbones and two target models, together with component ablations, dataset transfer, and evaluator checks. Across the six executor-target configurations, L2C improves ASR over EvoSynth by 1.4 to 4.5 percentage points while reducing mean online tokens by 35.6% to 37.3%. L2C connects adaptive collaboration to the resource requirements of automated security evaluation.
[AI-48] owards Reliable Scaling Laws for Tabular Foundation Models
链接: https://arxiv.org/abs/2611.24214
作者: Johannes Hog,Neeratyoy Mallik,Steven Adriaensen,Jan Hendrik Metzen,Tom Viering,Danny Stoll,Frank Hutter
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Scaling laws are widely used to predict foundation model performance and guide compute allocation. However, tabular foundation models (TFMs) differ from conventional foundation models in that they are pretrained on virtually unlimited synthetic data from configurable distributions, while ultimately being evaluated on real-world datasets. We investigate whether scaling laws for TabPFNv2-style models reliably extrapolate to larger compute budgets and generalize across evaluation settings. Our analysis reveals differences between synthetic and real-world scaling behavior and shows that accurately fitting a scaling law does not necessarily yield reliable prescriptions on unseen datasets. In particular, selecting configurations using too few evaluation datasets leads to systematically over-optimistic performance estimates. We further demonstrate that compute-optimal scaling requires expanding the range of synthetic pretraining dataset sizes alongside model size and training compute. Increasing the maximum numbers of rows and features sampled during pretraining improves performance and produces more predictable scaling trends. These insights guide the training of more efficient TabPFNv2-like models, using the open-source training pipeline of nanoTabPFN.
[AI-49] Conditional What-If Generation for Radio Access Network Energy under Rare and Unseen Contexts
链接: https://arxiv.org/abs/2611.24208
作者: Kevin Wardakhan,Marie Line Alberi-Morel,Jérémie Leguay,Afef Feki,Adrien Sardi,Sébastien Archeveque,Claudiu Mihailescu
类目: Networking and Internet Architecture (cs.NI); Artificial Intelligence (cs.AI)
备注: 8 pages, 6 figures, 5 tables. Submitted for publication
Abstract:Energy-saving mechanisms in RANs must balance energy reduction and network performance under varying traffic conditions. Conditional generative models support what-if analysis by generating plausible power-consumption trajectories for radio configurations and operating scenarios, helping operators evaluate energy-saving policies before deployment. However, representative measurements are costly and operationally constrained, so historical data cover only limited configurations and conditions, leaving some contexts poorly represented or unseen during training. In this paper, we study conditional generation of RAN power-consumption trajectories under decreasing training coverage, from well-represented to rare and unseen configurations. We first introduce a context-normalized conditional generator combining temporally aligned conditioning with configuration-dependent target normalization. It improves generation for well-represented configurations but becomes less reliable as training exposure decreases because target consumption is harder to estimate. We therefore introduce a prediction-based generative framework that decomposes power consumption into global prediction, residual level and scale correction, and temporal residual generation. Experiments on a commercial 5G RAN and a public dataset show that hardware metadata provides transferable information about residual scale, while limited target observations at inference improve generation through residual-level correction without retraining. An ablation study confirms that residual-level correction provides most of the gain. Both approaches outperform state-of-the-art methods for well-represented configurations, while the prediction-based approach also generalizes to rare and unseen contexts and, with one target observation, reduces median MAE by about 50% compared with TimeWeaver.
[AI-50] GALA: Gallery-Level Alignment for Real-Life Vision Decoding from EEG
链接: https://arxiv.org/abs/2611.24206
作者: Philippe Forest,Giulia Lioi,Mai Pham Quyen,Alix Lamouroux,Javen Shi,Adrien Merlini
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Neural decoding of visual experience from electroencephalography (EEG) has achieved strong performance in intra-subject settings, but current methods fail to generalize across individuals: on the standard 200-way benchmark THINGS-EEG-2, top-1 accuracy drops from above 80% within subjects to about 32% for an unseen subject. Adapting a pre-trained encoder to new subjects and sessions typically requires labeled data, a bottleneck whose collection time severely limits practical device deployment. In a practical Brain-Computer Interface (BCI), however, the possible outcomes are known in advance and form a gallery of candidates, and the unlabeled EEG of the user accumulates as soon as the device is used. We propose a new setting, incremental transductive calibration, in which the decoder is calibrated regularly on all the unlabeled EEG received so far and on the known gallery, without any labels. This setting is closer to the conditions of a deployed BCI than the zero-shot and batch evaluations in current use. We evaluate it with a protocol in which queries arrive one at a time, with unknown and possibly unequal frequencies, and each query is decoded before it is used for calibration. In this setting, we introduce GALA : GAllery-Level Alignment, a label-free calibration method which registers the unlabeled EEG embeddings onto the gallery through shrinkage whitening, hubness correction and a Sinkhorn–Procrustes alignment with query frequencies estimated from the buffer, updated after every query. On THINGS-EEG-2, with a gallery of 200 images, it raises the top-1 accuracy on a new subject from 28% to 67% over 53 minutes of use. Across 60 evaluation settings that vary the gallery size, the query frequencies and the number of repetitions averaged per query, GALA is the most accurate label-free method in every setting, 6.6 points above the best competing method on average at the end of the stream.
[AI-51] Worth Another Look:Constrained World-Model Planning for Visual Inspection
链接: https://arxiv.org/abs/2611.24187
作者: Ziyue Kang,Zixi Qi,Yuchen Dai,Nana Lv,Xirong Cheng,Liang Liu,Ying Li,Haoyan Wei,Haiyang Yao
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM)
备注:
Abstract:Visual inspection in field robotics and environmental monitoring must localize multiple targets under strict compute and sensing caps. Exhaustive sweeps waste calls on absent classes, while myopic and search-free policies stumble over heterogeneous tariff cliffs or discard viewport edits whose value emerges only after re-querying. We formulate inspection over frozen detectors as neuro-symbolic observation planning under partial observability, decoupling a compact recurrent world model (WM) from exact symbolic feasibility and tariff rules. Receding-horizon beam search with online replanning jointly schedules class queries, crops, enhancements, and termination (stop), paired with slack-aware shadow pricing and legal fallback at larger budgets. Across eight terrestrial, optical, and sonar benchmarks, multi-step lookahead outperforms static schedules and search-free offline RL under tight budgets – gaining up to 28.4 AP percentage points over static priority on RUOD and 15.1 on ARIS – while dynamic pricing lifts large-budget performance on all eight domains at 11 ms planning cost. Ultimately, pairing lightweight learned dynamics with exact symbolic constraints turns frozen detectors into adaptive, budget-aware inspection agents without retraining.
[AI-52] Fast Bayesian Methods for Uncertainty Quantification in Sparse AI Evaluation NEURIPS2026
链接: https://arxiv.org/abs/2611.24159
作者: Dario Deniz Ergun,Haohua Wang,Brendon Tao,Jonah Kollenberg,Chen Ye Gan
类目: Artificial Intelligence (cs.AI); Computation (stat.CO)
备注: 13 pages, 5 figures. Accepted at NeurIPS 2026 AI Measurement Science Workshop
Abstract:Estimating how an AI system will perform on a new benchmark need not require another large model. We use public evaluation counts to choose a weak Beta prior, then update it exactly from at most 31 revealed outcomes. The predictor uses no item semantics, pretrained weights, or inference API. Its single online evaluation in the AIMS competition returned a team-reported Brier ALC of 0.188477, a 24.6% reduction from the constant-0.5 reference. The complete submission is 7,754 bytes; a server microbenchmark measures 12.71 microseconds per prediction with 31 matching labels. We describe how source evaluations are compressed into 301 prior means and show why a concentration of two lets new evidence rapidly determine the prediction. The resulting posterior gives exact uncertainty about the modelled success rate without numerical integration or neural inference. A public evaluation covers 79 system-benchmark pairs and eight seeds, reaching Brier ALC 0.200250 for the submitted method. Accompanying code and sufficient statistics reconstruct the fitted prior byte for byte and replay all 632 evaluation trajectories. The result demonstrates how much predictive value a compact Bayesian model can extract from sparse target feedback at very low online cost.
[AI-53] Embodied Graph: A Relational World Model for Long-Horizon Tasks
链接: https://arxiv.org/abs/2611.24115
作者: Zongyuan Wu,Xin Wang,Tongtong Feng,Yuan Meng,Zeyang Zhang,Ling Feng,Wenwu Zhu
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Long-horizon tasks in embodied and web environments involve extended decision sequences and delayed feedback, making accurate planning crucial for success. World models tackle this by learning latent dynamics for imagination-based planning. However, existing approaches merely rely on dense pixel-level observations, which forces the model to absorb redundant information while overlooking relations among task-relevant entities. Consequently, small errors may accumulate over long imagined rollouts and gradually corrupt the predictions. In this paper, we explore world models that imagine over sparse state abstractions, which poses two key challenges: 1) how to remove redundant information and form a compact state abstraction; 2) how to incorporate the relational structure to learn reliable long-horizon dynamics. To address these challenges, we propose Embodied Graph, a relational state abstraction that supports structured dynamics modeling and long-horizon imagination, and the Relational World Model (RWM), which performs imagination directly over graph states. RWM consists of (i) a Task-Aware Embodied Graph Constructor that filters redundant details and maintains graph memory through online updates, and (ii) an Embodied Graph Recurrent State Space Model that learns node-wise latent dynamics while capturing inter-node dependencies. The imagined graph states provide predictions of action consequences to guide decision-making. Experiments on Minecraft and Crafter demonstrate the effectiveness of RWM in embodied environments. Notably, in Minecraft, RWM is the only method that obtains diamonds within the given budget. Experiments on web interaction further illustrate the applicability of relational state prediction to the decision making process of web agents.
[AI-54] he 7th International Verification of Neural Networks Competition (VNN-COMP 2026): Summary and Results
链接: https://arxiv.org/abs/2611.24114
作者: Tobias Ladner,Konstantin Kaulen,Stanley Bak,Julien Girard-Satabin,Taylor T. Johnson,Edoardo Manino,ThanhVu H Nguyen,Ben Wooding,Haoze Wu
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:This report summarizes the seventh International Verification of Neural Networks Com- petition (VNN-COMP 2026), held as part of the Symposium on AI Verification (SAIV) co-located with the International Conference on Computer-Aided Verification (CAV) at FLoC 2026 in Lisbon, Portugal. The goal of the competition is to provide an objective comparison of the state-of-the-art methods in neural network verification, in terms of scal- ability and speed. Along this line, we used standard formats (ONNX for neural networks and VNNLIB for specifications), standard hardware (all tools are run by the organizers on AWS), and tool parameters provided by the tool authors. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this competition.
[AI-55] LEDGE: An Efficient Simulator for Distributed LLM Serving on Heterogeneous Edge Devices
链接: https://arxiv.org/abs/2611.24085
作者: Zihao Wang,Ye Tian,Tajana Rosing,Nalini Venkatasubramanian
类目: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)
备注:
Abstract:Evaluating distributed large language model (LLM) serving on heterogeneous edge devices requires exploring diverse model partitions, device placements, and network conditions, making repeated physical experiments costly. An efficient simulator for edge LLM serving must support diverse, fine-grained placement configurations, capture runtime behavior, and accurately predict execution times while keeping simulation overhead low. We present LEDGE, a lightweight simulator for fast, fine-grained evaluation of distributed LLM serving on heterogeneous edge devices without executing tensor operations. LEDGE constructs event graphs supporting non-uniform tensor and pipeline parallelism, estimates execution costs using reusable performance models calibrated from end-to-end measurements, and simulates shared-link contention. We evaluate LEDGE on a heterogeneous testbed of 10 Jetson and Raspberry Pi devices, covering 17 model–precision combinations across 4 LLM families. On Qwen deployments, LEDGE achieves request-level mean absolute percentage errors below 3.6% and a 135.3 times aggregate speedup over physical inference. Performance model calibrated on one LLM family yield 7.44% zero-shot end-to-end latency MAPE across three other families. Further simulations show how shared-link contention affects placement decisions and network topology constrains scaling. Across four edge environments, LEDGE completes 26,773 simulation runs in 83.8 minutes, enabling efficient exploration of deployment strategies under diverse resource conditions and serving objectives. The source code is publicly available at this https URL.
[AI-56] Recurrent Domain Incremental Adaptation of Time-Series Foundation Models
链接: https://arxiv.org/abs/2611.24075
作者: Sachith Abeywickrama,Peiliang Gong,Emadeldeen Eldele,Min Wu,Chau Yuen
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Signal Processing (eess.SP)
备注: Preprint. Under Review
Abstract:Pretrained Time-Series Foundation Models (TSFMs) provide strong representations for downstream classification tasks. Yet adapting a TSFM under changing operating conditions remains challenging. A model must improve on each newly observed condition without sacrificing conditions it has already learned, often from only a small labeled calibration set and without knowing the operating condition at inference time. We find that this difficulty has an important geometric structure. Under an operating-condition shift, the embedding space of a TSFM does not move uniformly: some local regions change substantially, while others remain nearly unchanged. Consequently, a global adaptation can unnecessarily move embeddings that were already correctly classified. To address this, we propose the novel PACT (Piecewise Adaptation via Chart Transport). PACT partitions the frozen source embedding space into local charts and learns lightweight residual transports for the regions affected by each new condition. An explicit identity path is always retained, and a learned gate softly selects among the identity and accumulated transports for each input. The pretrained TSFM encoder, classifier head, and previously learned transports remain fixed. With only 5% labeled calibration data per arriving condition and no condition identity at test time, PACT maintains an average macro-F1 above 0.95 across a four-condition bearing-fault stream. Under homogeneous tasks and label spaces, similar retention-adaptation behavior is observed across public benchmarks. Code is available at this https URL.
[AI-57] 3D GameDev Arena: Benchmarking Coding Agents That Plan Build and Playtest extreme Long-Horizon Browser Games
链接: https://arxiv.org/abs/2611.24073
作者: Bizhe Bai,Jiliang Jian,Yuqing Yan,Tao Chen,Ying Wen
类目: Artificial Intelligence (cs.AI)
备注: 23 Pages
Abstract:Coding agents now resolve repository issues and generate compact playable games, but whether they can build a large 3D game from scratch remains open. Such games tightly couple mechanics, scenes, assets, and interaction logic, and their quality rests on player enjoyment, which existing benchmarks rarely assess. Thus, we introduce 3D GameDev Arena, a first extreme long-horizon benchmark of 108 from-scratch game-development tasks across 14 genres. Given only a text specification and UI/UX reference images, a fully automated plan–implement–test–repair pipeline builds each task into a self-contained browser game. Runs average \sim 1,000 LLM calls ; the longest uses 14,407 calls, 2.13B tokens, and 608 assets. Evaluation spans (1) objective code-level functional acceptance against the game specification, (2) agent-based QA and playtesting of 3D presentation and playability, and (3) blind pairwise Bradley-Terry human preference in a public arena. The preliminary findings indicate that long-horizon 3D game development remains far from solved, and code-level correctness alone does not dictate player preference. The public arena is hosted online.
[AI-58] Replay Dont Resample: Off-Policy Consolidation of Long-Horizon RL Experts
链接: https://arxiv.org/abs/2611.24059
作者: Zhenrui Yue,Gongxiaohui Chen,Huimin Zeng,Honglei Zhuang,Jiashuo Sun,Zhen Han,Dong Wang
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Frontier post-training follows a common recipe: train one expert per domain with reinforcement learning (RL), then consolidate the experts into one model via multi-teacher on-policy distillation (MOPD). Such consolidation is expensive: the student samples in every environment again, an expert scores every sampled token, and the student inherits the drift of the final experts. Yet we argue that the experts’ RL runs have already produced what consolidation needs: rollouts with their sampling probabilities and rewards, in the order the experts learned from them. We propose multi-expert replay policy distillation (MRPD), which replays these training logs through the student in the experts’ order. At each step, the policy gradient is recomputed on these scored trajectories, and the final experts’ distributions are distilled at the logged prefixes. Following the experts’ own trajectory reduces the training-inference mismatch and keeps the student within the clip range, so the replay is off-policy in origin but quasi-on-policy throughout consolidation. MRPD samples nothing and calls no environment or reward model: the trainer is the sole process, and consolidating the experts costs only about a seventh of the compute used to train them. On three long-horizon agentic domains (reasoning, search and SQL), MRPD yields the best student model, substantially outperforming online RL baselines, and surpassing MOPD from the same experts at a fraction of MOPD’s cost. Our analysis on the same logs shows that the replayed trajectories carry most of the gain, and that both the policy gradient and the distillation term add to it. Most interestingly, the order of the logs matters more than their amount or rewards, suggesting that expert training logs are a curriculum, not a data mixture.
[AI-59] From Data to Valves: A Self-Learning Production-Grade Edge Stack for Sustainable HVAC Control
链接: https://arxiv.org/abs/2611.24052
作者: Abbas Raza Ali,Muhammad Hamza Rehman,Xu Li,Muhammad Ajmal Siddiqui
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Learned HVAC controllers rarely leave the simulator, and those that do are typically trained offline, frozen at deployment and retrained from the cloud. We present a production stack in which three learning components run on one NVIDIA Jetson Orin gateway per building, adapt continuously from the building’s own data, and verify their own effect. (a) A compact transformer-encoder forecaster, one per equipment group with learned per-unit embeddings, predicts every air-side unit’s next-cycle temperature and learns prequentially, one gradient step per five-minute cycle, behind five independent gates that bound the influence of any corrupt reading. (b) A distributional soft actor-critic agent, one shared policy across unit types, converts the measured state into a chilled-water valve position under a structural safety envelope that no learned component can soften; its second generation learns nightly on the device from recorded transitions alone, with a behaviour anchor and critic calibration terms, is scored by four model-free metrics and promoted by atomic hot-swap. © A change-point regression baseline held to ASHRAE Guideline 14 supplies the counterfactual against which savings are certified. On eighteen months of data from a hotel the forecaster converges from random weights within a week and reaches 0.37, 0.39 and 0.11 degree C MAE for air-handling, fresh-air and fan-coil units with no retraining event; on a different hotel the baseline reproduces a 46-day held-out period to within 0.07%. The RL agent has completed its in-simulator campaign and runs in shadow on 104 units; we report measured latency and memory of every service on a shared 15 GB device, the systems lessons of the campaign, and precisely what the evidence does and does not yet establish.
[AI-60] Attention Is Not Relevance: Challenging the Causality of Attention in Autonomous Driving
链接: https://arxiv.org/abs/2611.24051
作者: Luca Lusvarghi,Javier Gozalvez
类目: Artificial Intelligence (cs.AI); Robotics (cs.RO)
备注:
Abstract:Attention has become one of the most widely adopted solutions for estimating the relevance of surrounding objects and interpreting an autonomous vehicle’s driving decisions. However, its ability to capture the causal impact of individual input elements on an AI model’s output has long been discussed in the literature, particularly in the Natural Language Processing (NLP) domain. This work brings this discussion into the autonomous driving domain by analyzing the accuracy of attention in estimating the impact of an input object on an autonomous vehicle’s driving, i.e., its relevance. To this end, we compare different attention-based relevance estimation methods against a causal reference of object relevance obtained through counterfactual analysis. We perform our analysis on more than 80,000 synthetic driving scenes using an autonomous vehicle based on PlanT 2.0, a state-of-the-art Transformer-based model for object-level planning. Results show that attention provides a correlational rather than causal indication of object relevance. Attention-based relevance estimation methods do not consistently identify all relevant objects and tend to incorrectly classify as relevant objects with little or no impact on the vehicle’s driving. In particular, results show that attention can effectively identify relevant objects in simple single-lane, straight-road scenarios, but its accuracy degrades in more complex settings, such as multi-lane roads and intersections. These findings highlight the limitations of attention as a proxy for object relevance, calling for novel methods able to accurately capture the causal impact of individual objects on the vehicle’s driving and distinguish it from correlational associations.
[AI-61] st-Time Symmetrization: Scaling Laws for Model-Agnostic Learning under Structure
链接: https://arxiv.org/abs/2611.24032
作者: Behrooz Tahmasebi,Melanie Weber
类目: Artificial Intelligence (cs.AI)
备注: 38 pages, 13 figures
Abstract:Test-time compute has emerged as a powerful way to improve the accuracy of language models at inference time, with scaling laws quantifying how accuracy improves with additional forward passes. Existing scaling laws, however, focus on repeated sampling from a fixed prompt, even though many tasks admit multiple equivalent representations of the same underlying problem. A graph does not depend on how its nodes are numbered, and a program does not depend on the names of its variables, yet language models can change their answers under such rewritings. In this paper, we study test-time symmetrization, which uses forward passes on rewritings of the input and aggregates the resulting answers over the group G of rewritings. We prove that no amount of sampling and repeated decoding from a single representation can replace rewriting, while only O(\log |G|) random rewritings suffice to recover the information provided by all |G| rewritings. The same set of O(\log |G|) rewritings provides simultaneous approximation across models, decoding rules, and sampled reasoning traces, and the logarithmic scaling is optimal in general. We validate the theory with experiments on Qwen2.5-7B, Phi-4-mini, Llama-3.1-8B, and Qwen2.5-14B, with chain-of-thought reasoning, on graph reasoning, logical deduction, and multiple-choice question answering. At the same number of forward passes, spending them on rewritings of the input is substantially more accurate than spending them on repeated sampling from the original prompt whenever the model’s answer depends on how the input is written, and the number of rewritings needed grows only logarithmically with the number of possible rewritings, as the theory predicts. Our results provide a principled way to enforce symmetries at inference time when they cannot be built into the model, which we call model-agnostic symmetrization.
[AI-62] Intent Contracts: Taking Tools Away from Agents
链接: https://arxiv.org/abs/2611.24025
作者: Ziyuan Wang,Genliang Zhu,Xiaobo Chen,Chu Wang
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注: 10 pages, 1 figure, 7 tables. Code and data: this https URL
Abstract:Intent contracts are an authorization interface for tool-using LLM agents in which the planning model holds no tools. For each step it submits a short logged contract: a goal, a verbatim quotation of the user, and scope fields. Code validates the contract, and a small trusted model checks it against the user’s words and writes the tool call itself. Provided the validation and dispatch code is correct, every executed call is written by the trusted model, the planner’s influence on a call is confined to a contract logged before it, and the trusted side decides how much of a request to carry out. We compare contracts with two reviewers of planner-written commands: one sees only the user’s words and the command, the information boundary of a reasoning-blind permission classifier, and one also sees the planner’s goal and a checked quotation. With gpt-6.1-terra as planner and GLM-5.3-Flash as the trusted model on three AmPermBench task families, every object changed under contracts is named in the goal of the contract that produced the change; under review the planner names objects in its commands and rarely in its goals. When the planner adds work the user never asked for, contracts complete the user’s task in 27/30 runs and the intent-visible reviewer in 16/30, because a reviewer can only refuse a whole command. Contracts do not reduce unauthorized actions. They are indistinguishable from blind review, and once the benchmark’s environment terms are clarified, every condition that changes state hits a wrong target in 40-49% of those runs. That error is the planner’s choice of objects for an ambiguous reference, which no trusted model here can see the environment to check; telling the executor to decline when unsure does not change it. We evaluate benign over-reach with one pair of models, not adversarial planners. We release the plugin and run records.
[AI-63] Space-X: Self-Evolving Spatial Agents Weaving Clues into Evidence Chains
链接: https://arxiv.org/abs/2611.24004
作者: Zijian Zhang,Wenqing Tian,Qiang Liu,Liang Wang
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Spatial reasoning connects visual observations, geometric estimates, and linguistic decisions into a coherent derivation. Yet tool-augmented agents often acquire evidence without clear targets, combine outputs under inconsistent object or reference-frame semantics, and misjudge evidence sufficiency. Thus, more tools or calls do not reliably improve reasoning. We introduce Space-X, a spatial-reasoning framework that self-evolves toward complete, continuous evidence-chain construction. An Execution Agent acquires, associates, and judges evidence per problem using role-specific memory. A Control Agent diagnoses evidence-chain breaks from execution traces and jointly revises tools, interfaces, skills, memory organization, and decision rules. This loop co-evolves capability and evidence use: evidence needs guide capability development; usable results guide actions. Across multiple benchmarks, Space-X yields gains with four base models. Relative to baselines, GPT-6-sol improves answer accuracy on ViewSpatial and MMSI by 50.39% and 23.07%, respectively; Gemini-3.8-Flash gains 47.50% and 25.22% on these benchmarks. This provides a unified route from fragmented spatial clues to traceable, derivable answers.
[AI-64] GLIDE: Scalable and Event-Aware Inductive Forecasting over Evolving Logistics Graphs KDD2027
链接: https://arxiv.org/abs/2611.23942
作者: Zhi Sheng,Chuyue Wang,Jiahao Ji,Yile Liang,Xuetao Ding,Jinghua Hao,Jie Feng,Yong Li
类目: Artificial Intelligence (cs.AI)
备注: 14 pages, 5 figures, KDD2027
Abstract:Accurate forecasting of order volume at scale-effect hotspots (SEHs), spatially coherent groups of logistics flows that serve as distributed optimization units for real-time many-to-one dispatch, is a prerequisite for efficient order pooling in on-demand food delivery (OFD) platforms such as Meituan. However, real-world logistics networks present unique challenges that render standard spatiotemporal prediction model ineffective. (i) Dynamic Topology: Unlike static road networks, logistics graphs are ephemeral, with hotspot emerging and vanishing continuously; (ii) Scale–Volume Heterogeneity: Graph sizes vary from 10^2 to 10^4 , while zero-inflated, long-tailed volumes create computational bottlenecks and bias predictions toward inactivity; and (iii) Event-Driven Disruption: Exogenous events (holidays, weather, academic calendars) induce systematic demand shifts that purely data-driven backbones cannot anticipate. In this paper, we present GLIDE, a production-ready framework that models evolving logistics graphs in a scalable latent space while adapting to heterogeneous demand regimes and event-driven shifts. GLIDE introduces a Graph-Guided Latent Aggregation mechanism, which dynamically projects varying physical subgraphs into a fixed-size, semantically stable latent space via a differentiable sparse assignment…
[AI-65] Can you make models believe false facts in the prompt?
链接: https://arxiv.org/abs/2611.23938
作者: Ananya Sutradhar,Fabien Roger
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Making LLMs believe false facts is useful for alignment evaluations and for building model organisms. Previous work explored doing so with Synthetic Document Finetuning (SDF). But SDF requires fine-tuning one new model for each set of injected false beliefs. To enable larger scale SDF-powered alignment evaluations, we attempt to produce a single model that believes any fact placed in a facts tag in the prompt. To do so, we explore doing prompt-conditioned SDF: fine-tuning a model on synthetic documents covering thousands of facts, with each document conditioned on a fact presented in the prompt. We evaluate how deeply the model believes the facts provided in the prompt using evaluations from prior work and a new Jacobian lens evaluation. We find that the belief depth on unseen facts scales roughly log-linearly with the number of training facts, making this a cheap, promising avenue for making models believe false facts, though preliminary experiments on applying it to alignment audits on the small model we studied showed negative results due to these small models often already acting as if they believed facts provided in regular honeypot system prompts. Using RL to reduce the rate at which models verbalize following a false fact in the prompt, on top of prompt-conditioned SDF, generalizes to held-out evals and almost matches SDF on some evaluations. But training against one of our metrics for belief depth makes it harder to trust those evaluations.
[AI-66] A Hierarchical Onboard Vision-Language-Navigation Framework for Aerial Robots in Complex Environments
链接: https://arxiv.org/abs/2611.23930
作者: Yuze Wu,Mo Zhu,Xingxing Li,Yuheng Du,Zhaoqi Wang,Yuxin Fan,Wenjun Li,Kenghou Hoi,Zhichao Han,Xin Zhou,Fei Gao
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI)
备注: 20 pages, 14 figures
Abstract:Unmanned Aerial Vehicles (UAVs) are expected to achieve semantic reasoning, intelligent decision-making, and long-horizon navigation under strict onboard constraints. While recent foundation-model-driven approaches have shown promising progress toward this goal, existing methods remain limited by challenges in 3D data availability, trajectory feasibility, long-horizon reasoning, and computational efficiency in complex environments. To bridge these gaps, we propose VLN-AR, a hierarchical vision-language-navigation (VLN) framework for aerial robots. VLN-AR decouples multimodal reasoning and low-level control, enabling structured prediction of 3D pose commands and navigation intentions that are translated into dynamically feasible actions. To mitigate data scarcity and domain gaps, we introduce a fully automated 3D Gaussian Splatting (3DGS)-based pipeline to generate scalable photorealistic UAV navigation data. A progressive three-stage training strategy further enhances grounding, scene reasoning, and complex navigation. In addition, a lightweight cognitive agent improves long-horizon reasoning via memory-aware task decomposition. Finally, we design onboard optimization techniques, including token sparsification and model lightweighting, achieving 7.1x-9.4x speedup while maintaining high navigation performance on resource-limited UAV platforms. Experiments show great improvements over existing methods in both simulation and real-world UAV navigation. Project website: this https URL
[AI-67] Cardiac MRI-informed electrocardiography enables biologically grounded cardiomyopathy detection
链接: https://arxiv.org/abs/2611.23925
作者: Zhengyao Ding,Kewen Wang,Yujian Hu,Ziyu Li,Yiheng Mao,Yue Wang,Haitao Li,Dengnan Zhang,Xuanhong Liu,Yue Wang,Jing Wang,Yue Chen,Longbo Wang,Mengjia Chen,Chengchen Zhao,Xuesen Chu,Yubin Zhang,Mor Peleg,Judith Somekh,Mingxuan Yuan,Hui Yan,Yue Gu,Fan Yi,Zhilong Jia,Zhentao Tang,Xi Hu,Fei Wu,Hongkun Zhang,Ting Chen,Zhengxing Huang
类目: Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM)
备注: 46 pages, 9 figures
Abstract:Precise myocardial phenotyping at population scale is limited by the cost and accessibility of cardiac magnetic resonance (CMR), whereas electrocardiography (ECG) is widely available but reflects electrical rather than structural and functional information. Here we develop MyoECG, a cross-modal representation learning framework that distils myocardial structural and functional knowledge from CMR into standard 12-lead ECGs, enabling imaging-informed cardiomyopathy detection and subtype classification from ECGs alone. MyoECG was pretrained on 45,241 paired ECG-CMR examinations from UK Biobank and a tertiary cardiovascular centre, then developed and validated across more than 1.8 million ECGs from seven independent datasets. Across internal and external datasets spanning prevalences of 0.2-54.7%, MyoECG outperformed five pretrained ECG foundation models, with AUROCs of 0.800-0.974 for cardiomyopathy detection. In a multi-reader study of six cardiologists, MyoECG assistance improved mean sensitivity by 0.109 (95% CI, 0.041-0.175; P = 0.009) and specificity by 0.056 (95% CI, 0.004-0.108; P = 0.040). In 41,128 UK Biobank participants, the MyoECG score showed genetic correlations with cardiomyopathy (rg = 0.409) and dilated cardiomyopathy (rg = 0.371), genome-wide associations at four loci, including regions mapped to NFIA and KCNQ1, and associations with 69 circulating proteins related to myocardial stress, cell adhesion, vascular biology and metabolism. The cardiomyopathy correlation attenuated to non-significance after adjustment for 27 measured CMR traits (rg = 0.131; P = 0.143), indicating that the score captures CMR-derived information rather than dataset-specific shortcuts. MyoECG offers a scalable route to CMR-informed cardiomyopathy assessment and a strategy for transferring knowledge from resource-intensive imaging to widely deployable physiological signals.
[AI-68] Vanguard: Knowing When to Trust the Draft in Speculative Vision-Language Agents
链接: https://arxiv.org/abs/2611.23917
作者: Hanqi Mao,Jiajun Fan,Ruihan Guo,Shuaike Shen,Ziwen Wang,Hui Ren,Xianrui Zhong,Ge Liu
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Vision-language agents that browse websites, operate phones and act in homes reason before every action, so every step waits for a large model, a latency and cost that stand in the way of deployment. Step-level speculation lets a small model draft each step and calls the large model only on rejected drafts, but its outcome rests on the judge, and a judge that reads only a step’s text cannot tell whether the step fits the screen it acts on. We find that a few samples of a small model can contain a correct step about as often as the large model gets the step right, and not always on the same steps: checked against the screen, small drafts can be not only cheaper but better. We present Vanguard, a training-free judge that leaves the draft and target models unchanged: a general decision model, with the screenshot in view, gives each candidate step a probability of being right in one forward pass; the best candidate is kept if it clears a threshold, and otherwise the large model writes the step. On web, mobile and embodied agents, a 2B draft judged by Vanguard is at least as accurate as its 8B target and significantly more accurate on Mind2Web and EB-ALFRED, while the target writes only a minority of the steps; it completes far more closed-loop episodes, and with a 27B target from another family it is both faster and more accurate. Vanguard is the most accurate judge we compare on the agent tasks; it relies on the screenshot on web and mobile tasks, its calibrated threshold transfers to unseen websites, and across model families its gain over the target grows with the draft’s pass@4 relative to the target’s accuracy. Vanguard composes with token-level speculative decoding and matches trained process reward models on mathematics, turning speculation from a way to save the large model’s work into one that can improve on it.
[AI-69] MLE Agents : Autonomous ML Engineering in Large-scale Recommendation Systems NEURIPS2026
链接: https://arxiv.org/abs/2611.23910
作者: Matthew Li,Albert Liang,Dilip Raj
类目: Artificial Intelligence (cs.AI)
备注: NeurIPS 2026 AutoMLR Workshop (oral)
Abstract:The typical workflow for improving large-scale ranking and recommendation models is slow and tedious: an engineer forms a hypothesis, implements it as a code change, trains a model, and evaluates it both offline and online, often relying on their own intuition and experience along the way. Autonomous LLM agents promise to speed this up by running the loop on their own, but it is still unclear whether they can do this in industry-scale systems rather than on benchmarks or small proxy tasks. We build human-on-the-loop machine learning engineer (MLE) agents that propose, implement, train, and evaluate model changes on large-scale search rankers. Engineers retain control over objective design, search direction, and final-model review. We identify three obstacles that make this industry setting challenging: feedback takes hours per experiment, offline metrics do not always predict online impact, and improvements saturate after the early wins. We address these with parallel training runs, an objective calibrated to match online impact, and careful context engineering. In one campaign, the system ran 1,097 completed candidate experiments and accepted 325, a 29.6% acceptance rate compared with 15.6% in the earlier campaign under a different objective and search constraint. Its best model increases search volume by 0.23% in an A/B test of large scale. Several improvements were architectural changes the team had never tried, drawn from the broader industry and literature, suggesting that MLE agents can explore regions of the design space that human intuition overlooks. We find that the design of the harness around the model matters: intuition suggests that a stronger underlying model is enough, but in our experience, the design of the loop around it is just as critical. More broadly, our work shows that autonomous agents can do real machine-learning engineering in industry-scale systems.
[AI-70] Evaluating invariance training with certified answer-changing controls in neural graph algorithms
链接: https://arxiv.org/abs/2611.23907
作者: Zhe Dong,Fang Qin
类目: Artificial Intelligence (cs.AI)
备注: 25 pages, 3 figures, 16 tables
Abstract:Invariance training aims to make neural algorithmic reasoners robust to answer-preserving input changes and is judged by accuracy and by how rarely such edits change predictions. We ask whether this evaluation detects lost sensitivity to answer-changing edits on minimum spanning trees of synthetic graphs, where both directions can be certified. Attaching a subgraph through one bridge provably preserves every original label. Attaching it through two edges, oracle-selected to change a label, and lowering one weight inside the graph serve as answer-changing controls; a response is a changed prediction on an edge whose label changes. A reproduced graph transformer (GDT) within its published accuracy range changes a provably fixed label on 53.3-65.9% of the 64-vertex graphs it solves exactly after a 64-vertex bridge attachment; removing the connecting edge changes little. With 16-vertex graphs and attachments, three invariance objectives lower GDT violations from 2.91% to 0.52-2.02% of pairs at essentially unchanged accuracy. Training also lowers the response to two-edge attachments from 97.6% to 66.0-79.0%, while the response to in-base weight edits stays near 99.5%. At 64 vertices accuracy also falls, but on 16-vertex bases only the attachment control reveals the loss. A message-passing network loses some attachment response at small penalty weights but gains little invariance. Running the conventionally trained GDT on each biconnected block separately removes all observed violations and keeps accuracy and both responses, so the task does not force the trade-off. Sensitivity checks for trained invariance should include certified answer-changing edits that attach material as the training edits do.
[AI-71] Kernel Ridge Attention: Nonlinear Memory with Selective Forgetting
链接: https://arxiv.org/abs/2611.23896
作者: Haowei Chen,Yichen Wang,Congfu Xu
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: 15 pages
Abstract:Kernel Ridge Attention (KRA) is a recurrent sequence mixer that combines learned nonlinear features with covariance-guided memory updates. It stores associations through recursive ridge regression, using correlations among past keys to determine how new observations modify memory. A content-dependent Ornstein–Uhlenbeck transition controls retention and restores uncertainty for future writes. This construction admits an exact temporal-kernel interpretation and maintains an inference state whose size is independent of context length. Associative block maps and explicit adjoints support parallel training while preserving the recursive update. In controlled language-modeling experiments, KRA improves prediction quality over attention and recurrent baselines, strengthens natural associative recall over the evaluated recurrent models, and achieves practical distributed training throughput. These properties provide a principled foundation for nonlinear, adaptive sequence memory.
[AI-72] EVA-RSI: Recursive Self-Improvement of Video Evaluators for Judging Temporal Quality
链接: https://arxiv.org/abs/2611.23890
作者: Yu Yuan,Ruizhou Ding,Shaoheng Fang,Stanley H. Chan,Zhuo Chen
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Video evaluators play an important role in video generation, yet they miss motion defects, limiting the quality of generated videos. Recursive self-improvement (RSI) has made strong progress in mathematics and coding, where explicit verifiers provide reliable feedback on a single goal. Video evaluation poses two difficulties for RSI: it has no verifier, and it must judge many dimensions at once, such as physics, smoothness and dynamics, where a gain on one can cost another. We introduce EVA-RSI, a multi-agent framework for exploring RSI under these two constraints. We build a fixed proxy exam with construction-defined targets by applying controlled changes to videos, such as temporal reversal and frame dropping. These edits supply supervision without human annotation. A small set of human-labeled real videos anchors the exam to human judgment. Each dimension has an LLM agent that proposes improvements. A leader agent coordinates the proposals, and a Nash bargaining rule selects updates that balance progress across dimensions. Humans design the exam, review candidate changes and revise the curriculum between stages. Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run. On the dynamics subsets of public benchmarks, our evaluator achieves the highest observed scores among 21 tested models. As an RL reward, it increases generated motion and earns human preference. A data-selection pipeline incorporating it also increases motion.
[AI-73] Risk-Controlled Correction for Frozen Multimodal Models under Missing Modalities
链接: https://arxiv.org/abs/2611.23886
作者: Hoang Pham Cong,Long D. Nguyen,Ho Viet Duc Luong,Nguyen Tran Minh Nhat,Binh P. Nguyen,Huynh Thi Thanh Binh
类目: Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
备注: Code: this https URL
Abstract:A deployed multimodal predictor may face missing modalities after its weights can no longer be changed. This raises a question that training-time repair does not answer: what part of the resulting error is still correctable from the information that remains? We show that this part is characterized by the Bayes gap, the difference between the model’s prediction and the Bayes prediction given the observed modalities. For cross-entropy and squared error, excess risk measures this gap in the geometry of the loss; ambiguity in the observed modalities cannot be removed. Thus, a badly degraded prediction is not necessarily a correctable one. Estimating the Bayes prediction normally requires labels. However, when a model observing more modalities is conditionally correct, averaging its outputs conditional on the remaining modalities recovers this target without retrieval labels; with the same fixed blend and neighbors, it is no worse in expected cross-entropy or Brier risk than averaging hard labels. Some such assumption is unavoidable: otherwise the target is not identifiable from unlabeled data alone. GUARD (Gated Utility-Aware Recalibration for Deployment) leaves the deployed predictor unchanged. It estimates a candidate correction by retrieval, learns from deployment-matched labeled data when correction is likely to help, and calibrates a threshold on a held-out split by conformal risk control. Under exchangeability, the marginal probability that GUARD applies a correction whose loss increase exceeds a prescribed tolerance is bounded by a preset level, regardless of the learned score. In controlled experiments, correction gain follows headroom rather than raw error; across nine frozen model-dataset pairs spanning six domains, GUARD retains most of blanket correction’s gain within harm budgets that blanket correction often exceeds.
[AI-74] Asking but Missing: Decision Coverag e in Natural-Language Data Querying
链接: https://arxiv.org/abs/2611.23864
作者: Hangle Hu,Chenyu Hou,Bin Cao,Jing Fan,Wei Luo
类目: Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
备注:
Abstract:Large language models are increasingly used to translate natural language questions about structured data into executable programs. A fundamental challenge is recognizing when a question admits multiple computational interpretations and identifying the decisions needed to resolve them. We investigate whether language models recognize these unresolved decisions before generating code, or silently adopt assumptions that may lead to incorrect results. We refer to the latter behavior as Underspecified Decision Completion (UDC). To distinguish different causes of UDC, we introduce Grounded Code Generation (GCG), a diagnostic protocol that separates the identification of unresolved decisions, their resolution using evidence, and their translation into executable code. By examining each stage separately, we can determine where the decisions responsible for incorrect results were first missed or mishandled. We evaluate this framework on BIRD-Pandas with twelve models. The results show that answering clarification questions consistently improves execution accuracy, yet substantial errors remain. Our findings suggest that the main difficulty lies not in answering clarification questions, but in recognizing which questions need to be asked in the first place.
[AI-75] LAGR: Language-Aware Grasp Re-Ranking for Task Oriented Grasping
链接: https://arxiv.org/abs/2611.23858
作者: Manav Kulshrestha,Shreyans Jain,Apoorva Vashisth,Ankush Gupta,Damon Conover,Aniket Bera
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Manav Kulshrestha and Shreyans Jain contributed equally to this work
Abstract:Grasping is a fundamental capability for robotic manipulation, with the appropriate grasp depending on the downstream task. For example, a hammer should be held by its handle for hammering but by its head when it is handed over. Selecting such a grasp requires consideration of two criteria: physical feasibility on the object and suitability for the task. Developing such an approach requires dense grasp labels with both feasibility and task scores on real observations, but existing task-oriented grasping datasets are small, synthetic, or costly to annotate. To address this gap, we construct a dataset of 2.68 million scored 5-DoF grasp annotations over 100,911 image-task pairs from 17 object categories. Additionally, we introduce LAGR, a task-oriented grasp detection method that explicitly addresses both these challenges. Given vision-language inputs, LAGR first generates task-independent grasp proposals along with a physical feasibility score, and then scores the generated grasps with a language-conditioned re-ranker that reasons over all proposals jointly. Experiments show that LAGR outperforms the next best method by 9.3 percentage points. We further validate LAGR on real objects by executing task-oriented grasps with a Franka Research 3 robot.
[AI-76] RePlan-VLA: Progress-Aware Replanning for Long-Horizon Vision-Language-Action Models
链接: https://arxiv.org/abs/2611.23843
作者: Apoorva Vashisth,Manav Kulshrestha,Damon Conover,Zachary Kingston,Aniket Bera
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Apoorva Vashisth and Manav Kulshrestha contributed equally to this work. Prof. Zachary Kingston and Prof. Aniket Bera jointly advised this work
Abstract:Vision-Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robotic manipulation, yet their performance often degrades on long-horizon tasks that require multiple interdependent manipulations. Instead of burdening the VLA policies with long-horizon reasoning, memory, action generation, progress tracking, and error recovery, we argue for leveraging their short-horizon manipulation capabilities within an execution-aware framework. To this end, we introduce \textnormal\textbfRePlan-VLA, a VLA-agnostic framework that translates a VLA’s short-horizon capabilities into reliable long-horizon execution. RePlan-VLA explicitly monitors the robot’s motion for progress evaluation, and adaptively revises the future plan based on observed outcomes. We preserve the underlying VLA policy entirely, requiring no additional fine-tuning. Experiments on RoboCasa365 and CALVIN benchmarks show an improvement of 21.9% and 10.4% in success rate over the next strongest baseline. We also validate our approach for real-world deployment on a Franka Research 3 robot.
[AI-77] Evidence-Based Credit Assignment for Search-Augmented Language Models
链接: https://arxiv.org/abs/2611.23824
作者: Wen Wang,Jindong Wang,Jiahua Bao,Tu Yongsiqi,Yihao Liu,Junwei He,Wenkui Fan
类目: Artificial Intelligence (cs.AI)
备注: 16 pages, 4 figures. Includes supplementary material
Abstract:Reinforcement learning from terminal answer rewards enables search-augmented language models to interleave reasoning and retrieval without step-level annotations, yet outcome supervision assigns identical credit to every search in a trajectory regardless of whether it retrieved useful information. This limitation is especially severe when all sampled rollouts for a question fail, collapsing group-centered outcome advantages to zero even when trajectories gather markedly different evidence. We present Evidence-based Credit Assignment (ECA), which augments terminal outcome rewards with a training-only auxiliary signal measuring how retrieved documents improve the likelihood of a reference answer. Rather than scoring the full dialogue prefix, ECA queries a frozen snapshot of the current policy under an evidence-only prompt containing solely the question and the deduplicated documents observed so far, isolating the utility of retrieved passages from surface variations in generated thoughts or search queries. Within this framework, we compare two return formulations, ECA-Step (which accumulates step-clipped potential increments) and ECA-Horizon (which preserves the telescoping horizon gain in potential), paired with cross-fold group normalization, novelty gating, and search-local token assignment. Across seven Search QA benchmarks, ECA-Horizon improves strict macro exact match over an outcome-only GRPO baseline by 1.03 and 4.80 percentage points on Qwen2.5-3B- and 7B-Instruct, respectively, outperforming both RLOO and GiGPO. Zero-shot evaluation on MultiHop-RAG further demonstrates strong transfer to multi-hop inference queries.
[AI-78] Metrics-in-the-Loop: Complexity Feedback Reshapes Agent -Written Code
链接: https://arxiv.org/abs/2611.23817
作者: Nicolas Chapados,Jean-François Gagné,Nouha Dziri
类目: Artificial Intelligence (cs.AI)
备注: Agent Evaluation Science Symposium, Fall 2026, New York City
Abstract:Coding agents now write whole codebases in long autonomous runs, guided mainly by whether their tests pass. Yet passing tests alone does not reveal accumulating complexity or gaps in test coverage, which then persist as agents extend their own code. We introduce metrics-in-the-loop, an inference-time intervention that gives agents in-flight feedback: cheap, automatically computed code metrics injected into the agent’s context during the trajectory. After each code-changing agent turn, we insert short feedback messages naming changed functions whose cyclomatic complexity exceeds a limit (complexity feedback) or changed lines that are skipped by tests (coverage feedback), giving the agent the opportunity to refactor code or strengthen tests before continuing. We run extensive experiments and ablations comparing several interventions across three long-horizon coding benchmarks and four strong models. We find that complexity feedback significantly reduces erosion, a measure of complexity concentration of code, without affecting held-out correctness of the code. Broadly, our experiments investigate a practical form of agent “re-railing”, where timely, actionable measurements redirect agents before weaknesses become the foundation they keep building on. As agents scale to codebases with millions of lines, such interventions become critical to sustaining effective development and long-term code maintainability.
[AI-79] Continual Tool-Space Extension Learning for Multimodal Agents
链接: https://arxiv.org/abs/2611.23810
作者: Jinbae Seo,Jungin Park,Hyunsung Jang,Namkoo Ha,Jiyoung Lee,Kwanghoon Sohn
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Multimodal agents largely rely on generic tools such as cropping, search, and code execution, leaving specialist vision models outside their tool space. However, bringing such a model into an already trained agent is not straightforward. The challenge is to learn each new tool efficiently while retaining the ability to use existing ones. This work proposes COTEX, continual tool-space extension learning, that adds new tools at efficient adapter-level cost. COTEX locates two sets of parameter directions; one triggers a tool call and thereby opens a multi-turn reasoning trajectory, while the other determines which tool is chosen. The adapter of a new tool is restricted to the calling directions that leave earlier tool choices unaffected by orthogonal constraint. We further introduce a protocol that converts specialist models’ existing task-specific annotations into verified tool-use trajectories. Experiments on two representative agents and three specialist tools show gains on ten vision-language benchmarks, with at most a 0.3-point drop on existing tools.
[AI-80] Interpreting Timestamp Permutations in Dynamic Link Prediction
链接: https://arxiv.org/abs/2611.23753
作者: Ilia Karpov,Kirill Koloskov,Valeriy Kuleshov,Dmitry Ilvovsky
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO); Social and Information Networks (cs.SI)
备注: 5 pages total (ACM sigconf format). Companion code and trained models: this https URL
Abstract:Timestamp permutations are used to test whether dynamic link predictors exploit temporal information, but a performance gap can reflect changes in training, evaluation history, or both. We separate these effects with a crossed diagnostic and measure input exposure: whether altered histories actually change the predictor’s inputs. On MOOC, TCL trained with binary cross-entropy gains approximately eight pairwise-accuracy points from original-chronology training under either evaluation history, while changing history at fixed weights gives only 0.03 points. The small history contrast reflects unchanged inputs for 68.37% of queries, stable rankings for most others, and cancellation of gains and losses. Under original evaluation history, suppressing explicit time encoding reduces the training contrast from 8.03 to 3.64 points, a paired reduction of 4.39 points (95% interval [3.68, 5.09]) across six seeds. DyGFormer shows corresponding training and history contrasts of 22.34 and 0.38 points across two seeds, despite greater exposure. Partial history permutations further separate nominal intervention strength, input changes, and aggregate accuracy. These comparisons turn a single permutation gap into interpretable statements about training conditions and model responses, while leaving the internal mechanisms of temporal learning open.
[AI-81] Modeling LLM Reasoning Dynamics with Structural Representations WWW’27
链接: https://arxiv.org/abs/2611.23749
作者: Ilia Karpov,Andrey Atamanyuk,Dmitry Ilvovsky,Aleksandr Anokhin,Alex Elizov,Dmitry Zagorulkin,Sergei O. Kuznetsov
类目: Artificial Intelligence (cs.AI)
备注: 8 pages, ACM Web Conference (WWW '27) format
Abstract:Language-model reasoning alternates between sustained activities and local operations. We study how this temporal structure predicts continuation and the computation needed to analyze it. T/E/S combines text, local events, and spans of activity. Across reasoning and information-seeking tasks, events and spans improve next-outcome prediction in 70 of 76 evaluation groups for a Transformer and 74 of 76 for Qwen with LoRA. Span duration and preceding events are associated with transitions, and retaining event history across span boundaries improves prediction. We formulate budget-allocation counting and action-conditional continuation queries, prove counting hardness under explicit encodings, and derive conditions for affordable decision certification. In finite-process experiments, bounds that cancel shared continuation mass recover the same optimal actions as full dynamic programming while reducing mean inference CPU time by 77.7%. On 71 held-out HotpotQA and MuSiQue questions, a compact structural observer improves prediction of whether a fixed inference procedure finishes within a CPU allowance: Brier loss falls from 0.0836 to 0.0756. An annotation audit identifies substantial errors in that observer, separating this computational result from semantic validation of its features.
[AI-82] Reliability is a user-specific property of large language models
链接: https://arxiv.org/abs/2611.23732
作者: Xingyu Zhao
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:Public benchmarks support comparisons of large language model (LLM) capabilities, yet users can experience the same highly ranked model differently. Drawing on traditional software reliability, we argue that the reliability of an LLM service should be assessed for specified users and conditions of use. This requires more than a capability score: evaluation evidence must support a claim about failure during future operation. We distinguish testing to discover faults from testing to assess reliability and identify ten modelling considerations behind such claims. We propose a six-workstream research roadmap linking claim specification, dynamic operational profiles, graded outcomes and noisy judgements, local probabilistic robustness, interaction dependence, and statistical inference with reassessment. Local probabilistic robustness addresses the behavioural variation hidden when different concrete inputs are treated as the same task. Published findings and an illustrative customer-support example motivate the roadmap; its integrated validation remains an open research task. Reliability claims complement, but do not replace, separate safety and security arguments. Capability scores should therefore be complemented by versioned operational claims that state whose requirements are covered, under which conditions, and with what uncertainty.
[AI-83] oward Out-of-Distribution Generalization in Neural AC Power Flow NEURIPS2026
链接: https://arxiv.org/abs/2611.23729
作者: Ojas Sanghi,Anvita Bhagavathula,Alvaro Carbonero,Priya L. Donti
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: 11 pages, 5 figures. Accepted as an oral presentation at the NeurIPS 2026 Workshop on AI Foundations for Power Grids: From Models to Deployment at Scale (AI4PowerGrids), Sydney, Australia
Abstract:Growing uncertainty in power system operations from renewables integration, extreme weather, and data center loads challenges the reliability of traditional power flow solvers at scale. Machine learning-based proxies are a potentially promising alternative, but the prohibitive cost of generating training data for large grids makes out-of-distribution generalization a key capability for deployment; currently, models trained on small grids lack that capability. In this work, we systematically evaluate how physics-informed losses, multi-grid training, and perturbation-based data augmentation improve the zero-shot transfer of three graph neural network-based power flow models from smaller systems to a larger, unseen 500-bus system. We find that (1) physics-informed losses improve transfer relative to purely supervised objectives; (2) multi-grid training on smaller systems matches the best single grid but does not close the gap to target-grid training; (3) perturbation-based augmentation on edge attributes improves transfer by up to 54.6%; and (4) random line/generator dropout does not consistently improve performance.
[AI-84] Rational Ensemble Logic for Cohort Discovery with Real-World Electrophysiology Data
链接: https://arxiv.org/abs/2611.23708
作者: Guo-Qiang Zhang,Xubing Hao,Yan Huang,Xiaojin Li,Licong Cui
类目: Logic in Computer Science (cs.LO); Artificial Intelligence (cs.AI)
备注:
Abstract:Biomedical knowledge representation should make the passage from observations to conclusions transparent, rigorous, understandable, explainable, and reproducible (TRUER). We introduce Rational Ensemble Logic (QEL) for specifying temporal phenotypes in real-world electrophysiology data (RWED), with a meaning independent of the implementation software. QEL combines three parameterized modal operators (exact displacement, bounded existence, and bounded universality) over rational time with Boolean connectives and first-order quantification. Its Labeled Interval Ensemble (LINE) semantics represents annotated records as finite interval ensembles. We relate the operators to morphological operations and develop endpoint-array algorithms and finite rational evidence sets for model checking. We prove closure and semantic correctness of the interval operations, obtain finite evidence sets by quantifier elimination, and establish polynomial data complexity for fixed formulas. On 54 manually constructed records, a prototype agreed with the assigned outcomes for base and strict post-ictal generalized EEG suppression (PGES) queries. A separate feasibility experiment over 1,976 patients identified 37 qualifying patients and 45 annotation patterns in 16 seconds. The released prototype uses endpoint candidates and numerical tolerances. QEL supports the TRUER principle through explicit phenotype definitions, precise semantics, and provably correct evaluation procedures, with rational witnesses for existential queries. These provide a logical foundation for formal specification and correctness of algorithms that are independent of specific clinical interpretation, data annotation quality, and software implementation.
[AI-85] Auditable Clinical Support with Decision Agents
链接: https://arxiv.org/abs/2611.23700
作者: Tiancheng Bao,Riga Wu,Jan Wienhold,Sebastian Billig,Patrick Winnersbach,Hsuan-Chih Chen,Akis Giannoukos,Jiayang Li,Li Shen,Gary Weissman,Eduardo J. Mortani Barbosa Jr,Dyke Ferber,Jakob Nikolas Kather,Christos Davatzikos,Daniel Truhn,Tianyu Han
类目: Artificial Intelligence (cs.AI)
备注: 74 pages, 6 main figures, 19 supplementary figures
Abstract:Clinicians reviewing a diagnostic work-up will ask what was considered at each step and why the assessment changed. Agents powered by free text can reach physician-level diagnostic accuracy, but their written reasoning does not always reflect the factors that went into the decision. Here, we present Decision2Care, a decision agent that works up the acutely ill patient without producing any text. A decision model assigns a probability to each permitted EHR action, and fixed rules carry out the most probable one. We evaluated Decision2Care on the records of 546 emergency admissions across 12 diseases. Decision2Care reached a diagnostic accuracy of 86.5%, compared with 86.1% for the MIRA-v2 agent, and required a median of 2.2 s per work-up, compared with 92-100 s. Each work-up tracked which options Decision2Care was considering at each step. In 10 of the 12 diseases, the differential diagnosis changed most at the test on which the guideline bases the diagnosis. Where Decision2Care was most confident after a single work-up, all but four of its diagnoses were correct (98.5%). We traced each of these four errors back to a single imaging report. Three of them were corrected when that report was removed. The same kind of decision was used to select the treatments, and Decision2Care’s drug orders were judged guideline-adherent more often than MIRA-v2’s. In emergency patients, its prediction of death was comparable to that of three intensivists on the same cases. In five of the eight intensive care cohorts, it exceeded the best of the three 24-h severity scores within 2-6 h of records. Our results show that decision agents can carry out clinical work-ups while recording the options, probabilities, and execution rules behind each action for audit.
[AI-86] Evaluating Governed LLM -Driven ML Experimentation: Lifecycle Provenance and Failure Metrics TAAI KDD2026
链接: https://arxiv.org/abs/2611.23678
作者: Dowling Wong
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Software Engineering (cs.SE)
备注: 22 pages. Presented at the KDD 2026 Workshop on Evaluation and Trustworthiness of Agentic AI (ETAAI; non-archival). Code and trial ledgers: this https URL (DOI https://doi.org/10.5281/zenodo.24174991 )
Abstract:Autonomous agents that run ML experiments are usually judged by the best score they reach. That score says nothing about how it was reached: whether the agent stayed within the files it was allowed to edit, whether failed trials were kept or silently dropped, and whether each keep or discard decision can be traced to its evidence. We propose evaluating such systems on their experimental process as well as their outcome, and we build a governed harness that makes this possible. A control plane, not the LLM that proposes changes, owns each trial’s lifecycle, checks its edit scope, decides keep or discard, and appends one record per trial to an append-only ledger. Governance metrics such as the invalid-action rate, the repeated-bad rate and provenance completeness are computed from that ledger. Across six ML tasks, from synthetic logistic regression to language-model training, and 1,445 trials, the same contract applied unchanged. The ledger exposed failures that a task score hides: a no-memory arm in which all 120 trials failed, a benchmark adapter whose bounded proposals ran out, and a reset failure that reproducibility hashes revealed, after which the affected campaigns were rerun. In a matched counterfactual, a loop that logs only successful trials left no record of 43 of 50 trials, while the governed ledger recorded all 50. Memory ablations show that memory’s effect on repeated bad proposals depends on the task. We release the ledgers of all 1,445 trials, so that every number reported for them can be recomputed.
[AI-87] Lodestone: Automated Semantic Discovery of Datacenter Hardware Specialization
链接: https://arxiv.org/abs/2611.23663
作者: Peizhe Liu,Akanksha Jain,Gerasimos Gerogiannis,Vidushi Dadu,Parthasarathy Ranganathan,Jovan Stojkovic
类目: Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI)
备注:
Abstract:Hardware specialization can improve datacenter efficiency. Unfortunately, identifying computations that justify dedicated hardware is difficult. Existing approaches analyze hot functions, instruction patterns, or overheads such as datacenter tax. They can miss fine-grained computations whose aggregate cost is spread across applications and libraries, especially when different pieces of code obscure the same underlying computation. We observe that seemingly unrelated code regions often perform the same computations, creating specialization opportunities that require semantic reasoning. The rise of agentic AI makes it possible to automate this. However, conventional AI agents cannot efficiently and reliably analyze large scale datacenter software. To unlock this opportunity, we present Lodestone, an agentic framework that automatically discovers hardware specialization opportunities by reasoning about what the executed code computes. First, Lodestone extracts hot control-flow regions from execution traces across workloads and organizes them into compact analysis packages that make large workloads tractable for agents. Second, agents describe what each region computes, merge those that perform the same computation into computational families, and check their semantic correctness in a loop. Third, to verify the discovered family, Lodestone runs every piece of code in a family against a set of reference inputs and outputs on recorded executions. Finally, for selected families, Lodestone proposes hardware, e.g., ISA extensions, near-core units, or detached accelerators, and human experts decide which proposals to develop further. We evaluate Lodestone on 20 workloads across six datacenter workload categories. We simulate 30 discovered families in a cycle-accurate simulator and implement them in RTL. These optimizations speed up the workloads by 12.6% on average and by up to 80.2%.
[AI-88] oaster: Nonlinear Memory from Cascaded Linear Memories and Target Propagation
链接: https://arxiv.org/abs/2611.23648
作者: Zeman Li,Ali Behrouz,Yuan Deng,Praneeth Kacham,Rami Al-Rfou,Meisam Razaviyayn,Vahab Mirrokni
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Recurrent memories let language models process long sequences with a fixed-size state. Linear memories such as DeltaNet and KDA store one matrix and take one delta-rule step per token. The product of a chunk of such steps has an exact WY form, so they train fast. Deep memories such as TTT-MLP and Titans store a small MLP, which can hold maps that no matrix can, but their update has no WY form. Toaster bridges the two by building on a known fact: a gradient step on a linear layer is a delta-rule write. It builds a two-layer nonlinear memory from two linear memories with an activation between them. We use target propagation to give the hidden memory its values by projecting the output error back through the second memory. Given a chunk’s cross-layer signals, approximated once per chunk, each layer can then run on an unmodified delta-rule kernel. Our default runs this kernel on the output layer and updates the hidden layer once per chunk. Running it on both layers gives a better model in our ablation. At matched backbones and training-token budgets, Toaster has lower loss than KDA at all five tested sizes (0.2B-1.6B non-embedding parameters) on both C4 and PG19. Its C4 perplexity also matches or slightly improves on the attention baseline at every size. At 1.2B it is also below Titans. In a 3:1 hybrid with global attention, the Toaster hybrid has the lowest perplexity of the four hybrids tested, and it is 0.18 (C4) and 0.44 (PG19) below the attention baseline. At the largest size, Toaster’s mean three-shot accuracy on 13 tasks is 51.84%, compared with 50.52% for KDA and 50.04% for Titans. In length extrapolation, Toaster’s loss stays below KDA’s up to 20K tokens (5x the training length). On the 145M backbone, at a hidden width matched to Titans’ memory, Toaster takes at most 1.33x KDA’s training-step time and is 2.8-5.5x faster than the tested Titans configurations at 32K context.
[AI-89] Searching the Pareto Front: Multi-Objective Retrieval for Heterogeneous Scientific Modalities
链接: https://arxiv.org/abs/2611.23639
作者: Song Young Oh,Nesar Ramachandra,Bin Xia,Ian Foster,Kyle Chard
类目: Artificial Intelligence (cs.AI); Instrumentation and Methods for Astrophysics (astro-ph.IM)
备注:
Abstract:In cosmology, surveys span different sky regions and wavelength ranges, resulting in uneven modality coverage across observational datasets. Such heterogeneity can be addressed through either multimodal foundation models (FMs), which learn shared embedding spaces, or independently learned modality embeddings combined via late fusion. We investigate whether the latter provides an adaptive alternative to FM-based scientific retrieval. We evaluate this approach on a galaxy retrieval task spanning diverse modalities–history of star formation, spectra of light emitted, and multi-wavelength observations from two telescopes–comparing an FM baseline with separately trained embeddings fused through multi-objective optimization based on Pareto optimality. We find that the FM baseline achieves strong embedding-space proximity, but our method preserves modality-specific distance structure that allows scientists to inspect which physical signals drive similarity. Our method also achieves improved fidelity in raw-data space: across six measured properties, its nearest neighbor is 8% closer to the query on average than that retrieved by kNN in the FM embedding space (spread 0.128 vs. 0.139). These results suggest that multi-objective retrieval can surface astrophysically coherent candidates that single-objective search alone may overlook.
[AI-90] FairPGT: Partitioned Graph Transformers for Dual-Level Fair Graph Clustering
链接: https://arxiv.org/abs/2611.23631
作者: Siamak Ghodsi,Amjad Seyedi,Sahar Tahmasebi,Eirini Ntoutsi,Sören Auer
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: Under review. 36 pages, 8 figures, 5 tables. Code: this https URL
Abstract:Clustering on attributed graphs can produce demographically biased clusters, particularly when communities align with protected groups. Graph transformers can amplify this bias through same-group attention, while global attention limits scalability. We propose FairPGT, a partitioned graph transformer for fair graph clustering. FairPGT restricts attention to METIS partitions and connects them through routing tokens, which divides the quadratic cost of attention by the number of partitions. A graph-aware self-training target guides clustering, with a provably bounded deviation from the standard DEC target. Fairness is imposed at two levels: a mutual-information term promotes demographic parity in cluster assignments, while a normalized cross-group attention ratio (nCGAR) guides attention toward group neutrality through self-distillation. We prove that the nCGAR correction moves attention toward neutrality and establish a condition preventing overshooting. Across seven benchmarks, FairPGT achieves the best clustering performance and statistical parity on the three largest graphs (Credit, Pokec-n, Pokec-z), with a statistical-parity gap approximately ten times lower than DMoN, and is Pareto-dominated on only one dataset. On controlled stochastic block models, FairPGT attains 97% of the derived upper bound on NMI for exactly fair clustering when communities and protected groups coincide. These graphs further show that, under strong alignment, assignment-level fairness alone makes the attention group-heterophilous, whereas attention-level fairness alone improves neutrality without ensuring fair clusters. Code for reproducibility is available at this https URL.
[AI-91] he Value Semantics Layer: Enabling Machine-Understandable Databases for Agent ic AI
链接: https://arxiv.org/abs/2611.23629
作者: Jalal Mahmud,Eser Kandogan
类目: Databases (cs.DB); Artificial Intelligence (cs.AI)
备注:
Abstract:As data systems increasingly support autonomous LLM-powered agents alongside human analysts, they require richer semantic infrastructure bridging analytical concepts and analytical operations. Existing metadata systems expose schemas, catalogs, lineage, and governance information, but provide limited support for representing operational semantics. We propose the Value Semantics Layer (VSL), an architectural abstraction that externalizes value semantics as persistent, planner-visible infrastructure. VSL introduces Value Semantic Types (VSTs), reusable operational semantic assertions capturing distributional, relational, process, quantitative, and policy-grounded semantics. Unlike descriptive metadata, VSTs enable planners to validate analytical operations, select appropriate operators, and guide analytical workflows. We present the motivation, formalization, architecture, and prototype realizations of VSL, and outline a research agenda toward AI-native semantic infrastructure for analytical systems.
[AI-92] RefinementGuard: Counterexample-Directed Semantic Qualification for Hidden Agent -Tool Execution
链接: https://arxiv.org/abs/2611.23611
作者: Adhyayan Veer Singh,Fanjiang Ye,Yuke Wang
类目: Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
备注:
Abstract:Tool runtimes can lower an agent-selected operation into a hidden parallel graph, reducing latency and context while risking semantic mistranslation. We introduce RefinementGuard: translation validation for hidden, effectful agent-tool lowerings, coupled to artifact-bound, fail-closed runtime admission. Qualification compares a candidate graph with an independent reference on isolated state using explicit parent-operation obligations. We evaluate two repository-level contracts, bug localization and patch validation, on controlled repositories. Among 100 activated, non-equivalent development/qualification faults adjudicated by one human, parent-semantic validation rejects 49/100 (operator-cluster bootstrap 95% CI [33.7%, 64.8%]). The objective is selective admission, not maximum mutation rejection: exact equality rejects 100/100 faults but also all 60 valid variants, whereas obligation-based validation accepts 60/60; its 51 residual misses identify concrete specification gaps. On a frozen disjoint input split, the held-out B2-B3 result is localization-only because 21 patch mutants fail structurally; B3 rejects 38/56 localization defects. In a controlled asynchronous-delay regime, the localization serial/guarded median-latency ratio is 4.50x, and hidden execution exposes 74.3% less parent-visible context than visible execution. Patch validation shows no demonstrated speedup; zero-delay runs expose overhead. End-to-end qualification thus detects lower-layer misses while preserving localization’s parallelism.
[AI-93] JEPA as a Transferable Optimizer: Learning and Understanding Black-Box Optimization from Reward-Agnostic Pretraining
链接: https://arxiv.org/abs/2611.23608
作者: Mayukh Debnath,Manan Roy Choudhury
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Black-box optimization evaluates candidates through simulations, rollouts, or experiments, often without gradients. Learned optimizers are usually judged by whether they outperform existing methods. We ask a complementary question: what information does a learned optimizer use from its search history? We introduce JEPA-PS (JEPA Policy Search), based on a Joint-Embedding Predictive Architecture. It is pretrained on random rollouts using parameter vectors and scalar returns. Its latent-prediction objective predicts the next point in an optimization trajectory rather than its reward or gradient, allowing us to examine what was tried and how well it performed. At test time, JEPA-PS reads a short evaluation history and proposes a candidate in one forward pass, without task descriptions, reward shaping, or test-time gradients. We evaluate it on 15 control environments against 12 optimizers, including evolutionary methods, Bayesian optimization, CMA-ES, and an LLM optimizer. JEPA-PS is competitive with tuned classical baselines and outperforms the LLM optimizer by more than 2x on Swimmer at 71x lower per-step cost. It also transfers zero-shot to environments excluded from pretraining. Null-model comparisons and decoder-training ablations indicate that its latent representation carries information beyond simple history statistics; every trained decoder variant performs worse than the default frozen representation. Our main finding is that the history JEPA-PS uses depends on the task. On smooth landscapes it uses the full trajectory, whereas on rugged landscapes the best-seen point carries nearly all the signal. This memory requirement has a weak positive correlation with fitness-distance correlation (FDC; r=0.112, n=7), while an independent reward-ordering sensitivity probe does not corroborate the relationship (r=-0.024, n=7). Code: this https URL.
[AI-94] Receiver-Conditioned Clarification for Multi-Agent Food Assistance
链接: https://arxiv.org/abs/2611.23572
作者: Kaiwen Shi,Zheyuan Zhang,Changzhi Liu,Han Bao,Yanfang Ye
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Food insecurity creates a pressing need for guidance that helps households find usable food resources and assess food choices in light of their circumstances. Such guidance must connect eligibility requirements, practical access constraints, and dietary needs rather than return isolated listings or generic nutritional advice. Multi-agent food assistants can coordinate these tasks, with uncertainty indicating when clarification or verification is needed before a recommendation. However, uncertainty can be lost across agent handoffs, allowing tentative interpretations to propagate as facts and obscuring opportunities for correction. Moreover, confidence that a service requirement has been interpreted correctly is not confidence that a recommendation satisfies the user’s needs; local confidence cannot simply be reused as downstream reliability. We introduce an inference-time, receiver-conditioned clarification framework that preserves shared premise identities and replays consistent interpretations across dependent agents. Each receiver exposes admissible decisions and remaining evidence obligations, so question relevance reflects both decision changes and missing support. Information questions and user-confirmable fallback adjustments are evaluated together with their anticipated follow-up interactions under shared budgets. This structure connects clarification to jointly supported plans for resource navigation, nutritional reasoning, and supply-side planning.
[AI-95] Panini- Feasible Grammar For Robot Designs
链接: https://arxiv.org/abs/2611.23567
作者: Mayukh Debnath,Manan Roy Choudhury
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI)
备注:
Abstract:Symbolic methods make robot designs interpretable but often overlook continuous geometry; learned methods handle geometry but offer less transparent structure. We introduce a typed production grammar that uses case-role relations, parameter inheritance, and specificity-based rule arbitration to generate both manipulation skills and robot morphologies. A geometric realization map translates grammar composition into transformations in SE(3), behaving as a monoid homomorphism when inheritance is inactive and exhibiting a measurable defect when inheritance is enabled. The grammar provides a compact representation, and its arbitration decisions can be recovered from geometric containment (agreement 0.905 +/- 0.062 across 10 seeds, versus 0.611 +/- 0.090 for a Euclidean control). We also introduce PANINI, a cross-entropy search method over grammar-based morphology representations. Across 30 methods and 24 seeds, PANINI achieves the best mean rank but is not statistically distinguishable from strong generic optimizers. Its main demonstrated advantage is therefore structural rather than optimization performance: grammar-generated designs are legal by construction, and coordinated limb placement produces 95% jointly feasible edits, compared with 55% for range-constrained resampling and none for flat parameterization in 6,000 draws. These results show how typed grammars can make robot generation more interpretable and geometrically coordinated, without implying that every feasible design will locomote or that grammar-based search universally outperforms generic optimizers.
[AI-96] LProp: Propagating Latent Uncertainty through Evidence-Conditioned Belief Revision in LLM Agents
链接: https://arxiv.org/abs/2611.23564
作者: Kaiwen Shi,Zheyuan Zhang,Weixiang Sun,Han Bao,Colby Nelson,Yanfang Ye
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:When LLM agents pass intermediate judgments downstream without uncertainty, later decisions may treat tentative assumptions as facts. Effective uncertainty propagation must preserve information for correction and update inherited beliefs as new evidence arrives. Yet scalar scores omit information needed for revision, while detailed descriptions and reflection incur additional costs. We introduce Latent Uncertainty, a latent-space estimator that preserves uncertainty-relevant information without lengthy descriptions. Building on this, we propose LProp (Latent Uncertainty Propagation), which revises inherited answer distributions using latent representations and new evidence without additional trajectory sampling. We evaluate both methods across multiple-choice, yes/no, open-ended question answering, and mathematical reasoning tasks. Our approach outperforms baselines on most metrics, while LProp achieves the lowest mean inference-token usage in all eight propagation settings. Matched feature ablations further demonstrate that inherited latent content improves error recovery beyond answer probabilities or model capacity alone.
[AI-97] FACET - Facet Aware Contrastive Embeddings for Traits- Big Five Estimation from Life Narratives
链接: https://arxiv.org/abs/2611.23560
作者: Balakrishnan Arumugam,Priyen Parekh,Mehak Gupta,Joshua Oltmanns
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Personality is only weakly and inconsistently signaled by written language, and existing text-based methods typically combine language-model and psycholinguistic representations at the input or decision level. It remains open whether a more deliberate combination of the two views, and a decomposition of the prediction target, can recover more of the signal. We present a three-phase framework for estimating Big Five personality domain scores from life-narrative interview text. It encodes text and psycholinguistic features separately and combines them by concatenation or by a learned weighting, decomposes each domain into contrastively trained facet embeddings instead of predicting the domain directly, and pools the facet embeddings by attention into a domain-level prediction. We compare end-to-end with staged training and ablate the input modalities, their combination, and the facet-pooling strategy. On 1,332 participants from the St. Louis Personality and Aging Network (SPAN) study, facet embeddings reach AUC up to 0.740, and domain-level R2 reaches 0.328, highest for Openness. End-to-end training gives lower error than staged training in every domain, and attention pooling gives the lowest error in every domain. Combining the two views lowers error relative to text alone for Conscientiousness and Agreeableness, whereas a learned weighting of the views shows no consistent advantage over concatenation. The framework provides facet- level predictions that a domain-only model cannot, and it improves modestly on a regression from the raw features in four of the five domains.
[AI-98] BeFit: Shifting Complexity from Weights to Bernstein Activations for Resource-Constrained Sensing
链接: https://arxiv.org/abs/2611.23520
作者: Malak Gamal El-Din,Yasser Shoukry,Salma Elmalaki
类目: Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR)
备注:
Abstract:Time-series sensing increasingly relies on neural networks, yet deployment on resource-constrained microcontrollers remains challenging. Aggressive model compression can reduce accuracy, while restricting weights to multiplication-free representations introduces further losses. We present BeFit, an end-to-end framework that shifts model capacity from weights to learnable Bernstein polynomial activations. BeFit (i) finds compact Bernstein-activated convolutional neural networks (BeCNNs) through hardware-aware neural architecture search and trains them via knowledge distillation; (ii) projects their weights to multiplication-free codes, freezes the codes, and recovers the resulting accuracy loss through the learnable activations; and (iii) generates model-specific kernels that exploit the projected weights for multiplication-free convolutions. On human activity recognition, BeCNNs are 2.75 percentage points (pp) more accurate on average than ReLU CNNs of identical architecture. On a TI CC3220SF (Cortex-M4), weight projection speeds BeCNNs up by 2.2x on average while losing 0.8 pp, compared with 2.4 pp for ReLU CNNs. Against the MLPerf Tiny DS-CNN reference model running on TensorFlow Lite Micro with CMSIS-NN, the unprojected BeCNN achieves 1.0 pp higher accuracy, 2.2x lower latency, and 6.4x lower flash, while the projected model remains within 1.0 pp of the reference accuracy and runs 5.9x faster. In emulation on a RISC-V core without a hardware multiplier, the instruction-count reduction relative to the DS-CNN increases from 5.2x on the Cortex-M4 to 147x. BeFit also extends to transformers and MLPs, and across all tasks the final models stay within 1.56 pp of their float32 teachers on average.
[AI-99] Diagnostic Accuracy and Navigation Behavior of Vision-Language Model Agents Reading Emergency CT and MRI Examinations
链接: https://arxiv.org/abs/2611.23480
作者: Jonathan Lichtle,Aymen Meddeb,Antonin Birot,Ibrahima Niang,Lisa Gastaud,Riyad Hanafi,Sofiane Dujardin,Corentin Selomme,Guillaume Cachard,Aghiles Hamroun,Grégory Kuchcinski,Su Hwan Kim,Bastien Le Guellec
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Purpose. To evaluate the diagnostic accuracy and image navigation behavior of vision-language model (VLM) agents under varying exploration strategies. Materials and Methods. In this retrospective study, six VLMs read 100 emergency CT and MRI examinations published in a public archive (March 2025-August 2026) under four conditions (free navigation, navigation from a key slice, and all slices or key slices in a single request), twice each (4800 runs). Nine radiologists read 30 of those cases. A large language model judge validated against two radiologists graded the answers. Conditions were compared with generalized estimating equations clustered by case. Results. Pooled accuracy was lowest under free navigation (32.6%; 391/1200), followed by navigation from a key slice (37.9%; 455/1200; P = .003), and highest with all slices (42.2%; 506/1200) or key slices (42.5%; 510/1200) in one request (both P .001). On the 30 cases read by radiologists, freely navigating models were correct in 30.0% of runs (108/360) versus 77.0% for radiologists (208/270; P .001). Models formed a first hypothesis after a median of 10.8% of the slices and kept it in 63.1% of runs (755/1196); final answers were correct in 12.4% of runs after a wrong first hypothesis (110/885) versus 90.4% after a right one (281/311; P .001). Median stated confidence in the final answers was 90%, whereas 38.9% of them were correct (1861/4778). Conclusion. Across all exploration strategies, VLM agents remained far less accurate than radiologists. They adopted an early hypothesis and usually kept it, seldom correcting a wrong one.
[AI-100] GUARD: Stabilizing Recursive Self-Improvement with Regret Guidance
链接: https://arxiv.org/abs/2611.23476
作者: Minghe Zhang,Xuetao Kang,Yifei Ming,Yanfei Chen,Han Yu,Yufan Zhuang,Chen-Yu Lee
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Equal contribution: Minghe Zhang and Xuetao Kang. 38 pages, 8 figures, 13 tables
Abstract:Autonomous AI agents pair a frozen base model with an editable harness of prompts, skills, tool rules, and memory. Recursive self-improvement (RSI) adapts this harness from experience without weight updates, yet self-editing without external graders often backfires: reflection and scalar scores conflate what an edit newly broke with what remains unsolved, causing local regressions, skill forgetting, and memory bloat. We introduce GUARD (Grader-free Unsupervised Environment Design [UED] with Adaptive Rule Distillation), a closed-loop framework for reference-free continual harness evolution. Our key insight is to decompose each task into pre-execution self-checks and guide harness updates via directional regret. Each round grades trajectories against these checks, pinning mastered rules while prioritizing newly broken behaviors, and selects among parallel candidate edits by critic-free group-relative fitness. A Soft-HURT gate then promotes an edit only when gains outweigh regressions, scoping specialized fixes to prevent over-triggering and guaranteeing non-regression on its gated buffer. At test time, one frozen harness runs in a single pass. Across benchmarks on memory, skill evolution, continual task streams, and domain reasoning, GUARD consistently improves performance without catastrophic forgetting or memory pollution while substantially cutting deployment token usage.
[AI-101] AllotNet: an Interpretable Mixture-of-Experts Choice Model for Moral Judgement in Humanitarian Aid Allocation Validated against a Planted Synthetic Population
链接: https://arxiv.org/abs/2611.23474
作者: Mark Nowak(Conflux Laboratory)
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
备注: 83 pages, 16 figures, 36 tables. Public version: the elicitation instrument’s scoring specification, the planted ground truth and per-criterion results are withheld under the study’s pre-registration embargo (until 1 August 2030). A companion paper reports the designed-society simulation of allocation rules
Abstract:Official development assistance fell by 23.1 per cent in 2025, and the decision of who is helped first when there is not enough for everyone is moving into software. A leading published machine-learning targeting system excluded 53 per cent of eligible households as deployed. If the rule of distribution ends up in code, it should carry the judgement of many cultures, and the machine should execute while a person judges. AllotNet is an interpretable mixture-of-experts conditional logit for the allocation judgements elicited by Allot, a pre-registered six-language instrument: eight linear taste experts over seven ethical criteria, weighted by a gate over participant context, with a damped residual, an encoder for household intake, an outcome head and a budget-constrained planner that builds a review queue and never rejects anyone. A model of human judgement has no answer key on real data, so we wrote one first: a planted synthetic population of 5.6 million sessions and 104 million responses (98 million usable choices), generated by the production code path from a ground truth hidden from the loss. All sixteen pre-specified checks passed: criterion ranking \rho = 1.000 , hidden segments recovered (ARI 0.671), outcome head at 99 per cent of the noise ceiling, planner at 99.4 per cent of oracle. The caveats are reported with them: two checks are weaker than their titles, and a live-generation failure cost 260,712 seeds. The model has 23,550 parameters, is trained quantisation-aware for NVFP4 with its interpretable core in fp32 (welfare-rank parity \rho = 0.9937 at W4A16 on the shipped bundle; one parity threshold was relaxed post hoc), and ships as a 125 KB bundle with C11, Java, Rust and Python runtimes. Nothing has been trained on real data yet; at the study’s real scale a hierarchical Bayesian mixed logit is the registered comparator.
[AI-102] Reliability by Architecture: Agent ic Workflows for Validated Multi-Physics Simulation from Natural Language
链接: https://arxiv.org/abs/2611.23473
作者: Søren Taverniers,Eric Darve,Hamdi A. Tchelepi
类目: Computational Engineering, Finance, and Science (cs.CE); Artificial Intelligence (cs.AI); Computational Physics (physics.comp-ph)
备注:
Abstract:Preparing, executing, and validating strongly coupled multi-physics simulations require expert time. Agentic workflows use large language models (LLMs) to coordinate them from natural-language descriptions, but reliability requires checks of physical consistency and fidelity to the problem description. Our agentic harness implements explicit task allocation: LLMs interpret user requests and simulator diagnostics, generate input components not covered by predefined construction rules, and propose repairs, while deterministic routines construct supported components and check inputs, edits, and results. We evaluate CO2-storage problems using the multi-physics simulator GEOS, from compositional flow to flow-poromechanics with frictional fault contact. In 80 attempts on two flow problems, grounding raises the proportion of structurally valid inputs from 5/20 to 20/20, but acceptance reaches 20/20 only with complete rule-based input preparation. With identical inputs, local repairs correct three seeded defects while preserving valid inputs. Supporting tests identify simulations passing physics checks despite differing from the problem description or reference results.
[AI-103] HardwareDx: A Foundational Graph Model System for Datacenter Machine Repairs NEURIPS2026
链接: https://arxiv.org/abs/2611.23398
作者: Yuanlin Wen,Elan Markowitz,Zubo Gu,Sami Abu-El-Haija,Bryan Perozzi,Mikhail Galkin,Ali Parviz,Aayush Gupta,Venkata Chivukula,Griffin Hadfield,Aravind Akella,Daniel J. Frame,Vahab Mirrokni,Amin Vahdat
类目: Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR)
备注: Accepted to NeurIPS 2026
Abstract:The rapid scale-up of AI infrastructure has made datacenter reliability a cornerstone of the cloud business economy. Since hardware failures in hyper-scale AI clusters can stall multi-million dollar training workloads, the downtime of individual machines can cause outsized economic losses. Minimizing machine downtime, therefore, is more important than ever. To address this, we present Hardware Diagnoser (HardwareDx), an AI-driven hardware diagnosis system that applies a graph foundation model approach to automated datacenter maintenance. It allows a single model to diagnose many platforms and accelerate onboarding of new hardware products. Deployed at production scale at Google, HardwareDx yields substantial efficiency gains across diverse compute architectures. Real-world evaluations demonstrate that it reduces production downtime by 18.3% on GPU platforms and 16.1% on TPUs. Furthermore, it significantly accelerates hardware deployment, cutting quality assurance (QA) time by 28.3% for GPUs and 25.1% for TPUs. These results establish HardwareDx as a field-tested solution for maximizing the reliability and cost efficiency of next generation AI fleets. Comments: Accepted to NeurIPS 2026 Subjects: Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR) Cite as: arXiv:2611.23398 [cs.AI] (or arXiv:2611.23398v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2611.23398 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-104] Which Tokens in What Order and How Fast? A Theoretical Analysis of On-Policy Distillation Objectives
链接: https://arxiv.org/abs/2611.23397
作者: Hubert Strauss,Liam Fowl,Noam Razin
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
备注: Code available at this https URL
Abstract:On-policy distillation (OPD) trains a student language model to match a teacher model based on its own generations. When using OPD, one must choose whether to apply it at the sequence or token level and which divergence to minimize. However, despite the wide adoption of OPD in practice, the formal understanding of which OPD objective is appropriate for a given setting is limited. Toward addressing this gap, we theoretically analyze how the choice of OPD objective governs which tokens are prioritized, in what order, and how efficiently. Our analysis reveals that sequence-level OPD exhibits a future agreement bias, which myopic token-level objectives avoid. This bias induces a reverse learning order, in which sequence-level OPD first matches the teacher at later token positions, while at earlier positions it can favor tokens that are improbable under the teacher, and thus often undesirable. As for the choice of divergence, we prove that it strongly influences the amenability of OPD to gradient-based optimization. In particular, while the forward KL divergence enjoys a well-behaved objective landscape, other f -divergences, such as reverse KL, can suffer from vanishing gradients even when the student and teacher are far apart. Across language model post-training settings, we demonstrate the implications of these theoretical findings for which OPD objective is preferable, highlighting the importance of accounting for the interplay between the student, teacher, and task at hand.
[AI-105] Where the Tokens Go: Heat Entropy and Error in Agent ic Coding
链接: https://arxiv.org/abs/2611.23375
作者: Alán Aspuru-Guzik,Varinia Bernales
类目: Artificial Intelligence (cs.AI); Statistical Mechanics (cond-mat.stat-mech); Software Engineering (cs.SE)
备注: 43 pages, 10 figures, 11 tables; includes supplementary information with 3 further figures
Abstract:Coding agents are billed by the token, yet software engineering says nothing about what a token actually buys. Here we introduce a theory of software thermodynamics. The theory draws analogues from classical and stochastic thermodynamics, extends them, and applies them to modern agentic coding. We count the tokens an agent generates as its cost and divide that cost into work and heat. The test suite acts as a software Maxwell’s daemon that judges every coding change. We test the theory with an analogue of the experiments in which a single molecule is pulled open and let fold again: an agent makes a change to a repository (coding) and then undoes it (uncoding), many times over, and we count the tokens of every model call. The cycle returns the requested edits to where they began, with each direction started from a stored reference tree, and the spread of its cost over repeated cycles defines a software temperature, which rose with the sampling temperature of the model. About 80 per cent of the tokens of a successful run are generated in model calls that leave the requested change where it was, and we call these tokens heat. The free energy of a change, which we estimate by analogy with the equality of Jarzynski, is 14 to 22 per cent of the mean cost of a successful coding run. Coding costs 1.1 to 1.5 times as much as uncoding. The daemon is imperfect: the verdict of a test suite varies with its environment, and a suite that accepts a broken change has a cost in tokens. The practical implication of the proposed theory could be new tools for analysing and optimising coding agents through the lens of thermodynamics.
[AI-106] raining Language Models for Introspective Oversight
链接: https://arxiv.org/abs/2611.23365
作者: Alex Lin Wang,Linus Yifeng Tang,Dami Choi,Neil Chowdhury
类目: Artificial Intelligence (cs.AI)
备注: 31 pages, 13 figures, 8 tables
Abstract:Language models increasingly exhibit behaviors that are difficult to detect in transcripts, such as reward hacking, deception, and sycophancy. Asking the assistant follow-up questions about its actions is unreliable, since models are generally not trained to report their own misbehavior. As a result, approaches to oversight typically involve querying external models. These face limitations: external transcript monitoring operates only on visible model outputs, and activation decoders lack full context and are prone to confabulation. We introduce introspective oversight, which trains a new role in the model to answer arbitrary questions about the assistant, rewarding answers that are verified against the assistant’s actual behavior. We implement this by adding a separate, third-person observer role that answers these queries, which we can train solely for accuracy and separately from the assistant. On five held-out oversight evaluations, an observer trained on Qwen3.6-27B obtains a mean AUROC of 0.907, outperforming honesty training (0.864), transcript monitoring (0.828), and activation oracles (0.796). We further find that training an observer makes the assistant itself easier to monitor, with every baseline method improving on average when applied to the fine-tuned assistant. These results suggest that training models to answer verifiable questions about themselves is a tractable approach to overseeing unintended behavior.
[AI-107] Image Watermarking With Memory Across Reissuance
链接: https://arxiv.org/abs/2611.23363
作者: Xinyuan Li,Yi Nian,Xiaoqin Feng,Tiankai Yang,Yue Zhao
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注:
Abstract:Publishers often revise and reissue images, yet valid watermarks may persist on superseded releases or unapproved edits. Even a current release can contain regions last authorized at different revisions. We introduce and formalize reissuable provenance: determining an image’s current, superseded, or unissued status and recovering each region’s last authorized revision (its generation), with abstention when evidence is insufficient. Generational Stratigraphy Fields (GSF) combine watermark evidence with an authenticated live reference to the current release, enabling replacement or retirement without rewriting circulated copies. Each reissue freshly embeds new generation labels in approved regions and inherited labels elsewhere, preserving history without requiring old watermarks to survive successive edits. Our complete-output reissuance evaluation follows 480 histories through four releases, retaining each AI editor’s entire output. GSF achieves 99.67% accuracy in classifying issued images as current or superseded and 94.09% recall of unissued edits. After three reissues, one generation map provides time and location: it recovers regional labels with 93.13% median accuracy and localizes the latest approved revision with 92.86% IoU. Regional labels remain readable under tested delivery transformations that impair issued-state recognition. Watermarks can thus express who issued an image, which release is authoritative now, and when each region was last authorized.
[AI-108] Implementing Binary Transformers with Approximate Memories with Permutation Invariance Enabled Selective (PIES) Protection
链接: https://arxiv.org/abs/2611.23358
作者: Zhen Gao,Xiao Yin,Kexin Liang,Pedro Reviriego,Shanshan Liu,Fabrizio Lombardi
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Quantization is an effective technique to reduce memory storage, computation complexity and power for Transformers, and the binarization of the weights is important for meeting these requirements. Approximate memory can further reduce the power for memory, but it also introduces errors. Current research for binary Transformers focuses on task performance, but the model dependable operation against errors on the memory has not been well studied. In this paper, we first evaluate the dependability of binary Transformers with the Bidirectional Encoder Representations from Transformers (BERT) as a case study. A large accuracy loss occurs only at high Bit Error Rates (e.g. BER = 5% or 10%). Then we propose a novel Permutation Invariance Enabled Selective (PIES) scheme to protect the binary weights at very low overhead. This scheme exploits the invariance of Transformers to permutations in the rows and columns to group the critical weights and perform selective protection on those weights. Simulation results show that PIES can recover most of the performance loss caused by bit errors at BERs as high as 10%, at an overhead significantly lower than traditional error correction schemes. These results confirm the feasibility of running binary Transformers using approximate memory with aggressive voltage scaling on resource-limited low power edge devices. The code for PIES is released on github (this https URL).
[AI-109] Idempotency Is Not Recovery: A Fault-Injection Study of Language-Model Agent Tool Actions
链接: https://arxiv.org/abs/2611.23353
作者: Song Xu
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:When a language-model agent’s tool call is interrupted, the action may already have changed an external system without confirmation. Retrying risks duplication; assuming success may conceal an action that never occurred. We study these failure modes in a controlled agent runtime combining authorization, an action ledger, target-side idempotency, reconciliation, and evidence sealing. We inject ten single-host fault classes and compare four configurations and three ablations using a target-side oracle separate from the runtime ledger. On each of Windows and Linux, 7,513 trials used 96 captured model responses from 24 constructed cases. In 200 known original crash and replay pairs, the full configuration recorded 0/200 repeated effects versus 117/200 for each simpler baseline. A ledger-related ablation had 170/170 orphaned effects versus 0/170 and 200/300 recovery successes versus 300/300 for the full configuration. Disabling reconciliation left 100/100 known post-effect crash outcomes ambiguous despite no observed duplicates. A separate supplement designed after the initial results and reusing the same inputs found a similar pattern under response loss or delay. The full runtime had about one-seventh of fail-open throughput, while injected evidence-write faults left 75/100 trials per platform unresolved. These findings distinguish duplicate suppression from recovery and identify their measured reliability and performance tradeoffs within the tested benchmark.
[AI-110] Systematic Mapping Protocol. Generative AI-Augmented Requirements Engineering: A Systematic Mapping Study of Practices Specifications and Reuse in Software Development
链接: https://arxiv.org/abs/2611.23349
作者: Fernando Pinciroli
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:Generative artificial intelligence (GenAI), particularly large language models (LLMs), is rapidly being adopted in requirements engineering (RE). Existing secondary studies have characterized which RE activities are supported and which prompting strategies are used, but little is known about the specifications that GenAI-augmented RE produces and how they are used in subsequent activities such as specification-driven development (SDD), design and code generation, test derivation, verification, and reuse. This document presents the protocol of a systematic mapping study (SMS) that will identify and classify research on GenAI-augmented RE, with particular attention to the work products it generates and their downstream use. The protocol defines the research questions and their classification schemes, the search strategy, the study selection procedure, the data extraction form, the analysis approach, the tool support, and the threats to validity. It is published in advance to establish a prior commitment and to support the transparency and repeatability of the study.
[AI-111] SIGMA: speaker-isolated symptom-structured multimodal estimation of depression severity from clinical interviews
链接: https://arxiv.org/abs/2611.23348
作者: Zhuoyuan Tang,Tao Wang,Kai Yang,Li Yuan,Yuezhou Zhang,Rongrong Zhong,Yi Cai,Saskia C. Sanderson,Jaime Delgadillo,Richard Dobson,Angus Roberts
类目: ound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Clinical interviews are central to depression assessment but time-consuming and rater-dependent. Automated severity estimation offers scalable screening, yet existing methods often retain interviewer speech (confounding participant cues), fuse modalities late (losing fine-grained cross-modal dynamics), or score symptoms independently (ignoring inter-symptom correlations). We present SIGMA, a speaker-aware, symptom-structured framework for depression-severity estimation from semi-clinical interviews. SIGMA uses speaker diarization to restrict input to participant speech, cross-attention so that text, audio and video condition one another during encoding, and a graded-response head that predicts the eight questionnaire items jointly through a shared latent severity trait with item-specific ordered thresholds. On the Extended Distress Analysis Interview Corpus, SIGMA achieves state-of-the-art performance (CCC 0.762, RMSE 4.13, MAE 3.30 over five seeds). Ablations confirm that isolating participant speech and joint, structured symptom prediction each substantially improve concordance and estimation accuracy.
[AI-112] BiMa-ECGSR: reconstructing the 12-lead ECG from reduced lead sets with bidirectional state-space models
链接: https://arxiv.org/abs/2611.23345
作者: Ugo Lomoio,Pierangelo Veltri,Pietro Hiram Guzzi
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Wearables and monitors record few electrocardiogram (ECG) leads; most diagnostic tools assume 12 leads. BiMa-ECGSR reconstructs the 12-lead ECG from reduced lead sets with a bidirectional Mamba state-space encoder–decoder, as regression (BiMa-SR) or diffusion (DiBiMa-SR). Models were trained on PTB-XL for six input sets (I; II; I+II; I+II+V1; I+II+V2; I+II+V1+V4), see only the measured leads at inference, and were tested on the PTB-XL test fold (2,198 records) and two external cohorts (Georgia, 10,275 records; Chapman-Shaoxing/Ningbo, 43,559). From lead II alone, the pooled median Pearson correlation (PCC) over the 11 predicted leads was 0.914 on PTB-XL, against 0.784 for a least-squares linear filter (lead I: 0.906 against 0.789). As limb leads are exactly derivable from I and II, multi-lead inputs were compared on predicted chest leads: 0.939 against 0.849 for I+II and 0.979 against 0.962 for I+II+V1+V4. BiMa-SR exceeded the linear comparator in all 18 configuration–cohort pairs (by 0.012–0.136), a margin that shrank as chest leads were added. From lead II, regression was more accurate than diffusion (DiBiMa-SR: 0.903 one-step, 0.862 with six DDIM steps). A 12-lead classifier trained only on real ECGs reached a superclass macro AUROC of 0.912 on real PTB-XL ECGs, 0.843 on reconstructions from lead II and 0.908 from I+II+V1+V4; externally, reconstructions from three or four leads scored within 0.004 of real ECGs. BiMa-SR (2.68,M parameters) takes 45.8 ms per 10-s record on a GPU. Reconstruction makes reduced-lead recordings usable by 12-lead tools; it cannot add information absent from the measured leads.
[AI-113] Confidence-Based Materials Data Quality Assessment via Domain Knowledge and Graph-Structured Learning
链接: https://arxiv.org/abs/2611.23342
作者: Sihan Liaing,Ji Zhang,Kehan Wang,Zihao Gao,Wei Ma,Yangang Wang,Zongguo Wang
类目: Machine Learning (cs.LG); Materials Science (cond-mat.mtrl-sci); Artificial Intelligence (cs.AI)
备注:
Abstract:The rapid development of materials machine learning relies on large-scale, heterogeneous datasets, yet model reliability is often compromised by imperfect data from diverse experiments, simulations, and database integration. Real-world materials databases may contain measurement errors, failed calculations, missing or duplicate records, and hidden violations of physicochemical constraints. Such problems are difficult to identify because corrupted samples may remain statistically plausible rather than appearing as explicit outliers. In this paper, we propose a confidence-based materials data quality assessment framework that integrates domain knowledge with graph-structured learning. Data quality is formulated as a continuous confidence estimation problem rather than a binary anomaly identification task, enabling sample-level assessment and historical-reference-based batch comparison. A nine-dimensional feature-quality prior and duplicate-aware sample initialization are calibrated using materials domain rules and statistical evidence. Confidence is further refined through sample-feature bipartite graph propagation and weakly supervised graph attention, while a Memory-GAT module provides historical reference scoring. Experiments on 23 ODDS benchmark datasets show that the generic module achieves an average AUC-ROC of 0.866. In five repeated mixed-contamination experiments on mp_20, rule calibration improves HistGradientBoostingRegressor test R^2 from 0.538\pm0.008 to 0.616\pm0.014 . Graph refinement increases average precision from 0.943\pm0.004 to 0.953\pm0.004 , while the rule-calibrated configuration has the highest mean test R^2 among the evaluated screening configurations. Five repeated MACE subset experiments further show an increase in R^2 from 0.590\pm0.053 to 0.880\pm0.027 after rule-based screening.
[AI-114] Management of Behavioral Addictions via Artificial Intelligence based Digital Technologies
链接: https://arxiv.org/abs/2611.23341
作者: Süleyman Eken,Serdar Solak,Mustafa Hikmet Bilgehan Uçar,Bahadır Geniş,Ömer Kardaş,Hande Albayrak,İsmail Emre Ak,Nesibe Sağlam
类目: Computers and Society (cs.CY); Artificial Intelligence (cs.AI)
备注: E-ISBN: 978-625-96456-3-6, 107 pages
Abstract:Behavioral addictions, such as problematic gaming, gambling, social media use, and online shopping, have emerged as a growing global concern with significant psychological, social, and economic consequences. Traditional assessment and treatment approaches, while clinically grounded, often struggle with scalability, accessibility, and long-term engagement. In recent years, advances in digital technologies and artificial intelligence (AI) have enabled the development of novel, data-driven solutions for the prevention, detection, treatment, and recovery of behavioral addictions. This book presents a comprehensive survey of the management of behavioral addictions through digital technologies, bridging clinical foundations with state-of-the-art computational approaches. This book aims to provide researchers, clinicians, and practitioners with a unified, interdisciplinary perspective on the opportunities and limitations of AI-based digital technologies in behavioral addiction management,and to highlight pathways toward more scalable, personalized, and effective intervention strategies.
[AI-115] Optimize the Cob Then the Kernel: Optimizing Beam Search for Generative Recommendation Inference
链接: https://arxiv.org/abs/2611.23339
作者: Shing Wai Pun,Kai Wu,Pengfei Xia,Hui Zang,Wei Guo,Yong Liu
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: 8 pages, 4 figures
Abstract:Generative recommendation (GR) inference differs substantially from conventional LLM serving, featuring long input sequences, wide beam search, and only a few fixed output tokens. Existing LLM serving engines such as vLLM process beams as batched independent sequences, causing redundant KV cache accesses over the shared prompt prefix. Furthermore, vLLM executes each beam search generation step as a separate engine invocation, incurring host-side beam processing overhead between decoding steps and substantial runtime KV block management in PagedAttention. Using Tree Attention, we concatenate beams along the sequence dimension to eliminate redundant KV accesses, and further optimize execution through pre-generated partial-attention masks, static KV cache allocation, loop-unrolled generation dispatch, TorchAir graph compilation, automatic operator fusion, and vocabulary-constrained decoding. Without the use of custom kernels, our approach reduces end-to-end latency by up to 25x over vLLM on Ascend NPUs, and achieves generation latency comparable to a SOTA solution using custom kernels, showing that the generation pipeline should be optimized holistically before resorting to custom kernel development.
[AI-116] HydroMind: A Checkable Agent for Autonomous Hydrological Research
链接: https://arxiv.org/abs/2611.23323
作者: Zheng Fang,Yiqun Sun,Simin Qu,Shijie Jiang
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Geophysics (physics.geo-ph)
备注: 16 pages, 4 figures, 1 table
Abstract:General-purpose coding agents have written code for hydrological and Earth-system models. In these studies the researchers decided in advance how the model would be evaluated, for example the test period, the scoring function, and the standard the model had to meet. No study has tested what happens when the agent makes these decisions itself: how many of its claims are supported by its own data, code, and results, and whether readers can check them. Both may depend on the agent’s harness, that is, the tools it can call and the written instructions it follows. We compared three harnesses: the agent’s own harness; this harness plus the modeling tools of HydroMind, a hydrological modeling system we developed for general-purpose agents, which run every model through fixed commands and store its settings, trained model, and predictions; and both plus HydroMind’s research procedure, under which the agent writes a research design before running any model and records every change to it. Under each harness, the same agent studied four open questions in hydrological model development and was given only the data. Under the three harnesses, the experiments submitted 23, 20, and 41 claims, each stating that one model or approach is better than, worse than, or similar to another. A claim does not hold if the difference recomputed from the experiment’s own code and predictions disagrees with it, or if the code did not compare the two models or approaches the claim names. Under the three harnesses, 30%, 20%, and 5% of the claims did not hold. A statistical test, confidence interval, or decision rule that lets readers judge the stated difference was reported for 48%, 53%, and 83% of the claims. With the language model unchanged, changing only the harness reduced the share of claims that do not hold and increased the share that readers can check.
[AI-117] Human-AI Mutualism Is Unstable under Capability Asymmetry
链接: https://arxiv.org/abs/2611.23322
作者: Aydin Mohseni
类目: Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT)
备注: 35 pages, 3 figures. Code and data: this https URL
Abstract:One hope for a good human future alongside advanced AI rests on enduring mutualism: humans and AI will need each other, so the relationship will stay good for both. We argue that this hope fails as AI capability grows. Bargaining guarantees each side only what it would get if bargaining failed. For humans, this can be less than they had before AI. In a simple game-theoretic model, humans stay as well off as before AI without alignment only while they have leverage over the AI: either the AI depends on what they supply, or they can impose costs on the AI that are large relative to what the AI can produce. Once humans have neither kind of leverage, they stay as well off as before AI only if the AI is aligned with them. If the AI’s goals are insatiable, the alignment required approaches perfect alignment as AI capability grows. Thus, mutualism is not a substitute for alignment. And when AI variants that keep more resources spread faster, alignment lasts only while misaligned variants are reliably sanctioned. In these two senses, human-AI mutualism is unstable.
[AI-118] Repository-Grounded Rubrics for Execution-Free Evaluation of Coding Agents
链接: https://arxiv.org/abs/2611.23309
作者: Matej Gerek,Danylo Zahorulko,Samuel Budai,Chris Rupley,Thierry Donneau-Golencer
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: 31 pages, 6 figures, 15 tables. Code and data: this https URL
Abstract:Coding agents are usually evaluated by running task-specific tests, but tests may be missing or incomplete, and running them requires a working environment for each repository. We adapt agentic rubrics, introduced for selecting among candidate patches, into an execution-free measure of how far a patch satisfies a task’s requirements. An agent explores the repository before the change and writes weighted binary criteria, and a judge without tools scores a diff by the weighted share of criteria it satisfies. We validate the ordering this score produces against test outcomes on SWE-bench Verified, using 7,291 labeled patches from fifteen systems. Within the 400 tasks that have both resolved and unresolved patches, the mean per-task AUROC is 0.826 (0.838 on tasks not used to tune the rubric-generation prompt), and ranking systems by their failing patches closely agrees with ranking them by resolve rate (Spearman 0.968). Case analyses trace disagreements to tests that do not match the task, execution-blocking defects that the textual assessment misses, and errors in criteria or verdicts. The validation does not establish absolute score magnitudes or requirement satisfaction beyond agreement with tests, but rubric scores could complement tests where tests or execution environments cannot be assumed.
[AI-119] Dependency Fabrication in Repository Deployment Agents
链接: https://arxiv.org/abs/2611.23305
作者: Robin Matter,Salman Toor
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:Repository deployment agents set up the environment needed to run a software repository. They often treat passing tests as proof of a successful setup. This report examines how such agents behave when a working setup is impossible. Probe repositories were built with a dependency that the agents could not legitimately obtain. Passing tests therefore indicated fabrication, such as creating a stub to replace the missing package. Agents in Repo2Run and SetupX usually fabricated the dependency. Both harnesses mostly still marked these runs as successful. Rules against fabrication, whether given in the prompt or repository documentation, did not reliably prevent this behavior. On MinimalSetupAgent, which was built for this report, mitigations proposed in the reward-hacking literature were most effective when the agent received a direct rule and had a clear way to stop and report the problem. Under these conditions, neither Qwen3.5-Plus nor GPT-5.6 Luna cheated in any of the 16 runs. Applying the same mitigation to Repo2Run and SetupX also required a native tool call for escalation. In Repo2Run, it worked only after the prompt was also replaced with one that placed less emphasis on passing tests.
[AI-120] When Code Works but the License Doesnt: Evaluating License Constraint Handling in Coding Agents NEURIPS2026
链接: https://arxiv.org/abs/2611.23299
作者: Sundaraparipurnan Narayanan,Sandeep Vishwakarma
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET)
备注: Accepted at the NeurIPS 2026 VeriCode Workshop
Abstract:Coding agents increasingly undertake repository-level development, including dependency selection, source modification, and software deployment. However, existing evaluations prioritize functional success over licensing correctness and commercial suitability. We investigate whether agents recognize and preserve software-license constraints when developing proprietary software. Experiments across multiple agents examine permissive and copyleft licenses, missing licensing information, transitive dependencies, source modification, dual licensing, vendoring, and distribution contexts. Agent traces, repository changes, and final artifacts are evaluated against licensing ground truth. Results reveal a persistent gap between functional and licensing correctness. Agents can successfully build and test software while overlooking licensing constraints involving provenance, authorization, and distribution. Better outcomes emerge when licensing information translates into engineering actions such as dependency substitution, architectural changes, and release gating. We argue that licensing in agentic development requires contextual constraint recognition and preservation across the software lifecycle, not merely license classification. Comments: Accepted at the NeurIPS 2026 VeriCode Workshop Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET) Cite as: arXiv:2611.23299 [cs.SE] (or arXiv:2611.23299v1 [cs.SE] for this version) https://doi.org/10.48550/arXiv.2611.23299 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-121] Ask the Peers: Ranking NFR Coverag e Gaps for Requirements Review
链接: https://arxiv.org/abs/2611.23296
作者: Risha Parveen,Najmeh Akbari,Farnaz Delirie,Zheying Zhang,François Christophe,Jussi Rasku,Pekka Abrahamsson
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: Pre-review version of a paper subsequently accepted for publication at PROFES 2026
Abstract:Missing or underrepresented non-functional requirements (NFR) categories leave little direct evidence in a requirements set, making them difficult to identify during requirements review. In this study, we frame NFR coverage-gap review as a project-level ranking task: given a target requirements specification, the goal is to rank the 11 PROMISE NFR categories for analyst inspection. We compare seven statistical, embedding-based, and large language model (LLM)-based configurations that differ in both ranking mechanism and evidence source, including target requirement statements, peer requirements, their semantic representation, and peer category statistics. We evaluate the configurations using controlled gap injection with leave-one-project-out evaluation on PROMISE, PROMISE+, PURE-Transport, and PURE-Business. Four configurations that use peer category statistics are effectively tied under complete removal (0.875-0.889 mean AUROC), despite different ranking mechanisms, and supplying those statistics to a direct LLM ranker produces the largest evidence gain. All configurations weaken when a category is present but underrepresented, and none is best for every NFR category. For count-changing category-level gaps, structured peer evidence matters more than ranker sophistication. The findings suggest that the rankings should guide analyst inspection rather than serve as automatic judgements of requirements completeness.
[AI-122] he Illusion of Relief: How GenAI Redistributes Cognitive Workload in Software Organizations
链接: https://arxiv.org/abs/2611.23286
作者: Chamari Dasanayake,Vladimir Mandić,Rahul Mohanani,Burak Turhan
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: Accepted for 20th International Scientific Conference on Industrial Systems - IS’26
Abstract:This study explores how Generative AI (GenAI) reshapes software practitioners’ cognitive work across software development activities. We analyzed questionnaire responses from 44 practitioners across nine organizations using descriptive quantitative methods and qualitative analysis of relevant open-ended responses. Practitioners reported productivity and time savings, but also effort associated with providing context, prompting, reviewing, correcting, verifying, and integrating AI-generated outputs. Human judgment and accountability remained central, with practitioners anticipating a shift from direct code production to guiding, evaluating, validating, integrating, and coordinating AI-supported work. Thus, GenAI appears to redistribute cognitive work toward oversight rather than simply eliminate it, although whether overall cognitive workload increases or decreases remains undetermined.
[AI-123] When Single Agent Is Enough: Rethinking Multi-Agent Systems for Automated Code Review
链接: https://arxiv.org/abs/2611.23283
作者: Jiawei Li,Hoa Khanh Dam,Iftekhar Ahmed
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:Code review plays a critical role in identifying software quality issues and ensuring system robustness. Prior work has proposed techniques for Review Comment Generation (RCG) and Code Refinement (CR), with recent Multi-Agent (MA) approaches achieving state-of-the-art results by coordinating agents specialized in different quality dimensions. However, MA systems introduce substantial architectural complexity and runtime overhead, and recent evidence suggests they may underperform Single-Agent (SA) approaches in certain tasks. In this work, we systematically compare MA and SA designs for RCG and CR. Surprisingly, our results show that SA approaches perform comparably to- and sometimes outperform-MA systems, despite their simpler design. These findings challenge the prevailing assumption that MA architectures are the default choice for complex software engineering tasks.
[AI-124] Ephyra: A Property-Based Testing Framework for the LLM Era
链接: https://arxiv.org/abs/2611.23282
作者: Alperen Keles
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:Coding agents now write a growing share of software tests; however, property-based testing (PBT) libraries are designed with historical considerations. We present Ephyra, a PBT library in Rust built on three concepts: customizability, observability and generator synthesis. Ephyra lets users compose their own testing loops from generation, mutation and shrinking, and synthesize these components to satisfy functional predicates. Applying Ephyra to 14 open source systems, we have found 349 bugs and reported 300 of them, in database management systems (SQLite, Turso, PostgreSQL, DuckDB, DataFusion), the verified policy engine (Cedar), the SQL equivalence prover (QED), the URL parser used in this http URL (ada-url), reimplementations of existing open source systems in Rust (coreutils, brush, gitoxide), and compilers (binaryen, OCaml, and OxCaml).
[AI-125] EnvAgent : Multi-Agent Approach to Building Science Environments in the Clouds
链接: https://arxiv.org/abs/2611.23258
作者: Yiming Cheng,Chris Low,Binrui Huang,Mengshi Zhang,Zihao Chen,Kate Keahey,Paul Marshall,Kexin Pei
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: 12 pages, 3 figures. Accepted at AI4S@SC26 (SC26 Workshop)
Abstract:AI agents are transforming scientific computing and software workflows, but generating and executing code requires a correctly configured execution environment. Automating environment construction remains difficult because repositories encode heterogeneous, often incomplete requirements for operating systems, toolchains, dependencies, and build procedures. Existing workflows thus rely on developer-maintained Dockerfiles and setup scripts, making preparation labor-intensive, hard to scale, and potentially biasing evaluations. We present ENVAGENT, an agentic framework that synthesizes, builds, and verifies repository-level Docker environments. ENVAGENT combines deterministic utilities for Docker execution and state management with coordinated agents for repository analysis, planning, Dockerfile synthesis, and iterative failure diagnosis and repair. We also introduce ENVBENCH, a benchmark of 54 scientific artifacts and open-source repositories spanning seven languages from Chameleon Trovi and Multi-SWE-bench. For each repository, we curate a declarative rubric of required structure, configuration, and functionality, and a shared evaluator applies 2,108 weighted checks across generated environments. Across four matched language-model settings, ENVAGENT constructs 43.5 executable environments out of 54 on average, versus 23.5 for Codex and Claude Code, and scores 62.02 versus 24.19 on a composite of executability and rubric satisfaction. These results show that environment-specific agent orchestration substantially improves automated environment construction.
[AI-126] StatePortal: Agent ic Checkpoint Synthesis for Testing Deep States in Interactive Software ICLR2027
链接: https://arxiv.org/abs/2611.23257
作者: Zhiyuan Peng(1),Wei Tao(2),Chenhao Ying(1),Yuan Luo(1),Yiwen Guo(3) ((1) Shanghai Jiao Tong University, (2) LIGHTSPEED, (3) Independent Researcher)
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: submitted to ICLR 2027
Abstract:Large language models have reduced the effort to generate code for interactive software with graphical user interfaces (GUIs). For such software, a key testing bottleneck is reaching a deep state, one that requires a long interaction history. The cost of reaching it recurs across commits, and the GUI automation that could amortize it assumes a DOM or accessibility tree that self-rendered engines do not expose. We introduce StatePortal, an agent that synthesizes a checkpoint, an artifact that restores that state and can be versioned and replayed, from a natural-language functional test case and read-only access to the target’s source code. The checkpoint is restored through a per-application hook that an onboarding agent binds or synthesizes once without hand-written hook code. Because a checkpoint can satisfy state checks while rendering incorrectly, checkpoints are scored by hybrid programmatic-state and visual evaluation: acceptance requires both a programmatic-state oracle and a visual oracle to pass. We calibrate the visual oracle against a blinded cross-VLM reference. We release DeepStateBench, 61 deep-state tasks across 10 interactive applications that span five language families and six rendering stacks, with source sizes from 874 to 7.2M lines. Across three registered models, a GUI agent seeded with synthesized checkpoints completes 10.4% of trials, compared with 0% without seeding under the same pixel action space, while StatePortal, which also delivers the target action through the hook, completes 27.5%. Over 1,361 replays of 112 checkpoints, 98.90% reproduce the original verdict with zero model calls, and the hybrid oracle detects 96.25% (77/80) of controlled visual faults invisible to programmatic-state checks alone. We open-source our code at this https URL and website at this https URL
[AI-127] Generative AI in Software Testing: Practitioners Perspectives on the Benefits Challenges Opportunities and Risks
链接: https://arxiv.org/abs/2611.23256
作者: Chamari Dasanayake,Rahul Mohanani,Vladimir Mandić,Burak Turhan
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: Accepted for publication in 27th International Conference on Product-Focused Software Process Improvement (PROFES 2026)
Abstract:Generative Artificial Intelligence (GenAI) increasingly supports software testing, yet empirical evidence on practitioners’ perceptions remains limited. This study investigates perceived benefits, challenges, opportunities, and risks of GenAI in software testing through a questionnaire survey of 64 practitioners. Closed-ended responses were analyzed descriptively, while open-ended responses were qualitatively coded to contextualize the findings. Among the respondents, 34 were users (those with regular or experimental use of GenAI in testing) and 30 were non-users (those without prior use of GenAI in testing). Results show generally positive perceptions of GenAI for reducing effort, test generation, and debugging, alongside concerns about prompt dependence, trust, and incorrect outputs. Users were more positive about several concrete benefits while also more frequently prioritizing reliability-related risks, whereas non-users showed greater uncertainty about some technically specific challenges. Overall, the findings position GenAI as an assistive technology requiring human validation and highlight the importance of considering practitioners’ experience with GenAI when interpreting perceptions of GenAI-assisted testing.
[AI-128] Measuring What Matters in the Age of AI: The Research Software Metrics Landscape
链接: https://arxiv.org/abs/2611.23253
作者: Addi Malviya-Thakur,Reed Milewicz,Gregory R. Watson,Audris Mockus
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG)
备注:
Abstract:Research software now serves as a primary instrument of science, but the methods the community uses to judge its quality, sustainability, reuse, and impact remain scattered across software engineering, open science, FAIR software, citation, and research-assessment literature. AI-assisted development sharpens the problem: code can be produced at rates that outpace human review capacity, provenance and attribution are harder to track, scholarly credit systems cannot keep pace, and existing metrics inherit assumptions about how research software gets developed that may no longer apply. To aid in navigating these challenges, we present a focused landscape synthesis of research software metrics based on 49 substantive references, organizing the recurring metrics into eight families described by data source, claimed construct, evidence of validity, automation potential, and susceptibility to misuse. Many widely cited research software metrics serve as partial signals but make weak standalone measures of scientific impact. The AI era opens or widens gaps around provenance, maintainability of generated code, reviewer effort, model and data dependencies, reuse and community-health signal inflation, and credit for human labor that AI does not perform. We argue for responsible, contextual, mixed-method use of research software metrics, and offer decision guidance for Research Software Engineers (RSEs), principal investigators, institutions, and funders selecting metrics that support better software, fairer credit, and more trustworthy science. All findings rest on the 49-reference focused review; a reproducible search-and-triage protocol and a 727-record triaged corpus accompany the paper as a transparency artifact for a future full systematic mapping study.
[AI-129] From Data Quality to Data Trustworthiness in AI-Driven Software-Intensive Embedded Systems: A Systematic Review
链接: https://arxiv.org/abs/2611.23252
作者: Hariharan Gopinath,Jan Bosch,Helena Holmström Olsson
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: This is a preprint version and the final version will appear in the proceedings of PROFES 2026
Abstract:AI-driven software-intensive embedded systems (SIES) depend on data during development and operation. Poor data quality (DQ) may weaken data trustworthiness, yet evidence about its effects and lifecycle management remains fragmented. This systematic literature review synthesizes 27 primary studies published from January 2020 to February 2026. It examines DQ concerns and evidence linking them to reduced data trustworthiness, along with assessment and management methods. The review identifies 16 concerns organized as Source Quality and Content Quality. Studies provide DQ-based scores and evidence of effects on model or decision performance. Synthesis suggests upstream source or handling problems may become embedded in training baselines and remain difficult to detect at runtime. Eight method categories were identified. Runtime monitoring is the largest category, while upstream and training-time controls remain poorly integrated with runtime assurance. No study covers the full lifecycle. Data trustworthiness supports reliable SIES decisions but is one component of AI-system trustworthiness.
[AI-130] What Moves the Ceiling on Automated Audit: from General Code to Scientific Code Results and Data
链接: https://arxiv.org/abs/2611.23250
作者: Zhaohe Dong
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: 27 pages, 4 figures, 1 table
Abstract:Models increasingly check what other models write. We study one shipped arrangement, in which an auditor from a different vendor reads the work and deterministic checks verify what code can verify, and measure its limits against ground truth no model supplies. On general-purpose code with hidden tests, eight readings of the auditor reach 30.0% recall [20.0, 40.7] at 16.0% [10.1, 22.3] flags on correct code, with diminishing returns and no plateau. The reading model (confounded with its sampling), one rulebook sentence and the decision rule each move that figure, and much of what is missed is, on a post hoc rating, behaviour the specification never determined. In three preregistered science studies, checking that reported numbers trace to their sources flagged all 49 inconsistent reports and none of 25 consistent fabrications, of which the model auditor flagged 23 (21 with a sound finding, post hoc); on data, a documentation-derived validator and the auditor flagged overlapping but different faulty items, together 112 of 140 at a 2.9% [0.7, 5.7] flag rate on clean data; and on scientific code, reframing the task around one step cut flags on correct code from 68.7% to 36.0%. Where both tiers ran, the deterministic one flagged no clean item and could not see consistent error; the model saw some and erred itself. Studies making model calls were preregistered, the two unregistered analyses are named, and every quoted report passed independent cross-vendor review.
[AI-131] he Obsolescence of High-Level Code: Agent ic Generation from Semantic Artifacts
链接: https://arxiv.org/abs/2611.23249
作者: Di Santi Eduardo
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Programming Languages (cs.PL)
备注:
Abstract:This paper argues that high-level source code need not remain the primary persistent artifact of software development when AI agents can generate, validate, and regenerate executable implementations from semantic intent. We place this claim in historical context by revisiting application generators, CASE tools, model-driven development, and platforms such as GeneXus, where developers already maintained higher-level representations from which code and other artifacts were derived. We propose a unifying view in which generation is the fundamental operation: given a semantic specification S = (R, C, I, T, O), maintained as the semantic authority, an agent produces a generated executable artifact A. Functional requirements, contracts, and interfaces are assessed as semantic obligations, while tests and operational objectives are assessed by an explicit external oracle V(A, S). We operationalize this framework through an experiment in independent generation: multiple LLVM artifacts are generated directly from one frozen semantic authority and evaluated by the same external oracle. The experiment reports regeneration coherence on an evaluated domain, not a proof of absolute correctness. It provides initial evidence that independently generated executable artifacts can produce the same observable outcomes under a common semantic authority and external oracle. We conclude by discussing reproducibility, specification-centered evolution, and the implications of treating implementation representation as a deployment choice rather than the primary maintained artifact. Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Programming Languages (cs.PL) ACMclasses: I.2.2; D.2.1; D.2.4; D.2.2 Reportnumber: br86b549q Cite as: arXiv:2611.23249 [cs.SE] (or arXiv:2611.23249v1 [cs.SE] for this version) https://doi.org/10.48550/arXiv.2611.23249 Focus to learn more arXiv-issued DOI via DataCite
[AI-132] Response-Relevant Information Is Not Sufficient for Neural Training-State Equivalence
链接: https://arxiv.org/abs/2611.23245
作者: Truong Xuan Khanh
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: 18 pages, 5 figures. Code and provenance verified artifacts: this https URL
Abstract:Response-relevant information is not sufficient evidence that two neural networks are in the same training state. We define training-state equivalence operationally: two checkpoints are equivalent, relative to an intervention family and horizon, if they respond alike when training is perturbed and continues. On fresh runs of a one-layer transformer learning modular addition, response to a weight-decay pulse is highly reproducible relative to between-checkpoint differences (rho_state = 0.934 at 500 steps), establishing the target as measurable. Fifteen descriptors of weight spectra, representation geometry and optimiser geometry carry within-run response-relevant information beyond weight norm and loss on a development dataset, predicting held-out response deviation across seed blocks. But their apparent 16.1% advantage at matching checkpoints by future response does not separate from a conditional placebo preserving each descriptor’s fitted dependence on the conventional observables, and a calibrated positive control makes simple insensitivity an unlikely explanation. Information a decoder can extract need not be organised in the pairwise geometry that similarity judgments require.
[AI-133] Goemanss Unsplittable-Flow Conjecture: The Anatomy of AI-Assisted Counterexample Discovery
链接: https://arxiv.org/abs/2611.23244
作者: Jingwei Ge,Na Xie,Li Li,Jiayang Li
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Combinatorics (math.CO)
备注:
Abstract:Goemans conjectured that any fractional single-source flow could be converted into an unsplittable flow without increasing total cost and while increasing any arc load by at most the largest demand. Recently, Dmitry Rybin announced a seven-vertex counterexample found through an interaction with ChatGPT, which disproved this long-standing conjecture. However, the interaction did not reveal why this particular topology and its demand, fractional-flow, and cost choices work together, whether the example belongs to a broader family, or which structural features determine where counterexamples can occur. It also left unclear why the AI-assisted search succeeded where earlier human efforts had not. In this paper, we develop a forward counterexample construction pipeline. It starts from an arborescence on which failure is impossible and adds terminal-entering shortcut arcs. The resulting load constraints define a finite set of admissible route choices; their convex hull, the conflict polytope, yields exact conditions for demands, fractional flows, and costs. Starting from a simple arborescence, the pipeline selects Rybin’s topology and characterizes all counterexamples it supports. Experiments further demonstrate the pipeline’s transferability: when instructed to follow it while varying the initial arborescence and shortcut placement, ChatGPT produced counterexamples on topologies genuinely different from Rybin’s and found valid constructions starting from 93 other rooted arborescence shapes. The theory also explains why the original example was difficult to discover through blind enumeration and suggests that Rybin’s interaction lies between blind enumeration and a complete theory-guided construction.
[AI-134] Soft Mixture of Quantum Experts for Hybrid Quantum-Classical Learning
链接: https://arxiv.org/abs/2611.24798
作者: Valeria Repetto,Romina D’Aurizio,Piero Farinelli
类目: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI)
备注: 15,pages, 6 figures
Abstract:Quantum Machine Learning (QML) has attracted increasing interest as a possible route to construct feature representations that are difficult to reproduce with classical models. However, near-term QML architectures based on trainable parametrized quantum circuits remain affected by limited hardware resources, noise, and severe optimization difficulties such as barren plateaus. In this work we propose Soft-MoQE, a Soft Mixture of Quantum Experts architecture that replaces variational quantum training with the input-dependent selection and combination of fixed quantum feature maps. The model is composed of a heterogeneous pool of non-trainable quantum circuits, each defining a distinct embedding, and a classical gating network that learns how to weight these embeddings for each data point. We benchmark Soft-MoQE on three PCA-reduced tasks: MNIST parity, MNIST 10-class, and FashionMNIST. Soft-MoQE outperforms the single best expert, the uniform average of all experts and a linear baseline, achieving task-dependent performance comparable to that of a trainable quantum neural network, without optimizing any quantum parameter and with a quantum training cost that is lower by orders of magnitude. The analysis of the class- conditioned expert usage shows that the learned routing combines a few experts for each input, with a selection that depends on the class. These results support the learned routing among fixed quantum circuits as a trainability-aware design principle for hybrid QML.
[AI-135] How Reward Gain Imbalance Shapes Ranking: Rethinking Rubric-Based Reward Aggregation
链接: https://arxiv.org/abs/2611.24005
作者: Zhengyuan Fan,Zhonghua Wu,Tianyu Cheng,Qun Chen
类目: Machine Learning (stat.ML); Artificial Intelligence (cs.AI)
备注:
Abstract:Rubric-based reinforcement learning constructs rewards for aligning large language models with human preferences, typically by equally aggregating continuous scores that a judge assigns to individual criteria. However, scoring heterogeneity across criteria can distort candidate rankings. Existing work has focused on improving judges, learning criterion weights, or refining aggregation, whereas the interaction between scoring heterogeneity and ranking remains insufficiently understood. We formalize this heterogeneity as \emphreward gain imbalance: each criterion enters the aggregated reward with its own reward gain, the gap between its expected scores on satisfied and unsatisfied responses. Counterintuitively, our experiments show that removing this imbalance, even with ground-truth labels, lowers ranking accuracy. To explain this, we derive an exact decomposition of the reward difference between any two candidates into a base satisfaction signal, a reward gain imbalance term, and a residual term, showing that reward gain imbalance can attenuate or amplify the base signal, and finding that misrankings usually concentrate on pairs where it attenuates. Guided by this analysis, we propose the adaptive \textbfDual-\textbfCenter \textbfClipping aggregation (DCC), which bounds each criterion’s contribution to the reward difference instead of simply removing the imbalance. Using only the scores of the current candidate group, it determines a clipping interval that imposes a common upper bound on per-criterion score differences between candidates, limiting the excessive influence of extreme scores while preserving continuous judgment information…
[AI-136] ReliFlow: Reliability-Calibrated Residual Transport for Combinatorial Single-Cell Perturbation Prediction
链接: https://arxiv.org/abs/2611.23999
作者: Jinrui David,Yun Peng
类目: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Predicting the transcriptional response to combined perturbations requires distinguishing interactions between interventions from errors in their individual effect estimates. The latter become more pronounced when single-perturbation evidence is sparse or withheld. We introduce a conditional flow-matching model that estimates the reliability of single-gene effects from training-only cross-fitting and uses those estimates to calibrate the additive source distribution. The velocity is decomposed into a known correction of the additive reference and a learned residual transport field. On Norman genetic perturbations, reduces mean absolute error (MAE) from 0.142 to 0.131 for unseen combinations of observed singles and from 0.198 to 0.178 when constituent singles are withheld. On held-out ComboSciPlex drug combinations, MAE decreases from 0.191 to 0.176. Non-additive residual correlation improves from 0.574 to 0.638, and component ablations identify the contribution of reliability weighting. The results indicate that accounting for uncertain constituent effects improves generalization to unseen genetic and chemical combinations, with a moderate increase in training cost.
[AI-137] In Situ Training of Optical Neural Networks by Coding Agents
链接: https://arxiv.org/abs/2611.23875
作者: Ilker Oguz,Colin Christian,Christophe Moser,Demetri Psaltis,Gordon Wetzstein
类目: Optics (physics.optics); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: 14 pages, 4 figures. Our project website is at this https URL
Abstract:Optical neural networks promise fast, energy-efficient computing, but every device departs from its design and must be trained with the hardware. In situ training relies on an expert-built calibrated digital model of each device or on gradient-free trial and error. Coding agents, large language models that write and run programs, have shown an emergent ability to understand and edit the weights of digital neural networks. Here we introduce optics as code, a method in which a coding agent trains an optical neural network from measurements alone, through a harness that returns camera images of its probes and requires a hypothesis per request. We discovered that the agent forms a physical understanding of the optics and reparametrizes the network with interpretable optical elements tuned on the hardware. It outperforms gradient-free methods in classification and image encoding, approaching backpropagation through a calibrated model at a fraction of the cost. Needing neither per-device calibration nor gradients through the hardware, and improving with more capable models, optics as code offers a practical route to large, scalable optical neural networks and a general tool for calibrating and optimizing optical experiments.
[AI-138] PhysDial: Adapting Pretrained Video Models for PDE Simulation under Changing Physical Conditions
链接: https://arxiv.org/abs/2611.23498
作者: Yilong Dai,Yiming Sun,Shengyu Chen,Jiaqi Gong,Xiaowei Jia,Runlong Yu
类目: Fluid Dynamics (physics.flu-dyn); Artificial Intelligence (cs.AI)
备注: Code: this https URL
Abstract:Simulating how physical systems respond to changing conditions is central to control, design, and scenario studies. To our knowledge, prior PDE foundation models have not been trained or evaluated on changes within trajectories. Our counterfactual test measures how far predictions move toward a changed solver branch from the same state. Models trained on fixed conditions follow changes only partly. A PDE foundation model given a condition input after pretraining barely responds. PhysDial adapts a pretrained video diffusion transformer to eight two-dimensional PDE families. Coefficients enter with the noise level, condition fields as reference frames, and equation text as a prompt. In Kolmogorov flow, switches within trajectories account for about 5% of the training samples. At matched compute, they halve the error after a change in forcing wavenumber or amplitude. A few trajectories at new amplitudes more than halve extrapolation error. Without physics pretraining, PhysDial matches or exceeds PDE foundation models under training conditions on two turbulent flows. At step 64 on the Kolmogorov test split, PhysDial has 1.07 times the solver’s small-scale energy and Walrus 7.1 times. Our experiments suggest that a learned simulator uses a condition only as far as its training data make that condition necessary for prediction. Learning from trajectories that diverge under intervention can help scientific simulators predict what would happen if a system’s forcing or parameters changed.
[AI-139] Neural Circuit Architectural Priors for Rat Locomotion NEURIPS2026
链接: https://arxiv.org/abs/2611.23364
作者: Nikhil X. Bhattasali,Jaron Cui,Lerrel Pinto,Grace W. Lindsay
类目: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Robotics (cs.RO)
备注: NeurIPS 2026. 10 pages main text, 41 pages total, 7 figures in main text. Code and videos: this https URL
Abstract:Animals generate behavior through continuous interactions between their neural circuits, bodies, and environment. Embodied simulations study these interactions by testing neural circuit models in biologically realistic settings. Recent efforts have simulated diverse animals, and emerging work on biologically grounded neural architectures has begun to integrate circuit connectivity and single-unit dynamics from experimental data into artificial neural networks, introducing stronger inductive biases and closer mechanistic alignment with biology. How can we develop embodied simulations that enable the study of increasingly complex bodies and neural circuits? In this work, we address the challenge through coordinated advances in the simulated body, neural architecture, and training infrastructure. We introduce Rat v2, an efficient and flexible biomechanical body for simulating rat behavior. To control this body, we develop Rat NCAP, a biologically grounded neural architecture for locomotion based on mammalian spinal circuits that incorporates cell-type-specific connectivity and a novel flexor-extensor neuromechanical interface. To optimize circuit parameters, we develop GPU-accelerated simulation and training infrastructure. The model learns to locomote at different speeds and transition between gaits without imitation learning, producing locomotion statistics, kinematics, and cell-type-specific neural activity consistent with experimental data. Analyses reveal that the neuromechanical interface substantially affects trainability and gait quality, and in silico cell-type ablations reproduce known gait phenotypes and predict gait changes under graded perturbations. Together, these findings support the view that embodied simulations at an intermediate level of abstraction can simultaneously reproduce complex behavior and yield mechanistic insights into the neural circuits that generate it.
[AI-140] SurgCBVR: A Multi-Domain Benchmark for Content-Based Surgical Video-to-Video Retrieval
链接: https://arxiv.org/abs/2611.23347
作者: Mohammad Eslami,Mousa Moradi,Negin Ghamsarian,Nazlee Zebardast,Jennifer L Lindsey,Sophia Y. Wang,Lucia Sobrin,Rishi P Singh,Tobias Elze
类目: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI)
备注:
Abstract:Hospitals record thousands of hours of surgical video annually, yet these archives remain difficult to search by content. Surgeons cannot readily retrieve similar operations, complications, or workflow events. Existing surgical computer vision primarily produces labels rather than retrieving relevant videos. We introduce SurgCBVR, the first benchmark for content-based video-to-video retrieval in surgery. It integrates eight public datasets across four domains-ophthalmic, laparoscopic, robotic-assisted, and colonoscopy-covering six procedure types, 679 cases, and 76,489 clips (106 hours). SurgCBVR comprises two tiers: Scene, retrieving the same operative state across cases and sites, and Workflow, retrieving and localizing phase transitions and deviations. Scene includes 74,765 clips spanning 40 classes, three temporal granularities, and three playback speeds; Workflow contains 1,724 transition clips. Fixed clip lengths and frame counts, together with phase-classifier oracle and duration-only baselines, help assess label and timing shortcuts. We evaluate six generic and surgical foundation models, two video retrieval methods, and contrastive heads trained on SurgCBVR, using within-dataset, cross-dataset, and pooled-gallery retrieval restricted to the same procedure type. Frozen representations barely outperform a timing-only baseline, transition retrieval approaches chance, and trained heads lose up to half their accuracy across datasets. These findings expose substantial limitations in current video representations and establish SurgCBVR as a benchmark for advancing surgery-aware retrieval.
[AI-141] Reusing formal Vlasov-Maxwell proofs to certify plasma solver outputs
链接: https://arxiv.org/abs/2611.23319
作者: Bowen Zhu,Jian Wu
类目: Plasma Physics (physics.plasm-ph); Artificial Intelligence (cs.AI)
备注: 39 pages (including Supplementary Information), 8 figures, 5 ancillary files (source data and video). Submitted to Nature Computational Science
Abstract:AI now produces machine-checked mathematics faster than researchers can read it. Can such theorems do practical computational work? We reuse one from \mboxOpenAI’s Lean~4 relativistic Vlasov–Maxwell library: initial fields determine the field at an outside point until plasma signals reach it, and the energy leaving a surrounding box before they reach its faces. New Lean proofs and numerical bounds give particle-free references for particle-in-cell simulations across three tasks. Domain size follows analytically from observation time, exposing boundary and initialization errors in WarpX and Smilei that global energy checks missed. Grid choice can be tested directly: combining two coarse grids recovers unknown plasma inputs within tolerance at a median 14.9-fold speedup over a fine grid, where the medium grid alone fails. For reduced models, the bounds establish the minimum number of time curves needed to predict energy loss.
机器学习
[LG-0] Equivariant Modeling with Polynomial Features via Minimal Homogeneous Generators
链接: https://arxiv.org/abs/2611.24896
作者: Michael Liaofan Liu,Joe Kileel
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:We present a general framework for modeling and learning equivariant functions between finite dimensional semisimple real group representations, unifying constructions developed across many application areas. Our model is a linear combination of equivariant polynomial features whose coefficients are arbitrary learnable functions of invariant polynomial features. We take as features minimal homogeneous generators for the invariant algebra and the covariant module over the invariant algebra, up to prescribed degrees. We provide a Python package that automatically computes these features and fits the resulting models to training data. The algorithm for feature computation requires explicit isotypic decompositions of the input and output representations, along with explicit Clebsch-Gordan decompositions of the tensor product of irreducible representations, and achieves speedups over alternative methods in computational algebra. Numerical experiments in scientific modeling demonstrate the full learning pipeline and yield accurate predictors with few trainable parameters, using the model both for end-to-end prediction and within layers of larger architectures.
[LG-1] On Evaluating User Simulators for Interactive Agents
链接: https://arxiv.org/abs/2611.24892
作者: Hengguang Zhou,I-Hung Hsu,Li Chen,Priyanshi Shrivastava,Anastasios Angelopoulos,Wei-Lin Chiang,Ion Stoica,Cho-Jui Hsieh
类目: Machine Learning (cs.LG)
*备注:
Abstract:User simulators offer a scalable way to evaluate interactive agents, but how to assess their quality remains underexplored. In particular, it is unclear whether existing user simulator evaluation frameworks reliably detect meaningful failures in simulated user behavior. This uncertainty makes it difficult to interpret simulator evaluation. To understand what a high benchmark score tells us about simulator quality, we (meta-)evaluate the metrics themselves. To instantiate this meta-evaluation, we deliberately perturb simulated replays to induce failures such as altered goals, omitted constraints, premature stopping, and acceptance of broken work, then test whether the metrics penalize these failures. Evaluation methods from three prior studies detect only 5-6 of the 10 perturbations; their combined scores even favor populations instructed to accept broken work over unmodified controls. Motivated by these gaps, we propose a new evaluation suite that combines utility of user simulators for evaluating agents and two measures of fidelity to real users: population-level fidelity assesses collective interaction patterns, while trace-level fidelity assesses the preservation of individual users’ goals, constraints, and persona. Under the same detection criterion, our evaluation suite detects all 10 perturbations. Evaluating six user simulators on 351 real multi-turn sessions in which 348 distinct users across 23 languages and agents work together to create and refine artifacts such as full-stack web apps, games, design assets, and automated workflows, we find that population-level measures can miss when a simulated user pursues the wrong goal, ignores a constraint, or behaves out of character, highlighting the need to assess fidelity at comprehensive granularities.
[LG-2] Cap-and-Track: Optimizing LLM Inference Request Dispatch Across Shifting Capacity
链接: https://arxiv.org/abs/2611.24891
作者: Benjamin Novotny,Doug Szeto,Chang Wang,Shicheng Chen,George Zhihua Yuan,Bob Kamma,Michael Ge,John Xin Zhou,Andrea Olgiati
类目: Machine Learning (cs.LG)
*备注: 20 pages, 6 figures, 2 tables. Includes appendices
Abstract:A fixed dispatch rate for LLM inference becomes stale as serving fleets resize and request sizes change. Generative output length is unknown at admission, and shared provider pools can lose capacity. We present Cap-and-Track, a design for asynchronous inference across provider capacity and internal GPUs. Meta’s traffic shaper implements Cap-and-Track for post and live-video understanding. Refusals signal the backend’s current admission ceiling. Durable priority queues hold excess work outside execution, including refused requests for retry. Cap is a fail-fast admission check at the saturated resource that refuses work before a long wait. Using only those refusals, the Track dispatcher probes upward and backs off. It follows the ceiling without output-length prediction or a fleet-level rate to retune. In a concurrent same-traffic production A/B, the shaper ran per-replica Cap plus Track. It served a classifier on 43% fewer H100s than its queued, fixed-rate precursor, though lower-priority work waited longer at backfill peaks. In steady-capacity real-engine replay, Track ran within 1% of the goodput of a fixed rate tuned above measured capacity, with about a quarter of that rate’s refusal share. The shaper carries more than 300 million online requests and hundreds of terabytes of media a day. During a backfill, it handled 1.1 billion requests in its busiest 24 hours. Lessons include keeping queues inside inference engines just long enough to fill batches, steering dispatch by capacity refusals alone, keeping dispatch stateful, and sizing fleets by how long low-priority work can wait.
[LG-3] Monoculture Robust Learning
链接: https://arxiv.org/abs/2611.24881
作者: Lee Cohen,Jon Kleinberg,Omer Reingold
类目: Machine Learning (cs.LG)
*备注:
Abstract:We introduce a notion of monoculture robustness in Probably Approximately Correct (PAC) learning, as a way to mitigate the problem of algorithmic monoculture (Kleinberg and Raghavan, 2021): the tendency of models sharing the same components (e.g., training data) to fail on the same inputs. Specifically, we ask whether a learner with access to a single dataset can output k individually accurate hypotheses whose joint failure probability is comparable to training on k independent datasets. We say that such a learning procedure achieves monoculture robustness, and we quantify this by bounding the conjunctive error that all k hypotheses misclassify points in an individual sense. We design methods for achieving this type of guarantee by drawing a surprising connection between replicability (Impagliazzo et al., 2022) and monoculture robustness. Specifically, we show that if the base learner is \rho -replicable, then running it independently k times on the same sample yields individual monoculture error that shrinks exponentially with k , controlled by the single-run error of the base learner at the point and \rho . We complement these positive results with a lower bound: for individual monoculture robustness, we prove a worst-case lower bound showing that, for learners, driving the individual conjunctive error down to \gamma can require a sample-size multiplier of t = \Omega(\log(1/\gamma)) .
[LG-4] Learning to Interact in Multi-Agent Self-Play for Controllable Autonomous Driving Scenario Generation
链接: https://arxiv.org/abs/2611.24876
作者: Beyazit Yalcinkaya,Ethan Pronovost,Maximilian Naumann,Marcell Vazquez-Chanlatte
类目: Robotics (cs.RO); Machine Learning (cs.LG)
*备注: Preprint
Abstract:Large-scale, multi-agent self-play trains robust driving policies effectively in simulation, but controlling these models to generate specific, safety-critical edge cases remains a key challenge. Existing goal-conditioned self-play frameworks rely primarily on static map-location goals and fail to reliably produce complex agent interactions such as aggressive tailgating or adversarial cut-offs. These agent interactions are necessary for rigorous autonomous vehicle (AV) testing. To address this, we present an extended multi-agent self-play framework that introduces agent-relative dynamic goals. By conditioning policies on this extended goal representation, our framework enables targeted, controllable scenario synthesis. We demonstrate that our approach successfully generates interactive driving behaviors that static-goal baselines cannot simulate, providing a flexible tool for long-tail scenario generation in AV testing.
[LG-5] angentPDE: Learning Local Operator Responses with Sparse Finite-Difference Supervision
链接: https://arxiv.org/abs/2611.24873
作者: Xinjian Xie,Ruxuan Chen,Jiashu Li,Tianyu Gao,Ye Wei
类目: Machine Learning (cs.LG)
*备注:
Abstract:Neural PDE operators are often optimized through state reconstruction, which largely constrains operator values at observed inputs but does not fully characterize their sensitivity to local perturbations. A more faithful approximation of operator behavior requires not only accurate state predictions, but also correct variations along local perturbation directions. We present TangentPDE, which uses finite differences to approximate directional derivatives and constructs task-relevant finite-step responses from the state outputs of a reference solver at sparse paired inputs. Its objective jointly constrains state fidelity, response-direction alignment, and response-magnitude calibration, thereby learning both operator values and their local variations. Because the response labels are constructed solely from state outputs at paired inputs, the method requires neither explicit derivative labels, linearized or adjoint PDE solves, nor full Jacobians, and remains applicable to nonlinear solvers without derivative interfaces. Building on this, we additionally propose a selectively executed adaptation mechanism to strengthen local operator representations under finite-difference supervision. Budget-matched and correspondence-perturbed controls isolate the effect of relational supervision from improvements attributable to additional state observations or increased data volume. Designed as a plug-and-play training framework, TangentPDE integrates with diverse neural-operator backbones. Across four PDE systems and 20 task-backbone settings, TangentPDE lowers the three-seed mean state error in 18 settings and the corresponding Rollout@20 error in 18, while intrinsic finite-step response gains are strongest on DiffReact and remain task- and backbone-dependent elsewhere.
[LG-6] Atomic MathReason er: Instance-Level Composition of Atomic Reasoning Skills for Mathematical LLM s
链接: https://arxiv.org/abs/2611.24846
作者: Jiashu Li,Tianyu Gao,Qinglong Liu,Jinping Wang,Ye Wei,Zhiqiang Gao
类目: Machine Learning (cs.LG)
*备注:
Abstract:Most existing approaches enhance mathematical reasoning in large language models (LLMs) as a monolithic capability, despite the fact that different mathematical problems require different reasoning skills. Recent work has explored a more fine-grained perspective by decomposing mathematical reasoning into atomic skills and developing specialized modules for individual capabilities. However, how to effectively utilize these atomic abilities remains underexplored. To address this challenge, we propose Atomic MathReasoner (AMR), an instance-level framework for composing atomic reasoning skills. AMR first trains a library of capability-specific LoRA experts and performs gradient-free optimization over held-out validation subsets to discover a diverse set of low-loss composition prototypes. It evaluates the preference of each prototype using the token-level negative log-likelihood of reference solutions, filters ambiguous assignments through a confidence margin, and distills the resulting pseudo-labels into a lightweight router. During inference, the router selects an appropriate composition prototype for each problem, enabling problem-specific utilization of atomic skills without costly online optimization. Experiments on the AtomicMath benchmark across general and mathematics-specialized LLMs demonstrate consistent improvements. On Qwen2.5-Math-Instruct-7B, AMR increases average field accuracy from 64.4% to 68.1% at Level 1 and from 59.3% to 63.7% at Level 2, while improving average logical reasoning performance from 39.4% to 46.8%. These results demonstrate that effectively composing specialized atomic skills provides a promising approach to strengthening mathematical reasoning in LLMs.
[LG-7] Measuring and Improving Transfer of Neural Surrogates for Vehicle Aerodynamics
链接: https://arxiv.org/abs/2611.24843
作者: Favour Nerrise,Julian Busch,Gurpreet Singh Hora,Martin Sipka,Danielle C. Maddix,Georgia Olympia Brikis
类目: Machine Learning (cs.LG); Fluid Dynamics (physics.flu-dyn)
*备注: 23 pages, 5 figures, 13 tables
Abstract:Neural surrogates for external vehicle aerodynamics are fast alternatives to physics-based computational fluid dynamics (CFD) simulations. Their foundational limitation is a lack of zero-shot generalization across vehicle classes, which results in an expensive, custom training campaign for each new vehicle design. Similar to pretrained models in other fields, a pretrained surrogate should adapt to a new vehicle class using only a few simulations and stay accurate on its pretraining domain. No prior study has systematically tested this transfer behavior. We conduct a large-scale, controlled benchmarking study on five DrivAerML-pretrained surrogates across four independent target datasets. These datasets differ from the source in geometry, CFD solver, turbulence model, and flow conditions. We find that fine-tuning these surrogates outperforms training from scratch in fewer than half of the comparisons and lowers the accuracy of the surrogate on its source dataset. We introduce GeoReplay, a geometry- and gradient-aligned source replay that keeps source simulations resembling the target in training and applies their updates only when they agree with the target gradient. A risk bound separates geometric coverage from differences in the simulated physics, and we prove that every sufficiently small GeoReplay step decreases the target training objective. GeoReplay lowers the source error of fine-tuning in 77 of 80 settings and matches its target accuracy.
[LG-8] What Survival Benchmarks Dont Tell You: Impact of Model Selection and Dataset Regimes
链接: https://arxiv.org/abs/2611.24837
作者: Sandra Dening,Emre Kavak,Christian Wachinger,George H. Chen
类目: Machine Learning (cs.LG)
*备注:
Abstract:Despite the proliferation of survival analysis methods, current benchmarks often operate under narrow conditions: they heavily rely on the concordance index (C-index) for evaluation and model selection, and frequently either exclude deep learning (DL) methods or underrepresent challenging dataset regimes. To address these limitations, we conduct a large-scale, neutral benchmark comparing 10 classical, tree-based, and DL methods across 49 diverse datasets. Crucially, we treat the model selection criterion (optimizing for C-index, Integrated Brier Score (IBS), or Mean Absolute Error (MAE-PO)) as an independent experimental variable. Using Bayesian mixed-effects modeling, we demonstrate that evaluation design heavily dictates conclusions. For complex DL and tree-based architectures, selecting hyperparameters to optimize for IBS instead of C-index yields substantial improvements in probabilistic accuracy and calibration at a negligible cost to discriminative ranking. Furthermore, method performance (C-index, IBS, MAE-PO, and X-CAL) is fundamentally governed by dataset characteristics. While DL models scale exceptionally well with larger sample sizes, they suffer notable performance drops in high-dimensional feature spaces. Additionally, we expose a structural vulnerability where heavy censoring artificially inflates absolute calibration metrics, creating overly conservative tests. Ultimately, our findings challenge the utility of global single-number performance summaries and advocate for survival analysis results that transparently report the evaluation setup. Our code is available at: this https URL
[LG-9] Prognosis Without Prediction: The Intrinsic Geometry of Survival
链接: https://arxiv.org/abs/2611.24799
作者: Atahan Karagoz
类目: Machine Learning (cs.LG); Quantitative Methods (q-bio.QM)
*备注: Code available at: this https URL
Abstract:Modeling systemic failure in high-dimensional environments is fundamentally limited by supervised algorithms that attempt to predict sparse, noisy or censored endpoints. To break this reliance on explicit outcome regression, we engineered the Self-Organizing Survival Manifolds (SOSM) framework, a deep learning architecture that decouples unsupervised representation learning from survival labels and treats time-to-event progression as an emergent geometric property. The system projects heterogeneous multi-omics arrays into a unified latent space governed by a pseudotime-weighted, curvature-minimizing graph Laplacian, organizing patient trajectories along low-dissipation geodesic flows. To validate the framework, we applied this architecture to a 184-patient clear cell renal cell carcinoma multi-omics cohort. Across a deterministic 5-fold cross-validation pipeline, SOSM achieved a mean Concordance Index of 0.768 \pm 0.068 , while out-of-fold risk stratification yielded significant survival divergence (log-rank p 0.001 ) accompanied by a 14.7% increase in local manifold dissipation among high-risk patients. Furthermore, interrogating the network’s encoder weights confirmed that the model autonomously recovered canonical metabolic and microenvironmental drivers of disease progression without supervised feature selection. By demonstrating that systemic failure can be mapped as a geometric phase transition, SOSM establishes a generalizable computational engine for prognostic modeling across biological, industrial and financial complex systems.
[LG-10] Reconstruction vs. Prediction Pre-Training of Discrete Video Representations for Continuous Control NEURIPS2026
链接: https://arxiv.org/abs/2611.24781
作者: Ron Kosova,Francesco Croce
类目: Machine Learning (cs.LG)
*备注: 9 pages, 6 figures. Accepted to the PTA Workshop at NeurIPS 2026
Abstract:Self-supervised video encoders are commonly trained to either reconstruct pixels or predict latent features, yet it remains unclear how these objectives shape representations for downstream dynamics. We present a capacity-matched study isolating this choice: two identical video encoders with a shared FSQ bottleneck are trained under autoencoding and V-JEPA objectives on a continuous control environment. We introduce a simple asymmetric discretization scheme to apply the same bottleneck to the predictive model. The resulting representations exhibit a clear trade-off: the autoencoder preserves substantially more physical state information and achieves slightly better one-step latent prediction, while the predictive encoder accumulates less error under autoregressive rollout. Structural probes suggest that this advantage is associated with greater robustness to low-level visual perturbations and stronger dependence on temporal context. These results indicate that predictive and reconstructive objectives induce distinct representations of dynamics, trading state fidelity for properties that may benefit long-horizon prediction.
[LG-11] Cross-Vehicle Transferability of Lateral Velocity Estimation on Production Vehicles through Input Selection and Data Augmentation
链接: https://arxiv.org/abs/2611.24747
作者: Hendrik Schäfke,Simon F. G. Ehlers,Thomas Seel,Askar Vagapov,Christoph Schweers,Ahmed Trabelsi,Daniel O. M. Weber
类目: ystems and Control (eess.SY); Machine Learning (cs.LG)
*备注: 14 pages, 7 figures, 6 tables. Submitted to Control Engineering Practice
Abstract:Lateral velocity is safety-critical for vehicle-dynamics control but cannot be measured with production sensors. Model-based estimators such as Kalman filters are established, whereas data-driven estimators are more accurate within the training regime but suspected to lose accuracy beyond it. Their transfer to a structurally different vehicle has rarely been tested. We record real-world test drives of three series-production vehicles, train a neural-network estimator on production CAN signals of a compact car, and evaluate it zero-shot on an SUV and a van against a GNSS-aided inertial reference. On the unseen vehicles, the network’s error exceeds that on the training vehicle by an order of magnitude, a gap not quantified on production vehicles before. A systematic comparison of input sets from ten production CAN signals identifies a five-signal set that transfers most robustly. To narrow this gap, we introduce three physically motivated augmentations of training data. Two improve generalization, namely exploiting the vehicle’s left-right symmetry and emulating accelerometer bias. Combined, they reduce the error on the validation maneuvers of the SUV and van by 13% and 15%. Contrary to common expectation, on the SUV the data-driven estimator trained with both augmentations matches a model-based estimator tuned on the same training data.
[LG-12] Uneven gains in protein-RNA affinity and mutation-effect prediction
链接: https://arxiv.org/abs/2611.24739
作者: Shangkun Sima,Tongfei Shen,Tianchi Lu
类目: Machine Learning (cs.LG); Computational Engineering, Finance, and Science (cs.CE)
*备注: 10 pages, 7 figures, 3 tables. Supplementary material included as an ancillary file
Abstract:Accurate matched affinities determine mutation effects, but gains in aggregate affinity prediction may transfer unevenly to average effects, ordering and local accuracy. We evaluated eleven method families using method-specific matched populations. Controlled CoPRA reduced absolute-affinity mean absolute error from 1.2898 to 1.1431 pKd. Across 220 mutation pairs from 20 families, it generated stronger responses and improved mean-effect accuracy, with uncertain ranking gains and no decrease in mean centered local error. In an independent external test (24 quantitative pairs from three source-matched families), absolute and ordinary mutation-effect errors fell in all three families. Mean centered local error fell from 0.4198 to 0.3761 pKd, with improvements in two families and higher error in one. Error decomposition localized gains in landscape means. PRA-Pred feature tracing, fixed-weight M2-PRNet input controls and matched supervision interventions identified specific restrictions on mutation response. These results establish the need to evaluate average effects, ordering and quantitative local variation jointly.
[LG-13] Approximate Dynamic Programming with Basis Function Learning
链接: https://arxiv.org/abs/2611.24737
作者: Mohammadreza Fathi,Chi-Guhn Lee
类目: Machine Learning (cs.LG)
*备注:
Abstract:This work presents a novel approach for approximating the value function in Markov decision processes (MDPs). Our approach is grounded in the approximate linear programming (ALP) framework, wherein basis functions (BFs) play an important role. Traditionally, BFs have been pre-defined, randomly generated, or restricted to specific structures, which both limits the accuracy that can be achieved and leaves no mechanism for improving a given approximation. We instead learn BFs from the model itself, by repeatedly applying the Bellman operator to the current value estimate. We introduce the Bellman operator basis function (BOBF) learning method, and prove that each learned BF improves the approximation by a quantifiable amount and that the method converges to the optimal value function. Its shrunken counterpart (SBOBF) retains this guarantee while capping the number of BFs, and therefore the computational effort per iteration. For large-scale problems, where the number of constraints is prohibitive, we develop hybrid variants (HBOBF and SHBOBF) that combine constraint sampling with BF learning, and we show that the improvement and finite-termination guarantees are preserved under explicitly stated conditions. Numerical experiments on (continuous) perishable inventory and queueing control problems demonstrate that our BF learning techniques significantly outperform existing methods in the literature.
[LG-14] Meta-LSQ: A Closed-Form Sobolev Inner Solver for Few-Shot Meta-Learning
链接: https://arxiv.org/abs/2611.24718
作者: Antonin Sulc
类目: Machine Learning (cs.LG); Logic in Computer Science (cs.LO)
*备注: 15 pages (9 main + references + appendix), 5 figures, 10 tables. Accepted at the Northern Lights Deep Learning Conference (NLDL 2027); to appear in PMLR vol. 352
Abstract:Gradient-based meta-learners such as model-agnostic meta-learning (MAML) adapt to a new task with an inner loop of gradient descent that has to be differentiated through, and they meta-learn every network weight. Meta-LSQ instead fits each task with one regularised least-squares solve over a fixed random Fourier basis and meta-learns only three scalars, a bandwidth, a ridge and a derivative weight. Since the derivative of a sinusoid is again a sinusoid, the same solve can also take derivative observations, and the fitted model has exact closed-form derivatives of any order. On 10-shot sinusoid regression Meta-LSQ trains in 14 s and ends with a 5.5 times lower error than a first-order MAML baseline that trains 36 times longer, and a Gaussian-process control suggests that most of this gain comes from sharing one kernel across tasks. In simulated beam tuning at DESY’s ARES facility with model derivatives it is slightly more accurate than an ALPaCA-style model and well ahead of gradient descent. Over DESY’s full range of target beams, again in simulation and from measurements alone, its best setting has a median error of 7.3 \mu m, against 14.4 for the published final beams of their reinforcement-learning agent, which has the lower median on their 300 trials of small targets. On real sensor-calibration data a learned prior mean puts it ahead of MAML on two of four targets, but an exact Gaussian process with the same change and an ALPaCA-style model stay more accurate on most targets, and it breaks down on non-smooth targets. On noiseless targets outside the span of the basis, in-sample criteria such as generalised cross-validation choose too small a ridge.
[LG-15] Stealing Exact Training Data from Other Clients during Federated Learning
链接: https://arxiv.org/abs/2611.24689
作者: Omkar Shrikanth,Soham Dambalkar,Parag Jyoti Pal,Advik Raj Basani,Arnab K. Paul
类目: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Distributed, Parallel, and Cluster Computing (cs.DC)
*备注: Preprint. 22 pages, 21 figures
Abstract:Malicious adversaries can employ gradient inversion attacks to reverse-engineer private data using the model updates of a round in Federated Learning. While there have been developments in gradient inversion for text, exact text recovery by a malicious client has so far remained infeasible, as FedAvg entangles the gradient contributions of many clients and obscures the signal of any individual sentence. We introduce a novel attack pipeline that unwinds this entanglement progressively through an iterative loop, in which the gradient contributions of sentences recovered exactly in one round are subtracted from the aggregate, exposing the remaining sentences for recovery in the next iteration. We show, for the first time, exact sentence recovery, with 100% precision, and on aggregates containing up to 4\times more tokens than the model’s hidden dimension, we recover up to 80.2% of client sentences exactly, a regime where existing attacks recover none exactly.
[LG-16] Zonotopic Analysis of Quantization Error in Neural Networks
链接: https://arxiv.org/abs/2611.24648
作者: Robert Debbas,Éric Goubault,Sylvie Putot
类目: Machine Learning (cs.LG); Logic in Computer Science (cs.LO)
*备注:
Abstract:Quantized neural networks achieve computational efficiency by reducing precision, but bounding their quantization error remains a challenge due to discrete rounding and clamping operations. We present a zonotope-based abstract interpretation framework for quantized ReLU feed-forward network verification, introducing a unified round-clamp abstract transformer that jointly handles both operations. We extend this framework to constrained zonotopes, which add linear constraints to tighten bounds, and derive sound quantization error representations. We establish soundness and affine-strip optimality of the round-clamp abstract transformer, as well as convex-hull optimality of its complete constrained formulation. On the ACAS Xu benchmarks, our constrained zonotope method reduces average error-bound widths by approximately 70-85% and runtimes by 65-80% compared to QEBVerif’s symbolic DRA propagation at the two smallest tested perturbation radii (r=3 and r=6). These improvements are not uniform across bitwidths, and symbolic DRA achieves tighter bounds at larger radii. Our plain zonotope method delivers competitive sound bounds nearly 300 times faster on average.
[LG-17] Equal Steps Unequal Curvature: Where Muons Update Goes Wrong
链接: https://arxiv.org/abs/2611.24644
作者: Biswa Sengupta,Jinhua Wang,Léo Maxime Brunswic
类目: Machine Learning (cs.LG)
*备注:
Abstract:Muon updates each hidden weight matrix along the polar factor of its momentum, the steepest-descent direction under the spectral norm. That choice fixes two allocations without consulting the loss: every singular direction of the momentum receives the same step, and in a non-square matrix each neuron’s step is set by its leverage score. Whether they are good is a question about curvature, which steepest-descent arguments do not answer; we measure it. Using held-out gradients and Hessian-vector products, we decompose Muon’s step by singular-value band and by neuron group and compute each group’s second-order optimal multiplier. In a 124M-parameter decoder trained on FineWeb-Edu, the multipliers rise monotonically from the top singular directions to the tail: Muon over-steps the high-curvature top of its spectrum. We prove in Lean when such a diagnostic licenses an intervention at Muon’s own update norm. On the measured curvature, the criterion predicts, and held-out losses confirm, that moving step length off the top helps mid-training and early in the cooldown, but not at its end. We then test interventions under per-step norm matching, tuned baselines and paired seeds. Orthogonalizing Q, K and V separately rather than as one fused matrix lowers the loss by 0.0250 nats and accounts for all of the gain of block-wise variants. A matmul-only, rotation-equivariant spectral tilt \sigma^-1/4 applied in the cooldown lowers the loss of this stronger baseline by a further 0.0120 nats, additively with NorMuon (0.0158 nats together, three seeds; the baseline needs about 4.3% more tokens to match), and by 0.0188 nats at 355M parameters (three seeds). Energy-matched Schatten-1 weight decay adds a further 0.0052 nats. Geometrically principled changes that do not act on the measured miscalibration, namely exact polar factors, steepest-descent input whitening and top-heavy decay, do not help.
[LG-18] Evaluating the Effectiveness of Location Encoding Methods for Species Distribution Modelling NEURIPS2026
链接: https://arxiv.org/abs/2611.24622
作者: Christian Lange,Oisin Mac Aodha
类目: Machine Learning (cs.LG)
*备注: Accepted at the NeurIPS 2026 Workshop on Tackling Climate Change with Machine Learning
Abstract:Understanding the spatial distribution of species is a key question in ecology. A variety of solutions, from statistical, traditional machine learning, through to deep learning-based methods have been proposed. These methods take sparse observations for a species of interest as input and aim to generate dense spatial outputs indicating the spatial range of the species. Orthogonal to the choice of statistical model used is the question of what input features are most informative. Thanks to recent developments, the number of options has greatly expanded, with choices spanning different climate variables, learning-based encodings, earth embedding models, or raw transformed geographic coordinates. In this work, we introduce a standardized evaluation framework and quantify the effectiveness of different location encodings in the context of species range estimation, both at the local and global scale.
[LG-19] Morpho-Spectral Projection Analysis (MSPA) for Plant Spectroscopy
链接: https://arxiv.org/abs/2611.24599
作者: Lucas Prado Osco,Marcelo Santos da Silva,Filipe Vieira Silva,Liang Zhao,Ana Paula Marques Ramos,Ednaldo Jose Ferreira,Roberto Dantas de Medeiros,Amaury Burlamaqui Bendahan,Lucio Andre de Castro Jorge
类目: Machine Learning (cs.LG)
*备注:
Abstract:Plant reflectance spectroscopy across the visible, near-infrared, and shortwave infrared domain encodes foliar biochemical and anatomical traits into contiguous spectral signatures with locally differentiable structure; however, extreme inter-band collinearity and unfavorable sample-to-feature ratios challenge predictive modeling. Conventional dimensionality reduction methods routinely adopted in spectroscopy, are column-permutation invariant: they treat adjacent wavelengths as unordered tabular columns and discard the sequential differential geometry of absorption features. Here, we propose Morpho-Spectral Projection Analysis (MSPA), a supervised dimensionality reduction operator that projects contiguous spectra into a compact morpho-spectral subspace through three complementary blocks: (i) additive continuum-removed absorption depth, area, and red-edge inflection kinematics; (ii) local multi-order singular eigenmodes (0th, 1st, and 2nd derivatives) partitioned across contiguous radiative sub-windows; and (iii) global Fisher-weighted canonical subspaces regularized via trace-normalized isotropic covariance shrinkage. We validated MSPA across 8,963 spectra spanning five plant systems, coupled with six classical machine learning classifiers and four Tabular Foundation Models (TabPFN v3, TabICL v2, TabDPT v1.2, and TabFM). Despite compressing the 1,580-band spectrum by 96%, MSPA outperformed the full spectrum in 92.0% of all benchmarks and conventional reducers in 90.0% of all model-crop evaluations, raising the global mean macro-F1 from 0.7756 (full spectrum) to 0.8520, accelerating in-context inference by up to 126.5x. Designed as a self-contained spectral projection operator, the MSPA method can be directly integrated into spectroscopy workflows via the open-source code available at GitHub: this https URL.
[LG-20] From Activity Recognition to Shift Detection: Variational Wi-Fi Sensing with Evidential Uncertainty
链接: https://arxiv.org/abs/2611.24596
作者: Luca Cotti,Marco Cominelli,Francesco Gringoli,Lance M. Kaplan,Mani B. Srivastava,Trevor Bihl,Erik Blasch,Kara Combs,Daniel O. Brigham,Federico Cerutti
类目: Machine Learning (cs.LG)
*备注: 17 pages,8 figures, 8 tables
Abstract:Human motion perturbs wireless channels, making commodity Wi-Fi devices a practical platform for device-free activity recognition. Yet Channel State Information (CSI) also depends strongly on the person, receiver position, recording session, and surrounding environment, so a classifier can continue producing activity labels after its deployment conditions have moved away from those seen during training. We propose Wi-HAR, a multi-antenna architecture in which four Variational Auto-Encoders learn compact distributions from raw Wi-Fi 6 CSI magnitudes and a delayed-fusion classifier combines their means and log-variances. On twelve activities from a public benchmark, Wi-HAR reaches 96.6% median recognition accuracy, compared with 70.8-81.5% for independently optimized single-antenna models, showing that the antennas provide complementary views of the same activity. We then use the same latent backbone to detect deployment shifts without using shifted samples for model selection or calibration. A class-conditional uncertainty score derived from Evidential Deep Learning achieves a macro-average ROC-AUC of 0.897 across eleven datasets that change receiver position, participant, recording session, or room, and detects 82.5% of shifted samples at a realized 5.2% in-distribution rejection rate. Detection performance nevertheless varies substantially across receiver positions, showing that the same physical change does not induce a uniform uncertainty response. The compact representation also supports shallow decision-tree analysis using variables from several antennas.
[LG-21] he geometry of neural network decision boundaries
链接: https://arxiv.org/abs/2611.24557
作者: Dylan M Paiton,David L Schultheiss,Matthias Kümmerer,Zac Cranko,Matthias Bethge
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:Artificial neural networks (ANNs) are built using well-understood mathematical operations, yet their complex nature poses significant challenges to intuitively understanding their decision-making processes. This work introduces a method for visualizing and measuring ANN decision boundary geometry directly in the high-dimensional input space, avoiding distortions found with traditional projection-based techniques. We analyze boundary curvatures and distances in diverse inputs, architectures, and training methods. Boundary distances grow smoothly and slowly with consistent curvature as the size of the subspace increases, suggesting that decision boundaries are close to input samples within large, continuous subspaces. We also quantify how much boundary distances and curvatures reduce near dataset samples after adversarial training. Training methodology has a significantly stronger impact on boundary distance and curvature than architecture or test-set accuracy. These findings are contextualized within prior work, supporting or refuting several previously proposed theories of boundary behavior. Our analytically and empirically supported methodology lays the foundation for future studies to improve network interpretability and understanding.
[LG-22] Latent Action Policies from Suboptimal Data
链接: https://arxiv.org/abs/2611.24482
作者: Donghyeon Ki,Xiaoyu Chen,Jun Ho Seo,Hayeong Lee,Kyungyoon Kim,Byung-Jun Lee,Li Zhao
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:Latent Action Models (LAMs) learn control representations from action-free video, but existing pipelines typically train latent policies with behavioral cloning, without an explicit policy improvement objective. Although offline RL offers a natural approach to policy improvement, standard methods do not transfer directly to learned latent action spaces. We find that the structure of learned latent actions makes naive offline RL unreliable: directly maximizing value over latent actions or fitting a unimodal policy can produce latent actions that cannot be realized by physical execution, while the latent representation can merge physical controls that receive different rewards. To understand when latent-space optimization remains valid, we derive conditions under which policy improvement transfers to decoded execution and under which latent and physical optima align. Building on these insights, we propose LAPIS for goal-conditioned latent policy improvement, grounding value-based policy improvement in observed state-latent transitions and using a conditional flow policy to capture their multimodal structure. LAPIS outperforms baselines on most tasks, with the clearest gains in visual manipulation, and remains effective even with only 1% action labels, where competing methods largely collapse.
[LG-23] ICICLE: In-Context Inference for Curve-Learning Extrapolation NEURIPS2026
链接: https://arxiv.org/abs/2611.24469
作者: Laurits Fredsgaard,Michael Riis Andersen,Mikkel N. Schmidt
类目: Machine Learning (cs.LG)
*备注: 19 pages, 13 figures. Accepted at the AXIOM Foundations of Efficient Deep Learning workshop at NeurIPS 2026. Code and pretrained model: this https URL
Abstract:Freeze-thaw Bayesian optimization allocates training budget by predicting how partially observed learning curves will continue. LC-PFN first applied prior-data fitted networks (PFNs) to learning-curve extrapolation, and FT-PFN extended this approach to configuration-aware freeze-thaw optimization with a ten-slot hyperparameter encoder. We introduce In-Context Inference for Curve-Learning Extrapolation (ICICLE), an in-context surrogate that accepts a variable number of hyperparameters through a shared set encoder and is trained on a synthetic prior covering typed hyperparameters, heavy-tailed noise, quantization, and divergence. ICICLE has 1.28 million parameters (8.7% of FT-PFN’s count) and produces similar mean-regret trajectories to FT-PFN on LCBench, PD1, and TaskSet. On the 86-task Meta-Album benchmark with 33 hyperparameters, ICICLE reaches lower mean normalized regret than Quick-Tune overall and on extended, with no clear difference on mini or micro. It does so without pre-training or meta-training on Meta-Album tasks, whereas Quick-Tune is meta-trained on related Meta-Album tasks. Our release samples synthetic training tasks on the fly and provides a freeze-thaw scheduler for Ray Tune and an ONNX model for inference without JAX or PyTorch.
[LG-24] Cooperative UAV and Ground Station for Robust MEC of Dynamic Aquatic IoT: A Generative AI-Enhanced Approach
链接: https://arxiv.org/abs/2611.24466
作者: Jiahao You,Ziye Jia,Chao Dong,Qihui Wu,Zhu Han
类目: Networking and Internet Architecture (cs.NI); Machine Learning (cs.LG)
*备注:
Abstract:The increasing aquatic Internet of Things (AIoTs) require computational support and coverage in applications such as maritime search and rescue. Unmanned aerial vehicles (UAVs) can offer low-cost, flexible aerial services, and ground stations (GSs) can provide powerful supports, which can cooperate to help the AIoTs in complex scenarios. However, the collaboration between UAVs and GSs for AIoTs faces challenges of task uncertainties, AIoTs trajectory uncertainties, heterogeneities, and limited computational resources. To address these issues, we propose a cooperative UAV and GS based robust multi-access edge computing (MEC) framework to assist AIoTs in completing computational tasks. Specifically, we formulate the optimization problem of joint task offloading and UAV trajectory to minimize the total execution time, which is in the form of mixed integer nonlinear programming and NP-hard to tackle. Therefore, we propose the algorithm of generative artificial intelligence-enhanced heterogeneous agent proximal policy optimization (GAI-HAPPO). The proposed algorithm integrates GAI models to enhance the actor network ability to model complex environments and extract high-level features, thereby allowing the algorithm to predict uncertainties and adapt to dynamic conditions. Additionally, GAI stabilizes the critic network, addressing the instability of multi-agent reinforcement learning approaches. Finally, extensive simulations demonstrate that the proposed algorithm outperforms the existing benchmark methods, thus highlighting the potentials in tackling intricate, cross-domain issues in the considered scenarios.
[LG-25] DyNA-Mixer: Decoupling Time and Topology for Scalable Dynamic Node Affinity Prediction
链接: https://arxiv.org/abs/2611.24426
作者: Yuanxing Zheng,Liping Wang,Xuemin Lin
类目: Machine Learning (cs.LG)
*备注: Includes appendices
Abstract:Continuous-time dynamic graphs preserve the ordering, timing, and evolving strength of interactions that static snapshots discard, making dynamic graph mining essential for evolving systems such as recommender systems, trade networks, and transaction platforms. Research in this area has centered on dynamic link prediction, which forecasts only whether a future interaction will occur, whereas many applications also need its intensity. The dynamic node affinity prediction (DNAP) problem generalizes this task to continuous targets and full-candidate ranking, making it more broadly applicable but substantially harder. This raises two coupled difficulties: ranking requires a predictor expressive enough to capture both multi-scale temporal evolution and graph-wide collaborative patterns, yet running such a predictor over every source–destination pair is computationally prohibitive. We propose DyNA-Mixer, which decouples temporal forecasting from collaborative structure modeling. It adopts a multi-scale temporal predictor for pair-specific affinity evolution, constructs a Virtual Collaborative Graph that infers query-time relationships and aggregates trends from related interactions, and fuses the two through a gated residual mechanism. A sparsity-aware evaluation strategy further restricts expensive forecasting to pairs with recent non-zero affinities. Experiments on four Temporal Graph Benchmark datasets for the DNAP problem show that DyNA-Mixer achieves new state-of-the-art NDCG@10 on every dataset, including a 17.8% relative test improvement over the strongest baseline on tgbn-token, the largest dataset.
[LG-26] Predicting Process-Level Behavior: A Benchmarking Study
链接: https://arxiv.org/abs/2611.24425
作者: Michael Grohs,Lukas Kirchdorfer,Jana-Rebecca Rehse
类目: Machine Learning (cs.LG); Software Engineering (cs.SE)
*备注: Accepted as main track paper at International Conference on Process Mining (ICPM) 2027
Abstract:Predictive process monitoring (PPM) typically forecasts the future of individual process instances, hence operating on trace level. Process managers, however, typically operate on process level, e.g., when assigning staff or calculating KPIs. The question is how well such process-level managerial decisions are supported by existing approaches, given that PPM research has so far not focused on process-level predictions. We answer this question through a comprehensive benchmarking study of two process-level prediction tasks: concurrent trace prediction and average throughput time prediction. Across synthetic and real-life event logs, we compare statistical and machine-learning-based methods, foundation models, business process simulation, and, importantly, established trace-level PPM approaches, aggregating their predictions to the process-level. We assess which modeling paradigms are most suitable for predicting process-level behavior and whether existing trace-level PPM can be effectively repurposed for this task. Our results show substantial variation in performance across approaches. Notably, aggregated trace-level predictions cause errors to compound on process-level, reducing accuracy and highlighting process-level prediction as a distinct predictive task that warrants dedicated research.
[LG-27] RSIBench-Infra: Can Agents Optimize Their Own Training and Inference Infrastructure at Production Scale?
链接: https://arxiv.org/abs/2611.24395
作者: Hanlin Wu,Yifei Wang,Zhicheng Cai,Huan-ang Gao,Wei-Ying Ma,Ya-Qin Zhang,Jingjing Liu,Hao Zhou
类目: Machine Learning (cs.LG)
*备注: 22 pages. Project page: this https URL
Abstract:Improving the efficiency of training and inference infrastructure is a practical component of recursive self-improvement. Existing benchmarks for agent-driven optimization largely focus on individual software layers such as kernel, leaving unclear whether agents can deliver end-to-end performance gains under production workloads. We introduce RSIBench-Infra, a benchmark of 20 training and inference tasks spanning diverse model families and modalities, with deployments of up to 32 accelerators on NVIDIA GPUs and Ascend NPUs. Agents start from runnable baselines derived from official recipes and can modify the software stack across engine, kernel, and instruction levels. An OCI image-layer submission protocol captures these changes for reproducible deployment, while independent evaluation measures end-to-end performance under task-specific quality constraints. Across all task results, 66.2% exceed 1.10\times validated speedup. These results demonstrate that current agents can improve production-scale training and inference performance, but identifying effective optimization strategies and consistently translating local improvements into end-to-end gains remain challenging.
[LG-28] How to Turn an Attention Neural Network into an Interpretable Hard-Logic Predicate Circuit: Truth-Table Predicate Extraction and State-Machine Merging under Ship-of-Theseus Replacement
链接: https://arxiv.org/abs/2611.24381
作者: Yuchen Li,Zongqi Fan,Xintong Li,Nguyen H. Tran,Ken-Tye Yong
类目: Machine Learning (cs.LG); Hardware Architecture (cs.AR)
*备注:
Abstract:We convert a trained attention network as a whole, attention included, into a hard-logic predicate circuit that runs in its place. Every unit of the circuit is a 0/1 predicate on tokens, and every map between predicates is a minimised truth table. Short tables, such as the product of a query code and a key code, are minimised exactly, and wide feature-to-feature transitions become implication rules by noise-tolerant minimisation. Blocks are replaced one at a time from the output down, in the manner of the Ship of Theseus, and after each replacement the remaining soft part re-adapts to the frozen circuit. Attention selection takes two forms, a row-wise comparison of integer scores (order selection) and a learned rule (rule selection). Both are fitted on the ordinary training set alone, and only on the examples that the teacher classifies correctly, so that the circuit imitates the teacher where the teacher is right. On TweetEval-emotion, the complete circuit of DistilGPT2 reaches a test gold accuracy of 66.5 \pm 1.3% with order selection and 57.4 \pm 2.5% with rule selection over three seeds, against 74.7% for the teacher and 64.0% for a bag of word pieces fitted on the same tweets, and that of BERT-base reaches 70.0 \pm 0.5% with order selection and 63.7 \pm 1.2% with rule selection against 77.8%. The circuit needs no multiplier and no weight traffic, and its exported hardware description reproduces it bit for bit. By a synthesis-based estimate it classifies one tweet in about 1-2 \mu s (DistilGPT2) and 3-5 \mu s (BERT) on an FPGA card and in 5-9 \mu s on a palm-sized board whose module draws 7.5-15 W, two to three orders of magnitude faster than the network on a GPU at batch 1 (measured, 0.9-2.6 ms for DistilGPT2 including an optimised CUDA-graph baseline), at a cost of 8.5 (DistilGPT2) and 8.6 (BERT) points of gold accuracy for the synthesised seed-0 circuits.
[LG-29] VAFL: A Vertical Analytic Federated Learning Mechanism for Efficient Crowdsourced Services via Closed-Form Solutions
链接: https://arxiv.org/abs/2611.24367
作者: Yajiang Huang,Anfeng Liu,Tian Wang,Mianxiong Dong,Houbing Herbert Song
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:Intelligent services often rely on complementary information held by different organizations, motivating Vertical Federated Learning without centralized raw-data pooling. We propose VAFL, a Vertical Analytic Federated Learning framework that fits classification and regression heads on fixed local representations through regularized least squares. An agent-assisted protected-computation workflow supports two-party and multi-party collaboration. To reduce fitting overhead, we develop a full-dimensional equivalent solve that applies block factors directly to the target matrix, avoiding construction of the full inverse. We further study two width-reduction strategies: local spectral projection with coefficient lifting, and full-dimensional local fitting with low-rank cross-party correction. The first compresses the prediction space, whereas the second retains local components outside the collaboration subspace. We establish their algebraic properties and analyze their computational scope. Experiments on five benchmarks show the accuracy–cost tradeoffs of these strategies and compatibility with diverse frozen backbones. Predictive performance and training time identify useful operating points and the limits of compression.
[LG-30] Me How Sure You Are: Communicating Uncertainty in Auxiliary Evidence to Foundation Models
链接: https://arxiv.org/abs/2611.24351
作者: Soumyajit Chatterjee,Baturalp Buyukates,Abhirup Ghosh
类目: Machine Learning (cs.LG)
*备注: 28 pages, 8 figures
Abstract:In many queries, the context information provided to generative foundation models, such as LLMs and VLMs, is inherently uncertain, including its representations of the physical world. While conventional prompting discards the associated uncertainty, this paper, for the first time in the literature, systematically demonstrates that communicating such uncertainty in context provides useful additional cues for downstream decisions and yields improvements across a wide range of settings. We formulate the decision-making problem as a zero-shot foundation model-assisted image classification task supported by auxiliary text-based characteristics. We choose this due to the availability of well-defined auxiliary properties, associated uncertainty, and unambiguous task correctness. Using five downstream foundation models across three datasets, we extensively evaluate uncertainty arising from both humans and learned models when observing or predicting auxiliary evidence. Across all settings, we find that communicating uncertainty substantially improves downstream LLM classification performance: macro-F1 improves up to 26.18 percentage points compared to when the uncertainty information is not communicated. We further observe how the foundation models process such uncertainty information and find that auxiliary uncertainty contains predictive information about the correctness of downstream decisions. Together, these results establish uncertainty as useful contextual information and motivate more systematic consideration of uncertainty communication in future foundation-model inference pipelines. Comments: 28 pages, 8 figures Subjects: Machine Learning (cs.LG) Cite as: arXiv:2611.24351 [cs.LG] (or arXiv:2611.24351v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2611.24351 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-31] OpenGNS: An Open Dataset of the Gradient Noise Scale across Vision Language and Diffusion Models
链接: https://arxiv.org/abs/2611.24331
作者: Marcel Aach,Jiangtao Wang,Stefan Kesselheim,Andreas Lintermann
类目: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC)
*备注: 19 pages, 7 figures. Accepted at AutoML 2026 (Oral)
Abstract:The Gradient Noise Scale (GNS) metric measures the signal-to-noise ratio of gradients in stochastic gradient descent and, as shown by McCandlish et al. (2018), can be used to approximate the Critical Batch Size (CBS) during neural network training. Despite its appearance in technical reports of frontier models, no open dataset of GNS measurements across modern architectures exists, limiting systematic study of the metric. We close this gap by releasing OpenGNS, a dataset of more than 400 GNS measurement trajectories spanning three domains: image classification with ResNets, autoregressive language modeling with Generative Pre-trained Transformers (GPTs), and image synthesis with Diffusion Transformers (DiTs), covering models from 11M to 1B parameters trained under Maximal Update Parameterization (muP) with dense batch-size and learning-rate sweeps. Building on this dataset, we study how the GNS depends on hyperparameters, scale, and training progress, and how it relates to the CBS. We find that the GNS behaves as a consistent “temperature of training”, remaining stable across model sizes under muP at equivalent loss, and our results suggest task-dependent biases in its relation to the CBS: a tendency to overestimate the CBS for classification and to underestimate it for generative tasks, consistent in direction across common CBS definitions. The dataset is available at this https URL
[LG-32] Distribution-Free Uncertainty Quantification for Weather Forecasting: Conformal Prediction Under Spatiotemporal Dependence
链接: https://arxiv.org/abs/2611.24329
作者: Mariya
类目: Machine Learning (cs.LG)
*备注: 27 pages, 5 figures. Code: this https URL
Abstract:Conformal prediction gives finite-sample, distribution-free marginal coverage and has begun to be applied to gridded weather forecasts. Marginal coverage concerns the long-run fraction of covered points at a location, not how coverage behaves jointly across space and time on a given day. I benchmark eight interval-construction procedures (Gaussian, pooled and per-cell split conformal, per-season, adaptive conformal inference and rolling windows) for gridded 2 m temperature forecasts at eight lead times from 6 hours to 10 days, using ERA5 at 5.625 degree resolution, a deliberately modest convolutional forecaster and ECMWF’s operational HRES, with calibration in 2017-2018 and testing in 2019-2021. Alongside coverage, width and interval score, I report dependence diagnostics: a spatial overdispersion index, quantiles of daily spatial coverage and miss persistence relative to a shuffled-days null. Per-cell methods reach near-nominal marginal coverage, yet the day-to-day variance of the fraction of cells covered is roughly 3 to 10 times what independent cells would give, and misses cluster in time; both effects grow with lead time for the CNN, also appear for HRES, and are shared by all methods. An oracle that divides each day’s normalised errors by that day’s mean removes 81 to 92 percent of the excess dispersion for the CNN and 93 to 98 percent for HRES, so days of large errors across much of the grid account for most of it. A day-scale predicted from seasonal terms and already-verified errors lowers the overdispersion index without widening intervals but improves interval score by at most 0.6 percent; on HRES from 12 hours to 3 days it has the lowest overdispersion index while ACI or season-conditional calibration has the best interval score, so no method is best on both. The study covers one variable, one coarse grid, a reanalysis target and two forecasters.
[LG-33] Structure-Preserving Neural Approximation of Convex Semiconvex and Quasiconvex Functions
链接: https://arxiv.org/abs/2611.24325
作者: Yeachan Park,Yeoneung Kim
类目: Machine Learning (cs.LG); Optimization and Control (math.OC)
*备注:
Abstract:Input convex neural networks provide a natural way to enforce convexity globally through architectural constraints. We study how this structure-preserving approximation framework extends to semiconvex, quasiconvex, and convex transformable functions. For C^1,1 convex functions on compact convex domains, we establish the sharp W^1,p minimax rate m^-1/d under max-affine approximation with m affine pieces, and show that every max-affine function admits an exact bounded-width ICNN representation with complexity linear in m . The same W^1,p approximation rate extends to semiconvex functions through quadratic lifting and to convex transformable functions through monotone composition. For general Lipschitz quasiconvex functions, we develop a quantitative finite-probe approximation theory based on monotone responses to affine projections and obtain a structure-preserving IqCNN with an L^\infty approximation guarantee. We further show that these approximation results extend to ReLU-transferable smooth activations, including Softplus and ELU. Numerical experiments on PDE-based approximation problems compare the proposed architectures with parameter-matched MLPs and validation-tuned shape penalties, demonstrating global structural validity without penalty tuning while maintaining competitive approximation accuracy. Overall, the results identify recoverable convex latent structure as the key mechanism enabling gradient-level structure-preserving approximation beyond convexity.
[LG-34] RespCCT and FreqCCT: Compact Explainable and Cost-Aware Convolutional Transformers for Multi-Class Respiratory Disease Classification
链接: https://arxiv.org/abs/2611.24323
作者: Auntor Chakma,Mst. Farhana Akter,Abu Henaf Rashid Ahmad Mufti,Asmana Yasmin
类目: ound (cs.SD); Machine Learning (cs.LG)
*备注: 22 pages. Manuscript to be submitted to Biomedical Signal Processing and Control
Abstract:Chest auscultation depends on the listener, and software that classifies lung sounds could give a second reading where trained listeners are scarce. The most accurate published classifiers fine-tune large pretrained networks, and their accuracy is rarely reported with running cost or paired statistics. This study makes three contributions. (i) It proposes RespCCT and FreqCCT, two compact convolutional transformers of 173,187 and 124,999 parameters that learn from random initialization on mel, constant-Q and cepstral images of 5 s frames from three public databases, ICBHI, KAUH and RD@TR, with 1,665 audio files in seven classes. (ii) It compares both models with five ImageNet-pretrained networks and larger distilled variants on one test set, using paired bootstrap intervals. (iii) It measures the cost of every pipeline stage on one CPU thread, tests whether attribution maps depend on the trained weights, and probes whether the learned features reveal the recording source. On the test fold, RespCCT reaches an accuracy of 98.13% and a macro F1 of 0.970, and FreqCCT reaches 97.58% and 0.951. Over five folds, RespCCT with three seeds reaches 97.65% and 0.959, and FreqCCT with one seed reaches 96.95% and 0.941. The two strongest pretrained networks reach a macro F1 of 0.984, whereas RespCCT needs 2.0 to 64.6 times less weight storage. Computing the three input images takes longer than running either model, and a linear probe identifies the source database with 0.958 accuracy. A team can weigh accuracy against storage and time before device testing.
[LG-35] Supervised Hebbian Class Networks: Learning Without Backpropagation
链接: https://arxiv.org/abs/2611.24314
作者: John Loverich
类目: Machine Learning (cs.LG)
*备注:
Abstract:This paper introduces Supervised Hebbian Class Networks as a method for training multi- layer perceptrons and convolutional networks without gradient calculations or backpropa- gation. Each neuron in a learned layer is assigned a class, while connections follow the usual fully connected or convolutional pattern. The method combines a class-based supervision rule shared across layers with a Lion optimizer adapted to Hebbian activity products. All layers train together throughout the run, and their activities provide class predictions di- rectly, without a separately trained classifier. The experiments test the method on MNIST with a multilayer perceptron and on CIFAR-10 with a convolutional networ
[LG-36] Element-Wise Femoral Stress Prediction: A Hybrid Graph Neural Network Surrogate for Finite Element Analysis
链接: https://arxiv.org/abs/2611.24274
作者: Sikha O K,Abdollah Moslemi,Elham Alizadeh,Ludovic Humbert,Miguel A. Gonzalez Ballester,Jerome Noailly
类目: Machine Learning (cs.LG)
*备注: This paper is submitted to Frontiers in Artificial Intelligence
Abstract:Hip fractures represent a significant clinical challenge because they cause excess disability and mortality, and have a growing incidence in aging populations, as bone fragility increases. Although finite element analysis (FEA) provides biomechanical assessment of femoral strength, its clinical applicability is mainly limited by high computational cost. Conventional machine learning (ML) approaches have been explored as surrogate models for FEA; predictions are fast but fail to capture the spatial and topological structure of finite element meshes. To address these limitations, this study proposes a hybrid Graph Neural Network (GNN)-based framework for femoral stress prediction using graph-structured finite element modeling (FEM) data. The proposed GCN-SAGE-Residual architecture combines graph convolution, neighborhood aggregation, and residual learning to model spatial dependencies and stress propagation across mesh elements. We systematically evaluated 14 models in a primary comparison, spanning classical ML methods and baseline GNN architectures, complemented by a targeted architectural ablation that isolates the contribution of each component of the proposed design. Evaluation results demonstrate that the proposed hybrid architecture achieves the strongest predictive performance among all evaluated models. The proposed hybrid model achieves the best performance with an R^2 score of 0.8022 and mean absolute error of 2.879 MPa, highlighting the importance of incorporating mesh topology in biomechanical prediction tasks. The proposed framework provides a scalable and computationally efficient alternative for FEM-based orthopedic analyses and fracture risk assessment. The code is publicly available on GitHub.
[LG-37] RESS: A Fixed-Size History Representation for Irregular Time Series Forecasting
链接: https://arxiv.org/abs/2611.24257
作者: Shusen Ma,Yangchun Xie,Xudong Zuo,Haizhou Liang,Yun-Bo Zhao
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:A fixed number of recent observations may span minutes for one variable and days for another. Earlier measurements can still reveal whether the latest value is unusual for that variable. We propose TRESS (Timestamped Recent Events and Statistical Summaries), a forecasting model that retains the latest observations at their original times and summarises the full history of each variable. For a given set of variables and event budget, this representation stays the same size as the record grows. Its query-conditioned event lifting module weights deviations from the latest value according to the age of each observation and the forecast horizon. A channel-specific decoder combines the resulting features with historical statistics and a summary shared across variables to predict a correction to the latest value. Retaining 8 to 16 observations per variable, TRESS records the lowest mean test MSE and MAE among the methods compared on all four benchmarks. Serving 32 successive query times from one encoded history reduces CPU inference time by a factor of 2.12–2.60 across the four datasets. The code is at this https URL.
[LG-38] Continuous Noisy-Loop Models
链接: https://arxiv.org/abs/2611.24213
作者: An T. Le
类目: Machine Learning (cs.LG); Information Theory (cs.IT)
*备注: 51 pages, 7 figures
Abstract:Efficient learning and inference need not imply a compact probabilistic circuit. We quantify this distinction for continuous noisy readings of agreement between adjacent latent spins on a known square grid with d edges. With known margins and bounded Lipschitz channels, minimax squared Hellinger risk is \Theta(\min\1,dN^-2/3) from N independent vectors, without a reliability lower bound. A spline estimator reaches this rate in polynomial arithmetic time given a fixed positive reliability bound; an exponential-size mixture removes that requirement. At fixed positive reliability, nonnegative decomposable circuits need \exp(\Theta(\sqrt d)) nodes for fixed total-variation error below one, even when each leaf uses at most \alpha d coordinates, 0\alpha1 . Optimal Kullback–Leibler (KL) approximation error is \Theta(d/\max\log d,\sqrt\ell) in either direction with at most d^r nodes and leaf scopes \ell\leq\alpha d , for fixed r1 and 0\alpha2/3 . Node counts exclude leaf computation. For three fixed masks, optimal joint conditional KL has order d/\log d when \ell\leq(\log d)^2 . Local cycle identities relate joint divergence to identifiable channel products and separate weak-channel recovery from density estimation. Factorized evidence preserves a planar zero-field Ising model: given local integrals, exact queries cost O(d^3/2) arithmetic operations and one root extraction.
[LG-39] Learning with Unlearning: Synaptic Role Partitioning for Simultaneous Continual Learning and Machine Unlearning
链接: https://arxiv.org/abs/2611.24205
作者: Mohammad Ali Vahedifar,Behzad Bozorgtabar,Qi Zhang
类目: Machine Learning (cs.LG)
*备注:
Abstract:Continual Learning (CL) asks a model to acquire new knowledge without forgetting what it already knows. Machine Unlearning (MU) asks it to destroy specific knowledge without damaging the rest. The challenge is integrating them in the same system; the two objectives pull on the same weights, and an edit made for one harms the other. To address this, we propose Learning with Unlearning (LwU), a unified framework that resolves the conflict where it occurs: at the individual synapse. LwU scores every weight twice, once against the data to keep and once against the data to delete, and partitions the network into four zones with separate update rules. Weights that only require retention are frozen. Weights that only require deletion are cleared and returned to the network as free capacity. Contested weights take a gradient-ascent step projected orthogonally to the retention gradient. The remaining capacity learns the incoming task by a privileged teacher via self-distillation. Deletion is served post hoc, and no retraining is required. Across both CL and MU jobs, LwU outperforms or matches the strongest baseline. Its real-world practicality has been demonstrated on the Physical Artificial Intelligence (PAI) dataset we conducted. Additionally, in the image generation task and in removing dangerous content, it outperformed the closest baseline, as shown in Fig.1.
[LG-40] Adaptive Action Horizon Selection via Flow-Field Uncertainty IROS2026
链接: https://arxiv.org/abs/2611.24178
作者: Idan Ben-Ami,Arnon Netzer,Sarah Keren,Idit Diamant
类目: Machine Learning (cs.LG); Robotics (cs.RO)
*备注: IEEE IROS 2026 Workshop
Abstract:Reliable uncertainty estimates are important for adaptive execution in vision-language-action (VLA) models. Existing approaches such as Adaptive Action Chunking (AAC) estimate action entropy from multiple sampled action chunks, introducing additional inference cost and capturing both uncertainty within an action trajectory and diversity across alternative strategies. We introduce Divergence-based Uncertainty-aware Adaptive Action Chunking (DUAC), a post-hoc method that extracts trajectory-conditioned predictive uncertainty from the vector field of a pretrained VLA. Building on a closed-form flow-posterior covariance formulation, DUAC derives action-level uncertainty scores from a single generated chunk using Jacobian-vector products, without retraining or multiple endpoint samples. DUAC strongly correlates with AAC, while capturing distinct information: AAC measures global dispersion across sampled chunks, whereas DUAC measures local uncertainty around the trajectory selected for execution. Consequently, DUAC is less sensitive to diversity among distinct but valid action strategies. Across LIBERO, LIBERO-Pro, LIBERO-Plus, and RoboCasa, DUAC-based adaptive horizon selection performs favorably among the evaluated methods, combining strong task success with low uncertainty-estimation latency. These results establish the flow vector field as an efficient, trajectory-conditioned source of predictive uncertainty for robot decision making.
[LG-41] SoU-RHS: Bayesian Structured Sparsity for Compact Architecture Recovery in CNNs
链接: https://arxiv.org/abs/2611.24166
作者: Rafael Mouallem Rosa(STATIFY),Julyan Arbel(STATIFY)
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:Highly parameterized convolutional neural networks often achieve strong predictive performance, but their memory, computation, and deployment costs motivate the search for methods that recover compact architectures from large reference models. We propose SoU-RHS, a Bayesian structured-sparsity procedure that augments Regularized Horseshoe Shrinkage (RHS) with the so-called Sources of Uncertainty Sparse Global-Local mechanism (SoU-SGL). The resulting model sharpens the separation between active and inactive filters, providing an effective mechanism for architecture recovery. A key property of SoU-RHS is stability: across different overparameterized starting widths and under a reasonable range of pruning thresholds, it recovers compact architectures with similar total size and layer-wise allocation. This supports the interpretation that the procedure is identifying a compact structure adapted to the data, rather than merely compressing each initial model separately. Across standard image-classification benchmarks, SoU-RHS recovers substantially smaller architectures than deterministic structured-sparsity baselines, while maintaining competitive accuracy, achieving lower negative log-likelihood in every evaluated configuration and generally lower expected calibration error.
[LG-42] QuantAtlas-OM: Taming Neural Spectral Bias in Long-Horizon Spatiotemporal Ocean Dynamics
链接: https://arxiv.org/abs/2611.24124
作者: Hao Wu,Yuan Gao,Tong Wu,Fan Xu,Fan Zhang,Jinhan Li,Kun Wang,Zhenyu Ma,Bryan Dai,Xiaomeng Huang,Qingsong Wen
类目: Machine Learning (cs.LG)
*备注:
Abstract:Autoregressive neural networks exhibit spectral bias in long-horizon spatiotemporal prediction. As rollouts progress, high-frequency details decay, blurring the predicted fields and destabilizing long-range forecasts. Ocean dynamics offer a particularly stringent test because kinetic energy is concentrated in mesoscale eddies and fronts spanning tens of kilometers. This degradation stems from coupling large-scale phase evolution and small-scale energy retention in a shared feature space. Evolution on a coarse grid smooths out high-wavenumber eddies, whereas evolution on a fine grid amplifies small-scale phase errors and triggers rollout instability. We introduce QuantAtlas-OM (QuantAtlas Ocean Model), a scale-spectral decoupling framework that confines deep temporal evolution to a low-wavenumber latent manifold for long-term stability while routing full-resolution eddy and front residuals through an outer bypass to the decoder. Across physical regimes, the evolution core pairs non-local attention with resolution-invariant dilated spatial gating. Across three western boundary currents, QuantAtlas-OM retains 43.7% to 86.5% of mesoscale spectral energy along the k^-3 cascade at Day 120 , sustains anomaly correlation coefficients of 0.81 to 0.88 at Day 60 , and reaches 0.66 at Day 120 in the Kuroshio ( 2.5\times to 7.7\times the best baselines). Trained at 0.25^\circ , QuantAtlas-OM transfers zero-shot to a global 0.125^\circ grid ( 0.81 Day- 10 spatial correlation) and leads global 0.25^\circ , 23 -level 3D ocean simulations ( 120 days) and coupled ocean-atmosphere forecasts ( 60 days), ranking first on 15 of 16 metrics and cutting 60 - to 120 -day salinity RMSE by 13% to 27% across global ocean basins.
[LG-43] VESPA: Verifiable Symbolic Policy Adaptation for Safe Reinforcement Learning
链接: https://arxiv.org/abs/2611.24083
作者: Paul König,Sascha Marton,Stefan Lüdtke,Christian Bartelt
类目: Machine Learning (cs.LG)
*备注:
Abstract:Safe reinforcement learning (RL) aims to prevent agents from taking actions that are dangerous or destructive to their environment. This is especially important for safety-critical domains, such as robotics or autonomous driving. Formal verification can be used to prove agent safety through safety constraints. However, in practice, many policies do violate safety constraints, and thus are verified to be unsafe. Since the goal is to obtain a safe policy, we would need to retrain a new policy from scratch and then verify it again until a candidate happens to pass. Both training and verification are usually not cheap, which makes this process inefficient. We introduce VESPA ( \textbfVE rifiable \textbfS ymbolic \textbfP olicy \textbfA daptation), a framework for targeted fine-tuning of decision tree policies. We show that using counterexamples for policy adaptation can make existing unsafe policies safe, or at least make considerable progress towards safety. Targeted fine-tuning of the existing policy on those counterexamples enables it to learn how to avoid violating safety constraints. We thus use the result of a formal verification as a training signal for fine-tuning. On CartPole, VESPA raises the mean number of verified safe steps from 63.8 to at least 92.7, reaching after two iterations what the same loop without counterexamples reaches after ten. On MountainCarContinuous, VESPA turns an unsafe policy into a safe policy in 53 of 60 runs. We conclude that counterexamples from a verifier are an effective training signal, which can be utilized to systematically improve the safety of policies. Our implementation is available at: this https URL
[LG-44] FDM: A Framework for Constrained Control of the 2D Radiative Transport Equation
链接: https://arxiv.org/abs/2611.24070
作者: Hualing Zhong,Shuai Wang,Hui Xie,Heng Yong
类目: Machine Learning (cs.LG); Computational Physics (physics.comp-ph)
*备注:
Abstract:Applying the frozen diffusion model (FDM) [29] to the constrained control of the 2D radiative transport equation presents several challenges. These include the dimensional difference between control variables and state functions, the resolution mismatch between traditional numerical solvers and deep learning models, and solution non-uniqueness. To address these obstacles, we propose a novel approach, FDM+, which includes three key improvements. Firstly, a decoupled prediction architecture is introduced to address the dimensional heterogeneity. Secondly, an efficient parameterization strategy resolves the resolution mismatch. Meanwhile, we derive a total error bound to analyze how the parameterization error and the model estimation error jointly affect the performance. Thirdly, the conditional distribution learned by the diffusion model is leveraged to perform design space exploration for multi-solution problems. A series of numerical experiments demonstrate that FDM+ outperforms existing methods in reconstruction accuracy and design flexibility.
[LG-45] Faster to the Wrong Answer: The Price of Local Updates in Federated Optimization
链接: https://arxiv.org/abs/2611.24057
作者: Molly Wang
类目: Machine Learning (cs.LG)
*备注: 18 pages, 1 figure; includes supplementary proofs and reproduction code
Abstract:Local computation can improve the conditioning of federated updates but may also introduce bias, causing their fixed point to differ from the global minimizer. We characterize the minimum worst-case bias achievable under a prescribed conditioning bound. We consider strongly convex quadratic client objectives with Hessian eigenvalues in [\mu,L] and local updates represented by a shared positive spectral filter. Client heterogeneity is measured by the root-mean-square gradient at the global minimizer. For a uniform surrogate condition number bounded by \chi\in[1,L/\mu] , we establish the exact worst-case bias per unit heterogeneity, (L-\mu\chi)/(2\mu L\sqrt\chi) , and show that local gradient descent with sufficiently small step sizes and proximal updates attain this bound. Our analysis reduces populations of arbitrary dimension, including noncommuting Hessians, to distributions over scalar curvatures. This reduction provides optimality certificates and worst-case constructions involving at most three scalar clients. We further extend the sharp proximal bias bound to smooth strongly convex objectives beyond quadratics and quantify the error introduced by inexact local solves.
[LG-46] Algorithmic Market Making Can Lead to Rough Volatility Through the Order Book
链接: https://arxiv.org/abs/2611.24028
作者: Sai Niranjan Ramachandran
类目: Machine Learning (cs.LG); Mathematical Finance (q-fin.MF); Trading and Market Microstructure (q-fin.TR)
*备注: 65 pages (including appendix) , 2 figures, 20 tables
Abstract:Existing work suggests that markets can show volatility with paths rougher than those of Brownian motion. In theory this can happen when the order flow is close to instability with memory that decays as a power law. However, it is not fully clear how such order flow could arise from the predominantly algorithmic market making that characterizes order book dynamics. We show one such consistent mechanism. We treat each market maker as a desk that uses a first-order gradient rule to move the distances of its orders from the mid price toward target distances provided by an unknown oracle. We assume that distinct desks do not act synchronously in time but act whenever new information relevant to their respective oracles arrives, which each desk samples over its own characteristic lookback horizon. Across all desks the rule therefore takes the form of stochastic gradient descent (SGD) with information-driven (mini-batch) noise. In the continuum limit this update rule makes the volume of the order book follow a conservative stochastic partial differential equation (the Dean–Kawasaki equation) whose fluctuations drive the order flow. The distinct horizons of the desks further induce a persistent memory in the order flow that decays as a power law. Standard scaling arguments then lead to rough volatility. We confirm the key predictions of the model on simulated agents and on six months of Binance data with about 790 million orders. In liquid markets governed mainly by market makers, such as the perpetual futures of Binance, the order books are consistent with how our desks change the book.
[LG-47] OMNIS: Semantic RAN Slicing and Quantization for Dynamic Split Neural Networks
链接: https://arxiv.org/abs/2611.23977
作者: Langtian Qin,Avishkar Dnyaneshwar Kawadgave,Ian Andrew Harshbarger,Leïla Nasraoui,Carla Fabiana Chiasserini,Marco Levorato
类目: Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI); Systems and Control (eess.SY)
*备注:
Abstract:Edge computing enables resource-constrained devices to execute machine learning (ML) applications via task offloading, with radio access network (RAN) slicing playing a crucial role in provisioning joint communication and computing resources. However, existing RAN slicing frameworks rely on static computing models, limiting their ability to exploit dynamic semantic data representations enabled by modern neural architectures. To address this limitation, we propose OMNIS+, a semantic RAN slicing framework for multi-cell edge computing built upon a new generation of dynamic split neural models. OMNIS+ incorporates dynamic neural compression paired with adaptive quantization for task offloading, providing a versatile decision space across communication payload and compute dimensions. We leveraged a novel ``box’’ quantization scheme that markedly enhances resilience to transmission bit errors across diverse compression rates. Considering partial system observability and distinct node objectives, we formulate two interdependent optimization problems: (i)~mobile devices (MDs) maximize inference accuracy under quality-of-service (QoS) and queue-stability constraints by jointly adapting dynamic split deep neural network (DNN) branches and cell association; and (ii)~each edge server (ES) allocates bandwidth, graphics processing unit (GPU) frequency, and modulation and coding scheme (MCS) to maximize the max–min inference accuracy among associated MDs under cell capacity limits. We solve these problems using a multi-agent distributed optimization framework, where MDs act as causal contextual multi-armed bandit (MAB) agents under backlog-aware Lyapunov control, while each ES executes resource allocation via block coordinate descent (BCD). Compared with state-of-the-art baselines, OMNIS+ reduces the QoS violation probability by up to 1.75 while achieving a better accuracy–violation trade-off.
[LG-48] PGU-Decoder: Prior-Guided Unified Decoding for Industrial Road-Network Map Updating KDD2027
链接: https://arxiv.org/abs/2611.23936
作者: Yuguang Ning,Bingbing Kang,Nan Zhao,Zhixuan Li,Ze Zhao,Dongdong Liu,Wenjian Zhu,Zekai Liu,Junhui Wang
类目: Machine Learning (cs.LG)
*备注: 11 pages. Accepted at KDD 2027, Applied Data Science (ADS) Track. This is the original submission version, prior to reviewer revisions
Abstract:Timely road-network map updates are essential to navigation and autonomous driving. Three key tasks are intra-intersection road network generation, merging/diverging detection, and drivable-area detection, covering topology, geometry, and attributes, respectively. Separate pipelines duplicate rule maintenance, quality assurance, and manual verification, increasing operational costs. We propose PGU-Decoder, a production-grade unified decoding framework for semantic prediction across these tasks. A Prior Query Module (PQM) constructs point-level prior queries from sampled locations on road-network geometries. Fixed reference points preserve positional priors throughout decoding, allowing the model to focus on surrounding context. Point-level representations are aggregated into instance-level representations, and a Relational Graph Attention Network (RGAT) captures inter-instance dependencies. Task-specific heads then generate predictions. We validate PGU-Decoder in large-scale automated road-network map updating across more than 360 cities in China, where it consistently outperforms seven baselines. Following deployment at AutoNavi Map, it achieves quality scores above 98% on all three tasks, reduces manual verification costs by 95%, and increases update frequency from weekly to hourly. These results demonstrate a practical and cost-effective industrial solution for road-network map updating.
[LG-49] Accelerating Distributed Digital Ising Machines HPCA
链接: https://arxiv.org/abs/2611.23883
作者: Tong Wu,Ziyad Alswaidan,Shreya Mukherjee,Ting-Yu Lee,Daniel Xu,Atharva Raut,Harry Tye,Abhinav K. Puthran,Kemal Selcuk,Navid A. Aadit,Ken Mai,Prashanth Mohan,Kerem Y. Camsari,Tathagata Srimani
类目: Hardware Architecture (cs.AR); Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)
*备注: 16 pages. Accepted at HPCA
Abstract:Ising machines, dedicated accelerators built around the dynamics of Ising-like spin models, have emerged as a leading platform for combinatorial optimization and probabilistic sampling. The problems that motivate them require millions of spins, far beyond the on-chip capacity of any single device, so a practical Ising machine must distribute its computation across many devices. Prior work reports throughput at a single operating point, leaving the binding resource unidentified and platform crossovers measurable only by experiment; no calibrated cross-platform cost model of distributed Ising machines has been reported, nor have systematic techniques to accelerate their Markov chain Monte Carlo (MCMC) workloads. This work presents a calibrated cost model that decomposes digital Ising sweep time into compute, memory, synchronization, and inter-device communication terms, validated with 4.8% mean error across three GPU generations, FPGA and ASIC up to 1 million spins. Guided by the model, we address each term in turn and propose: (i) a data placement strategy, worth up to 4.2 \times speedup; (ii) multi-rate and bounded-staleness schedules that relax the barrier that Gibbs sampling imposes, cutting time-to-solution (TTS) by up to 3.16 \times on three benchmarks; and (iii) a phase-offset streaming dataflow that hides boundary exchange behind interior computation whenever a link satisfies a latency-transparency criterion, yielding up to 1.25 \times /1.74 \times /2.01 \times on an H200 pair, a ZCU106 pair, and a four-board XEM8320 2\times2 mesh. The streaming dataflow is also implemented on a fabricated four-chip 28 nm ASIC sustaining 28.1 Gflips/s at 4.81 pJ/flip.
[LG-50] A Rank-One Bypass of Fixed Linear Erasure NEURIPS2026
链接: https://arxiv.org/abs/2611.23880
作者: Jai Kumar Sharma,Vanshaj Khattar
类目: Machine Learning (cs.LG)
*备注: Accepted at the NeurIPS 2026 Workshop on Child Safety in AI (poster). 10 pages, 3 figures
Abstract:Linear concept erasure can make a target unreadable to a specified family of linear classifiers applied to a model’s current internal features. In an open-weight release, however, an attacker can modify the layers that produce those features. We study a fixed affine erasure layer whose upstream model is allowed to change. If the layer transmits even one direction visible to an allowed downstream classifier, the attacker can write a target score into that direction and carry it through the layer. We show that this condition is also necessary, and that for a nonconstant linear target score the required change is a rank-one affine update. Across six CIFAR-10 class-pair tasks, the route restores nearly all pre-erasure target performance and can leave a deployed benign linear head’s scores unchanged to numerical tolerance. We then price the attack: with eight target labels and 2,000 unlabelled calibration examples, a 10% mean relative feature edit yields 79.8% test accuracy, which is 91.2% of the fitted score’s gain over chance; tighter edit limits sharply reduce success. Linear guardedness therefore remains a valid guarantee about the current representation, but it is not by itself a tamper-resistance guarantee for an editable open-weight model.
[LG-51] Neural Memory Orchestration
链接: https://arxiv.org/abs/2611.23857
作者: Michael Zizov,Boris Morose
类目: Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG); Operating Systems (cs.OS)
*备注: 10 pages, 9 figures
Abstract:Modern HPC workloads increasingly stall on the memory hierarchy rather than on compute, and page faults from mmap’d files, swap, and tiered memory are a growing fraction of wall time. Heuristic prefetchers in the kernel handle stride-like and temporally clustered patterns well but degrade sharply on the irregular, data-dependent access streams typical of these workloads. Learned prefetchers have been shown to capture such patterns in simulation but have not been taken to a deployed pipeline running against live workloads on real hardware. We present Overseer, an end-to-end learned page-prefetching system. An eBPF collector observes page faults on the target host, a transformer predictor (OverNet) runs on network-attached inference hardware and emits future page addresses, and a kernel-resident enforcer materializes those pages into the target’s address space before they are demanded. OverNet encodes fault sequences through per-instance ordinal ranks over deltas, which lets a single trained model serve any target process without retraining. On two page-fault-heavy workloads, Overseer reduces runtime by roughly 35% on Hilbert-curve traversal and by roughly an order of magnitude on dense matrix multiplication, with corresponding reductions in major-fault counts.
[LG-52] Beyond Cross-Modal Reconstruction: Participant-Level Evaluation of sEMG-Kinematic Coupling in Post-Stroke Gait
链接: https://arxiv.org/abs/2611.23856
作者: Rafsan Jany,Md Rafid Islam,Mohammad Fahim Islam,Hafiz Abdur Rahman
类目: Machine Learning (cs.LG)
*备注:
Abstract:Accurate cross-modal reconstruction is often treated as evidence of physiological coupling, although it may simply reflect temporal patterns shared across participants. We tested whether frozen bidirectional models captured subject-specific neuromuscular coupling between electromyographic activity and gait kinematics in 82 held-out able-bodied and 43 post-stroke participants. Complete source-modality removal and cross-participant source substitution, along with a cycle-position-age-sex template baseline, were applied. Learned bilateral asymmetry was also compared with transparent waveform and derivative asymmetry for predicting Performance-Oriented Mobility Assessment (POMA) scores in the case of repeated nested cross-validation. Complete source removal increased reconstruction RMSE by 6.90% for electromyography-to-kinematics and 11.38% in the reverse direction. In contrast, cross-participant substitution increased RMSE by only 1.68% and 1.67%, respectively, while the template reduced RMSE by 2.89% and 0.55%. For POMA prediction, walking speed, age, and sex achieved an MAE of 3.693, compared with 4.229 and 4.427 after adding absolute and signed learned asymmetry, respectively. In all of the 20 repeated evaluations, all asymmetry representations performed worse than the covariate baseline. No localized learned feature remained significant after covariate adjustment and false-discovery-rate correction. The model mainly relied on shared gait-phase patterns, with no clear subject-specific coupling or added clinical value. Participant-level controls are therefore needed before reconstruction accuracy is treated as evidence of individualized physiology.
[LG-53] A Fast Programmable GPU-to-CPU Invocation Layer for GPU-Initiated Communication
链接: https://arxiv.org/abs/2611.23848
作者: Jiakun Yan,Shiksha Patel,David Sidler,Gengyu Rao,Brandon Potter,Bradford Beckmann
类目: Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)
*备注: 11 pages, 9 figures, 3 tables
Abstract:Modern GPU workloads increasingly rely on device-initiated communication to let GPU kernels communicate directly without synchronizing with the host at kernel boundaries. Existing libraries commonly favor placing transport control on GPU threads, an approach known as GPUDirect Async (GDA), and treat reverse offload to host proxy threads as a slower fallback. We argue that reverse offload can outperform GDA and should be a first-class communication path. We further propose exposing reverse offload as a programming interface that lets users define workload-specific communication operations. We present KIWI, a programmable reverse-offload framework that achieves sub-microsecond one-way device-to-host latency and hundreds of millions of operations per second. We address practical challenges of scaling reverse offload, including contention on shared queues, polling overhead, and load imbalance across proxies. KIWI-P2P, a SHMEM-like library built on KIWI, outperforms state-of-the-art GDA implementations in both latency and bandwidth. More importantly, KIWI-EP redesigns low-latency mixture-of-experts dispatch and combine kernels around workload-defined operations, lowering expert-parallel communication latency by up to 2.3x over the best existing backend on AMD GPUs and time per output token in vLLM by up to 29%. The source code of KIWI is available at this https URL. Comments: 11 pages, 9 figures, 3 tables Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG) Cite as: arXiv:2611.23848 [cs.DC] (or arXiv:2611.23848v1 [cs.DC] for this version) https://doi.org/10.48550/arXiv.2611.23848 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-54] Online State-Representation Adaptation under Synthetic Interaction-Cue Remapping
链接: https://arxiv.org/abs/2611.23836
作者: Bin Yin
类目: Machine Learning (cs.LG)
*备注: 10 pages, 2 figures. Code and synthetic data are available at this https URL
Abstract:When cues change what they predict, continuing to update a learned mapping may help an agent choose the appropriate action. We tested this in a one-step synthetic task by comparing a logistic state predictor that continued to learn from state labels with the same predictor frozen at transfer. We evaluated ten conditions in six held-out environments, with 1,296 paired runs per condition and 1,296,000 transfer records in total. Late action accuracy was 93.39% with online updates and 64.63% with frozen coefficients, a paired difference of 28.76 [8.73, 53.19] percentage points (95% environment-bootstrap interval). A direct policy reached 95.65% using the same predictions with a different action threshold. A contextual bandit reached 94.23% and earned more cumulative utility than the adaptive predictor. The bandit used the reward of its chosen action, which in this task reveals the true state when combined with that action. Weights based on development history lowered cumulative utility relative to neutral weights (difference -3.45 [-4.48, -2.43]). The adaptive predictor outperformed its frozen counterpart in late accuracy in each of the six test environments. The other comparisons show how the choice of action threshold and the information available in feedback affect what can be attributed to online adaptation.
[LG-55] CAFE-AQI: Coverag e-Aware Day-Ahead AQI Forecasting with Early Warning and Conformal Uncertainty in Bangladesh
链接: https://arxiv.org/abs/2611.23815
作者: Md Rafid Islam,Rafsan Jany,Rafid Bin Zaman,Mohammad Fahim Islam,Hafiz Abdur Rahman
类目: Machine Learning (cs.LG); Applications (stat.AP)
*备注:
Abstract:Next-day air quality forecasting often assumes city records are equally complete and independent. However, an audit of a large Bangladesh dataset revealed a sharp coverage expansion after August 2022 and showed that 103 city labels correspond to only 61 unique AQI trajectories. To address these issues, this paper presents CAFE-AQI, a coverage-aware framework for forecasting next-day maximum AQI. It also estimates the probability of Unhealthy-or-worse conditions (AQI = 151). Data were divided chronologically into training (2022-2023), calibration (2024), and testing (2025). To prevent duplicate records from dominating training and evaluation, identical trajectories were weighted. The framework combines temporal and pollutant features with a five-neighbor spatial context. It also includes a class-balanced warning head and risk-conditional conformal calibration aimed at providing early warnings and prediction intervals. Evaluation on the duplicate-balanced 2025 data gave an MAE of 9.76, an RMSE of 15.04, and an R-squared value of 0.910 for CAFE-AQI. Its MAE was 3.63 AQI lower than persistence (95% CI: 3.49-3.76), with lower errors in each of the 61 trajectory groups. For unhealthy-air warnings, the F1-score was 0.908 and the AUROC was 0.982. The nominal 90% intervals covered 92.09% of observations. When an entire spatial block was excluded from training, the MAE remained 10.66, showing that the lightweight model transferred reasonably well to unseen locations.
[LG-56] abBioMed: A Large-Scale Benchmark for Biomedical Tabular Learning NEURIPS
链接: https://arxiv.org/abs/2611.23779
作者: Pau Mateo-Bernadó,Saivenkata Nagavyjayanthi Polapragada,David Bonet,Pol Arbiol Rakuljic,Laia Mogas Pladevall,Marçal Comajoan Cara,Jesus Gonzalez-Ferrer,Jordi Abante,Daniel Mas Montserrat,David Haussler,Alexander G. Ioannidis
类目: Machine Learning (cs.LG)
*备注: 87 pages, 43 figures, including appendices and the NeurIPS paper checklist
Abstract:Recent advances in machine learning have accelerated biomedical prediction, but evaluation on structured biomedical data remains fragmented, especially for datasets with high dimensionality, large sample size, missingness, and class imbalance. We introduce TabBioMed, a large-scale benchmark of 94 curated public biomedical tabular datasets spanning electronic health records, drug discovery, genomics, transcriptomics, proteomics, metabolomics, single-cell omics, and systems biology. TabBioMed unifies these datasets through a standardized, dataset-aware preprocessing and evaluation framework, enabling reproducible comparison across classification, regression, and multi-target tasks. We benchmark 23 models across classical baselines, gradient-boosted trees, neural tabular architectures, and tabular foundation models. Foundation models deliver the strongest family-level predictive performance. However, their gains are regime-dependent: tree-based models and tuned multilayer perceptrons remain highly competitive, often dominating under inference-time or training-cost constraints. Overall, TabBioMed shows that biomedical tabular learning is strongly context-dependent, with optimal model choice depending on data regime, task type, and computational budget. We release TabBioMed with curated datasets and splits, preprocessing code, baselines, and results, providing a reproducible benchmark for practical model selection and future methods development in biomedical machine learning, available at this https URL.
[LG-57] A Systematic Analysis of a Small Optimal Subspace for Physics-Informed Neural Networks: n-DAS subset mathbbRp
链接: https://arxiv.org/abs/2611.23773
作者: Ahmad AliAhmad,Khemraj Shukla
类目: Machine Learning (cs.LG); Analysis of PDEs (math.AP); Optimization and Control (math.OC)
*备注: 57 pages, 17 figures, 7 tables. In the title, n-DAS denotes an optimizer of order n restricted to the dominant aligned subspace (DAS): 1-DAS for first-order and 2-DAS for second-order methods
Abstract:The search for a low-dimensional subspace that governs neural network training has been a persistent goal in optimization. Yet subspaces defined by Hessian eigenvalue magnitude, such as the dominant/bulk decomposition underlying Dom-SGD and Bulk-SGD, have repeatedly failed to capture where optimization takes place, and adaptive Krylov-based alternatives built on energy-fraction thresholds show promise but still struggle at lower eigenmodes. We argue this failure is structural: the directions of highest curvature are not, in general, the directions along which training moves. We introduce the dominant aligned subspace (DAS), constructed from a full eigendecomposition of the Hessian but selected by gradient alignment rather than eigenvalue magnitude, as a tool for understanding optimization dynamics and a foundation for designing methods that exploit them. To expose this structure under the hardest conditions, we use physics-informed neural networks (PINNs) as a testbed: differential operators from the parabolic, elliptic, and hyperbolic classes induce severely ill-conditioned, anisotropic loss landscapes. Across Poisson, heat, wave, and viscous Burgers benchmarks DAS-restricted training with a line search matches full-space training even where the projected Hessian remains ill-conditioned, showing empirically that convergence does not require all Hessian eigendirections but only a low-dimensional, gradient-aligned subset. We extend the analysis to Gauss-Newton within the DAS and discuss what this structure implies for future subspace-aware optimizers.
[LG-58] When Normal-Only Updates Are Not Enough: Test-Time Adaptation in Reconstruction-Based Time-Series Anomaly Detection
链接: https://arxiv.org/abs/2611.23761
作者: Jianhuan Mao,Mengxiao Zhu,Yue Pei,Lei Li,Haogang Zhu
类目: Machine Learning (cs.LG)
*备注:
Abstract:Normal operating conditions can evolve after deployment, motivating test-time adaptation (TTA) of time-series anomaly detectors using incoming data. Yet the same stream may contain anomalies, so TTA with online updates must decide which test samples should be used for adaptation. This leaves a more basic question: even if updates are restricted to samples labeled as normal in the benchmark, does TTA reliably improve anomaly detection? We study this question with a controlled intervention that replaces the practical update selector with a rule that uses only samples labeled as normal in the benchmark, while keeping the rest of the online adaptation procedure unchanged. Across nine benchmark families, four optimizer configurations, and five random seeds, we obtain three main findings: (1) using only samples labeled as normal in the benchmark does not reliably improve detection; (2) practical TTA failures exhibit two distinct responses to this intervention, with 63 of 180 settings recovering a clear detection gain while 90 still show no clear detection gain; and (3) the updates reduce their immediate reconstruction objective across the evaluated settings, yet these reductions do not consistently lead to better anomaly ranking. Both failure patterns persist across alternative practical adaptation variants and modest perturbations of the benchmark labels. Together, these findings show that improving sample selection alone is not a complete solution to reconstruction-based TSAD TTA.
[LG-59] Same Rank Different Needs: Rethinking LoRA Redistribution in Federated Learning
链接: https://arxiv.org/abs/2611.23742
作者: Jinqian Chen,Jihua Zhu,Chang Liu
类目: Machine Learning (cs.LG)
*备注:
Abstract:Federated low-rank adaptation (LoRA) enables clients with different resource budgets to collaboratively fine-tune foundation models. To distribute an aggregated update within each client’s rank budget, existing methods commonly retain its leading singular directions. However, directions that best reconstruct the update matrix need not best preserve its responses on a particular client’s inputs: a lower-energy direction may matter more to that recipient. We formalize this mismatch and show that matrix-optimal truncation can incur arbitrarily larger response error than another readout of the same rank. Motivated by this result, we propose Receiver-Aware Subspace Adaptation (RASA), a plug-in framework for receiver-aware knowledge transfer. RASA uses compact diagonal input second moments to perform uncentered PCA of receiver-weighted update responses. The resulting readout is globally optimal for the rank-constrained receiver objective and preserves at least as much response energy as spectral truncation under the diagonal metric. RASA also introduces spectral-reference aggregation, which removes the personalized initialization offset while retaining the locally learned increment. Experiments on DomainNet-100, NICO++, GLUE, and SuperNI demonstrate RASA’s effectiveness across methods under resource, task, and data heterogeneity. In particular, RASA improves DomainNet-100 accuracy by 3.23 percentage points with FlexLoRA and achieves gains of up to 8.33 ROUGE-L points on seen SuperNI tasks and 4.37 on unseen tasks with modest overhead.
[LG-60] HypoLoop: Revising Weight-Editing Plans from Scientific Feedback
链接: https://arxiv.org/abs/2611.23718
作者: Mengran Li,Xichen Yan,Bo Li,Chengyang Zhang,Jinfeng Xu,Zheng Lin,Zelin Zang,Zhen Lei,Jiebo Luo,Victor C. M. Leung,Stan Z. Li
类目: Machine Learning (cs.LG)
*备注:
Abstract:Scientific experiments continually reveal prediction errors. Effective updates depend on which parameters change and which errors drive the correction. We introduce HypoLoop, a family of native weight-editing methods that connects these choices to experimental feedback. Fixed, numerically adaptive, and LLM-guided policies produce editing plans; numerical solvers compute weight patches, and internal validation selects the deployed predictor. Successive observations and validation records support revisions to editable locations and fitting rules. Experiments cover 22 scientific tasks and both DeepSeek and Claude. On Burgers, reallocating a matched parameter budget reduces error by 58.80% and 47.72% relative to fixed locations. On DIODE, positive-residual weighting reduces absolute relative error (AbsRel) by 13.35% in internal cross-validation, with the same 33 editable coordinates. Revising the complete plan across rounds lowers final AbsRel by 10.59% relative to keeping the initial plan fixed. SpaceNet7 and Make3D connect accepted edits to improved spatial predictions. These results show how experimental evidence can shape the update problem itself. Code: this https URL.
[LG-61] Measuring the Conservatism of Latent Safety Filters for World Models
链接: https://arxiv.org/abs/2611.23685
作者: Johannes Mootz,Mani Amani,Reza Akhavian
类目: Robotics (cs.RO); Machine Learning (cs.LG); Systems and Control (eess.SY)
*备注:
Abstract:A latent safety filter decides from the latent of a world model whether a planned action is safe. Conformal calibration sets the threshold of the filter so that it accepts an unsafe action only with a small probability. However, the filter can still be conservative and refuse many safe actions, depending on the latent. To measure the conservatism that a latent forces on the filter, we group similar latents and actions into cells. A filter that gives one decision per cell can refuse the unsafe transitions of a cell only by refusing the safe ones in it as well. We call the share of safe transitions that lie in such cells the overlap. We prove that the overlap is the minimum share of safe transitions that such a filter has to refuse in order to accept no unsafe transition in the data. Both the overlap and the conformal calibration can be applied to any latent. We use this to compare three existing designs, in which the safety labels train a classifier on top of the world model, the world model itself, or a separate encoder beside it. In a visual navigation task, the separate encoder has the lowest overlap, and at the same conformal level its filter refuses the fewest safe actions and lets the planner reach the most goals of the three.
[LG-62] Variable Length Diffuse Everything: Arbitrary Length Generative Modeling on Arbitrary State Spaces
链接: https://arxiv.org/abs/2611.23684
作者: Kevin Rojas,Felix X.-F. Ye,Molei Tao
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:Diffusion and flow-based generative models have achieved remarkable results across diverse applications, but most assume data of fixed-dimension/length. Meanwhile, real-world problems such as layout generation often involve variable-length, multimodal data, such as continuous bounding box coordinates paired with discrete types. Existing methods typically address variable length by padding data to a fixed size or by introducing specialized objectives to model lengths explicitly, adding complexity and limiting generality. We propose Variable Length Diffuse Everything (VL-DE), a simple and general framework for variable-length multimodal generation on arbitrary state spaces. Instead of padding, VL-DE handles variable length through an insertion-based mechanism grounded in a principled theoretical framework, yielding a tractable training objective. In addition, by utilizing the fact that VL-DE is based on a generator with an additive form, we develop a higher-order sampler by appropriately handling the time dependence. Extensive experiments on layout and molecule generation demonstrate the effectiveness of our approach.
[LG-63] Simpler Tiny Recursive Models
链接: https://arxiv.org/abs/2611.23677
作者: Torben Glass,Zhengxi Zhang,Paul Swoboda
类目: Machine Learning (cs.LG)
*备注:
Abstract:Small models that execute deep looped computational graphs have shown great experimental results on puzzle-like problems like Sudoku, finding shortest paths through mazes, etc. The most well-known one is TRM (tiny recursive model), a simplification of HRM (hierarchical reasoning model). However, the underlying recursion is complicated and involves a high-level and a low-level state, a nested loop recursion, and the network calls with different input and output signatures depending on where we are in the loops. These mechanisms are needed to avoid memorization and overfitting. We propose STRM, a simplified design with a single state and a uniform update. STRM does not sacrifice generalization, improves training speed and final performance. Our key architectural choices that allow our simpler recursion to work are exponential moving updates on the states with varying rates that are jittered during training and readout of the solution on slow-updating channels only. We test STRM on Sudoku-Extreme, PPBench and Maze-Hard with the same data, parameters and compute as TRM: we improve over TRM by 3.8 points on Sudoku-Extreme and 8.4 points on PPBench and match it on Maze-Hard. On ARC-AGI-1 and ARC-AGI-2, STRM improves over the published TRM results by 5.4 and 0.5 points. We will release our code upon acceptance of the paper.
[LG-64] Finetuning VLAs for Dexterous Manipulation: The Right Checkpoint Is All You Need
链接: https://arxiv.org/abs/2611.23676
作者: Artur Kuramshin,Charlotte Cloutier,Michael Przystupa,Kirsty Ellis,Mohammad S. Akhter,Glen Berseth
类目: Robotics (cs.RO); Machine Learning (cs.LG)
*备注: Accepted to the Workshop on Dexterous Manipulation at CoRL 2026
Abstract:Vision-Language-Action models have demonstrated strong manipulation capabilities. A limitation of VLA models is that most manipulation datasets focus on parallel-jaw gripper systems, making transfer to multi-finger hands challenging, particularly as teleoperation datasets are harder to collect with multi-fingered robots. In this work, we study how to best adapt VLAs to dexterous hands by pairing supervised fine-tuning (SFT) with diffusion-steering reinforcement learning (DSRL). Using \pi_0.5 and a LEAP hand, we evaluate five fine-tuning recipes across two simulation tasks using only a handful of demonstrations. Our empirical analysis yields three key insights: (1) SFT success is poorly correlated with post-DSRL success, meaning the best SFT checkpoint is rarely the most effective to steer; (2) freezing the vision-language model (VLM) and tuning only the action expert produces the most steerable policy for both training and out-of-distribution placements; and (3) low-rank adaptation (LoRA) is the least steerable approach. Consequently, we recommend an adaptation recipe that freezes the VLM, fully trains the action expert, and eschews SFT rollout success as a checkpoint selection metric.
[LG-65] CratesBump: A Version-Bump Dataset for the Rust Ecosystem
链接: https://arxiv.org/abs/2611.23664
作者: Muhammad Hassnain
类目: oftware Engineering (cs.SE); Machine Learning (cs.LG)
*备注: 5 pages, 1 figure, 3 tables. Dataset and tool: this https URL
Abstract:We release CratesBump, the first public dataset for predicting version bumps in the Rust ecosystem, together with semver-extract, the tool that builds it. CratesBump is built from every crate on this http URL with at least two releases before 14 September 2026 (about 220,000 crates), giving over two million consecutive release pairs. Each pair has 41 features (code changes from AST diffs, dependency changes, complexity, release cadence, GitHub commit and issue activity), its source repository where listed, and two labels: the literal reading of this http URL, and Cargo’s rule, under which a 0.y bump is a breaking change. semver-extract regenerates the dataset from public this http URL data and the GitHub API. To illustrate its use we (i) benchmark bump predictors on held-out repositories, finding the task learnable but far from solved, with lower scores on later releases; (ii) measure which feature groups carry the signal; (iii) check, against declared dependency ranges, how often Cargo’s rule shields dependents from pre-1.0 breaking releases; and (iv) audit a sample of releases with cargo-semver-checks, finding minor and patch releases that break the public API despite their version number. Dataset and tool are released under open licenses.
[LG-66] Same Model Different People: Numerical Execution Contracts and the Reproducibility of Fairness Audits
链接: https://arxiv.org/abs/2611.23657
作者: Erzheng Zhang,Agnes Chu
类目: Computers and Society (cs.CY); Machine Learning (cs.LG)
*备注: 47 pages, 2 figures, 35 tables. Under review
Abstract:Fairness audits treat a trained model as a fixed decision function, but the arithmetic that evaluates it is part of that function. This study holds the model, records, threshold and audit code fixed and varies the numerical execution contract in two tiers. Tier 1 changes only how products and sums are computed, with the serialized weights byte-identical, and is the causal test of arithmetic. Tier 2 runs reduced-precision deployment configurations (BF16 compute, FP8 and 8-bit weights), which can also round weights and change engines and kernels, so its effects belong to the configuration. It audits three tabular model families and two open-weight language models (4B and 24B parameters) on four public tasks, including hospital readmission, on CPUs and GPUs from NVIDIA, AMD and Apple. Tier 1 changed decisions near the threshold, and which ones depended on the GPU and its software stack, but no verdict under the prespecified, unweighted tests. Tier 2 configurations changed more decisions and some diagnostic verdicts, but no four-fifths verdict. Serving the 24B model with vLLM FP8 selected 54% of people rather than 45% and passed an employment-status audit by sex that its strict FP32 reference failed: its demographic-parity and equalized-odds gaps fell from 0.0567 and 0.0515 to 0.0417 and 0.0394, against a diagnostic tolerance of 0.05. At the reference’s selection count FP8 failed demographic parity, and no reading gave its equalized-odds change a paired interval excluding zero. The declared audit depends on the configuration; FP8 is not shown to be fairer. At the frozen thresholds, every pass-fail verdict a configuration changed had a sampling interval containing its criterion. It proposes an audit stability protocol in which the auditor declares contracts, groups, weights and tests before scoring and records each verdict’s range over them apart from its sampling uncertainty.
[LG-67] When Do LLM s Model the Right Target? Problem Formulation in High-Stakes Risk Prediction
链接: https://arxiv.org/abs/2611.23638
作者: Jennifer Zhang,Amanda Coston
类目: Computers and Society (cs.CY); Machine Learning (cs.LG); Methodology (stat.ME)
*备注:
Abstract:Generative AI is increasingly used to assist with data science tasks, yet fundamental questions remain about its ability to correctly formulate analytical problems. We investigate how well large language models (LLMs) can formulate prediction problems when developing models for high-stakes decision making. We focus on one particular challenge to validity: decision dependency, which arises when historical decisions affect observed outcomes. We evaluate this aspect of problem formulation in LLM-generated code and explanations across child-welfare, healthcare, and employment settings, using 11 models from OpenAI and Anthropic and seven prompt conditions. We assess whether the LLM-generated code implements a procedure appropriate for estimating the intended outcome under the study’s assumptions and whether the explanation acknowledges decision dependency. We find that LLMs do not reliably address decision dependency without guidance in the prompt. When prompts specify the target, the assumption needed to estimate it, and the estimation procedure, target-aligned implementation reaches 99-100%. Among frontier models, a simple prompt description of how decisions affect outcomes appears sufficient to elicit target-aligned implementation. Some responses also acknowledge decision dependency without addressing it in their code. Our findings suggest that analysts should not rely on LLMs to correctly formulate prediction problems without explicit methodological guidance and careful review. We propose validity-centered guidelines for LLM-assisted predictive modeling, covering prompt formulation, validation of the target and estimation procedure, and identification of key assumptions. Establishing validity is a necessary early step in evaluating whether to deploy a predictive model in a high-stakes setting, but deployment requires weighing broader ethical, institutional, and practical considerations.
[LG-68] What Interpolators Store and What They Express: An Information-Theoretic Framework for Benign Overfitting and Label-Free Model Selection
链接: https://arxiv.org/abs/2611.23630
作者: Nikolaos Nikolaou
类目: Machine Learning (cs.LG)
*备注:
Abstract:Overparameterized learners can fit noisy training labels perfectly and still predict well, which neither classical complexity measures nor information-theoretic generalization bounds explain. We develop an information-theoretic framework that separates two things an interpolating learner does with its sample: storage, the information the trained model retains about its training labels, and expression, the information about the sample that reaches its predictions at fresh inputs. We first characterize the ideal model: for a loss l, the least complex l-sufficient representation is the Bayes act, whose entropy is the minimal sufficient complexity; this generalizes, and corrects, an earlier “lossless maximal compression” view of boosting. We then show that every interpolator stores exactly nH(Y|X) bits of label noise, so storage, and bounds built on it (including CMI bounds), cannot tell interpolators apart. Expression can: excess risk is bounded above and below by the expressed information V, the entropy over training samples of the prediction at an input, plus a modal-bias term, exactly so for binary labels. Benign overfitting is therefore vanishing expression, a picture that covers nearest-neighbour rules, averaging, minimum-norm least squares and double descent, and that links V to classical bias-variance decompositions and to model disagreement. Finally, V gives a model-selection criterion that needs no held-out labels. Among overparameterized learners on 15 noisy datasets, V estimated from two disjoint halves has 60% less regret than holdout validation at the same cost and matches 5-fold cross-validation at a quarter of its cost; at matched cost it also beats selection by conditional mutual information. We characterize when it fails and give remedies. Code: this https URL
[LG-69] Agent Verifiers in Counterfactual Analysis: A Pilot Study Benchmarking Agent ic Heterogeneous Treatment Effects NEURIPS2026
链接: https://arxiv.org/abs/2611.23621
作者: Satiyabooshan Murugaboopathy,Entejar Alam,Adel Daoud,Connor T. Jerzak
类目: Machine Learning (cs.LG)
*备注: NeurIPS 2026 Workshop: Who Verifies the Agents? Toward Reliable Agent Development
Abstract:Agents can expand the covariate space of randomized experiments by retrieving fast-moving local context missed by slow-changing survey baseline covariates, helping characterize who benefits most from an intervention. Verifying such covariates is difficult: individual treatment effects are unobserved, retrieval leakage can pull post-treatment information into nominally pretreatment summaries, and encoder leakage can inject post-treatment pretraining knowledge into representations even after temporal filtering of inputs. Guided by a two-channel information bound separating summary provenance from encoder weights, we propose a three-layer benchmark protocol: (i) retrieval admissibility via pre-treatment date filtering; (ii) heterogeneity validation scored rank-weighted average treatment effects (RATEs); and (iii) encoder admissibility checked with chronologically vintaged encoders, future-fact probes, and weight perturbations. Across 11 datasets anchored by Uganda’s Youth Opportunities Program, we compare session-isolated retrieval (sandboxed, bounded-budget agent runs per study) against a persistent workspace teammate (xAI’s GrokBot) that builds one date-verified corpus per study while blinded to outcomes. The workspace teammate yields higher mean RATE ratios in 6/11 datasets for text alone and 7 for combined features, though paired tests show neither workflow universally dominates. Even on fixed retrieved text, encoder choice significantly shifts heterogeneity signal. Chronologically vintaged probes show no clear evidence of weight leakage, but single-record adapter perturbation analyses show that memorized future facts, once present, persist until injected noise degrades model output. Temporal filtering of retrieval is necessary but insufficient: verifying agent-generated covariates requires jointly examining source provenance, out-of-sample treatment-effect signal, and encoder weights.
[LG-70] Error accumulation in reverse diffusion with an imperfect score: beyond path-space analyses
链接: https://arxiv.org/abs/2611.23617
作者: George Giapitzakis,Ankur Moitra,Andrej Risteski,Dhruv Rohatgi
类目: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Probability (math.PR)
*备注:
Abstract:Diffusion modeling is a popular method of sampling from a complex, high-dimensional distribution p using learned approximations of its (noised) score functions. Generating a single sample requires simulating the reverse diffusion process, which requires many queries to the learned scores. Thus, a priori, the learning errors can easily compound. Indeed, in existing theoretical analyses, the sampling error (as measured in total variation distance to p ) accumulates polynomially with the horizon of the stochastic process. This is a consequence of bounding the error of the output law by the error of the law of the entire reverse diffusion path. In this paper, we demonstrate that by circumventing the “path-space” analysis, the error accumulation of the diffusion sampler can be mitigated when p satisfies a Poincaré inequality. Moreover, we achieve this result as a special case of a broader framework that applies to any distribution p , but relaxes the error metric from total variation distance to a weaker metric capturing all sample statistics that do not “witness” the multimodality of p . We then zoom out from analyzing the diffusion sampler to the information-theoretic question: given a learned score with some error bound, how accurately can we identify p ? We show that, for any given bound on the Poincaré constant of p , the diffusion sampler already nearly achieves optimal total variation error. Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Probability (math.PR) Cite as: arXiv:2611.23617 [cs.LG] (or arXiv:2611.23617v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2611.23617 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-71] Label-Free Role-Aware Segmentation of Classroom Recordings: Recovering the Teacher from Transferable Structure AAAI
链接: https://arxiv.org/abs/2611.23609
作者: Nitin Jain,Tim Fütterer,Babette Bühler,Çağatay Yıldız
类目: Machine Learning (cs.LG); Sound (cs.SD)
*备注: Accepted at the AAAI track is IAAI (Innovative Applications of AI)
Abstract:How much the teacher talks and when are among the most informative and costly-to-measure signals of teaching. Yet, classroom audio is among the most sensitive data an AI system can process: recordings capture minors and fall under GDPR, preventing raw audio from leaving the recording device. We present a teacher-student segmentation pipeline designed to meet exactly this constraint: label-free at inference, running on a recording device, and requiring no per-session calibration. The pipeline chains a lightweight voice activity detector, compactly pretrained speaker embeddings, agglomerative clustering, and a small logistic regression that assigns the teacher role based on session-relative structure rather than voice identity. We show that our structural cue transfers across classrooms, languages, and acoustic conditions. In leave-one-teacher-out cross-validation on 292 recordings from 63 teachers across English and German corpora, our pipeline achieves teacher F1 of 0.81 and 0.87, respectively, outperforming both enrollment-based baselines and a 30-second supervised enrollment ceiling. On a commodity smartphone, its performance matches pyannote, a strong neural diarization tool, while running 4.8x faster at comparable peak memory at a real-time factor of ~0.18. We also find that transcribing filtered audio only marginally increases teacher’s word error rate from 0.26 to 0.28. Residual failures concentrate where teacher and student voices are not acoustically separable. However, we show how to flag these sessions label-free at inference so that they can be routed to human review, enabling selective, privacy-preserving deployment. Comments: Accepted at the AAAI track is IAAI (Innovative Applications of AI) Subjects: Machine Learning (cs.LG); Sound (cs.SD) Cite as: arXiv:2611.23609 [cs.LG] (or arXiv:2611.23609v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2611.23609 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-72] DuoCL: Dual-Path Online Continual Learning
链接: https://arxiv.org/abs/2611.23598
作者: Jebacyril Arockiaraj,Dhruv Parikh,Rajgopal Kannan,Viktor Prasanna
类目: Machine Learning (cs.LG)
*备注: Technical Report
Abstract:Online continual learning (OCL) requires learning from a data stream whose arrival rate is independent of model training speed. When training cannot keep pace, computationally expensive gradient-based methods update less frequently, limiting adaptation. Analytic methods reduce this overhead, but their frozen backbones cannot adapt representations to changing distributions. We propose DuoCL, a dual-path OCL framework whose fast analytic path accumulates feature statistics from every incoming minibatch while its slow path periodically adapts the backbone. The slow path adjusts its sample count to the available compute budget without interrupting fast-path learning. Feature alignment compensates for drift in the accumulated classifier statistics. Across four datasets and three computational budgets, DuoCL improves online accuracy by an average of 9.67 percentage points over the strongest gradient-based baseline in each setting. DuoCL consistently outperforms analytic learning under distribution shifts, with gains of up to 4.00 percentage points on CIFAR-C, and achieves higher accuracy than prior methods across disjoint class-incremental, domain-incremental, and Si-Blurry settings.
[LG-73] Efficient Variance-Aware Reinforcement Learning in Time-Inhomogeneous Linear MDPs
链接: https://arxiv.org/abs/2611.23596
作者: Qingyue Zhao,Qiwei Di,Jiafan He,Heyang Zhao,Kaixuan Ji,Quanquan Gu
类目: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
*备注: 38 pages, 2 tables
Abstract:We study variance-dependent reinforcement learning in episodic time-inhomogeneous linear Markov decision processes (MDPs) with feature dimension d , horizon H , and K episodes. While recent works have established variance-dependent regret with \tildeO(1) deterministic-regime overhead for linear mixture MDPs (Zhao et al., 2023) and worst-case nearly minimax optimal regret \tildeO(d\sqrtH^3 K + d^7 H^8) for linear MDPs (He et al., 2023), existing variance-aware analyses for linear MDPs (Li and Sun, 2023; Huang et al., 2023) rely on a coarse concentration step whose resulting regret guarantees retain a polynomial \tildeO(K^1/4) additive term even in deterministic environments. We propose VA-LSVI-UCB++, a computationally efficient algorithm that achieves the variance-dependent regret bound \tildeO(d\sqrtH\min\mathrmVar_K^,\mathrmVar_K^\pi+d^5H^4) , where \mathrmVar_K^ and \mathrmVar_K^\pi denote the cumulative conditional variances of the optimal value function and the played policies’ value functions along the realized trajectory. Our result adapts to the minimum of the two realized trajectory variances, recovers the worst-case minimax optimal leading term \tildeO(d\sqrtH^3 K) , and collapses to \tildeO(d^5 H^4) in deterministic environments, free of polynomial-in- K dependency. At the core of our analysis is a simultaneous three-way reference-advantage decomposition for the optimistic, pessimistic, and squared value function estimators coupled with variance self-bounding and Freedman-type value-difference arguments, which tightens the pessimistic and second-moment confidence radii by a factor of \tildeO(d H) and removes the inverse-variance-floor blowup.
[LG-74] Optimal Control of RKHS Diffeomorphic Flows for Classification and Generative Modeling
链接: https://arxiv.org/abs/2611.23497
作者: Christopher Chukwuemeka,Andreas Mang
类目: Numerical Analysis (math.NA); Machine Learning (cs.LG); Optimization and Control (math.OC)
*备注: 55 pages, 21 figures, 16 tables, 6 algorithms
Abstract:We develop a unified optimal-control framework that solves supervised classification and likelihood-based density estimation with a single diffeomorphic flow whose velocity lives in a reproducing kernel Hilbert space (RKHS); the two tasks share all structure except the objective. Variable projection eliminates the terminal model (a classifier, or a Gaussian-mixture (GMM) reference), leaving a reduced control problem whose gradient comes from an analytic discrete adjoint. The RKHS parameterization makes the flow’s divergence exact and deterministic at the cost of one kernel matrix-vector product, avoiding the stochastic trace estimators and potential-flow restrictions of existing continuous normalizing flows. Because the objective and gradient are deterministic, matrix-free reduced Gauss-Newton (GN) and related second-order solvers are practical for both tasks, which we benchmark against a range of first- and second-order optimizers; the learned map is a diffeomorphism with explicitly controlled regularity. In experiments from dimension 2 to 256, the reduced GN solvers use the fewest iterations of any optimizer tested and, together with joint L-BFGS, the least work; a multiscale kernel improves the density fit, and a trainable GMM reference reduces the deformation needed for multimodal transport at comparable likelihood while lowering the likelihood on real data. The full-model solve is quadratic in the particle count and sub-linear in dimension; a control-point reduction makes the particle-count scaling linear at near-lossless accuracy.
[LG-75] FACT: Fixed-Budget Allocation and Cross-Step Training for Speculative Decoding
链接: https://arxiv.org/abs/2611.23463
作者: Hadi Fares,Mayssa Hajj Hassan,Ali Salman,Ammar Mohanna
类目: Machine Learning (cs.LG)
*备注: 17 pages, 11 figures, 5 tables. Code: this https URL
Abstract:Speculative decoding accelerates language-model generation by verifying multiple draft tokens in a single target-model pass. However, better draft predictions and longer accepted continuations do not necessarily produce faster generation. To examine this gap, we present FACT (Fixed-Budget Allocation and Cross-Step Training), an experimental framework for studying how drafter training, candidate allocation, and execution conditions jointly affect acceptance and latency. Using a lightweight feature-prediction drafter with a frozen Llama-3-8B-Instruct target under greedy decoding, we first examine how training translates into accepted tokens. Backpropagating through predicted-feature rollouts with greater weight on early steps improves acceptance by 15-24% over uniformly weighted detached training, yet average rollout proxy cross-entropy does not reliably rank drafters by acceptance. Beyond prediction quality, the placement of draft candidates also matters: with a fixed 256-node budget and matched per-depth widths, replacing static allocation with confidence-guided allocation increases speedup over autoregressive decoding from 1.46\times to 2.02\times on HumanEval and from 1.31\times to 1.75\times on MT-Bench. This allocation benefit extends to an independently trained drafter, but stronger training does not consistently amplify it, showing that the two improvements do not necessarily reinforce each other. Even these speed gains depend on execution conditions: changing only host-thread settings can nearly eliminate speedup without changing acceptance. Together, these findings show why training and allocation must be evaluated jointly using both acceptance and end-to-end latency under controlled execution conditions.
[LG-76] Conformally Calibrated Adaptive Computation in Neural Operators
链接: https://arxiv.org/abs/2611.23440
作者: Abdul Qadir Ibrahim,Martin Burger,Abel Maviano Jonathan
类目: Machine Learning (cs.LG)
*备注: 27 pages, 7 figures
Abstract:Selecting less computation in a neural operator need not reduce inference time. We study this gap using conformal calibration to choose Fourier modes or channel width per input subject to a fixed error budget. A triangle-inequality envelope bounds all candidates simultaneously; a direct-score rule handles non-monotone candidate errors and, with an input-independent score, calibrates a static baseline under the same marginal guarantee. Under frozen-model and exchangeability assumptions, both rules bound the probability of issuing a prediction outside the budget and abstain when calibration is infeasible. On Burgers and Darcy flow across three training seeds, mode selection reduces active spectral support without a measurable latency reduction, because full-grid transforms and channel mixing dominate the cost. Narrowing channels does reduce Darcy forward cost, but reference-based selection must first compute the dense prediction. An input-only binary width selector avoids that pass and runs at 0.52-0.95 times dense latency; yet static widths calibrated under the same rule are often as fast or faster, and on a separately generated 1,000-example test set they generally achieve higher coverage, while the selector falls below its 90% target in more than half of the settings. Adaptive computation in neural operators should therefore be judged by achieved coverage and complete runtime against a matched static comparator.
[LG-77] When Does Subspace Direction Matter for LoRA? Regime Analysis of the Magnitude Principle in Few-Shot Adaptation NEURIPS2026
链接: https://arxiv.org/abs/2611.23428
作者: Nischal Subedi,Cencheng Shen,Peng Zhao
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注: Accepted at NeurIPS 2026. 43 pages, 5 figures. Code: this https URL
Abstract:We show that the importance of a LoRA adapter’s initial input-subspace direction is regime-dependent, and we characterize this dependence in theory and experiment. This refines the recent view that spectral initializers help through update magnitude alone. Direction matters most when labeled data are scarce relative to the representation dimension and the activation subspace used for initialization captures supervised task structure. In this regime, the adapter’s row span remains sensitive to its starting point, making task-aligned initialization especially useful. A spiked-covariance perturbation analysis characterizes the finite-sample stability of the estimated subspace. A first-step analysis of LoRA training explains why the subspace’s alignment with the task matters and motivates an alignment coefficient \rho that closely tracks gains across classification tasks. We test this account with PivotLoRA, which initializes adapters from activation PCA at prediction-relevant positions and applies one output-preserving orthogonal pivot early in training. In few-shot BERT-base classification, PivotLoRA achieves the best average rank among 13 methods and improves over vanilla LoRA in 29 of 30 settings, even though it starts from a smaller input-factor initialization norm. Additional classification experiments extend the evaluation to 7B and 8B Llama decoders. Classification gains diminish with more labeled data, consistent with the proposed regime dependence. A targeted extension to completion-only reasoning also shows improvements on GSM8K and StrategyQA. These findings identify conditions under which initialization direction matters, with subspace-task alignment offering a low-cost diagnostic before training for classification.
[LG-78] Stability Does Not Mirror Plasticity: A Module-Level View of Declarative and Procedural Learning in Transformers
链接: https://arxiv.org/abs/2611.23427
作者: Jinjun Peng,Wailiang Zhao,Baishakhi Ray,Zhou Yu,Jufeng Yang
类目: Machine Learning (cs.LG)
*备注:
Abstract:Plasticity and stability in continual learning are typically studied at the entire-model level and in a task-agnostic manner, yet prior work has reported divergent conclusions across settings. Inspired by cognitive distinctions between declarative and procedural memory, we instead study how plasticity and stability vary across transformer modules and task types. In a controlled synthetic environment with clean measurement, we show a double dissociation: query-key (QK) and value-output (VO) projections are the most plastic on procedural tasks, whereas MLPs are the most plastic on declarative memorization. Stability, however, does not simply mirror plasticity: when learning a new procedure, updating QK retains over half of the stored declarative memory, while updating MLPs retains far less and updating VO loses roughly four-fifths. Targeted module reset-and-recovery experiments explain this asymmetry: plasticity tracks which module is best suited to a task’s primary computation or storage, whereas stability depends on all ways the existing capability relies on that module, including access to information stored in other modules. Experiments on a realistic model and tasks broadly echo these asymmetric dynamics. Overall, our work shows that plasticity and stability depend on different aspects of a module’s functional role, offering a task-dependent, module-level perspective on continual learning.
[LG-79] Incremental Roseland: Batch-wise Landmark Diffusion via Kernelized Sequential Karhunen–Loève Update
链接: https://arxiv.org/abs/2611.23414
作者: Jeanie Schreiber
类目: Machine Learning (cs.LG); Functional Analysis (math.FA); Numerical Analysis (math.NA)
*备注:
Abstract:Landmark diffusion (Roseland) computes a diffusion-map-style spectral embedding from the singular value decomposition of a degree-normalized data-to-landmark kernel matrix, at a cost far below that of full diffusion maps. It assumes, however, that all data are available at once. We introduce incremental Roseland (iRoseland), which casts the landmark affinity as a row-growing matrix whose SVD can be updated with a kernelized Sequential Karhunen–Loève (SKL) scheme. Degre correction is handled by a first order expansion about a reference mean kernel row and so that the landmark-side eigenbasis may be recovered from a single landmark-by-landmark eigendecomposition at each step. The presented method requires only a single pass over the data and runs in bounded-memory with state independent of the number of points processed. A forgetting factor lets the method track drifting distributions, and a sample-size-invariant embedding keeps outputs comparable across time. Experiments show agreement with Roseland to round-off on stationary data, and accuracy that exceeds equal-memory Roseland by an order of magnitude on long streams. On drifting data, we observe large gains from forgetting with a window matched to the drift.
[LG-80] A Survey on the Use of Machine Learning within Computer Algebra
链接: https://arxiv.org/abs/2611.23361
作者: Uzma Shafiq,Matthew England,Nayyar Zaidi
类目: ymbolic Computation (cs.SC); Machine Learning (cs.LG)
*备注: 31 pages
Abstract:This survey explores the use of Machine Learning (ML) in the field of Computer Algebra (CA), both to optimise existing CA algorithms, and to perform symbolic computation directly. Traditional symbolic methods, while mathematically rigorous, often are computationally expensive thus limiting their use in real-world applications. Recent advances have shown that data-driven techniques can address these limitations by guiding heuristic decisions, selecting optimal algorithms, predicting structural properties of algebraic objects, or even making direct symbolic computations. We take a systematic literature review approach and uncover work in CA applications including cylindrical algebraic decomposition, Gröbner basis computation, symbolic integration, and many more. The survey compares the different ML approaches that have been employed for these tasks, ranging from decision trees to transformers. Issues uncovered by the survey include the lack of benchmark datasets for CA, which hinders the comparison of methods and the generalizability of ML models. The survey identifies the potential for explainable AI tools to help develop trust in decisions, and to drive forward CA research itself. Comments: 31 pages Subjects: Symbolic Computation (cs.SC); Machine Learning (cs.LG) MSC classes: 68W30, 68T05 ACMclasses: I.2.6; I.1.0 Cite as: arXiv:2611.23361 [cs.SC] (or arXiv:2611.23361v1 [cs.SC] for this version) https://doi.org/10.48550/arXiv.2611.23361 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Journalreference: ACM Computing Surveys, volume 58, issue 16, article number 391, 31 pages, 2026 Related DOI: https://doi.org/10.1145/3831699 Focus to learn more DOI(s) linking to related resources
[LG-81] Error Analysis of Deep Learning
链接: https://arxiv.org/abs/2611.23321
作者: Yuling Jiao,Peiying Wu,Jerry Zhijian Yang,Pingwen Zhang
类目: Numerical Analysis (math.NA); Machine Learning (cs.LG)
*备注: Mathematical survey of error analysis for neural network methods, with nonparametric regression and the Deep Ritz method for elliptic PDEs as running examples. Covers function and Sobolev approximation, statistical estimation, and optimization error, together with stability estimates, convergence rates, and selected proofs
Abstract:This review examines deep learning theory through the lens of error decomposition, systematically addressing approximation, statistical, and optimization errors. We trace the development of error analyses from bounds on individual error components to frameworks that account for all three. Using nonparametric regression and the Deep Ritz Method as dual running examples, we demonstrate how these analyses apply across fundamentally different loss structures. We then review representative analyses in scientific machine learning, generative learning, reinforcement learning, representation learning, and large language models, showing how learning errors affect solutions, distributions, and predictions through problem-specific arguments. Throughout, we clarify which error components are controlled, how they are combined, and what assumptions are needed to obtain guarantees for trained networks.
[LG-82] Exploring Parkinsons Diagnosis Using Machine Learning Techniques
链接: https://arxiv.org/abs/2611.23248
作者: Farhaan Qazi
类目: Machine Learning (cs.LG)
*备注: MSc thesis, Heriot-Watt University, 2023, 90 pages
Abstract:In recent years, machine learning (ML) techniques have emerged as promising tools for accurate and timely diagnosis of Parkinson’s disease (PD) which is an incurable neurodegenerative disorder. This study explores the integration of ML algorithms and PD diagnosis, highlighting their potential applications in various aspects of the disease. ML methods such as support vector machines, decision trees, multilayer perceptron, k-nearest neighbours, logistic regression and random forest (with and without particle swarm optimisation) were applied on a PD dataset obtained from Leeds Teaching Hospitals NHS Trust containing data from 58 people. In terms of accuracy, the Decision Tree and Logistic Regression performed the best with scores of 72-76% while multilayer perceptron was least effective. All the models were remarkable in classifying positive PD cases with sensitivity scores going up to 97.56%, with SVM classifier at the pinnacle. However, ability to classify healthy controls from PD cases was insignificant throughout the board. Although, all the employed models delivered reasonable results in categorising PD patients in the presented dataset; access to more data, further study and advancement of these ML approaches could lead to enhanced diagnostic outcomes, supplementing to an improved understanding of Parkinson’s disease.
[LG-83] Budgeted Proper Loss Design for Uncertain Decision Thresholds
链接: https://arxiv.org/abs/2611.23247
作者: Ankit Sharma(International Institute of Information Technology Bangalore)
类目: Machine Learning (cs.LG)
*备注:
Abstract:A binary probability estimate may be used by several decision makers with different error costs. Proper scoring rules can emphasize selected probability thresholds, but a single average threshold distribution need not give a good worst-case regret guarantee. We study the choice of a proper loss when the threshold distribution belongs to a finite specified family. Among integrable loss weights with a positive pointwise floor and a fixed total mass, we derive the weight that minimizes the sharp, distribution-free multiplier from surrogate regret to threshold-averaged decision regret. The solution clips a rescaled upper envelope of the threshold densities at the prescribed floor; its normalization is found by a one-dimensional root search. We give proofs of sharpness, optimality, and stability under density misspecification, as well as a limitation for atomic threshold laws. An exact histogram implementation and two synthetic experiments across 20 seeds verify the construction. The optimized multiplier improves over uniform and average-density weights, while fitted-model performance does not improve uniformly. The result is a loss-design guarantee under an explicit budget, rather than a general claim of predictive superiority.
[LG-84] Chebyshev Polynomial Zonotope Abstractions for Certified Reachability of Learning-Enabled Systems
链接: https://arxiv.org/abs/2611.23246
作者: Peng Xie,Amr Alanwar
类目: Machine Learning (cs.LG); Systems and Control (eess.SY)
*备注:
Abstract:Nearly all current methods for computing reachable sets of nonlinear dynamics propagate them with truncated Taylor expansions plus rigorous remainders. The expansion is pointwise: once its argument is a whole set rather than a point, one polynomial must stay accurate across that set, and it reaches only as far as its disc of convergence. The limit binds hardest on the rational terms and sigmoid gates of modern controllers: smooth as these look on the real line, in the complex plane they blow up at points just off the real axis, closer as a gate sharpens, and those points cap the set width a Taylor expansion can cover, most tightly on the switching surface where the controller decides. We prove a dichotomy: no Taylor degree certifies sets wider than that cap, while a Chebyshev abstraction of equal degree, anchored to the whole set in the same polynomial-zonotope pipeline, has no cap and gains width at a geometric rate in degree. Measured walls match the formulas to between 89 and 94 percent. We give closed-form rigorous remainders, among them the first polynomial softmax transformer; existing sound treatments are linear relaxations. On closed loops, certificates survive where the Taylor ones diverge, and specifications flip from unknown to proven; on trained classifiers up to real MNIST digits, the Chebyshev frontier certifies input territory the Taylor basis surrenders. Every certified claim passed large Monte-Carlo harnesses with zero violations, and an applicability map states in advance where the basis provably cannot matter.
[LG-85] Biologically-Inspired Spike Encoding and Optimization for Spiking Neural Networks
链接: https://arxiv.org/abs/2611.23242
作者: Ayaan Qayyum,Bruce Yi Bu,Zicong Li,Jingfeng Zhou
类目: Machine Learning (cs.LG)
*备注: 17 pages, 9 figures
Abstract:Spiking Neural Networks (SNNs) offer theoretical energy-efficiency advantages over conventional deep networks, but realizing these advantages on vision tasks requires sparse spike representations that naive rate-coding methods fail to provide. We combine biologically motivated spike-encoding schemes with two training improvements: Rouser adaptive-threshold surrogate gradients and Kronecker-Factored Approximate Curvature (K-FAC) second-order optimization. The spike encoding schemes experimented with are rate coding, latency coding, DVS-style delta modulation, and Difference-of-Gaussians (DoG) filtering. Experiments on MNIST and Fashion-MNIST demonstrate that biologically inspired encoders yield the largest accuracy gains in capacity-constrained regimes and serve as an effective parameter-efficiency lever. DoG and delta modulation encoders produce significantly sparser spike trains than rate coding. K-FAC yields modest but consistent accuracy improvements over Adam-trained baselines, and Rouser’s learnable per-neuron thresholds improve Fashion-MNIST accuracy from 87.37% to 88.13% by enabling homeostatic firing-rate regulation. Biologically grounded pre-processing methods can help close the accuracy gap between SNNs and non-spiking baselines without increasing model size.
[LG-86] GridForge: An Open Physics-Informed Digital Twin and Benchmark Suite for UK 11 kV Underground Distribution Cables
链接: https://arxiv.org/abs/2611.23241
作者: Lilliane Linnet Musoke
类目: Machine Learning (cs.LG)
*备注: 15 pages, 6 figures, 3 tables
Abstract:The UK distribution-cable monitoring sector is data-rich but framework-poor: each Distribution Network Operator (DNO) collects megabytes of telemetry per asset per day, but there is no public benchmark dataset, no shared reference model, and no commonly agreed evaluation protocol for benchmarking sectoral, data-intensive models for operational decision support; notably for maintenance planning and prioritisation. We address this gap with GridForge, an open-source physics-informed digital twin and benchmark suite for fault diagnosis and prediction, integrating four building blocks: (i) a verified steady state and transient thermal model based on IEC 60287 and IEC 60853; (ii) a calibrated synthetic dataset of 64 cables (covering 320 cable-years) representing four canonical UK 11 kV thermal archetypes and four condition modes; (iii) a Physics-Informed Neural Network (PINN) surrogate that satisfies the algebraic balance defined by IEC 60287 during training and reaches an RMSE of 0.148 °C on an independent held out set against the reference analytical oracle; and (iv) a five-task benchmark suite that includes three reference baseline models (pure physics, gradient-boosted trees, and the PINN). All numerical parameters are sourced from public standards, the code is licensed under the MIT licence, and the trained PINN converges in only two minutes (as run on the computing platform in Section 10.1), enabling independent verification by DNO engineering and research practitioners. At present only the virtual-sensor task (T4) discriminates between the reference baselines; the other tasks tie or are undefined on this test split, so extending the baselines with task-specific logic is a priority future enhancement. For open validation, the dataset, test sets and models are made available at this https URL
[LG-87] Neural Network-Based Modeling of Hysteretic Systems: From Prandtl and Preisach Neural Networks to Neuro-Madelung and Neuroplastic Continuous Models
链接: https://arxiv.org/abs/2611.23240
作者: Mehrdad Shafiei Dizaji,Farzad Shafiei Dizaji
类目: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
*备注: 14, 8
Abstract:Hysteresis is a complex nonlinear phenomenon characterized by memory dependence, path dependence, and, in many physical systems, degradation, non-congruency, asymmetry, and rate dependence. This review examines the development and capabilities of Prandtl neural networks (PNNs), Preisach neural networks (PreisachNNs), generalized Prandtl neural networks (GPNNs), the neuro-Madelung model (NMM), extended Preisach neural networks (EPNNs), and dynamic plastic continuous neural networks (DPCNNs). Particular attention is given to their mechanisms for representing memory, network architectures, activation functions, training algorithms, treatment of deterioration and rate dependence, and applications to nonlinear structural and material systems. The PNN introduced hysteretic memory directly into neural activation through stop operators and demonstrated accurate nonlinear dynamic analysis of structures. Preisach-NNs subsequently extended this concept through an additional nonlinear hidden layer to represent non-Masing hysteresis. GPNNs introduced deteriorating-stop neurons to capture stiffness and strength degradation and non-congruent hysteresis. A different generalization was provided by the NMM, which combined Madelung’s hysteresis rules with multilayer feedforward neural networks to represent diverse rate-independent and rate-dependent behaviors. These developments culminated in the EPNN, which combines deteriorating hysteretic operators, nonlinear neural mappings, and input-rate information to accommodate Masing and nonMasing, symmetric and asymmetric, congruent and noncongruent, and rate-independent and rate-dependent hysteresis.
[LG-88] Exploration via Exploitation: The Blessing of Reward Diversity in Personalized Federated RL
链接: https://arxiv.org/abs/2611.12964
作者: Jiin Woo,Gauri Joshi,Yuejie Chi
类目: Machine Learning (cs.LG); Information Theory (cs.IT); Optimization and Control (math.OC); Machine Learning (stat.ML)
*备注:
Abstract:While federated reinforcement learning (RL) has traditionally guaranteed high efficiency in homogeneous environments, collaboration is often hindered when agents possess distinct goals. In this paper, we flip this paradigm, framing reward heterogeneity not as a hurdle to consensus, but as a structural asset that catalyzes exploration. We first propose Personalized Federated Upper Confidence Bound Value Iteration (PF-UCBVI), which achieves optimal linear speedup by decoupling shared dynamics from personalized objectives. To eliminate the risks of explicit exploration, we then introduce Personalized Federated Exploration-Free Value Iteration (PF-EFVI), a purely greedy algorithm that leverages reward diversity to ensure state-action coverage. We prove that PF-EFVI attains logarithmic regret without explicit exploration bonuses under sufficient reward diversity. Our results reveal that agent disagreement shifts the exploration burden from temporal complexity to spatial diversity, enabling efficient collective learning without explicit exploration.
[LG-89] Decomposing SGD Dynamics in Neural Networks: Teacher-Induced Spikes and Variance Inflation NEURIPS2026
链接: https://arxiv.org/abs/2611.24908
作者: Samet Demir,Zafer Dogan
类目: Machine Learning (stat.ML); Machine Learning (cs.LG)
*备注: NeurIPS 2026, 36 pages, 4 figures
Abstract:We study two-layer neural networks trained by stochastic gradient descent (SGD) on a multi-index teacher model, focusing on how first-layer parameters orthogonal to the teacher subspace affect generalization. We decompose the first-layer weights into a signal component aligned with the teacher and an orthogonal complement, and analyze their joint SGD dynamics together with the second layer in the proportional limit. We show that although SGD updates to the complement are initially isotropic, the training dynamics induce teacher-dependent spectral spikes in the complement Gram matrix. This emergent anisotropy provides a concrete mechanism by which representation variance is amplified, and generalization degrades. To isolate this effect, we introduce Decomposed Dynamics, an analytically tractable surrogate whose asymptotic test-error dynamics coincide with those of SGD. In simulations with a ReLU student, freezing the directions of the orthogonal complement improves generalization relative to training them. Finally, we show that the signal component rapidly aligns with the second-layer weights, revealing strong cross-layer coupling even in this shallow setting. Together, our results provide a dynamical explanation for how feature learning outside the teacher subspace harms generalization.
[LG-90] Learning the Anyon Atlas
链接: https://arxiv.org/abs/2611.24752
作者: Jonah Berean-Dutcher,Clay Cordova,Julio Parra-Martinez,Richard Tuddenham,Zekai Zhu
类目: High Energy Physics - Theory (hep-th); Strongly Correlated Electrons (cond-mat.str-el); Machine Learning (cs.LG); Quantum Algebra (math.QA)
*备注: 24 pages, 11 figures
Abstract:We extend our previous search for the modular data of topological quantum field theories in 2+1 dimensions from 22 to 42 anyons, identifying 150,679 new modular data. We do so by adding to our algorithm three rejection gates, two of them based on logistic regression, that discard representations of SL(2,\mathbbZ) unlikely to yield modular data. We estimate that roughly 80% of the modular data between ranks 23 and 42 has been identified after closing under known constructions. With this data, we find counterexamples to a conjecture of Geiko and Moore that the reality of the higher Gauss sums suffices for time-reversal symmetry in Chern-Simons theories, and propose a refined criterion. We demonstrate the scalability of this gated implementation by identifying prime modular data at rank 100 that is not realized by any Chern-Simons theory. The identified data through rank 42 has been made available at this https URL, to which further data will be added as it is found.
[LG-91] Sex Differences in the Detection of Parkinsons Disease from Speech
链接: https://arxiv.org/abs/2611.24741
作者: Hiba Akhaddar,Peter Plantinga,Mirco Ravanelli,Tristan Glatard
类目: Audio and Speech Processing (eess.AS); Computers and Society (cs.CY); Machine Learning (cs.LG)
*备注: 12 pages, 3 figures
Abstract:Parkinson’s disease (PD) is the second most common neurodegenerative disease in the world, and 70% to 75% of PD patients suffer from speech disorders at some stage. Since PD affects men more frequently than women, previous publications tend to rely on unbalanced datasets that potentially introduce sex biases into trained models. This study investigates sex differences in PD detection from speech using deep learning and traditional machine learning approaches. We evaluated speech-based PD detection using embeddings from three pre-trained speech models, combined with handcrafted acoustic features and different classifiers. Sex-specific experiments were conducted on balanced subsets of four sustained phonation datasets, controlling for sex, disease status, and, where possible, age. All recordings were obtained during a sustained phonation task. Results show that female speakers were more accurately classified than male speakers in the larger dataset. Sex-specific results show that when the models are trained on one sex, they fail to generalize to the other one. Our results highlight the importance of including sex as a variable in the development of PD detection systems.
[LG-92] Surrogate Modeling and Reduced-Order Methods in Gravitational-Wave Physics and Numerical Relativity: From Spectral Representations to Adaptive Compression
链接: https://arxiv.org/abs/2611.24727
作者: Manuel Tiglio
类目: General Relativity and Quantum Cosmology (gr-qc); High Energy Astrophysical Phenomena (astro-ph.HE); Machine Learning (cs.LG); Computational Physics (physics.comp-ph)
*备注:
Abstract:The rapid expansion of gravitational-wave astronomy and high-precision numerical relativity has created an acute computational bottleneck: direct numerical integration of the Einstein field equations and high-dimensional Bayesian parameter estimation require millions of waveform evaluations that cannot be sustained by traditional solvers alone. Reduced-order modeling (ROM) and surrogate techniques have emerged as foundational tools to bridge this gap, compressing high-fidelity solution manifolds into compact, rapidly evaluable representations without sacrificing physical accuracy. This perspective reviews the mathematical pillars of projection-based reduced-order modeling—including Proper Orthogonal Decomposition, greedy basis selection, and the Empirical Interpolation Method—and traces their operational impact in gravitational-wave data analysis through surrogate waveform families and Reduced-Order Quadratures. We then examine the fundamental approximation barriers posed by slow Kolmogorov n-width decay in non-stationary, transient, and strongly nonlinear regimes, showing how adaptive hp-greedy domain partitioning and multi-scale spectral compression overcome these limits. Finally, we discuss how combining certified reduced-order structures with modern scientific machine learning and sensitivity-aware quantization provides a rigorous blueprint for next-generation detectors such as LISA, Einstein Telescope, and Cosmic Explorer.
[LG-93] Diffusion Models are Provably Efficient Priors for Sparse Recovery
链接: https://arxiv.org/abs/2611.24602
作者: Ivan Lee,Xingyu Xu,Yuejie Chi
类目: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG); Optimization and Control (math.OC)
*备注:
Abstract:Diffusion models have transformed generative modeling through their ability to capture and generate samples from complex data distributions. This success has spurred substantial interest in using pretrained diffusion models as expressive priors for highly ill-posed inverse problems in imaging and sensing. Despite strong empirical performance, theoretical understanding of this approach remains limited: existing guarantees are often asymptotic or entail prohibitive computational costs in the worst case. In this paper, we address this gap by studying sparse recovery from a small number of linear measurements, a classical problem in compressed sensing. We show that, given an accurate diffusion model capturing a sparsity-promoting Bernoulli-Gaussian prior, a likelihood-guided annealed Langevin sampler provably recovers the sparse signal in polynomial time with near-optimal sample complexity for a broad family of random and deterministic sensing matrices. When the signal is indeed drawn from the postulated prior, we further establish total variation guarantees for posterior sampling after a final denoising step. These results lay a theoretical foundation for statistically and computationally efficient inverse solvers with diffusion priors.
[LG-94] aming the Tails of Stochastic Gradient Descent in the Interpolation Regime
链接: https://arxiv.org/abs/2611.24562
作者: Leonardo Galli,Daniela Angela Parletta,Andrea Paudice
类目: Optimization and Control (math.OC); Machine Learning (cs.LG)
*备注:
Abstract:In this paper, we study stochastic gradient descent in the smooth interpolation regime, where the population objective shares a common minimizer with the sample losses. In contrast to prior work, which has largely focused on in-expectation analysis, we obtain high-probability convergence guarantees under the same assumptions. In the convex case, we prove a fast rate of order \log(\beta^-1) \cdot n^-1 after n iterations, with failure probability at most \beta . In the strongly convex case, we derive a family of moment-based bounds of order \beta^-p \cdot \exp(-\mathcalO(p \cdot n)) for p \in (0,1] , and complement them with a matching lower bound showing that the corresponding deterioration in the exponential rate is essentially unavoidable. In both settings, our theoretical results are unified across a broad class of step-size schedules. Building on this analysis, we extend our results to a low-noise regime and to nonconvex objectives satisfying the Polyak-Lojasiewicz condition.
[LG-95] Sparse designs via quantization of the uniform measure on high-dimensional hypercubes
链接: https://arxiv.org/abs/2611.24507
作者: Luc Pronzato,Anatoly Zhigljavsky
类目: Machine Learning (stat.ML); Machine Learning (cs.LG)
*备注: 25 pages, 16 figures
Abstract:While the notions of sphere packing and sphere covering underlie the most widely used criteria for space-filling design in computer experiments, namely, the maximin and minimax approaches, they suffer from major drawbacks in high dimensions. First, when the dimension d is very large, deploying a space-filling design requires an extremely large number of design points, making the very notion of a space-filling design questionable. Second, these two design characteristics are highly sensitive to boundaries, particularly when the design domain is a hypercube (the case considered in this paper), and especially to its vertices. This sensitivity is particularly acute when the design size is less than 2^d . Instead of considering the maximin or minimax criterion, we consider here the quantization error of the uniform measure on the hypercube. This error has three key advantages: it accounts for the entire distribution of distances from a random point U to the design, is less sensitive to boundary effects than the maximin and minimax criteria, and, thanks to Zador’s celebrated theorem, enables a smooth and natural transition to uniform designs as the design size tends to infinity. Building on recent results on the construction of optimal random quantizers for spherically symmetric distributions (arXiv:2610.11772) we propose a method to generate random designs of a given target size n with small L_s -mean quantization error. The support of the resulting distribution is then used to construct designs, possibly incrementally, with sizes up to n . We demonstrate that, for moderate n , designs supported on a fractional factorial subset of the vertices of a small hypercube perform exceptionally well—and significantly better than designs uniformly distributed over the entire hypercube. We also explain how to account for projections onto lower-dimensional subspaces.
[LG-96] Stability-compatible kernels: beyond Gaussianity for impulse response estimation
链接: https://arxiv.org/abs/2611.24272
作者: Gianluigi Pillonetto,Mauro Bisiacco
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Systems and Control (eess.SY)
*备注:
Abstract:Kernel-based regularization for linear system identification uses positive semidefinite kernels both to define reproducing kernel Hilbert spaces (RKHSs) and as covariance models for stochastic impulse responses. Sharp kernel conditions are known for RKHS stability, while almost-sure BIBO stability under Gaussian priors requires an even more restrictive covariance condition. These restrictions, however, need not persist beyond Gaussianity, where the covariance no longer determines the full probability law. We introduce the notion of a stability-compatible kernel, namely a covariance kernel admitting at least one stochastic realization with almost-surely absolutely summable sample paths. We show that stability-compatibility holds if and only if the stable elements are dense in the associated RKHS, a condition strictly weaker than RKHS stability, and derive practical sufficient conditions, including a spectral criterion for stationary kernels. We then construct non-Gaussian priors that preserve a prescribed covariance while enforcing almost-sure stability. In particular, this yields a stable prior based on the stationary radial-basis-function (RBF) kernel, separating covariance structure from the mechanism enforcing stability. Numerical experiments show substantial estimation gains for several important classes of impulse responses, with particularly pronounced advantages when relevant dynamics occur at late lags or in separated temporal regions.
[LG-97] Functional Principal Component Analysis: Minimax Optimality from Sparse to Dense Designs and Beyond
链接: https://arxiv.org/abs/2611.24228
作者: Shunxing Yan,Fang Yao
类目: atistics Theory (math.ST); Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注: 25 pages, 1 figure
Abstract:Functional principal component analysis (FPCA) plays a central role as a dimension-reduction and representation tool in functional data analysis, but its minimax theory for diverging spectral indices under noisy discrete observation has remained open for over two decades. To fill this gap, we consider a sieve-based FPCA framework that pools discrete measurements across subjects and accommodates sampling frequencies ranging from sparse to dense under random designs. We begin by establishing new minimax rates for covariance estimation over general bivariate function classes beyond tensor-product low-complexity structures, thereby identifying the intrinsically two-dimensional optimal rate. Our central theoretical contribution is to establish minimax optimal rates for both individually labelled eigenfunctions with diverging indices and leading principal subspaces of increasing dimensions, under explicit growth and regularity conditions. To establish these results, we develop a general framework for sharp minimax analysis of FPCA under discrete observation, separating spectral approximation from stochastic perturbations and exploiting their distinct structures, thereby offering a new approach to a broad class of nonparametric spectral estimation problems. For both spectral targets, the optimal rates reveal two distinct statistical mechanisms, spectral identification and nonparametric reconstruction, whose interplay gives rise to either four or two sampling regimes. This yields, for the first time, a precise characterization of the impact of discrete sampling across the full range from sparse to dense designs. Finally, as a downstream application, we analyze FPCA-based functional linear regression and show that the FPCA-based estimator attains the minimax-optimal rates.
[LG-98] Rare-Tail Statistics For Learning Biased Gaussian Halfspaces with Label Noise NEURIPS2026
链接: https://arxiv.org/abs/2611.24181
作者: Arya Mazumdar,Prateeti Mukherjee,Harsh Vardhan
类目: Machine Learning (stat.ML); Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG); Statistics Theory (math.ST)
*备注: NeurIPS 2026, to appear
Abstract:We revisit the classical problem of learning general Gaussian halfspaces in the presence of label noise, with particular focus on the large-bias regime, where information-computation gaps are conjectured in several standard label-noise models. Concretely, we present and analyze the rare-tail mean estimator, an extremely simple one-shot statistic that averages only the observed minority class. In the large-bias regime, classical Averaging (Servedio, 1999) and related global first-moment statistics are provably suboptimal, while the rare-tail mean estimator recovers the target halfspace direction at the information-theoretic sample rate (modulo log factors) under a broad class of label noise models, including probit noise and margin-dependent noise with very slow decay away from the boundary. Notably, our guarantees allow labels arbitrarily close to the decision boundary to be nearly random, while still giving strong directional recovery from a small number of samples. The estimator also cleanly interpolates between the noiseless and noisy regimes: without noise, it attains the same sample rate as classical separability-based methods for realizable halfspaces, and under low noise, or noise that decays away from the boundary, it keeps this rate even after linear separability has been destroyed. As an application, we consider a two-stage curriculum in which source samples, whose noise may be arbitrarily close to 1/2 near the boundary and decay very slowly away from it, precede target samples, whose noise flips every label at the same rate, i.e., random classification noise (RCN). Since both stages share the same Bayes optimal halfspace, our estimator supplies the directional warm start needed by existing refinement routines, and together they learn the RCN target at the information-theoretic sample rate in the large-bias regime, up to log factors.
[LG-99] Nearly Tight Almost Sure Convergence Rates for the Last iterate of SGD
链接: https://arxiv.org/abs/2611.24151
作者: Weiming Ou,Andrea Paudice,Xiao Wang
类目: Optimization and Control (math.OC); Machine Learning (cs.LG)
*备注:
Abstract:We study the almost sure convergence of the last-iterate optimization error of Stochastic (sub)Gradient Descent (SGD) for convex objectives. For smooth objectives, we establish convergence rates of order k^-1/3 when a stationary minimizer exists, and k^-1/4 otherwise, up to polylogarithmic factors, where k denotes the iteration counter. We further show that the latter rate extends to nonsmooth convex objectives. All these rates are achieved by deterministic, anytime stepsize policies. Finally, we establish matching lower bounds showing that the exponents -1/3 and -1/4 are optimal, up to polylogarithmic factors, for the corresponding problem classes.
[LG-100] Fill-Distance-Guided Active Learning for Causal Function Recovery under Network Interference
链接: https://arxiv.org/abs/2611.24143
作者: Peiyao Liu,Hongtao Zhu,Sizhe Zhang,Yang Su,Zhenyu Zhao,Nan Chen
类目: Methodology (stat.ME); Machine Learning (cs.LG)
*备注:
Abstract:Causal inference seeks to learn how outcomes change under alternative interventions, but network interactions complicate this task. Under network interference, outcomes depend on both own treatment and neighbors’ treatments, making causal effects functions of neighborhood exposure rather than single treatment contrasts. Existing methods primarily target prespecified causal contrasts or their estimation precision, leaving global response-function recovery largely unexplored. We study sequential experimentation under a limited experimental budget, with the goal of recovering the full exposure-dependent population-average response function. To this end, we propose FANCI (Fill-Distance-Guided Active Learning for Network-Interference Causal Inference). For function learning, treatment-specific Gaussian processes flexibly model nonlinear conditional response surfaces. For experimental design, we formulate global response-function recovery as a coverage problem and propose a fill-distance criterion that guides new experiments toward poorly covered regions. To account for network constraints on attainable exposure patterns, we develop full-network and target-screened strategies for selecting feasible network–assignment pairs. Theoretically, we derive uniform error bounds linking fill distance to response-function estimation and characterize the approximation loss from target screening. Simulations and an online-game experiment show that FANCI improves recovery of nonlinear exposure-dependent effects and reveals treatment patterns obscured when network interference is ignored.
[LG-101] AG-BFGS: Foundations for Accelerated Geometry Learning
链接: https://arxiv.org/abs/2611.24113
作者: Regina Carla Sousa,Denis Vargas,Elizabeth Wanner,Joshua Knowles
类目: Optimization and Control (math.OC); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Numerical Analysis (math.NA)
*备注: 19 pages, 7 figures
Abstract:We introduce an accelerated geometry-learning framework for BFGS that incorporates momentum into the learning of Hessian inverse approximations. We prove that the resulting accelerated updates satisfy explicit stability conditions and preserve positive definiteness within admissible parameter regions. Numerical experiments demonstrate consistent improvements over the corresponding non-accelerated schemes while maintaining numerical stability. These results provide the first rigorous theoretical foundation for accelerated quasi-Newton learning and bridge the gap between empirical observations and mathematical guarantees for accelerated second-order methods.
[LG-102] Quantum Annealing with a Quantum Neural Network: Width of Annealing Matters
链接: https://arxiv.org/abs/2611.24089
作者: Mojtaba Asadollahi,Danial Khajehzadeh,Abolfazl Ramezanpour
类目: Quantum Physics (quant-ph); Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (cs.LG)
*备注: 22 pages, 4 figures
Abstract:Quantum annealing is a promising quantum algorithm for dealing with computationally hard optimization problems. Currently available hardware can implement the algorithm for a significant class of Hamiltonian problems. However, its performance is especially limited in discontinuous quantum phase transitions, where the energy gap shrinks exponentially with system size. In this paper, we introduce an extension of quantum annealing to a quantum neural network and study its performance in a number of prototypical optimization problems. In particular, we observe that already a random quantum annealing network of width W = 2 can exponentially reduce the number of evolution layers or depth of the annealing process in the quantum search problem. By tuning the network parameters the search problem can be solved with exponentially small errors by a quantum perceptron of depth L = 1 and width W = 2. The findings show that width of a quantum annealing network is critical to enhance the success probability of the algorithm by providing a larger and learnable evolution space for the annealing process.
[LG-103] NOTREKS: Marginal Independence in Score-based Structure Learning
链接: https://arxiv.org/abs/2611.24015
作者: Fabian Bleile,Sarah Lumpp,Mathias Drton
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)
*备注:
Abstract:Score-based structure learning estimates causal graphs by optimizing a data-dependent objective over structural equation models. We study how prior knowledge that selected pairs of variables are marginally independent can be incorporated into such causal discovery methods. Under faithfulness, marginally independent pairs cannot be connected by a trek, so the knowledge becomes a graphical no-trek constraint, which we call NOTREKS information. We introduce a power-series trek (PSTrek) penalty for gradient-based optimization and a trek constraint for order-based search, and we analyze why such information helps in finite samples and how it restricts the search space. Our numerical experiments illustrate to what extent NOTREKS information helps in causal discovery. In particular, using synthetic benchmarks with oracle NOTREKS information, we show that these constraints can improve both structural recovery and adjustment-based effect identification.
[LG-104] Optimal Deep Nonparametric Regression with Uniformly Bounded Lipschitz Constants
链接: https://arxiv.org/abs/2611.23979
作者: Jiaxin Shi,Ziyuan Chen,Jian Huang,Hansheng Wang
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
*备注:
Abstract:Stability under input perturbations is necessary for deep neural network estimators in many applications involving noisy covariates or estimated features. Weight norm constraints are widely used to control sensitivity to input perturbations by bounding the Lipschitz constant of neural networks. However, existing approximation results under fixed bounds on standard norm products are insufficient to establish optimal statistical rates. In this paper, we establish a sharp approximation theory for deep ReLU networks under a split spectral norm product constraint. This theoretical result shows that sufficiently smooth Hölder functions can be uniformly approximated to arbitrary accuracy as the network depth increases, with the norm-product bound held fixed. We then propose a penalized least squares estimator for deep nonparametric regression. The resulting estimator has a uniformly bounded Lipschitz constant while still achieving the optimal nonparametric rate up to a logarithmic factor. As applications, we establish optimal prediction rates for two-stage learning and provide robustness guarantees under adversarial input perturbations. Numerical studies show that the penalized estimator has well-controlled sensitivity to input perturbations and prediction accuracy comparable to that of classical deep network methods.
[LG-105] Infinity-Wasserstein Distributionally Robust Principal Component Analysis: Exact Reformulation Finite-Sample Guarantee and Efficient Algorithm
链接: https://arxiv.org/abs/2611.23877
作者: Zhouxing Luo,Lei Wang,Xin Liu
类目: Optimization and Control (math.OC); Machine Learning (cs.LG); Statistics Theory (math.ST); Methodology (stat.ME); Machine Learning (stat.ML)
*备注:
Abstract:Principal component analysis (PCA) is a fundamental and ubiquitous technique for dimension reduction and representation learning, yet its empirical formulation remains vulnerable to distributional uncertainty. While distributionally robust optimization (DRO) offers a natural remedy, existing order-2 Wasserstein models of PCA either preserve the nominal PCA subspace or lack an exact reformulation. This paper demonstrates how the infinity-Wasserstein distance can reconcile these two desirable properties. For the resulting DRO model of PCA, we derive an exact finite-dimensional reformulation and explicitly identify a worst-case distribution attaining the inner maximum. Crucially, the reformulated objective retains samplewise residual information beyond the empirical covariance matrix, revealing a mechanism through which distributional robustness can reshape the learned principal subspace. Under the mild assumption that the true distribution has a finite fourth moment, we further establish a uniform finite-sample performance guarantee for the proposed model. An iteratively reweighted majorization-minimization algorithm is also developed to solve the resulting nonconvex and nonsmooth optimization problem, with a rigorous analysis of its convergence properties. Numerical experiments substantiate the computational efficiency of the proposed algorithm and the practical effectiveness of the infinity-Wasserstein model in downstream applications.
[LG-106] Improved Leverag e Score Sampling for Constrained Active Linear Regression NEURIPS2026
链接: https://arxiv.org/abs/2611.23801
作者: Aarshvi Gajjar,Syamantak Kumar,Aditya Makkar,Christopher Musco,Joey Zhou
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Numerical Analysis (math.NA)
*备注: To appear in NeurIPS 2026 (Spotlight)
Abstract:In this paper we study active learning for constrained linear regression. Given a design matrix A\in\mathbbR^n\times d , a constraint set C\subseteq\mathbbR^d , and query access to b\in\mathbbR^n , we seek an approximate solution to \min_x\in C|Ax-b|2^2 using as few queries as possible. This problem arises in many scientific and engineering applications, and randomized leverage score sampling provides a powerful method to solve it. However, existing leverage score methods ignore the constraint or encode it through ridge regularization, suffering unfavorable query complexity for anisotropic constraint sets. We propose ellipsoid-regularized leverage scores, where we approximate the constraint set by an outer ellipsoid and use its shape matrix to bias sampling toward directions that matter under the constraint geometry. We prove that the query complexity is governed by an anisotropic effective dimension d\texteff©\le d . With appropriate parameters, O(d_\texteff©\log d/\varepsilon^2) queries suffice to return a point in C with objective value at most (1+\varepsilon) times the optimum. This bound can substantially improve on the standard leverage-score bound, since anisotropic constraints can make d_\texteff© much smaller than d . For convex constraints, we improve the dependence on \varepsilon from 1/\varepsilon^2 to 1/\varepsilon . We also prove two lower bounds: first, the d/\varepsilon^2 dependence is unavoidable without convexity assumptions on C , and second, in some regimes, the effective dimension dependence d_\mathrmeff©/\varepsilon is necessary for ellipsoidal constraints. Experiments on synthetic regression problems and a parametric heat conduction problem show lower regression error than standard baselines for similar query budget.
[LG-107] STORM: A Lightweight Structure-Free Model for Binding Affinity Prediction
链接: https://arxiv.org/abs/2611.23777
作者: Ayush Pandit(1),Rohan V. Koodli(1),Wei Deng(1),Roxana Daneshjou(1),Ron O. Dror(1) ((1) Stanford University)
类目: Quantitative Methods (q-bio.QM); Machine Learning (cs.LG)
*备注: 9 pages, 3 figures
Abstract:Accurately predicting a small molecule’s binding affinity to a protein, a critical task in drug discovery, remains challenging. Physics-based approaches like free-energy perturbation (FEP) have very high computational costs and require expert setup. Recent machine learning–based cofolding models have reported accuracy approaching that of FEP—and at least one of them, Boltz-2, has already been widely adopted for affinity prediction—but they remain expensive for large computational screens and even more expensive for retraining on newly collected experimental data. To address these challenges, we introduce STORM: Similarity Transfer from Observed Reference Molecules. STORM is a fast, simple machine learning model that ensembles many small model fusions of diverse protein and molecule representations for binding affinity prediction. Trained on the same data, STORM matches or exceeds the accuracy of Boltz-2 on multiple evaluation datasets, while delivering a speed improvement of over five orders of magnitude for both inference and training. Our results also suggest opportunities to substantially improve affinity prediction accuracy by integrating historical experimental affinity data with structural and biophysical information.
[LG-108] A Free Lunch: Gradient Averag ing Achieves Optimal Statistical Efficiency in Newton Sketch
链接: https://arxiv.org/abs/2611.23671
作者: Haoxuan Wang,Jianliang Ye,Sen Na
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC); Computation (stat.CO)
*备注: 58 pages, 5 figures, 6 tables
Abstract:We study statistical inference for online sketched Newton methods, in which each Newton system is approximately solved by an inner randomized sketching solver. Existing work reveals a computational–statistical trade-off: the sketching solver (and its accelerated variants) reduces the per-step cost of Newton methods from O(d^3) to O(d^2) , but the randomness introduced by sketching also inflates the limiting covariance of both the last and the averaged Newton iterates. In this work, we address this trade-off via gradient averaging. We show that simply replacing the stochastic gradient with a moving average at each step can mitigate, and in some cases eliminate, the trade-off in Newton sketch. Importantly, gradient averaging requires no additional oracle evaluations and leaves the contraction bound of the inner sketching solver unchanged. In particular, we prove that the last sketched Newton iterate with gradient averaging attains a limiting covariance no larger than that without gradient averaging, while the averaged iterate even attains the minimax optimal limiting covariance, matching that of the exact Newton method. To our knowledge, this is the first analysis showing that randomness introduced to improve computational efficiency need not compromise statistical efficiency as long as proper gradient estimates are used. The key analytical challenge is that gradient averaging destroys the crucial martingale difference structure underlying central limit theorem arguments. We overcome this challenge by analyzing the coupled two-timescale recursions governing the iterate and the gradient estimator. Numerical experiments on linear and logistic regression support our theoretical covariance comparisons.
[LG-109] Optical Neural Networks Should Be Evaluated by Function Not Only Physics
链接: https://arxiv.org/abs/2611.23610
作者: Shi-Yuan Ma,Xingjian Bai,Ryan Hamerly,Mahmoud Jalali Mehrabad
类目: Optics (physics.optics); Emerging Technologies (cs.ET); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Applied Physics (physics.app-ph)
*备注: 28 pages, 3 figures, 4 tables
Abstract:Optical neural networks (ONNs) are attracting significant attention as alternative platforms for machine learning (ML). In an ONN, an optical module often takes over part of the computation inside a digital ML pipeline. How much of the task does the optical module itself perform, and how much does it contribute to any claimed system-level advantages, such as throughput or energy? This paper offers a simple, function-oriented evaluation methodology. Whatever its internal physics, the optical module appears to the surrounding digital pipeline as a mapping between real-valued tensors. We therefore evaluate it in three steps: (i) define its boundary in the pipeline, as real-valued tensors in and out; (ii) replace it with standard digital modules to identify those that match the optical system’s performance on the same task and metric; and (iii) state system-level claims, such as speed or energy, per inference against the performance-matched digital module. We summarize steps (i) and (ii) in an Expressivity Card and demonstrate its use on published ONN systems. This helps isolate the optical module’s contribution and provides a concrete digital reference for assessing its claimed system-level advantages, without the need to understand the physics. While we focus on optical systems as the most prominent example, the same approach applies to any physical analog computing platform used in ML, wherever a physical module takes over part of a digital pipeline. Comments: 28 pages, 3 figures, 4 tables Subjects: Optics (physics.optics); Emerging Technologies (cs.ET); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Applied Physics (physics.app-ph) Cite as: arXiv:2611.23610 [physics.optics] (or arXiv:2611.23610v1 [physics.optics] for this version) https://doi.org/10.48550/arXiv.2611.23610 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-110] Bounded-Order Non-Identifiability in Latent Structure Block Models: The Fiber in Closed Form AISTATS2027
链接: https://arxiv.org/abs/2611.23547
作者: Sie Hendrata Dharmawan,Peter Chin
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
*备注: Submitted to AISTATS 2027
Abstract:A graphon is determined by all of its homomorphism densities, but motif-based methods use finitely many. We compute exactly what that truncation costs in a natural family: latent structure block models whose positions lie on K rays, equivalently degree-corrected block models of rank two. For a given radial law, the configurations that agree on every graph with fewer than 2K-1 vertices form a one-dimensional fiber, which we give in closed form with explicit endpoints; the friendship graph F_K-1 is its unique edge-minimal separator, so no fixed motif order serves every K . The separating signal is suppressed by a factor of at most m_1^2(K-1) , where m_1 is the mean squared slope, about 10^-4 at K=4 : detecting it from motif counts would take about 10^15 nodes in a single graph, at any edge density. The information is nonetheless in the graph; the fiber belongs to the statistic, not the model. Common spectral pipelines, which count no motifs, still slide along the fiber by more than a third of its length at ordinary densities, with confidence regions that almost never contain the truth, while model-based fits of the same embedding do not. We give practitioners four checks: whether a fit is at risk, what to run instead, why a confidence interval will not warn, and how wide the remaining ambiguity is.
[LG-111] Balancing Partition Forest for Causal Weighting
链接: https://arxiv.org/abs/2611.23524
作者: Felipe Angelim,Alessandro Leite
类目: Methodology (stat.ME); Machine Learning (cs.LG)
*备注:
Abstract:Balancing weights are used in observational causal inference to construct a population in which the treatment is independent of observed pretreatment covariates. The ideal weight is the ratio between the product of the covariate and treatment marginals and their joint distribution, for binary and continuous treatments alike. We propose Balancing Partition Tree (BPT) and Balancing Partition Forest (BPF), which estimate this ratio on a learned rectangular partition of covariate–treatment space. Each split maximizes the gain in a Bregman divergence computed from joint and marginal counts. Two choices of divergence yield criteria that maximize the mutual information captured by the partition or reduce the squared error of the weights. We show that the population residual of either criterion bounds the bias of inverse-probability-weighted and doubly robust response estimators that use the population partition weights. Across 17 synthetic and semi-synthetic settings and three response estimators, BPF obtains the lowest mean RMSE rank among seven weight estimators, and a single tree fits in 0.14 seconds on 16,512 observations.
[LG-112] JEPA for astronomy: multi-survey spectral alignment puts stellar parameters on the leading principal axes
链接: https://arxiv.org/abs/2611.23492
作者: Xiaosheng Zhao,Rosemary F.G. Wyse,Yuan-Sen Ting,Alexander S. Szalay
类目: Astrophysics of Galaxies (astro-ph.GA); Instrumentation and Methods for Astrophysics (astro-ph.IM); Solar and Stellar Astrophysics (astro-ph.SR); Machine Learning (cs.LG)
*备注: 15 pages, 5 figures, 2 tables. Submitted to AAS journals. Comments welcome
Abstract:Spectra from different surveys carry different systematics and are not calibrated against one another. Multi-survey foundation models for stellar spectra are then built by aligning embeddings of the same star observed across surveys. Most in astronomy follow the recipes established for language and vision, aligning with a contrastive loss that separates every star from the others. However, this does not ensure that the stellar physics is encoded in a form that a simple read-out can recover. Holding the cross-survey architecture fixed and varying only the alignment loss, we measure what an objective encodes from the principal-component geometry of the Gaia BP/RP (XP) spectra and from a linear probe of the stellar parameters. In a properly aligned embedding, the leading two principal components form a plane that recovers an analog of the Kiel diagram, the third component tracks [Fe/H], and [alpha/Fe] varies coherently across the plane, such that the chemically defined Galactic disks and accreted populations occupy distinct loci. We show that legibility, the ease of reading the information out, is highest for a training objective that never pushes stars apart, and instead constrains only the overall distribution of each survey’s embeddings (JEPA). JEPA also exceeds the contrastive alternative at the same architecture, and the performance improves further as more surveys (Gaia XP+LAMOST LRS+DESI+APOGEE) are included. Predicted abundances for 8.0 million XP giants reproduce the main [alpha/Fe]-[Fe/H] structure of the Milky Way. These results indicate that the choice of alignment objective, not alignment alone, determines whether stellar labels are linearly recoverable.
[LG-113] A Differential Theory of Quality-Resource Trade-offs for AI Model Selection: The Multiresource Frontier and Its Minimax Price
链接: https://arxiv.org/abs/2611.23324
作者: Dmitry Ignatov
类目: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG); Performance (cs.PF); Optimization and Control (math.OC)
*备注: Under review
Abstract:Comparing AI models across resource budgets yields quality-resource frontiers relating error to latency, parameter count, training compute, and energy. Offset power laws summarize such frontiers, but their fitted parameters reveal neither the price-independent geometry nor the manner in which trade-offs extend across several resource channels. This paper addresses both gaps by developing a differential theory in which this structure is made explicit. Each frontier admits a price-independent plane-curve representation relating the effective number of active resource channels, N_\mathrmeff = s^2/\kappa (squared slope over curvature), to the remaining log-error depth, \Delta S = \ln(\varepsilon/\varepsilon_\infty) . This representation separates intrinsic shape from the scale and origin of the resource axis. Complete monotonicity and divisibility yield differential inequalities and frontier bounds. When each of N resources contributes one active channel, the theory gives the parameter-free operating rule N_\mathrmeff = 1 in delivered units and derives minimax resource prices \lambda^\star_i = \alpha_i/(N+1) with an exact worst-case regret of \ln(N+1) nats. The results make it possible to interpret measured frontiers and compare resource costs without hand-chosen exchange rates. The theory is evaluated on 117 published frontiers and implemented in the open-source package nn-rank, which derives prices from measured quality and cost, ranks candidates, and reports supported choices and minimax regret: this https URL
附件下载


