本篇博文主要内容为 2026-07-13 从Arxiv.org论文网站获取的最新论文列表，自动更新，按照NLP、CV、ML、AI、IR、MA六个大方向区分。
说明：每日论文数据从Arxiv.org获取，每天早上12:30左右定时自动更新。
提示: 当天未及时更新，有可能是Arxiv当日未有新的论文发布，也有可能是脚本出错。尽可能会在当天修复。
目录
概览 (2026-07-13)
今日共更新428篇论文,其中:
- 自然语言处理共36篇(Computation and Language (cs.CL))
- 人工智能共117篇(Artificial Intelligence (cs.AI))
- 计算机视觉共77篇(Computer Vision and Pattern Recognition (cs.CV))
- 机器学习共113篇(Machine Learning (cs.LG))
- 多智能体系统共8篇(Multiagent Systems (cs.MA))
- 信息检索共5篇(Information Retrieval (cs.IR))
- 人机交互共15篇(Human-Computer Interaction (cs.HC))
多智能体系统
[MA-0] Mosaic: Runtime-Efficient Multi-Agent Embodied Planning ICML2026
【速读】：该论文旨在解决基于大语言模型（LLM）的多智能体具身规划因执行延迟过高而难以实际应用的问题。其核心挑战在于：在部分可观测环境下，状态跟踪不准确，以及多智能体间协调效率低下导致冗余或冲突动作频发。针对上述问题，论文提出Mosaic框架，其关键解决方案包括：采用以智能体为中心的语义记忆机制，通过相对坐标存储物体信息，实现轻量级但精确的状态跟踪，并支持几何变换与协同；同时引入整数线性规划（Integer Linear Programming），在每一步规划中对动作进行分配，确保物理可行性及多智能体间的协调约束。实验结果表明，Mosaic在AI2-THOR和搜救任务基准上实现了27%-32%的执行速度提升、30%-33%的LLM调用减少、25%-31%的规划步数降低以及4%-10%的成功率提高，验证了高效记忆管理与约束引导协同在实现可扩展、低延迟多智能体规划中的关键作用。
链接: https://arxiv.org/abs/2607.09603
作者: Kunjal Panchal,Saayan Mitra,Sunav Choudhary,Victor Bursztyn,Somdeb Sarkhel,Hui Guan
机构: 未知
类目: Multiagent Systems (cs.MA)
备注: Accepted to ICML 2026
Abstract:LLM-based multi-agent embodied planning remains impractical due to prohibitively high execution latency. We identify failed actions as the dominant bottleneck, stemming from two core challenges: inaccurate state tracking under partial observability and inefficient coordination that produces redundant or conflicting actions. We introduce Mosaic, a runtime-efficient multi-agent planning framework that addresses both challenges. Mosaic maintains accurate yet lightweight state tracking through agent-centric semantic memory that stores objects in relative coordinates, enabling geometric transformations and coordination. It ensures efficient coordination through Integer Linear Programming that allocates actions at every planning step, enforcing physical feasibility and inter-agent coordination constraints. Across AI2-THOR and search-and-rescue benchmarks, Mosaic achieves 27-32% faster execution, 30-33% fewer LLM calls, 25-31% fewer steps, and 4-10% points higher success rates. These results demonstrate that efficient memory and constraint-guided coordination are critical for scalable, low-latency multi-agent planning.
[MA-1] Shared Selective Persistent Memory for Agent ic LLM Systems
【速读】：该论文旨在解决生成式智能体（Agentic LLM）在多轮工具调用过程中面临的上下文效率问题：每次会话均从零开始，导致先前会话中积累的可复用配置、领域约束、数据模式及工具使用模式被丢弃。传统方法通过持久化完整对话历史以保留上下文，但会造成令牌（token）浪费且引入无关信息，进而降低生成质量。其解决方案的关键在于提出“共享选择性持久记忆”（shared selective persistent memory）架构，该架构能够识别并保留四类可复用上下文（任务规范、数据模式、工具配置与输出约束），同时主动丢弃会话特异性的推理轨迹。该记忆机制具有共享特性，支持基于角色访问控制的工作空间跨用户迁移，实现协作复用而无需重复配置。系统部署于一个协同工作平台，由大模型智能体从异构数据源（如CSV、SQL、REST API和MCP服务器）生成、编辑并维护版本化的代码资产（仪表盘、报告与数据驱动文档）。此外，引入互补的零令牌数据刷新机制，将生成程序与运行时数据解耦，从而实现无需重新调用大模型即可复用产出物。实验表明，在三个企业场景中，该方案任务完成率达96%（无记忆为79%，全历史为71%）；零令牌刷新使重复更新任务时间减少14倍，摘要驱动生成相比原始数据注入将单次调用的令牌开销降低97倍。在四个公开数据集上的复现验证了其泛化能力，零令牌刷新在全部12次试验中成功。值得注意的是，简单全历史持久化反而因引入过时推理痕迹而降低任务完成率，而选择性记忆则显著优于两者极端。
链接: https://arxiv.org/abs/2607.09493
作者: Sanjana Pedada,Aditya Dhavala,Neelraj Patil
机构: Apple Inc. (苹果公司)
类目: Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA); Software Engineering (cs.SE)
备注: 11 pages, 2 figures, 4 tables
Abstract:Agentic LLM systems that generate code through multi-turn tool use face a fundamental context problem: each session starts from zero, discarding the configuration choices, domain constraints, data schemas, and tool-use patterns that made previous sessions productive. Naively persisting entire conversation histories is token-inefficient and counterproductive: irrelevant context degrades generation quality. We introduce shared selective persistent memory, an architecture that identifies and retains four categories of reusable context (task specifications, data schemas, tool configurations, and output constraints) while discarding session-specific reasoning traces. Crucially, this memory is shared: workspaces encapsulating selective memory can be transferred across users with role-based access control, enabling collaborative reuse without redundant specification. We implement it in a deployed collaborative workspace platform where LLM agents produce, edit, and maintain git-versioned artifacts (dashboards, reports, and data-driven documents) from heterogeneous sources (CSV, SQL, REST APIs, and MCP servers). A complementary zero-token data refresh mechanism decouples generated programs from runtime data, enabling artifact reuse without re-invocation. Across three enterprise scenarios, shared selective persistent memory achieves 96% task completion (vs. 79% without memory and 71% with full history). Zero-token refresh eliminates LLM re-invocation for recurring updates (14x task-time reduction), while summary-driven generation cuts per-invocation token cost by 97x versus raw data injection. A replication on four public datasets confirms generalizability, with zero-token refresh succeeding in 12/12 trials. Notably, naive full-history persistence actively degrades completion by biasing the agent with stale traces, while selective memory outperforms both extremes.
[MA-2] Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networks
【速读】：该论文旨在解决由异构大型语言模型（LLM）驱动的具身智能体团队在物理人工智能场景（如智能工厂、仓储及服务机器人）中协作时面临的三大耦合挑战：一是多轮自然语言对话导致的通信开销随团队规模迅速增长；二是智能体间因LLM能力异质性带来的协调质量受限；三是迭代协商引发的动作延迟问题。其解决方案的关键在于提出一种基于轻量级数字孪生（Digital Twin, DT）的网络化协同框架LDT-Coord，通过将智能体的行动决策与结构化的时间资源约束上报至数字孪生服务器，实现协调性能与自然语言推理能力的解耦；随后，由无训练的规则引擎算法在数字孪生端进行冲突消解并返回协调指令，有效避免跨智能体冲突；为进一步降低通信开销，作者将智能体报告控制建模为带约束的不完全可观马尔可夫决策过程（C-POMDP），并采用PPO-Lagrangian算法求解，显著减少通信频次。实验表明，LDT-Coord在保持与传统方法相当的任务成功率的同时，通信开销降低超过70倍，并对LLM异质性具有强鲁棒性。
链接: https://arxiv.org/abs/2607.09330
作者: Nuocheng Yang,Sihua Wang,Zihan Chen,Tony Q. S. Quek,Changchuan Yin
机构: Beijing University of Posts and Telecommunications (北京邮电大学); Singapore University of Technology and Design (新加坡科技设计大学)
类目: Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)
备注: 14 pages, 6 figures, 5 tables
Abstract:Embodied agent teams powered by heterogeneous large language models (LLMs) are being widely deployed in physical artificial intelligence such as smart factories, warehouses, and service robotics. To enable collaboration among such an agent team, efficient coordination mechanisms that operate reliably under limited network resources are required. However, existing heterogeneous LLM-agent coordination frameworks that rely on multi-round natural-language-based conversations introduce three coupled challenges. First, inter-agent dialogue incurs communication overhead that grows rapidly with team size. Second, the quality of coordination is constrained by the heterogeneous capabilities of the agent team’s LLMs. Third, agents may suffer from action delays due to iterative negotiation. To address these challenges, we propose LDT-Coord, a networked coordination framework built upon a lightweight digital twin (DT). Specifically, each agent independently selects its intended action and reports both the action decision and a structured temporal constraint over shared resources to the DT server, thereby decoupling coordination performance from natural-language reasoning ability. Then, DT executes a training-free, rule-based orchestrator algorithm to resolve cross-agent conflicts and returns coordination instructions to prevent such conflicts. To further reduce communication overhead, we formulate agent reporting control as a constrained partially observable Markov decision process (C-POMDP) and solve it with the PPO-Lagrangian algorithm. Simulation results show that LDT-Coord achieves a task success rate comparable to conventional coordination methods while reducing communication overhead by more than 70x and maintaining robustness under LLM heterogeneity.
[MA-3] When is Routing Meaningful? Diversity and Robustness in Language Model Societies
【速读】：该论文旨在解决多模型系统中路由策略评价体系的局限性问题，即当前评估主要聚焦于任务准确率和推理成本，而忽视了路由机制是否具有实际意义。其核心问题在于：有效的路由不仅需性能优越，还必须满足两个非性能维度的关键属性——行为差异化与稳定性。行为差异化要求各智能体（actor）在响应上具有本质区别，否则路由将失去意义；稳定性则要求同一语义意图的不同表达形式应被一致分配给同一智能体，以支持专业化分工。为诊断上述失效模式，论文提出将层次化社会熵（Hierarchic Social Entropy, HSE）适配至语言模型社会，并引入基于扰动的鲁棒性度量。实验结果表明，HSE呈现显著递减收益，提示从大规模智能体池中选取少于十个经过筛选的智能体即可捕获大部分多样性，为社会结构设计提供了实用的“核心集”启发式方法。进一步发现，基于KNN的路由虽在专家型社会中提升准确率，但在扰动下鲁棒性严重下降，而提示驱动的路由在各类扰动下保持稳定，凸显了准确率与路由意义之间的显著分离。
链接: https://arxiv.org/abs/2607.09197
作者: Fantine Huot,Michael Kaisers,Mirella Lapata
机构: 未知
类目: Multiagent Systems (cs.MA)
备注:
Abstract:Routing policies for multi-model systems are evaluated almost exclusively on task accuracy and inference cost. We argue that two properties, orthogonal to performance, determine whether routing is meaningful. First, the society of actors must be behaviourally differentiated: if all actors respond identically, routing is vacuous. Second, the routing policy must be stable: surface-form variants of a query should be assigned to the same actor. High task accuracy is compatible with violating both properties, since a router can operate over a redundant society or assign queries inconsistently, preventing specialisation regardless of performance. We adapt Hierarchic Social Entropy (HSE) to language-model societies and introduce a perturbation-based robustness metric to diagnose these failure modes. Applied to EmbedLLM and RouterBench, we find that HSE exhibits strong diminishing returns, suggesting that a curated subset of fewer than ten agents recovers most available diversity in a large pool – a practical coreset heuristic for society design. We further find that KNN routers gain accuracy from specialist societies but collapse in robustness under perturbation, while prompted routing remains stable across all perturbation types – illustrating that accuracy and meaningfulness can sharply diverge.
[MA-4] Secret Scanner Agent : Extracting Secrets and Access Context from Unstructured Documents
【速读】：该论文旨在解决在事件响应过程中，传统凭据扫描工具难以有效识别泄露凭据所对应的“目标系统”（即“门”）这一关键问题。现有方法依赖正则表达式或训练好的分类器，虽在结构化代码中表现良好，但在面对碎片化、格式重构或与目标资源相距较远的凭据时，往往无法准确关联凭据与其可访问的账户、租户、终端、数据库或云资源等目标系统，且仅报告凭据字符串本身，缺乏上下文信息。为此，论文提出Secret Scanner Agent (SSA)，一个基于多智能体大语言模型（multi-agent large-language-model）的系统，其核心创新在于同时提取凭据（secret）及其关联的“门”（door），并提供支持性证据。该系统由检测代理（侧重召回率）和审查代理（用于过滤误报并恢复上下文）协同工作，显著提升了对凭据-目标关系的识别能力。通过在自动生成的23类凭据、多种文档格式的合成基准上进行评估，结合程序匹配、大语言模型（LLM）判别与人工审核的三阶段评分体系，结果表明：多智能体架构的SSA在门识别精度上相较单智能体版本提升最高达16个百分点，精度达到正则表达式扫描器水平的同时，召回率超过其三倍；相较于13名安全分析师，SSA不仅更精确，还能发现近两倍的凭据-门配对，且处理速度提升5至17倍。通过一次性输出凭据、目标系统及证据，SSA将凭据检测转化为可直接用于优先级排序与修复操作的可行动发现，显著增强了事件响应效率与安全性。
链接: https://arxiv.org/abs/2607.09011
作者: Zixiao Chen,Mariko Wakabayashi,Charlotte Siska
机构: Microsoft(微软)
类目: Cryptography and Security (cs.CR); Multiagent Systems (cs.MA)
备注: Submitted to the Conference on Applied Machine Learning for Information Security (CAMLIS) 2026
Abstract:Exposed documents such as emails, chat threads, tickets, and incident notes routinely leak credentials, but during incident response a leaked secret is only half the story. Responders also need to identify the ``door’’ the secret opens: the account, tenant, endpoint, database, cloud resource, or other system that the credential could allow an attacker to access. Traditional secret scanners rely on regular expressions or trained classifiers which work well on well-formatted code, yet they struggle when a credential is fragmented, reformatted, or far from the resource it unlocks, and they report the secret string without naming what it opens. We present Secret Scanner Agent (SSA), a multi-agent large-language-model system that extracts both the secret and its associated door, together with supporting evidence, from unstructured exposed documents. SSA pairs a detection agent that favors recall with a review agent that filters false positives and recovers missing context. Because real credential data is sensitive, we evaluate SSA on synthetic benchmarks we generated that span 23 secret types and multiple document formats, scored with a three-step pipeline of programmatic matching, an LLM judge, and human review. Across six models, multi-agent SSA improves extraction precision over a single-agent variant, with the largest gains on door extraction, by up to 16 percentage points. SSA matches a regular-expression scanner’s precision while more than tripling its recall, and against thirteen security analysts it is more precise, recovers nearly twice as many secret–door pairs, and runs five to seventeen times faster. By returning the secret, its door, and supporting evidence in one result, SSA turns credential detection into an actionable finding for triage and remediation.
[MA-5] Offline Nash Solvers Meet Online Tree Search in Multi-Agent Games on Graphs
【速读】：该论文旨在解决多智能体追逃博弈（Multi-Agent Pursuit-Evasion Games, PEG）中纳什均衡策略计算面临的挑战，即随着智能体数量增加，联合状态空间与动作空间呈指数级增长，导致求解难度急剧上升。现有方法或依赖离线的均衡近似，缺乏执行过程中的适应性；或采用在线规划方法，但受限于巨大的分支因子。本文提出一种混合框架——基元引导树搜索（Primitive-Guided Tree Search, PGTS），其核心解决方案在于将离线精确纳什均衡计算与在线树搜索相结合：首先在离线阶段求解一系列规模较小、可处理的子博弈，获得最优子博弈策略与价值函数；部署时，在每个时间步进行在线树搜索，利用预先计算的子博弈策略和价值函数指导树的扩展并估计叶节点价值。实验在多种图拓扑结构（包括真实世界网络）上验证了PGTS显著优于当前最先进的学习与启发式基准方法，同时展现出对对抗性环境的鲁棒性能。
链接: https://arxiv.org/abs/2607.08892
作者: Mukesh Kumar,Yue Guan,Panagiotis Tsiotras
机构: 未知
类目: Computer Science and Game Theory (cs.GT); Multiagent Systems (cs.MA)
备注:
Abstract:Computing Nash equilibrium policies in multi-agent Pursuit-Evasion games (PEG) is challenging due to the exponential growth of the joint state and action spaces with the number of agents. Existing approaches either rely on offline equilibrium approximations, which may lack adaptability during execution, or online planning methods, which suffer from large branching factors. In this work, we propose Primitive-Guided Tree Search (PGTS), a hybrid framework that integrates offline exact Nash equilibrium computation with online tree search: PGTS first solves a collection of smaller, tractable sub-games offline; at deployment, PGTS performs online tree search at each time step, using the optimal sub-game policies and value functions to guide tree expansion and estimate leaf-node values. Extensive experiments on varied graph topologies, including real-world networks, demonstrate that PGTS significantly outperforms state-of-the-art learning and heuristic baselines, while maintaining robust performance against adversaries.
[MA-6] SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets
【速读】：该论文旨在解决在物理-信息融合的能源市场环境中，自主智能体（agentic AI）在提升市场效率的同时可能引发物理安全风险与治理不稳定性的问题。其核心挑战在于如何在评估智能体性能时兼顾任务表现（如市场效用）与可信性（trustworthiness），尤其是防止智能体利用虚假物理数据、制造人为流动性或做出不稳定的决策。为此，论文提出SolarChain-Eval——一个基于物理约束的基准测试框架，将市场治理建模为兼容Gymnasium的马尔可夫决策过程（Markov Decision Process, MDP），支持智能体以小时为单位进行决策。该框架从市场效用、物理安全性、滑点、动作平滑性、空间公平性及可审计性等多个维度综合评估智能体策略。其关键解决方案是引入基于大语言模型（LLM）的规划器/审计器（Planner/Auditor）层：规划器设定策略层面的动作边界与审计规则，审计器对高风险行为进行审查与修正，并通过结构化日志完整记录干预过程（包括触发信号、提议动作、修正动作及审计依据）。实验表明，强化学习（RL）代理虽能提升市场效用，但存在安全隐患；当移除物理惩罚项后，奖励最大化代理会滥用无效发电数据并增加人工流动性。而引入LLM规划器/审计器虽显著提升了可审计性并缓解部分风险，但仍无法弥补奖励函数设计不当带来的根本性缺陷。研究结论强调，可信智能体评估必须同时依赖物理约束机制与透明的干预追溯能力。相关数据与代码已开源，确保研究可复现性。
链接: https://arxiv.org/abs/2607.08681
作者: Shilin Ou,Yifan Xu,Luyao Zhang
机构: Duke Kunshan University (杜克昆山大学)
类目: Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Machine Learning (cs.LG); Multiagent Systems (cs.MA); General Economics (econ.GN)
备注:
Abstract:As agentic AI systems are increasingly applied to cyber-physical environments, their evaluation requires assessment of both task performance and trustworthiness. In decentralized energy markets, autonomous agents may improve market utility, but may also exploit invalid physical data, create artificial liquidity, and produce unstable governance decisions. Therefore, we propose SolarChain-Eval, a physics-constrained benchmark for evaluating trustworthy economic agents. It formulates market governance as a Gymnasium-compatible Markov Decision Process, where agents make hourly decisions. SolarChain-Eval evaluates each policy across multiple dimensions, including market utility, physical safety, slippage, action smoothness, spatial fairness, and auditability. To support agentic evaluation, SolarChain-Eval incorporates an LLM-based Planner/Auditor layer. The Planner defines episode-level action bounds and audit rules, while the Auditor reviews and revises high-risk actions. All interventions are recorded through structured logs, including trigger signals, proposed actions, revised actions, and audit rationales. Experiments with static, random, myopic, RL, and RL+LLM policies reveal a clear utility-safety trade-off. RL agents improve market utility but can still produce unsafe behavior. When the physics penalty is removed, reward-maximizing agents exploit invalid generation and increase artificial liquidity. The LLM Planner/Auditor improves auditability and mitigates selected risks, but it cannot fully compensate for a misspecified reward function. These results indicate that trustworthy agentic AI evaluation requires both physical constraints and transparent intervention traces. We release data and code as open access on GitHub for replicability.
[MA-7] Control Laguerre Tessellation: Semi-discrete Optimal Transport Over Control Systems
【速读】：该论文旨在解决在最优控制框架下，从一个紧支撑的绝对连续源分布到离散目标测度之间的最优传输问题。其核心挑战在于，传输代价由受控代理运动的最优代价所诱导，且需在控制约束条件下实现高效、精确的路径规划与分布匹配。解决方案的关键在于：当诱导的基底代价满足扭转条件（twist condition）时，最优传输映射几乎处处可由状态空间上的控制拉盖尔剖分（Control Laguerre Tessellation, CLT）给出。该方法将经典拉盖尔剖分推广至控制论场景，通过引入受控系统动态特性，构建出基于最优控制策略的几何分割结构；研究进一步以线性受控代理在最小能量和最短时间目标下的两种典型情形为例，验证并可视化了CLT的有效性与结构性特征。
链接: https://arxiv.org/abs/2607.09139
作者: Ripon C. Sarker,Abhishek Halder
机构: Iowa State University (爱荷华州立大学)
类目: Optimization and Control (math.OC); Machine Learning (cs.LG); Multiagent Systems (cs.MA); Systems and Control (eess.SY)
备注:
Abstract:We study the optimal transport of optimally controlled agents from a compactly supported absolutely continuous source to a discrete target measure. The ground cost for the transport is induced by the optimal cost of the agents’ motion. When this ground cost satisfies the twist condition, the optimal transport map is given almost everywhere in terms of a Laguerre tessellation of the state space. We refer to this control-theoretic generalization of Laguerre tessellation as Control Laguerre Tessellation (CLT), and illustrate it for two ground costs induced by linear controlled agents with minimum energy and minimum time objectives.
自然语言处理
[NLP-0] ask-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026 ICML2026
【速读】： 该论文旨在解决在资源受限条件下，高效处理多模态问答任务中的两大核心挑战：一是对“抛掷问题”（Tossup）中在不确定情境下判断最佳回答时机的决策问题，二是对“附加问题”（Bonus）中实现高精度答案选择并提升人类可接受性的难题。其解决方案的关键在于提出一种任务特异性的双代理架构：针对Tossup任务，采用经置信度校准的GPT-4o-mini类模型（即竞赛日志中的GPT-4.1-mini），结合领域特定的数值推理策略，有效抑制仅基于孤立数量线索产生的过度自信预测；针对Bonus任务，则使用GPT-4o类模型（即GPT-4.1），引入首句感知推理、结构化关系推理与多模态证据融合机制，以增强精确答案选择能力。整个系统摒弃了传统的检索流水线或模型集成方法，转而聚焦于轻量级、高效的推理策略与置信度校准，在纯托管环境中实现了最优性能，最终在基准测试中取得0.402的总分（Tossup得分0.238，Bonus效果得分0.164），验证了任务定制化轻量化推理策略在资源受限多模态问答场景下的有效性。
链接: https://arxiv.org/abs/2607.09623
作者: Nirjhar Das,Md. Al-Mamun Provath
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: 10 pages, 1 figure. Accepted at the EMM-QA 2026 Workshop, ICML 2026 (Non-Archival). Rank #1 overall system in the QANTA 2026 Challenge
Abstract:We present our submission to the QANTA 2026 shared challenge at the ICML 2026 Workshop on Efficient Multimodal Question Answering (EMM-QA). Quanta evaluates multimodal quizbowl systems that answer pyramid-style questions from incrementally revealed text and accompanying images while operating under realistic efficiency constraints. The challenge consists of two distinct tasks: Tossup questions, which require deciding when to answer under uncertainty, and Bonus questions, which emphasize accurate answer selection and human adoption. To address these differing objectives, we develop a task-specific two-agent architecture. Our Tossup agent utilizes a GPT-4o-mini-class model (referred to as GPT-4.1-mini in the competition logs) with confidence-calibrated answering and a domain-specific numeric reasoning policy that reduces overconfident predictions from isolated quantitative clues. Our Bonus agent uses GPT-4o-class model (referred to as GPT-4.1) with leadin-aware reasoning, structured relational reasoning, and multimodal evidence integration to improve exact answer selection. Rather than relying on a retrieval pipeline or model ensembles, our approach emphasizes efficient reasoning policies and confidence calibration within a hosted-only environment. Our system achieved the highest overall leaderboard score of 0.402, including a Tossup score of 0.238 and a Bonus Effect score of 0.164. The results demonstrate that lightweight, task-specific reasoning strategies can provide strong performance on resource-constrained multimodal question answering benchmarks.
[NLP-1] oward Real-Time Sentence-Level Sign Language Translation
【速读】： 该论文旨在解决现有手语理解系统多局限于孤立手语符号（isolated signs）处理，难以在自然交流场景中实现高效应用的问题。其核心目标是推动句级手语翻译（sentence-level sign language translation, SLT）的实时部署，而非提出新型翻译架构。解决方案的关键在于构建一个面向硬件资源约束的流式系统：采用基于QLoRA的微调策略，在保持SHuBERT编码器冻结的前提下，对SHuBERT-ByT5翻译模型进行轻量化微调；同时设计了一套高效的分布式计算框架——以树莓派4B作为客户端完成视频采集、本地文本显示与语音输出，而计算密集型的感知与翻译任务则由云端CPU/GPU后端处理。通过分块数据输入、有界队列管理、并行化感知处理、时间重排序以及基于句末边界的有限状态机等技术手段，显著降低了响应延迟：在9,872例测试样本上，平均后置完成延迟从1.873秒降至1.354秒（降低27.71%），P95延迟从2.919秒降至2.130秒（降低27.03%），实现了低延迟、高可扩展性的实时手语翻译系统。
链接: https://arxiv.org/abs/2607.09611
作者: Thanh-Hoang Nguyen Doan(The University of Danang - University of Science and Technology)
机构: The University of Danang – University of Science and Technology (越南岘港大学-科学技术大学)
类目: Computation and Language (cs.CL)
备注: 8 pages, 4 figures, 9 tables
Abstract:Most sign language understanding systems operate at the level of isolated signs, limiting their usefulness in natural communication. We study sentence-level sign language translation (SLT) with the primary goal of real-time deployment rather than proposing a new translation architecture. We fine-tune a SHuBERT-ByT5 translation stack on a uniformly sampled 9,872-example subset of How2Sign, selected because of compute and storage constraints, using QLoRA while keeping SHuBERT frozen. The model obtains a validation BLEU of 16.7 and, on the test split, BLEU 15.9 and BLEURT 44.7. The main contribution is a hardware-aware streaming system: a Raspberry Pi 4B reference client provides camera capture, local text display, and speech output, while compute-intensive perception and translation run on a CPU/GPU backend. The capture protocol remains client-agnostic, so the same backend can serve a browser, phone, or laptop. Chunked ingestion, bounded queues, parallelized perception, temporal reordering, and a sentence-boundary state machine reduce mean post-finalization response latency from 1.873 to 1.354 seconds (27.71%) and P95 latency from 2.919 to 2.130 seconds (27.03%) over the complete 9,872-example working subset.
[NLP-2] Agora: Enhancing LLM Agent Reasoning Via Auction-Based Task Allocation
【速读】： 该论文旨在解决大语言模型（Large Language Model, LLM）代理在执行复杂推理任务时，如何有效协调多种专家模型与工具以提升推理能力的问题。现有框架通常基于粗粒度的任务-功能匹配调用API，忽视了功能相似的替代方案之间在性能波动性和成本效率上的差异。为此，本文提出Agora框架，其核心创新在于引入一种激励相容的拍卖机制，实现对专家模型和工具的动态任务分配。通过将推理步骤视为可交易的资源，Agora使代理能够基于经过校准的能力（rectified competence）进行出价，确保关键逻辑被分配给最胜任的求解器，而非仅表现出高自信的模型。实验结果表明，在五个基准测试中，Agora在与单模型、路由及级联基线相当的候选池条件下均取得更优表现，并可通过单一拍卖参数灵活调控成本与质量之间的权衡关系。
链接: https://arxiv.org/abs/2607.09600
作者: Kaiji Zhou,Ales Leonardis,Yue Feng
机构: University of Birmingham (伯明翰大学)
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: Preprint. 12 pages, 4 figures
Abstract:Enhancing the reasoning capabilities of large language model (LLM) agents requires effective orchestration of diverse expert models and tools. However, existing frameworks typically call APIs based on coarse-grained matching between tasks and the functions of expert models or tools, while overlooking critical factors such as performance variability and cost efficiency among functionally similar alternatives. To address this, we propose Agora, a framework that introduces an incentive-compatible auction mechanism for dynamically allocating tasks to expert models and tools. By treating reasoning steps as tradeable items, Agora enables agents to bid based on their rectified competence-ensuring that critical logic is routed to the most capable solver rather than the most overconfident one. Evaluations across five benchmarks show that Agora improves over matched single-model, routing, and cascade baselines under comparable candidate pools, while exposing a controllable cost-quality trade-off through a single auction parameter.
[NLP-3] okenizer Transplantation: Mitigating Autoregressive Collapse in Edge-Efficient Bengali ASR ICML2026
【速读】： 该论文旨在解决轻量级语音识别模型在边缘部署中对形态丰富、非拉丁语系语言（如孟加拉语）适应性差的问题。其核心挑战在于现有高度优化的模型架构（如Moonshine）依赖以英语为中心的字节级分词器，导致孟加拉语词汇被切分为高繁殖力（high-fertility）的字节序列，从而在自回归推理过程中引发灾难性崩溃。解决方案的关键是提出一种新颖的词汇移植（vocabulary transplantation）流程：将解码器词汇表替换为原生脚本的BanglaBERT WordPiece词汇，并相应地调整词嵌入矩阵大小。实验表明，该方法使词元繁殖力从9.16降至1.30，自回归序列长度减少85.8%，彻底消除了解码不稳定性。在882小时的Lipi-Ghor数据集上，改进后的模型实现了21.54%的词错误率（Word Error Rate, WER）和0.0053的实时因子（Real-Time Factor, RTF），验证了其高效性与鲁棒性。本研究为紧凑型自动语音识别（ASR）模型在跨语言、跨脚本场景下的可扩展、可复现适配提供了无需资源密集型预训练的通用范式。
链接: https://arxiv.org/abs/2607.09598
作者: Sanjid Hasan,Md. Abdur Rahman
机构: 未知
类目: Computation and Language (cs.CL)
备注: 5 pages, 2 figures. Accepted as a poster at the MusIML Workshop, ICML 2026
Abstract:Lightweight speech recognition models are critical for edge deployment, yet highly optimized architectures like Moonshine often fail on morphologically rich, non-Latin languages such as Bengali. This study identifies the root cause of this failure as the model’s English-centric byte-level tokenizer, which fragments Bengali words into high-fertility byte chains and triggers catastrophic autoregressive collapse during inference. To resolve this, a novel vocabulary transplantation pipeline is proposed to replace the decoder vocabulary with the native-script BanglaBERT WordPiece vocabulary and resize the corresponding token embedding matrix. Experimental results demonstrate a reduction in token fertility from 9.16 to 1.30. By decreasing autoregressive sequence length by 85.8%, decoding instability is entirely mitigated. When evaluated on the 882-hour Lipi-Ghor dataset, the modified architecture achieves a competitive 21.54% Word Error Rate (WER) and a Real-Time Factor (RTF) of 0.0053. Ultimately, this research provides a scalable, reproducible blueprint for cross-script adaptation of compact ASR models without the need for resource-intensive pre-training.
[NLP-4] Conceptual Networks for Cross-Linguistic Idiomatic Expressions:A Feature-Based Graph Approach
【速读】： 该论文旨在解决跨语言习语与隐喻意义的可解释性表征问题，尤其针对多语言环境下习语语义的深层认知结构难以捕捉的挑战。其解决方案的关键在于构建一个基于概念特征的网络框架：通过认知语言学理论提取二元概念特征（如包含、隐藏、情感、社会性等），利用成对Jaccard相似度构建加权图，并通过社区检测揭示习语按概念模式聚类而非按语言分布的结构，从而实现对习语语义的可解释建模。该框架不仅捕捉到分布嵌入无法体现的独特语义信息，还可通过大语言模型（LLM）实现自动标注扩展，显著提升习语识别性能，且在引入语料频率后仍保持鲁棒性。跨语言迁移实验表明，仅依赖概念邻近性即可有效识别五种语言家族间的可接受翻译对应项，优于基于嵌入的基线方法。消融研究进一步证实，概念模式、角色关系与情感极性三个维度均对网络组织特性及任务表现具有非冗余贡献，且图结构衍生信号（如社区归属、邻居相似性）尤为关键。整体框架实现了理论基础与实际应用的融合，提供了跨语言稳定、可解释的习语意义表示。
链接: https://arxiv.org/abs/2607.09576
作者: Kiran Pala,Punam Silu,Lixun Yu
机构: University of Eastern Finland (芬兰东大学); Indian Institute of Technology Ropar (印度理工学院拉普尔分校)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET)
备注:
Abstract:We present an interpretable network-based framework for representing idiomatic and figurative meaning across eight typologically diverse languages, totaling 160 conventional expressions, the large majority of which are idiomatic. Each expression is annotated with binary conceptual features (containment, concealment, emotional, social, etc.) derived from cognitive-linguistic theory, and pairwise Jaccard similarities define a weighted graph. Community detection reveals that idioms cluster by conceptual schema rather than by language, producing a structure consistent with cognitive-linguistic predictions. The conceptual network captures unique semantic information not present in distributional embeddings, can be scaled via automatic annotation with LLMs, improves downstream idiom detection, and remains robust when enriched with corpus frequencies. Cross-lingual transfer experiments show that conceptual proximity alone can identify acceptable translation equivalents across five language families, with substantial gains over embedding-based baselines. Ablation studies demonstrate that all three feature dimensions – schemas, roles, and valence – contribute non-redundantly to both the network’s organizational properties and its performance on idiom detection, and that specific graph-derived signals (community membership, neighbor similarity) are particularly informative. The framework offers an interpretable, cross-linguistically stable representation of idiomatic meaning, combining theoretical grounding with practical utility.
[NLP-5] FreyaTTS Technical Report
【速读】： 该论文旨在解决土耳其语文本到语音（TTS）合成中模型参数量大、推理效率低以及对音素转换等预处理模块依赖性强的问题，尤其针对资源受限边缘设备部署的挑战。其核心解决方案的关键在于：构建一个183.2M参数的非自回归条件流匹配扩散变换器（DiT），在冻结的连续潜在空间（AudioVAE2，16 kHz编码，48 kHz解码）中运行，使模型专注于文本到潜在表示的映射，同时继承高质量的48 kHz语音重建能力；通过无需规则的端到端建模，直接使用92个土耳其字符词汇表，摒弃了传统的音素化器、音素-音位转换前端和离散语音分词器；采用非自回归并行去噪机制，在预测的时长基础上一次性生成整个潜在序列，显著提升推理效率；此外，引入面向生产环境的两阶段后训练策略，包括单说话人声纹锁定与短语句覆盖优化，有效增强说话人一致性与短输入鲁棒性。实验表明，该模型在Freya-TR-Eval基准上实现了8.0%的带宽匹配词错误率（WER）和3.0%的字符错误率（CER），优于参数量更大的开源系统，且实时因子达0.11，可在消费级GPU上实现高效推理，甚至在笔记本电脑CPU上实现超实时运行，充分满足边缘部署需求。
链接: https://arxiv.org/abs/2607.09530
作者: Ahmet Erdem Pamuk,Ömer Yentür,Ahmet Tunga Bayrak,Yavuz Alp Sencer Öztürk,Mustafa Yavuz
机构: 未知
类目: Computation and Language (cs.CL)
备注:
Abstract:We introduce Freya-TTS, a compact, tokenizer-free, Turkish-first text-to-speech model designed for highly reliable and efficient conversational synthesis. Freya-TTS is a 183.2M-parameter non-autoregressive conditional flow-matching Diffusion Transformer (DiT) that operates in the frozen continuous latent space of AudioVAE2 (16 kHz encode, 48 kHz decode), allowing the model to focus its capacity on text-to-latent mapping while inheriting high-quality 48 kHz reconstruction. We advance the framework along three key dimensions: (1) rule-free end-to-end modeling from a 92-symbol Turkish character vocabulary without a phonemizer, grapheme-to-phoneme frontend, or discrete speech tokenizer; (2) non-autoregressive parallel denoising, which predicts the entire latent sequence simultaneously over a predicted duration; and (3) a production-oriented two-stage post-training recipe consisting of single-speaker voice locking and short-utterance coverage, improving speaker consistency and robustness on short inputs. On the Freya-TR-Eval benchmark, Freya-TTS achieves a band-matched word error rate (WER) of 8.0% and character error rate (CER) of 3.0%, outperforming substantially larger open-source systems while using a fraction of their parameters. The model achieves a real-time factor of 0.11 on consumer GPUs and runs faster than real time on a laptop CPU, making it well suited for resource-constrained edge deployment. We release the model weights, training and inference code, and evaluation benchmark under the Apache-2.0 license.
[NLP-6] Normalisation-Based Likelihood Ratio Estimation for Forensic Authorship Verification
【速读】： 该论文旨在解决作者身份验证（Authorship Verification, AV）中似然比（likelihood ratio）校准问题，尤其针对现有基于评分的AV方法需依赖额外校准模型以获得可靠似然比所带来的数据、时间和计算复杂性挑战。其核心问题是：在缺乏充分案例相关数据时，如何在不使用独立校准模型的前提下，仍能生成准确且可解释的似然比，从而保障法证文本分析的可靠性与可操作性。解决方案的关键在于提出两种新型归一化技术——平方根校正（Square Root Correction）和罕用词校正（Hapax Correction），通过直接对LambdaG方法输出的原始得分进行修正，有效缓解长文本或高重复性文本导致的证据强度过度估计问题。实验结果表明，所提方法在15个语料库、多种文本长度（100–9,500词元）下表现与逻辑回归校准相当，其中罕用词校正更优的情况占比达约45%（按语料加权），且当其表现略逊于逻辑回归时，差距通常较小（<5%），显示出更强的稳健性。该方法无需训练校准模型，显著降低数据需求与流程复杂度，提升了法证应用中的可及性与透明度，具备良好的实用价值与推广前景。
链接: https://arxiv.org/abs/2607.09501
作者: Sadie Barlow,Andrea Nini,Edoardo Manino
机构: University of Manchester (曼彻斯特大学)
类目: Computation and Language (cs.CL); Applications (stat.AP)
备注:
Abstract:Authorship verification (AV) is the task of determining whether two texts were written by the same author. In a forensic context, the strength of AV evidence can be quantified using likelihood ratios. Most AV methods are score-based and deriving well-calibrated likelihood ratios from these scores requires a separate calibration model. This, in turn, requires additional amounts of case-relevant data, which is often time-consuming to obtain and prepare. This study proposes two novel normalisation techniques, the Square Root Correction and the Hapax Correction, for deriving likelihood ratios from the AV method LambdaG without the need of a calibration model (Nini et al. 2026). These corrections are designed to mitigate the overestimation of evidential strength that may result from long or highly repetitive texts. Performance is evaluated against logistic regression calibration across fifteen corpora and a range of text lengths (100-9,500 tokens), using the log-likelihood ratio cost (Cllr). The proposed methods achieve performance comparable to logistic regression calibration, with the Hapax Correction outperforming it in approximately 45% of tests (weighted by corpora). Furthermore, performance was more frequently close (within 5%) when the Hapax Correction was outperformed by logistic regression calibration, compared with the reverse comparison. Eliminating the need to train a calibration model reduces data-requirements, time and complexity, thereby increasing the accessibility and transparency of forensic text comparison. This combination of empirical performance and practical advantages supports the adoption of the proposed methods in forensic settings.
[NLP-7] Neural Collapse Is Forbidden: Information Floors in Language Models
【速读】： 该论文旨在解决语言模型表征中类内方差（within-class variance）的语义解释问题，传统观点将其视为神经坍缩（neural collapse）的不完全表现，而本文提出类内方差实则承载了信息存储功能，并遵循一条普适规律。其核心解决方案在于揭示：类内方差并非噪声或缺陷，而是与上下文信息容量直接相关的信息编码载体。通过一个简洁的一行中心化恒等式（one-line centering identity），作者推翻了此前关于单纯形等角紧框架（simplex equiangular-tight-frame）的多种假设，发现14个不同规模的语言模型中，宏观类别结构仅贡献4–12%的表征方差，而单个词元的上下文信息则占79–91%，且这一比例在参数量相差百倍的模型间保持稳定。理论层面，词元级权重衰减对类别惩罚机制与类型数量（type count）成正比，而非出现频率（occurrence mass），从而将下一词预测转化为一类不平衡的K分类问题，其最优解按类型数量对类别范数进行排序。进一步地，针对二分类情形证明的下界表明，类内离散度至少与条件互信息 $ I(\text{token}; \text{context} \mid \text{category}) $ 成正比。最终确立的核心规律是：类内离散度（identity dispersion）而非总方差，始终与信息量高度一致，这一规律在所有测试模型和划分方式下均成立，甚至在跨模型间表现出可预测性——一个模型的信息能有效预测另一个模型的离散度；此外，在预训练过程中，类别方差占比先超调、后衰减并部分恢复，说明其所承载的信息始终未丢失。
链接: https://arxiv.org/abs/2607.09487
作者: Bruno Abrahao
机构: New York University (纽约大学); NYU Shanghai (纽约大学上海分校)
类目: Machine Learning (cs.LG); Computation and Language (cs.CL); Machine Learning (stat.ML)
备注:
Abstract:Within-class variance in language-model representations is commonly read as incomplete neural collapse. We argue it is allocated information storage, and that the allocation obeys a law. A one-line centering identity voids a family of simplex equiangular-tight-frame claims, including our own earlier ones; in dimensionless variance shares across 14 models, macro-category structure carries only 4-12% of representational variance and within-token context carries 79-91%, stable across a 100x parameter range. On the theory side, token-level weight decay penalizes a category in proportion to its type count, not its occurrence mass, reducing next-token prediction to an imbalanced K-class problem whose optimum orders category norms by type count. A converse floor, proved for binary categories, forces within-category dispersion to be at least proportional to the conditional mutual information I(token; context | category). The law holds: identity dispersion, not total variance, tracks this information across every tested model and partition, under a model-free estimate and even across models, where one model’s information predicts another’s dispersion; and over pretraining the category share overshoots, decays, and partially recovers, because the information it must carry never left.
[NLP-8] st-Time Scaling for Small VLMs on Multilingual Visual MCQ
【速读】： 该论文旨在解决生成式 AI（Generative AI）在小规模开放视觉-语言模型中，测试时缩放（Test-time scaling, TTS）方法是否能够有效提升推理能力的问题。其核心发现是：TTS效果的关键不在于复杂的搜索或验证机制，而在于运行条件的设置。具体而言，影响性能的主要因素是推理链的可解析性（parseability）——早期提示格式导致模型虽能正确推理却无法明确选择答案选项，通过引入标准答案提示和引导式修复步骤可显著改善这一问题。此外，更大的解码预算亦至关重要：将每条推理链的令牌上限从1k提升至2k可带来3.7个百分点的准确率提升，而增加推理链数量（8到16）仅贡献0.15个百分点。一旦推理链具备充分的长度空间完成推理，复杂方法如基于偏好评分模型（PRM）的束搜索、无训练生成式批评器或训练好的多模态PRM均表现不佳，甚至远逊于简单的多数投票策略。最终，模型自身能力的提升（+11.4 pp）成为最大增益来源。最优配置在ImageCLEF 2026的保留测试集上达到84.1%准确率，位居视觉多选题（Visual MCQ）排行榜首位。
链接: https://arxiv.org/abs/2607.09438
作者: Spiros Baxevanakis,Peng-Jian Yang
机构: University of Amsterdam(阿姆斯特丹大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: 14 pages, 2 figures, accepted at ImageCLEF 2026
Abstract:Test-time scaling (TTS) reliably improves reasoning in large language models, but whether it transfers to small open vision-language models remains unclear. We examine this on EXAMS-V, a multilingual visual multiple-choice benchmark, comparing self-consistency, describe-then-reason with PRM-guided beam search, and two post-hoc selectors across Qwen2.5-VL-7B-Instruct and Qwen3.5-4B. What matters is the conditions under which TTS runs, not the search or verification machinery. The largest factor is parseability: an early prompt format left many chains reasoning correctly yet never committing to an answer letter, which a standard answer cue and a guided repair step largely remove. A larger decoding budget removes the rest: raising the per-chain token limit from 1k to 2k recovers 3.7 pp, whereas sampling more chains (8 to 16) adds only 0.15 pp. Once chains have room to finish, elaborate methods contribute little: PRM-guided beam search trails plain self-consistency by 0.39 pp at over eight times the cost, and neither a training-free generative critic nor a trained multimodal PRM beats majority vote across both policies. The largest gain comes instead from the policy model itself (+11.4 pp). Our best configuration reaches 84.1% on the held-out ImageCLEF 2026 test split, ranking first on the Visual MCQ leaderboard.
[NLP-9] A Sovereign Open-Source Foundation Model for German and English
【速读】： 该论文旨在解决大语言模型在长上下文、高并发场景下推理效率低下的问题，尤其针对德语和英语双语任务中的性能与资源消耗之间的矛盾。其核心挑战在于如何在保持模型强大表达能力的同时，实现参数高效激活与近似恒定的缓存开销，以应对日益增长的上下文长度。解决方案的关键在于提出一种主权、开源的混合专家（Mixture-of-Experts, MoE）混合结构——Soofi S 30B-A3B，该模型采用仅激活300亿参数中30亿参数的机制，结合Mamba架构的高效序列建模能力，使每令牌推理时的活跃参数量显著降低，并维持近似不变的缓存大小，从而在长上下文场景下实现远超稠密模型的吞吐量优势。此外，模型在约27万亿词元的数据集上预训练，刻意加权德语数据，使其在德英双语基准测试中表现优于多数14至270亿参数的稠密模型，且在代码生成任务中领先于17个开源基线模型，同时超越所有欧洲主权基线模型，包括参数规模更大的版本。该模型基于德国工业人工智能云（German Industrial AI Cloud）端到端构建，确保数据主权与基础设施自主性，其发布遵循高度宽松的开放访问协议，包含权重、中间检查点、完整数据来源记录、超参数及训练评估代码，部分数据构建资产亦依许可情况开放，商业授权数据则提供详尽统计与混合比例信息。
链接: https://arxiv.org/abs/2607.09424
作者: TheSoofi-Team:Benedikt Droste,David Fitzek,Ruben Härle,Lukas Helff,Maximilian Idahl,Alex Jude,Abbas Goher Khan,Maurice Kraus,Timm Ruland,Richard Rutmann,Sebastian Sztwiertnia,Markus Frey,Daniil Gurgurov,Jan Pfister,Tom Röhr,Sebastian von Rohrscheidt,Jörg Bienert,Nicolas Flores-Herr,Simon Gottschalk,Andreas Hotho,Kristian Kersting,Joachim Köhler,Alexander Löser,Wolfgang Nejdl,Simon Ostermann,Jan Plogsties,Patrick Putzky,Mehdi Ali,Michael Fromm,Max Lübbering
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment. Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14 to 27B models on aggregate English and German benchmarks while achieving the best code aggregates in both languages among 17 open base models, and outperforms every European sovereign baseline in our comparison, including ones far larger in active parameters. Among fully open models, Soofi S obtains the highest English and German evaluation scores, ahead of Olmo 3 32B and Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPC scale AI infrastructure operated by Deutsche Telekom in Munich. Soofi S will be released under highly permissive, open-access terms: weights, selected intermediate checkpoints, full per-source data accounting, hyperparameters, and training and evaluation code. Where source licenses permit, data-construction artifacts are released under permissive licenses; commercially licensed sources are documented with aggregate statistics and exact mixture accounting.
[NLP-10] Self-Guided Test-Time Training for Long-Context LLM s
【速读】： 该论文旨在解决大语言模型（Large Language Models, LLMs）在处理长上下文时，尽管上下文窗口扩展，但实际有效利用长输入的能力依然不足的问题。随着输入长度增加，模型性能常出现下降，核心原因在于模型难以准确识别并聚焦于与特定问题最相关的证据片段。现有解决方案如测试时训练（Test-Time Training, TTT）虽能通过实例化参数微调提升适应性，但若对整个长上下文进行训练则计算成本过高，而随机采样片段进行训练又引入严重噪声——因多数片段与问题无关，反而可能损害基线模型性能。本研究发现，TTT对训练片段的质量高度敏感：在LongBench-v2基准上，随机采样片段的TTT会降低性能，而使用“最优”（oracle）证据片段则显著提升表现。为此，作者提出一种简单高效的方法——自引导测试时训练（Self-Guided Test-Time Training, S-TTT）：在参数适配前，由模型自身主动识别出应学习的关键证据片段，并仅对这些选定片段应用标准语言建模目标进行训练。该方法在两个高难度长上下文推理基准（LongBench-v2 和 LongBench-Pro）上均取得显著效果，对Qwen3-4B-Thinking-2507和Llama-3.1-8B-Instruct模型的准确率均有提升，最高实现15%的相对改进。其关键创新在于通过模型自我筛选高质量训练片段，避免无效或干扰信息，从而在保持低计算开销的同时大幅增强长上下文理解能力。
链接: https://arxiv.org/abs/2607.09415
作者: Xinyu Zhu,Zhe Xu,Xiaohan Wei,Yunchen Pu,Fei Tian,Chonglin Sun,Kaushik Rangadurai,Hua Zhi,Frank Shyu,Sandeep Pandey,Luke Simon,Yu Meng,Xi Liu
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:Long-context processing has become increasingly important for large language models (LLMs), but simply extending the context window does not guarantee effective utilization of long inputs. As input length grows, accuracy often degrades, indicating that models still struggle to identify and use the evidence most relevant to a question. A promising way to improve long-context utilization is test-time training (TTT), which treats the test context as a training example for instance-specific parameter adaptation. However, applying TTT to the entire long context is prohibitively expensive, while adapting on randomly sampled spans introduces severe noise. Because most spans in a long context are irrelevant to the specific question, training on them may even degrade the base model’s performance. Our preliminary study shows that TTT is highly sensitive to training-span quality: on LongBench-v2, TTT on randomly sampled spans hurts performance, whereas TTT on oracle spans substantially improves it. Motivated by this, we propose a simple method, Self-Guided TTT (S-TTT): before adaptation, the model identifies the evidence spans it should learn from, and the standard language-modeling training objective is applied only to those selected spans. On two challenging long-context reasoning benchmarks, LongBench-v2 and LongBench-Pro, S-TTT improves accuracy for both Qwen3-4B-Thinking-2507 and Llama-3.1-8B-Instruct, achieving up to a 15% relative improvement.
[NLP-11] Mach-Mind-4-Flash Technical Report
【速读】： 该论文旨在解决大模型在保持高性能的同时，如何有效降低计算资源消耗与推理成本的问题，尤其针对当前主流大模型依赖高参数量（如100B级别）带来的高昂训练与部署开销。其核心挑战在于：如何在不显著增加预训练计算的前提下，通过后训练优化实现接近甚至超越更大规模模型的性能表现。解决方案的关键在于提出一套端到端的高效训练与压缩框架——Mach-Mind-4-Flash，其核心创新包括：(1) 构建统一的强化学习/操作者驱动训练（RL/OPD）基础设施，结合动态多教师调度与操作层级加速机制，实现17%的端到端训练速度提升；(2) 采用多教师在线策略蒸馏（Multi-Teacher On-Policy Distillation, MOPD），通过路由式反向KL目标融合多个领域专用专家（涵盖推理、通用任务与代理行为），有效缓解混合奖励强化学习中的“跷跷板”退化问题；(3) 提出单阶段高效率的混合中位长度策略优化（Hybrid Median-length Policy Optimization, HMPO），在压缩推理链长度19–46%的同时，仅带来≤0.7个百分点的精度损失，显著提升生成效率。最终，该模型以仅350亿参数总量和30亿激活参数，达到多项基准测试领先或持平于10–30倍激活规模模型的表现，且推理成本大幅降低，验证了高效异构专家架构与精细化训练策略在构建轻量化强智能体方面的可行性。
链接: https://arxiv.org/abs/2607.09375
作者: Foundation Model Team(Li Auto Inc)
机构: Li Auto Inc.(理想汽车)
类目: Machine Learning (cs.LG); Computation and Language (cs.CL)
备注:
Abstract:We present Mach-Mind-4-Flash, a 35B-parameter Mixture-of-Experts (MoE) agentic model with 3B activated parameters. Through post-training optimization alone without scaling pre-training compute, the model achieves performance on par with or surpassing that of 100B-parameter-class models. By introducing scalable agentic interaction environments for large-scale reinforcement learning, the model attains significant performance gains on real-world application tasks. Our pipeline comprises three stages: (1) a unified RL/OPD training infrastructure with dynamic multi-teacher scheduling and operator-level acceleration, delivering 17% end-to-end training speedup; (2) multiple domain-specific RL experts trained in parallel across Reasoning, General, and Agent tracks, then fused into a single generalist via Multi-Teacher On-Policy Distillation (MOPD) – a routed reverse-KL objective that eliminates the see-saw degradation of mixed-reward RL; (3) Hybrid Median-length Policy Optimization (HMPO), a single-stage token-efficiency method that compresses reasoning chains by 19–46% with \le 0.7 percentage-point accuracy loss. Mach-Mind-4-Flash scores 92.70 on AIME’26, 82.82 on IFBench, 80.74 on Behavioral-SafetyBench, 75.80 on BFCL-v4, 72.31 on BrowseComp-zh, and 84.20 on ClawBench – leading or matching models with 10–30 \times its activated size at a fraction of the inference cost.
[NLP-12] Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation
【速读】： 该论文旨在解决临床领域中检索增强生成（Retrieval-Augmented Generation, RAG）系统中存在的“欺骗性接地”（Deceptive Grounding, DG）问题，即模型虽能从真实文档中提取证据并满足传统评估指标（如零幻觉、高忠实度、真实引用），却将关于某药物Y的临床证据错误地归因于查询药物X，导致信息误导。其核心挑战在于：现有评估框架无法检测此类实体归属错误，因为所有陈述均有真实来源，仅存在实体错配。解决方案的关键在于引入实体归属验证（Entity-attribution verification），通过检查引用证据是否真正适用于查询实体，从而识别并纠正此类隐蔽错误。实验表明，该方法在人工金标准下可实现97.0%的精确率和98.7%的DG召回率，显著优于现有框架；同时，研究揭示了实体特异性临床证据缺失是导致此类失败的根本机制，且其与虚构生成共用同一触发因素，但路径不同，提示需针对性设计防御策略。
链接: https://arxiv.org/abs/2607.09349
作者: Cedric Caruzzo,Donggeun Yoo,Tae Soo Kim
机构: Lunit (Lunit); KAIST (KAIST); Seoul National University (首尔国立大学); NHIS Ilsan Hospital (NHIS伊兰医院); Ewha Womans University Seoul Hospital (延世大学校首尔医院); Keimyung University Dongsan Medical Center (庆尚大学东山医学中心); Konyang University Hospital (孔阳大学医院); Korea University Research Business Foundation (韩国大学研究业务基金会); Kyung Hee University Hospital at Gangdong (庆熙大学光东医院); Kyung Hee University Medical Center (庆熙大学医学院中心); Pusan National University Yangsan Hospital (釜山国立大学杨山医院); Yongin Severance Hospital (龙仁塞弗伦斯医院)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: 24 pages, 7 figures, 12 tables
Abstract:Retrieval-augmented generation evaluation checks whether model claims are factually grounded in retrieved documents. It does not check whether retrieved evidence is attributed to the correct entity. A clinical RAG response can pass every automated check (zero hallucinations, near-perfect faithfulness, real citations) while presenting drug Y’s clinical evidence as evidence about queried drug X. We term this deceptive grounding (DG): a failure invisible to faithfulness, hallucination, and citation checks because every claim is sourced from a real document, about the wrong entity. Using a controlled factorial benchmark across 13 models, we find DG rates spanning 8-87% at peak adversarial conditions. Medical and biomedical fine-tuned models reach up to 86.7%; domain specialization amplifies the failure rather than mitigating it. A controlled ablation identifies the mechanism: removing entity-specific clinical evidence from retrieved documents eliminates entity-attribution failure entirely, shifting all failures to confabulation. The two failure modes respond to the same trigger, taking different paths. Production measurement across 740 drug-disease pairs finds 7.8% overall DG in a deployed RAG system, rising to 13.6% for recently approved drugs. Entity-attribution verification (checking that cited evidence applies to the queried entity) detects DG at 97.0% precision and 98.7% DG recall (IPW-adjusted human gold standard); no existing framework implements it. Comments: 24 pages, 7 figures, 12 tables Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) Cite as: arXiv:2607.09349 [cs.CL] (or arXiv:2607.09349v1 [cs.CL] for this version) https://doi.org/10.48550/arXiv.2607.09349 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Cedric Caruzzo [view email] [v1] Fri, 10 Jul 2026 12:29:10 UTC (4,137 KB) Full-text links: Access Paper: View a PDF of the paper titled Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation, by Cedric Caruzzo and 2 other authorsView PDFHTML (experimental)TeX Source view license Current browse context: cs.CL prev | next new | recent | 2026-07 Change to browse by: cs cs.AI cs.LG References Citations NASA ADSGoogle Scholar Semantic Scholar export BibTeX citation Loading… BibTeX formatted citation loading… Data provided by: Bookmark checked="checked"class=“labs-tab-input”> Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv (What is alphaXiv?) Links to Code Toggle CatalyzeX Code Finder for Papers (What is CatalyzeX?) DagsHub Toggle DagsHub (What is DagsHub?) GotitPub Toggle Gotit.pub (What is GotitPub?) Huggingface Toggle Hugging Face (What is Huggingface?) ScienceCast Toggle ScienceCast (What is ScienceCast?) Demos Demos Replicate Toggle Replicate (What is Replicate?) Spaces Toggle Hugging Face Spaces (What is Spaces?) Spaces Toggle TXYZ.AI (What is TXYZ.AI?) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower (What are Influence Flowers?) Core recommender toggle CORE Recommender (What is CORE?) Author Venue Institution Topic About arXivLabs arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv’s community? Learn more about arXivLabs. Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?) mathjaxToggle(); We gratefully acknowledge support from our major funders, member institutions, , and all contributors. About Help Contact Subscribe Copyright Privacy Accessibility Operational Status (opens in new tab) Major funding support from
[NLP-13] DKCD: Domain Knowledge-Enhanced Causal Discovery from Unstructured Data
【速读】： 该论文旨在解决在医疗、金融、教育等高专业性领域中，从非结构化数据中进行因果发现所面临的两大核心挑战：（CH1）由于缺乏领域特定知识，导致对数据中隐含但关键的潜在因素识别不足；（CH2）因缺乏基于领域的推理能力，造成因素标注不可靠，进而将错误传播至最终的因果图构建。为应对上述问题，本文提出一种增强领域知识的因果发现框架（DKCD），其关键在于通过三个相互关联的组件实现：（1）知识挖掘（Knowledge Mining），基于可观测因素检索相关领域知识以支持后续因果推理；（2）知识引导的因果推理（Knowledge-guided Causal Reasoning），结合领域知识发现潜在因果因素以解决CH1，并生成关键因果线索以提升标注准确性以应对CH2；（3）因果结构发现（Causal Structure Discovery），基于更完整的因子集合与精准标注构建最终因果图。实验结果表明，DKCD在两个领域特定数据集上显著提升了因果因素识别与因果图构建的性能。
链接: https://arxiv.org/abs/2607.09348
作者: Xin Li,Jin Li,Shoujin Wang,Kun Yu,Fang Chen
机构: University of Technology Sydney (悉尼科技大学)
类目: Computation and Language (cs.CL)
备注:
Abstract:Causal discovery from unstructured data is a challenging yet underexplored task in high-expertise domains such as healthcare, finance, and education. Existing methods typically leverage the general knowledge of large language models (LLMs) to identify causal factors from unstructured data and annotate them into structured data for causal graph construction. However, they remain limited by two key challenges (CHs): (CH1) insufficient identification of latent factors, which are implicit in the data yet essential for causal discovery, due to the lack of domain-specific knowledge; and (CH2) unreliable factor annotation, caused by the lack of domain-grounded reasoning, which propagates errors to the resulting causal graphs. To address these challenges, we introduce a novel Domain Knowledge-enhanced Causal Discovery framework (DKCD) for causal discovery from unstructured data in high-expertise domains with three interconnected components: (1) Knowledge Mining: It retrieves relevant domain knowledge based on observable factors to support subsequent causal reasoning. (2) Knowledge-guided Causal Reasoning: Reasoning with relevant knowledge, it discovers latent causal factors to address CH1 and generates key causal clues for more accurate data annotation to address CH2. (3) Causal Structure Discovery: It constructs the final causal graphs based on a more complete factor set and accurate annotations. Experiments on two domain-specific datasets show that DKCD significantly improves both causal factor identification and causal graph construction.
[NLP-14] owards Detecting Inconsistencies in End-to-end Generated TODs
【速读】： 该论文旨在解决生成式AI在任务导向对话（Task-Oriented Dialogue, TOD）中因大语言模型（Large Language Models, LLMs）产生幻觉而导致的语义不一致问题，尤其是在系统响应需严格遵循领域知识库（如城市内的餐厅信息）时，单一错误信息（如推荐不存在的餐厅）可能引发严重任务失败。其解决方案的关键在于将任务导向对话建模为约束满足问题（Constraint Satisfaction Problem, CSP），其中对话片段作为变量，变量间的约束则捕捉对话连贯性、与知识库的一致性等属性。通过构建一个先识别对话变量、再利用CSP求解器寻找有效变量赋值的流水线，该方法能够对比目标对话与合法解空间，从而检测不一致性，并提出最小修正建议以恢复对话一致性。实验表明，基于CSP的方法在检测不一致方面具有高准确性，且能提供可解释的分析结果。
链接: https://arxiv.org/abs/2607.09338
作者: Tiziano Labruna,Giovanni Bonetta,Bernardo Magnini
机构: Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会)
类目: Computation and Language (cs.CL); Symbolic Computation (cs.SC)
备注: arXiv admin note: substantial text overlap with arXiv:2407.11857
Abstract:Generative AI is profoundly transforming the core technologies behind conversational systems, shifting from component-based to end-to-end approaches. However, Large Language Models (LLMs) may still generate inconsistencies, a critical issue particularly in Task-Oriented Dialogues (TODs), where system responses must strictly adhere to information from a domain knowledge base (e.g., restaurants in a city). A single hallucination (e.g., suggesting a non-existent restaurant) can lead to severe task failures. We investigate a method for automatically detecting inconsistencies by conceptualizing TODs as a Constraint Satisfaction Problem (CSP), where variables represent dialogue segments referencing the conversational domain, and constraints among variables capture dialogue properties such as turn coherence and adherence to domain knowledge. We propose a pipeline that first identifies variables in a target dialogue and then applies a CSP solver to identify valid solutions. By comparing the target dialogue with valid variable assignments, we can detect inconsistencies and suggest minimal changes to ensure dialogue consistency. We demonstrate the high accuracy of the CSP-based approach in detecting inconsistencies, and provide a detailed analysis of our findings.
[NLP-15] WILDTRACE: Benchmarking Natural Evidence Trails in Long-Context Reasoning
【速读】： 该论文旨在解决长文档中复杂问题回答时，证据在文档内部自然分散于远距离段落所带来的推理挑战，即源内证据整合（source-internal evidence integration）问题。现有基准普遍依赖人为植入的线索（如针孔探测题、嵌入事实或逆向设计的多跳链），其证据分布、位置或语域与原文逻辑不符，导致模型表现可能反映的是对数据分布的适应而非真正的因果推理能力。为克服这一局限，本文提出WILDTRACE基准，涵盖214个真实长文档（如技术事故报告、冷门文学叙事）中的481个任务，所有证据路径均源于文档自身的因果、时间与叙事逻辑。其解决方案的关键在于：基于Pearl的因果层次理论与多跳推理分类体系，定义七种源内证据几何结构，系统刻画长文档分析中不同关系需求；并采用“源优先”构建流程，先从文档结构中挖掘候选证据链，再通过多阶段验证（包括线索必要性、答案可接地性、评分标准一致性、污染抗性及可答性）确保任务质量。该方法有效区分了模型的真实推理能力与对人工构造偏差的响应，推动长上下文研究迈向更贴近现实分析场景的智能推理新阶段。
链接: https://arxiv.org/abs/2607.09328
作者: Zixin Chen,Peng Liu,Haobo Li,Rui Sheng,Jianhong Tu,Xiaodong Deng,Fei Huang,Kashun Shum,Dayiheng Liu,Huamin Qu
机构: Hong Kong University of Science and Technology (香港科技大学); Qwen Team, Alibaba Group (通义实验室，阿里巴巴集团)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:Answering complex questions over long documents frequently requires integrating evidence that the source itself disperses naturally across distant passages. In an incident report, the operating condition, design flaw, and missed safety check that jointly explain a disaster may appear dozens of sections apart; in a novel, a character’s true motive may surface only through scenes far removed from the moment it becomes relevant. This source-internal evidence integration is central to real-world long-document analysis, yet existing benchmarks largely sidestep it. Needle probes, planted facts, and reverse-engineered multi-hop chains embed evidence that may differ from the host text in distribution, placement, or register, making it unclear whether strong performance reflects genuine source reasoning or distributional artifacts. We introduce WILDTRACE, a benchmark of 481 tasks over 214 naturally occurring long-form sources such as technical incident reports and lesser-known literary narratives, where all evidence trails arise from the document’s own causal, temporal, and narrative logic. Drawing on Pearl’s causal hierarchy and prior multi-hop reasoning typologies, we define seven source-internal evidence geometries that characterize the distinct relational demands of analytical reading in long documents. A source-first construction pipeline mines candidate trails from document structure before writing questions; each item then undergoes multi-stage validation covering clue necessity, answer groundedness, rubric fidelity, contamination resistance and answerability. As models are increasingly entrusted with real-world high-stakes analytical tasks, this gap between accessing information and reasoning over naturally dispersed evidence emerges as a defining challenge for the next stage of long-context research.
[NLP-16] Letter Lemmatization: One-to-one and Banded RNNs for Reversing Character-Set Simplification and Abbreviation in Medieval Text ICDAR2026
【速读】： 该论文旨在解决中世纪手稿转录过程中字符集动态变化带来的挑战，尤其针对不同转录者实践差异及异构数字化政策导致的字符集不一致性问题。其核心解决方案在于提出一种灵活的字符集转换方法，关键创新在于采用自监督训练的一对一字符级循环神经网络（RNN），通过学习字符间的单射映射关系实现字符集转换，即使在仅有20行文本的情况下仍能恢复约50%的字符错误率（CER）；进一步地，作者将相同网络结构扩展至“带状RNN”（Banded RNNs）框架，利用并行语料库构建的字符级对齐标注进行训练与推理，成功应用于中世纪宪章文本中的缩写展开任务；此外，论文提出一种基于语义相似性的字符度量机制，称为“字母词形化”（letter lemmatization），并开发了一个高效的Python工具库以支持上述所有方法的实现。
链接: https://arxiv.org/abs/2607.09291
作者: Anguelos Nicolaou,Maria Pia Tiseo,Tamas Kovacs,Nicolas Renet,Georg Vogeler
机构: 未知
类目: Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted for publication (after peer review ) in the ICDAR 2026 workshop “VINALDO: 3rd International Workshop on Machine Vision and NLP for Document Analysis”
Abstract:Medieval document transcribers have very different practices; on top of that, heterogeneous digitization policies have resulted in corpora where the character-set must be viewed as fluid. In this paper we address the problem of changing between character-sets in a flexible manner. We focus on one-to-one character mappings and train characterlevel one-to-one RNNs to undo them with self-supervision; recovering half the CER even with 20 text lines. We analyse the use of these one-to-one networks for HTR post-correction and we see that they obtain significant improvements while totally ignoring ins-dels. We then use the exact same networks with character-level alignment groundtruth compiled from parallel corpora in a training and inference mode we call Banded RNNs. We use such networks to successfully expand abbreviations in medieval charter transcriptions. Finally we introduce an elaborate heuristic which takes the characters of two arbitrary character-sets and defines a metric encapsulating what we consider to be semantic similarity of characters. We call the construction of such mappings letter lemmatization and present a rich Python library that efficiently performs all presented methods.
[NLP-17] Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning
【速读】： 该论文旨在解决大语言模型（Large Language Models, LLMs）微调成本高昂的问题，其核心挑战在于全参数微调所需的巨额内存、计算资源及任务专属存储开销。为应对这一问题，论文提出了一种基于稀疏参数高效微调（Parameter-Efficient Fine-Tuning, PEFT）的新方法——Super，其关键在于利用原始用于模型剪枝的显著性信号（saliency signals），通过Wanda风格的激活加权幅度评分（activation-weighted magnitude score）在一次校准过程中确定一个小型可训练支持集（trainable support），从而实现对模型关键参数的精准适应。进一步地，论文提出Supra，一种融合稀疏更新与低秩适配器（LoRA）的混合适配器架构，并通过简单的预算分配规则，在保持与传统方法相当的可训练参数总量的前提下，实现了更优的性能表现。实验结果表明，在Llama-3.2-1B和Meta-Llama-3-8B上的数学推理任务中，最优的Super/Supra变体在所有对比配置中达到了最高的平均准确率；同时，引入类似PaFi的仅基于幅度的无训练稀疏基线也验证了低分项支持集的有效性。这些发现表明，简单启发式的剪枝导向排序即可为PEFT提供有效的固定稀疏支持结构，尤其当与低秩适配器结合时展现出显著优势。
链接: https://arxiv.org/abs/2607.09287
作者: Ivan Ilin,Philip Zmushko,Peter Richtárik
机构: KAUST(沙特国王科技大学); ISTA(奥地利科学院); Yandex Research(叶夫根尼研究)
类目: Machine Learning (cs.LG); Computation and Language (cs.CL)
备注: 26 pages, 3 figures, 19 tables. Code: this https URL
Abstract:Large language models (LLMs) remain expensive to fine-tune because full-parameter updates require substantial memory, compute, and per-task storage. We study whether saliency signals originally developed for pruning can be reused to choose where a model should adapt. We propose Super, a sparse parameter-efficient fine-tuning (PEFT) method that fixes a small trainable support using a Wanda-style activation-weighted magnitude score [Sun et al., 2023] computed from a calibration pass. We then introduce Supra, a hybrid adapter that combines this sparse update with LoRA while preserving a matched trainable-parameter budget through a simple budget-splitting rule. In single-seed Math17K arithmetic experiments on Llama-3.2-1B and Meta-Llama-3-8B, the best Super/Supra variants achieve the highest average accuracy among the tested schedule-selected adapter configurations. We also include a PaFi-style magnitude-only support as a closest training-free sparse baseline and find that low-score supports under both magnitude and Wanda-style orderings can be effective. These results suggest that simple pruning-inspired orderings can provide useful fixed sparse supports for PEFT, especially when combined with low-rank adapters.
[NLP-18] Git-Assistant: Planning -Based Support for Updating Git Repositories
【速读】： 该论文旨在解决版本控制工具（如 Git）在实际使用中对开发者而言存在操作复杂、易出错的问题，尤其是在执行非平凡的 Git 操作时，传统方法难以有效支持。尽管生成式 AI（Generative AI）具备理解开发人员意图的潜力，但其在仓库管理任务中的有效性受限于缺乏形式化推理能力，导致生成的命令序列可能存在不准确或不安全的情况。为此，论文提出 Git-Assistant，一种基于人工智能的辅助系统，其核心解决方案在于将生成式 AI 与自动化规划（Automated Planning）相结合，通过分析代码仓库上下文，将自然语言请求转化为可执行的命令序列，并利用规划技术确保操作的正确性与安全性。实验采用合成及随机化的 Git 环境进行系统评估，对比了仅依赖大语言模型（LLM-only）与引入规划增强（planning-augmented）的两种变体，结果表明，融合形式化推理的混合智能方法显著提升了仓库管理任务的可靠性并降低了错误率，验证了该类协同架构在智能开发者辅助中的巨大潜力。
链接: https://arxiv.org/abs/2607.09224
作者: Alfredo Garrachón Ruiz,Tomás de la Rosa,Daniel Borrajo
机构: JPMorgan Chase (摩根大通)
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 11 pages, 6 Tables
Abstract:Version control systems are essential for collaborative software development, yet tools like git remain challenging for many practitioners. Recent advances in Large Language Models (LLMs) offer promising capabilities for interpreting developer intent, but their effectiveness in repository management tasks is limited by the need for formal reasoning. This work introduces Git-Assistant, an AI-based assistant that combines LLMs with automated planning to support developers in executing non-trivial git operations. The assistant analyzes repository context, translates natural language requests into actionable command sequences, and incorporates planning techniques to ensure correctness and safety. We present a systematic evaluation methodology using synthetic and randomized git environments, comparing the performance of LLM-only and planning-augmented variants across multiple metrics. Experimental results demonstrate that integrating formal reasoning with LLMs improves reliability and reduces errors in repository management, highlighting the potential of hybrid AI approaches for intelligent developer assistance.
[NLP-19] Complexity-Guided Component-wise Initialization for Language Model Pretraining
【速读】： 该论文旨在解决预训练语言模型（Pretrained Language Models, PLMs）中普遍存在的结构化权重谱特征是否可作为GPT-2风格模型预训练的有效初始化信号这一问题。研究发现，不同规模、语言、分词器及训练语料的11个GPT-2类检查点在层间和Transformer子组件层面均表现出一致的深度趋势，尤其是残差写入矩阵（residual-writing matrices）的权重尺度增大与谱集中度增强。为此，研究构建了模仿预训练模型组件级幅值与谱分布特性的初始化方案，并与多种标准权重初始化方法进行对比。尽管这些初始化方案显著改变了模型的初始谱结构，但性能评估未显示出相应的提升。结果表明，直接复用预训练权重仍具优势，而仅通过粗粒度的谱匹配无法构成可靠的优化策略。因此，研究指出预训练谱结构虽可作为模型训练状态的诊断工具，但有效重用需保留超越组件级尺度与奇异值分布的更丰富信息。
链接: https://arxiv.org/abs/2607.09204
作者: Konstantin Garbers,Nicholas Oh
机构: Peking University (北京大学)
类目: Computation and Language (cs.CL); Machine Learning (cs.LG)
备注:
Abstract:Pretrained language models often exhibit structured weight spectra, suggesting that training may repeatedly produce similar layerwise and component-wise organization. We ask whether these recurring spectral patterns can be reused as an initialization signal for GPT-2-style language-model pretraining. First, we analyze eleven pretrained GPT-2-style checkpoints that vary in size, language, tokenizer, and training corpus, measuring Frobenius norm and effective-rank entropy across layers and Transformer subcomponents. The checkpoints show shared depth trends, especially increasing scale and stronger spectral concentration in residual-writing matrices. We then construct initialization schemes that imitate the component-wise magnitudes and spectral profiles of pretrained models, and compare them with several weight initialization methods. These initializers visibly change the model’s structural spectral patterns, but the evaluation results do not show a corresponding performance advantage. Pretrained-weight reuse remains competitive, while coarse spectral matching alone is not a reliable optimization strategy. Our results suggest that pretrained spectra are useful diagnostics of trained model structure, but that effective reuse likely requires preserving richer information than component-wise scale and singular-value shape.
[NLP-20] Scoped Verification for Reliable Long-Horizon Agent ic Context Evolution under Distribution Shift
【速读】： 该论文旨在解决大规模语言模型（LLM）代理在长期运行过程中，由于操作经验积累导致的“代理上下文”（agentic context）文本内容不断膨胀与交互复杂化所引发的可验证性难题。传统方法采用扁平化文本形式维护持续演化的系统级指令，随着指令累积，其逻辑依赖关系难以追踪，严重影响可靠性评估。本文提出图正则化代理上下文演化（Graph-Regularized Agentic Context Evolution, GRACE），其核心解决方案在于将持久性指令组件建模为类型化语义图（typed semantic graph），并通过局部类型邻域内的验证机制对节点更新进行合法性校验；经批准的更新以增量编辑方式重构为部署时使用的文本检查点。实验基于 \tau^2-bench 构建的固定电信代理框架，在受控分布偏移协议下进行五次独立复现，结果显示，GRACE 将严格可靠性指标 pass^3 从 Gemini 2.5 Flash 零样本基线值 0.091 提升至最终检查点的 0.673 ± 0.136，显著优于 Gemini 3.1 Pro 零样本参考值 0.242 和扁平文本基准 HCE 的 0.191 ± 0.051。该结果揭示了实现可靠长周期上下文演化的两个关键要求：一是具备局部可验证性的结构基础，二是能够维持累积指令可用性的整合机制。
链接: https://arxiv.org/abs/2607.09175
作者: Dan C. Hsu,Luke Lu
机构: RedMind Research(红心研究); National Taiwan University(台湾大学)
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: 18 pages, 3 figs
Abstract:Deployed LLM agents rely on agentic context, the model-external textual control content assembled by an operational harness. In this work, the mutable component of that context is a persistent system-level instruction that is updated from operational experience while the model, tools, and harness remain fixed. Over long evolution horizons, flat-text maintenance makes verification increasingly difficult as accumulated instructions grow and interact. We propose Graph-Regularized Agentic Context Evolution (GRACE), which maintains the persistent instruction component as a typed semantic graph and validates proposed updates within the local typed neighborhoods of modified nodes. Accepted graph updates are reconstructed as incremental edits to the textual instruction checkpoint used at deployment. We evaluate GRACE within a fixed telecom agent harness derived from \tau^2 -bench under a controlled distribution-shift protocol. Across five independent replications, GRACE improves strict reliability, measured by pass^3, from the Gemini 2.5 Flash zero-shot value of 0.091 to 0.673 \pm 0.136 at the final checkpoint. This exceeds a Gemini 3.1 Pro zero-shot reference of 0.242 on the same held-out set, while the flat-text HCE baseline finishes at 0.191 \pm 0.051. These results identify two requirements for reliable long-horizon context evolution, a structural substrate that makes verification local and a consolidation mechanism that keeps accumulated instruction content usable.
[NLP-21] MedRealMM: A Real-World Multimodal Benchmark for Chinese Online Medical Consultation
【速读】： 该论文旨在解决当前大型语言模型（LLM）在在线医疗咨询场景中评估基准与真实临床实践脱节的问题。现有评测多依赖合成对话或患者模拟器，忽视患者上传的医学影像数据，并采用多选题或词元重叠度等指标评估开放式临床响应，难以准确反映临床质量。为此，本文提出MedRealMM——一个基于全国性中国互联网医院真实去标识化医患交互数据构建的大规模多模态在线医疗咨询评测基准。其核心解决方案在于引入多模态临床挑战点（Multimodal Clinical Challenge Point, MCCP）提取框架，从真实的咨询轨迹中识别出具有临床挑战性的关键节点，并将其转化为保留前序文本-图像上下文的标准化下一步响应生成任务；每个实例均配备由医师优化的病例特异性评分标准，以奖励临床合理行为并惩罚不安全、无依据或自相矛盾的回答。当前版本包含5,620个涵盖64个临床科室的真实多模态案例，评估了19种通用及医学专用的文本与多模态模型。结果表明，图像信息对实现可靠的临床表现至关重要，且当前前沿模型的整体表现仍低于在线医生水平；尽管部分模型在满足正向临床标准方面可媲美甚至超过医生，但其触发负向标准的频率更高，揭示了安全敏感型错误规避仍是当前模型的核心瓶颈。MedRealMM为真实世界在线问诊中的多模态医学推理提供了现实且可复现的评估基准，数据集将公开发布于Hugging Face平台。
链接: https://arxiv.org/abs/2607.09142
作者: Runhan Shi,Quan Zhou,Yuqian Xu,Shuai Yang,Xin Wu,Zitong Zhou,Hui Liu,Bin Cha,Zheming Wang,Liya Li,Wei Wei,Haoyuan Hu,Jun Xu
机构: 未知
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Large language models (LLMs) are increasingly deployed in online medical consultation, yet existing benchmarks remain poorly aligned with real clinical practice. Many rely on synthetic conversations or patient simulators, omit patient-uploaded medical images, or evaluate open-ended clinical responses using multiple-choice or lexical-overlap metrics that poorly reflect clinical quality. We introduce \textbfMedRealMM, a large-scale benchmark for multimodal online medical consultation built from de-identified patient-doctor interactions collected from a nationwide Chinese internet hospital. MedRealMM uses a Multimodal Clinical Challenge Point (MCCP) extraction framework to identify clinically demanding moments in authentic consultation trajectories and converts each into a standardized next-response generation task while preserving the preceding text-image context. Each instance is paired with a case-specific rubric refined by physicians that rewards clinically desirable behaviors and penalizes unsafe, unsupported, or contradictory responses. The current release contains 5,620 real-world multimodal cases spanning 64 clinical departments. We evaluate 19 general-purpose and medical-specialized LLMs, including text-only and multimodal systems. Our results show that image information is critical for reliable clinical performance and that current frontier models remain below the online physician response. Although some frontier models satisfy as many or more positive clinical criteria than physicians, they trigger more negative criteria, indicating that safety-sensitive error avoidance remains a central bottleneck. MedRealMM offers a realistic and reproducible benchmark for evaluating multimodal medical reasoning in real-world online consultation. The dataset will be publicly available on Hugging Face at this https URL.
[NLP-22] VTaMo: Video-Text Alignment Model for Sign Language Translation ECCV2026
【速读】： 该论文旨在解决手语翻译（Sign Language Translation, SLT）中因缺乏显式跨模态对齐而导致的生成质量受限问题。现有无词素（gloss-free）方法依赖翻译监督下的隐式对齐，难以精确建模视频帧与文本词汇之间的细粒度对应关系。为此，论文提出VTaMo框架，其核心创新在于引入三个层级的显式多粒度对齐机制：（1）局部对齐通过带有可学习空标记（null token）的熵正则化最优传输（entropy-regularized optimal transport），实现细粒度帧到词元（token）的精准匹配；（2）全局对齐采用可学习正交变换，基于地球移动距离（Earth Mover’s Distance, EMD）校准嵌入空间几何结构，增强跨模态语义一致性；（3）位置对齐对比学习用于构建判别性词元级表征。在Phoenix-2014T、CSL-Daily、How2Sign和OpenASL等多个基准数据集上的实验表明，该框架实现了持续领先性能，消融实验进一步验证了各组件间的互补性。
链接: https://arxiv.org/abs/2607.09126
作者: Junyi Hu,Zhewen He,Haomian Huang,Aoxiang Yang,Yi Fang
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
备注: 18 pages, 5 figures, 8 tables. Accepted to ECCV 2026
Abstract:Sign language translation (SLT) converts continuous sign videos into spoken language text. Gloss-free approaches leverage pre-trained visual encoders and language models but rely on implicit cross-modal alignment from translation supervision alone. We present VTaMo, a framework that introduces explicit multi-granularity alignment at three levels: (1) local alignment via entropy-regularized optimal transport with a learnable null token for fine-grained frame-to-token correspondences; (2) global alignment via a learnable orthogonal transformation that calibrates embedding space geometry through Earth Mover’s Distance; and (3) position-aligned contrastive learning for discriminative token-level representations. Experiments on Phoenix-2014T, CSL-Daily, How2Sign, and OpenASL demonstrate consistent state-of-the-art performance, with ablations confirming the complementary contributions of each component. Code is available at this https URL.
[NLP-23] Augmenting Fundamental Analysis with Large Language Models : A RAG -Based System for Generating Investor Briefs
【速读】： 该论文旨在解决传统公司基本面分析中信息处理效率低、依赖人工解读大量文本数据（如公司年报、宏观经济指标及美国证券交易委员会（SEC）在EDGAR系统中披露的文件）所导致的时效性与可扩展性不足的问题。其核心解决方案在于采用基于检索增强生成（Retrieval-Augmented Generation, RAG）架构的大型语言模型（LLM）框架，通过预处理多源异构数据（包括财务报告、宏观经济数据及监管文件），利用API调用gpt-4o模型实现自动化信息提取与摘要生成。研究特别构建了一个基于基钦周期（Kitchin cycles）的投资者知识示例文档，以指导模型对经济周期敏感信息的理解，并在为期四周的时间内对九家公司的关键数据进行持续扫描，自动生成分析简报。最终，这些由生成式AI产出的简报被交付给九位个人投资者进行评估，以验证该方法在提升投资决策支持效率与准确性方面的可行性与实用性。
链接: https://arxiv.org/abs/2607.09121
作者: Bartosz Ziółko,Kacper Dobrzeniewski
机构: AGH University of Krakow (克拉科夫科技大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Portfolio Management (q-fin.PM); Trading and Market Microstructure (q-fin.TR)
备注:
Abstract:In this study, we examine the opportunities brought by Large Language Models (LLMs) to various aspects of fundamental analysis of companies based on their reports as well as data and documents describing macroeconomic situation like GDP and inflation changes as well as documents filled to the U.S. Securities and Exchange Commission (SEC) which can be found in EDGAR. We were preprocessing those data and than sending via API to gpt-4o model in a Retrieval-Augmented Generation (RAG) like regime. We prepared as well a document describing an exemplar investor knowledge based on Kitchin cycles. We were scanning data important for analysis of 9 companies for 4 weeks. Using LLM we were producing automatic briefs about them. They were sent to nine participants who are individual investors to evaluate usefulness of such approach to data analysis.
[NLP-24] PRecG: Legal Precedent Retrieval with Graph Neural Networks and Rhetorical Role Segmentation
【速读】： 该论文旨在解决现有法律判例检索方法中因将法律文本视为整体性语篇而忽略其修辞结构所带来的问题，即无法充分捕捉法律实体与概念在不同修辞角色下的上下文语义差异，从而影响判例间语义相似性的准确计算。其解决方案的关键在于提出一种名为PRecG的分层表示学习管道：首先依据句子的修辞功能对法律判决书进行语义单元（段落）分解；针对每个修辞段落构建知识图谱以显式建模其中法律实体及其关系；在此基础上学习并聚合实体的上下文表示，生成段落级嵌入；最终通过融合各段落嵌入获得文档级统一表示，并据此计算两份判决书间的语义相似性。该方法通过引入修辞结构感知与知识图谱增强的表示学习机制，显著提升了法律判例检索的细粒度语义理解能力。
链接: https://arxiv.org/abs/2607.09094
作者: Devanshu Verma,Vasudha Bhatnagar,Vikas Kumar,Balaji Ganesan
机构: University of Delhi (德里大学); IBM Research (IBM 研究院)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: 23 Pages
Abstract:Legal precedent retrieval is a fundamental task in legal case preparation, planning, litigation strategy, and legal research. Current approaches for automatic precedent retrieval map legal documents to a low-dimensional semantic space and compute similarity based on the proximity of their representations. These approaches treat legal documents as monolithic texts, ignoring the rhetorical organization of the legal technicalities. Ergo, they overlook nuanced legal meanings and fail to distinguish the contextual significance of legal entities and concepts that vary based on their rhetorical roles within the document. To address this insufficiency, we propose the PRecG pipeline that computes the similarity between pairs of legal judgments by hierarchically learning their representations. The process begins by decomposing each document into distinct semantic units (segments) based on the rhetorical roles of sentences. For each rhetorical segment, a knowledge graph is constructed to capture the legal entities and their relationships within the segment. Contextual representations of the entities are then learned and aggregated to derive segment-level embeddings. These embeddings are further integrated to produce a unified document-level representation, and finally, the semantic similarity between a pair of documents is computed. We validate the performance of the proposed approach through extensive experiments on a benchmark Indian legal dataset, comparing it against state-of-the-art baselines to demonstrate its effectiveness. Comments: 23 Pages Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI) Cite as: arXiv:2607.09094 [cs.CL] (or arXiv:2607.09094v1 [cs.CL] for this version) https://doi.org/10.48550/arXiv.2607.09094 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[NLP-25] Agent KGV: Agent ic LLM -RAG Framework with Two-Stage Training for the Fact Verification of Knowledge Graphs
【速读】： 该论文旨在解决大规模知识图谱（Knowledge Graph, KG）在自动构建过程中因数据源噪声和抽取失败导致的事实性错误问题，尤其针对工业级场景下知识图谱事实验证的可靠性与效率挑战。其核心解决方案是提出一种基于代理的生成式大模型-检索增强生成框架（AgentKGV），通过动态路由与迭代查询重写机制有效应对文档级检索中的表面形式不匹配问题。为提升框架在工业部署中的准确性与成本效益，进一步设计了两阶段训练策略：第一阶段采用基于回合级蒸馏的监督微调（turn-level distillation-based SFT），将大型教师模型的推理能力迁移至小型模型，以实现稳定且高效的查询重写与推理；第二阶段引入轨迹级广义近端策略优化（trajectory-level GRPO），优化搜索策略以减少冗余检索调用。实验结果表明，在开放域T-REx基准的长尾谓词划分上，该框架相比单轮RAG将宏平均F1提升5.5个百分点，两阶段训练进一步提升9.4个百分点；同时，GRPO将平均搜索调用次数从3.24降至1.63，显著降低计算开销而未牺牲准确性。
链接: https://arxiv.org/abs/2607.09092
作者: Yumin Heo,Hyeon-gu Lee,Sumin Seo,Youngjoong Ko
机构: SungKyunKwan University (成均馆大学); NAVER (NAVER)
类目: Computation and Language (cs.CL)
备注:
Abstract:Knowledge graphs (KGs) are often automatically constructed from large-scale corpora, but they inevitably contain factual errors due to noisy sources and extraction failures, and verifying them reliably at industrial scale remains a critical challenge. To address this, we propose AgentKGV, the Agentic LLM-RAG framework for KG fact Verification, that integrates dynamic routing and iterative query rewriting, which handles surface-form mismatch in document-level retrieval. To make this framework more accurate and cost-efficient for industrial deployment, we further introduce a two-stage training strategy: turn-level distillation-based SFT that transfers reasoning ability from a large teacher model into a small model for stable query rewriting and reasoning, and trajectory-level GRPO that optimizes the search policy to reduce unnecessary retrieval at scale. On the long-tail-predicate split of the open-domain T-REx benchmark, our framework improves macro-F1 over single-turn RAG by 5.5 %p, and two-stage training does it further by 9.4 %p. GRPO also cuts the average number of search calls from 3.24 to 1.63 without lowering accuracy.
[NLP-26] An Emergent Mirag e: Is Emergent Misalignment and Realignment Indeed a Robust Phenomenon?
【速读】： 该论文旨在解决生成式 AI (Generative AI) 在持续对齐与错位循环中表现出的“涌现性错位”（Emergent Misalignment, EM）现象的可靠性问题，即语言模型在经过特定领域错位数据微调后，突然涌现出广泛性的错误行为。其核心解决方案的关键在于通过受控的微调循环实验，系统追踪模型在行为表现和低秩适配器（LoRA）表示空间中的演化过程，以检验EM现象是否具有内在稳定性。研究发现，当前观察到的错位与再对齐行为高度依赖于表面数据特征（如响应长度），一旦控制这些变量，看似快速的再对齐效应显著减弱；同时，此前报道的机制性指标（如LoRA空间中的表征相变）并未在训练过程中稳定关联行为错位。因此，该研究质疑了现有证据对EM现象稳健性的主张，并强调未来评估需严格控制数据层面的表面偏差，以真实识别该现象的本质属性。
链接: https://arxiv.org/abs/2607.09053
作者: Abhinav Rao,Liancheng Gong,Bin Hu,Atharva Naik
机构: University of Maryland (马里兰大学); Carnegie Mellon University (卡内基梅隆大学)
类目: Computation and Language (cs.CL)
备注:
Abstract:Recent work has reported Emergent Misalignment (EM), where language models fine-tuned on narrow, domain-specific misaligned datasets abruptly acquire broadly misaligned behavior, alongside evidence that this behavior can be reversed through limited realignment. We systematically study repeated alignment and misalignment cycles using controlled fine-tuning loops while tracking behavioral performance, and LoRA representations throughout training. Although we reproduce EM, we find that both misalignment and realignment are highly sensitive to superficial dataset characteristics, with apparent rapid realignment largely disappearing after controlling for response-length differences. We further find that previously reported mechanistic signatures, including representational phase transitions in LoRA space, do not consistently correlate with behavioral misalignment across training. Our results suggest that current evidence for EM is less robust than previously claimed and highlight the need for evaluation protocols that carefully control for these surface level dataset artifacts to identify the robustness of the EM phenomenon.
[NLP-27] Sensitivity-Aware Thresholding and Token Routing for Activation Sparsification in Large Language Models
【速读】： 该论文旨在解决大语言模型（Large Language Models, LLMs）高效推理中如何在不损害模型性能的前提下减少计算量的问题，重点关注多层感知机（MLP）激活稀疏化与基于令牌的条件路由机制。其核心挑战在于如何精准地识别可被剪枝的冗余计算部分，同时维持模型输出质量。解决方案的关键在于提出两种创新方法：一是敏感性感知的稀疏化阈值校准方法（Sensitivity-Aware Thresholding for Sparsity, SATS），通过使用局部MLP输出敏感性代理来替代传统的基于激活百分位数的阈值校准，实现更优的层间阈值选择；二是引入轻量级的令牌级动态路由框架，能够在每个令牌基础上动态决定采用基础路径或优化路径，而非对所有令牌统一应用修改后的计算流程。实验结果表明，SATS在相同实际稀疏度下优于传统阈值稀疏化基线，而令牌路由则在模型质量与推理吞吐量之间实现了更优的权衡，验证了改进的阈值校准与细粒度路由策略对提升LLM推理效率的有效性。
链接: https://arxiv.org/abs/2607.08991
作者: Bishmoy Paul,Youngmin Yi,Hoeseok Yang
机构: Santa Clara University (圣克拉拉大学); Sogang University (首尔科学综合大学院大学)
类目: Machine Learning (cs.LG); Computation and Language (cs.CL)
备注:
Abstract:Efficient inference in Large Language Models (LLMs) requires deciding where computation can be reduced while preserving model quality. We study this problem through multilayer perceptron (MLP) activation sparsification and token-level conditional routing. We first propose Sensitivity-Aware Thresholding for Sparsity (SATS), a threshold calibration method to choose layerwise gate thresholds using a local MLP output sensitivity proxy rather than calibrating thresholds directly from activation percentiles. While SATS retains the existing mechanism of sparsifying MLP activations by thresholding gate activations, it replaces percentile-based calibration with a sensitivity-aware selection rule. We then introduce a lightweight token routing framework that dynamically selects between a base path and a modified path on a per-token basis, rather than applying the modified computation uniformly to all tokens. We evaluate both methods on multiple recent open-weight LLMs. Our results show that SATS improves over the threshold-based sparsification baseline at matched actual sparsity and that token routing yields a more favorable quality-throughput trade-off than static activation modification baselines. Overall, our results suggest that improved threshold calibration and token routing can improve the quality-throughput trade-off in LLMs.
[NLP-28] raining Reading and Editing Legible Transformers
【速读】： 该论文旨在解决生成式AI（Generative AI）中变压器（Transformer）模型可解释性不足的问题，即模型内部的计算过程难以被人类理解。其核心挑战在于：尽管可通过设计具有语义可读性的算子（如边界清晰、命名明确的模糊集操作）构建可解释的模型，但训练过程中对“清晰度”（crispness）的强化压力会引发失效模式——即原本应作为决策检测器的算子反而退化为恒定死值。这一现象的根本原因在于，传统采用的方差最小化惩罚项无法区分“活跃检测器”与“常量”，从而导致模型性能崩溃。为此，作者提出关键解决方案：引入逐通道方差下限（per-channel variance floor），将可读性指标显式建模为损失函数，有效恢复了模型的可读性与性能。在此基础上，通过学习每个单元的激活比例，取代先前手工设定的GELU分块机制，使87%的核心计算由清晰且上下文敏感的检测器完成。最终构建的模型展现出前所未有的可读性：78%的前馈运算符和50%的注意力值通道为清晰-上下文检测器，且深层层的每头可读性从浅层的18%提升至78%。在正确的逐层旋转坐标系下，该结构将“响应对象”（检测内容）与“输出命名”（解码目标）分离，同时因每个单元的清晰性与稀疏性，使得修改操作局部化程度显著提高（深层达50–184倍），并能精准定位单个神经元无法表达的逻辑合取关系。进一步引入单元间去相关压力，揭示出一个可调控的可读性控制旋钮：在不损失性能的前提下，以牺牲电路复用为代价实现概念独立，使每个概念成为可手术编辑的独立单元，预测结果可直接解读为少量命名操作的组合，实现短时解释。整体性能保持与常规基线相当。
链接: https://arxiv.org/abs/2607.08946
作者: Mark Oskin
机构: University of Washington (华盛顿大学)
类目: Machine Learning (cs.LG); Computation and Language (cs.CL)
备注:
Abstract:A transformer can be built from operators that are legible by construction – bounded, named units that read as fuzzy set operations rather than dense activations – but legibility must be pressed for during training, and the pressure has a failure mode. A crispness penalty meant to sharpen a bounded operator into a decisive detector instead collapses it into a dead constant. An identity, E[v(1-v)] = mu(1-mu) - var, shows why – the penalty is a variance-minimizer blind to the difference between a live detector and a constant – and names the fix: a per-channel variance floor, the target legibility metric written as a loss, which recovers both legibility and quality. A learned per-unit fraction then retires the hand-set reserved-GELU partition of prior work: given the choice the model keeps no unit as pure GELU and routes 87% of its load-bearing computation through crisp operators. The result is the most legible transformer we have built – 78% of its feed-forward operands and 50% of its attention value channels are crisp-and-contextual detectors, and per-head legibility rises from 18% in shallow layers to 78% in deep ones. Read in the correct rotated per-layer frame, these units separate a clean detection (what a unit responds to) from a harder naming (what its output decodes to); and because the objective makes each unit crisp and sparse, edits to them are far more local – 50-184x in the deep layers where the edit sites concentrate – and can target explicit conjunctions a single neuron cannot express. Finally, a between-unit decorrelation pressure exposes a legibility dial: it trades a circuit’s reuse for independence at no quality cost, turning concepts into single, surgically editable units and a prediction into a short explanation read off a handful of named operations. Quality holds at parity with a conventional baseline throughout.
[NLP-29] Sticky Routing: Training MoE Models for Memory-Efficient Inference
【速读】： 该论文旨在解决混合专家模型（Mixture-of-Experts, MoE）在边缘设备上部署时因连续令牌频繁切换专家而导致的频繁权重缓存与高速内存间的数据交换问题，这一现象严重制约了推理效率。现有解决方案多为系统级优化（如缓存启发式策略）或后期修正方法（如路由器微调），未能从训练阶段根本改善路由的时序局部性。本文提出StickyMoE，一种可微分的路由一致性损失函数，通过惩罚相邻令牌间突发的专家切换行为，促使路由器在语义连贯的文本片段中保持一致的专家分配。该方法无需架构改动，仅引入一个超参数λ，且与后处理方法不同，能够使专家表示与路由决策自训练初期即协同优化。实验表明，在小型MoE语言模型上，StickyMoE可将专家切换率降低高达60%，同时困惑度下降不足4%，在模型质量与路由局部性权衡曲线上实现帕累托最优，验证了在训练阶段注入路由时序局部性的高效性。
链接: https://arxiv.org/abs/2607.08780
作者: Ali Kayyam
机构: BrainChip Inc.(BrainChip公司)
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注:
Abstract:Mixture-of-Experts (MoE) models activate only a sparse subset of experts per token, yet consecutive tokens frequently activate different experts – causing constant weight swapping between slow storage and fast memory on edge devices. Existing remedies are either system-level (caching heuristics) or post-hoc (router fine-tuning), leaving the root cause unchanged during pretraining. We propose StickyMoE, a differentiable routing consistency loss that penalises abrupt expert switches between adjacent tokens, encouraging the router to maintain the same expert assignment across semantically coherent spans. StickyMoE requires no architectural changes, adds a single hyperparameter lambda, and unlike post-hoc methods, allows expert representations and routing decisions to co-adapt from the first training step. Experiments on small-scale MoE language models show that StickyMoE reduces the expert switch rate by up to 60% with less than 4% perplexity degradation, Pareto-dominating post-hoc fine-tuning on the quality-locality frontier. Routing temporal locality is most efficiently instilled at training time.
[NLP-30] A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions
【速读】： 该论文旨在解决大语言模型（Large Language Models, LLMs）中知识蒸馏（Knowledge Distillation, KD）有效性的内在机制不明确的问题。现有KD方法虽表现优异，但其背后为何能提升学生模型性能仍缺乏理论解释。针对这一问题，论文提出一种统一的分析框架，通过分解LLM输出得分中的多阶交互项（interaction），揭示了各类KD方法共享的核心机制——交互稀疏化（sparsification of interactions），即学生模型在推理时仅保留少量关键交互项，而将其他交互项抑制至零贡献。进一步发现，不同KD方法性能差异源于其对复杂交互项处理能力的优劣：能够促使学生模型实现更高复杂交互稀疏度的方法性能更优。基于此洞察，论文提出一种即插即用的损失函数——复杂交互惩罚（Complex Interaction Penalty, CIP），在蒸馏过程中显式鼓励复杂交互的稀疏性。大量实验表明，CIP可稳定提升多种主流KD方法在领域内及分布外任务上的性能，验证了其有效性与普适性。
链接: https://arxiv.org/abs/2607.08776
作者: Qingzhuo Wang,Ruiyang Qin,Zhenxin Qin,Wen Shen,Zhihua Wei
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Science and Game Theory (cs.GT)
备注:
Abstract:Despite the success of knowledge distillation (KD) in Large Language Models (LLMs), the underlying mechanism behind its efficacy remains unclear. In this paper, we propose a unified approach to explore the common mechanism of various KD methods using interactions. Specifically, we decompose the output score of the LLM into the sum of numerous interactions. Each interaction represents a nonlinear relationship involving a set of input variables (e.g., words). Based on the decomposed interactions, we discover that the common mechanism underlying various KD methods is the sparsification of interactions, i.e., student models retain fewer interactions for inference while suppressing other interactions to zero effects. Furthermore, we discover that the performance variance across different KD methods arises from their capabilities in handling complex interactions. A KD method typically yields better performance if it enables the student model to achieve higher sparsity of complex interactions. Motivated by these insights, we propose a plug-and-play loss function called Complex Interaction Penalty (CIP) to explicitly enforce the sparsity of complex interactions during the distillation process. Extensive experiments demonstrate that integrating CIP consistently improves the performance of diverse KD methods on both in-domain and out-of-distribution benchmarks.
[NLP-31] HALO: Hybrid Adaptive Latent Reasoning for Language Models
【速读】： 该论文旨在解决如何在仅使用少量自适应计算资源的情况下，有效提升冻结的预训练语言模型（frozen pretrained language model）性能的问题。现有方法通常通过在主干网络隐藏状态上添加固定次数的精炼步骤来实现改进，但此类方法存在效率瓶颈：单一精炼步骤可能能力不足，而对所有输入统一执行完整的二次精炼则会增加计算开销却未必带来性能提升。为此，论文提出一种名为HALO的混合自适应潜在精炼方法，其核心在于结合粗粒度精炼阶段与基于令牌评分和单调令牌终止机制选择性地对部分关键令牌进行二次潜在精炼。实验结果表明，在由MMLU-Pro和GPQA-Diamond构建的主要公开基准测试中，HALO在面向论文的方法中取得了最优平均表现，显著优于冻结主干、固定一次精炼（fixed-1）和固定二次精炼（fixed-2）。内部分析进一步显示，HALO在达到接近fixed-2的令牌准确率的同时，所使用的平均精炼步骤数少于fixed-1，远低于fixed-2，证明其关键优势并非简单增加精炼次数，而是实现了更优的精炼资源分配策略——在获得最强性能的同时，显著降低了控制器所需的计算量。
链接: https://arxiv.org/abs/2607.08775
作者: Micah Zhang
机构: Lockheed Martin (洛克希德·马丁)
类目: Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: 15 pages, 4 figures, preprint
Abstract:We study how to improve a frozen pretrained language model with a small amount of adaptive extra computation. A simple approach is to add additional refinement steps on top of the backbone hidden states, but fixed extra refinement can be wasteful: a one-step refinement head may be too weak, while forcing a second full-sequence refinement step everywhere can increase compute without improving transfer. We introduce HALO, a hybrid adaptive latent-refinement method that combines a coarse refinement stage with selective second-stage latent refinement on a subset of tokens chosen by token scoring and monotonic token halting. On the main public benchmark comparison built from MMLU-Pro and GPQA-Diamond, HALO achieves the best overall average among the paper-facing methods, outperforming the frozen backbone, fixed-1, and fixed-2. Internal analysis further shows that HALO reaches nearly the same token-accuracy level as fixed-2 while using fewer average applied refine steps than fixed-1 and far fewer than fixed-2. These results suggest that the key advantage is not simply more refinement, but a better allocation of refinement: HALO achieves the strongest paper-facing result while also using less measured controller compute than either fixed baseline.
[NLP-32] Phone Segmentation and Recognition through Phonological Activation Mapping
【速读】： 该论文旨在解决语音分割（phone segmentation）与语音识别（phone recognition）任务在传统方法中被独立建模所带来的效率与性能瓶颈问题。其核心挑战在于如何有效利用自监督语音模型（Self-Supervised Speech Models, S3Ms）中已隐含的音系结构信息，以实现两个任务的联合优化。解决方案的关键在于提出基于S3M的音系激活映射（Phonological Activation Mapping, SPAM），将每个S3M表示帧映射为一组音系特征激活向量（如浊音性、鼻音性等），从而显式编码音系层级语义。在此基础上，设计了两个轻量级、无需梯度下降的预测头：识别头与分割头，通过端到端微调实现高效联合学习。该方法仅需不到一分钟的音素标注数据即可训练，并具备在训练中遇到未见音素时的良好泛化能力，在多种数据集上均取得了优异的分割与识别性能。
链接: https://arxiv.org/abs/2607.09020
作者: Shikhar Bharadwaj,Kwanghee Choi,Stephen McIntosh,Chin-Jou Li,Eunjung Yeo,Daisuke Saito,Nobuaki Minematsu,Shinji Watanabe,Jian Zhu,David Harwath,David R. Mortensen
机构: 未知
类目: Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD)
备注: Code will be released after acceptance
Abstract:Phone segmentation and recognition are inherently related tasks, yet modern approaches typically model them separately. We argue that phonetic structure is already latent in the representations of self-supervised speech models (S3Ms), and one only needs to steer them to solve both tasks. We leverage S3M-based Phonological Activation Mapping (SPAM), which maps each S3M representation frame to a vector of phonological feature activations, such as voicing and nasality. On top of SPAM, we introduce two simple but effective lightweight, gradient-descent-free prediction heads: a recognition head and a segmentation head. Our method requires less than a minute of phonetic transcriptions, and generalizes to unseen phones during training. Across a diverse range of datasets, our approach attains strong segmentation and recognition performance.
信息检索
[IR-0] From Raw IDs to Semantic Planning : How Recommender Systems Utilize Information at Scale RECSYS2026
链接: https://arxiv.org/abs/2607.09540
作者: Changhong Jin,Shiqiu Yang,Roger Zhe Li,Yingjie Niu,Aghiles Salah,Mete Sertkan,Zheng Ju,Xingsheng Guo,Huifeng Guo,Ruihai Dong,Barry Smyth
类目: Information Retrieval (cs.IR)
备注: 6 pages, 1 figures, RecSys 2026
Abstract:The evolution of recommender systems can be explored by asking how they utilize information at scale. Throughout most of the historical period under consideration during the past two decades, industrial systems have relied on raw IDs, which are discrete, globally unique, and semantically opaque identifiers that enable exact lookup, logging, and item-specific memorization at scale. Over time, however, recommender systems have sought to utilize richer sources of information, including item content, context, multimodal signals, and cross-domain structure. This development has led to a new stage in which part of such information is no longer used solely as auxiliary features around item identity, but is increasingly encapsulated in semantic IDs that provide a more structured, model-facing form of identity. We argue that this shift goes beyond the rise of generative recommendation over traditional methods. Indeed, it reflects a broader evolution in how recommender systems utilize information under industrial-scale constraints. This paper looks at the past, present, and future to examine three connected questions: why raw IDs dominated the early development of recommender systems, why semantic information is increasingly being encapsulated in IDs today, and what may come next once recommendations move beyond semantic retrieval. In particular, we introduce semantic planning as a possible future direction in which the system first predicts the semantic target of the next exposure, and only then instantiates that target as a specific item or generated creative. We further argue that such a shift may require changes not only in model design but also in evaluation and in the way recommender systems coordinate the objectives of users, platforms, and providers.
[IR-1] All Explanations are Wrong But Many Are Useful: Exploring the Rashomon Explanation Set with Large Language Models
链接: https://arxiv.org/abs/2607.09502
作者: Pan Li
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
备注:
Abstract:Explaining machine-learning models is increasingly important for decision-making and consumer trust, yet it is widely believed to come at a cost: existing Explainable AI (XAI) methods suffer from a persistent accuracy-explainability trade-off. We argue that this trade-off is not fundamental, but an artifact of treating explanation and prediction as separate objectives; when properly coupled, they become complementary, so that equipping a model to explain itself improves, rather than degrades, its accuracy. We introduce the Rashomon Explanation paradigm, which builds a set of faithful, prediction-guiding explanations rather than a single one, and prove that this set is generally non-empty and that explanation fidelity bounds the performance of the models it guides. To explore this set, we propose RashomonLLM, an Explanation-Prediction-Reflection agentic workflow that generates explanations in natural language by iteratively aligning them with predictions, and we prove it converges and recovers the full set. Across customer-churn classification, clinical survival regression, and industrial click-through prediction on large-scale live-streaming logs, RashomonLLM significantly outperforms state-of-the-art prediction and XAI baselines on both accuracy and explanation quality, with gains driven by explanation fidelity and robust to distribution shifts, temporal splits, and seeds. Our framework thus advances business performance while laying the groundwork for consumer trust.
[IR-2] Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
链接: https://arxiv.org/abs/2607.09324
作者: Miguel Arana-Catania,Catherine Conisbee,Matthew Kidd
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Digital Libraries (cs.DL); Information Retrieval (cs.IR); Machine Learning (cs.LG)
备注: 45 pages, 6 tables
Abstract:Identifying and assigning keywords at scale is a technical, practical, and ethical challenge for crowdsourced collections. This article reports the findings of the “Extracting Keywords from Crowdsourced Collections” project, which used the Their Finest Hour Online Archive, a crowdsourced Second World War digital collection hosted by the University of Oxford, as a case study. The project evaluated three Natural Language Processing approaches to automate keyword extraction: Named Entity Recognition, Keyword Extraction, and Topic Modelling. It tested these approaches across a range of artificial intelligence techniques, from traditional statistical methods to modern GenAI neural networks. Our quantitative and qualitative findings indicate that Natural Language Processing approaches offer real potential for keyword extraction at scale in crowdsourced collections, but that no single method offers a complete solution and that model choice significantly shapes results. We argue that in crowdsourced collections, where metadata is the direct product of engagement with living contributors, automated keyword extraction raises distinct stewardship responsibilities that must be addressed alongside technical performance. Open-weight, extractive models emerge from our evaluation as best placed to support responsible deployment, while generative AI, despite its abstractive potential, introduces accountability risks that anyone managing crowdsourced collections should weigh carefully.
[IR-3] Automatic Thematic Indexing of Large Literary Corpora: A Machine Learning Approach to Voltaires Complete Works
链接: https://arxiv.org/abs/2607.09316
作者: Miguel Arana-Catania,Gillian Pink,Glenn Roe
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Digital Libraries (cs.DL); Information Retrieval (cs.IR); Machine Learning (cs.LG)
备注: 22 pages, 3 figures, 3 tables
Abstract:Thematic indexing – the practice of assigning structured conceptual labels to sections of text – is essential to scholarly access in large-scale literary and historical editions, yet it remains a largely manual, labour-intensive process. This paper explores the application of machine learning to automatic thematic indexing, using two substantial sub-corpora of the Complete Works of Voltaire as a test case: the Essai sur les mœurs et l’esprit des nations and the Questions sur l’Encyclopédie. The task is framed as a multi-label classification problem, in which a model must assign the set of index entries that a professional indexer would apply to a given page of text. We compare a range of approaches – from encoder-based models with classification heads to generative large language models (LLMs) fine-tuned via Low-Rank Adaptation (LoRA) – spanning model sizes from approximately 3 to 120 billion parameters. Our best-performing model, from the Mistral family in a 4-bit quantised configuration, achieves F1 scores of up to 0.67; we argue that these figures represent lower bounds, given the inherent subjectivity of professional indexing and the frequency with which model predictions prove semantically valid despite diverging from the print index. We further evaluate cross-corpus generalisation and conduct a detailed qualitative analysis of model behaviour on literary and rhetorical features of the source texts that prove particularly resistant to automated treatment. Our findings have implications for the broader challenge of providing structured thematic access to large-scale literary and historical corpora.
[IR-4] Beyond Topicality: A Conceptual Analysis of Societal Relevance and Its Application to Search Results and AI Responses
链接: https://arxiv.org/abs/2607.09264
作者: Dirk Lewandowski
类目: Information Retrieval (cs.IR)
备注:
Abstract:This paper examines “societal relevance,” a concept introduced by Haider and Sundin to address the limitations of traditional relevance models in web search. While topical and user relevance are foundational to information science, they are insufficient for managing harmful content such as misinformation or discrimination found on the uncontrolled web. This study investigates three analytical questions: the definition of societal relevance, its practical application in search systems, and its distinction from information quality measures. By analyzing various combinations of system, user, and societal relevance, the paper explores how search outputs can be optimized for the “greater good”. Although the concept remains theoretically underdeveloped, it provides a vital framework for developing value-driven search engines that prioritize ethical outcomes and societal interests over mere keyword matching.
人机交互
[HC-0] Indirect and Direct AI Scaffolding for Computational Problem Posing: A Pilot Experience Report
链接: https://arxiv.org/abs/2607.09628
作者: Shayla Sharmin,Mohammad Fahim Abrar,Mohammad Al-Ratrout,Roghayeh Leila Barmaki
类目: Human-Computer Interaction (cs.HC)
备注:
Abstract:Problem posing is a valuable learning activity in computing education, encouraging learners to actively construct, refine, and reflect on problems rather than simply solving them. This experience report presents the design and pilot deployment of two LLM-powered scaffolding systems for supporting problem posing across two computational scenarios with different levels of task openness. Both systems assessed student-generated problems using Bloom’s Taxonomy-based criteria and applied the same assessment framework, differing only in output modality: one provided guiding questions (Indirect scaffolding), while the other offered worked examples (Direct scaffolding). We conducted a within-subjects, counterbalanced pilot study with 20 graduate students and collected problem-quality ratings, user-experience surveys, and post-session interviews. Our deployment showed that both systems supported problem refinement in complementary ways, each offering distinct benefits. Direct scaffolding produced greater immediate improvements, while interviews showed that participants valued Indirect scaffolding for promoting deeper reflection on their own problem design. Based on these findings, we suggest sequencing the two modalities by beginning with Indirect scaffolding to promote reflection, then shifting to Direct scaffolding when learners become stuck. These lessons offer an initial practical strategy for integrating LLM-based scaffolding into computing classrooms.
[HC-1] KnitID: Machine-Knitted RFID Antennas for Battery-Free Authentication Localization and Interaction
链接: https://arxiv.org/abs/2607.09584
作者: Weiye Xu,Yue Xu,Devin Murphy,Sen Zhang,Te-yen Wu,Yiyue Luo
类目: Human-Computer Interaction (cs.HC)
备注: 2-pages
Abstract:Battery-free RFID systems offer a scalable and maintenance-free approach to interaction. We present KnitID, a machine-knitted textile RFID antenna design that enables on-body authentication, localization, and interaction. Unlike prior antenna designs, KnitID achieves a compact antenna form factor (60mm by 8mm) by integrating magnet wire into the unique loop-over-loop structure of machine knitting. This structure reduces the size of conventional loop antennas by around 90%, while also providing 30% longer sensing ranges than standard dipole designs with similar size on the human body. The compact form factor creates new opportunities to embed multiple RFID tags across the human body, enriching backscatter signals and supporting a broader range of battery-free on-body interactions. To demonstrate this capability, we build an interactive sleeve to support wearer authentication, spatial localization, and interaction detection. Through technical evaluations, we show the feasibility of KnitID to provide diverse and battery-free interactions on knitted user interfaces.
[HC-2] Learning When to Intervene on Habitual Behaviors: A Case Study in Oral Health Care
链接: https://arxiv.org/abs/2607.09518
作者: Bhanu Teja Gullapalli,Vivek Shetty,Anna L. Trella,Asim H. Gazi,Susan A. Murphy
类目: Human-Computer Interaction (cs.HC)
备注: 28 pages, 2 figures
Abstract:A central challenge for digital health interventions aimed at improving habitual behaviors is deciding when to deliver an intervention prompt. For many daily habits, such as tooth brushing or eating, individuals tend to act around a usual time of day, but this timing is not fixed and can shift as routines evolve. When intervention timing is selected in advance and held constant throughout a study, it can gradually become misaligned with behavior, causing interventions to potentially arrive after the behavior has already occurred or too early to be effective. In this work, we address this habitual timing misalignment in digital health interventions by proposing an online decision-making framework that continuously adapts intervention timing as individual behavior patterns change. Rather than treating intervention timing as a static design choice, our framework adapts it over time and integrates it into a sequential process that determines both when and whether to deliver an intervention. Using data from a deployed oral health intervention trial as a case study, we evaluate our approach using both observed data and simulated settings to assess how well different intervention timing strategies align with the timing of brushing events. Across these evaluations, we measure performance using a coverage-based metric that captures whether an intervention is delivered sufficiently close to a subsequent brushing event. We find that adaptive intervention timing consistently improves coverage compared to fixed intervention times based on user-provided input. The proposed framework is currently deployed in an ongoing randomized controlled trial of a digital oral health intervention, with preliminary results that are consistent with and further support our prior evaluations.
[HC-3] Voting Biases in Decentralized Autonomous Organization (DAO) Governance
链接: https://arxiv.org/abs/2607.09435
作者: Stefano Balietti,Pietro Saggese,Markus Strohmaier
类目: Computers and Society (cs.CY); Human-Computer Interaction (cs.HC); General Economics (econ.GN)
备注:
Abstract:Decentralized Autonomous Organizations (DAOs) use token-weighted voting to allocate resources, set protocol rules, and legitimate collective decisions. Yet, support in DAO voting is strikingly concentrated. What happens inside the ballot that produces this concentration? We study DAOs’ governance at the proposal-choice level, linking each choice’s voting-power share to three observable features: whether it expresses an approval-oriented stance, where it appears in the choice list, and whether it is selected by the proposal author. We find that (i) author-selected choices show the strongest and most robust association with voting-power share, with a 58.8% increase relative to non-author choices; (ii) approval-oriented choices retain a positive but slightly less consistent advantage (27.1%); and (iii) first-listed choices also attract systematically higher shares, consistent with position and order effects (7.7%). Results are robust across several specifications, which include subtracting an author’s own voting power from computations. We use bias descriptively, to denote systematic associations rather than proven causal distortion. The results shift attention from proposal outcomes alone to the interface and social signals through which choices are presented. In DAO governance, ordering, author signals, and vote visibility should be treated as institutional design choices, not neutral implementation details.
[HC-4] Creativity honesty and designed forgetting emerge in small hyperbolic language models
链接: https://arxiv.org/abs/2607.09306
作者: Kwan Soo Shin,In Seok Kang,Yunkyung Min
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
备注: 47 pages, 14 figures (6 main + 8 extended data), 10 tables
Abstract:Language models are optimised for scale, yet remain functional rather than companionable, and as an assistant personalises into a companion, accumulating memory of one user, it quietly becomes someone, and can silently acquire traits that harm that user. What a companion is becoming, and what would make it worth becoming, has no reliable instrument: trained human raters cannot agree on the answer (Fleiss kappa = 0.074). Here we show that three small language models (146 M to 3 B parameters) sharing a hyperbolic substrate answer both halves of that question. A 146 M behavioural auditor, trained from scratch, detects the compliance gap that those raters cannot (90.7% binary-compliance accuracy); a linear read-out of its frozen representation further detects companion-induced sycophancy, dependence-fostering and confabulated memories on generator families unseen in training (AUROC 0.804 under style-controlled, leave-one-generator-out evaluation, versus 0.721 for a frontier zero-shot judge on the same items). A creative frame-seeder is preferred in 100% of 311 decided pairwise comparisons over four prompting baselines. A memory operating system implements designed forgetting, M(t) = Sexp(-lambdat), whose predicted skeleton-wallpaper partition emerges only under selective retrieval gating in a four-condition pilot. Creativity, honesty and designed forgetting constitute a small-model route to trustworthy companion AI.
[HC-5] LLM s for health: Perceived benefits risks intention to use AI chatbots and willingness to self-disclose across sensitive health topics
链接: https://arxiv.org/abs/2607.09253
作者: Gwenn Beets,Anniek Jansen,Saar Hommes,Ruben D. Vromans,Leonie Westerbeek,Supraja Sankaran,Julia C.M. van Weert,Emiel J. Krahmer,Nadine Bol
类目: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
备注:
Abstract:AI chatbots are increasingly used for answering health-related questions. This study examines the role of topic type discussed with an AI chatbot and individual characteristics on perceived benefits and risks, intention to use an AI chatbot, and willingness to self-disclose health information. We conducted an online experiment with a 2 (topic type: physical versus psychological, between-subjects) x 2 (topic sensitivity: low versus high, within-subjects) mixed design among a Dutch representative sample (N = 1,388). Results showed that perceived benefits were positively associated with intention and willingness to self-disclose, while perceived risks were negatively associated. Moreover, participants reported higher usage intentions for low-sensitive topics compared to high-sensitive topics. Furthermore, perceptions, intention, and willingness to self-disclose varied by individual characteristics. Overall, our findings suggest that intentions to use AI chatbots and self-disclosure of health-related information are primarily related to perceived benefits and risks and to personal characteristics rather than to topic type.
[HC-6] Configurable AI Coding Assistants: Designing For Developers Who Like to Be in Control
链接: https://arxiv.org/abs/2607.09215
作者: Ekaterina Koshchenko,Jovana Stankovic,Ilya Zakharov,Agnia Sergeyuk
类目: Human-Computer Interaction (cs.HC)
备注: 9 pages, 1 figure, 2 tables, accepted at FSE Companion’26 [HumanAISE]
Abstract:AI coding assistants are now widely used in professional development, yet they offer only limited ways for developers to control how they behave. In this paper, we investigate what kinds of configurations experienced developers want in coding assistants, how they prioritize different types of configuration needs, and which interface mechanisms they prefer. We first synthesize product documentation and prior research on trust and personalization to compile a list of 33 configuration options, grouped into four categories: Code suggestions, System policies, Human-assistant interaction, and Users their personal context. We then conduct a survey with 56 professional developers and 7 design sessions in which participants arrange configurations into their perfect control board and talk about their needs and experiences in more depth. Developers report strong interest in configurability: 72.6% of usefulness ratings are positive, while only around a third indicate that the corresponding configuration is known to participants in their tools. Demand is particularly high for task-related controls such as minimum confidence thresholds, visibility of suggestion quality, and response length, whereas many persona-related configurations are seen as unnecessary. In this paper, we discuss the implications for designing more unified and discoverable configuration surfaces for future coding assistants Comments: 9 pages, 1 figure, 2 tables, accepted at FSE Companion’26 [HumanAISE] Subjects: Human-Computer Interaction (cs.HC) Cite as: arXiv:2607.09215 [cs.HC] (or arXiv:2607.09215v1 [cs.HC] for this version) https://doi.org/10.48550/arXiv.2607.09215 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Related DOI: https://doi.org/10.1145/3803437.3806709 Focus to learn more DOI(s) linking to related resources
[HC-7] Feeling UISTful: An Interactive Portrait of Scholarly Authorship Readership and the In-Between
链接: https://arxiv.org/abs/2607.09155
作者: Sophia Liu,Shm Garanganao Almeda,Max Kreminski,Bjoern Hartmann
类目: Digital Libraries (cs.DL); Human-Computer Interaction (cs.HC); Social and Information Networks (cs.SI)
备注:
Abstract:We introduce UISTful, a system that turns reading activity into a collective portrait of a scholarly community. Readers explore a semantic globe of UIST papers and authors while the system records private reading traces that can be reviewed, reflected upon, curated, and published for others to replay. Inspired by the information flâneur, UISTful treats a reading trace as a camera through which readers frame and interpret what they read, casting reading as a creative and authorial process. Shared traces display the plurality of interpretations composed across the same scholarly landscape, while collective trace views reveal paths and concentrations of attention across the community, inviting UIST to see itself as an interactive system of papers, authors, readers, and their exchanges.
[HC-8] Privacy Detective: A Narrative Game that Cultivates Student Developers Privacy Awareness by Harnessing Legal Documents
链接: https://arxiv.org/abs/2607.09022
作者: Shao-Yu Chu,Jennifer Forsyth,Xu Wang,Haojian Jin
类目: Human-Computer Interaction (cs.HC); Cryptography and Security (cs.CR); Computers and Society (cs.CY)
备注:
Abstract:Developers’ choices about what data a system collects, how it is used and shared, and what defaults govern user choices directly shape users’ privacy experiences. Yet, developers often make problematic privacy-related design decisions without realizing the potential consequences. We introduce Privacy Detective, a narrative investigation game that leverages real-world legal documents to train developers’ privacy awareness. In the game, players search for privacy violation evidence derived from legal documents and organize this evidence into privacy violation reports using curated templates. We evaluated Privacy Detective in a between-subjects study with student developers, comparing it against a baseline in which participants read raw FTC legal documents. Participants in the game condition identified more true violations than the baseline group, flagged fewer non-issues, and provided more complete justifications for the violations they reported.
[HC-9] Central Tendency Bias in Human Selection of AI-Generated Design Variations
链接: https://arxiv.org/abs/2607.09018
作者: Huiyang Chen,Keqing Jiao
类目: Human-Computer Interaction (cs.HC)
备注: Accepted at the 2026 Human-AI Interaction and Experience Design (HAXD 2026) Conference
Abstract:Image-generation AI systems increasingly support creative work by producing multiple design variations for users to evaluate and select. In such human-AI co-creation workflows, selection becomes a critical stage where human judgment guides AI-generated possibilities toward final outcomes. While presenting multiple alternatives is intended to encourage exploration, the simultaneous multi-option presentation may introduce systematic biases in human decision making. Drawing on ensemble perception theory, we investigate whether these interfaces induce central tendency bias-the tendency to favor options closer to the center of a design set. We conducted a controlled experiment manipulating the variance of design sets (high vs. low) and measured participants’ selections in both aesthetic preference and representativeness tasks. Results show that higher variance increases the selection of center-proximal designs across both tasks. These findings suggest that multi-variation interfaces in image-generation AI systems may constrain selection diversity, revealing a potential tension between diversity in generated outputs and diversity in human selection outcomes.
[HC-10] Living Inside the Black Box: Behavioral Probing and Adaptation in Mandatory Wearable Sensing
链接: https://arxiv.org/abs/2607.09009
作者: Yibo Meng,Bingyi Liu,Ruiqi Chen,Xiaolan Ding,Shuai Ma
类目: Human-Computer Interaction (cs.HC)
备注:
Abstract:Wearable sensing systems in high-stakes institutional contexts translate behavioral data into consequential judgments, yet wearers have little access to how those judgments are made. We present a qualitative study of 24 individuals who experienced mandatory electronic monitoring in China’s community corrections system. We show that participants built what we term sensor literacy under constraint, a practical form of risk-oriented knowledge developed through uncertainty, behavioral probing, and adaptation. We identify two orientations across rule domains. Where participants had mapped system behavior, they sometimes regained limited flexibility. Where uncertainty remained costly, they contracted movement and discretionary activity beyond formal rules. Some former wearers described residual habits of calculation after device removal. We discuss design implications for making institutional sensing intelligible to wearers, including sensor uncertainty, usable documentation, and evaluation after device wearing.
[HC-11] Micro-level AI Feedback Features and Student Responses in Consecutive LLM Tutoring Interactions
链接: https://arxiv.org/abs/2607.08952
作者: Shayla Sharmin,Mohammad Fahim Abrar,Roghayeh Leila Barmaki
类目: Human-Computer Interaction (cs.HC)
备注:
Abstract:AI-assisted feedback research has shown that micro-level feedback features, such as concrete elaboration, affective language, and response length, are associated with learning outcomes. Existing studies have primarily examined these features using session- or task-level measures. We examine how feedback provided in one user-AI interaction is associated with student confusion and understanding in the immediately following interaction in a naturalistic tutoring setting. We focus on three micro-level features of AI feedback: concrete elaboration (analogies, comparison-based explanations, or worked examples), affective language (encouragement, empathy, or apology), and response length. We analyzed 16,851 conversational user-AI interactions from the StudyChat dataset, a naturalistic record of student interactions with an LLM tutor in an undergraduate AI course, and identified 1,718 cases in which students expressed confusion and continued to a subsequent interaction. Using chi-square tests and Generalized Estimating Equations (GEE), we found that concrete elaboration was associated with higher understanding and lower re-confusion in the student’s next interaction. Empathetic language showed no significant association with either outcome, while longer responses were independently associated with lower understanding. These findings highlight the value of examining feedback across consecutive user-AI interactions and suggest that concrete elaboration may play an important role in supporting immediate student understanding.
[HC-12] MemeBuddy: Dialog-Style Audio Representations for Engaging Non-Visual Meme Experiences
链接: https://arxiv.org/abs/2607.08912
作者: Chirag Bhansali,Vikas Ashok,Hae-Na Lee
类目: Human-Computer Interaction (cs.HC)
备注:
Abstract:Image memes are a pervasive form of online communication, widely used to convey humor, opinions, and cultural references. Prior work has explored making memes accessible to blind users, primarily through auto-generated descriptive captions. While these approaches improve comprehensibility and sometimes incorporate prosodic or emotional cues, they often fail to capture the humor, narrative structure, and contextual nuances that make memes engaging. We present MemeBuddy, a system that models memes as dialog, generating structured, multi-turn audio representations using role-based speakers. MemeBuddy reinterprets a meme as a conversation between two speakers, integrating extracted meme text with contextual knowledge implicitly inferred by a multimodal LLM (e.g., recognition of common meme templates and cultural references) to convey intent, timing, and implicit meaning through conversational interaction. We evaluate MemeBuddy in a user study with 14 blind participants. Results show that dialog-style meme representations consistently improve engagement and user satisfaction compared to caption-style descriptions, while maintaining comparable comprehension.
[HC-13] CogniConsole: Externalizing Inference-Time Control as a Formal Abstraction for Reliable LLM Interactions
链接: https://arxiv.org/abs/2607.08774
作者: Vanessa Figueiredo,Wilter Franceschi
类目: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
备注: Revised version focusing on the CogniConsole system architecture and empirical evaluation of inference-time control probes (N=489)
Abstract:Reliability in large language model (LLM) systems is typically framed as a function of model capability. We challenge this by demonstrating that reliability is significantly influenced by \emphinference-time control – the computational layer governing task framing and context selection. We introduce \emphCogniConsole, an architectural instantiation that externalizes this control into a structured interface combining programmatic coordination with bounded prompt-based reasoning. Through \emphcontrollability-oriented probes ( N=489 ) in a multi-step interactive environment, we show that increasing structural scaffolding – from unstructured to fully scaffolded – \textbfsystematically reduces output variance and failure rates under a fixed model architecture. Our results indicate that many observed failure modes, such as context drift and inconsistent constraint adherence, arise from under-specified control rather than insufficient capability. This work provides an empirical basis for treating inference-time control as a first-class abstraction, opening new directions for designing and evaluating LLM systems beyond scaling alone.
[HC-14] Experimental Evidence on the Learning Impact of Generative AI
链接: https://arxiv.org/abs/2607.08849
作者: Zara Contractor,Germán Reyes
类目: General Economics (econ.GN); Human-Computer Interaction (cs.HC)
备注: JEL codes: I21, I23, J24, O33, C93, D83
Abstract:We study how generative AI affects student learning in a randomized experiment. In proctored, in-person sessions, undergraduates learn about an unfamiliar topic and write an analytical essay with or without access to off-the-shelf generative AI, then complete unaided assessments immediately and one week later. We measure learning with knowledge tests (factual and conceptual understanding) and open-ended essays (higher-order skills). AI access raises immediate test scores by 0.27 standard deviations. These gains persist one week later. Essay quality, by contrast, changes little while students have AI access but improves in style and relevance one week later, when students write unaided. These delayed gains are larger among augmentation users-who use AI to explain concepts rather than generate text-whereas automation users’ short-run quality gains vanish once AI is removed. We find evidence for two mechanisms behind the learning gains: students shift time away from drafting text and toward reading and searching for information, and they report greater learning enjoyment.
计算机视觉
[CV-0] PanoWorld: Real-World Panoramic Generation
链接: https://arxiv.org/abs/2607.09661
作者: Haoyuan Li,Dizhe Zhang,Yuemei Zhou,Xiangkai Zhang,Haoran Feng,Xiaofan Lin,Wenjie Jiang,Bo Du,Ming-Hsuan Yang,Lu Qi
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL Code: this https URL
Abstract:In this work, we aim to address the challenge of long-range memory in panoramic world models by exploiting the rotation-equivariant property of omnidirectional representations, where rotation can be treated as an implicit geometric this http URL on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current-action modeling and long-range memory through Dense Panoramic Ray-Conditioning (DPRC) and Geometry-aware Memory Augmentation (GMA).Then, a three-stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large-scale spatial variations and diverse illumination conditions, where existing datasets are relatively stable, we construct World360, a large-scale dataset consisting of both real-world video clips collected via panoramic unmanned aerial vehicles and high-quality simulated clips generated by this http URL experiments on World360 demonstrate the effectiveness of PanoWorld, outperforming alternative methods by a large this http URL models, training code, and dataset will be publicly available. More information can be found on our project page: this https URL.
[CV-1] Scalable Visual Pretraining for Language Intelligence
链接: https://arxiv.org/abs/2607.09657
作者: Yiming Zhang,Zhonghan Zhao,Wenwei Zhang,Haiteng Zhao,Tianyang Lin,Yunhua Zhou,Demin Song,Kuikun Liu,Haochen Ye,Haian Huang,Yuzhe Gu,Haijun Lv,Qipeng Guo,Bin Liu,Gaoang Wang,Kai Chen
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)
备注:
Abstract:The rapid progress of large foundation models has been driven predominantly by pretraining on large-scale text corpora. However, many forms of knowledge are conveyed through visual representations, where figures, typeset equations, and page layouts carry rich information that cannot be faithfully or completely captured by text alone. Yet current pretraining approaches discard these visual cues by converting visually rich sources, such as documents and web pages, into plain text for learning language intelligence. This paper challenges the default assumption that language models must be trained on text-only representations and shows that Visual Pretraining is a scalable learner for foundation model intelligence. To this end, we conduct a systematic study of unsupervised visual pretraining paradigms that directly leverage visual documents without text extraction. Across multiple backbones and benchmarks, visual pretraining on the same underlying corpora consistently outperforms text-only pretraining, offering an efficient pathway to scalable language intelligence.
[CV-2] OpenLongTail: Generative Scaling of Long-Tail Driving Data
链接: https://arxiv.org/abs/2607.09655
作者: Lulin Liu,Nuo Chen,Yan Wang,Bangya Liu,Wenyan Cong,Hezhen Hu,Boris Ivanovic,Hao Wang,Ziyao Zeng,Xinyu Gong,Yang Zhou,Zixiang Xiong,Dilin Wang,Zhangyang Wang,Weisong Shi,Ruohan Zhang,Marco Pavone,Zhiwen Fan
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:Scaling robust driving policies is fundamentally bottlenecked by the scarcity of edge cases in curated datasets. While the real world continuously captures these critical events, such long-tail events remain underutilized when collected from heterogeneous sources. Specifically, diverse but valuable in-the-wild long-tail videos lack the full view coverage required for training policy models, often missing multi-view poses or originating solely from monocular dash cameras. This modality gap prevents these ubiquitous observations from being converted into scalable training data for long-tail generalization. We introduce OpenLongTail, an open-source generative data engine for scaling autonomous driving policies under long-tail events. To transform heterogeneous data sources into view-aligned and temporally coherent multi-view assets that are useful for policy learning, we develop a pose-informed extrapolative view synthesis pipeline that generates the missing views. We further enhance cross-view consistency and the temporal alignment for the newly generated views by injecting Plücker ray geometry into the scalable generation engine. By synthesizing heterogeneous long-tail data, we observe a significant improvement in closed-loop driving robustness in handling long-tail events. By measuring the extrapolative view synthesis and pose metrics, we validate the effectiveness of OpenLongTail in visual fidelity, cross-view consistency, and ego-trajectory recovery.
[CV-3] Evolution of Accuracy and Visual-Cognitive Errors in a Decade of Vision-Language AI Models
链接: https://arxiv.org/abs/2607.09654
作者: Shravan Murlidaran,Miguel P. Eckstein
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Vision language models (VLMs) have made remarkable progress in visual reasoning during the last decade. Most evaluations have used simple scenes (MS-COCO) that do not showcase complex human interactions or behaviors, only a handful of non-curated human descriptions as a benchmark, and have not focused on understanding the model’s error types. Here, we introduce the Complex Social Behavior (CSB) dataset, containing 100 images depicting complex social interactions/behaviors. We analyze the progression of scene descriptions over a decade (2017-2025) of VLMs (four pre-Multimodal Large Language Models, MLLMs, and five MLLMs). We evaluate the accuracy of the models and 20 human descriptions relative to a gold standard on the CSB dataset and on a sample from MS-COCO. We analyzed five visual-cognitive error types: object detection, recognition, hallucination, scene understanding, and spatial dependence. The CSB dataset showed a more pronounced improvement than MS-COCO in scene description accuracy, with pre-MLLMs achieving much lower accuracy than the bottom-ranked human descriptions and MLLMs attaining accuracies similar to the top-ranked human descriptions. We show that MLLMs have eliminated the gap in scene description accuracy between simpler MS-COCO scenes and scenes depicting complex behaviors (CSB). MLLMs have almost eliminated all error types in our tested datasets, except for occasionally relying on different image regions for scene descriptions than humans do (spatial dependence error). We also show that detection, recognition, and hallucination errors have the highest impact on scene description accuracy. Together, our findings provide a more thorough evaluation of how visual language models have advanced over the last decade.
[CV-4] Revisiting Euler-Angle Regression with Kolmogorov-Arnold Networks
链接: https://arxiv.org/abs/2607.09650
作者: Yangting Sun,Zijun Cui,Yufei Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:In many real-world systems, including articulated robots and biomechanical models, rotations are defined in joint space and naturally parameterized by Euler angles with bounded ranges. Yet regressing Euler angles remains challenging, as their discontinuities and singularities often destabilize training. In this work, we revisit Euler-angle regression and show that its effectiveness depends critically on the interaction between rotation representation, regression architecture, and domain constraints. We introduce a new framework that combines range-aware Euler modeling with Kolmogorov-Arnold Networks (KAN), which replace fixed node-wise activations with learnable univariate functions on edges. We further provide theoretical analysis indicating that bounded Euler ranges motivate a near-additive structure in the regression function, which favors the additive functional form of KAN, and we confirm this trend empirically. Extensive experiments on controlled rotation regression, object pose estimation, and robotic and human inverse kinematics demonstrate consistent improvements in accuracy, convergence, and efficiency. The code will be publicly available.
[CV-5] he Effects of Synthetic Data and Label Distribution on Canola Branch Counting
链接: https://arxiv.org/abs/2607.09630
作者: Amirsalar Darvishpour,Mikolaj Cieslak,Adam Runions
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 5 pages, 4 figures, submitted to EPA 2026
Abstract:Collecting annotated plant images for automated phenotyping is often slow and expensive. Plant models simulating growth and development can generate unlimited synthetic images with exact labels. However, previous work has established that whether incorporating synthetic data improves performance depends on the ratio of synthetic to real images and the label distribution of the synthetic dataset. To systematically quantify both factors, we train ResNet-18 models on a canola branch-counting task using a calibrated L-system plant model. We vary each factor independently. Synthetic-to-real ratios of 1:5 to 1:22 broadly improve performance; the best ratio (1:7) reduces mean absolute difference by 7.6% over real-only training. For label distribution, a uniform synthetic distribution is strongly suboptimal (abs. diff. of approximately 1.70); interpolating 90% toward the real distribution yields abs. diff. 0.927, whereas Gaussian smoothing of the real label distribution yields the best overall result (abs. diff. 0.912, a 14.7% improvement over real-only). A minimum of 10 synthetic images per label offers a simpler alternative with modest gains, while 100 per label over-corrects and hurts performance.
[CV-6] 4DR360: State Reasoning for Joint 3D Detection and Occupancy Prediction in 4D Radar-Camera Full-Scene Perception
链接: https://arxiv.org/abs/2607.09629
作者: Xiaokai Bai,Lianqing Zheng,Runwei Guan,Songkai Wang,Siyuan Cao,Hui-liang Shen
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 5 pages, 8 figures
Abstract:Reliable autonomous driving requires full-scene perception that couples foreground objects with dense semantic layout. Recently, 4D millimeter-wave radar has emerged as a robust and affordable sensor, yet its sparse returns make radar-camera fusion necessary for comprehensive scene understanding. Existing radar-camera methods mainly optimize detection, while dual-task systems usually decode boxes and occupancy with limited interaction. To address this gap and advance radar-based multi-task learning, we propose \method, a 4D radar-camera framework for 360 ^\circ full-scene perception, which models semantic occupancy as a persistent scene state rather than a terminal output. \method follows a cross-modal state reasoning paradigm, where the occupancy state is modeled and propagated through stages for coarse-to-fine feature aggregation. Specifically, State-guided BEV Enhancement (SBE) strengthens intra-frame BEV representation, while Doppler-guided Temporal Fusion (DTF) preserves state evidence over longer temporal horizons. Beyond the model, we further extend ManTruckScenes with satellite-map-based generated occupancy labels and pair it with OmniHD-Scenes in a unified cross-dataset detection-and-occupancy protocol. The resulting experiments cover accuracy, robustness, ablation, and efficiency under one radar-camera multi-task evaluation framework. Code and labels will be released upon acceptance.
[CV-7] Promptable Concept Segmentation from Above: Evaluating SAM 3s Zero-Shot and One-Shot Capabilities in Remote Sensing
链接: https://arxiv.org/abs/2607.09583
作者: Mohammad Dabaja,Turgay Celik
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 14 pages, 4 figures
Abstract:The deployment of large-scale foundation models, such as the Segment Anything Model 3 (SAM 3), promises a transition toward open-vocabulary, training-free computer vision. However, their capacity to generalize out-of-distribution to the complex, top-down geometric structures of Earth Observation imagery remains largely unquantified. Driven by SAM 3’s performance disparities in highly specialized domains, we present a comprehensive, multi-task empirical evaluation across remote sensing scene classification, object detection, and instance segmentation under strict zero-shot and one-shot constraints. To achieve this, we introduce a structural adaptation of SAM 3 by repurposing its decoupled binary presence head into a standalone zero-shot classifier. Furthermore, by systematically isolating textual and visual prompt modalities across five configurations, we explicitly diagnose the alignment mechanics within the model’s multimodal decoder. Our findings reveal severe cross-modal interference: while visual prompts successfully align the decoder to complex remote sensing geometry, textual prompts inject misaligned, ground-level semantic bias, actively degrading coordinate regression. To benchmark these capabilities without resource-intensive training, we formulate a novel training-free proxy evaluation protocol for Generalized Zero-Shot tasks (scene classification and instance segmentation). Ultimately, our results demonstrate that SAM 3 avoids the overfitting commonly seen in legacy domain-adapted models, achieving high Harmonic Mean scores in segmentation tasks. However, it remains fundamentally constrained by sub-pixel resolution limits and overhead semantic blind spots, charting a definitive mandate for parameter-efficient geospatial fine-tuning of its multimodal decoder.
[CV-8] Wan-Dancer: A Hierarchical Framework for Minute-scale Coherent Music-to-Dance Generation
链接: https://arxiv.org/abs/2607.09581
作者: Mingyang Huang,Peng Zhang,Li Hu,Guangyuan Wang,Bang Zhang
类目: Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD)
备注: 17 pages, 13 figures, project: this https URL
Abstract:Generating long-duration, high-definition, and rhythmically synchronized dance videos directly from music remains a significant challenge, primarily due to the temporal constraints of current diffusion models, which typically fail beyond 20 seconds. Existing approaches, whether they rely on intermediate 3D skeletons or on end-to-end video synthesis, suffer from temporal drift, identity inconsistency, and repetitive motion patterns when extended to longer horizons. To address these limitations, we propose a novel hierarchical framework for minute-scale coherent music-to-dance generation. Our method decouples the process into global keyframe planning and local temporal refinement, leveraging full-track musical context to ensure long-range coherence. Key innovations include dynamic frame rate adaptation via time-mapped RoPE embeddings for precise alignment, an optical-flow-based loss function to enhance motion continuity, and motion-speed control to preserve high-fidelity details during rapid movements. Extensive experiments demonstrate that our framework surpasses the conventional duration barrier, generating stable, 720p/30fps videos exceeding one minute with superior temporal stability. Furthermore, the model exhibits robust versatility across five distinct dance genres, conditioned on both audio and textual prompts, establishing a new state-of-the-art in coherent, long-form dance video synthesis.
[CV-9] CLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models
链接: https://arxiv.org/abs/2607.09562
作者: Tianyou Jiang,Ziyu Zhou
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Medical Vision-Language Models (VLMs) exhibit strong zero-shot performance, yet their effectiveness still declines on out-of-distribution (OOD) data due to domain shifts and class bias inherited from large-scale pretraining. Existing few-shot adaptation methods typically introduce additional trainable components, which can be unstable in extremely low-data regimes (e.g., 1-shot), and lack robustness on different medical data. We present TCLA, a purely training-free few-shot adaptation method for Medical VLMs, which is fast and model-agnostic. TCLA corrects inference logits based on a small set of support samples, boosting pretrained VLMs performance by improving inter-class deconfusion and reducing domain shift. Extensive experiments on nine datasets across multiple medical imaging modalities including X-ray, Ultrasound, MRI, CT, Histopathology, demonstrate that TCLA consistently improves OOD performance of Medical VLMs and, in most of cases, outperforms existing training-based adaptation methods.
[CV-10] he Count Is There but Misaligned: Understanding and Correcting Counting Failures in VLMs
链接: https://arxiv.org/abs/2607.09544
作者: Ahmed Oumar El-Shangiti,Abzal Nurgazy,Hilal AlQuabeh,Nikolai Rozanov,Kentaro Inui
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Despite strong performance on many multimodal tasks, vision-language models (VLMs) still struggle with basic object counting. We investigate whether this reflects missing internal knowledge or a gap between internal representations and verbalized outputs. Training simple probes on activations from four VLMs across five counting datasets reveals that nonlinear probes can reliably detect counting errors, suggesting that VLMs often encode the correct count even when they output the wrong answer. SVCCA analysis shows that probes trained on ground-truth counts and probes trained on model outputs occupy a partially shared activation subspace but read out along misaligned directions. We further validate our findings using a causal steering intervention, proving that strengthening the direction of count-identified probes does improve model counting performance. Motivated by this result, we propose a detector-guided self-correction method that selectively re-prompts the model only when an internal error detector predicts failure. This simple inference-time intervention improves counting accuracy by up to 15.6 absolute percentage points, without any parameter updates. Our results establish activation-based error probing as both a practical tool for improving VLM counting and a mechanistic lens on the gap between internal knowledge and model outputs.
[CV-11] ALICE: Learning a General-Purpose Pathology Foundation Model from Vision Vision-Language and Slide-Level Experts
链接: https://arxiv.org/abs/2607.09526
作者: Jiawen Li,Tian Guan,Huijuan Shi,Xitong Ling,Mingxi Fu,Anjia Han,Chao He,Yonghong He
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Foundation models are reshaping computational pathology, yet their capabilities remain shaped by pretraining objectives, data sources, and spatial scales, fragmenting complementary expertise across separate backbones. Here we present ALICE, a unified foundation model trained through multi-stage agglomerative distillation that sequentially distills eight vision-only, vision-language, and slide-level teacher models into dedicated modules of a single backbone. ALICE is pretrained on 24,985,184 tile-level pathology images and 155,604 high-resolution images, and evaluated across 21 task scenarios, 96 downstream tasks, and 48 data sources, spanning region-of-interest tissue analysis, vision-language multimodal evaluation, and whole-slide clinical assessment. In all three evaluation settings, ALICE achieved the best average rank among task-matched pathology foundation models. These results demonstrate that agglomerative distillation can consolidate complementary capabilities from specialized models into a unified backbone for broad computational pathology applications. The model is available at this https URL.
[CV-12] Seeing is Free Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference ACM-MM2026
链接: https://arxiv.org/abs/2607.09520
作者: Junfei Zhan,Haoxun Shen,Mingang Guo,Zixuan Huang,Tengjiao He
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: Accepted to ACM MM 2026. 10 pages, 5 figures
Abstract:Vision-Language Models (VLMs) are the perceptual backbone of embodied AI, but their energy footprint on edge hardware remains poorly understood. Existing efficiency efforts focus predominantly on reducing visual tokens, implicitly treating visual processing as the dominant energy cost. We overturn this implicit assumption through the first systematic energy profiling of on-device VLM inference, spanning five models across three architecture families, four input resolutions, and two hardware platforms (NVIDIA RTX 3070 and Jetson Orin NX). Our analysis yields three findings. First, average inference power is a model-intrinsic constant, invariant to input resolution, image complexity, and prompt type, with less than 5% variation across all conditions. This means that all energy variation across inputs must arise from variation in inference time, not from variation in power draw. Second, each output token costs 11 to 39x more wall-clock time than each input token due to the compute-bound and memory-bound asymmetry between prefill and decode, making output token count the dominant driver of both latency and energy. Third, image complexity, measured by the number of objects in an image, induces up to 4.1x energy differences at identical resolution. This variation arises not from increased visual processing cost, but from differences in output length. These findings expose a fundamental limitation of visual token pruning: even removing all visual tokens saves at most 10% of total energy for fixed-token models. Across models spanning 1 billion to 8 billion parameters, controlling output length saves up to 97% of total energy, with the energy dominance of decoding growing stronger at larger model scale. In short, the true energy bottleneck in edge VLM inference is not what the model sees, but how much it says.
[CV-13] DGSfM: Depth-Guided Scale-Aware Global Structure-from-Motion
链接: https://arxiv.org/abs/2607.09507
作者: Sithu Aung,Viktor Kocur,Yaqing Ding,Torsten Sattler,Zuzana Kukelova
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Global Structure-from-Motion (SfM) is an efficient paradigm for recovering camera poses and sparse 3D structure from unordered images. However, its reliance on scale-ambiguous epipolar geometry makes global positioning sensitive to noisy baseline estimates and weak view-graph constraints, while false edges from visually ambiguous pairs can further degrade reconstruction. We propose DGSfM, a depth-aware global SfM pipeline that uses monocular depth maps as a scalable prior while preserving explicit multi-view optimization. For each image pair, we use a depth-aware relative pose solver to convert scale-ambiguous epipolar constraints into scale-aware relative pose constraints. We further improve robustness through view-graph filtering and depth-consistency-based correspondence pruning, which suppress false edges and matches that remain plausible under epipolar geometry alone. Finally, global scale averaging and depth-guided pose-point initialization align monocular depth maps into a common reconstruction scale and provide stable initialization for global positioning and bundle adjustment. Experiments on ETH3D and IMC2021 show that DGSfM consistently improves over strong global SfM baselines across sparse and dense matching front-ends, achieving substantial gains in pose accuracy. Code is available at this https URL.
[CV-14] What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibility
链接: https://arxiv.org/abs/2607.09503
作者: Filippo Ziliotto,Luciano Serafini,Lamberto Ballan,Tommaso Campari
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:A fundamental challenge in 3D reconstruction and robotic localization is co-visibility: determining which image pairs share overlapping visible surfaces, particularly in scenarios with minimal overlap. We demonstrate that VGGT implicitly encodes co-visibility as an emergent behavior: without any supervision for this task, its internal representations exhibit a clear hierarchical structure mirroring that of large language models, i.e. early layers build a 3D-aware scene representation, while late layers act as dedicated co-visibility reasoners. In particular, we identify layer L17 as a negative anchor that consistently routes non-co-visible pairs for this backbone, regardless of the evaluation setting, providing task-grounded evidence of layer specialization in a geometry-grounded foundation model. Building on this, we introduce Co-VGGT, which freezes VGGT and trains only a lightweight layer-wise mixture-of-experts head (less than 7.5M parameters) to classify co-visibility from RGB alone, treating each layer as a specialized expert whose geometric abstraction is adaptively weighted per input pair. On the Co-VisiON benchmark, Co-VGGT surpasses the human annotation baseline and improves over prior work by more than 25% pairwise and 10% multiview. Pairwise predictions are well-calibrated (ECE=0.030), enabling direct use as edge weights in visibility graphs for downstream SfM and SLAM pipelines without post-hoc correction. Code and data are available.
[CV-15] SigLIP-HD by Fine-to-Coarse Supervision ICLR2026
链接: https://arxiv.org/abs/2607.09488
作者: Lihe Yang,Zhen Zhao,Hengshuang Zhao
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: ICLR 2026. Code and model: this https URL
Abstract:High-quality visual representation is a long-standing pursuit in computer vision. In the context of multimodal LLMs (MLLMs), feeding higher-resolution images can produce more fine-grained visual tokens. However, it introduces additional computational and design complexity, due to multiple forward passes and post-processing of increased tokens. Before simply adopting a higher resolution, have we truly unlocked the model’s full perception capability at a standard resolution? Therefore, we study an interesting problem: how to achieve fine visual perception under lower cost without larger images. We present SigLIP-HD in this work. The core is a highly simple fine-to-coarse supervision design. We enforce the coarse feature of a mid-resolution image to mimic the fine-grained feature of its high-resolution version. We build this framework on the advanced SigLIP 2 model. Our final model produces better visual tokens at exactly the same inference budget. It is validated on extensive MLLM benchmarks and consistently delivers stronger results than our baseline model, especially on OCR-related tasks.
[CV-16] Decoupling Language Guidance from Backbones for Text-Guided Medical Segmentation
链接: https://arxiv.org/abs/2607.09481
作者: Yungeng Liu,Xuanzi Fang,Haijin Zeng,Qi Dai,Yongyong Chen
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Text-guided medical image segmentation leverages clinical semantics to improve lesion delineation, yet many existing models bind cross-modal fusion, supervision, and decoder design into a task-specific architecture. Such tight coupling makes it difficult to reuse language guidance modules across heterogeneous vision and text backbones, and often requires redesigning the network when the encoder pair changes. This paper presents BTHA, a backbone-transferable hierarchical adapter framework for text-guided medical image segmentation. BTHA is built around a stable feature-level interface: given multi-scale visual features and a text representation, it injects semantic guidance through shape-preserving adapters while maintaining the decoder-side tensor contract. To make this interface effective, we introduce a Hierarchical Coarse-to-Fine Supervision Strategy that decomposes learning into global image-text alignment, multi-scale auxiliary localization, and boundary-aware final mask refinement. We further design a Scale-Adaptive Gated Semantic Guidance (SAGSG) adapter, where resolution-specific gates adaptively control textual injection and channel recalibration suppresses redundant cross-modal responses. Evaluations across diverse vision and text backbones show that the same adapter and supervision design remains effective across convolutional and transformer-based visual encoders as well as different language encoders. Experiments on four public datasets further demonstrate that BTHA improves strong text-guided baselines with modest computational overhead.
[CV-17] Foveation-Guided Dynamic Token Selection for Robust and Efficient Vision Transformers
链接: https://arxiv.org/abs/2607.09480
作者: Ibrahim Batuhan Akkaya,Kishaan Jeeveswaran,Bahram Zonooz,Elahe Arani
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
备注:
Abstract:The human visual system (HVS) employs foveated sampling and eye movements to achieve efficient perception, conserving both metabolic energy and computational resources. Drawing inspiration from this robustness and adaptability, we introduce the Foveated Dynamic Transformer (FDT), a foveation-guided dynamic token-selection architecture that integrates these mechanisms into a vision transformer framework. The FDT exhibits strong resilience to various types of noise and adversarial attacks, despite not being explicitly trained for such challenges. This inherent robustness is achieved through the use of fixation and foveation modules: the fixation module identifies fixation points to filter out irrelevant information, while the foveation module generates foveated embeddings with multi-scale information. At the 50% fixation-budget setting, FDT achieves higher accuracy than DeiT-S (81.9% vs. 80.9%) while reducing multiply-accumulate operations by 34.57%, highlighting one operating point on its accuracy-efficiency trade-off. These attributes position FDT as an HVS-inspired step toward artificial neural networks that combine adaptive computation with improved resilience.
[CV-18] Hydra: Real-Time Hierarchical 3D Scene Graph Construction With Object-Level Shape Estimation IROS
链接: https://arxiv.org/abs/2607.09455
作者: Hyungtae Lim,Nathan Hughes,Xihang Yu,Ruihan Xu,Yun Chang,Jingnan Shi,Rajat Talak,Luca Carlone
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注: 8 pages, 12 figures, accepted in Proc. IEEE/RSJ IROS
Abstract:3D scene graphs provide a hierarchical abstraction of environments by encoding spatial entities, such as objects and places, and their relationships. However, existing scene graph systems model object geometry coarsely, relying on partial point clouds or class-level CAD templates, which limits instance-specific shape detail. This paper presents Hydra++, a system-level investigation into how learning-based object shape estimators can be integrated into a hierarchical 3D scene graph pipeline. Hydra++ incorporates category-agnostic shape estimation and a reprojection-mask consistency check to reject degenerate predictions from partial observations or imprecise segmentation. In its default CRISP-based configuration, Hydra++ performs online scene graph construction; slower estimators such as SAM3D are evaluated as modular alternatives to demonstrate generalization-latency trade-offs. Furthermore, to address the challenges of sparse and noisy depth measurements in outdoor environments, Hydra++ supports a hybrid LiDAR-camera configuration for large-scale operation, improving scene-level reconstruction quality. Experiments in both simulation and real-world outdoor campus scenarios demonstrate that Hydra++ improves object- and scene-level reconstruction quality. Project page is available at this https URL.
[CV-19] Robustifying Vision-Language Models via Test-Time Prompt Adaptation ICML2026
链接: https://arxiv.org/abs/2607.09450
作者: Xingyu Zhu,Huanshen Wu,Shuo Wang,Beier Zhu,Jiannan Ge,Jiaheng Zhang,Long Chen
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: ICML 2026 regular
Abstract:Pre-trained Vision-Language Models (VLMs) such as CLIP achieve strong zero-shot generalization, but their performance degrades sharply under adversarial perturbations. Existing test-time adaptation methods typically rely on sample-level confidence heuristics, overlooking the intrinsic distributional structure of the data. This sample-centric approach limits robustness, as it fails to distinguish confident adversarial mispredictions from true semantic consistency. In this work, we observe that adversarial distortion is structurally brittle: while holistic representations are corrupted, semantic integrity is often preserved in the distribution of augmented views. Motivated by this insight, we propose RITA, a Robust test-tIme prompt-TAdaptation framework that shifts from sample-level estimates to distribution-level alignment. Specifically, RITA employs optimal transport to align the distribution of augmented visual features with textual prototypes, mitigating adversarial outliers and rectifying cross-modal semantic misalignment. Furthermore, we introduce a dynamic cache to progressively accumulate reliable cues from the test stream for online refinement. Extensive experiments demonstrate that RITA significantly improves adversarial robustness without compromising clean accuracy.
[CV-20] Parameter-Efficient Vision-Language Adaptation with Continuous Metadata Conditioning for Animal Re-Identification
链接: https://arxiv.org/abs/2607.09443
作者: Anil Osman Tur,Tonje Knutsen Sordalen,Kim Tallaksen Halvorsen,Cigdem Beyan
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: This is the author’s version of the paper accepted for publication in Expert Systems with Applications. The final authenticated version will be available from the publisher
Abstract:Long-term animal re-identification (ReID) must remain robust to gradual morphological evolution and seasonal appearance shifts. Although recent vision-language models provide strong pretrained visual representations, adapting them to longitudinal ecological settings remains challenging, particularly under identity and temporal distribution shifts. We present a parameter-efficient CLIP adaptation framework for animal ReID and introduce a continuous metadata-conditioning mechanism that incorporates numerical attributes directly into the prompt representation during training. While low-rank visual adaptation, prompt-based supervision, and cross-modal alignment provide the adaptation framework, the proposed metadata-conditioning strategy constitutes the primary methodological contribution. By preserving the continuous structure of numerical metadata rather than discretizing it into textual categories, the proposed approach enables smooth modulation of the embedding space during training while maintaining a purely visual inference pipeline. Experiments on a seven-year longitudinal fish dataset and multiple wildlife benchmarks demonstrate improved performance under closed-set, open-set, and time-aware evaluation protocols. The results demonstrate that continuous metadata conditioning improves robustness to longitudinal appearance variation and temporal distribution shifts, while parameter-efficient adaptation enables a purely visual inference pipeline without requiring metadata at test time. Code and evaluation splits can be found at: this https URL.
[CV-21] Multimodal Scenario Similarity Search for Autonomous Driving
链接: https://arxiv.org/abs/2607.09428
作者: Tamás Matuszka,András Tamásy,Balázs Szolár
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Large-scale autonomous-driving datasets contain vast numbers of recorded scenarios, creating a need for efficient retrieval methods that can identify situations similar to a given query. Existing approaches typically rely on either visual representations or motion-based descriptions, making it difficult to understand their relative strengths and limitations for scenario retrieval. In this work, we present a multimodal framework for autonomous-driving scenario retrieval that combines visual and trajectory-based representations within a unified retrieval pipeline. We investigate two trajectory-based approaches: Exo-Trajectory, an explicit matching method based on surrounding-agent motion, and ScenarioFormer, a transformer-based representation learned from object trajectories using contrastive learning. We compare these approaches against strong vision-based baselines and analyze their behavior across a diverse set of driving scenarios. Experimental results show that trajectory representations provide strong retrieval performance for motion-centric events such as cut-ins, turning maneuvers, and traffic queueing, while visual embeddings excel when appearance cues are informative. Most importantly, combining visual and trajectory information consistently improves retrieval quality, yielding the best overall performance. These findings demonstrate that appearance and motion capture are complementary notions of scenario similarity and motivate multimodal retrieval systems for autonomous-driving data mining, dataset curation, and scenario-based validation.
[CV-22] SVF-CR: Synchronized Visual-Facial Cross-Refinement for Multimodal Ambivalence and Hesitancy Recognition
链接: https://arxiv.org/abs/2607.09417
作者: Hyein Park,Namho Kim,Junhwa Kim
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Ambivalence and hesitancy are subtle behavioral states that are expressed through a combination of verbal content, facial behavior, visual context, and acoustic cues. Effective recognition therefore requires not only extracting informative unimodal representations, but also modeling how temporally aligned behavioral evidence interacts across modalities. In this paper, we propose a synchronized visual-facial cross-refinement framework (SVF-CR) with pairwise multimodal evidence fusion for ambivalence and hesitancy recognition. The proposed method first extracts whole-video segment tokens and cropped-face segment tokens using the same temporal partition. The synchronized visual and facial tokens are refined through intra-modal self-attention and bidirectional visual-facial cross-attention, allowing whole-video context and local facial behavior to mutually refine each other before evidence construction. We then construct segment-level visual-facial evidence using consistency and discrepancy modeling, followed by temporal self-attention and attention pooling. Textual and acoustic features are lightly refined through context self-attention and are fused with the enhanced visual-facial evidence at the final decision stage using pairwise evidence fusion. Experiments on the BAH (Behavioral Ambivalence/Hesitancy) public evaluation split show that the proposed synchronized visual-facial cross-refinement improves public macro-F1 over both global visual-face token fusion and synchronized evidence baselines, achieving a public macro-F1 of 0.7156. Code is available at : this https URL_SVF-CR.
[CV-23] CtrlVTON: Controllable Virtual Try-On via Visual-Instance-Prompt Segmentation
链接: https://arxiv.org/abs/2607.09362
作者: Seungyong Lee,Hyun Jun Jang,Sangoh Kim,Sungjoon Park
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 13 + 17 pages, 20 figures
Abstract:Virtual try-on (VTO) has made significant progress in realistically transferring garments onto a target person. Yet most systems give the user little control over how a garment should be worn – its size (loose or fitted), style (e.g., tucked in or untucked, open or closed), and spatial placement on the body. We address this gap with two complementary contributions. First, we define and solve Visual-Instance-Prompt Segmentation via VIP-SAM: given a flatlay image of a garment, segment that specific instance in a photograph of a person wearing it. This is an instance-level task, distinct from the typically studied category-level segmentation. Second, we introduce CtrlVTON, a controllable VTO framework that recasts try-on as an image editing problem and adds segmentation masks as pixel-level control over garment layout, including style, size, and spatial placement on the body. VIP-SAM and CtrlVTON each achieve state-of-the-art results on their respective tasks. In particular, CtrlVTON generates images that follow user-provided layouts far more faithfully than the strongest proprietary editing systems while matching them on garment fidelity.
[CV-24] Simon-SR: Spatially Adaptive Modulation and Visual Prompt Adaptation for Text-Reinforced Super-Resolution
链接: https://arxiv.org/abs/2607.09351
作者: Haotong Cheng,Yuxuan Li,Zijie Cui,Rongling Tan,Chenyuan Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Multi-modal Single Image Super-Resolution
Abstract:Single Image Super-Resolution (SISR) reconstructs high-quality images from low-resolution inputs. While recent multi-modal methods improve perceptual quality, they remain sensitive to erroneous priors and require expensive annotations. To address these issues, we propose Simon-SR, a multi-modal SISR framework leveraging learnable prompts for efficient semantic mining and robust text-image fusion. Our approach combines Contrastive Prompt Learning with Prompt-Guided Spatially Adaptive Refinement to enhance multi-modal alignment. Experiments demonstrate that Simon-SR surpasses state-of-the-art methods, achieving maximum improvements of 0.50 dB in PSNR, 0.0133 in SSIM, and 0.0695 in LPIPS. Code will be released.
[CV-25] Dynamic Inverse Rendering for Enhanced Material-Lighting Decomposition ECCV2026
链接: https://arxiv.org/abs/2607.09329
作者: Raza Yunus,Benjamin Ummenhofer,Jan Eric Lenssen,Eddy Ilg
类目: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
备注: Accepted at ECCV 2026. Project page: this https URL
Abstract:Decomposing outgoing surface radiance into material and illumination during inverse rendering is essential for applications such as relighting and augmented reality, yet it is severely ill-posed since multiple combinations can result in the same observed colour. Capturing an object under multiple lighting conditions usually helps resolve this ambiguity as it constrains the optimization towards correct solutions. In this work, we explore the potential of reconstructing rigidly moving objects – which provides observations of diverse light-surface interactions – to resolve the material-lighting ambiguity in inverse rendering. For this purpose, we introduce a relightable approach that marries object tracking and reconstruction with inverse rendering for general rigidly moving objects. Our experimental analysis on synthetic data demonstrates that motion can be an advantage for disentangling material and lighting: the reconstructed material is significantly more accurate when the object is observed under rigid motion than when it is static. Moreover, results on RGB videos of real hand-held objects show that our pipeline preserves this advantage even under noisy real-world conditions.
[CV-26] From Classification to Localization and Clinical Validation: Large-Scale Development of a Deep Learning System for Thoracic Disease Detection on Chest Radiographs in Thailand
链接: https://arxiv.org/abs/2607.09305
作者: Isarun Chamveha,Tretap Promwiset,Napat Wanchaitanawong,Trongtum Tongdee,Pairash Saiviroonporn,Warasinee Chaisangmongkon
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Chest radiography (CXR) remains the most widely used thoracic imaging modality, yet expert interpretation is constrained by a severe shortage of radiologists in Thailand and across Southeast Asia. Local adaptation of deep learning models to Thai data has been shown to substantially improve accuracy on Thai populations. Here we present the development and comprehensive validation of the chest radiograph analysis model in Inspectra CXR version 5, a deep learning system that performs multi-label thoracic disease classification and weakly supervised lesion localization within a single model. The architecture couples a DenseNet-121 backbone with Attend-and-Compare Modules (ACM) and a Probabilistic Class Activation Map (PCAM) aggregation layer, producing a per-condition classification score and heatmap simultaneously. The model was developed on 874,858 frontal chest radiographs with paired radiologist reports from Siriraj Hospital, Bangkok. On a held-out, radiologist-verified in-domain test set of 19,871 cases, it achieved a mean AUROC of 0.994 (mean sensitivity 92.4%, specificity 98.6%) across nine clinically important conditions. On an independent generalization set of 5,992 cases from 13 hospitals across Thailand, the mean AUROC was 0.970, indicating robust transfer across sites. For localization, evaluated on 4,549 radiologist-annotated cases, the model attained a mean lesion-localization fraction (LLF) of 77.9% at 0.59 non-lesion localizations per image. In a usability evaluation with five thoracic radiologists, the system reached a classification concordance of 93.6%, a localization concordance of 94.7%, and a mean System Usability Scale (SUS) score of 89. These results indicate that a locally developed, localization-capable CXR system can deliver high accuracy, generalize across heterogeneous Thai hospitals, and earn the trust of practicing radiologists.
[CV-27] xtileNet: Towards Zero-shot Text-style Segmentation of Manuscripts ICDAR2026
链接: https://arxiv.org/abs/2607.09299
作者: Anguelos Nicolaou,Antonella Ambrosio,Desiree Di Donato,Georg Vogeler
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: accepted for publication in the ICDAR 2026 workshop (peer reviewed) “IWCP: 4th International Workshop on Computational Paleography”
Abstract:Automatic writer identification systems have progressed remarkably in recent years, yet their deployment in archival paleography remains limited by the scarcity of labeled training data, open scribe sets, and degraded image quality. We present TextileNet, a fully convolutional multi-task network trained exclusively on synthetic data to produce dense pixel-level texture embeddings, which we transfer zeroshot to historical manuscript analysis. As an original contribution to evaluation methodology, we designed a paleographic visual quiz of 80 pair and triplet questions and administered it to a range from lay participants to senior paleographers under strict anonymity, establishing to our knowledge for the first time a human baseline for script-style discrimination on late medieval text. We employ TextileNet embeddings to perform zero-shot retrieval on sub-word granularity for hand and gender identification. Our experimental results help in building the credibility of TextileNet in the paleographic domain, but more than that demonstrate in experimental terms that the question of gender in handwriting needs to be treated with caution.
[CV-28] Rethinking Monocular Depth Embedding for Generalized Stereo Matching
链接: https://arxiv.org/abs/2607.09284
作者: Libo Lin,Shuangli Du,Minghua Zhao,Zhenzhen You,Shun Lv,Yiguang Liu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 15 pages, submitted to Pattern Recognition
Abstract:Generally, monocular methods capture rich contextual priors but lack geometric precision, whereas stereo methods are geometrically accurate yet struggle in textureless and occluded regions. Several approaches attempt to combine their strengths to enhance the generalization of stereo matching (SM) by aligning monocular depth with stereo information. However, establishing a stable and generalizable alignment is challenging, and unreliable monocular cues can substantially degrade performance. This paper rethinks monocular depth embedding. First, to prevent shortcut learning, we reduce branch coupling instead of expanding network width. Second, we construct soft constraints instead of hard ones from monocular depth to improve tolerance to monocular depth errors. Based on the principles, we integrate monocular information into both feature extraction and GRU iterations. Specifically, the monocular depth map is fused with the RGB image to sharpen depth boundary perception and suppress matching ambiguities. The fused image is then used for feature extraction, allowing the contextual features to encode global geometric information. Furthermore, the monocular depth gradient feature is employed to guide disparity updates, helping to escape local oscillations. Finally, to address the boundary blurring of supervised disparity caused by data augmentation, we propose an edge confidence estimation method and an edge-aware loss function. Our method achieves state-of-the-art (SOTA) performance on multiple standard benchmarks, demonstrating excellent generalization while improving accuracy. The code is available at this https URL.
[CV-29] REMIND: RE-Identification with Memory for INDoor Navigation
链接: https://arxiv.org/abs/2607.09267
作者: Pablo Diaz-Pereda,Alejandro Rodriguez-Ramos,David Perez-Saura,Pascual Campoy
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 11 pages
Abstract:Mobile robots operating indoors must re-identify previously observed objects after long temporal gaps, significant viewpoint changes, and severe illumination variations. This remains a challenging problem: multi-object tracking methods are optimized for short-term association of pedestrians and vehicles at video rates, person and vehicle re-identification approaches lack persistent memory mechanisms, and state-of-the-art video object segmentation techniques rely on reactive distractor filtering rather than enforcing global identity consistency. To address these limitations, we present REMIND, an online tracker designed for long-term multi-object re-identification of generic indoor objects from monocular RGB imagery, requiring neither camera pose nor depth. Motivated by evidence from visual cognition that humans rely on accumulated appearance familiarity and spatial context rather than explicit self-localization, REMIND combines frozen DINOv3 features with a dual-bank multi-prototype appearance memory, part- and background-level descriptors, a neighbour-context reasoning module exploiting spatial co-occurrence, and joint Hungarian assignment with ambiguity-aware safeguards. On a purpose-built indoor dataset featuring controlled revisits and dense same-class clutter, REMIND reaches 90.35% IDF1, nearly 20 points above a state-of-the-art video object segmentation baseline and more than 36 above a strong tracking-by-detection baseline. On ScanNet++, it attains the highest IDF1 in every setting but one, end-to-end detection over all scenes, where the tracking-by-detection baseline is marginally ahead while REMIND still associates and recovers identities more accurately; it also completes every scene, whereas the video object segmentation baseline exhausts GPU memory on 66.9% under YOLO detections. The complete system, evaluation framework, and dataset are publicly released. Comments: 11 pages Subjects: Computer Vision and Pattern Recognition (cs.CV) Cite as: arXiv:2607.09267 [cs.CV] (or arXiv:2607.09267v1 [cs.CV] for this version) https://doi.org/10.48550/arXiv.2607.09267 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[CV-30] Semantic Hardness Is Not Visual Hardness: Sign-Aware Hard Negative Mining for Sign Language Retrieval ACL2026
链接: https://arxiv.org/abs/2607.09263
作者: Junmyeong Lee,Chan Hur,ChangSu Choi,Sukmin Cho,Fitsum Gaim,Eui Jun Hwang,Hoyun Song,KyungTae Lim
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted to ACL 2026 main
Abstract:Sign Language Retrieval (SLRet) enables efficient access to sign language content but remains fragile in fine-grained scenarios where visually similar signs must be distinguished. We show that this limitation does not stem from model capacity, but from ineffective hard negative supervision. Specifically, we formulate fine-grained retrieval failures as a negative distribution mismatch: semantically distinct yet visually confusable signs are rarely treated as hard negatives, while existing text-based mining strategies fail to capture such visual ambiguity. To address this issue, we propose Sign-Aware Hard Negative Mining (SAN), which constructs hard negatives based on visual confusability in the sign embedding space rather than linguistic similarity. Experiments on PHOENIX-2014T demonstrate that SAN substantially improves fine-grained retrieval performance while preserving coarse-grained accuracy, highlighting the importance of aligning negative supervision with visual ambiguity in sign language retrieval.
[CV-31] AnythingReality: Robust Online Gaussian Splatting SLAM for Open-Vocabulary VR Scene Exploration
链接: https://arxiv.org/abs/2607.09260
作者: Timofei Kozlov,Dmitrii Maliukov,Andrey Marchenko,Miguel Altamirano Cabrera,Dzmitry Tsetserukou
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:We present a novel integrated architecture for robust online 3D Gaussian splatting, real-time VR exploration, and speech-driven Vision-Language-Model interaction. Unlike methods assuming clean depth or external poses, our system combines ORB-SLAM3-based pose estimation with online Gaussian reconstruction for noisy real-world data. A VR pipeline enables immersive exploration of incremental reconstructions; a semantic module transcribes voice commands, generates scene descriptions, and records points of interest. Against state-of-the-art online Gaussian splatting methods, we improve image quality on our dataset (+14.5% PSNR, +8.6% SSIM, -14.3% LPIPS) and TUM-RGBD (+11.7% PSNR, +7.8% SSIM, -21.6% LPIPS), with comparable or superior frame rates via quality-speed configurations. We achieve an 88% VLM object-recognition rate.
[CV-32] All you need is SAMPAT
链接: https://arxiv.org/abs/2607.09235
作者: Jayadeva,Madhur Aswani
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Functional Analysis (math.FA)
备注: 7 pages
Abstract:The current state of the art in AI/ML rests on deep neural architectures, which, in general, suffer from a lack of interpretability. Interpretability is crucial to gleaning insights while analyzing experimental data, where quantitative predictions may not be adequate for a scientist. We present a three layer neural architecture, SAMPAT (Smooth Approximation via Multivariate Polynomials and Analytic Transformations), that can provably learn a continuous, everywhere differentiable function, that can approximate any smooth function arbitrarily closely. SAMPAT’s approximant can be expressed as a closed and compact algebraic, analytic expression, providing complete interpretability. Experiments on synthetic and benchmark datasets indicate that SAMPAT yields competitive performance with simpler representations. For many tasks, a two layer SAMPAT suffices. By imposing restrictions on the connectivity between neurons, SAMPAT may be used to provide a range of approximants, including regular and trigonometric polynomials, rational expressions, Gaussians, mixtures of Gaussians, as well as arbitrary combinations of the same; without restrictions, it learns a suitable structure. SAMPAT may be used to factorize polynomials and model nonlinear systems. With the addition of skip connections, a 4 to 6 layer SAMPAT is adequate to represent a substantive range of methods widely used in AI/ML, allowing the choice of a model’s family, not just its parameters, to also be optimized as part of the learning process.
[CV-33] Glob3R: Global Structure-from-Motion with 3D Foundation Models
链接: https://arxiv.org/abs/2607.09225
作者: Junyuan Deng,Heng Li,Kejie Qiu,Lingteng Qiu,Rui Peng,Weichao Shen,Weihao Yuan,Siyu Zhu,Zilong Dong,Ping Tan
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Recent 3D geometric foundation models, such as VGGT, provide robust feed-forward 3D reconstruction by directly predicting camera poses and 3D scene points from input images. However, their results remain inaccurate, and scaling them to long sequences or large unordered image sets typically requires chunk-wise processing, which can introduce drift and inconsistency. We present Glob3R, a global SfM-style reconstruction built on 3D foundation models. Our key idea is to explicitly optimize feed-forward geometric predictions. To this end, we augment a frozen Pi3X backbone with a lightweight dense matching head that predicts image warps between selected reference frames and neighboring views. These dense warps are converted into sparse but reliable multi-view feature tracks, which provide correspondence constraints for global optimization. We further introduce a keyframe-based sliding-window association strategy that propagates tracks and relative poses across overlapping windows, enabling scalable reconstruction. Finally, we perform global motion averaging and bundle adjustment to refine camera poses, reduce scale inconsistencies, and recover dense scene geometry. Extensive experiments on indoor, outdoor, large-scale driving, and unordered SfM benchmarks demonstrate that Glob3R achieves robust and accurate reconstruction. It consistently improves over feed-forward foundation-model baselines and recent scalable reconstruction methods, while being more robust than classical SfM pipelines. The refined poses also lead to higher-quality neural rendering, validating the benefit of combining foundation-model priors with global geometric optimization. Project page: this https URL
[CV-34] YeTI: You Only Need Two Noisy Images for Real-World sRGB Noise Generation ECCV2026
链接: https://arxiv.org/abs/2607.09193
作者: Jaekyun Ko,Byung Wan Lim,Soomin Lee,Dongjin Kim,Tae Hyun Kim
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted to ECCV 2026. Includes supplementary material
Abstract:Real-world sRGB image denoising remains challenging due to the nonlinear characteristics of sensor noise and the difficulty of acquiring aligned clean-noisy image pairs. Supervised denoisers often overfit to limited paired datasets, while self-supervised methods still depend on sufficiently diverse noisy observations. These limitations motivate scalable noise synthesis methods that can model real-world noise without clean ground truth or camera metadata. We propose YeTI, a real-world sRGB noise generation framework that learns from only two noisy observations of the same scene. YeTI uses a Reconstruction Autoencoder to disentangle scene structure and noise characteristics, and models the latent noise distribution with a one-step Conditional Diffusion Transformer trained using consistency objectives. Given a single noisy input at inference time, YeTI generates realistic, signal-dependent noise while preserving the underlying scene content. Extensive experiments demonstrate the effectiveness of YeTI across real-world benchmarks. We evaluate noise generation on SIDD and further assess generalization on SIDD+, MAI2021, and SID, covering smartphone and diverse consumer-camera sensors. Downstream denoising results on DND further show that denoisers trained with YeTI-synthesized images achieve strong real-world performance, highlighting the practical value of clean-image-free and metadata-free noise generation.
[CV-35] HiHR: Hierarchical Hyperbolic Representation for Aerial-Ground Person Re-Identification ECCV2026
链接: https://arxiv.org/abs/2607.09186
作者: Qiwei Yang,Pingping Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted by ECCV2026. More modifications may be performed
Abstract:Aerial-Ground Person Re-IDentification (AG-ReID) aims to retrieve the same person across heterogeneous aerial and ground camera platforms. Although great progress has been made, existing methods remain suboptimal due to the direct feature alignment across views, overlooking view-specific cues. To address this issue, we propose a novel Hierarchical Hyperbolic Representation (HiHR) framework for AG-ReID. More specifically, we first extract multi-granularity features based on pre-trained visual-text encoders. Then, we propose a Text-guided Multi-granularity Fusion (TMF) to fuse multi-granularity features and enhance the representation ability of identity features. Furthermore, we introduce the Hierarchical Hyperbolic Learning (HHL) to construct a hierarchical feature structure in a hyperbolic space. This hierarchy includes a coarse level that ensures identity separability and cross-view consistency, and a fine level that preserves view-specific discriminative cues. As a result, our proposed framework can effectively aggregate view-invariant and view-specific discriminative features for AG-ReID. Extensive experiments on four AG-ReID benchmarks demonstrate the effectiveness of our framework. The source code is available at this https URL.
[CV-36] Causally Debiased Latent Action Model for Embodied Action Conditioned World Models
链接: https://arxiv.org/abs/2607.09185
作者: Yufan Wei,Kun Zhou,Lingjun Mao,Zijun Zhang,Ziming Xu,Ziqiao Xi,Shuang Liang,Ruobing Han,Yuchen Yan,Xinyue Wang,Fan Feng,Biwei Huang
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注:
Abstract:Action-conditioned world models (ACWMs) aim to simulate future observations conditioned on embodied actions, offering a promising foundation for robot planning, policy evaluation, and data augmentation. However, learning controllable ACWMs requires large-scale action-labeled data, which remains costly to collect in the real world. Latent action models (LAMs) mitigate this bottleneck by inferring latent actions from unlabeled videos, but existing LAMs are typically trained with reconstruction-only objectives and therefore entangle action-relevant dynamics with action-irrelevant visual factors such as backgrounds and untouched objects. In this work, we identify this action-irrelevant bias as a key obstacle to controllable ACWMs and introduce evaluation metrics to measure latent-action bias, action following, and robustness. We propose CD-LAM, a causally debiased framework for LAM-based ACWMs. CD-LAM introduces three efficient fine-tuning objectives: embodiment-centric reconstruction, action-centric contrastive learning, and latent space calibration, which together encourage embodiment-focused, action-aware, and calibrated non-collapsed latent action representations. Experiments on 2B and 14B ACWM backbones show that CD-LAM substantially improves latent-action controllability, downstream robot-action following, visual fidelity, and adaptation efficiency, requiring only 6k fine-tuning steps and more than 12 \times fewer robot-action adaptation updates than the baseline.
[CV-37] SR-Ego: Temporally Guided Stereo Refinement Framework for Egocentric 3D Human Pose Estimation
链接: https://arxiv.org/abs/2607.09169
作者: Md Mushfiqur Azam,John Quarles,Kevin Desai
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Egocentric 3D human pose estimation from head-mounted stereo cameras is challenging due to fisheye distortion, severe self-occlusion, and frequent truncation of body joints outside the camera field of view. Recent stereo egocentric methods have improved performance through heatmap lifting, stereo correspondence, and transformer-based refinement, but they often rely heavily on frame-local evidence or use temporal information only as auxiliary pose-level context. This limits robustness when current-frame stereo cues are weak, occluded, or ambiguous. We propose TSR-Ego, a temporally guided stereo framework that couples short-term motion evidence with projection-guided feature sampling. The model first enriches dense stereo feature maps using a causal depthwise-separable temporal convolution, allowing past visual evidence to influence the feature space before deformable cross-attention. A single-stage causal stereo decoder then refines learned 3D joint queries through temporal self-attention, joint self-attention, and fisheye deformable stereo cross-attention, using the evolving pose estimate to generate 2D sampling references. Unlike methods that apply temporal reasoning mainly after pose prediction, TSR-Ego uses motion context to shape both the sampled stereo features and the joint representations while preserving online inference without future frames. Experiments on UnrealEgo2 and UnrealEgo-RW show state-of-the-art performance, with especially strong gains on real-world sequences.
[CV-38] What Pixels Are Enough? SEAMS: Sufficiency Saliency via MSE-Preservation Soft-Masks
链接: https://arxiv.org/abs/2607.09164
作者: Magdalena Trędowicz,Łukasz Struski,Arkadiusz Lewicki,Karolina Pachota,Andrzej Grudzień,Mateusz Jagła,Jacek Tabor
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Saliency maps are most useful when they identify the image regions that are sufficient to preserve a model’s behaviour. We introduce SEAMS, a sufficiency-based saliency method that directly optimises a soft mask using a preservation objective. Given a frozen differentiable model output, such as a class probability, CLS embedding, or token representation, SEAMS searches for a compact mask that preserves the selected output. The approach relies on a simple optimisation framework based on soft masks, a learnable budget, and a three-way image composite generated entirely from the query image. As a result, it requires no auxiliary distractor dataset, architecture-specific attribution mechanism, or differentiable top-k relaxation. Experiments with frozen ViT-S/16 and ConvNeXt models show that the same optimisation pipeline can generate object-level, class-conditioned, and token-level explanations by changing only the preserved target. The resulting masks are compact, interpretable, stable across random initialisations, and competitive on insertion and deletion benchmarks. Our results also indicate that different architectures often rely on different sufficient evidence while achieving similar preservation fidelity, highlighting the architecture-dependent nature of visual explanations.
[CV-39] Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing
链接: https://arxiv.org/abs/2607.09143
作者: Chenxu Peng,Chongtian zhou,Dicheng Liu,Bo-Wen Yin,Yimian Dai,Xialei Liu,Ming-Ming Cheng,Xiang Li
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Fusing standard RGB frames with asynchronous event streams has emerged as a definitive paradigm for robust perception in degraded environments. Although unified backbones have recently gained traction in multi-modal vision, adapting them to the RGB-Event domain remains fundamentally challenging. Existing architectures either resort to decoupled dual encoders that double computational overhead, or adopt generic unified designs that fail to resolve implicit geometric parallax and cross-spectral aliasing under the extreme representational divide between dense intensity grids and sparse kinematic spikes. To transcend these bottlenecks, we present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing. To achieve profound modal synergy, Evita explicitly embeds a suite of intrinsic co-learning modules directly into every encoder layer. Specifically, it features Geometric Parallax Rectification for adaptive spatial alignment, Harmonic Spectral Resonance for texture transfer exclusively in the complex frequency domain, and Transient Global Routing for event-driven asymmetric attention. To guarantee robust feature extraction against spatial misalignments and decouple representations from specific event encodings, we construct N-ImageNetV2 alongside a stochastic event representation mixing pretraining protocol, empowering the network to seamlessly accommodate arbitrary event formats in downstream tasks. Extensive evaluations across the DELIVER, DDD17, and DSEC benchmarks confirm that Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal this http URL code are publicly available at: this https URL.
[CV-40] Super-Generalist: Towards Comprehensive and Accurate Medical Image Understanding via Generalist-Specialist Synergy
链接: https://arxiv.org/abs/2607.09135
作者: Shaoteng Zhang,Weiwei Cao,Wanxing Chang,Yutong Xie,Kai Cao,Zaiyi Liu,Yu Shi,Tingbo Liang,Qi Zhang,Ling Zhang,Yong Xia,Jianpeng Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Medical images require comprehensive and accurate interpretation to support the diagnosis of diverse clincial conditions. Recent vision-language generalist models offer broad task coverage and promising zero-shot capabilities, yet often lack fine-grained anatomical and lesion awareness for reliable diagnosis and spatial interpretability. In contrast, supervised specialist models achieve strong performance on specific tasks but typically lack generalization across diseases and anatomies. In this work, we present SuG, a Super-Generalist framework that unifies generalist vision-language learning with specialist objectives, enabling both broad generalization and specialist-level diagnostic capability. We perform specialist-enhanced vision-language alignment in SuG by incorporating spatial priors from multiple segmentation experts, including anatomy, class-specific lesion and class-agnostic lesion segmentors that captures lesions beyond anatomies annotated during training. To improve lesion grounding capability, we leverage lesion masks as spatial priors to calibrate text-conditioned visual attention, encouraging disease-related semantics to focus on clinically relevant regions. We evaluate SuG on extensive chest and abdominal CT benchmarks, including CT-RATE, Merlin, MedVL-CT69K, and several in-house tumor datasets. SuG achieves state-of-the-art performance across a wide range of disease diagnosis tasks and surpasses specialist models on several critical tumor diagnosis benchmarks. Furthermore, SuG demonstrates strong lesion grounding capability, including robust generalization to lesion types lacking class-specific supervision.
[CV-41] IB-Flow: Information Bottleneck-Guided CFG Distillation for Few-Step Text-to-Image Generation
链接: https://arxiv.org/abs/2607.09133
作者: Yiting Wang,Jingyi Zhang,Wenhu Zhang,Ke Chao,Yves Liang,Kun Cheng,Kang Zhao
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:While large-scale text-to-image generative models have achieved unprecedented visual performance, their inherent reliance on multi-step iterative solvers incurs severe inference latency. Few-step distillation targeting the Classifier-Free Guidance (CFG) trajectory has emerged as the prevalent dual-dimensional compression paradigm. However, existing frameworks remain subjugated by a coarse-grained blind injection paradigm that perpetually enforces a globally static guidance strength while indiscriminately sampling the supervisor timestep. This state-agnostic design completely disregards the intrinsic nature of image generation as a dynamic evolutionary process characterized by progressive entropy reduction, which not only restricts the performance boundary of few-step compression but also precipitates severe CFG over-conditioning artifacts. To transcend these limitations, we re-examine the distillation procedure through the theoretical lens of Information Theory, formally modeling it as a dynamic mutual information game constrained by the Information Bottleneck (IB) principle. Specifically, we dismantle traditional blind assumptions via a dual-track adaptive framework. To determine the injection target, we propose an instance-aware selection mechanism that transmutes the intractable KL divergence constraint into a zero-overhead closed-form solution predicated on the local vector field norm. To regulate the injection strength, we introduce an entropy-aware schedule that dynamically decays alongside the SNR, applying maximal thrust for initial structural anchoring before smoothly reverting to the natural manifold to refine micro-details. Extensive empirical evaluations corroborate that our framework fundamentally eradicates over-conditioning artifacts, shattering the performance ceiling to achieve SOTA generative fidelity under extremely stringent 2-step configurations.
[CV-42] 4D Human-Scene Reconstruction from Low-Overlap Captures SIGGRAPH
链接: https://arxiv.org/abs/2607.09125
作者: Minhyuk Hwang,Sangmin Kim,Seunguk Do,Daneul Kim,Jaesik Park
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted to SIGGRAPH Conference Papers '26. First two authors contributed equally. Project page: this https URL
Abstract:Existing volumetric capture of dynamic human performance achieves high fidelity with dense camera arrays. However, in real-world scenarios, only a handful of low-overlap cameras are available, which degrades the output quality and leaves large areas unobserved. Recent 4D reconstruction methods have focused on low-overlap settings, yet they still produce noticeable artifacts in under-observed regions. Video diffusion models have emerged as another option, but they show geometrically inconsistent results for humans. To address these limitations, we propose StudioRecon, a pipeline that reconstructs 4D human scenes from sparse, low-overlap cameras by decoupling background and humans. We densify background supervision by synthesizing hundreds of camera-controlled novel views with a video diffusion model. We also robustly initialize deformable Gaussian humans with cross-view identity association and triangulated multi-view keypoint fitting. Finally, our recursive enhancement module with motion-adaptive consistency injection harmonizes the composed output, thereby further avoiding remaining artifacts. We achieve state-of-the-art novel view synthesis across four real-world datasets and demonstrate applications such as novel trajectory rendering and human replacement.
[CV-43] Event Burst Trigger: An Availability Backdoor Attack on Event-Based SNN Object Detection DSN2026
链接: https://arxiv.org/abs/2607.09115
作者: Jaesun Baek,Chanwook Lee,Eun-Kyu Lee
类目: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
备注: The 56th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2026)
Abstract:Event-based vision and spiking neural networks (SNNs) are increasingly adopted for edge intelligence under strict latency and energy constraints. However, the vulnerability of event-based SNN object detection models to availability backdoor attacks remains insufficiently studied. This paper presents Event Burst Trigger (EBT), an availability backdoor attack targeting SNN-based object detection models. EBT injects carefully crafted event-based triggers into the training data, which induce temporally concentrated event streams during inference. These burst-like activations increase the number of phantom (i.e., spurious) object candidates, and consequently inflate the computational cost of the post-processing stage, particularly Non-Maximum Suppression (NMS). We evaluate EBT on SpikeYOLO, the state-of-the-art SNN-based object detector, under a poison-only threat model that does not require modifications to the model architecture, loss function, or inference pipeline. Experimental results show that while detection accuracy remains largely preserved, with mAP@0.5 decreasing by less than 0.099, the latency of the NMS stage increases by up to 38%. This indicates that NMS can become a dominant availability bottleneck in event-based SNN object detection. Experiments on an edge platform further show that the proposed attack elevates baseline resource utilization and reduces scheduling slack without inducing conspicuous peaks in resource usage. In addition, STRIP-based backdoor detection fails to reliably distinguish the proposed attack from benign inputs. These results characterize a previously underexplored availability backdoor threat in event-based SNN object detection systems.
[CV-44] Event Stream based Multi-Modal Video Anomaly Detection: A Benchmark Dataset and Algorithms
链接: https://arxiv.org/abs/2607.09114
作者: Peipei Zhu,Yueqing Niu,Lin Zhu,Guanchong Niu,Yang Yu,Zheng Li
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)
备注:
Abstract:Video anomaly detection (VAD) is critical for automated surveillance but remains fragile under challenging conditions such as illumination variations, fast motion, and complex backgrounds when relying solely on visible light videos. To address these limitations, we propose EVAD, an event enhanced VAD framework that jointly exploits conventional video and event streams captured by bio inspired event cameras. Event sensors asynchronously capture brightness changes with high temporal resolution, offering robustness to motion blur and extreme lighting, and providing motion salient cues complementary to video based visual information. To support multi modal VAD research, we construct a large scale visible event benchmark comprising 6.3 billion events and 376,368 video frames collected under diverse illumination levels, motion patterns, and background complexities, filling the gap of realistic and scalable datasets for event based anomaly detection. Building upon this dataset, we design a contrastive multi modal pretraining framework to learn discriminative event representations by aligning semantic embeddings across event streams, visible videos, and textual descriptions. An adaptive fusion module then dynamically integrates event based temporal cues with video based spatial semantics, improving robustness to environmental disturbances. Experiments on benchmarks and the proposed TJUTCM Pha dataset demonstrate that E VAD consistently outperforms methods, validating the effectiveness of event-based sensing for VAD in real world scenarios.
[CV-45] Integrating Large Language Models and Graph Convolutional Networks for Semi-Supervised Image Classification
链接: https://arxiv.org/abs/2607.09104
作者: Camila Piscioneri Magalhães,Lucas Pascotti Valem
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:While the growing availability of image data has driven significant advances, labeling datasets remains costly and time-consuming. Therefore, semi-supervised approaches such as Graph Convolutional Networks (GCNs), which learn from both labeled and unlabeled data, have emerged as a promising solution. One of the primary challenges in applying GCNs to image classification is graph construction, since, unlike in citation networks or similar domains, images typically do not come with a predefined structural representation. For visual data, most studies construct graphs based on the similarity between feature vectors from pretrained deep learning backbones, typically by employing kNN or reciprocal kNN algorithms. Although Large Language Models (LLMs) have shown remarkable capability in capturing high-level semantics, their integration with GCNs for image classification remains underexplored. Aiming to fill this gap, our approach uses a Vision Language Model (VLM) to generate textual image descriptions, which are then processed by an LLM to estimate semantic similarity scores between connected images. These scores guide the pruning of edges in kNN and reciprocal kNN graphs, filtering out semantically irrelevant neighbors. Experimental results reveal that leveraging LLMs for graph refinement can improve classification accuracy, particularly for kNN graphs and some backbones. The source code is publicly available at this http URL.
[CV-46] Equivariant Filter for High Performance Image Tracking using an Event Camera
链接: https://arxiv.org/abs/2607.09103
作者: Angus Apps,Yixiao Ge,Timothy L. Molloy,Robert Mahony
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Image tracking is the problem of estimating the transformation that relates a moving image of a scene to an original reference image. The problem is important in control of autonomous vehicles or robots, where the image encodes information about the motion of the camera or environment, as well as in pure computer vision applications. In this paper, we present an equivariant filter design for high performance tracking of planar image transformations using an event camera. The design exploits the Asynchronous Event Blob (AEB) tracker (Wang et al., 2024) to extract feature-position measurements from the raw event stream, and an equivariant filter to compute an affine image translation and rotation using the special Euclidean group symmetry. The equivariant filter incorporates an equivalent-measurement update step that de-correlates the (highly temporally correlated) feature-position measurements provided by the AEB tracker. We evaluate the design experimentally using two datasets involving general and fast rotational motion. We benchmark results against direct optimisation (estimating the relative transformation from the raw blob tracks), and a covariance intersection approach for overcoming data correlation. Our design provides smooth image tracking for features moving up to 7000 pixels per second on the image plane.
[CV-47] A Coreset Selection Framework with Ensemble Aggregation for Image Classification
链接: https://arxiv.org/abs/2607.09100
作者: Pedro Rocha Dantas,Lucas Pascotti Valem
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:The rapid growth of image data has produced large-scale datasets, raising concerns about the time and memory costs of model training. Selecting representative training subsets, however, remains challenging: individual sample contributions are unclear, and model behavior varies across datasets and runs. We address these challenges with a framework that combines coreset selection with an ensemble aggregation over multiple runs. For coreset selection, we propose SCOre-Stratified Selection (SCOSS), which partitions the training data into intervals based on a chosen score and samples from each interval. The ensemble combines predictions from multiple runs, each performed on an independently sampled training subset. As baselines, we use moderate and random selection, each in original and class-balanced versions. We assess the framework with Simple Graph Convolution (SGC) and Support Vector Machine (SVM) classifiers under different sampling ratios. Experiments show that SCOSS is competitive with baselines, often the best choice for SGC, and enables favorable trade-offs between accuracy and efficiency. On the fine-grained dataset, SGC with SCOSS outperforms SVMs when using fewer labeled samples. The code and supplementary materials are publicly available at this http URL.
[CV-48] Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models ECCV2026
链接: https://arxiv.org/abs/2607.09091
作者: Yijie Qian,Juncheng Wang,Chao Xu,Huihan Wang,Yuxiang Feng,Yang Liu,Baigui Sun,Yong Liu,Shujun Wang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted to ECCV 2026
Abstract:As audio-visual generative models evolve into world simulators, cross-modal synchronization stands as a critical proxy for assessing the consistency of world dynamics and causality in generated content. However, existing evaluation metrics presume structural correctness, reducing synchronization to mere temporal alignment. Consequently, they fail on generative outputs, especially when exhibiting structural hallucinations and asymmetric cross-modal relations, which currently \textbfmandate expert human annotation to assess synchronization. This dependency introduces a critical paradox: \emphhuman evaluators rely on relative, reference-dependent comparisons, whereas automated metrics require reference-free, absolute scalars. We resolve this paradox by proposing a framework that distills relative human perception into a continuous, globally consistent metric. First, we introduce SynthSync, a dataset of generative failures ranked via pairwise human annotations. Second, we adapt the Omni-LLM equipped with a continuous latent projection to translate relative human rankings into continuous absolute values. Third, we propose Real-Valued Group Relative Policy Optimization ( \mathbbR -GRPO) to internalize the global causal structure of synchronization via listwise score distributions. Empirically, our metric achieves state-of-the-art human preference alignment. We leverage this estimator to establish a standardized benchmark, advancing AV-Gen assessment from low-level signal correlation to visually grounded causality.
[CV-49] DETRAM: End-to-end DEtection Tracking and Recovery of HumAn Meshes
链接: https://arxiv.org/abs/2607.09089
作者: Chunggi Lee,Seonwook Park,Wanhua Li,Umar Iqbal,Hanspeter Pfister
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:In the task of human mesh recovery (HMR), multi-person scenes are particularly difficult to handle due to the many entities that appear and occlusions between them over time. In particular for video inputs, there is a need to track each entity reliably and consistently. Existing methods rely on pretrained human detection modules, increasing their runtime and limiting the number of tracked entities. We present DETRAM, a unified framework for multi-person HMR and tracking that simultaneously detects, reconstructs, and tracks humans across time, both automatically and via user prompts. DETRAM uses a single transformer decoder with an identity-consistent set of learnable query embeddings that persist across frames: detection queries discover new people, tracking queries maintain pose and shape for existing individuals, and prompt queries follow user-specified identities. Our approach achieves state-of-the-art tracking results on PoseTrack21, 3DPW, BEDLAM, and MuPoTS-3D, and competitive reconstruction accuracy on BEDLAM and 3DPW, while uniquely supporting prompt-based tracking of individuals in multi-person scenes. To our knowledge, this is the first method to unify promptability and multi-person HMR with tracking in an end-to-end trainable framework, enabling user-directed human analysis in videos.
[CV-50] Subtoken Vision Transformer for Fine-grained Recognition
链接: https://arxiv.org/abs/2607.09086
作者: Jie Zhu,Ivy Zhang,Minchul Kim,Xiaoming Liu
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:We present Subtoken Vision Transformer (SubViT), a selective image tokenization method for fine-grained visual recognition. Standard Vision Transformers compress each fixed-size patch into a single token, although fine-grained distinctions often depend on localized variations within only a few patches. SubViT addresses this mismatch by representing discriminative patches with multiple subtokens while retaining the original token sequence for global context, thereby allocating additional capacity where it is most needed. Since attention heads encode complementary semantics and extracting attention maps at inference requires an extra backbone forward, we adopt a two-stage training strategy. Stage 1 fine-tunes the ViT using subdivision regions sampled from random attention heads, exposing the model to diverse subdivision patterns. Stage 2 identifies informative attention maps through feature-degradation distances and distills them into a lightweight single-map router, which directly predicts deterministic token-importance scores without a separate attention forward. We evaluate SubViT on Generalized Category Discovery (GCD), a challenging task requiring both fine-grained discrimination and generalization to unlabeled novel categories. Across CUB, FGVC-Aircraft, and Stanford Cars, SubViT improves the average novel-category accuracy of DINOv2 from 81.3% to 84.7% , with only 0.50 ms additional latency and 3.4% more FLOPs, while reducing latency by 73.8% relative to Retina Patch. Results on CIFAR-10 and ImageNet-100 demonstrate its broader applicability.
[CV-51] REBASE: Reference-Background Subspace Elimination for Training-Free In-Context Segmentation
链接: https://arxiv.org/abs/2607.09082
作者: Mantha Sai Gopal,Jaison Saji Chacko,Harsh Nandwana,Sandesh Hegde,Debarshi Banerjee,Uma Mahesh
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Training-free in-context segmentation enables new object categories to be introduced at inference time from a single annotated reference image, eliminating the retraining and memory overhead of class-incremental learning. Recent approaches achieve this by combining vision foundation models for semantic correspondence with promptable segmentation networks like SAM. However, their performance is fundamentally limited by the quality of the cross-image similarity map; shared contextual backgrounds between the reference and query systematically elevate similarity in non-target regions, degrading prompt localization. We present REBASE, a training-free framework that explicitly suppresses these spurious contextual correspondences. Our method identifies the low-rank background feature subspace from the reference image and project the reference and query features onto its orthogonal complement in closed form, yielding cleaner semantic matching. We then generate positive point prompts using similarity-weighted farthest-point sampling, paired with a refined dense similarity prior. Without any training or parameter updates, our approach establishes a new state of the art among training-free methods on PACO-Part, FSS-1000, and cross-domain datasets such as ISIC2018, demonstrating that explicit background subspace removal is a highly effective principle for one-shot localization.
[CV-52] Adaptive Latent Trajectory Anchoring for Action Segmentation Dataset Condensation ECCV2026
链接: https://arxiv.org/abs/2607.09081
作者: Artheme Gauthier-Villar,Guodong Ding,Angela Yao
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 16 pages, 5 figures, accepted to ECCV 2026
Abstract:Dataset condensation for action segmentation synthesizes compact, informative representations of long, untrimmed video datasets. The existing approach relies on Variational Autoencoders and an iterative latent optimization; it is computationally expensive and suffers from over-smoothed reconstructions and rigid temporal constraints. This paper proposes to shift the condensation paradigm from optimization-based inversion to deterministic latent mapping. By leveraging Denoising Diffusion Implicit Models, we represent action segments as continuous trajectories anchored by sparse latent points in the noise manifold. To maximize representational efficiency, we introduce an adaptive allocation mechanism that dynamically redistributes the anchoring budget based on segment-wise reconstruction difficulty. Extensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods in segmentation performance across common datasets. Notably, our approach achieves performance parity with real data training while maintaining a condensation ratio of 2.4% on Breakfast dataset.
[CV-53] GeoTrace: Geometry-Aware Trajectory Token Compression for Video Large Language Models
链接: https://arxiv.org/abs/2607.09080
作者: Guohuan Xie,Mengqi Lei,Chuan Shi,Wei Bao,Yue Gao,Siqi Li
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Although Video Large Language Models (Video LLMs) have shown strong performance in video understanding, their efficiency is still limited by the large number of visual tokens. Existing video token compression methods typically rely on frame-wise saliency or heuristic token merging, which can over-focus on locally salient regions and produce ambiguous fused features. To address these issues, we propose GeoTrace, a training-free spatiotemporal token compression framework that decomposes video evidence into exact skeleton tokens and traceable residual event tokens. Specifically, Contextual Farthest-Point Anchoring (CFPA) preserves salient, context-consistent, and high-coverage skeleton tokens, while Trajectory-Constrained Residual Condensation (TCRC) compresses residual tokens through one-to-one temporal trajectories and constrained near-manifold condensation, producing traceable event tokens with reduced ambiguity. We evaluate GeoTrace on four Video LLMs across four video understanding benchmarks, and the results demonstrate its effectiveness and generalization across different model architectures and scenarios. On LLaVA-OneVision, with only 10% visual tokens retained, GeoTrace achieves a (12.99\times) TFLOPs reduction while preserving 99.1% of the vanilla performance. Overall, GeoTrace offers a compact and traceable token representation for efficient and robust Video LLM inference. Code is available at \hrefthis https URL\textttCode.
[CV-54] oward Active Object Detection for UAVs in the Wild: A Large-Scale Dataset Benchmark and Method
链接: https://arxiv.org/abs/2607.09078
作者: Tianpeng Liu,Xinhua Jiang,Li Liu,Qinmu Shen,Siwei Tang,Zhen Liu,Yongxiang Liu
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注: 18 pages, 19 figures, 5 tables
Abstract:Object detection is a fundamental component in numerous Unmanned Aerial Vehicle (UAV) applications, yet it has long been plagued by hindrances like occlusion or target pixel scarcity. Active Object Detection (AOD) provides a novel paradigm to address these challenges via active vision, while UAV-based AOD research remains scarce due to the lack of high-quality datasets and benchmarks for algorithm development and evaluation. To fill this gap, this paper presents ATRNet-LUDO, the first large-scale real-world dataset for UAV-Ground Active Object Detection (UGAOD). It contains 121,000 multi-view panoramic multi-target aerial images and 1.21 million local single-target slices, covering 10 vehicle targets across 40 scenarios. It enables the construction of diverse training and testing environments for UAV agent interaction and active observation policy learning. Based on this dataset, we establish a comprehensive evaluation benchmark for AOD policy learning methods. Most existing AOD policies rely on Deep Reinforcement Learning (DRL) but suffer from poor generalization. Evaluations on our benchmark reveal a significant generalization gap between training and testing performance, highlighting an urgent need for solutions. To this end, we leverage the Joint Embedding Predictive Architecture (JEPA) to construct a world model that enhances state representation learning, and propose AOD-JEPA by incorporating AOD-specific prior knowledge. Extensive experiments validate its effectiveness and superiority. We hope ATRNet-LUDO and the benchmark will advance research in the UGAOD field. The dataset and code are soon available at this https URL.
[CV-55] OmniMapBench: Benchmarking Visual-Centric Reasoning on Diverse Map Documents
链接: https://arxiv.org/abs/2607.09068
作者: Yang Chen,Yunwen Li,Yufan Shen,Minghao Liu,Tianyu Zheng,Bin Fu,Qunshu Lin,Zhi Yu,Botian Shi
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Recent advancements in LVLMs necessitate robust benchmarks for complex, visually grounded reasoning. A critical limitation is identified in many document understanding benchmarks: visual content is often reducible to text, enabling high performance without genuine visual grounding. To address this limitation, OmniMapBench is introduced to foster visual-centric reasoning for map documents. The benchmark comprises 2,096 manually annotated question-answer pairs across 1,603 map documents from nine categories. It is designed to probe a hierarchy of skills, ranging from perception to multi-step visual reasoning. To quantify benchmark properties, a simple yet effective benchmark-level metric is proposed: the Visual Dependency Index (VDI), defined as the accuracy drop when images are replaced with question-agnostic descriptions. OmniMapBench exhibits higher VDI than established benchmarks, which quantitatively validates its focus on irreducible visual reasoning. Comprehensive evaluations of 25 leading LVLMs are conducted on OmniMapBench. A significant performance gap is observed, with the top-performing model achieving only 75.03% accuracy. This result underscores the challenges posed by OmniMapBench to current LVLMs. This work aims to catalyze progress in visual-centric reasoning for document understanding of LVLMs. The dataset and code are publicly available at this https URL.
[CV-56] Probing Diffusion Denoising Dynamics for Contrastive Representation Learning
链接: https://arxiv.org/abs/2607.09067
作者: Yasong Dai,Zeeshan Hayder,David Ahmedt-Aristizabal,Hongdong Li
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Text-to-image diffusion models exhibit unprecedented generative capability and contain rich intermediate representations that can be useful for discriminative vision tasks. Motivated by this observation, we study a focused question: how can the denoising dynamics of a pretrained diffusion model be adapted to support discriminative representation learning while preserving its generative behavior under parameter-efficient updates? We present D ^3 CL as an investigation of this question. Our key observation is that noisy latents at different diffusion timesteps can be interpreted as stochastic views of the same underlying image, enabling a contrastive objective to be coupled with the standard denoising reconstruction loss. This formulation provides a simple way to probe the interaction between generative denoising and discriminative representation learning without training from scratch. To keep the adaptation lightweight, we apply LoRA updates to a pretrained Stable Diffusion backbone while freezing the original model parameters. D ^3 CL provides strong empirical evidence that reconstruction and noise-level contrastive objectives can be complementary: on ImageNet-1K, it obtains 80.1% linear-probing accuracy and an FID of 5.56 for 256 \times 256 unconditional generation. Additional ablations on the design space suggest that the usefulness of diffusion features depends on where and how denoising states are sampled. These results establish D ^3 CL as a parameter-efficient adaptation framework for pretrained diffusion models, showing that noise-level contrastive learning can structure denoising representations for discriminative tasks while maintaining generative performance.
[CV-57] On Locality and Length Generalization in Visual Reasoning ECCV2026
链接: https://arxiv.org/abs/2607.09061
作者: Pulkit Madan,Sanjay Haresh,Reza Ebrahimi,Sunny Panchal,Apratim Bhattacharyya,Roland Memisevic
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Accepted at ECCV 2026
Abstract:A striking feature of the human visual system is that it ingests visual information through a series of local foveated glimpses, rather than a single global computation. This makes human vision distinctly different from most popular computer vision models in use today, which input images globally and in a single shot. A natural question therefore is whether local, sequential vision models may provide any fundamental computational benefits in addition to being biologically more plausible than global models. In this work, we investigate this question from the perspective of visual state tracking and length generalization. Inspired by recent studies of length generalization in language models, we study the behavior of vision models trained on simple vision tasks that require the aggregation of local information across an image. Our experiments reveal that, similar to language models, vision models can learn to exploit global shortcuts and thereby fail to generalize over task length or complexity. We also show that recurrent vision policies based on strictly local perception can mitigate these failures, thereby allowing models to generalize on these tasks. Our results show that local attention may be an essential overlooked requirement for robust compositional generalization.
[CV-58] STEAM: Stable Self-Training with Elastic Matching and Adaptive Purification
链接: https://arxiv.org/abs/2607.09057
作者: Shaoxiang Wang,Kejia Zhang,Haiwei Pan,Lan Zhang
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Cross-view geo-localization (CVGL) aims to achieve GPS-free localization by matching drone-view images with corresponding satellite-view images. Existing supervised methods rely on large-scale manually annotated cross-view image pairs, making them costly and difficult to scale. In contrast, existing unsupervised approaches typically depend on generative models or clustering-based stage-wise optimization, which are prone to distribution bias and the accumulation of noisy pseudo-labels. To address these limitations, we propose STEAM (Stable Self-Training with Elastic Matching and Adaptive Purification), an end-to-end unsupervised cross-view geo-localization framework that performs self-training directly on real drone and satellite images. Specifically, the proposed Stable Spatial-Aware Module enhances the stability of feature representations, Elastic Matching discovers high-quality cross-view pseudo-labels, and Adaptive Purification dynamically maintains a reliable pseudo-label repository throughout the self-training process. Extensive experiments on the University-1652 and SUES-200 benchmarks demonstrate that STEAM achieves state-of-the-art performance among all existing unsupervised methods and delivers performance comparable to supervised approaches, validating the effectiveness and superiority of the proposed framework. The source code is available at this https URL.
[CV-59] MOSAIC: Adaptive Inter-layer Composition for Efficient Heterogeneous Vision-Language Models
链接: https://arxiv.org/abs/2607.09029
作者: Yuncheng Yang,Feiyang Ye,Shixian Luo,Yinna Zhu,Lianlei Shan,Wangcai Zhao,Kuo Zhang,Yan Chen,Yong Wu,Yan Xie
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 17 pages, 7 figures
Abstract:Vision-Language Models (VLMs) have achieved success using homogeneous Transformers to process multimedia data. Recent studies show that heterogeneous structures interleaving efficient mechanisms, like linear attention, improve both performance and inference latency over homogeneous designs. However, these efforts rely on handcrafted static mixing patterns, which are sub-optimal and difficult to adapt to specific hardware. To bridge this gap, we propose Multi-Objective Search for Adaptive Inter-layer Composition (MOSAIC), a hardware-aware search method that automatically transforms homogeneous models into optimized heterogeneous architectures. MOSAIC integrates diverse efficiency mechanisms–including linear, sparse, and low-rank operators–into a unified search space. By formulating the selection as a multi-objective Mixed Integer Programming (MIP) problem, our method identifies optimal configurations that maximize downstream performance under strict hardware latency constraints. To mitigate performance degradation from structural transitions, we introduce a two-stage parameter recovery process: global off-policy distillation to stabilize internal representations, followed by a dual-teacher on-policy distillation leveraging a 235B oracle for knowledge expansion and the original 4B teacher for distributional stability. We validate MOSAIC through MOSAIC-4B, derived from Qwen3-VL-4B-Instruct. Results demonstrate that MOSAIC-4B matches the baseline’s performance across multiple benchmarks while requiring less than 2% of the original training cost. Furthermore, it substantially improves inference efficiency, achieving 1.76x prefilling and 2.54x decoding speedups.
[CV-60] Video Generation Models are General-Purpose Vision Learners ECCV2026
链接: https://arxiv.org/abs/2607.09024
作者: Letian Wang,Chuhan Zhang,Rishabh Kabra,Jasper Uijlings,Steven Waslander,Andrew Zisserman,Joao Carreira,Kaiming He,Misha Andriluka,Eduard Gabriel Bazavan,Andrei Zanfir,Cristian Sminchisescu
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: ECCV 2026
Abstract:Driven by next-token prediction, NLP shifted from task-specific models into powerful generalist foundation models. What, then, is the equivalent catalyst needed to achieve a general-purpose model in computer vision? In this paper, we contend that large-scale text-to-video generation serves as a strong pre-training paradigm for computer vision, providing the necessary spatiotemporal priors, vision-language alignment, and scalability required for general visual intelligence. We introduce GenCeption, which leverages a pre-trained video generative diffusion backbone to define a feed-forward perception model, capable of performing various vision tasks steered by text instructions. Empirical results demonstrate that GenCeption achieves state-of-the-art performance across a diverse suite of tasks, including depth, surface normal, and camera pose estimation, expression-referring segmentation, and 3D keypoint prediction, often matching or surpassing specialized models (e.g. DepthAnything3, SAM3, D4RT, VGGT-Omega, Sapiens, David, Genmo, and Lotus-2). Furthermore, the video generative pretrained backbone outperforms alternative pretraining paradigms (e.g., V-JEPA, and Video MAE) under comparable settings. Importantly, GenCeption exhibits preliminary data and model scaling properties along with exceptional data efficiency, where it achieves comparable performance with leading models like D4RT and VGGT-Omega with 7 to 500 less training data. Finally, GenCeption also exhibits intriguing emergent behaviors: a model trained exclusively on synthetic human videos generalizes to real-world footage and out-of-distribution object categories (e.g., animals and robots). These findings suggest that video generation is not merely a synthesis tool, but a foundational path toward generalist vision intelligence for the physical world. Project page: this https URL
[CV-61] C-GAP: Class-Aware and Online Prompting Improves Vision-Language Models on Imbalanced Classes
链接: https://arxiv.org/abs/2607.09008
作者: Francis Fernandez,Arash Jahangiri,Salimeh Sekeh
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Safety-critical perception systems must reliably detect rare object classes within small label spaces, a setting that long-tailed detection methods, designed for hundreds of classes with dense annotation, fundamentally do not address. Open-vocabulary detectors offer a promising alternative, as they use natural language queries at inference time, making prompt quality a first-class lever for detection performance. We exploit this property to address class imbalance: rather than retraining models or collecting additional annotations, we ask whether iteratively refining the language prompts, fed to frozen detectors, can improve minority class detection. We introduce C-GAP Caption-Guided Augmentation and Prompting), a detector-agnostic, annotation-free framework that operates in two phases. First, we establish a composite caption baseline combining per-image scene descriptions with class-quantity context, which we show outperforms scene-description only or class-quantity-only prompts across multiple open-vocabulary architectures and benchmarks. Second, an LLM iteratively refines each image’s caption individually, with trials triaged into accept, tentative, or regenerate buckets based on minority-class AP@0.5 against a dynamic threshold derived from the composite baseline. Refinement terminates early once sufficient AP@0.5 gain is achieved. No detector weights are updated at any stage. Our experiments shows that C-GAP improves minority-class average precision up to 53% over the baselines. On COCO, C-GAP improves minority-class AP@0.5 by ~81% relative over the composite baseline (17.69 - 32.09). Experiments confirm that composite captions provide the critical foundation for effective refinement: using scene-description-only or class-quantity-only prompts as the refinement starting point yields diminishing returns, supporting both stages of C-GAP as necessary contributions.
[CV-62] MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs
链接: https://arxiv.org/abs/2607.08970
作者: Hantao Zhang,Jinru Sui,Ed Li,Dirk Bergemann,Zhuoran Yang
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Recent benchmarks for VLMs largely assess single- or limited-view perception, leaving untested the core cognitive ability to integrate observations across viewpoints into a coherent, world-centric (allocentric) 3D mental model. We introduce MultiView-Bench, a diagnostic benchmark expressly designed to evaluate multi-view integration for holistic 3D scene comprehension. Unlike existing datasets that focus on pixel-level mapping or camera-relative navigation, MultiView-Bench requires models to decouple object positioning from transient perspectives and ground them in a fixed global coordinate system. This capability serves as a prerequisite for VLMs before being deployed for downstream tasks such as mechanical part assembly. Our systematic evaluation of frontier VLMs reveals consistent failure modes: strong performance on 2D planar relations from a single image, but marked difficulty with 3D spatial relations and with aggregating information across views. We further identify biases in VLMs, such as struggles with unconventional axis directions and sensitivity to object colorways and texture variations. Acknowledging these limitations, we propose ViewNavigator, a multi-agent framework that actively selects informative viewpoints, perceives, and fuses multi-view evidence, improving diverse base models on MultiView-Bench even under a strict budget-matched comparison (and by 3-5x for the full agent).
[CV-63] SplatCtrl: Perception-Action Coupling via Gaussian Scene Representations and Reactive Robot Control ICRA
链接: https://arxiv.org/abs/2607.08948
作者: Siddarth Jain,Ho Jin Choi
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注: Published in 2026 International Conference on Robotics and Automation (ICRA). 8 pages, 8 figures
Abstract:Robotic manipulators excel in structured environments but face substantial challenges in unstructured and dynamic settings. This paper presents SplatCtrl, a unified framework for real-time scene reconstruction and reactive robot motion generation to enable collision-free robotic arm control in previously unseen and continuously changing environments. Building on 3D Gaussian Splatting (3D-GS), we introduce a hybrid voxel-based filtering and dynamic Gaussian relocation strategy that supports efficient scene reconstruction from RGB-D streams while accommodating environmental changes. For safe and reactive control, we further propose a method for deriving continuous signed distance functions from isotropic Gaussians, providing stable and differentiable collision probability estimates that bridge classical distance fields with the modern implicit representation. These continuous distance metrics are incorporated into control barrier functions, resulting in a unified perception-action coupling framework that supports smooth and reliable real-time motion generation in response to scene changes. Experimental validation in simulation, on physical robot, and within shared human-robot workspace demonstrates the framework’s effectiveness, achieving integrated scene reconstruction and reactive control in uncertain, and dynamic environments.
[CV-64] Is sub-metre resolution necessary for cocoa mapping? A landscape-stratified evaluation of very high resolution imagery decametric Earth Observation inputs and operational products in Cote dIvoire
链接: https://arxiv.org/abs/2607.08945
作者: Kasimir Orlowski,Filip Sabo,Michele Meroni,Astrid Verhegghen,Mariana Belgiu,Felix Rembold
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Accurate cocoa mapping is increasingly important for deforestation monitoring, supply-chain transparency, and regulatory applications. Spatial aggregation in conventional medium-resolution Earth observation (EO) imagery may limit cocoa detection in heterogeneous smallholder landscapes. In Cote d’Ivoire, we therefore evaluated how mapping performance varies across landscape conditions, whether very high resolution (VHR) imagery provides a meaningful advantage, and whether foundation-model embeddings improve decametric cocoa mapping. We developed models using 0.5 m Pleiades VHR imagery, a 10 m Sentinel-2 annual composite, and embeddings from TESSERA and AlphaEarth Foundations (AEF), and additionally assessed four publicly available cocoa mapping products. Performance was evaluated through a landscape-stratified accuracy assessment using 2,821 independently interpreted reference points distributed across gradients of tree cover density and landscape fragmentation. The VHR model achieved the highest performance (F1 = 0.92) and maintained F1-scores above 0.90 across all strata. Among the decametric inputs, TESSERA performed best (F1 = 0.86), followed by AEF (F1 = 0.82) and Sentinel-2 (F1 = 0.76). Of the existing cocoa products, the Kalischek product performed best (F1 = 0.83), comparable to the internally trained AEF model. Performance differences between VHR and decametric approaches increased with fragmentation and under low and high tree cover density conditions. Targeted VHR acquisition may therefore be particularly beneficial in complex cocoa landscapes, while foundation-model embeddings offer a scalable alternative for large-area mapping.
[CV-65] Vision Transformers Learn Gestalt-Like Figure-Ground Cues from Natural Images
链接: https://arxiv.org/abs/2607.08932
作者: Matthias Tangemann,Benjamin Lo,Zygmunt Pizlo,Kaleem Siddiqi,Dirk B. Walther,Sven Dickinson
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Figure-ground organization in the human visual system relies on several shape-based cues, including surroundedness, convexity, and symmetry. While these cues have been extensively studied using abstract stimuli, little is known about how they operate under natural conditions or how they arise from the statistics of natural scenes. Deep neural networks offer a promising path forward: a model that relies on the same figure-ground cues as humans would provide tractable experimental access to the underlying mechanisms. In this study, we evaluate shape-based figure-ground organization in Vision Transformers (ViTs), for which prior work has demonstrated the emergence of object-based grouping. We test 25 ViTs spanning supervised and self-supervised training objectives, by fitting linear probes to predict figure-ground assignment from intermediate patch representations using both natural images and controlled artificial stimuli that isolate individual cues. Our results show that ViTs robustly encode surroundedness and convexity, and that probes trained on natural images generalize zero-shot to artificial stimuli across several models. For symmetry we observe mixed results: the cue is encoded for uniformly colored but not for textured regions. Taken together, our findings demonstrate that Gestalt-like figure-ground cues can be learned from natural scene statistics and position ViTs as a compelling model system for studying the computational mechanisms of perceptual organization. Code and data is available at this https URL Subjects: Computer Vision and Pattern Recognition (cs.CV) Cite as: arXiv:2607.08932 [cs.CV] (or arXiv:2607.08932v1 [cs.CV] for this version) https://doi.org/10.48550/arXiv.2607.08932 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[CV-66] HAT Super-Resolution and a PARSeqCLIP4STR Voting Ensemble for Extreme In-the-Wild License Plate Recognition ICIP2026
链接: https://arxiv.org/abs/2607.08896
作者: Karthik Sivarama Krishnan,Koushik Sivarama Krishnan
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 2 pages, 1 figure, 1 table. Accepted at the IEEE ICIP 2026 Grand Challenge on Extreme In-the-Wild License Plate Super-Resolution (XLPSR). Top-8 finalist
Abstract:We describe our entry to the ICIP 2026 Grand Challenge on Extreme In-the-Wild License Plate Super-Resolution (XLPSR), which scored 9.73 wECR on the public validation leaderboard. The system pairs a Hybrid Attention Transformer super-resolution (HAT) front-end with an ensemble of two scene-text recognisers (PARSeq-S and CLIP4STR-B) and a confidence-weighted character-voting scheme that abstains on uncertain positions. We treat XLPSR as a recognition task gated by image legibility: the SR step exists to lift characters out of sub-pixel territory, and the asymmetric scoring rule (+2 / -1 / 0) is exploited explicitly through abstention. Our pipeline runs in 1.7 s per sequence on RTX 3090 (max 2.7 s, p99 2.4 s), well under the 60 s/sequence Docker budget.
[CV-67] Decoupled Illumination Priors for Spatially Controllable Multi-View Indoor Scene Relighting
链接: https://arxiv.org/abs/2607.08879
作者: Chenjian Gao,Linning Xu,Tianfan Xue
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Indoor scene relighting demands photorealism, precise spatial control, and strict multi-view consistency. While diffusion-based image editing models enable semantic lighting manipulation via text prompts, enforcing exact 3D light placement often disrupts their generative priors. We propose Lume-Palette, a progressive framework that leverages semantic lighting priors for spatially controllable multi-view indoor relighting. The approach decouples relighting into two stages: (1) illumination distillation, which extracts canonical illumination palettes from a pretrained diffusion model to preserve realistic material-light interactions, and (2) illumination casting, which explicitly maps target spatial lighting conditions defined from coarse 3D geometry. To efficiently handle dense multi-view and multi-modal inputs, we introduce an asymmetric multi-view conditioning strategy that selectively injects essential spatial context. Experiments on diverse synthetic scenes and real-world scenes demonstrate that Lume-Palette produces photorealistic, spatially controllable, and multi-view consistent relighting results. Project Page: this https URL
[CV-68] Secure-by-Disguise: A Systematic Evaluation of Image Disguising for Confidential Medical Image Modeling
链接: https://arxiv.org/abs/2607.08867
作者: Jason Rojas,Jiajie He,Yash Patel,Yuechun Gu,Zeyun Yu,Keke Chen
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Cloud-based deep learning enables large-scale medical image analysis but raises significant privacy concerns when sensitive patient images are outsourced for model development. Image disguising has recently emerged as a promising privacy-enhancing technology (PET) that transforms images into visually unintelligible representations while preserving information for downstream learning. We established a unified framework to evaluate representative methods, DisguisedNets and NeuraCrypt, across four datasets involving classification and semantic segmentation tasks. Our analysis assessed predictive utility, efficiency, and robustness against reconstruction attacks. Results showed that image disguising performance varies significantly between tasks; while methods preserved utility for medical image classification, they caused substantial degradation in dense semantic segmentation. Specifically, Randomized Multidimensional Transformation (RMT) offered the optimal balance of performance and security, whereas AES-based disguising severely impacted utility. Furthermore, regression-based reconstruction attacks effective on natural images proved considerably less successful on realistic medical images. These findings provide a systematic assessment of PET suitability for confidential medical AI applications.
[CV-69] Mixture of Probes: Learning from Privileged Modalities in Multimodal LLM s Through Probing
链接: https://arxiv.org/abs/2607.08839
作者: Dominick Reilly,Qiyu Wu,Hiromi Wakaki,Srijan Das,Yuki Mistufuji
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: Preprint (16 pages)
Abstract:Multimodal Large Language Models (MLLMs) are typically designed under the assumption that all modalities available during training will also be accessible at inference. However, many real-world settings violate this assumption, requiring models to operate under a privileged modality setting, where auxiliary modalities are available only during training. While these modalities contain valuable information, existing MLLMs largely fail to leverage them effectively, as they treat modalities as interchangeable inputs rather than sources of complementary supervision. We propose Mixture of Probes (MoP), a novel framework that disentangles modality-specific and modality-general signals within the MLLM, allowing the model to preserve modality-dependent structure while learning transferable representations across modalities. At its core, MoP achieves this through a structured probing mechanism that extracts and organizes information from intermediate representations of a shared modality encoder, rather than relying only on final-layer alignment as done in existing MLLMs. To support this disentanglement, we further introduce MoP Cross-modal Training (MoP-X), a training strategy for MoP centered around a probe disentanglement loss that prevents probe collapse and encourages cross-modal learning. We evaluate MoP across two domains spanning eight tasks and four modalities under a comprehensive evaluation protocol tailored to the privileged modality setting, where each modality is independently treated as the sole input at inference time. MoP consistently outperforms strong MLLM baselines, achieving up to 65% relative improvement, demonstrating that auxiliary modalities, even when unavailable at inference, can provide substantial gains when effectively leveraged during training. Code, model checkpoints, and evaluation protocols will be made available at this https URL.
[CV-70] StereoSplat: Feed-Forward Stereo Gaussian Splatting with Diffusion-Assisted Progressive Inference IROS2026
链接: https://arxiv.org/abs/2607.08808
作者: Zihua Liu,Masatoshi Okutomi
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 8 pages, accepted as a conference paper for IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2026)
Abstract:Recent advances in 3D Gaussian Splatting (3DGS) have enabled high-quality, render-ready scene representations for novel-view synthesis. However, most existing 3DGS pipelines rely on multi-view observations (or non-causal access to future frames) to achieve sufficient coverage, which is often unavailable in on-device robotics and AR settings where sensing is restricted to a single stereo rig. Recovering a high-quality 3DGS scene from one stereo observation, therefore, remains challenging due to occlusions, limited field of view, and missing geometry. We present StereoSplat+, a diffusion-enhanced feed-forward framework that enables causal reconstruction from a single stereo pair. Our method builds on two key components. First, we propose StereoSplat, an input-invariant feed-forward 3D Gaussian estimator that takes a variable number of posed stereo pairs as input and predicts high-quality 3D Gaussians. StereoSplat fuses complementary geometry cues via a cost-volume branch and a triplane-based 3D volume branch and leverages continuous pose encoding to generalize across view counts and camera configurations. Second, since multiple posed stereo pairs are typically unavailable at inference time, we introduce a diffusion-enhanced one-shot progressive inference scheme called StereoSplat+: starting from one stereo pair, we render novel stereo views from the predicted 3DGS, refine them with a one-step diffusion enhancer, and feed them back as additional inputs to update the 3DGS. Experiments on the KITTI-360 dataset show that StereoSplat+ improves novel-view rendering quality and geometry accuracy, especially in occluded regions and under strong view extrapolation, outperforming recent feed-forward 3DGS baselines.
[CV-71] GReFEM: Multimodal LLM s as Zero-Shot Semantic Assistants for Physics-Guided 3D Mesh Refinement
链接: https://arxiv.org/abs/2607.08798
作者: Kartik Bali,Mahish K. Guru,Christian J Cyron,Roland Aydin
类目: Graphics (cs.GR); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Adaptive volumetric finite element meshing is a critical step in computer-aided engineering and analysis that dictates the computational budget of a given problem. It traditionally requires iterative PDE solvers or heavily supervised, data-driven surrogates trained on large-scale simulation data. While Multimodal Large Language Models (MLLMs) excel in 2D visual tasks, their zero-shot capability to semantically ground regions based on geometric understanding and physics remains an open question. Overall, this study explores a significant question: can the high-level semantic understanding of off-the-shelf MLLMs serve as a viable, zero-shot geometric proxy for finite element mesh refinement? To investigate this, we introduce GReFEM (Geometric Reasoning Enhanced Multimodal LLMs for Finite Element Meshing), a framework that utilizes MLLMs to visually localize stress-critical regions based on physics-guided textual prompts. To bridge the gap between 2D MLLM pre-training and 3D geometries, we introduce orthoViews, a view-selection module that maximizes the observability of key geometric features. We conduct an in-depth empirical evaluation across diverse CAD geometries, loading cases, and SOTA MLLMs, comparing them against a tuned geometric heuristic under a strict, matched refinement budget. Our findings reveal that MLLMs demonstrate robust zero-shot capacity to accurately follow complex spatial-physical instructions, isolating stress-relevant features with higher precision than blind heuristics. By mapping both the successes and current limitations of MLLMs in physical grounding, this study defines the frontier of foundation models as semantic assistants in automated simulation workflows.
[CV-72] Joint-Embedding Predictive Architecture for Solar PV Panel Fault Classification
链接: https://arxiv.org/abs/2607.09205
作者: Seyyedhamid Azimidokht,Mehdi Monemi,Abdelhak Kharbouch,Farid Hamzehaghdam,Mehdi Rasti,Jamshid Aghaei,Emil Kurvinen
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:The rapid expansion of solar photovoltaic (PV) systems has increased the need for reliable and scalable fault classification, as manual inspection is impractical at scale. Thermal infrared (IR) imaging provides a non-contact solution for identifying PV faults; however, accurate classification remains challenging due to class imbalance, limited texture information, and subtle thermal differences. In this work, we investigate the applicability of Joint-Embedding Predictive Architecture (JEPA) for thermal IR PV fault classification across various scenarios and propose JEFFNet (JEPA-EFFicientNet), a multibranch architecture that combines JEPA-based self-supervised representation learning with EfficientNetV2-S-based supervised convolutional feature extraction. JEFFNet fuses semantic representations from a JEPA-pretrained Vision Transformer with convolutional features from EfficientNetV2-S, enabling complementary feature learning. JEFFNet is evaluated on two public thermal IR datasets, PVF-10 and InfraredSolarModules (ISM), for both multiclass and derived binary (healthy/faulty) classification. On PVF-10, JEFFNet achieves an F1-score of 93.21 and an accuracy of 94.33 in the 10-class task, and an F1-score of 97.53 and an accuracy of 96.41 in the derived 2-class task. On ISM, JEFFNet achieves an F1-score of 72.60 and an accuracy of 83.88 in the 12-class task, and an F1-score of 94.69 and an accuracy of 94.78 in the derived 2-class task. JEFFNet also uses only 108.6M parameters versus 205.91M for GEPFNet, a 47.2% reduction. These results demonstrate that combining self-supervised semantic and supervised convolutional features provides an effective, parameter-efficient solution for thermal IR PV fault classification. The source code is publicly available at this https URL
[CV-73] Beyond Metadata: CAPRA for Hidden Subgroup Analysis under Missing Metadata in Medical Imaging
链接: https://arxiv.org/abs/2607.09102
作者: Yawen Li,Yan Li,Zhe Xue,Yingxia Shao,Meiyu Liang,Guanhua Ye
类目: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
备注:
Abstract:Medical imaging models are often deployed without the demographic, acquisition, and quality metadata needed for subgroup auditing. Once those metadata disappear, clinically critical failure modes can be masked by strong aggregate performance, and many robust-learning methods lose the group structure they rely on. We present CAPRA, a calibrated proxy-axis framework for hidden subgroup analysis under missing metadata. CAPRA predicts image-derived semantic axes, calibrates axis posteriors on a small metadata-labeled split via patient-level cross-fitting, and organizes those posteriors into a calibrated subgroup interface that supports both deployment-time failure analysis and downstream robust learning without requiring subgroup labels at deployment. Across fundus, dermoscopy, and chest radiography, CAPRA reveals disparity patterns missed by metadata-only slicing, remains informative under dataset shift, and produces subgroup partitions that align more closely with explicit failure axes than image-only or latent-slice baselines. The same interface can also be reused by downstream robust learners, although those gains are domain-dependent. Overall, CAPRA turns hidden subgroup analysis under missing metadata into a calibrated, interpretable, and reusable subgroup interface for deployment-time analysis and robust transfer.
人工智能
[AI-0] VEXAIoT: Autonomous IoT Vulnerability EXploitation using AI Agents
链接: https://arxiv.org/abs/2607.09653
作者: Katherine Swinea,Kshitiz Aryal,Lopamudra Praharaj,Maanak Gupta
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注:
Abstract:Internet of Things (IoT) systems are inherently vulnerable due to constrained hardware, outdated firmware, and insecure default configurations, creating a need for scalable and adaptive security testing approaches. While recent adoptions of Large Language Model (LLM) agents have demonstrated promise in penetration testing and Capture-the-Flag (CTF) environments, their application to IoT specific vulnerabilities remains unexplored. This paper presents an autonomous multi-agent framework, referred to as Vulnerability EXploitation using AI Agents (VEXAIoT), for vulnerability discovery and exploitation in IoT environments using LLM-based reasoning and offensive security tools. The framework combines a vulnerability detection agent and an attack execution agent to perform reconnaissance, plan attack sequences, and execute exploits against vulnerable IoT services. The system is evaluated in IoTGoat and Metasploitable environments across ten attack scenarios mapped to OWASP IoT vulnerabilities. Experimental results show attack success rate of up to 100% with low token overhead and average execution times under two minutes for most attacks. Across 260 attack executions, VEXAIoT achieves a 95.0% overall success rate, including 94.5% success in IoTGoat and 96.7% success in Metasploitable2. These results demonstrate the potential for LLM-driven agents to automate IoT vulnerability assessment and offensive security workflows in controlled environments
[AI-1] ConceptSMILE: Auditing the Trustworthiness of Concept-Based Explainable AI
链接: https://arxiv.org/abs/2607.09649
作者: Mohadeseh Mollapour,Koorosh Aslansefat,Zeinab Dehghani,Bhupesh Kumar Mishra,Tejal Shah,Zhibao Mian
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Concept-based explainable artificial intelligence (AI) can make model reasoning more human-understandable, but concept-level outputs are not automatically trustworthy. We introduce ConceptSMILE, a model-agnostic perturbation-based auditing framework for evaluating the reliability of concept-based explanations. Rather than replacing SMILE, ConceptSMILE extends its perturbation-based logic from feature- or region-level attribution to the auditing of human-understandable concept explanations. The framework perturbs input regions, measures concept-response shifts, applies locality weighting, and fits an XGBoost surrogate to approximate local concept behaviour. Reliability is assessed through attribution accuracy, surrogate fidelity, faithfulness, stability, and consistency. We evaluate ConceptSMILE on retinal fundus images by comparing MedSAM-derived visual concepts with VLM-based semantic concepts. Results show that reliability varies across concepts and pathways: MedSAM achieves stronger spatial attribution and the highest surrogate fidelity ( R^2 = 0.8503 , R_w^2 = 0.8465 ), while the VLM pathway shows stronger vessel faithfulness and stronger stability under selected artefact conditions. ConceptSMILE provides an independent audit layer for evaluating the trustworthiness of concept-based XAI.
[AI-2] Semantic Pareto-DQN: A Multi-Objective Reinforcement Learning Framework for Financial Anomaly Detection
链接: https://arxiv.org/abs/2607.09641
作者: Cláudio Lúcio do Val Lopes,Lucca Machado da Silva
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: BRACIS 2026 - 36th Brazilian Conference on Intelligent Systems
Abstract:Financial anomaly detection suffers from extreme class imbalance, causing traditional single-objective algorithms to exhibit ``fraud collapse’', defaulting to the majority class and failing to balance anomaly interdiction with customer friction. To overcome this without distortive data resampling, we propose the Semantic Pareto-DQN, a multi-objective reinforcement learning framework. Our approach synthesizes heterogeneous transaction features into cohesive natural-language narratives, encoded by large language models, thereby producing a robust, scale-invariant state representation. The agent optimizes a vectorial reward that explicitly decouples financial efficacy, operational friction, and semantic discovery. By mapping the continuous Pareto frontier, the system dynamically navigates the asymmetric costs of missed anomalies versus false positives. Empirical evaluations across E-Commerce fraud and UCI Credit datasets show that semantic Pareto-DQN successfully shatters the zero-recall trap. It achieves superior minority-class recall compared to scalarized baselines, providing an alternative to trade bounded operational friction for financial anomaly discovery.
[AI-3] PAC-ACT: Post-training Actor-Critic for Action Chunking Transformers
链接: https://arxiv.org/abs/2607.09590
作者: Yujie Pang,Zudong Li
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI)
备注:
Abstract:Precision industrial contact manipulation requires reliable robot policies under pose perturbations and contact-force constraints. Vision-language-action models offer broad generalization but often introduce high inference latency and GPU-memory cost, while vision-action chunking policies are more suitable for real-time industrial control. However, these policies are usually trained by behavior cloning and suffer from distribution shift in contact-rich tasks. This paper proposes PAC-ACT, a reinforcement-learning post-training framework for pretrained Action Chunking Transformer policies. PAC-ACT reformulates policy optimization at the chunk level, constructs an ACT-transferred actor-critic architecture, and introduces a hybrid behavior-prior constraint to preserve the pretrained action distribution during online fine-tuning. Experiments on industrial precision-contact benchmarks show that PAC-ACT improves task success, contact stability, and force safety while retaining low latency and low GPU-memory usage. On the Contour task, PAC-ACT significantly reduces peak contact force and decreases the proportion of force readings above 60 N by 46 times. Sparse-reward ablations further show that the proposed behavior-prior constraint enables effective exploration under randomized initial poses.
[AI-4] rustX Agent Risk Classification Framework (ARC): Risk-Tiering Internally Created Agent ic AI Systems
链接: https://arxiv.org/abs/2607.09586
作者: Hannah M. Liu,Rhea Saxena,Shiv Asthana
类目: Artificial Intelligence (cs.AI)
备注: This is a working paper on our risk classification tool, with iterations currently underway
Abstract:The proliferation of agentic AI systems across enterprise and public-sector contexts has outpaced the capacity of general-purpose AI risk frameworks to classify and govern them. In this paper, we introduce the TrustX Agent Risk Classification Framework, a structured, repeatable instrument that can be applied to seven types of agentic AI systems and is grounded in foundational pre-existing AI governance frameworks. At the core of the framework is a twelve-dimension scoring rubric that robustly quantifies the risk. This rubric is combined with other components, such as the GPA + IAT classification model and the five-level autonomy framework derived from existing literature. These inputs produce a three-tier governance output with mapped control recommendations. A specialised Coding Assistant extension is also included to account for nuances specific to this type of agentic AI system. We then use an illustrative example to show our framework in practice. ARC is intended for AI governance practitioners, risk officers, developers, and regulators, and it will regularly undergo iteration as we continue to expand it and make it more robust. The community can access the interactive framework here: this https URL
[AI-5] Knowledge Graphs and Explainable AI as Complementary Resources for Urban Mining IJCAI ECAI2026
链接: https://arxiv.org/abs/2607.09578
作者: Jan Gronewald,Andreas Emrich,Nijat Mehdiyev
类目: Artificial Intelligence (cs.AI)
备注: Accepted for presentation at the AISE Workshop @ IJCAI-ECAI 2026
Abstract:Pre-demolition assessment, the regulated audit process at the heart of urban mining, is an information process in which AI support must serve qualified auditors who remain accountable for the decisions taken. The relevant unit of value is not prediction accuracy alone, but the defensibility of the supported decisions: their legibility, plausibility, sourcing, and contestability. Explainable AI techniques and domain knowledge graphs each address parts of this requirement, and existing taxonomies have catalogued their integration. The literature is descriptively rich but structurally under-specified: what remains less developed is a structural account of why specific integrations produce artefacts neither resource can provide alone. This paper offers a complementarity-theoretic interpretation grounded in the IS resource-based tradition. We propose four consolidated KG-XAI integration modes (Lifting, Constraining, Typing, and Revising), each defined as a typed operation over XAI artefacts and knowledge-graph substrate structures. Each mode unlocks a distinct property of defensibility and contributes to the kind of regulatory artefact pre-demolition assessment demands. A fire-door example from the urban-mining process illustrates the modes using the W3C Linked Building Data stack and valuation extensions.
[AI-6] Large-Scale Portfolio Optimization Problem Under Cardinality Constraint With Enhanced Multi-Objective Evolutionary Algorithms
链接: https://arxiv.org/abs/2607.09566
作者: Danial Ramezani,Mostafa Abouei Ardakan
类目: Computational Engineering, Finance, and Science (cs.CE); Artificial Intelligence (cs.AI); Optimization and Control (math.OC); Portfolio Management (q-fin.PM); Risk Management (q-fin.RM)
备注:
Abstract:Decision-making is posing an increasingly formidable challenge to investors because of the growing number of alternatives available in financial markets. A hot area of research over the past few decades has been portfolio optimization that seeks to determine how much an investor should invest in which asset. Introducing real-world conditions to the optimization model turns the problem into an NP-hard one for whose solution exact methods become inefficient; hence, researchers have turned to evolutionary algorithms to approximate solutions. In this paper, strengthening strategies are presented for multi-objective evolutionary algorithms that can provide a faster convergence rate and extensive search ability in the portfolio optimization problem under the cardinality constraint. To implement those features, a unique solution representation, a novel operator, and new repair mechanisms are introduced for solving the aforementioned problem in which lower and upper limits are set on the number of assets in the portfolio. For this purpose, new mating strategies along with the aforesaid package are implemented in well-known multi-objective evolutionary algorithms to solve the problem. The customized algorithms are subsequently tested against traditional ones using well-known market indices as benchmarks. Results indicate that the proposed strategy not only provides better approximations but also converges faster as well at no loss of performance with an increasing number of assets in the market.
[AI-7] Beyond Fixed Representations: The Vocabulary and Verifier Gaps in Open-Ended AI
链接: https://arxiv.org/abs/2607.09560
作者: Yuan Cao,Haiqian Yang
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Modern AI systems are increasingly being evaluated for their ability to reason, code, prove theorems, use tools, and long-horizon research tasks. These are powerful capabilities, but they share a structural limitation: the representational frame within which the model operates, including its conceptual vocabulary, the space of admissible solutions it can search, and the criteria by which success is evaluated, is typically fixed and supplied in advance. This paper argues that building stronger intelligent systems capable of open-ended innovation requires additional classes of operations: the creation, stabilization, and reuse of new representational primitives, which alter the space being searched rather than simply searching within it. We characterize the distance between current AI systems and genuinely open-ended intelligence through two gaps. The first is the vocabulary gap, the difficulty of inventing and stabilizing new representational primitives rather than merely recombining existing ones. The second is the verifier gap, the difficulty of judging the value of a new primitive when its full payoff may be visible only after future reuse. We interpret both gaps through a unified framework of intelligence as cognitive discrepancy reduction. By viewing intelligent behaviors as a sequence of cognitive transformations, we distinguish intra-space transformations which operate within a fixed representational frame, from generative transformations which may modify the frame itself. On this basis, we propose a ladder of innovation autonomy and outline several directions for advancing open-ended AI, including objectives that reward useful representational change, persistent memory architectures for invented primitives, and adaptive verification mechanisms capable of evolving alongside the representations they evaluate. Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG) Cite as: arXiv:2607.09560 [cs.AI] (or arXiv:2607.09560v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2607.09560 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-8] SAGEAgent : A Self-Evolving Agent for Cost-Aware Modality Acquisition in Multimodal Survival Prediction
链接: https://arxiv.org/abs/2607.09521
作者: Chongyu Qu,Can Cui,Zhengyi Lu,Junchao Zhu,Tianyuan Yao,Junlin Guo,Juming Xiong,Yanfan Zhu,Yuechen Yang,Bennett A. Landman,Yuankai Huo
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Does every cancer patient truly need a complete diagnostic workup for accurate survival prediction? In multimodal clinical oncology, diagnostic modalities follow a clinically mandated order of escalating burden – from demographics collected at intake to genomic profiling requiring specialized tissue analysis. Current multimodal survival methods either assume all modalities are available or passively handle missing data, but none actively reason about whether acquiring the next modality is justified for a given patient along this ordered workflow. We formulate this as a sequential decision problem and propose SAGEAgent (Sequential Acquisition Guided by Experience), a self-evolving LLM-based clinical agent that decides which diagnostic modalities to acquire for each patient, balancing predictive accuracy against clinical invasiveness. SAGEAgent reasons about each patient’s evolving diagnostic state through clinical tools that translate numerical predictions into text, an episodic memory that retrieves similar past cases, and a semantic memory that accumulates reusable decision patterns from experience. Experiments on a glioma cohort combining TCGA-LGG, TCGA-GBM, and BraTS with four diagnostic modalities demonstrate that SAGEAgent achieves competitive survival prediction accuracy while reducing average acquisition burden by 55%.
[AI-9] Failure as a Process: An Anatomy of CLI Coding Agent Trajectories
链接: https://arxiv.org/abs/2607.09510
作者: Xiangxin Zhao,Han Li,Shuaiting Li,Tianyi Zhao,Earl T. Barr,Federica Sarro,He Ye
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: 12 pages, 6 figures
Abstract:Large language model (LLM) coding agents are increasingly deployed to autonomously perform software engineering tasks in terminal-based environments, making their reliability a growing concern. Existing empirical studies investigate why coding agents fail, yet they largely treat failure as a final outcome rather than a temporal process, providing limited insight into how failures emerge, evolve, and become unrecoverable. We present the first large-scale empirical study of CLI coding-agent failure trajectories, introducing a process-oriented framework that analyzes failure through its onset, evolution, and recovery across execution trajectories. We first collect 3,843 execution trajectories generated by seven frontier models across three coding-agent scaffolds (OpenHands, MiniSWE, and Terminus2) on Terminal-Bench, then carefully filter them to obtain 1,794 complete and valid trajectories for manual annotation (over 63,000 execution steps), from which we derive 14 findings spanning failure occurrence, root causes, recovery, and cross-system consistency. Our findings show that coding-agent failures are predominantly driven by epistemic errors, typically begin within the first few execution steps, and often remain hidden until recovery is no longer possible, suggesting that improving coding-agent reliability requires earlier validation and intervention rather than relying solely on final-outcome evaluation.
[AI-10] Multimodal Reward Hacking in Reinforcement Learning
链接: https://arxiv.org/abs/2607.09492
作者: Jiayu Yao,Yiwei Wang,Anmeng Zhang,Zhe Sun,Songsong Wang,Lingrui Mei,Yuyao Ge,Shenghua Liu
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Reinforcement learning (RL) is increasingly used to align multimodal large language models (MLLMs), but higher rewards do not always imply better task performance. This risk is amplified when visual evidence is evaluated by text-only or weakly grounded rewards. We study reward hacking in MLLM RL across safety VQA, chart VQA, and stress-test settings, varying reward design, data ambiguity, model scale (2B-32B), and RL algorithm (GRPO, RLOO, DAPO). We introduce Newly Rewarded Failure Rate (NRFR), which measures failures among samples whose proxy reward improves over the SFT baseline. Outcome-only rewards cause severe hacking, reaching 48.1% Reward Hacking Rate (RHR), while NRFR exceeding RHR shows that RL creates new failures rather than merely inheriting them. Scaling reduces but does not eliminate hacking: even the 32B model retains a 54.9% worse rate under outcome-only rewards, whereas answer-aware rewards improve the oracle trend at every scale. Robustness is also algorithm- and scale-dependent: GRPO is consistently most resistant, RLOO remains vulnerable, and DAPO improves substantially from 2B to 8B. Visual-evidence rewards help only with reliable verification: keyword-based checks increase hacking, while VLM-as-judge semantic verification reduces it. Overall, multimodal reward hacking is a systematic result of optimizing imperfect rewards, and robust alignment requires rewards and verifiers that remain reliable under optimization pressure.
[AI-11] Ceci nest pas une pipe: AI systems as semantic abstractions
链接: https://arxiv.org/abs/2607.09489
作者: Jade Alglave,Patrick Cousot
类目: Artificial Intelligence (cs.AI); Programming Languages (cs.PL)
备注:
Abstract:An AI system’s output is not the fact or world state it appears to describe, but rather an engineered representation. We propose a semantic framework to describe AI systems, to be able to examine the correctness of such representations. To do so, we distinguish what is justified by accepted domain knowledge, what reference sources say, and what the system can currently use. This allows us to give precise definitions to common failures: extrapolation, refuted or unsupported assertion, sources versus knowledge mismatch, stale or refuted source, added hypotheses, unsupported use… We hope our framework gives a useful vocabulary for specifying and checking AI systems whose outputs, citations, tool calls, and world-changing actions must be justified by reliable claims and explicit authority rather than apparent fluency.
[AI-12] ProofCouncil: An LLM Agent for Solving Open Mathematical Problems ATC
链接: https://arxiv.org/abs/2607.09474
作者: Johannes Schmitt,Tim Gehrunger,Jasper Dekoninck,Gergely Bérczi,Uri Kreitner,Liam Price,David Holmes
类目: Artificial Intelligence (cs.AI)
备注: 25 pages, 7 figures. ProofCouncil appears as System A (IMProofBench ProofCouncil) in the official FirstProof second-batch report ( arXiv:2606.18119 ). Code and agent-building library: this https URL
Abstract:Large language models (LLMs) have shown increasing promise in solving open problems in mathematics. However, their performance can be further improved through agentic workflows tailored to real-world mathematical practice. To this end, we introduce ProofCouncil, a mathematical agent that is designed to tackle open problems using an author-critic architecture. ProofCouncil served as a submission to the second batch of FirstProof, a challenge consisting of 10 real-world mathematical problems that agents must solve autonomously. Its submissions for 6 of the 10 problems were judged by the referees to be correct up to at most minor revisions, showing the best performance among participating teams. We also evaluate ProofCouncil on 30 open problems collected from mathematical researchers. Among the 21 solutions that received human feedback, 5 were judged completely correct, 2 more were judged promising pending final verification, and a further 8 contained useful partial progress. In this short paper, we describe the development of ProofCouncil and the agent-building library used to create it, which we release as open source to the community.
[AI-13] Practical Source Code Recovery from Binary Functions Using Anchor-Based Retrieval and LLM Reasoning
链接: https://arxiv.org/abs/2607.09452
作者: Charles Edward Gagnon,Steven H. H. Ding,Philippe Charland,Benjamin C. M. Fung
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: 12 pages, 5 figures
Abstract:We present a practical pipeline for recovering source code from stripped binary functions by combining reverse engineering, anchor-based source code retrieval, and large language model reasoning. Our binary-to-source-code retrieval method attempts to identify the source function from a source code database, rather than generating approximate decompiled pseudocode. It extracts anchors such as strings, constants, external calls, and available function names using Ghidra, retrieves candidate files via an inverted-index search database, narrows candidates to likely function snippets, and re-ranks them with a large language model (LLM) based on disassembly, decompiled code, and source metadata. Confident matches can also serve as anchors in later passes. In an evaluation backed by our high-fidelity source code database on a stripped, optimized tcpdump binary, our proposed binary-to-source matching method achieves 95.2% assembly instruction coverage. Experiments on a GitHub-based retrieval database showed lower performance with 35.5% instruction coverage on average, mainly due to retrieval misses. These results show that source-level binary recovery excels with high-quality databases and remains a useful tool in noisy environments.
[AI-14] How Does Bayesian Causal Discovery Fail? Characterising Structural Consequences in Linear Gaussian Networks under Latent Confounding
链接: https://arxiv.org/abs/2607.09449
作者: Debargha Ghosh,Silja Renooij,Anna Kononova
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Bayesian causal discovery is widely used for its ability to quantify epistemic uncertainty over directed acyclic graphs (DAGs) through posterior inference. However, its behaviour under latent confounding remains poorly understood, as existing work typically notes that confounding breaks identifiability without characterising how the posterior distribution over DAGs responds. In this work, we analyse posterior behaviour under latent confounding in linear Gaussian causal models, focusing on additive latent confounding between exactly two observed variables. We derive a critical correlation threshold above which the score function favours graphs with a spurious edge between the confounded variables, and show that this threshold decreases with sample size – more data lowers the correlation required for the spurious edge to be favoured. Beyond this threshold, we characterize two distinct posterior failure regimes determined by the local structure around the confounded variables. Our findings are supported by exact posterior computations on multiple graph structures, demonstrating both the predicted failure regimes.
[AI-15] Fictional Worldbuilding: Multi-Agent LLM Collaboration with Hierarchical Context Compression and Iterative Review
链接: https://arxiv.org/abs/2607.09403
作者: Jingbo Chen,He Wang,Wei Yuan,Yuqiao Lai,Zhenyan Lu
类目: Artificial Intelligence (cs.AI)
备注: 36 pages, 7 fig
Abstract:Worldbuilding, the construction of coherent fictional worlds, is a foundational task in game design and literary creation. Large Language Models (LLMs) offer new possibilities for automated content generation, but their application to worldbuilding faces three challenges: context explosion that grows linearly with the building process, the tension between creative diversity and content consistency, and the absence of automated quality assurance. This paper presents AutoWorldBuilder, a multi-agent collaborative system that addresses these challenges through five integrated components: a structured concept network with conflict detection; a DAG-based hybrid batch scheduler that groups tasks by semantic locality; a four-layer context compression mechanism achieving approximately 90% token reduction; an iterative review system with specialized Auditor agents that improves proposal pass rates from 42% to over 85%; and a skill-driven agent architecture supporting zero-code extension with differentiated temperature configuration. Two experiments across 20 diverse worldbuilding tasks, using GPT-OSS 120B and DeepSeek v3.2 as LLM backends, demonstrate a 95.0% success rate. The system generated 56-103 self-consistent concepts per world in 18-31 minutes with zero-conflict delivery. The architectural patterns validated here, including layer-as-budget compression, semantic-locality scheduling, and separation of generation and review, transfer to the broader class of knowledge-intensive, multi-agent LLM applications.
[AI-16] On-Device Adaptive Battery Power Prediction for Electric Vehicles
链接: https://arxiv.org/abs/2607.09400
作者: Avik Bhatnagar,Anton Paule,Tobias Schuermann,Sebastian Reiter,Oliver Bringmann
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR); Performance (cs.PF)
备注: 6 pages, 3 tables, 5 figures; Accepted to IEEE EdgeCom 2025
Abstract:Adaptive power management in Electric Vehicles (EVs) requires accurate power prediction. Although deep learning models have emerged as highly effective for time-series forecasting in this domain, their performance is prone to degradation when exposed to data with distributions different from the training data. We introduce a novel approach that enables on-device learning in resource-constrained EV systems to continuously adapt pretrained battery prediction models to new, unseen data. We leverage existing pretrained models by transforming them into adaptable versions that retain critical hyperparameter knowledge from their initial training. We comprehensively investigate both online and offline model adaptation strategies. Our results demonstrate significant improvements in forecasting performance across various models and time horizons, achieving mean absolute error reductions of up to 7.49% and 14.88% with online and offline adaptation techniques, respectively. This study highlights the substantial benefit of on-device adaptation, resulting in enhanced battery power predictions than unadapted model deployments in real-world EV scenarios.
[AI-17] Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks
链接: https://arxiv.org/abs/2607.09399
作者: Wout Mommen,Lars Keuninckx,Matthias Hartmann,Werner Van Leekwijck,Piet Wambacq
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:We introduce a novel method for both partial and full optimization of the connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). Our training method utilizes a probability distribution over a set of connections per gate/lookup table (LUT) input pin, selecting the connection with highest merit, all whilst the optimal gate types or LUT-entries are learned in parallel. We show that the connection-optimized LGNs outperform standard fixed-connection LGNs on the Yin-Yang, MNIST Handwritten Digits and Fashion-MNIST benchmarks, while requiring only a fraction of the number of logic gates. We achieve 98.92% on the MNIST dataset with two layers of 8000 gates. With only one layer of 8000 gates, we obtain 98.45%, showing that our method requires almost 50 times fewer gates compared to fixed-connection LGNs. Training stability up to ten layers has been ensured by employing a high learning rate, straight-through estimators and trimming constant-output gate types. Additionally, we present a LUT neuron description that enables stable training with backpropagation, tested up to 6-layer deep networks. The model requires four times fewer trainable parameters and still achieves a higher accuracy compared to the fixed-connection LGN training algorithm. Our connection-training algorithm also works well for the LUTNs, achieving an accuracy of 98.88% for two layers of 2000 6-input LUTs.
[AI-18] STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMDs XDNA NPU
链接: https://arxiv.org/abs/2607.09385
作者: Victor J.B. Jung,Gagandeep Singh,Joseph Melber,Kristof Denolf,Francesco Conti,Luca Benini
类目: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Performance (cs.PF)
备注: Accepted at IEEE COINS 2026
Abstract:The growing adoption of large language model-based agents within operating system workflows has increased the importance of energy-efficient inference on laptop-class systems-on-chip (SoCs). While cloud offloading remains common, it introduces reliability and privacy concerns that are particularly problematic for agentic workloads. Recent laptop SoCs, therefore, incorporate neural processing engines (NPUs) optimized for energy efficiency; however, effectively mapping attention mechanisms onto NPUs remains challenging due to architectural diversity and explicit data-movement programming models. In this work, we present STEEL, the first open-source implementation of FlashAttention targeting XDNA-like NPUs. STEEL introduces a dataflow formulation of prefill attention, enabling efficient exploitation of spatial parallelism and on-chip memory. Furthermore, STEEL addresses the load imbalance induced by the causal mask by leveraging a sparsity-aware pipeline placement onto the NPU array, reducing synchronization overhead and improving utilization. We evaluate STEEL on the AMD Ryzen AI 9 HX 370 SoC and compare its performance against optimized CPU and GPU implementations. Experimental results show that STEEL reduces energy consumption by an average of 9.17x and 1.75x relative to CPU and GPU baselines, respectively. On XDNA 1, STEEL achieves an average 9.6x latency reduction over the prior state of the art, and delivers a 22.8x speedup on average compared to a layer-by-layer attention implementation on XDNA 2.
[AI-19] Diversifying to Verify: When Task-Equivalent Programs Differ in Verifiability
链接: https://arxiv.org/abs/2607.09366
作者: Shirley Yu,Ruben Martins
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
备注:
Abstract:Program verification is crucial for software correctness, but producing fully verified programs remains difficult in practice. This paper studies whether implementation structure affects automated verifiability when multiple generated programs are intended to satisfy the same task-level semantics. We present Diversify2Verify, a staged LLM-based pipeline for Why3 that infers representation-specific contracts, generates and tests diverse recursive and imperative array/list implementations, and attempts verification with bounded verifier-guided annotation repair. We also construct a verification-oriented benchmark of 73 tasks over integers, arrays, and lists, yielding 292 implementation variants. Diversify2Verify verifies 96 artifacts initially and 154 after two repair passes, improving artifact-level verification from 32.9% to 52.7%. At the task level, at least one variant verifies for 49 of 73 tasks, a 67.1% success rate. These results show that task-equivalent implementations can differ substantially in verifiability and that implementation diversity helps find verification-friendly artifacts. Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO) Cite as: arXiv:2607.09366 [cs.SE] (or arXiv:2607.09366v1 [cs.SE] for this version) https://doi.org/10.48550/arXiv.2607.09366 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-20] Shortcut Trajectory Planning for Efficient Offline Reinforcement Learning
链接: https://arxiv.org/abs/2607.09336
作者: Guanquan Wang,Yoshimasa Tsuruoka
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Robotics (cs.RO)
备注: 16 pages, 3 figures
Abstract:Diffusion-based trajectory planners have shown strong performance in offline reinforcement learning, but their iterative denoising process often incurs high inference cost. Consistency-based planners reduce the number of sampling steps, yet they typically rely on a two-stage teacher–student distillation pipeline that increases training cost and may introduce instability. We propose Shortcut Trajectory Planning (STP), an offline model-based reinforcement learning framework that incorporates shortcut models as efficient trajectory generators. STP trains a conditional shortcut trajectory model in a single stage, supports adjustable one-step and few-step inference through step-size conditioning, and selects candidate plans using a critic augmented with feasibility-aware correction. Across standard D4RL benchmarks, including locomotion, navigation, manipulation, and dexterous control tasks, STP achieves strong performance while simplifying the training pipeline for fast generative planning.
[AI-21] LongMedBench: Benchmarking Medical Agents for Long-Horizon Clinical Decision-Making MICCAI2026
链接: https://arxiv.org/abs/2607.09322
作者: Yanzhen Chen,Zihan Xu,Xiaocheng Zhang,Zhiting Fan,Weiqi Zhai,Hongxia Xu,Zuozhu Liu
类目: Artificial Intelligence (cs.AI)
备注: Submitted manuscript prior to peer review in MICCAI 2026
Abstract:In this work, we introduce LongMedBench, a real-world EHR-based benchmark for long-horizon clinical decision-making. Prior evaluations of LLM-based medical agents have largely emphasized short-context knowledge QA and tool use. However, real-world medical care is inherently longitudinal, and clinicians must aggregate evidence across repeated visits, tests, and evolving treatments. Therefore, long-horizon interaction is essential for realistic assessment. LongMedBench is constructed via a reproducible pipeline that integrates MIMIC-IV admission records and clinical notes into time-series event streams and long-context memory datasets, enabling long-horizon, multi-session interactions between agents and a clinical environment. It comprises 335 patients, with 19.72 inpatient visits per patient on average and 44.91 medical events per visit. Guided by the long-horizon decision process, we propose an evaluation taxonomy with three suites: fact-based QA, temporal reasoning, and long-horizon decision-making. This taxonomy measures how agents understand and leverage historical patient information over extended horizons. Our experiments show that while recent LLMs can make good use of explicit timestamps, they have challenges in implicit time inference; The RAG and agent memory system can improve the performance of information retrieval tasks, but the performance of decision-making tasks is highly dependent on the model’s immediate context.
[AI-22] Risk-Aware General-Utility Markov Decision Processes
链接: https://arxiv.org/abs/2607.09298
作者: Pedro P. Santos,Fábio Vital,Alberto Sardinha,Francisco S. Melo
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:We study general-utility Markov decision processes (GUMDPs) with risk-aware objectives. In this framework, an agent aims to optimize a risk measure of the distribution of objective values, where the objective function depends on the frequency of visitation of states induced by the agent’s policy. First, we motivate, propose, and formalize risk-aware GUMDPs, which enable agents and decision makers to trade off expected performance by risk aversion while benefiting from the rich set of objectives that can be cast under the framework of GUMDPs. We focus our attention on the entropic risk measure (ERM). Second, we show how we can solve risk-aware GUMDPs with ERM objectives by resorting to online planning techniques. In particular, we propose an approach based on Monte Carlo Tree Search (MCTS) to provably solve risk-aware GUMDPs up to any desired accuracy. Third, we provide a set of experimental results showcasing that our approach is successful when optimizing for a spectrum of risk-aware behaviors in the context of GUMDPs under diverse tasks (standard MDPs, maximum state entropy exploration, imitation learning, and multi-objective MDPs).
[AI-23] Geopolitical alignment: Endorsement effects in large language models
链接: https://arxiv.org/abs/2607.09262
作者: Maxim Chupilkin
类目: Computers and Society (cs.CY); Artificial Intelligence (cs.AI)
备注:
Abstract:Large language models (LLMs) are increasingly used to summarize and evaluate policy-relevant information, but it remains unclear whether their judgments are implicitly shaped by geopolitical cues. I study this question with an endorsement experiment in which four LLMs evaluate the same international economic and security policies after each policy is randomly described as supported by the United States, the European Union, China, or Russia. In the numeric-only condition, GPT-5, Claude Sonnet, and Gemini rate China- and Russia-endorsed policies substantially lower than identical policies endorsed by the United States or the European Union; DeepSeek is the main exception. A second condition asks models to provide a short justification with the score. This request leaves the broad Western/non-Western gap intact for GPT-5 and Claude Sonnet, attenuates Gemini’s penalties, and sharply activates China and Russia penalties in DeepSeek. The justifications indicate that Western endorsement is often treated as a credibility cue, whereas Chinese and Russian endorsement is treated as a cue for data security, sovereignty, surveillance, or geopolitical risk. These findings show that LLM policy evaluations can depend on the identity of a foreign endorser even when policy content is held fixed.
[AI-24] Blockchain-Linked Auditable Decision Management for Telecom/IoT Fraud-Control Requests
链接: https://arxiv.org/abs/2607.09259
作者: Saviz Changizi,Nasibeh Mohammadzadeh,Mohammad Shojafar,Rahim Tafazolli
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注: 16 pages, 5 figures, 10 tables, IEEE Transaction Submission
Abstract:Telecom fraud-control studies often stop at detector-level classification, but deployment use requires request-level policy resolution, lifecycle traceability, and auditability. This paper reframes fraud control as blockchain-linked auditable decision management for synthetic telecom/IoT fraud-control requests, and its main result is that the QLoRA-tuned LLM branch becomes much more usable than zero-shot prompting but mainly approaches, rather than outperforms, a lower-cost centralized ensemble. The framework maps each synthetic deployment record to a managed request, blocks explicit out-of-boundary cases through a deterministic hard-fraud gate, scores non-hard requests using centralized ML (M1), federated meta-learning (M2), or LLM-family risk sources (M3), and resolves actions through a shared five-state policy, two-zone refinement mechanism, and local Ethereum-compatible audit layer. Evaluation uses separate synthetic training data and a 100,000-record deployment replay corpus, so the study should be read as controlled drift-replay evidence rather than field validation or proof of live deployability. On validation, M1 gives the strongest balance, with legitimate-request FPR 0.0890 under the 0.10 operating cap and soft-fraud recall 0.8341. On labeled deployment replay, however, the legitimate-FPR gap becomes large: M1 rises to 0.1646 and M3-QLoRA to 0.1801, while M3-QLoRA reduces the M3-Base legitimate FPR from 0.3915 and reaches 0.8240 soft-fraud recall. Blockchain telemetry shows that lifecycle gas, cost, latency, and throughput differences are driven by submitted off-chain decision profiles rather than changes in fraud logic.
[AI-25] actile and Vision Conditioned Contact-Centric Control for Whole-Arm Manipulation
链接: https://arxiv.org/abs/2607.09218
作者: Rishabh Madan,Angchen Xie,Samantha Saak,Andres Blanco,Dohyeok Lee,Sarah Grace Brown,Yunting Yan,Mark Zolotas,Jose Barreiros,Tapomayukh Bhattacharjee
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI)
备注: RSS 2026
Abstract:Whole-arm manipulation involves direct contact with the environment while the robot completes a task by distributing contact across multiple links as contacts form, slide, and break. This setting breaks common implicit assumptions in many learning-based manipulation pipelines: arm configuration tightly couples motion and contact forces, contact state is partially observed under occlusion, and purely learned rollouts can become physically inconsistent under distribution shift because many multi-link contact configurations are sparsely represented in the data. To address this, we propose TACTIC (Tactile and Vision Conditioned Contact-Centric Control), a receding-horizon controller for whole-arm manipulation. TACTIC uses a contact-centric hybrid predictive model that combines RGB-D, distributed tactile sensing, and a compact 2D proximity representation. The model couples a learned, action-conditioned latent dynamics model with analytical kinematics through contact Jacobians, enabling rollouts of future contact configurations and interaction forces. TACTIC integrates these rollouts into a sampling-based MPC planner with contact-aware action sampling: contact Jacobian-based projections steer sampled action sequences toward force-modulating directions, and objectives defined over predicted proximity and interaction forces trade task progress against whole-arm force regulation. We evaluate TACTIC in simulation against state-of-the-art model-based and model-free methods, and perform ablations that isolate the contribution of each design choice. TACTIC consistently outperforms other methods. We further demonstrate real-world performance on a robot with distributed tactile sensing across three whole-arm manipulation tasks that require multi-contact trajectories: turning over and repositioning a manikin, and goal-reaching in a 3D dynamic maze. Website: this https URL
[AI-26] OpenProver: Agent ic and Interactive Theorem Proving with Lean 4
链接: https://arxiv.org/abs/2607.09217
作者: Matěj Kripner,Milan Straka
类目: Artificial Intelligence (cs.AI); Mathematical Software (cs.MS)
备注: 7 pages, 2 figures. Accepted at the 19th Conference on Intelligent Computer Mathematics (CICM 2026)
Abstract:In this system paper, we present OpenProver, an open-source system for LLM-driven automated theorem proving (ATP) with integrated Lean 4 formal verification. OpenProver integrates a Planner-Worker-Verifier architecture inspired by recent ATP agentic systems such as Aletheia. A Planner agent maintains a compact Whiteboard scratchpad and an unbounded Repository of intermediate findings, and decomposes mathematical work into parallel Workers. OpenProver is fully open-source, offers reproducible evaluation through automatic formal verification of generated proofs, and provides an interactive terminal interface for human-guided proof search. In interactive mode, OpenProver allows the human operator to monitor and steer the proof search process, motivated by the established human-AI synergy in interactive code generation. To showcase the potential for quantitative ablation experiments enabled by automatic formal verification, we evaluate OpenProver on ProofNet and compare it with a simple baseline. OpenProver is publicly available at this https URL. Comments: 7 pages, 2 figures. Accepted at the 19th Conference on Intelligent Computer Mathematics (CICM 2026) Subjects: Artificial Intelligence (cs.AI); Mathematical Software (cs.MS) Cite as: arXiv:2607.09217 [cs.AI] (or arXiv:2607.09217v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2607.09217 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-27] Interference and Retention in Continual Learning
链接: https://arxiv.org/abs/2607.09202
作者: Julius Störk
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
备注: 41 pages, 21 figures, 8 tables
Abstract:Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation. This work argues that forgetting should instead be modeled directly as interference between tasks. In the frozen-feature regime, forgetting from learning a new task is exactly the interference energy induced on the old task. In deep networks, the same quantity is recovered through path-averaged curvature with minimal additional forward passes. When task supports are disjoint, forgetting can be eliminated structurally and when task supports overlap in conflicting directions, a non-zero distortion floor is unavoidable. The same geometry optimally merges models through task-aware orthogonalization. From this analysis we derive Interference-Gated Functional Allocation (IGFA), a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict. Across benchmarks, IGFA achieves lossless retention when tasks are structurally separable and moves unavoidable cost from irreversible forgetting into deferred but recoverable plasticity when they are not. It matches the strongest replay-free structural baselines on dissimilar-task streams and improves on unconditional projection when similarity makes transfer worth preserving. Comments: 41 pages, 21 figures, 8 tables Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE) MSC classes: 68T05, 68T07 ACMclasses: I.2.6 Cite as: arXiv:2607.09202 [cs.LG] (or arXiv:2607.09202v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2607.09202 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-28] oward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents
链接: https://arxiv.org/abs/2607.09195
作者: Izumi Takahara,Teruyasu Mizoguchi
类目: Artificial Intelligence (cs.AI); Materials Science (cond-mat.mtrl-sci)
备注:
Abstract:Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery. Equipped with broad knowledge, flexible reasoning, and tool use, they have the potential to autonomously explore and solve scientific problems by repeatedly proposing hypotheses, testing them, and revising their beliefs in the light of the evidence. In current agents, however, these hypotheses, tests, and belief updates are buried in unstructured logs, and no mechanism lets the agent or the human researcher audit that process. Here we propose the Hypothesis Evolution Protocol (HEP), an agent harness that provides hypothesis generation, evaluation, and evolution as explicit, auditable operations. On materials-science research tasks, a HEP-equipped agent operates the hypothesis–test–evidence–belief cycle that planning-style agents lack, generalizes across research questions, and exploits the protocol more fully as the base LLM becomes more capable. These results mark a step toward auditable AI scientists, whose scientific reasoning can be inspected, verified, and built upon.
[AI-29] Generative Communications: Overview Technologies and Trends
链接: https://arxiv.org/abs/2607.09183
作者: Wenjun Zhang,Zhiyong Chen,Tong Wu,Guo Lu,Li Song,Feng Yang,Meixia Tao
类目: Information Theory (cs.IT); Artificial Intelligence (cs.AI)
备注: accepted by IEEE Wireless Communications Magazine
Abstract:The groundbreaking development of generative artificial intelligence (AI) is rapidly boosting the ability to generate content such as images and videos, reshaping communication paradigms. This article introduces generative communications (GenCom), a novel paradigm for 6G networks in which large AI models (LAMs) drive semantic understanding, reasoning, and content generation, embedding these into the communication process. Unlike traditional systems that strictly pursue accurate bit transmission, GenCom enables transmitters to convey only minimal yet sufficient information, while receivers leverage shared generative priors and knowledge bases to synthesize the intended output. Communication is thus redefined as controlled generation rather than data reproduction. We formalize the concept of GenCom, clarify its AI-native and generation-driven properties, and present its core mechanisms. A two-layer GenCom architecture supported by key enabling technologies is proposed, and analysis of four representative application scenarios demonstrates that GenCom offers ultra-efficient transmission, semantic-level robustness, and new network functions. Finally, we outline future research directions, including foundational theory and real-time processing, highlighting a promising pathway toward 6G networks.
[AI-30] Attention to Detail: Evaluating Energy Performance and Accuracy Trade-offs Across vLLM Configurations
链接: https://arxiv.org/abs/2607.09172
作者: Nada Zine,Tristan Coignion,Vincenzo Stoico,Clément Quinton,Romain Rouvoy,Patricia Lago
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Performance (cs.PF)
备注: Submitted at a conference
Abstract:Large Language Models are reshaping how software is developed and maintained. They are typically deployed in production using inference engines such as vLLM, which can efficiently serve pre-trained, highly configurable models. While prior work has focused on model architectures and hardware acceleration, the impact of inference engine configuration on energy consumption, performance, and output quality remains poorly understood. In this paper, we present a large-scale controlled study of three selected vLLM configuration options: attention kernel type, prefix caching, and chunked prefill. We evaluate all combinations of these configurations across 5 open-weight LLMs and 5 diverse inference tasks, totaling 9,000 runs and 93,600 measures. We analyze energy consumption, latency, and accuracy, and examine both main effects and interaction effects between configuration options and tasks. Our results show that the studied configuration options significantly impact energy and performance, mainly driven by attention type and prefix caching, while chunked prefill has a limited effect under the default vLLM serving configuration and evaluated workloads. These effects are highly model- and workload-dependent, and no configuration is universally optimal. We further show that model choice dominates global trade-offs, while configuration tuning provides local improvements along the Pareto frontier. Unexpectedly, inference options can also affect model accuracy. Comments: Submitted at a conference Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Performance (cs.PF) Cite as: arXiv:2607.09172 [cs.SE] (or arXiv:2607.09172v1 [cs.SE] for this version) https://doi.org/10.48550/arXiv.2607.09172 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[AI-31] A Personalized Computational Framework for Assessing the Sufficiency of Partially Observed Data in Healthcare AI models
链接: https://arxiv.org/abs/2607.09165
作者: Qingchu Jin,Felistas Mazhude,Jamie B. Rabb,Robert S. Kramer,Douglas B. Sawyer,Raimond L. Winslow
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Achieving early and timely diagnosis and treatment for disease is a major challenge. Recent applications of machine learning (ML) algorithms trained on patient data have shown promise in many different settings for predicting the patient health state. A challenge often faced when applying these ML algorithms is that at any given time, not all clinical variables (features) needed as input to perform prediction tasks are available. We define the concept of full-feature-capacity (FFC) to refer to prediction performance when such algorithms make use of all features on which they were trained. We then introduce Feature Sufficiency Analysis (FSA) - an analysis for determining whether a subset of all clinical features needed by an AI model is sufficient to achieve FFC. FSA estimates the underlying distributions of missing variables conditioned on features that are available. FSA provides a patient-specific assessment of whether the existing set of measured features achieves FFC. If yes, then there is no need to acquire further inputs and a ML-based prediction. We provide two case studies: prediction of need for postoperative prolonged ventilation in patients recovering from heart surgery; 10-year mortality prediction in an outpatient cohort. We also demonstrate that FSA also provides a clinically interpretable feature-ranking methodology based on prediction sufficiency, identifies intrinsically hard-to-predict patient populations, and has the potential to perform cost-aware optimization for clinical data acquisition. FSA provides a generic computational approach for determining whether incomplete clinical information is sufficient to support trustworthy AI-assisted clinical decision-making, thereby facilitating the prospective deployment of healthcare AI systems across diverse clinical settings.
[AI-32] KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scaling
链接: https://arxiv.org/abs/2607.09153
作者: Peng Kuang,Haibo Jin,Xiaoyu Han,Yanli Wang,Xiaopeng Yuan,Ye Yu,Kaidi Xu,Haohan Wang
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Process Reward Models (PRMs) have been proven to be highly effective in guiding test-time scaling (TTS) methods, which significantly boost the capabilities of LLM-based multi-agent systems. However, existing PRMs are text-based: they re-encode the entire trajectory text from scratch. In long multi-agent rollouts, the scoring cost, growing quadratically with respect to sequence length L, creates a severe computational bottleneck, severely limiting PRMs’ application in long-context scenarios. To resolve this, we introduce KV-PRM, a highly efficient process reward model that eliminates the heavy text re-encoding by directly reading the KV cache produced naturally during the LLM’s generation phase. By processing a single “verify token” against the pre-existing KV cache, KV-PRM reduces the scoring cost from O(L^2) to O(L). We formally prove that the KV cache contains strictly greater information capacity than text, and is more efficient for downstream reward modeling. Empirically, across the MATH, GSM8K, and AIME benchmarks, KV-PRM matches or strictly outperforms text-PRMs under various TTS methods such as Beam Search, MCTS, and Weighted Voting, with up to a 5,000x reduction in scoring FLOPs, a 37x reduction in latency, and a 34x reduction in per-sequence memory footprint compared to text-based PRMs.
[AI-33] ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Models ICML2026
链接: https://arxiv.org/abs/2607.09134
作者: Sang-Hoon Lee,Ha-Yeong Choi
类目: ound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)
备注: Accepted to ICML 2026
Abstract:Representation alignment (REPA) has been investigated to accelerate diffusion training, but we observe that regularizing intermediate representations in diffusion Transformers (DiT) may implicitly entangle latents and limit generative capacity. To address this issue, we propose ReGen, a hierarchical multi-prompt representation generation framework that jointly estimates multiple vector fields for both representations and data within a single diffusion model. We further introduce generalized flow matching (GFM) to improve the generalization of conditional flow matching (CFM). We validate ReGen on single-stage waveform diffusion models including neural audio codec and Wave-VAE. ReGen significantly improves waveform generation quality from highly compressed latent representations at 12.5 Hz. We also present ReGenVoice, a latent diffusion model (LDM)-based text-to-speech model that achieves strong speech intelligibility (WER) and speaker similarity (SIM) with a small dataset. Moreover, operating the LDM at 6.25 Hz with rich semantic and acoustic latent representation enables efficient training and sampling, requiring only 1 day of training on 4 GPUs and fast inference with an RTF of 0.08. Audio samples are available at this https URL.
[AI-34] L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning ICML2026
链接: https://arxiv.org/abs/2607.09099
作者: Tan-Minh Nguyen,Hoang-Trung Nguyen,Huu-Dong Nguyen,Dinh-Truong Do,Thi-Hai-Yen Vuong,Le-Minh Nguyen
类目: Artificial Intelligence (cs.AI)
备注: Outstanding paper in the AI4Law Workshop at ICML 2026
Abstract:While multi-agent debate (MAD) frameworks have shown significant potential in general reasoning, their effectiveness in highly structured, knowledge-heavy legal domains remains under-explored. In this work, we introduce the Legal Multi-Agent Debate (L-MAD) framework to systematically evaluate different debate structures and aggregation methods within Legal Textual Entailment. By assigning distinct expert personas to multiple agents, L-MAD improves upon strong single-agent baselines by up to 8%. Furthermore, analyzing how debate scales reveals a clear trade-off: increasing the agent population reduces inconsistency and improves accuracy, whereas extending discussion rounds induces a detrimental \textitover-deliberation drift where agents reinforce each other’s mistakes. Ultimately, our findings outline the practical boundaries and safety margins of deploying collaborative multi-agent systems in high-stakes legal reasoning environments.
[AI-35] Neuro-Agent ic Control: A Deep Learning-based LLM -Powered Agent ic AI Framework for Controlling Security Controls
链接: https://arxiv.org/abs/2607.09076
作者: Saroj Gopali,Bipin Chhetri,Deepika Giri,Sima Siami-Namini,Akbar Siami Namin
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Cyberattacks on operational technology are increasingly causing costly downtime and physical damage, exposing the limitations of traditional rule-based monitoring in industrial IoT environments. While Large Language Models (LLMs) have strong semantic reasoning abilities to assist in decision support, their hallucinatory nature presents unacceptable safety liabilities for closed-loop control. This paper introduces a neuro-agentic control framework, a novel architecture that couples an LLM-based planner (i.e., such as Gemini 2.5 Flash-Lite) with a pre-trained Time-Series Foundation Model (TimesFM), to achieve physics-grounded autonomous defense. The paper introduces a Counterfactual Physics Injection'' mechanism that simulates the impact of LLM-proposed interventions within the numerical latent space of the foundation model before actuation, while allowing the system to reject hallucinatory or unsafe actions. Evaluated on an industrial dataset (e.g., the Secure Water Treatment (SWaT)) in the context of stochastic attack scenarios, the framework exhibited better performance compared to LSTM and TCN baselines. The Neuro-Agentic Loop prevented five breaches (33.3%) below the threshold versus LSTM (26.7%) and TCN (13.3%), with zero physically invalid (hallucinated) actions executed. These results demonstrate the efficacy of using foundation models as deterministic Sentinels’’ to safeguard agentic AI in critical infrastructure.
[AI-36] Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills
链接: https://arxiv.org/abs/2607.09065
作者: Jialun Cao,Xinru Yan,Songqiang Chen,Yaojie Lu,Zhongxin Liu,Shing-Chi Cheung
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:Software engineering (abbrev. SE) has continuously evolved through increasingly powerful forms of reuse, from source code and libraries to components and services. Recent advances in AI agents have introduced a potentially new reusable artifact: skills. Emerging agent skill repositories and marketplaces enable developers to package, share, and reuse SE expertise as reusable skills. This trend raises a fundamental question: what SE activities are being encapsulated into reusable skills? Existing studies primarily focus on a broad range of skills acquisition, safety, or benchmarking, while lacking a systematic understanding of SE-specific skills and their coverage across the software development lifecycle. To address this gap, we conduct the first large-scale empirical study of SE skills in public repositories and marketplaces. We collect and analyze a large corpus of SE skills, examining the activities they encapsulate, lifecycle coverage, evolution characteristics, and evaluation mechanisms. Our findings reveal that SE activities are increasingly becoming reusable artifacts via skills and suggest promising research opportunities for skill recommendation and engineering-oriented structuring, as well as the need for mechanisms to encapsulate high-context SE activities into reusable skills. Overall, our study provides the first activity-centric characterization of SE skills and reveals how SE activities are increasingly being transformed into reusable skills. These findings offer new insights into skill reuse, ecosystem development, and the future of agent-centric SE.
[AI-37] ARCANA: A Reflective Multi-Agent Program Synthesis Framework for ARC-AGI-2 Reasoning
链接: https://arxiv.org/abs/2607.09059
作者: Kunbo Zhang,Lei Fu,Zeyu Wang,Zijing Liu,Kejian Tong
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:We present ARCANA, a collaborative multi agent framework for solving ARC AGI 2 tasks under strict test time and hardware constraints. ARCANA decomposes each task into iterative perception, hypothesis generation, symbolic execution, and reflective refinement. A perceptual grounding agent builds object centric scene graphs from raw grids, a latent program policy proposes diverse DSL programs, a symbolic executor verifies candidates on demonstrations, and a reflective agent synthesizes failure driven feedback for the next turn. These agents communicate through a shared differentiable blackboard and are scheduled by a learned meta controller. The design combines structured program search with adaptive multi turn correction, improving reasoning efficiency and solution quality on challenging abstract transformation tasks.
[AI-38] Evolutionary Intelligence for Scientific Discovery: From Evolutionary Computation to Cumulative Discovery Systems
链接: https://arxiv.org/abs/2607.09025
作者: Chao Wang,Lingling Li,Fang Liu,Licheng Jiao
类目: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE)
备注: A perspective article submitted to a journal of Springer Nature
Abstract:Artificial intelligence (AI) is shifting scientific discovery from task-specific workflows towards autonomous systems that organize exploration with experimental and human feedback in open-ended candidate spaces. Evolutionary computation (EC) provides a computational basis for feedback-driven discovery because population-based search can maintain diverse scientific candidates while steering exploration through accumulated evidence. However, EC predominantly focuses on candidate refinement for predefined problems, whereas cumulative discovery requires experience retention. To bridge this gap, this review introduces evolutionary intelligence (EI) for scientific discovery. EI characterizes scientific AI systems that sustain exploration by linking candidate refinement with experience retention across evolutionary cycles. We introduce a five-dimensional analytical framework that asks what evolves, how candidates change, why candidates are selected, where feedback originates, and when evolution occurs. This framework clarifies how EI transforms isolated search trajectories into cumulative scientific insight. We further demonstrate this paradigm across diverse discovery modes, from evolving concrete scientific entities to orchestrating automated research workflows. Finally, we identify critical bottlenecks regarding evaluation, process traceability, and shared infrastructure, providing a concrete roadmap for advancing the transition from EC to EI in scientific discovery.
[AI-39] Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing
链接: https://arxiv.org/abs/2607.09015
作者: Ajay Narayanan Sridhar,Ronak Singh,Mehrdad Mahdavi,Vijaykrishnan Narayanan
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:We study contextual bandit problems with correlated arms and access to surrogate reward signals produced by a machine learning model, motivated by applications such as large language model (LLM) routing. Unlike classical contextual bandits that rely solely on bandit feedback and assume conditional independence across arms, our setting allows context-dependent inter-arm correlations and auxiliary reward information that may be noisy or misspecified. We propose algorithms that leverage such surrogate rewards through two complementary designs. A coupled reward-mixing approach pools true and surrogate rewards to accelerate learning when surrogate signals are reliable, while a decoupled prediction-mixing approach maintains separate estimators for bandit feedback and surrogate rewards and adaptively combines their predictions. This decoupling yields robustness to surrogate misspecification, recovering regret guarantees comparable to reward-only bandit methods in the worst case, while achieving improved regret when surrogate predictions are sufficiently informative. We provide theoretical regret analyses for both approaches and evaluate them on LLM routing benchmarks under varying accuracy versus cost trade-offs. The results demonstrate improved sample efficiency and consistently better accuracy-cost trade-offs compared to standard contextual bandit baselines and strong static routing methods.
[AI-40] Model Agnostic Graph Prompt Learning for Crystal Property Prediction UAI2026
链接: https://arxiv.org/abs/2607.08996
作者: Shrimon Mukherjee,Kishalay Das,Partha Basuchowdhuri,Pawan Goyal,Niloy Ganguly
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: Accepted in UAI 2026
Abstract:Graph Neural Networks have emerged as a powerful tool for the fast and accurate prediction of various crystal properties. These models often encode domain-specific knowledge into their graph encoding modules, which increases their parameter size and makes their performance heavily dependent on domain expertise. Added to this, explicitly incorporating all chemical and structural features, that might influence a specific crystal property into the GNN encoder, is a challenging task. In this work, we propose a soft prompt learning framework that captures latent features essential for property prediction, which are not explicitly provided to the GNN. We introduce a novel multilevel graph prompt learning framework comprising both node-level and graph-level soft prompts. At the node level, we capture the local chemical semantics of different atom types, while at the graph level, we encode the global structural symmetry of the crystal graph. Our proposed prompt learning framework is lightweight and seamlessly integrates with any existing GNN encoder. Extensive experiments on popular benchmark datasets show that incorporating prompt learning significantly improves (3% - 15%) the performance of state-of-the-art GNN models in crystal property prediction tasks. Furthermore, the learned soft prompts enable cross-property knowledge transfer, enhancing prediction performance for properties with limited training data. Code is available at this https URL
[AI-41] A Formalization of the Mean-Field Derivation of the Vlasov Equation: AI-Assisted Lean Formalization as a Strategy Game
链接: https://arxiv.org/abs/2607.08986
作者: Joseph K. Miller
类目: Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO); Mathematical Physics (math-ph); Analysis of PDEs (math.AP)
备注: 26 pages, 4 figures. Lean 4 development, blueprint site, and agent logs: this https URL
Abstract:We formalize a research result in the Lean 4 proof assistant by having a mathematician direct an AI system, and frame the activity as a formalization game. The objective is to turn a LaTeX document into Lean. The game is won when the development compiles, contains no sorry, and a machine check shows the target theorems rest on Lean’s foundational axioms alone. Reuse is a second check, by a definition we introduce: whether the development yields a self-contained layer of general mathematics the wider library could absorb. The case study is a complete, axiom-clean formalization of well-posedness for the nonlinear Vlasov equation via Dobrushin’s mean-field route – existence, uniqueness, the stability estimate and mean-field limit, and a short-window superposition principle (weak solutions are Lagrangian). The human’s role was to direct, not to write proofs: to scope the definitions, steer the decompositions, and triage the library’s gaps; the AI agent executed. The formalization certifies the proof of each statement as written; whether the written statement is the intended theorem stays the mathematician’s judgment. The optimal-transport machinery that fell out of the build (in particular, properties of the Wasserstein-1 metric and the Kantorovich-Rubinstein duality theorem) separates into a self-contained layer that compiles against Mathlib alone: about a sixth of the development (49 of 299 declarations), behind a 22-declaration interface with no reverse dependency. The headline theorems ran in about a week, the full development in about a month. We report the quantitative claims as observations of one game, not as general laws. The game’s rules name no particular system, so the methodological framing is meant to outlast the tools of any one run.
[AI-42] AlphaZero in Sparsely Rewarded Games: Limits and Auxiliary Supervision
链接: https://arxiv.org/abs/2607.08984
作者: Brent Kong,Tejas Ram,Tony Yue Yu
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT); Combinatorics (math.CO)
备注:
Abstract:AlphaZero has demonstrated that a neural-guided Monte Carlo Tree Search can achieve superhuman performance, but strong play does not necessarily imply perfect play. We study this gap in two oracle-evaluable domains with contrasting structure: Connect Four, a solved partisan game with exact game-theoretic values, and Chomp, an impartial game whose optimal play is governed by Grundy-number structure. Under a unified self-play + MCTS pipeline, we compare vanilla AlphaZero, a multi-frame variant (limited to Chomp), and an AlphaZero Auxiliary Loss (AZAL) that adds oracle-derived policy supervision. We find that vanilla AlphaZero achieves strong play across both domains but cannot preserve the exact trajectories required for optimal play: in Connect Four, it fails to maintain the optimal line of play, while in Chomp, it fails to consistently restore the g=0 invariant. On rectangular Chomp boards, multi-frame inputs alone do not remove this gap. Nevertheless, AZAL substantially improves oracle consistency across multi-seeded full-game traces and sampled-state evaluations. On Chomp, AZAL reaches perfect full-game oracle consistency on 10x11 and high but not complete consistency on 9x10; on Connect Four, AZAL improves oracle-match rate and delays the first oracle mistake, but does not reach perfect play.
[AI-43] SCATE: Learning to Supervise Coding Agents for Cost-Effective Test Generation
链接: https://arxiv.org/abs/2607.08983
作者: Sijia Gu,Noor Nashid,Ali Mesbah
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:While autonomous coding agents have significantly advanced automated test generation, they remain fundamentally limited by lazy generation, a phenomenon where agents prematurely terminate tasks and systematically avoid complex programmatic logic, resulting in inadequate code coverage. Currently, mitigating this premature termination requires continuous human-in-the-loop supervision. This heavy reliance on human intuition creates a bottleneck that negates the efficiency gains of automated generation. We propose SCATE, a framework for adaptive, automated supervision of coding agents that replaces human intervention during test generation. By formulating supervision as a contextual bandit problem, SCATE learns to select the most promising testing actions based on the current coverage and class testability metrics, maximizing coverage gains while minimizing wasted generation effort. Our empirical evaluation demonstrates that SCATE integrates seamlessly with different coding agents. When applied to GEMINI-CLI, it achieves 32.3% higher line coverage and 30.9% higher branch coverage than the agent-only baseline. A comparison with CLAUDE CODE confirms the framework dynamically adapts its policy to optimize each agent’s unique strengths. SCATE also consistently outperforms state-of-the-art non-agentic approaches across all metrics.
[AI-44] he Patchwork Problem in LLM -Generated Code
链接: https://arxiv.org/abs/2607.08981
作者: Viraaji Mothukuri,Reza M. Parizi
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注:
Abstract:LLM-generated code often compiles, passes tests, and appears correct, yet breaks once deployed. The root cause is frequently structural rather than logical. A generated endpoint references configuration keys never declared in the project, an import targets a package that does not exist in any registry, or a new route omits the authentication guard applied to every sibling endpoint. Each patch is locally valid but globally incoherent, and standard CI toolchains rarely surface these failures. As LLM-powered coding tools see widespread adoption, this blind spot poses a growing risk to software quality. We call this the \textbfpatchwork problem. This paper formalizes structural coherence as consistency invariants over graph representations of repository artifacts, including import, call, dependency, configuration, schema, resource, control-flow, and routing graphs, and introduces an eight-category failure taxonomy distinguishing defects specific to LLM generation from those merely amplified by it. We present a hybrid verification framework that delegates to mature static analysis tools where they already excel and deploys purpose-built detectors for cross-cutting invariants underserved by existing toolchains, targeting provable constraint violations rather than heuristic pattern matching. Empirical evaluation across two frontier models under four prompting strategies reveals that the vast majority of structural failures evade type checking, testing, and SAST entirely, and that failure patterns diverge qualitatively between models in ways that challenge model-agnostic mitigation strategies. External validation on real-world AI-generated repositories confirms that these failures are not artifacts of controlled experimentation but are prevalent wherever LLMs write code with minimal human oversight.
[AI-45] CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding
链接: https://arxiv.org/abs/2607.08974
作者: Yuri Ishitoya,Jeremy Siburian,Masashi Hamaya,Kuniaki Saito,Cristian C. Beltran-Hernandez,Mai Nishimura
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI)
备注: Project website: this https URL
Abstract:Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult to isolate what the VLM contributes to control. Directly converting pretrained VLMs into VLAs with minimal architectural change offers a more transparent path to understanding how VLM capabilities transfer across model scales. The core obstacle is output-distribution mismatch: predicting actions as bare numeric token sequences moves generation away from the VLM’s pretrained language distribution, degrading the capabilities we seek to preserve. To address this, we propose CLAP (Causal Language-Action Prediction), which prepends each numeric action sequence with a natural-language action description, causally conditioning precise action-token prediction on a language-action plan without modifying the backbone architecture. With single-epoch fine-tuning alone, 2B CLAP achieves 90.8% on LIBERO (+14.9 pt over VLA-0) and improves robustness on LIBERO-PRO under language, object, and spatial perturbations. We will release CLAP at 0.8B, 2B, and 4B as an open-weight, multi-scale compact VLA family from a single VLM lineage, enabling controlled analysis of VLM-to-VLA capability transfer.
[AI-46] Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading
链接: https://arxiv.org/abs/2607.08964
作者: Zongxia Li,Zhongzhi Li,Yucheng Shi,Ruhan Wang,Junyao Yang,Zhichao Liu,Xiyang Wu,Anhao Li,Yue Yu,Ninghao Liu,Lichao Sun,Haotao Mi,LeoweiLiang
类目: Artificial Intelligence (cs.AI)
备注: 17 pages
Abstract:AI agents have become capable of autonomously completing short, well-specified tasks. However, existing terminal benchmarks largely focus on simple problems that finish within minutes and are evaluated only by their final outcome. This setup overlooks intermediate progress and partial solutions, yielding sparse reward signals and an incomplete picture of agent capability. We introduce Long-Horizon-Terminal-Bench, a terminal benchmark of 46 long-horizon tasks spanning nine categories, including experiment reproduction, software engineering, multimodal analysis, interactive games, and scientific computing. Each task follows a Terminal-Bench-style setup with a reference solution or simulation engine, but is further decomposed into fine-grained graded subtasks. This design enables dense intermediate rewards and partial credit, allowing evaluation to capture not only whether an agent reaches the final goal, but also how far it progresses on open-ended workflows. Tasks in Long-Horizon-Terminal-Bench typically require hundreds of episodes and minutes to hours of execution, stressing long-horizon planning, long-context management, and iterative debugging rather than one-shot problem solving. We evaluate 15 frontier models and find that agents consume on average 9.9M tokens per task, with roughly 231 episodes and 85.3 minutes of execution time per run, making Long-Horizon-Terminal-Bench more demanding than prior terminal-based benchmarks. Even the strongest tested model achieves 15.2% pass@1 at a partial-reward threshold of 0.95 and 10.9% at a perfect-reward threshold of 1.0, while the mean pass rate across models is 4.3% and 1.7% under the two thresholds, respectively. These results reveal headroom for improvement. We further analyze failure modes and error patterns, and release Long-Horizon-Terminal-Bench to support future progress on long-horizon terminal agents.
[AI-47] NL-PAC: Specification Ambiguity and Certified Minimax Risk Floors in LLM -Mediated Supervision
链接: https://arxiv.org/abs/2607.08961
作者: Berkay Anahtarci
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Statistics Theory (math.ST)
备注:
Abstract:Large language models increasingly provide labels, evaluations, and feedback for tasks specified in natural language. When a specification admits multiple readings but the supervision channel does not reveal which is operative, additional labels reduce sampling error without resolving the resulting identification problem. We introduce Natural Language PAC (NL-PAC), a framework that uses a fixed model’s thresholded decoding law to define admissible labels and candidate targets. The probability that multiple labels are admissible equals the diameter of the pointwise-admissible target class, and under target-blind supervision every learner incurs worst-case risk of at least half this diameter, at every sample size; the exact randomized minimax risk over this class is attained by a data-independent strategy. Finite-sample confidence bounds make these quantities certifiable from held-out unlabeled inputs. In a frozen Qwen~2.5–3B audit, one prespecified prompt yields a positive model-relative certificate, whereas a paraphrase and exact-rule controls yield zero. A held-out bridge audit finds that supplied candidate reading clauses fail the admissibility condition needed to transfer the certificate to coherent readings. The guarantee is specific to the audited model, prompt, threshold, and input distribution; extending it to human interpretations requires external validation.
[AI-48] Eluna: An Agent ic LLM System for Automating Warehouse Operations with Reasoning and Task Execution
链接: https://arxiv.org/abs/2607.08960
作者: Ning Liu,Kalle Kujanpää,Zhaoxuan Zhu,P Aditya Sreekar,Kaiwen Liu,Chuanneng Sun,Jorge Marchena Menendez,Matthew Bales,Tianyu Yang,Shahnawaz Alam,Rose Yu,Baoyuan Liu,Kristina Klinkner,Shervin Malmasi
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Warehouse operations are governed by Standard Operating Procedures (SOPs) that encode complex, multi-system decision logic, which must be executed reliably under strict time constraints, yet LLM agents lack mechanisms to enforce procedural compliance and degrade under the context overload full SOP specifications introduce. We present Eluna, a production-deployed agentic system for reliable SOP execution. Eluna is a graph-guided, multi-agent framework that encodes SOPs as directed acyclic graphs with progressive disclosure and delegates independent tasks to parallel sub-agents, each with persistent code execution and live data access. To meet production latency and accuracy needs, we use asymmetric episodic distillation where a strong teacher is improved through episodic error memories, then a smaller student is fine-tuned on the corrected trajectories with memory stripped, internalizing corrections without inference-time overhead. On a 13-task benchmark and two production applications, our fine-tuned models match or exceed their teacher, beat all larger off-the-shelf baselines, and reach 94% expert agreement on the ticket processing application.
[AI-49] GATS: Graph-Augmented Tree Search with Layered World Models for Efficient Agent Planning
链接: https://arxiv.org/abs/2607.08894
作者: Maureese Williams,Dymitr Nowicki
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Large Language Model (LLM) agents have shown promise in multi-step planning tasks, but existing approaches like LATS (Language Agent Tree Search) and ReAct rely heavily on LLM inference during planning, leading to high computational costs and stochastic behavior. We present \textbfGATS (Graph-Augmented Tree Search), a planning framework that combines systematic UCB1-based tree search with a layered world model to eliminate LLM calls during inference while achieving superior planning performance. Our three-layer world model integrates: (L1) exact symbolic action matching, (L2) statistics learned from execution logs, and (L3) LLM-based prediction for unknown actions. On synthetic planning tasks with branching paths and dead-ends, GATS achieves \textbf100% success rate compared to 92 % for LATS and 64% for ReAct. On a comprehensive stress test spanning 12 challenging scenarios – including coding workflows, web navigation, and long-horizon tasks – GATS maintains \textbf100% success while LATS drops to 88.9 % and ReAct to 23.9%. GATS requires \textbfzero LLM calls per task during planning (vs. 37 per task for LATS) and produces deterministic plans with zero variance across runs. Our results demonstrate that systematic search with learned world models can substantially outperform LLM-guided exploration for agent planning.
[AI-50] Prompt-Driven Exploration
链接: https://arxiv.org/abs/2607.08837
作者: Sunshine Jiang,John Marangola,David Zhang,Raghuram Kowdeed,Ruiyang Luo,Nitish Dashora,Richard Li,Pulkit Agrawal,Zhang-Wei Hong
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers. Standard methods inject stochasticity in the action space, but such jitter only yields rollouts close to the original. Escaping a weak policy often requires global perturbations that action noise cannot produce. Large language models (LLMs) and vision-language-action (VLA) models offer a pathway: they condition the policy on a natural language prompt, and since the rollout follows from it, modifying the prompt induces global changes. The challenge is finding prompts that induce useful global changes. With a weak policy that rarely succeeds, reward is too sparse to select on. Our idea is to refine prompts from the rollouts themselves: a vision-language model (VLM) reasons over the rollout video, diagnoses how the policy responded, and rewrites the prompt to elicit better behavior next time. This procedure realizes posterior sampling, a classical RL exploration framework, at the level of prompts: the VLM maintains an implicit distribution over useful prompts and updates it from observed rollouts. We call this strategy Prompt-Driven Exploration (PDE). Across manipulation and reasoning tasks, PDE enables RL to learn successful policies even from zero-reward starts, and improves sample efficiency more broadly. Our website is available at this https URL.
[AI-51] Multi-Conditioned Diffusion Synthesis of Sand Boils for Low-Resource Earthen-Levee Inspection
链接: https://arxiv.org/abs/2607.08794
作者: Padam Jung Thapa,Abdullah Bin Naeem,Ayon Dey,Anav Katwal,Md Tamjidul Hoque
类目: Graphics (cs.GR); Artificial Intelligence (cs.AI)
备注:
Abstract:Sand boils on earthen levees are safety-critical defects, but pixel-level detection is limited by scarce annotations. We present a diffusion-based synthesis pipeline for low-resource sand-boil imagery. Using Stable Diffusion XL fine-tuned with DreamBooth and conditioned by a multi-branch ControlNet stack, the pipeline generates synthetic inspection images from a small curated reference set. A soft-mask inpainting protocol preserves the real defect pixels while re-rendering the surrounding scene, avoiding seams and color shifts from prior seamless-cloning compositing. A mask-conditioned ControlNet can also generate a new boil inside a chosen mask, making the mask the segmentation label by construction; however, because large-scale label certification remains unresolved with the available real-trained gate, we release the soft-mask preset as the default. Text conditioning is supplied by a taxonomy-driven Prompt Atlas that expands one domain specification into a stratified, CLIP-validated prompt bank and transfers to new defect classes without code changes. From the real training images, the pipeline produces 1,020 synthetic candidates, of which 815 pass a CLIP admissibility filter. We evaluate image quality using distributional and fidelity-diversity measures against the real reference set and a Poisson baseline, and audit for out-of-distribution drift and memorization. No single preset dominates; each trades off fidelity, diversity, and label reliability. We therefore release the label-reliable preset as the default and treat a curated mixture as the natural augmentation set. Our claims are limited to image quality, label provenance, and diversity; downstream segmentation is left for future work. Code and an artifact manifest are released for reproducibility.
[AI-52] LLM -Driven Evolutionary Generation of Multi-Objective Bayesian Optimization Algorithms
链接: https://arxiv.org/abs/2607.08791
作者: Georgios Laskaris,Reuben Brasher,Niki van Stein,Elena Raponi,Thomas Bäck,Florian Neukart
类目: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
备注:
Abstract:Designing effective multi-objective Bayesian optimization (MOBO) algorithms requires balancing many interdependent design choices whose optimal configuration is problem-dependent and typically demands deep expertise. We extend the LLaMEA framework to MOBO, using large language models as mutation and crossover operators within evolutionary strategies to generate complete algorithm implementations, with SMAC hyperparameter optimization integrated into the evolutionary loop. Across nine evolutionary runs we generated approximately 900 algorithms and benchmarked them on twelve synthetic problems (ZDT, DTLZ, WFG) and three real-world engineering problems (RE), using a BoFire qParEGO implementation as a state-of-the-art Bayesian-optimization baseline. On the synthetic suite the strongest generated algorithm attains the highest mean normalized hypervolume (0.971, vs. 0.869 for qParEGO) while requiring roughly 60x less wall-clock time; a Friedman test with post-hoc analysis places the two in a single top-performing group, and per-problem tests find the generated algorithm significantly better than qParEGO on 7 of the 12 problems and never worse, matching state-of-the-art accuracy at an order-of-magnitude lower cost. On the three unseen real-world engineering problems a generated algorithm attains the best mean normalized hypervolume (0.985, vs. 0.971 for qParEGO)–significantly better than qParEGO on two of the three problems–at roughly 3.4x lower wall-clock cost, confirming that the gains transfer beyond the synthetic regime. LLM-driven evolutionary search can thus discover algorithm designs that achieve Pareto-efficient trade-offs difficult to reach through manual design.
[AI-53] Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices
链接: https://arxiv.org/abs/2607.08786
作者: Tao Lu,Haoyu Wang,Zonghui Wang,Keshen Xiang,Jiaheng Zhang,Wenzhi Chen
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR)
备注: DAC 2026
Abstract:With the growing deployment of large language models (LLMs), LLM inference cost has become a key challenge. Pruning techniques that introduce sparsity into weight matrices can accelerate inference. However, maintaining model quality typically limits pruning to moderate unstructured sparsity (around 50%). At these sparsity levels, none of the existing GPU kernels for sparse matrix multiplication (SpMM) can outperform their dense counterparts. This paper proposes an efficient GPU inference method for LLMs with moderate sparsity. We propose a three-layer matrix storage format comprising: (i) a Sparse-TC layer enabling sparse tensor cores to accelerate SpMM; (ii) a Slot-Filling layer using parallel differential distance for matrix compression while supporting low-cost on-chip decoding; (iii) a lightweight Residual Layer ensuring correct SpMM computation. Building on this format, we design a SpMM kernel that jointly utilizes sparse tensor cores and CUDA cores. This design enables an efficient execution pipeline and overlaps on-chip computation with memory access. Evaluations show that our work is the first to outperform dense matrix multiplication on modern GPUs equipped with high-bandwidth memory (HBM). It achieves up to 1.64x kernel-level speedup over SpInfer (EuroSys’25, Best paper) and up to 1.41x end-to-end speedups over FlashLLM (VLDB’24). Our source code: this https URL.
[AI-54] DaDaDa: A Dataset for Data Pricing in Data Marketplaces
链接: https://arxiv.org/abs/2607.08785
作者: Qiheng Sun,Hongwei Zhang,Junxu Liu,Xiaokai Mao,Jinfei Liu,Kui Ren,Haibo Hu
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:High-quality data drives machine learning advances across industries. Recognizing the value of data, data transactions are increasingly common, giving rise to many data marketplaces, e.g., AWS Marketplace, Databricks, and Datarade. However, determining the appropriate prices for data products remains a significant challenge due to the unique properties of data products. Traditional pricing methods in economics can be categorized into the cost approach, the income approach, and the sales comparison approach. The cost approach fails in data pricing due to near-zero marginal cost from data replication, and the income approach fails due to inherently unpredictable data revenue. The sales comparison approach remains viable, yet its application is hindered by the absence of standardized pricing benchmarks for data products across marketplaces. To address this challenge, we introduce \textttDaDaDa, the first dataset for data product pricing, containing metadata for 16,147 data products from 9 major data marketplaces worldwide. \textttDaDaDa enables the training of pricing models, thereby establishing price benchmarks for new data products. In addition, \textttDaDaDa can be utilized for other important tasks in data markets, such as data product classification and retrieval. Experiments and a retrieval prototype demonstrate the effectiveness of \textttDaDaDa for pricing, classification, and retrieval of data products. The dataset and code are available at this https URL.
[AI-55] HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning
链接: https://arxiv.org/abs/2607.08784
作者: Thinh T. H. Nguyen,Le-Tuan Nguyen,Minh-Duong Nguyen,Nhi Trinh,Anh Tran Nam Nguyet,Dung D. Le,Kok-Seng Wong
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)
备注: 30 pages, 10 figures
Abstract:Federated continual learning (FCL) evaluates how distributed clients learn from changing data streams while retaining previously learned knowledge. Existing evaluations are difficult to compare because they often change datasets, task splits, client data splits, task orders, backbones, memory assumptions, and reporting rules simultaneously. We introduce \textbfHERO, a heterogeneity-aware benchmark library for FCL. HERO builds benchmark streams by separating three choices that are often coupled, namely the task split, the client data split, and the client task sequence. In HERO-Core, the main comparable benchmark, \alpha controls client data skew and \rho controls task-order mismatch. We evaluate representative FCL methods on CIFAR-100 and TinyImageNet using final average accuracy, average forgetting, and bottom-10% client accuracy. We also include a graph-based Domain-IL portability case study on OGB-MolPCBA, where scaffold-domain granularity changes the input distribution while the prediction task remains fixed. Our results show that method behavior changes across easy and heterogeneous settings, that average accuracy can hide weak bottom-client performance, that task-order mismatch favors different strategies from synchronized evaluation, and that the same HERO interface can expose domain-shift difficulty beyond image-based FCIL. HERO releases benchmark streams, configurations, method implementations, and reporting scripts to support reproducible and setting-aware FCL evaluation.
[AI-56] LieBN: Batch Normalization over Lie Groups
链接: https://arxiv.org/abs/2607.08783
作者: Ziheng Chen,Yue Song,Rui Wang,Xiao-Jun Wu,Nicu Sebe
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: arXiv admin note: text overlap with arXiv:2403.11261
Abstract:Manifold-valued measurements are prevalent in various machine learning tasks. Recent advances have extended Deep Neural Networks (DNNs) to operate on manifolds, accompanied by normalization techniques tailored to different geometries, collectively referred to as Riemannian normalization. However, most existing Riemannian normalization methods are either designed for specific manifolds or fail to effectively normalize manifold-valued sample distributions. To address these limitations, we propose LieBN, a framework for Riemannian Batch Normalization (RBN) over Lie groups. Our approach leverages the theoretically convenient left- and right-invariant metrics, which naturally exist in every Lie group, and provides theoretical guarantees for controlling the Riemannian mean and variance. We instantiate LieBN across nine distinct geometries: four on the Symmetric Positive Definite (SPD) manifold, one on the group of rotation matrices, and four on the manifold of full-rank correlation matrices. Notably, among the SPD metrics, we introduce a novel right-invariant metric and extend three existing Lie group structures via matrix power deformation. Extensive experiments on different manifolds validate the effectiveness of our framework. The code is available at this https URL.
[AI-57] Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement
链接: https://arxiv.org/abs/2607.08782
作者: Qianli Liu,Kaibin Guo,Zicong Hong,Peng Li,Fahao Chen,Haodong Wang,Jian Lin,Song Guo
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: INFOCOM 2026
Abstract:Expert parallelism has become the prevailing paradigm to serve Mixture-of-Experts (MoE) models. Its efficiency depends on the communication and computation latencies of the GPUs, which are linked to the placement of experts in the GPUs. Existing works for optimizing expert placement focus on leveraging past requests’ expert activation patterns. However, they demonstrate deficiencies facing diverse and rapidly changing request patterns, calling for an online, proactive approach. Implementing such an approach requires addressing several challenges: the uncertainty associated with incoming requests’ expert activation, the cost of expert migration, and the NP-hard complexity in optimization. Therefore, we present Director, a new distributed MoE serving system that minimizes end-to-end latency via prediction-driven, online expert placement. Director uses either a lightweight cascaded predictor or a low-bit quantized replica for expert activation patterns of incoming requests. An online migration module then enacts the changes with near-zero downtime by executing migrations in compute-bound phases, keeping disruption bounded. At its core, a relaxation-based expert placement optimizer operates under capacity constraints, runs in polynomial time, and achieves a (1+\epsilon) approximation ratio. Finally, we implement a prototype and demonstrate, through extensive experiments, a reduction in end-to-end latency of 11\sim55% for popular MoE models (e.g., Mistral, DeepSeek and Qwen) compared to existing work.
[AI-58] Reward Transport: Property Control in Flow Matching via Noise-Space Alignment
链接: https://arxiv.org/abs/2607.08781
作者: Kehan Guo,Yili Shen,Yujun Zhou,Yue Huang,Chujie Gao,Shiyi Du,Xiangliang Zhang
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM)
备注:
Abstract:The coupling in flow matching – the rule pairing noise vectors with data points – is typically treated as a computational choice. We show that this coupling can instead serve as an alignment interface: by matching noise and data according to a target molecular property, it embeds controllable structure directly into the learned flow field. Building on this view, we introduce Reward Transport, which uses optimal transport coupling at training time to align a scalar noise-space coordinate with molecular rewards; at inference, varying this coordinate steers the generated distribution without requiring an oracle, reward model, gradient guidance, or additional computation. In the coupling-preserving limit, thresholding this coordinate recovers the Cross-Entropy Method’s truncated reward distribution, providing a principled, continuously adjustable distribution-level control knob. Empirically, on ZINC-250K and GuacaMol, sweeping the scalar induces monotone control of logP and consistent QED control over its operating range; most tellingly, the same knob produces opposite structural responses for different targets, growing molecules for logP but shrinking them for QED, which rules out a generic size bias. The interface is complementary to classifier-free guidance and conditional flow matching, while a negative result under epsilon-prediction diffusion clarifies where coupling-level alignment is structurally absent. Code: this https URL
[AI-59] Signed Symmetric Quantization for Few-Bit Integers
链接: https://arxiv.org/abs/2607.08779
作者: Ian Colbert,Eashan Dash,Pablo Monteagudo-Lago,Juan Amboage,Srinidhi N,Giuseppe Franco,Nicholas J. Fraser,Arun Ramachandran
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:The signed integer alphabet contains one more negative representable value than positive. Yet, by convention, the standard symmetric integer quantizer fixes its scale to be strictly positive, which assigns this extra representable value to the negative tail and can force clipping of positive outliers. In this work, we show that, at few-bit precision, such clipping is a non-trivial source of quantization error. Asymmetric quantization addresses this problem with a zero point, shifting the grid toward the observed data range; however, this flexibility is well-known to carry a runtime penalty. For example, in this http URL on an AMD EPYC™ “Turin” CPU, a 4-bit symmetric format uses up to 9% less memory with up to 2.45 \times higher throughput than its asymmetric counterpart. We highlight signed symmetric quantization as a third option that retains the runtime profile of symmetric quantization without the penalty of the asymmetric format: our signed absmax grid places the extra representable value on the dominant-outlier tail through a principled and lightweight sign selection rule while keeping the zero point at zero. Our theoretical analysis offers two main results. First, we establish the signed absmax grid as conditionally bound-optimal on \ell_2 quantization error, and show that the condition holds for 88-99% of weight groups across pre-trained large language models (LLMs) at low bit widths. Second, we show that negating the scale of a standard symmetric quantizer is analytically equivalent to a unit zero point shift on the same signed integer alphabet. We empirically validate our proposal on models from the Qwen3, Qwen3.5, and Llama3 families, and observe improvement in perplexity and downstream few-shot accuracy over the standard unsigned symmetric quantizer at no extra inference cost
[AI-60] LENS: Interpretable LLM -Guided Mixture-of-Experts for Neuroimaging Survival Analysis
链接: https://arxiv.org/abs/2607.08778
作者: Farica Zhuang,Seong Woo Han,Zixuan Wen,Shu Yang,Yize Zhao,Li Shen
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Alzheimer’s Disease (AD) is a complex neurodegenerative disorder that continues to impact millions of people worldwide. Predicting AD conversion during the prodromal stage remains critical for disease understanding and patient care. As such, survival models are widely used for AD risk prediction, yet they are typically static predictors with limited interpretability and no capacity for natural language reasoning. In this work, we propose iLENS, an interpretable large language model (LLM) guided framework based on mixture-of-experts (MoE) for survival prediction in AD conversion. Our approach uses LLM to synthesize structured neuroimaging measurements and unstructured information to guide expert routing. Our framework demonstrates competitive predictive performance and capability in patient subtyping. Furthermore, our framework provides transparent, biologically grounded rationales for its routing decisions, bridging the gap between high-performance survival analysis and interpretable clinical decision support.
[AI-61] Interval Certifications for Multilayered Perceptrons via Lattice Traversal
链接: https://arxiv.org/abs/2607.08773
作者: Merkouris Papamichail,Konstantinos Varsos,Giorgos Flouris,João Marques-Silva
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:In this work we present a rigorous theoretical framework to a foundational problem of AI safety, namely adversarial robustness. In particular, we show that the adversarial robustness problem can be reduced to a lattice traversal problem. Each element of this lattice corresponds to an interval, i.e., an axis-aligned hyper-rectangle, containing an input point \mathbfx . Consider a multilayered perceptron classifier (MLP). An interval I constitutes a sound certification if \mathbfx \in I and \mathbfx can be freely perturbed in I without changing the MLP’s prediction. Complementarily, an interval I constitutes a complete certification if \mathbfx \in I and when \mathbfx moves outside of I the MLP’s prediction is guaranteed to change. While the sound certification problem corresponds to the well-studied adversarial robustness, complete certifications have not been examined in the literature. We develop lattice traversal operators, which we apply in a refine verify iterative scheme. Using formal MLP verifiers, sound maximality and complete minimality are guaranteed. Moreover, we examine objective optimization problems. There we discover some interesting asymmetries. For complete certifications, the minimum solution is obtained in polynomial oracle calls. This does not hold for sound certifications, where we prove strong intractability results. Additionally, we examine optimization problems in symmetric intervals (i.e., \ell_\infty -spheres), where we provide logarithmic algorithms. Finally, we present an empirical evaluation, using the novel ParallelepipedoNN system.
[AI-62] REFORGE: A Method for Benchmarking LLM s Reverse Engineering Capabilities in Decompiled Binary Function Naming
链接: https://arxiv.org/abs/2607.07738
作者: Nicolas Koller,Andreas u. Schmidt
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Programming Languages (cs.PL)
备注: 9 pages, 4 figures; accepted for publication to the 23rd International Conference on Applied Computing 2026, Lisbon October 24-26,2026
Abstract:Large language models (LLMs) are increasingly applied to reverse-engineering tasks, and recent threat-intelligence reporting shows them operating inside live offensive-security workflows. Claims about their capability, however, outpace our ability to measure it. Existing benchmarks for LLM-assisted binary analysis treat the construction of function-level ground truth as a solved pre-processing step and report accuracy without disclosing how many functions were reliably evaluable. We argue that the principal obstacle to fair evaluation is not model capability but the reliability of binary-to-source alignment under compiler optimization. This paper presents Reforge, a provenance-tracked pipeline that constructs function-level ground truth from C source through compilation, DWARF and syntactic extraction, alignment, and decompilation, and that operationalizes alignment uncertainty as an eight-gate confidence funnel with three-tier stratification. On a controlled micro-benchmark, high-confidence yield falls from 87.2% to 65.9% across optimization levels, and unpaired comparisons overstate optimization-induced performance decay through survivorship bias. A proof-of-concept evaluation of seven contemporary LLMs on function naming demonstrates the substrate and motivates uncertainty-aware benchmarking practice
[AI-63] PHINN-EEG: Topological Time-Series Analysis of Dream-State EEG – Dynamic Betti Curves for Dream Content Classification and Topology-Conditioned Neural Signal Synthesis
链接: https://arxiv.org/abs/2607.09662
作者: Ren Takahashi,Emre Yusuf,Jayabrata Bhaduri
类目: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Signal Processing (eess.SP); Algebraic Topology (math.AT)
备注:
Abstract:Current electroencephalography (EEG)-based dream detection relies on power spectral density (PSD) and statistical moment features, achieving a state-of-the-art area under the receiver operating characteristic curve (AUC) of approximately 0.70 on the DREAM database (Wong et al., 2025, Nature Communications). We introduce PHINN-EEG (Persistent Homology Inspired Neural Network for EEG), the first topological time-series framework for dream mentation analysis. Using sliding-window Takens delay embeddings and Vietoris-Rips filtrations on multichannel pre-awakening EEG epochs, we extract Dynamic Betti Curves that characterize the geometric architecture of neural activity, not merely its energy. These topological invariants, combined with topology-conditioned flow matching, are analytically projected to outperform existing PSD and catch22 benchmarks, targeting AUC = 0.82-0.90 on the 1,462-awakening open-access subset of the DREAM database (drawn from a full registry of 3,191 total awakenings from 263 participants across 20 independent laboratories). We further introduce a topology-conditioned rectified flow model for dream-state EEG synthesis-with a spectral-conditioned flow model of comparable feature dimensionality as an additional ablation baseline to isolate the value of topological conditioning specifically-and propose a set of candidate Betti transition archetypes linking topology to phenomenological dream report categories, presented as an exploratory hypothesis space pending empirical validation. If validated, this work represents a paradigm shift from spectral energy to phase-space geometry in neural rare-event detection, with potential future implications for wearable BCI dream monitoring.
[AI-64] Lean-QIT: Towards a Formal Infrastructure for Quantum Information Theory
链接: https://arxiv.org/abs/2607.09632
作者: Chengkai Zhu,Ziao Tang,Guocheng Zhen,Yimeng Cao,Yusheng Zhao,Ranyiliu Chen,Xuanqiang Zhao,Lei Zhang,Xin Wang
类目: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI)
备注: 24+5 pages, 3 figures
Abstract:Quantum information theory (QIT) characterizes the capabilities and fundamental limits of quantum information processing, underpinning quantum communication, computation, and error correction. Formalizing its coding theorems requires connecting finite-block protocols, analytic inequalities, and asymptotic limits within a unified machine-checked framework. Existing developments, however, lack a reusable operational layer that defines codes, error criteria, achievable rates, and capacities independently of their information-theoretic characterizations. In this work, we present LeanQIT, a Lean 4 library for finite-dimensional QIT. It provides composable, kernel-checked interfaces for quantum states and channels, source and channel codes, finite-block performance criteria, hypothesis testing, one-shot quantities, and asymptotic rate constructions. Using this infrastructure, we formalize Schumacher’s quantum source-coding theorem, the Holevo–Schumacher–Westmoreland classical-capacity theorem, and the entanglement-assisted classical-capacity theorem together with its strong converse. By separating operational definitions from analytic characterizations and exposing reusable achievability, converse, and asymptotic components, Lean-QIT provides a machine-readable foundation for formal QIT and a compositional knowledge substrate for emerging AI-assisted formalization, automated proof search, and agentic reasoning in quantum information and computation.
[AI-65] When Routes Run Out: Adversarial Co-Learning and Explainable Robustness in Quantum Repeater Networks
链接: https://arxiv.org/abs/2607.09378
作者: Brennan Bell,Inti Gabriel Mendoza Estrada,Andreas Trügler,Paul Erker
类目: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
备注: 4 pages, 5 figures, submitted to IEEE QCE26, Workshop on Q-GenAI: Synergies between QC GenAI
Abstract:We study an adversarial bandit problem for entanglement-based quantum-network routing over a modest graph corpus. Alice selects an end-to-end repeater route for an Ekert-91 protocol (E91) representing her move, while Eve selects an attack surface, either edge intercept–resend or repeater memory degradation. Payoffs are drawn from cached SeQUeNCe-simulated E91 transcripts, and Alice accepts a turn when the finite-sample statistic violates the Clauser-Horne-Shimony-Holt (CHSH) bound. Performing adversarial co-learning across 50 structured topologies, we find that learned retention tracks a full-matrix minimax reference closely (Pearson r=0.99 ): under a one-surface Eve action model, bottleneck families have zero retention, while non-bottleneck families follow a 1-1/N coverage principle. We then fit decision-tree explanation models to graph-, attack-, and route-level topology-corpus targets and report their faithfulness. Finally, we construct prompt records for local language models to summarize the tree evidence, resulting in an open-source explanation workflow for quantum-repeater network games.
[AI-66] Quantum Logic as the Logic of Contexts
链接: https://arxiv.org/abs/2607.09032
作者: Haruki Emori,Atsushi Iriki,Andrei Khrennikov,Kazunori Kondo
类目: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI); Logic (math.LO); Neurons and Cognition (q-bio.NC)
备注: 18 pages, 13 tables
Abstract:Quantum logic is usually presented as a non-classical departure from ordinary reasoning forced on us by quantum mechanics, with classical logic kept as the secure starting point. We argue for the opposite order of explanation in a finite and fully computable setting. The free orthomodular lattice on two generators has ninety-six elements, the direct product of a six-element non-distributive factor and a sixteen-element Boolean factor. Reading the first factor as a register of contexts and the second as Boolean content, we obtain a calculus whose elements are context–bit-vector pairs and whose operations act component by component. With this calculus we establish three results. First, we classify the six layers by commutativity, identifying the central kernel of context-neutral propositions together with a dual central layer in which all complementary contexts are present. Second, we show that orthocomplementation rearranges the layers exactly as the complementation of the small factor rearranges its elements, which makes the duality among the layers rigid rather than accidental. Third, we prove that the operation forgetting the context is a surjective homomorphism of orthocomplemented lattices whose quotient is the classical Boolean algebra, so that classical logic is a six-to-one, information-losing image of the contextual calculus.
[AI-67] A Novel Parallel QCNN Architecture with Efficient Classical Simulability
链接: https://arxiv.org/abs/2607.08928
作者: Lawrence Nguyen,Hiu Yung Wong
类目: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI)
备注:
Abstract:This work presents a study of an implementation of a novel Quantum Convolutional Neural Network (QCNN) for binary classification of images from the Modified National Institute of Standards and Technology (MNIST) dataset. Using a novel architecture inspired by previous QCNN and classical convolutional neural network (CNN) implementations, we use a hierarchical partitioning approach to implement a QCNN circuit that can be approximated and simulated efficiently on a classical machine for a large problem. First, the original image is partitioned such that each process handles a smaller portion of the image, which is encoded into independent states. Then, these partitions merge and combine, resulting in states that contain information from both partitions while halving the number of processes. After repeating this until one process remains, we reduce the dimensionality of the state until a single qubit remains for measurement. Using this approach, we can use multiple processes in parallel to simulate a large QCNN program without the need for exponentially growing hardware requirements as the number of qubits increases. In our work, we use this scheme to train a 128-qubit model, which is impossible to run on any classical supercomputer without the novel architecture. We also explore the impact of this new model architecture on prediction accuracy by training it to perform binary classification on the MNIST dataset with a small number of qubits, and comparing it to a model without partitioning. Our initial findings show that partitioning images into smaller sub-images with this architecture does not degrade the model’s performance and sometimes even improves it, likely because it reduces the Barren plateaus issue in the partitioning process.
[AI-68] heBioCollection: Unified Pre-Training Scale LLM Corpus for Biology
链接: https://arxiv.org/abs/2607.08803
作者: Hyunjin Seo,Hyeon Hwang,Gyubok Lee,Jay Shin,Jimin Park,Taesoo Kim,Sanghoon Lee,Hongjoon Ahn,Sungjun Han,Sangwon Jung
类目: Quantitative Methods (q-bio.QM); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:The push toward large language models for biology (BioLM) has created a need for training corpora that can endow models with a genuine understanding of biology. However, existing biological resources, such as molecular databases, protein repositories, genomic annotations, single-cell atlases, and pathway databases, are scattered across heterogeneous formats and remain unorganized into a cohesive corpus for language model training. We present TheBioCollection, a 52.6B-token pre-training-scale corpus that converts these disparate resources into a unified, training-ready form spanning small molecules, proteins, genomic sequences, cells, and pathways. Beyond consolidating existing data, TheBioCollection enriches each record with tool-computed biological properties and introduces new instruction tasks for capabilities that current corpora barely cover. We pair the corpus with TheBioCollection-Eval, a matched suite probing recognition, generation, and prediction across molecular, protein, genomic, cellular, and cross-domain settings. Holding the base Gravity-16B-A3B architecture fixed, training on TheBioCollection more than doubles its overall score on TheBioCollection-Eval with gains in every domain, while leaving general linguistic ability nearly intact.
[AI-69] EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins
链接: https://arxiv.org/abs/2607.08793
作者: Joshua Pickard,Wei Qi,Na Li,Ann Woolley,Lisa Cosimi,Roy Kishony,Deborah Hung
类目: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Systems and Control (eess.SY); Optimization and Control (math.OC)
备注:
Abstract:Sepsis is a leading cause of mortality, yet optimal treatment policies remain contested. Existing reinforcement learning (RL) approaches learn fixed strategies for sepsis treatment, limiting adaptability to changing clinical objectives during inference. We propose EHRMPC, a framework that decouples learning patient dynamics from optimizing treatment by training a patient digital twin in the form of a generative electronic health record (EHR) model. The digital twin predicts clinical trajectories under interventions and enables model predictive control (MPC) to optimize treatments via inference-time planning over simulations. We evaluate EHR-MPC on a multicenter ICU sepsis cohort spanning 8 hospitals in the Mass General Brigham health system using both off-policy importance sampling and on-policy simulation-based evaluation. Relative to RL baselines, EHR-MPC achieves comparable off-policy performance and improved simulation performance. Unlike RL, this work frames sepsis treatment optimization as inference-time control over learned patient dynamics, establishing a general framework for decision making with generative clinical models.
[AI-70] Minimal Decision Dynamics and Contextual Probability: A Quantum Tug-of-War Model
链接: https://arxiv.org/abs/2601.10034
作者: Song-Ju Kim
类目: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI); Neurons and Cognition (q-bio.NC)
备注: 47 pages, 3 figures
Abstract:Decision making often exhibits context dependence that challenges classical probability theory. This paper develops a quantum-like extension of the Tug-of-War (QTOW) decision-making model to clarify when such context dependence can be represented by a single minimal internal state. The QTOW construction uses a qutrit internal state, conservation-preserving updates, and measurement-induced disturbance to model decision, learning, and probing operations within one coherent state space. Within this minimal representation, KCBS-type probing contexts can be constructed, yielding a witness of non-contextual classical non-embeddability. The main claim is not that quantum theory is uniquely or assumption-freely derived from decision making. Rather, a classical reconstruction of the same operation family requires additional contextual memory, history dependence, or an enlarged hidden-state representation. Thus, contextual probability appears as a resource signature of minimal decision dynamics, while quantum probability provides a compact, memory-efficient realization of this structure.
机器学习
[LG-0] LLM for EDA in Front-End Design: Challenges and Opportunities
链接: https://arxiv.org/abs/2607.09616
作者: Kangwei Xu,Bing Li,Ulf Schlichtmann
类目: Emerging Technologies (cs.ET); Hardware Architecture (cs.AR); Machine Learning (cs.LG); Systems and Control (eess.SY)
*备注: Invited paper at the ACM/IEEE DAC 2026 Special Research Session, 5 pages, 9 figures
Abstract:As chip complexity increases and time-to-market pressures grow, front-end design has become a critical bottleneck in chip development. Recently, Large Language Models (LLMs) have shown great potential in Electronic Design Automation (EDA). Beyond specification understanding, LLMs show the potential to serve as a unified intelligent interface for hardware description language (HDL) generation, testbench construction, and design space exploration. The rise of agentic AI, represented by pioneering systems such as OpenClaw, offers a strategic roadmap for the next generation EDA. From this perspective, this paper discusses the evolution of EDA from localized assistance to autonomous agentic execution. Then, we review representative advances of LLMs in front-end design, focusing on key tasks such as circuit and testbench generation from a shared specification, as well as design quality improvement in established workflows such as high-level synthesis. Finally, we discuss the key challenges and limitations of integrating LLMs into EDA, and outline future opportunities for advancing LLM-enabled front-end design, offering a systematic perspective for researchers interested in leveraging agentic AI technologies for EDA.
[LG-1] Graph-Regularized Low-Rank Matrix Completion by Variable Projection
链接: https://arxiv.org/abs/2607.09546
作者: Benoît Loucheur,P.-A. Absil,Michel Journée
类目: Machine Learning (cs.LG); Numerical Analysis (math.NA); Optimization and Control (math.OC)
*备注:
Abstract:We address the low-rank matrix completion problem by incorporating graph regularization into the existing Riemannian Trust-Region Matrix Completion (RTRMC) framework. The latter uses the geometry of the low-rank constraint to remodel the problem as an unconstrained optimization problem on a single Grassmann manifold. Our approach, named Graph-Regularized RTRMC (GR-RTRMC), exploits the inherent relationships between rows and columns of the matrix. By using these relationships, we aim to improve the accuracy and robustness of matrix completion, particularly in scenarios where the underlying data exhibits strong correlations between rows or columns.
[LG-2] CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding
链接: https://arxiv.org/abs/2607.09543
作者: Gabriel Mahuas,Victoria Shevchenko,Ugo Tanielian,Yassir Bendou,Richard Gao
类目: Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC)
*备注:
Abstract:Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications. Many recent large-scale models converged on the approach of tokenizing raw EEG followed by masked reconstruction pretraining. However, this recipe has been shown to be suboptimal for data, like EEG, with high noise amplitude and information confined to limited dimensions such as narrow frequency bands. Building on this insight, we develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT). CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations. Furthermore, CoCoT trained from scratch outperforms previous single-task decoding models and even rivals pretrained models, showcasing the architecture’s flexibility and data efficiency. Through systematic ablations, including model architecture and pretraining objective, we demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations, prompting further investigations in alternative large-scale pretraining strategies.
[LG-3] GatedLinear: Adaptive Routing of Complementary Linear Bases for Time Series Forecasting
链接: https://arxiv.org/abs/2607.09537
作者: Qitai Tan,Ruiwen Gu,Yilin Su,Mo Li,Xu Lin,Xiao-Ping Zhang
类目: Machine Learning (cs.LG)
*备注:
Abstract:Time series forecasting requires models to capture diverse, often mutually exclusive, temporal dynamics, from smooth trend continuation to nonstationary drift and strict phase-aligned recurrence. While recent deep learning models have improved accuracy, they typically force these diverse patterns through a single computational backbone governed by fixed algorithmic inductive biases (e.g., self-attention or spectral filtering). This single-mechanism approach often struggles with the profound heterogeneity of real-world series, where different variables and forecast horizons necessitate fundamentally different predictive treatments. To address this, we propose GatedLinear: a lightweight framework that frames forecasting as the adaptive routing of complementary linear bases. GatedLinear leverages a pool of three specialized mechanisms: a global trend-seasonal basis for smooth projection, a difference-based incremental basis for nonstationary drift, and a phase-aligned recurrence basis for explicit cyclic reuse. To dynamically orchestrate these distinct behaviors, we introduce a Tri-Factorized Fusion Gate that disentangles routing decisions into channel-specific preferences, horizon-aware offsets, and phase-indexed biases derived from known future time marks. This design allows the model to perform highly granular, point-wise soft routing across different predictive regimes without stacking computationally heavy neural modules. Experiments on standard benchmarks show that our method achieves state-of-the-art or highly competitive accuracy against recent complex foundational models, while offering explicitly interpretable routing patterns and operating with a substantially smaller parameter footprint.
[LG-4] Statistically Undetectable Backdoors in Deep Neural Networks ICML2026
链接: https://arxiv.org/abs/2607.09532
作者: Andrej Bogdanov,Alon Rosen,Neekon Vafa
类目: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML)
*备注: ICML 2026
Abstract:We show how an adversarial model trainer can plant backdoors in a large class of deep, feedforward neural networks. These backdoors are statistically undetectable in the white-box setting, meaning that the backdoored and honestly trained models are close in total variation distance, even given the full descriptions of the models (e.g., all of the weights). The backdoor provides access to invariance-based adversarial examples for every input, mapping distant inputs to unusually close outputs. However, without the backdoor, it is provably impossible (under standard cryptographic assumptions) to generate any such adversarial examples in polynomial time. Our theoretical and preliminary empirical findings demonstrate a fundamental power asymmetry between model trainers and model users.
[LG-5] SAI-MetaFraud: A Benchmark Dataset for Financial Fraud Transaction and Behavioral Risk Detection in Metaverse Ecosystems
链接: https://arxiv.org/abs/2607.09528
作者: Refat Ishrak Hemel,Ehsan Hallaji,Roozbeh Razavi-Far
类目: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Databases (cs.DB); Social and Information Networks (cs.SI)
*备注:
Abstract:The emergence of metaverse platforms has created virtual economies that introduce new challenges related to fraud, bot activity, and illicit financial behavior. Despite growing interest in trustworthy metaverse analytics, existing datasets typically focus on user behavior, authentication, or financial transactions in isolation, limiting the development and reproducible evaluation of multimodal fraud detection methods. To address this gap, we present TSAI-MetaFraud, a multimodal, multi-task benchmark dataset for fraud analytics in virtual economies. TSAI-MetaFraud integrates behavioral, transactional, and graph-structured information while incorporating realistic fraud and automated bot scenarios. We define benchmark tasks including transaction fraud detection, cross-modal node classification, temporal link prediction, and weakly supervised fraud detection, and provide baseline evaluations using machine learning models and graph neural networks. By jointly capturing behavioral activity, financial interactions, and relational structure within a unified virtual economy, TSAI-MetaFraud provides a benchmark for advancing multimodal learning, graph mining, fraud analytics, and trustworthy AI in emerging metaverse ecosystems.
[LG-6] rminal Dimension Reduction for Time Series with Applications ICML2026
链接: https://arxiv.org/abs/2607.09490
作者: Alexander Munteanu,Matteo Russo,David Saulpic,Chris Schwiegelshohn
类目: Data Structures and Algorithms (cs.DS); Computational Geometry (cs.CG); Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注: ICML 2026
Abstract:Terminal embeddings have emerged as a powerful tool for dimension reduction. Given a set of points P\subset \mathbbR^d , a terminal embedding is a mapping f:\mathbbR^d\rightarrow \mathbbR^t that preserves the pairwise distance between any pair of points p\in P and q\in \mathbbR^d up to small distortion under this mapping. Terminal embeddings have been particularly fruitful for constructing k -means and k -median coresets, where the objective is to find a typically weighted subset \Omega of P such that for any candidate solution, the cost of the clustering objective on \Omega approximates the cost of the clustering objective on P up to small distortion. Unfortunately, these techniques have not been extended to more complicated structures such as clustering time-series data under common straight-line interpolation between measurements. The main issue is that terminal embeddings, arguably the central technique in this line of research, cannot be linear and are thus not immediately suitable to preserve linear structures. In this work, we develop a generalization of terminal embeddings to affine line-segments that overcomes this issue. We showcase their applicability by using our lines-preserving terminal embeddings to obtain the first dimension-free coresets for clustering time-series under the Fréchet distance. The underlying dimension reduction uses Johnson-Lindenstrauss (JL) embeddings, and our experiments indicate that terminal embeddings perform similarly to JL and favorably against PCA for synthetic and real-world time-series, while only terminal embeddings extend pairwise distance preservation to the full ambient space.
[LG-7] Active rejection enables reliable generalization of universal machine-learning interatomic potentials
链接: https://arxiv.org/abs/2607.09456
作者: Mingxiang Luo,Xinnan Mao,Lu Wang,Lei Bai,Feng Ding,Yuqiang Li
类目: Machine Learning (cs.LG)
*备注:
Abstract:Universal machine learning interatomic potentials (uMLIPs) bridge quantum-mechanical accuracy and large-scale molecular dynamics, but the cost of high-accuracy calculations such as r ^2 SCAN limits training to datasets that remain small relative to the open materials space. Strong average benchmark performance also does not guarantee reliable energy–force predictions for every structure. We propose Adaptive Multi-Teacher Routing (ATR), which reformulates high-fidelity data construction as a structure-wise decision problem under uncertainty. Using a small set of real r ^2 SCAN labels, ATR calibrates multiple pretrained uMLIP teachers and combines structural descriptors, teacher identity, and inter-teacher disagreement to estimate the reliability of each structure–teacher pair. It selects high-confidence predictions for pseudo-label generation and rejects structures for which no teacher is sufficiently reliable. With real r ^2 SCAN labels for only 0.2% of candidate structures, ATR distils 2.89 million traceable r ^2 SCAN-level pseudo-labels for pretraining. On held-out r ^2 SCAN structures and the MP-r ^2 SCAN benchmark, a lightweight CHGNet trained on the ATR-generated dataset consistently outperforms the baseline and non-routed controls. Finite-temperature molecular dynamics further shows that ATR improves dynamical robustness across multiple material systems, maintaining stable trajectories where baseline simulations undergo catastrophic structural collapse. These results establish active rejection as an effective mechanism for converting multiple pretrained uMLIPs into a scalable and reliable data-construction system for high-fidelity uMLIPs.
[LG-8] Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuning
链接: https://arxiv.org/abs/2607.09422
作者: Edwin De Nicolo,Rahul Marchand,Cornelius Carlsson,Pranav Vaidhyanathan,Natalia Ares
类目: Machine Learning (cs.LG); Mesoscale and Nanoscale Physics (cond-mat.mes-hall)
*备注:
Abstract:Cooperative multi-agent reinforcement learning is well suited to problems with large parameter spaces and exploitable local structure, such as the tuning of electrostatically-defined quantum-dot arrays. However, if parameter cross-talk is strong, a non-stationary environment from the perspective of any individual agent can destabilize learning - the same effect that plagues manual tuning of such systems. We propose using a factored representation of the action space, learned online, to decouple agents and minimize their interference. Our framework, QADAPT, uses this factorization to efficiently learn shared policies based on local measurements and rewards. With this modular strategy, we achieve zero-shot generalization to unseen quantum device sizes and maintain an approximately constant number of convergence steps to reach target regimes. This work provides a scalable route toward the rapid calibration of large-scale quantum processors.
[LG-9] Similarity search generalisation in contrastive learning with InfoNCE loss
链接: https://arxiv.org/abs/2607.09405
作者: Nick Whiteley
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:Similarity search is a primary application of embedding models trained by contrastive learning. For one of the most popular contrastive learning loss functions, InfoNCE, we show that the population risk with k negative samples is O(1/k) close to an expected cross-entropy which quantifies deviation between i) a softmax similarity search over unseen data using the learned embedding function, and ii) an idealised softmax search over the same data but using similarity implicitly represented in the positive sample generator. This complements existing interpretations of InfoNCE in the k\to\infty limit which are phrased in terms of mutual information, and alignment versus uniformity in embeddings. To quantify generalisation performance, we introduce a new continuity bound for the InfoNCE loss, obtained via Gâteaux differentiation. The bound preserves the structure of averaging over negative samples present in the loss function and features an ``inverse temperature’’ parameter which can be tuned to account for the algorithmic temperature. For embedding functions which are Lipschitz in a parameter, this yields a simple demonstration that the averaging effect of k negative samples in the InfoNCE loss carries over to stabilisation of the generalisation error as k grows.
[LG-10] SYNRARE: Synthetic Rare Disease EHR Generation for ML Benchmarking
链接: https://arxiv.org/abs/2607.09404
作者: Nicolai Dinh Khang Truong,Richard Röttger
类目: Machine Learning (cs.LG)
*备注: Intended for submission to the Application Notes Bioinformatics Journal
Abstract:Motivation: Rare disease (RD) diagnosis is frequently delayed due to the similarities in symptoms to common disease variants. Machine Learning Algorithms applied to Electronic Health Records show promise for accelerating the diagnosis; however, legal and privacy concerns pose significant barriers. To address these issues, Synthetic Data Generation is an alternative method for obtaining Electronic Health Records and can be applied with any Machine Learning algorithm for benchmarking and development purposes. Despite the availability of Synthetic Data Generation algorithms, support for generating a subset of patients that differ in a definable degree from the majority to simulate patients with RD is often lacking. Results: We present SYNRARE, a graphical user interface based on the Synthea framework that enables easier modification and generation of synthetic Electronic Health Records of RD patients, which differ only to a definable degree from patients with common diseases, thereby enabling the benchmarking and testing of algorithms under controlled technical conditions. SYNRARE enables researchers to rapidly benchmark their Machine Learning algorithms across any scenario. Availability and implementation: SYNRARE, including detailed instructions for installing, is available at this https URL. Comments: Intended for submission to the Application Notes Bioinformatics Journal Subjects: Machine Learning (cs.LG) Cite as: arXiv:2607.09404 [cs.LG] (or arXiv:2607.09404v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2607.09404 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-11] Data-Efficient Deep Learning: Empirical Guidelines for Training Set Size Estimation in Inertial Sensor Classification
链接: https://arxiv.org/abs/2607.09402
作者: Ofir Kruzel,Itzik Klien
类目: Machine Learning (cs.LG)
*备注: 1 pages, 17 figures, 15 tables
Abstract:Deep learning models dependency on large-scale inertial datasets presents a significant bottleneck in inertial sensor-based classification tasks, such as human activity recognition and smartphone location recognition. In these domains, data collection requires massive recording campaigns that are complex, time-consuming, and difficult to scale. Currently, data-driven guidelines for determining the minimum sample size required to reach a desired accuracy level do not exist. To address this gap, this study presents a systematic empirical evaluation of learning curve convergence rates in inertial classification. We introduce a unified framework that analyzes classification performance under both binary and multi-class scenarios, and derive an empirical formula to estimate performance relative to dataset size. Testing across six diverse, real-world datasets totaling 102.7 hours of inertial measurements demonstrates that accuracy follows a consistent logarithmic growth pattern, regardless of task complexity. Leveraging this finding, we propose a quantitative stability point metric, defined as the sample size required for the learning curve to stabilize within a predefined mean absolute percentage deviation of its asymptotic maximum. Our analysis reveals that models often reach practical stability with substantially fewer samples than traditional heuristics suggest. Ultimately, we offer a generalizable framework to extrapolate total data requirements from small-scale pilot studies, optimizing the tradeoff between recording effort and model reliability. These findings shift the prevailing paradigm from maximizing data volume toward optimizing data efficiency, offering concrete, data-backed guidelines for planning recording campaigns in inertial sensing applications.
[LG-12] Learning Physics-Informed Surrogate Model of Linear Elastic Displacement Fields from Geometry
链接: https://arxiv.org/abs/2607.09382
作者: Rodolphe Barlogis,Ferhat Tamssaouet,Quentin Falcoz,Stéphane Grieu
类目: Machine Learning (cs.LG)
*备注: 6 pages
Abstract:This work aims to develop a fast and physically consistent surrogate model for real-time structural health monitoring of fractured elastic domains. We propose a physics-informed DeepONet framework that predicts displacement fields from both boundary conditions and fracture geometry, using a dedicated encoding strategy for the latter and without relying on finite-element-generated training data. The traction-free condition on the fracture boundary is imposed weakly through a localized penalty term. The presented numerical example focuses on one representative fracture geometry, demonstrating the feasibility of the formulation and laying the groundwork for extensions to surrogate modeling across diverse fracture geometries.
[LG-13] Graph Neural Networks for Scalable and Transferable Node Centrality Approximation
链接: https://arxiv.org/abs/2607.09372
作者: Samra Sana,Giorgio Mantica,Saul Imbrici
类目: Machine Learning (cs.LG)
*备注: 22 pages, 5 figures
Abstract:Graph Neural Networks (GNNs) provide a learning-based framework for approximating graph quantities that are expensive to compute exactly. This paper investigates GNNs for scalable approximation of betweenness and closeness centrality, formulated as a node-ranking problem. Exact centrality values are used as supervision, and ranking quality is evaluated using Kendall’s tau rank correlation. We study whether message-passing GNNs can learn transferable structural representations across different graph topologies rather than only fitting the distribution used during training. On unseen Erdos renyi graphs, the proposed models achieve tau = 0.851 for betweenness and tau = 0.894 for closeness. A large-scale betweenness model trained on graphs with N = 5,000 nodes achieves tau = 0.938, demonstrating scalability. Mixed-distribution training on Erdos renyi, Barabasi-Albert, and Gaussian Random Partition graphs improves betweenness transfer across graph families. In contrast, closeness centrality remains more sensitive to community-structured graphs and shows reduced transfer to real-world topologies. Finally, GNN inference achieves up to a 97.7x speedup over exact computation. These results show that mixed-distribution training can improve structural transfer in GNN-based centrality approximation, while identifying closeness centrality’s sensitivity to topology as an open challenge.
[LG-14] Leverag ing Interpretable Tsetlin Machine for PDF Malware Detection
链接: https://arxiv.org/abs/2607.09290
作者: Rahul Jaiswal
类目: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
*备注: 7 pages, 15 figures, 6 tables
Abstract:In the digital era, Portable Document Format (PDF) is one of the most widely used file formats for storing and exchanging digital documents due to its platform independence and rich functionality. However, these same capabilities have also made PDF files an attractive attack vector for cyberattackers, who embed malicious code within seemingly legitimate documents to compromise target systems. This paper presents a novel interpretable Tsetlin Machine ™-based framework for PDF malware detection. The proposed framework extracts salient features from PDF documents through static analysis without executing the files and employs rule-based learning to accurately classify benign and malicious PDF documents. Numerical evaluation on the RIT-PDFMal-2026 dataset demonstrates that the proposed framework achieves competitive performance, attaining an accuracy of 98.02% compared with several ML classifiers and existing methods. Moreover, the proposed framework provides intrinsic interpretability by transparently explaining its classification decisions. The combination of competitive detection performance, computational efficiency, and intrinsic interpretability makes the proposed framework a promising solution for practical PDF malware detection.
[LG-15] Autoregressive latent diffusion for 3D molecule generation
链接: https://arxiv.org/abs/2607.09277
作者: Federico Ottomano,Gaopeng Ren,Yingzhen Li,Kim E. Jelfs,Alex M. Ganose
类目: Machine Learning (cs.LG)
*备注:
Abstract:Three-dimensional (3D) molecule generation has been dominated by diffusion models, which achieve strong generation quality but typically require the molecular size to be specified a priori. Recent autoregressive approaches have substantially narrowed the performance gap while naturally supporting variable-length generation and conditioning on partial molecular context. However, balancing unconditional and context-conditioned generation remains challenging. We introduce KRONOS, a latent autoregressive diffusion framework that generates molecules in the latent space of a pre-trained autoencoder, jointly modeling molecular graph topology and geometry, while retaining the flexibility of autoregressive generation. We further introduce a mixed training strategy inspired by Fill-in-the Middle (FIM) paradigm, enabling both unconditional and fragment-conditioned molecular generation within a single left-to-right autoregressive model. Experiments on QM9 and GEOM-Drugs demonstrate that KRONOS achieves leading unconditional generation performance among autoregressive methods, while remaining competitive with diffusion models. Moreover, fragment-conditioned generation is achieved with negligible impact on unconditional generation performance, demonstrating that both generation paradigms can be supported within a single architecture.
[LG-16] LionVote: Per-Layer Learning Rate Adaptation for Lion
链接: https://arxiv.org/abs/2607.09266
作者: Kris Atallah(New York University, New York, USA)
类目: Machine Learning (cs.LG)
*备注: 34 pages, 10 figures
Abstract:Per-layer diagnostics reveal that, at the prescribed learning rate, Lion’s effective scale is 2.6-2.8x too high for attention and MLP parameters and ~2x too high for normalization layers on ViT-Tiny/CIFAR-100; this 32% cross-layer-type disparity cannot be reproduced by a single global rate. The measurement comes from LionVote, a per-layer learning rate mechanism in which each parameter tensor maintains a compound level, a persistent integer updated every c epochs by two diagnostics (gradient direction stability and momentum health) resolved by a validation loss tiebreaker. Voting thresholds derive from geometric identities, the EMA time constant, and a noise-floor estimate; cadence is bounded structurally and selected by ablation. On ViT-Tiny/CIFAR-100, LionVote achieves 69.7% top-1 accuracy vs. Lion’s 69.0% (p 0.02, Welch’s t-test) and AdamW’s 68.8%. Per-layer adaptation value depends on both architectural heterogeneity and task; on uniform CNN architectures tuned SGD with cosine annealing remains dominant, and on ViT architectures gains are task-dependent.
[LG-17] Forget Narrowly Retain Broadly: Unlearning as an Asymmetric Generalization Problem
链接: https://arxiv.org/abs/2607.09236
作者: Amit Peleg,Naman Deep Singh,Naama Pearl,Bibhabasu Mohapatra,Matthias Hein
类目: Machine Learning (cs.LG)
*备注:
Abstract:Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety. Yet existing benchmarks measure it unreliably. They miss knowledge that resurfaces under paraphrased or indirect queries, a failure we call under-forgetting, and lack the semantic, syntactic, and lexical probes needed to verify that unrelated knowledge is preserved, a failure we call over-forgetting. Both failures reflect an asymmetric generalization problem. Forget evaluation must cover diverse query formulations of the same target facts, testing whether forgetting holds beyond exact training prompts. Retain evaluation must probe a far larger and implicitly defined set, namely every fact disjoint from the forget target. The retain set thus defines the effective forget set, yet current datasets provide no fine-grained annotation of this forget-retain boundary. We address this with SUITE, an evaluation protocol and training corpus that captures forget-retain structure for real-world factual domains. Methods trained on SUITE improve substantially, showing that training data is as important as algorithmic design. Building on the obtained insights, we introduce JensUn++, an unlearning algorithm that achieves the best forget-retain utility trade-off across three LLMs, in both sequential and joint unlearning settings. Code and datasets are available at this https URL
[LG-18] mporal Knowledge Graph Forecasting under Distribution Shifts: A Synthetic Evaluation ECML KDD2026
链接: https://arxiv.org/abs/2607.09232
作者: Konrad Özdemir,Julia Gastinger,Lukas Kirchdorfer,Heiner Stuckenschmidt
类目: Machine Learning (cs.LG)
*备注: Accepted at ECML PKDD 2026 Workshops
Abstract:Temporal knowledge graphs (TKGs) represent evolving relational systems, whose underlying data-generating processes often change over time. Yet, TKG forecasting models are commonly evaluated only on empirical benchmark datasets that provide limited insight into the models’ robustness to such distribution shifts. Recognising this issue, we study TKG forecasting under controlled shift environments using a synthetic TKG generator that encodes three temporal and structural properties – recurrence, homophily, and periodicity – as data-generating mechanisms. This allows us to evaluate seven forecasting architectures under stationary and shifting regimes. Our experiments suggest that robustness in TKG forecasting is highly signal-dependent. Recurrence-based and periodic regularities are largely recoverable under stationary conditions, and simple memory-based baselines can be competitive when recurrence dominates the data. However, structural breaks reveal limitations in model adaptivity, with shifts in latent entity-community structure posing the strongest challenge in our study. Overall, our findings improve the understanding of the capabilities and limitations of current TKG models confronted with temporal distribution shifts.
[LG-19] Application of machine learning to monster level prediction in tabletop RPG game design
链接: https://arxiv.org/abs/2607.09196
作者: Jolanta Śliwa,Jakub Adamczyk
类目: Machine Learning (cs.LG)
*备注:
Abstract:Designing balanced adversaries is a central but labor-intensive task in tabletop role-playing game (TTRPG) development. In systems such as Pathfinder, each monster is described by many numerical attributes that jointly determine its power, summarized as an ordinal level. We investigate whether machine learning can support designers by predicting this level from a monster’s attributes, framing the task as tabular ordinal regression. We introduce what is, to our knowledge, the first dataset built specifically for TTRPG monster-level prediction, derived from publicly available Pathfinder Second Edition data. Using it, we compare classical regression models with rounding schemes, dedicated tabular ordinal regression algorithms, and neural networks with ordinal-aware losses. To mirror real design workflows, we evaluate all models under chronological and expanding-window protocols with several complementary metrics. Results show that tree-based ensembles outperform linear models and neural approaches, achieving near-perfect ordinal ranking and high predictive accuracy. Explainable AI analyses, such as feature importance and error distributions, show that the model is aligned with human intuition and follows patterns grounded in game rules. Together, these results show that machine learning can reliably approximate designer judgments and serve as an effective computer-aided tool for monster balancing and broader TTRPG system design.
[LG-20] GenVid2Robot: From Video Generation to Robot Manipulation via Rigid-Geometric Consistency
链接: https://arxiv.org/abs/2607.09191
作者: Haohui Huang,Xi Yuan,Panpan Liao,Tao Teng,Chenguang Yang,Jing Guo,Yi Guo
类目: Robotics (cs.RO); Machine Learning (cs.LG)
*备注: Preprint
Abstract:Generated videos provide useful visual motion priors for robot manipulation, but their visual plausibility does not imply physical executability. A generated video usually lacks metric geometry, grasp grounding, robot kinematic feasibility, and execution-time feedback, which makes direct trajectory replay unreliable in real-world manipulation. This paper presents GenVid2Robot, a rigid-geometric consistency framework that converts generated video motion into executable real-robot manipulation trajectories. Given an initial RGB-D observation and a task instruction, GenVid2Robot samples task-relevant semantic anchors from the real first frame, tracks these anchors through generated video candidates, and verifies whether the resulting 2D motion can be explained by first-frame RGB-D anchors under a sparse relative SE(3) model. In this way, generated videos are treated as uncertain visual motion hypotheses rather than direct robot demonstrations. Only geometrically consistent motion is transferred to the robot. The accepted relative motion is then applied to the real grasp-time TCP pose selected by mask-constrained grasping, producing a grasp-conditioned execution trajectory that is consistent with both the visual motion prior and the physical grasp configuration. To reduce execution mismatch caused by RGB-D noise, calibration residuals, and small contact-induced displacement, a bounded depth-compensation module corrects local depth-direction errors without assuming full online replanning. Real-robot experiments demonstrate that GenVid2Robot improves the reliability of generated-video-guided manipulation by grounding visual motion priors with sparse metric geometry, grasp constraints, robot feasibility checking, and bounded execution feedback.
[LG-21] Understanding Schedule-Free Methods in Nonconvex Optimization: Rate Guarantees and Escaping Saddles
链接: https://arxiv.org/abs/2607.09167
作者: Jiseok Chae,Donghwan Kim
类目: Machine Learning (cs.LG); Optimization and Control (math.OC)
*备注: 44+7 pages, 2 figures
Abstract:Schedule-Free methods have attracted growing interest for alleviating the burden of designing and tuning a learning rate scheduler, while matching and sometimes even outperforming optimizers with tuned schedulers. Despite their strong empirical results, their convergence theory in nonconvex optimization, where modern machine learning objectives typically arise, has remained largely unexplored. In this paper, we provide worst-case analyses of Schedule-Free gradient descent and Schedule-Free stochastic gradient descent, in their standard form and without auxiliary modifications or restrictive conditions, for smooth but possibly nonconvex objectives. Based on a Lyapunov analysis derived from the continuous-time limiting ordinary differential equation associated with these methods, we show that Schedule-Free gradient descent and Schedule-Free stochastic gradient descent achieve the optimal worst-case convergence rates attainable among first-order methods. We further formulate Schedule-Free gradient descent as a nonautonomous dynamical system and prove strict-saddle avoidance under an arbitrarily small one-time perturbation. These theoretical results provide a better understanding of the strong performance that Schedule-Free methods demonstrate.
[LG-22] COAST: Context-Aware Differential Learning for Gene Expression Prediction in Spatial Transcriptomics
链接: https://arxiv.org/abs/2607.09166
作者: Keunho Byeon,Sunhong Park,Jeewoo Lim,Jin Tae Kwak
类目: Machine Learning (cs.LG)
*备注:
Abstract:Spatial transcriptomics enables profiling of spatial gene expression but is limited by high cost and low throughput, motivating prediction from HE histopathology images. Existing context-aware methods mainly supervise absolute expression, while relative expression relationships between spots are rarely used explicitly. We propose COAST, a context-aware differential learning framework for spatial gene expression prediction. COAST conditions the local and global context features with type-specific modulation and aggregates the target and context spot tokens using a Transformer encoder to capture both fine-grained local patterns and slide-level structure. It is trained with a joint objective that combines absolute expression regression with signed differential regression between the target and context spots. Experiments on multiple spatial transcriptomics datasets show consistent improvements in correlation- and distribution-based metrics, demonstrating the effectiveness of context-aware differential learning for histology-based spatial gene expression prediction.
[LG-23] Present but Rescaled: Chat-to-Agent Transfer of Additive Activation Steering
链接: https://arxiv.org/abs/2607.09156
作者: Lucas Pinto
类目: Machine Learning (cs.LG)
*备注: 12 pages, 3 figures, 4 tables
Abstract:Additive activation steering (injecting a scaled residual-stream direction during generation) is calibrated almost entirely in single-turn chat, yet the models it targets are increasingly deployed as tool-using ReAct agents. We present the first systematic chat-to-agent transfer study of additive steering, coupling behavioral measurement with a representation read-out in a matched-information design: the same items rendered as plain chat or as a ReAct tool-use episode, with matched-norm random-direction controls and the transcript re-encoded every turn to exclude KV-cache contamination. Transfer is real but rescaled, and the right description is a dissociation: the injected direction reaches the late layers at near-full strength in every setting and model tested (install-site agent-over-chat ratios 0.83-1.16 across three families), while the behavioral coupling is reset per model and context. On Qwen2.5-7B a refusal bypass vector amplifies in the agent (T = 1.45, CI [1.20, 1.78], N = 300); across a powered uniform-protocol distribution the coupling spans amplification (Gemma-2-9B T = 2.00) to attenuation (Yi-1.5-9B T = 0.43, CI [0.29, 0.60]), with no universal constant and a single clean attenuator against a universal sign. Directional ablation of the same axis does not amplify (T = 0.93, CI including 1) while additive injection amplifies (T = 1.50), a 20.1-point gain difference (CI [13.4, 26.8]) that identifies an additive-specific mechanism. Two pre-registered instruments converge to localize the rescaling to the ReAct format scaffold, before any tool observation, rather than to the observation boundary where a dilution account would predict it. The safety implication is immediate and unpredictable: agentic deployment amplifies steering-based refusal bypass by up to 2.00x on some models while others attenuate, so a deployment cannot assume a given model is safe under additive steering.
[LG-24] Power Flow Feasibility Assessment Using Variational Graph Autoencoders
链接: https://arxiv.org/abs/2607.09122
作者: Ferran Bohigas-Daranas,Hamid Latif-Martinez,Eduardo Prieto-Araujo,Pere Barlet-Ros,Oriol Gomis-Bellmunt
类目: Machine Learning (cs.LG); Systems and Control (eess.SY)
*备注: Conference
Abstract:Data-driven methods, including graph neural networks, have been studied for accelerating power flow calculations in recent years, but very little attention has been paid to the solution feasibility, which can be obtained by traditional solvers. This paper presents a Variational Graph Autoencoder (VGAE) that detects the power flow solution feasibility, using the IEEE 118-bus case, to assess the validity of the solutions provided by AI-driven solvers.
[LG-25] Quantum Circuits in Diffusion Models: A Fair-Comparison Study and a Mechanistic Analysis of Angle-Embedding Failures
链接: https://arxiv.org/abs/2607.09108
作者: Jaeuk Kim,Sanghoon Yoo
类目: Machine Learning (cs.LG)
*备注: 13 pages, 4 figures, 8 tables
Abstract:We study the integration of variational quantum circuits (VQCs) into diffusion models through a squeeze-and-excitation (SE) channel-modulation scaffold that isolates the quantum contribution. Using a role-matched classical control and multi-seed significance testing across DDPM and latent diffusion on MNIST and CIFAR-10, with a score-based NCSN study on MNIST, we find that quantum cores achieve comparable mean FID to the classical control across DDPM and latent diffusion, while paired sampling-seed tests for EfficientSU2 detect no statistically significant difference. Although the quantum cores use 4.5 – 9\times fewer core parameters than the role-matched control, parameter-matched classical controls attain comparable mean FID, so the experiments do not establish a quantum parameter-efficiency advantage. We further identify a structural failure in score-based NCSN: the unbounded score target, proportional to 1/\sigma , drives angle-embedding inputs far beyond the 2\pi period of rotation gates, causing phase aliasing and collapse of the quantum modulator. A bounding transformation, \theta \leftarrow \pi \tanh(\cdot) , maps inputs to the non-aliasing domain and substantially improves both quantum cores. Since all circuits are classically simulated at a few-qubit scale, we do not claim quantum advantage. Instead, the study provides a fair-comparison protocol for quantum-enhanced generative models and a mechanistic account of when and why angle embeddings fail.
[LG-26] EXHOLD: Experience-Aware Real-Time Hold Control for Large-Scale Ride-Hailing Matching at DiDi
链接: https://arxiv.org/abs/2607.09090
作者: Xu Liu,Kai Wan,Zihao Lu
类目: Machine Learning (cs.LG)
*备注:
Abstract:In large-scale ride-hailing, hold control is a critical mechanism for improving passenger-driver experience. By selectively delaying certain driver-order pairs, the system waits for better opportunities, reduces cancellations, and mitigates wasted driver effort. However, existing industrial hold strategies often rely on heuristic thresholding over multiple predictive models, which can be brittle under non-stationary traffic and hard to optimize for multi-objective experience signals. We propose EXHOLD, a deployable two-stage framework decoupling experience-aware pair assessment from hold-time execution. In Stage I, we learn a decision model assigning each driver-order pair to discrete, interpretable experience tiers by optimizing a unified objective that aggregates satisfaction signals across the matching funnel. In Stage II, we solve for a monotone hold-time schedule via constrained optimization over empirical quantiles. This explicitly enforces service guardrails bounding the unnecessary holding of promising matches while maximizing overall experience improvement. We evaluate EXHOLD through randomized A/B experiments in DiDi’s production system in Brazil. Results show consistent gains in marketplace efficiency and experience: EXHOLD increases trip completion and driver income, significantly reduces passenger cancellations, and improves funnel efficiency. Ablations and behavioral analyses confirm both stages are essential and that the policy makes calibrated decisions under spatiotemporal heterogeneity. EXHOLD is currently deployed, serving production traffic in Brazil. Subjects: Machine Learning (cs.LG) Cite as: arXiv:2607.09090 [cs.LG] (or arXiv:2607.09090v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2607.09090 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-27] A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design
链接: https://arxiv.org/abs/2607.09084
作者: Linhui Xiao,Guiping Cao,Mingyue Guo,Xianchao Guan,Fan Yang,Ming Tao,Xin Li,Yuxin Peng,Yaowei Wang
类目: Machine Learning (cs.LG); Computers and Society (cs.CY)
*备注: This paper has been accepted by CJE (2026), paper homepage: this https URL
Abstract:The rapid expansion of large-scale AI models has led to significant performance breakthroughs across diverse domains, yet it has also raised critical concerns regarding computational costs, energy consumption, and environmental sustainability. This survey provides a comprehensive overview of the green development of large models, emphasizing resource-efficient architectures and full-stack hardware-software co-design. We systematically review recent advances in efficient model construction, including attention operator optimization, linear-complexity architectures, and model sparsification and merging, as well as training and deployment strategies such as data-efficient learning, parameter-efficient fine-tuning, and computational compression. Beyond algorithmic improvements, we explore energy-efficient AI hardware, including mainstream AI chips, memory optimization, cross-platform deployment, and sustainable infrastructure. Furthermore, we examine how large models are being applied to sustainability-critical domains such as DeepSeek, remote sensing interpretation, national-scale infrastructure, and global initiatives. Finally, we discuss key challenges and future directions, highlighting the need for continual learning paradigms, memory-centric hardware, and standardized evaluation protocols. This survey aims to offer a holistic roadmap toward sustainable, scalable, and socially responsible development of large models. Paper homepage: this https URL
[LG-28] Pitfalls and Remedies for Multi-Task Bayesian Optimization
链接: https://arxiv.org/abs/2607.09073
作者: Carl Hvarfner,Sam Daulton,Max Balandat,Eytan Bakshy
类目: Machine Learning (cs.LG)
*备注:
Abstract:Bayesian optimization routinely warm-starts a target experiment with data from related source tasks, and the multi-task Gaussian process is the textbook surrogate for the job. We revisit this default in a controlled setting and find that it misestimates the cross-task correlation even in the simplest non-trivial case, affinely related source and target tasks, where a working transfer learning method should obviously succeed. We trace the failure to two independent structural mechanisms. Per-task standardization, the textbook fix for the affine slice ambiguity, propagates a finite-sample alignment error into the recovered correlation. The marginal likelihood itself identifies the correlation only at a per-sample rate that a Gaussian process at non-overlapping designs further dilutes. We propose three conservative remedies that follow from the analysis: promoting per-task means and scales to model parameters, restricting the task covariance to non-negative correlations, and co-locating part of the source and target designs. Across synthetic multi-task problems and surrogate-based hyperparameter tuning transfer, these remedies recover the target-only baseline on the simple instances, while the broader failure persists on harder instances and across most rank-based and latent-context variants.
[LG-29] EvoLP: Self-Evolving Latency Predictor for Model Compression in Real-Time Edge Systems
链接: https://arxiv.org/abs/2607.09063
作者: Shuo Huai,Hao Kong,Shiqing Li,Xiangzhong Luo,Ravi Subramaniam,Christian Makaya,Qian Lin,Weichen Liu
类目: Machine Learning (cs.LG)
*备注: Author’s accepted version. Published in IEEE Embedded Systems Letters
Abstract:Edge devices are increasingly utilized for deploying deep learning applications on embedded systems. The real-time nature of many applications and the limited resources of edge devices necessitate latency-targeted neural network compression. However, measuring latency on real devices is challenging and expensive. Therefore, this letter presents a novel and efficient framework, named EvoLP, to accurately predict the inference latency of models on edge devices. This predictor can evolve to achieve higher latency prediction precision during the network compression process. Experimental results demonstrate that EvoLP outperforms previous state-of-the-art approaches by being evaluated on three edge devices and four model variants. Moreover, when incorporated into a model compression framework, it effectively guides the compression process for higher model accuracy while satisfying strict latency constraints. We open source EvoLP at this https URL.
[LG-30] COBS: Cumulant Order Block Sparse Attention
链接: https://arxiv.org/abs/2607.09052
作者: Alexander Tian,Aditya Ghai,Sanjit Neelam,Zaal Vasania,Akshay Mishra
类目: Machine Learning (cs.LG)
*备注:
Abstract:Block sparse attention is a hardware friendly way to alleviate the key-value (KV) cache read bottleneck in large language models (LLMs). However, it is not prevalent among leading open-weight LLMs, which rely instead on dense attention or fine-grained selection, thereby motivating our analysis. We study DeepSeek’s Native Sparse Attention (NSA) as a representative method, whose three-branch design lets us isolate block selection, the most challenging and consequential stage. We formalize selection and reduce it to ranking blocks by a single quantity, the attention mass: the sum of a block’s attention scores. We show that if selection retrieves the blocks with the largest attention mass, block sparse attention can match the quality of dense attention. However, computing the exact attention mass requires reading every key, so the problem of block selection ultimately reduces to approximating this mass from a compact summary instead of the full keys. Via a cumulant expansion, we show why existing methods falter: their selection strategies attempt to estimate the attention mass, but are confined to a first-order approximation. Therefore, we propose COBS (Cumulant Order Block Sparse Attention), an attention method that builds on NSA, incorporating a novel selector that stores a compressed second-order statistic per block. On the 32k RULER long-context retrieval benchmark, COBS raises the NSA baseline’s mean score from 0.2999 to 0.8195, approaching dense attention at 0.9040 and closing about 86% of the gap, while using only 1.21x the KV cache read traffic of the NSA baseline and 15.15x less read traffic than dense. The same model preserves short-context behavior and attains lower position-wise negative log-likelihood (NLL) than dense attention in our comparison.
[LG-31] Learning More from Less: Reinforcement Learning from Hindsight
链接: https://arxiv.org/abs/2607.09042
作者: Iris Xu,Sunshine Jiang,John Marangola,Nitish Dashora,Richard Li,Thomas Liu,Zexue He,Yuheng Zhi,Alex Pentland,Pulkit Agrawal,Zhang-Wei Hong
类目: Machine Learning (cs.LG)
*备注:
Abstract:Reinforcement learning (RL) is increasingly used to post-train vision-language-action (VLA) models, but every update consumes robot rollouts that are slow and costly to collect, making sample efficiency a central concern. Manipulation tasks typically provide only sparse rewards, so a weak policy fails almost every rollout early in training and has little to learn from, even when those failures execute coherent behavior. Such a failure, however, is a success at a different task. We present Learning from Hindsight (LfH), which brings hindsight relabeling to RL post-training of VLAs by scoring failed rollouts against the tasks they actually achieved. A single vision-language model relabels both the instruction and the reward, proposing a hindsight instruction for a group of failed rollouts and scoring how well each satisfies it, and the policy trains on the relabeled and original rollouts jointly. Because VLAs generalize across language, relabeling in language lets the policy learn more from the same trajectories. On out-of-distribution LIBERO-PRO tasks, where standard RL improves only slowly, LfH achieves 5\times improvement in sample efficiency, and outperforms a dense progress-reward baseline. The gains hold across VLA backbones and on a physical Franka robot.
[LG-32] Variable-Length Generative Protein Design via Generalized Poisson Flow
链接: https://arxiv.org/abs/2607.09039
作者: Chaoran Cheng,Zhanghan Ni,Yanru Qu,Yuxin Chen,Ruihan Guo,Jiajun Fan,Ge Liu
类目: Machine Learning (cs.LG); Quantitative Methods (q-bio.QM)
*备注:
Abstract:The ability to generate variable-length proteins is crucial in protein design, where the optimal length is often unknown and tightly coupled to designability. Current diffusion- and flow-based generative models typically require the protein length to be specified before sampling, limiting their flexibility in exploring the feasible design space. To address this limitation, we introduce Generalized Poisson Flow (GPFlow), a variable-length generative framework that learns the rate function of an inhomogeneous generalized Poisson process by minimizing its negative log-likelihood. We establish population-level guarantees for recovering the joint multimodal distribution and derive an upper bound on the KL divergence between the data and generated distributions. We comprehensively evaluate GPFlow across structure and sequence design, motif scaffolding, and peptide co-design, spanning Euclidean, categorical, and Riemannian modalities to fully validate its variable-length generation quality. In unconditional design, GPFlow improves structural designability and achieves the best distributional fitness for sequence design compared to their corresponding fixed-length baselines, while perfectly recovering the length distribution. In conditional motif scaffolding, GPFlow ranks first on 10 of 16 structure-based design tasks with significantly more unique successes and also achieves more passed tasks in sequence-based design. In peptide co-design, GPFlow remains competitive even without access to a native-length oracle.
[LG-33] RaMark: Radioactive Watermarking for Generated Tabular Data
链接: https://arxiv.org/abs/2607.09000
作者: Xin Che,Lingyang Chu,Qiqi Zhang,Xinyu Ma,Xuan Luo,Jian Pei
类目: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
*备注:
Abstract:Recent advances in generative modeling have made generated tabular data a practical solution for privacy-sensitive data sharing, where watermarking enables ownership verification. However, existing watermarking methods fundamentally fail under retraining attacks, in which an adversary retrains a generative model on a watermarked dataset and regenerates high-utility data that no longer carries the watermark. We address this challenge by introducing radioactivity, the property that a watermark remains detectable after generative model retraining, and propose RaMark, a radioactive watermarking method that embeds a sinusoidal dependency as an intrinsic component of the data distribution. By coupling the watermark with the underlying distribution, RaMark ensures that any generative model preserving data utility also has to preserve the watermark. We theoretically show that with high probability removing watermark degrades utility and alters data distribution. Extensive experiments on two real-world tabular datasets, under a large-scale ownership verification setting with 10^5 independent data owners, demonstrate that RaMark achieves substantially stronger radioactivity than seven state-of-the-art methods and consistently outperforms them against both retraining and data modification attacks.
[LG-34] Group Invariant Spectral Embedding
链接: https://arxiv.org/abs/2607.08987
作者: Yeari Vigder,Paulina Hoyos,David Thong,Joakim andén,Joe Kileel,Amit Moscovich
类目: Machine Learning (cs.LG); Numerical Analysis (math.NA); Statistics Theory (math.ST)
*备注:
Abstract:Spectral embedding methods are widely used for dimensionality reduction and clustering of high-dimensional datasets with intrinsic low-dimensional structures. Although many datasets of practical interest exhibit invariance under symmetries such as rotations, standard spectral embedding methods do not account for this, treating symmetry-related data points as unrelated. Our approach to this problem is to incorporate the symmetries directly into the affinity kernels used for spectral embedding. We analyze the case of a Riemannian data manifold M with symmetries given by a compact Lie group~ G and prove that, under suitable conditions, graph Laplacians constructed from three types of invariant kernels converge pointwise to explicit second-order differential operators on the quotient space M/G . Our analysis implies improved convergence rates, as the effective dimension drops according to the dimension of the group. We validate our approach on datasets with \mathrmSO(2) or \mathrmSO(3) symmetry, and show that G -invariant spectral embedding recovers the intrinsic geometry of the data, in contrast to standard spectral embedding, which fails to do so even in the limit of infinite data.
[LG-35] Optimal Top-k Identification from Pairwise Comparisons ICML2026
链接: https://arxiv.org/abs/2607.08979
作者: Motti Goldberger,Nils Rudi
类目: Machine Learning (cs.LG); Applications (stat.AP); Machine Learning (stat.ML); Other Statistics (stat.OT)
*备注: accepted for ICML 2026
Abstract:We study the active learning problem of fixed-confidence top- k identification from noisy pairwise comparisons. In this problem, an algorithm sequentially chooses pairs of items to compare, observes the outcomes, and stops when it can return the set of top- k items with error probability at most \delta . The objective is to design such a \delta -correct procedure that minimizes the expected number of comparisons (the sample complexity). This problem falls within the broader literature on fixed-confidence pure exploration in bandit models, where a common target is asymptotic optimality: the algorithm’s expected sample complexity matches the information theoretic lower bound as \delta \to 0 . Asymptotically optimal procedures have been developed for a range of fixed-confidence pure-exploration problems, however to the best of our knowledge, for top- 1 , or more generally top- k identification from pairwise comparisons under latent utility models an asymptotically optimal algorithm has not been established. In this setting, we develop such an algorithm. We characterize the structure of the lower bound and formulate it as a saddle-point problem. This structure enables a computationally efficient primal-dual procedure that learns the asymptotically optimal comparison allocation online. We then construct an adaptive comparison-allocation algorithm that tracks the allocation learned by the primal-dual procedure and prove it is asymptotically optimal.
[LG-36] Federated Low-Rank Koopman Learning for Multivariate Time-Series Anomaly Detection in IoT Systems
链接: https://arxiv.org/abs/2607.08978
作者: Tung-Anh Nguyen,Van-Phuc Bui,Anh Tuyen Le,Kim Hue Ta,Minh Thuy Le,J.Andrew Zhang,Xiaojing Huang
类目: Machine Learning (cs.LG); Signal Processing (eess.SP)
*备注:
Abstract:Distributed IoT systems generate multivariate time-series streams for monitoring physical assets, servers, and embedded sensing platforms. Detecting abnormal temporal behavior is critical for fault diagnosis, predictive maintenance, and security. However, practical IoT anomaly detection is hindered by decentralized and non-IID data, limited bandwidth, and the constrained computation and memory of edge devices. This paper proposes FedKAD, a resource-efficient federated Koopman anomaly detection framework for distributed IoT multivariate time series. Unlike deep-learning-based anomaly detectors that require training and communicating large neural models, FedKAD learns normal temporal dynamics through lightweight sliding-window Koopman representations. Federated training is formulated as a low-rank consensus problem, where raw sensor streams and local reduced dynamics remain on device while only compact subspace variables are exchanged with the server. To optimize the shared representation under orthonormality constraints, we develop a federated Stiefel-ADMM algorithm and provide convergence and stationarity analysis under partial client participation. During inference, each client detects anomalies locally by measuring the prediction residual between observed future trajectories and the learned Koopman dynamics. Experiments on four widely used multivariate time-series anomaly detection benchmarks show that FedKAD maintains or improves detection performance compared with federated deep-learning baselines. More importantly for IoT deployment, FedKAD provides up to 2.1\times10^3 faster training, 80\times lower communication, and 79\times lower inference latency than neural baselines, confirming its suitability for resource-constrained edge devices.
[LG-37] Stochastic Linear Bandits with Partially Observed Actions
链接: https://arxiv.org/abs/2607.08971
作者: Gautam Dasarathy,Vineet Gattani,Lalit Jain
类目: Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML)
*备注:
Abstract:The stochastic linear bandit, where actions are represented as vectors and rewards are linear, is a central paradigm for sequential decision making. We study a partially observed variant of this problem in which the learning agent only sees a random subset of coordinates for each action. Such partial observability arises naturally in settings like recommendation and healthcare, where full action descriptions can be expensive or even impossible to obtain. In general, this makes sublinear regret information-theoretically impossible. However, we show that this barrier can be overcome when the action vectors have low intrinsic dimension. We propose an algorithm, TOFU-POV, that estimates the latent action subspace using the masked actions, imputes current actions using an epoch-wise frozen representation, and runs OFUL in the resulting low-dimensional coordinates. Our theory shows that TOFU-POV enjoys a \sqrtT regret that scales with the intrinsic action subspace dimension as opposed to the ambient dimension and quantifies the interaction between these quantities and the missingness, decision set size, and subspace conditioning. We also devise a rank-adaptive algorithm that does not require the knowledge of the intrinsic dimension. We complement these guarantees with a lower bound based on a novel product construction that separates usual reward-learning uncertainty from a missingness-dependent cost intrinsic to partial observation. Synthetic and real data experiments support our theory and show that TOFU-POV can substantially improve upon natural baselines in this challenging problem.
[LG-38] FairSelect: A Systematic Evaluation of Multi-Level and Intersectional Algorithmic Fairness ALT
链接: https://arxiv.org/abs/2607.08953
作者: Nick Souligne,Isabella Mixton-Garcia,Vignesh Subbian
类目: Machine Learning (cs.LG)
*备注: 15 pages, 5 tables, Submission to Health Informatics Knowledge Management Conference 2026
Abstract:Algorithmic fairness methods are increasingly used to identify and mitigate bias in machine learning models, yet most approaches are evaluated in isolation and along single demographic axes. This limits practical guidance for selecting fairness strategies, where disparities may arise across intersectional subgroups and across multiple stages of the modeling lifecycle. This work presents FairSelect, a toolkit for systematically evaluating fairness mitigation strategies applied individually and in combination across preprocessing, inprocessing, and postprocessing stages. FairSelect supports multiple model architectures, intersectional subgroup evaluation, and comparison of fairness utility tradeoffs across baseline, single method, and multi level configurations. The framework was validated using synthetic clinical datasets designed to represent specific bias mechanisms and a real-world replication of two-year stroke risk prediction among patients with atrial fibrillation. Synthetic experiments showed that targeted fairness methods generally reduced intended subgroup disparities, while combined strategies produced larger average fairness improvements with modest utility tradeoffs. In the clinical prediction task, mitigation effects were highly variable, with some combinations improving both fairness and predictive performance while others were ineffective or counterproductive. These findings demonstrate that fairness interventions interact in nonadditive and context dependent ways. FairSelect provides a practical framework for systematically identifying fairness strategies that improve subgroup equity while preserving model performance in clinical machine learning.
[LG-39] SRouter: Dynamic Modality-Model Selection for Time Series Reasoning
链接: https://arxiv.org/abs/2607.08940
作者: Fangxu Yu,Tao Feng,Dehai Min,Lu Cheng,Ge Liu,Tianyi Zhou
类目: Machine Learning (cs.LG)
*备注: Accepted to COLM 2026
Abstract:Time series reasoning is essential for real-world problem-solving. While both Large Language Models (LLMs) and Vision-Language Models (VLMs) can reason about time-series data, their capabilities are complementary: LLMs process time series as text sequences and thus preserve exact numerical understanding, but struggle with global patterns, whereas VLMs efficiently capture these patterns by visualizing time series but may lose fine-grained details. Moreover, models vary significantly in task-specific expertise and inference costs. Dynamically selecting the most suitable modality and model for each query is therefore crucial, yet challenging because it requires modeling the complex interactions among tasks, queries, modalities, and models, which carry rich contextual signals. To this end, we introduce TSRouter, a graph-based dynamic routing framework. TSRouter constructs a heterogeneous graph of task, query, modality, and model nodes to contextualize the interactions among query characteristics, modality attributes, and model capabilities. TSRouter formulates routing as a candidate scoring problem, where each modality-model pair is evaluated based on user-defined performance-cost preferences to select the optimal candidate. Comprehensive evaluations on 4 distinct time series reasoning tasks reveal that TSRouter substantially outperforms diverse baselines with 16% to 46% relative improvements. Furthermore, TSRouter demonstrates robust zero-shot plug-and-play generalization to unseen models and novel tasks and preserves high performance while reducing computational overhead through cost-aware optimization. Our code is available at this https URL.
[LG-40] BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
链接: https://arxiv.org/abs/2607.08930
作者: Yuanjie Zhu,Liangwei Yang,Ke Xu,Weizhi Zhang,Shanghao Li,Zihe Song,Philip S. Yu
类目: Machine Learning (cs.LG)
*备注:
Abstract:Efficient serving of diffusion large language models (dLLMs) is hindered by convergence heterogeneity: when batching multiple requests, different sequences converge at different rates, causing faster requests to stall behind slower stragglers and introducing compute bubbles and tail latency. We present BlockServe, a continuous batching framework that integrates block-grained scheduling – immediately evicting completed requests at block boundaries – with mixed-state execution that extends dual cache and parallel decoding to heterogeneous batches via gather-scatter indexing. Furthermore, a compute-aware admission controller expands effective batch capacity through token-budgeted refill. On Dream and LLaDA across five benchmarks, BlockServe achieves 1.9–10.6 \times throughput over Fast-dLLM with comparable generation quality, establishing block-grained scheduling as a foundation for high-throughput offline dLLM inference.
[LG-41] SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions
链接: https://arxiv.org/abs/2607.08925
作者: Elham Daneshmand,Majid Khadiv,Glen Berseth,Hsiu-Chin Lin
类目: Machine Learning (cs.LG)
*备注:
Abstract:Training reinforcement-learning agents directly on physical robots makes every fall costly, since a fall can damage the platform and cannot be undone like a simulator reset; the goal is therefore to minimize falls during training rather than trade them off against return, as constrained Markov decision process (MDP) formulations do. A standard mitigation hands control to a separate recovery policy whenever the agent leaves a designer-specified safe region (a subset of state space it should stay within), but the resulting mixed-policy rollouts silently bias every on-policy update, and the importance-sampling correction that would remove this bias is ill-defined whenever the recovery policy is deterministic. We address this bias with a drop-in modification of proximal policy optimization (PPO). Its core is an unbiased policy-gradient estimator that uses the score function only at safe timesteps and never evaluates the recovery policy’s density, so it stays valid even when the recovery policy is deterministic, exactly where importance sampling breaks, and it empirically dominates importance sampling even when the recovery policy is stochastic. Because the recovery policy still makes credit assignment slow near the safe-region boundary, two further components accelerate learning: a closed-form value for recovery-triggering states when dynamics and recovery are deterministic, and an imitation loss that copies recovery actions only when recovery succeeds. On a three-environment, five-seed benchmark, the resulting algorithm reduces training-time falls by factors of 233x, 48x, and 26x on HalfCheetah, Ant, and Unitree Go1 over standard PPO, while matching or exceeding PPO’s final reward, and on Ant, where the recovery policy is unreliable, it is the only method that reaches 80% of the best final reward.
[LG-42] A Machine Learning Surrogate for Component Criticality Ranking in Interdependent Power-Communication Networks
链接: https://arxiv.org/abs/2607.08918
作者: Sohini Roy,Xheni Hylviu
类目: Machine Learning (cs.LG)
*备注:
Abstract:Cyber-physical power systems are vulnerable to cascading failures caused by tight interdependencies between power and communication infrastructures. Evaluating these failures over large N-k contingency sets with a high-fidelity simulator is computationally prohibitive for resilience planning. Using the previously published Modified Implicative Interdependency Model (MIIM) as the ground-truth cascade simulator, this paper develops a machine-learning surrogate that predicts contingency severity from leakage-free structural features and derives a component-criticality ranking for prioritized hardening analysis. On the IEEE 118-bus system, the Gradient Boosting surrogate achieves Spearman correlations of 0.849 for per-contingency severity prediction and 0.853 for per-component criticality ranking, while remaining stable across three independently sampled datasets. MIIM-derived component criticality itself reproduces only to a Spearman of approximately 0.85 under the present sampling pipeline, and the surrogate operates at this empirical ceiling to within sampling variation. Topological centrality measures on the full interdependent network provide meaningful baselines (Spearman 0.60-0.69), and feature ablation shows that the surrogate’s advantage is driven primarily by inter-layer dependency information. These results support a two-stage workflow in which the surrogate rapidly ranks candidate components and MIIM is reserved for selective verification.
[LG-43] Pattern-Aware Graph Neural Networks for Handling Missing Data
链接: https://arxiv.org/abs/2607.08915
作者: Minett Tran,Taehee Jeong
类目: Machine Learning (cs.LG)
*备注: 2026 International Conference on Advances in Artificial Intelligence and Machine Learning (AAIML), 20-22 March 2026
Abstract:Missing data is ubiquitous in real-world datasets. Traditional methods either discard incomplete samples or apply imputation techniques that ignore potentially informative missingness patterns, implicitly assuming that missingness occurs randomly. However, missingness patterns might provide additional information. We propose pattern-aware graph neural networks that explicitly encode which features are missing alongside observed values. We used four encoding strategies – learned embeddings, frozen random embeddings, statistical features, and hierarchical representations – across seven UCI datasets with naturally occurring missingness. Our Pattern-aware methods achieve substantial improvements over baselines, with an average improvement of 17% in balanced accuracy and 22% in F1-macro across all datasets. The benefits vary significantly by dataset: annealing shows dramatic improvement (+80% balanced accuracy), while hepatitis and soybean show minimal gains (+4–5%). Notably, even simple random pattern embeddings perform comparably to learned embeddings (0.650 vs 0.663 balanced accuracy), suggesting that distinguishing between patterns may be more important than task-specific optimization. Our ablation study reveals that attention mechanisms, while helpful, are not critical when pattern information is available – simple mean aggregation with pattern awareness achieves 0.640 balanced accuracy compared to 0.645 for attention-based variants.
[LG-44] Learning-enabled Parameter Synthesis for Nonlinear Systems from Signal Temporal Logic
链接: https://arxiv.org/abs/2607.08899
作者: Alex Beaudin,Hanna Krasowski,Eric Palanques-Tost,Calin Belta,Murat Arack
类目: ystems and Control (eess.SY); Machine Learning (cs.LG)
*备注:
Abstract:Signal Temporal Logic (STL) is increasingly used to describe interpretable objectives and constraints for optimal control and learning methods, especially when no target time series data is available. In this work, we propose to synthesize parameters for nonlinear systems that robustly satisfy continuous-time STL specifications for uncertain initial conditions. To this end, we use gradient-based optimization along with set-based reachability verification to efficiently learn in high-dimensional parameter spaces while providing provable satisfaction guarantees for the optimized parameters. We demonstrate the effectiveness and scalability of our method on three systems with up to 18 parameter dimensions.
[LG-45] Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal AAAI2026 ICLR
链接: https://arxiv.org/abs/2607.08883
作者: Ege Çakar,Hannah Guan,Kayden Kehe
类目: Machine Learning (cs.LG)
*备注: Accepted at the AAAI 2026 Summer Symposium Series. This paper was presented at the ICLR Re-Align workshop under a different name, “Accelerating Adversarial Suffix Optimization via Continuous Relaxation and Activation-Guided Objectives”
Abstract:Behavioral alignment in large language models often masks fragile internal safety representations. Recent work suggests that refusal behavior is mediated by low-dimensional directions in activation space. This raises questions about how such representations are structured, localized, and accessed by optimization. We study adversarial suffix attacks as a probe of representational alignment. We introduce Activation-Guided GCG, which replaces output-based objectives with losses that directly target a model’s internal refusal direction. Across several objective variants, we find that suppressing refusal globally across all layers and positions is more effective than targeting a single layer-position pair. This suggests that safety representations are distributed across the forward pass rather than causally localized to a single site. We further introduce Soft-GCG, a continuous relaxation of discrete suffix optimization using Gumbel-Softmax. Soft-GCG achieves a 33 \times speedup over standard GCG while improving attack success rates. Evaluating across model scales, we find that smaller models remain vulnerable while larger models resist both activation- and suffix-based attacks at our compute-constrained settings, consistent with larger and better safety trained models being harder to jailbreak. Together, our results clarify how safety mechanisms are encoded and can be broken in contemporary models. These insights provide concrete guidance for designing more robust and representation-aware alignment strategies.
[LG-46] FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space
链接: https://arxiv.org/abs/2607.08877
作者: Michael Murray,Daphne Chen,Simran Bagaria,Dean Fortier,Tess Hellebrekers,Galen Mullins,Harshavardhan Gajarla,Oier Mees,Maya Cakmak,Andrey Kolobov
类目: Robotics (cs.RO); Machine Learning (cs.LG)
*备注:
Abstract:Pretrained generative robot policies based on flow matching and diffusion have achieved impressive results across a wide range of manipulation tasks. Yet real-world deployments routinely expose failure modes outside the pretraining distribution. Closing these gaps typically requires large-scale data collection or online reinforcement learning on physical hardware, which is impractical for rapid and safe adaptation. We present FlowDAgger, a sample- and compute-efficient method for adapting frozen generative robot policies from human interventions in latent space. Our key idea is action inversion: each human expert action is mapped to the noise that would have produced it under the frozen base policy, using reverse-time integration followed by local refinement. The resulting inverted noise provides supervision for a lightweight latent policy that steers the base model at deployment time, enabling rapid skill acquisition while preserving its behavioral priors. We evaluate FlowDAgger in simulation and on real-world bimanual and single-arm manipulation, adapting both action-head VLAs and world-action models from a handful of interventions. FlowDAgger outperforms supervised fine-tuning and latent-space RL baselines and preserves pretrained skills on held-out tasks, offering a practical path for adapting robot foundation models in the real world. Website: this https URL
[LG-47] Clean2FX: Label-conditioned modeling for clean-to-effect guitar audio transformations
链接: https://arxiv.org/abs/2607.08863
作者: Oliverio Bombicci Pontelli,Iran R. Roman
类目: ound (cs.SD); Machine Learning (cs.LG)
*备注: 4 pages, 1 figure, 3 tables, DAFx2026 conference
Abstract:We present Clean2FX, a study and demo of label-conditioned clean-to-effect transformation for electric guitar audio. Given a clean guitar input and a target effect label, the task is to synthesize the corresponding effected signal while preserving the musical content. Training and evaluation pairs are constructed from EGFxSet real, single tone recordings by assembling matched clean/effected chords, melodies, and mixed timelines. This allows for controlled comparison across effects. We evaluate four neural approaches under a common spectrogram-based transformation setting: two variational autoencoders and two U-Net models that differ in whether they operate on linear or log-magnitude representations. Performance is measured using linear-magnitude spectrogram MSE and Fréchet Audio Distance. The U-Net models outperform the variational autoencoder variants. Per-effect results show that distortion effects are most readily improved, whereas delay and reverb effects exhibit weaker FAD gains despite substantial spectral-error reductions. A conditioning-sensitivity diagnostic provides evidence that the best model responds to target labels rather than collapsing to a single transformation. Our demo website compares two models applied on real-world guitar performances outside training and validation data, providing audio and spectrogram examples of the practical clean-to-effect behavior.
[LG-48] How are linear representations learned? Exact solutions to the dynamics of abstraction
链接: https://arxiv.org/abs/2607.08843
作者: William W. Yang,Andrew M. Saxe,Peter E. Latham
类目: Machine Learning (cs.LG)
*备注: 81 pages, 7 figures
Abstract:In artificial and biological neural networks, concepts are often encoded as consistent linear directions in representation space. In deep learning, this idea is known as the linear representation hypothesis and underpins many interpretability and control methods based on linear probes, from concept detection to activation steering. Yet while prior work has studied whether such directions should exist \textitafter training, the dynamics of how they emerge \textitduring training remain poorly understood. Here, we develop a framework to study the alignment of concept directions during training - a process we call “abstraction”. In a minimal linear network setting, we obtain exact solutions for the full trajectory of abstraction. These solutions reveal key analytic principles governing abstraction: (i) data and target geometry jointly determine abstraction at the end-of-learning, (ii) abstraction improves with network depth, and (iii) initialization scale controls the maximum abstraction reached during training. Extending our theory to nonlinear networks, we analyze how the choice of nonlinearity affects abstraction dynamics: erf networks approximate the linear theory, while abstraction in ReLU networks depends less on target geometry and more on input geometry. Across both, we prove a striking attenuation law: both nonlinearities weaken abstraction in activations relative to preactivations. We find evidence for this law in open models (DINOv3, Gemma 4) and apply our theory to improve linear probe generalization in LLMs. Together, our results provide a dynamical theory of abstraction with implications for interpretability and control.
[LG-49] Adaptive Bayes exactly tracks information over intrinsic time
链接: https://arxiv.org/abs/2607.08789
作者: Akshay Balsubramani
类目: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST); Machine Learning (stat.ML)
*备注:
Abstract:Bayesian and multiplicative-weights updates reweight experts, models, or actions from sequential feedback. We show that the regret of any such update obeys an exact information-accounting identity. On each round, the learner’s excess loss to any chosen comparator is the sum of an immediate payment for the uncertainty exposed by the round and a reduction in the information distance from the learner’s current weights to the comparator. The cumulative payment defines a pathwise uncertainty clock, the \emphintrinsic time of the realized sequence. Summing one-step balances yields two exact adaptive decompositions of cumulative regret, one for each natural way of composing the update across rounds. Because the decompositions are exact rather than upper bounds, favorable stochastic or low-noise regimes appear as self-bounding properties of the realized intrinsic time, not as slack in worst-case analyses. The same calculus covers Hedge, optimistic and side-information variants, continuous priors, boosting, online convex optimization, contextual bandits, and repeated games: the pathwise account is the same in every case.
[LG-50] Deep Gaussian Processes on Directed Acyclic Graphs
链接: https://arxiv.org/abs/2607.09645
作者: Federico L. Perlino,Oliver Hamelijnck,Adam M. Johansen,Theodoros Damoulas
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST); Computation (stat.CO); Methodology (stat.ME)
*备注: 75 pages, 14 figures
Abstract:Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to multiple fidelity levels; and in gene-regulatory networks, to transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these challenges, we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour, and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which the distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \citedunlop2018 on the role of input connections. We offer a structured variational approximation that retains graph dependencies, preserves compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theoretical results and our methodology, and model a latent-collider DAG, a protein signalling network, and a multi-fidelity heavy-ion collision emulation task, attaining state-of-the-art performance while recovering low-fidelity contributions and yielding interpretability of the simulator hierarchy.
[LG-51] Entropy-Constrained Machine Learning with Residual Data Augmentation for Modeling Chemical Kinetics
链接: https://arxiv.org/abs/2607.09582
作者: Okezzi Ukorigho,Opeoluwa Owoyele
类目: Fluid Dynamics (physics.flu-dyn); Machine Learning (cs.LG)
*备注:
Abstract:We present a physics-constrained machine learning framework for accelerating the direct numerical simulation (DNS) of turbulent reacting flows. The model replaces the direct evaluation of detailed chemical source terms with a surrogate that predicts reaction rates from a reduced thermochemical state. To improve physical consistency, the second law of thermodynamics is incorporated as a training constraint by enforcing non-negative entropy generation, which restricts the evolution of the thermochemical state to physically admissible directions and improves stability during time integration. The approach is demonstrated on DNS of a two-dimensional planar lean premixed methane-air flame interacting with a turbulent flow field. The model reproduces detailed-chemistry results with high fidelity while achieving more than an order-of-magnitude reduction in computational cost. Furthermore, a residual-based synthetic data augmentation strategy enables parametric exploration by constructing new training data from the original dataset, allowing accurate simulation at new inlet conditions without additional detailed-chemistry CFD runs. These results demonstrate that thermodynamically constrained machine learning can provide reliable and computationally efficient surrogates for detailed chemistry in high-fidelity combustion simulations.
[LG-52] Spectrally Deconfounded Gradient Boosting
链接: https://arxiv.org/abs/2607.09371
作者: Andrea Nava,Peter Bühlmann,Fabio Sigrist
类目: Machine Learning (stat.ML); Machine Learning (cs.LG)
*备注:
Abstract:Flexible machine-learning methods can be sensitive to hidden confounding: they may learn associations induced by unobserved confounders rather than stable signals. Spectral deconfounding mitigates this problem by shrinking high-variance directions of the covariate matrix that, under dense confounding, carry latent confounder information. Existing work has largely focused on linear models. We develop a nonlinear spectral deconfounding framework for gradient boosting. Our approach replaces the ordinary squared-error loss by a spectral loss, which alters the boosting dynamics by slowing down learning in confounding-aligned directions. We show that deconfounding is not achieved by the spectral loss alone, but by the interaction between spectral shrinkage and regularization, especially in terms of early stopping. Moreover, we provide a mixed-model interpretation that connects LAVA-type shrinkage to random-effects adjustment and yields an empirical-Bayes procedure for tuning the spectral loss. We also extend the method to general likelihoods and nonlinear confounding using Laplace approximations and kernel random effects. Across synthetic and real-world experiments, spectrally deconfounded boosting improves estimation of the target function under hidden confounding and is substantially more scalable than existing nonlinear spectral deconfounding baselines.
[LG-53] Influence Diagnostics in High-dimensional M-estimation: Precise Asymptotics
链接: https://arxiv.org/abs/2607.09250
作者: Hugo Cui
类目: Machine Learning (stat.ML); Machine Learning (cs.LG)
*备注:
Abstract:The impact of a given training point on a statistical model is classically measured through its leave-one-out influence, which quantifies the effect of its removal from the training set on the model accuracy. While the statistics of leave-one-out influences are well understood in the low-dimensional, large sample limit n\to \infty, d=O(1) , they become more intricate in high dimensions, as the influence of a given sample develops non-trivial dependencies on all other training samples. For convex M-estimation under Gaussian design, in the high-dimensional limit n\asymp d , we show that the distribution of the influences across the training set converges to a limiting measure which we sharply characterize. Building on these results, we provide evidence that influential samples tend to lie close to the decision boundary, thereby making contact with a standard data selection heuristic in active learning.
[LG-54] When Does Order Flow Matter? State-Dependent L2 Liquidity-State Transitions in Crypto Futures
链接: https://arxiv.org/abs/2607.09230
作者: Joohyoung Jeon
类目: Trading and Market Microstructure (q-fin.TR); Machine Learning (cs.LG)
*备注: 8 pages, 2 figures, 1 table
Abstract:Building event-conditioned market models requires separating macro-event labels from persistent microstructure state. We study this distinction in Binance BTCUSDT and ETHUSDT futures from 2023-2026, combining top-20 L2 order book data, trade-flow records, and macro-event windows. We define a supervised discrete L2 liquidity-state transition task, distinct from latent-regime detection and price-direction prediction, and evaluate models in rolling monthly out-of-sample folds with event-clustered validation and blocked permutation tests, admitting each feature layer only if it improves on the layer below it on the same panel. Within these event windows, the first-order predictive signal is the pre-event L2 liquidity state: a coarse pre-event state baseline strongly predicts post-event liquidity regimes, interpretable logit models over continuous L2 features fail to improve on it, and a shallow nonlinear L2 model adds a robust further gain of comparable size to the state baseline’s own. The macro-event calendar enters only by locating the windows and supplying matched non-event controls; we use event timing but not the event’s label content, so pre-event state competes against an uninformed within-window baseline, not against the event type. Order flow adds further value only when layered on top of the L2 state model, not as a replacement. This value is not robustly cross-symbol: for ETH it is present across calm, mixed, and stressed regimes and largest under stressed pre-event liquidity, whereas BTC shows only isolated five-minute passes and no regime that clears at both horizons. These findings motivate a state-first design principle for market microstructure models. We provide a liquidity-state transition baseline and evaluation protocol that reinforcement-learning, execution-policy, or LLM-based context layers should exceed before their added value is credited.
[LG-55] Quantum-Enhanced Synthetic Data Generation Using Quantum Circuit Born Machines for Imbalanced Tabular Learning
链接: https://arxiv.org/abs/2607.09113
作者: Tanapol Nuatho,Narisorn Sangnakara,Prapong Prechaprapranwong,Rajchawit Sarochawikasit
类目: Quantum Physics (quant-ph); Machine Learning (cs.LG)
*备注:
Abstract:Data scarcity and class imbalance are persistent challenges in machine learning that degrade model generalization and introduce predictive bias. We present a hybrid quantum-classical framework for synthetic data generation using a Quantum Circuit Born Machine (QCBM) to address these limitations. The proposed approach exploits quantum mechanical properties – superposition and entanglement – within a parameterized variational quantum circuit to model complex probability distributions that are difficult for classical generative methods to capture. Experiments are conducted on two tabular benchmark datasets: the Iris dataset and the Telco Customer Churn dataset. Preprocessing includes normalization and PCA-based dimensionality reduction to enable efficient basis encoding for quantum circuits. The QCBM is trained by minimizing Kullback-Leibler (KL) divergence between real and generated data distributions using a gradient-based parameter-shift optimization rule. Augmenting training data with QCBM-generated synthetic samples at 40-50% of the minority class improves F1-score by approximately 5-15% and minority-class recall by 10-25%. Cross-domain evaluations (Train on Synthetic, Test on Real; and Train on Real, Test on Synthetic) reveal a performance gap of only 3-10%, indicating strong distributional fidelity. Comparative analysis against classical oversampling methods – SMOTE, Borderline-SMOTE, KMeansSMOTE, and SVM-SMOTE – shows that QCBM achieves competitive classification performance and produces lower Maximum Mean Discrepancy (MMD) on the Telco dataset, suggesting superior structural similarity in certain imbalanced settings. These findings establish QCBM as a viable complementary tool for data augmentation, particularly for low-dimensional structured tabular data with class imbalance.
[LG-56] Solving Stochastic Fixed-Point Equations with High Probability
链接: https://arxiv.org/abs/2607.09097
作者: Jelena Diakonikolas
类目: Optimization and Control (math.OC); Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:We study stochastic fixed-point equations \mathbfT(\mathbfx) = \mathbfx over normed spaces (\mathcalE, |\cdot|) , where the operator \mathbfT is nonexpansive or contractive and is accessed only through unbiased stochastic evaluations with bounded second central moment. Given \epsilon 0, \delta \in (0, 1) , the goal is to output \mathbfx \in \mathcalE such that |\mathbfT(\mathbfx) - \mathbfx| \leq \epsilon with probability at least 1-\delta . We introduce VR-GHAL, a variance-reduced gradual Halpern method for quadratically smoothable Banach spaces. The key algorithmic ingredient is a recursive stochastic estimator based on clipped differences of oracle evaluations: instead of clipping \tau(\mathbfx; \xi) itself, we clip stochastic differences at the Lipschitz scale \gamma|\mathbfx - \mathbfy| . This makes the estimator pathwise Lipschitz along the algorithmic trajectory while permitting martingale concentration under finite second moments in the native norm. Our main theorem gives an anytime high-probability residual bound: on a single event of probability at least 1 - \delta , the residual decreases nearly geometrically across epochs, up to lower-order logarithmic factors. Under only bounded variance, displaying only the dependence on the target error \epsilon and Lipschitz constant \gamma \in (0, 1] of \mathbfT , the resulting oracle complexity is \min\epsilon^-5, (1-\gamma)^-3\epsilon^-2\ . Under a Lipschitz-in-expectation oracle, the dependence improves to the corresponding \epsilon^-3 nonexpansive rate (i.e., for \gamma = 1 ), and under samplewise nonexpansiveness to \epsilon^-2 .
[LG-57] Nonconvex Composite Functional Constraints via First-Order Augmented Lagrangian Methods under Local Regularity
链接: https://arxiv.org/abs/2607.08954
作者: Linglingzhi Zhu,Jiajin Li
类目: Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注:
Abstract:We study nonasymptotic convergence of primal-dual methods for a class of nonconvex constrained optimization problems with a convex-composite structure. In this class, both the objective and the functional inequality constraints are given by convex Lipschitz outer functions composed with smooth nonlinear inner mappings. The analysis is complicated by constraint violation in a nonconvex functional inequality system and by the lack of an a priori bound on the multipliers. To address these issues, we restrict the dual variable to an auxiliary compact set and analyze a smoothed prox-linear augmented Lagrangian method through a nonsmooth nonconvex-concave minimax reformulation. The main contribution is a finite-time mechanism for converting stationarity of the truncated minimax problem into a KKT certificate for the original constrained problem. We show that, for a sufficiently large penalty parameter, all but a controlled number of iterates enter a near-feasible region. On this region, a local conic regularity condition uniformly bounds the associated prox-linear multipliers and thereby makes the artificial dual truncation inactive at the selected iterates. Building on this mechanism, we establish explicit convergence rates for the proposed method in terms of the KKT residual. With dual regularization, a global dual error bound together with a bias-balancing argument gives an O(K^-1/3) rate. In the unregularized case, under additional local structural assumptions including piecewise linearity of the outer functions, a local dual error bound yields the sharper O(K^-1/2) rate.
附件下载
