本篇博文主要内容为 2025-06-25 从Arxiv.org论文网站获取的最新论文列表，自动更新，按照NLP、CV、ML、AI、IR五个大方向区分，若需要邮件定时接收，请在评论区留下你的邮箱号。
说明：每日论文数据从Arxiv.org获取，每天早上12:00左右定时自动更新。
友情提示: 如何您需要邮箱接收每日论文数据，请在评论处留下你的邮箱。
目录
概览 (2025-06-25)
今日共更新506篇论文,其中:
- 自然语言处理共69篇(Computation and Language (cs.CL))
- 人工智能共141篇(Artificial Intelligence (cs.AI))
- 计算机视觉共117篇(Computer Vision and Pattern Recognition (cs.CV))
- 机器学习共138篇(Machine Learning (cs.LG))
自然语言处理
[NLP-0] ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality Debiasing
【速读】： 该论文旨在解决大型视觉语言模型（LVLM）在图像描述生成中存在的固有偏差问题，包括多模态偏差导致的描述粒度不平衡以及语言偏差引发的虚构对象描述。解决方案的关键在于提出一种可扩展的去偏图像描述策略——ScaleCap，该策略通过持续增加推理预算来丰富和校准描述内容。其核心创新包括启发式问答和对比句评分两个组件：前者基于图像生成特定内容的问题并回答，逐步注入相关信息；后者通过句级离线对比解码有效识别并消除由语言偏差引起的幻觉。
链接: https://arxiv.org/abs/2506.19848
作者: Long Xing,Qidong Huang,Xiaoyi Dong,Pan Zhang,Yuhang Zang,Yuhang Cao,Jinsong Li,Shuangrui Ding,Weiming Zhang,Nenghai Yu,Jiaqi Wang,Feng Wu,Dahua Lin
机构: University of Science and Technology of China (中国科学技术大学); Shanghai Artificial Intelligence Laboratory (上海人工智能实验室); The Chinese University of Hong Kong (香港中文大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
备注: Code is available at this https URL
Abstract:This paper presents ScaleCap, an inference-time scalable image captioning strategy that generates comprehensive and detailed image captions. The key challenges of high-quality image captioning lie in the inherent biases of LVLMs: multimodal bias resulting in imbalanced descriptive granularity, offering detailed accounts of some elements while merely skimming over others; linguistic bias leading to hallucinated descriptions of non-existent objects. To address these issues, we propose a scalable debiased captioning strategy, which continuously enriches and calibrates the caption with increased inference budget. Specifically, we propose two novel components: heuristic question answering and contrastive sentence rating. The former generates content-specific questions based on the image and answers them to progressively inject relevant information into the caption. The latter employs sentence-level offline contrastive decoding to effectively identify and eliminate hallucinations caused by linguistic biases. With increased inference cost, more heuristic questions are raised by ScaleCap to progressively capture additional visual details, generating captions that are more accurate, balanced, and informative. Extensive modality alignment experiments demonstrate the effectiveness of ScaleCap. Annotating 450K images with ScaleCap and using them for LVLM pretraining leads to consistent performance gains across 11 widely used benchmarks. Furthermore, ScaleCap showcases superb richness and fidelity of generated captions with two additional tasks: replacing images with captions in VQA task, and reconstructing images from captions to assess semantic coverage. Code is available at this https URL.
zh
[NLP-1] Orthogonal Finetuning Made Scalable
【速读】： 该论文试图解决正交微调（Orthogonal Finetuning, OFT）在实际部署中因高运行时间和内存需求而受到限制的问题。其核心计算瓶颈在于OFT的以权重为中心的实现方式，该方式依赖于具有三次复杂度的昂贵矩阵-矩阵乘法。解决方案的关键在于提出OFTv2，这是一种以输入为中心的重新表述，采用矩阵-向量乘法（即无矩阵计算），将计算成本降低到二次复杂度，并引入Cayley-Neumann参数化方法，通过截断的Neumann级数近似Cayley变换中的矩阵求逆，从而显著提升训练速度并降低内存占用。
链接: https://arxiv.org/abs/2506.19847
作者: Zeju Qiu,Weiyang Liu,Adrian Weller,Bernhard Schölkopf
机构: Max Planck Institute for Intelligent Systems (马克斯·普朗克智能系统研究所); The Chinese University of Hong Kong (香港中文大学); University of Cambridge (剑桥大学); The Alan Turing Institute (艾伦·图灵研究所)
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
备注: Technical report (17 pages, 7 figures, project page: this https URL )
Abstract:Orthogonal finetuning (OFT) offers highly parameter-efficient adaptation while preventing catastrophic forgetting, but its high runtime and memory demands limit practical deployment. We identify the core computational bottleneck in OFT as its weight-centric implementation, which relies on costly matrix-matrix multiplications with cubic complexity. To overcome this, we propose OFTv2, an input-centric reformulation that instead uses matrix-vector multiplications (i.e., matrix-free computation), reducing the computational cost to quadratic. We further introduce the Cayley-Neumann parameterization, an efficient orthogonal parameterization that approximates the matrix inversion in Cayley transform via a truncated Neumann series. These modifications allow OFTv2 to achieve up to 10x faster training and 3x lower GPU memory usage without compromising performance. In addition, we extend OFTv2 to support finetuning quantized foundation models and show that it outperforms the popular QLoRA in training stability, efficiency, and memory usage.
zh
[NLP-2] MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration ACL2025
【速读】： 该论文旨在解决当前统一多模态医学大语言模型（Large Language Models, LLMs）在知识更新成本、全面性和灵活性方面的局限性。其解决方案的关键在于提出一种模块化多智能体框架（Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis, MAM），该框架将医学诊断过程分解为多个专业角色，包括全科医生、专科团队、放射科医生、医疗助理和主任，每个角色由基于LLM的智能体实现，从而实现高效的知識更新和对现有医学LLMs及知识库的有效利用。
链接: https://arxiv.org/abs/2506.19835
作者: Yucheng Zhou,Lingran Song,Jianbing Shen
机构: 未知
类目: Computation and Language (cs.CL)
备注: ACL 2025 Findings
Abstract:Recent advancements in medical Large Language Models (LLMs) have showcased their powerful reasoning and diagnostic capabilities. Despite their success, current unified multimodal medical LLMs face limitations in knowledge update costs, comprehensiveness, and flexibility. To address these challenges, we introduce the Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis (MAM). Inspired by our empirical findings highlighting the benefits of role assignment and diagnostic discernment in LLMs, MAM decomposes the medical diagnostic process into specialized roles: a General Practitioner, Specialist Team, Radiologist, Medical Assistant, and Director, each embodied by an LLM-based agent. This modular and collaborative framework enables efficient knowledge updates and leverages existing medical LLMs and knowledge bases. Extensive experimental evaluations conducted on a wide range of publicly accessible multimodal medical datasets, incorporating text, image, audio, and video modalities, demonstrate that MAM consistently surpasses the performance of modality-specific LLMs. Notably, MAM achieves significant performance improvements ranging from 18% to 365% compared to baseline models. Our code is released at this https URL.
zh
[NLP-3] How Effectively Can BERT Models Interpret Context and Detect Bengali Communal Violent Text?
【速读】： 该论文试图解决社交媒体上煽动群体暴力文本的分类问题，这一领域在现有研究中仍处于探索阶段。其关键解决方案是引入一种针对该任务进行微调的BanglaBERT模型，并通过扩展数据集以缓解数据不平衡问题，进而构建了一个集成模型，显著提升了宏F1分数至0.63，表明其在该领域的有效性。
链接: https://arxiv.org/abs/2506.19831
作者: Abdullah Khondoker,Enam Ahmed Taufik,Md. Iftekhar Islam Tashik,S M Ishtiak Mahmud,Farig Sadeque
机构: 未知
类目: Computation and Language (cs.CL)
备注:
Abstract:The spread of cyber hatred has led to communal violence, fueling aggression and conflicts between various religious, ethnic, and social groups, posing a significant threat to social harmony. Despite its critical importance, the classification of communal violent text remains an underexplored area in existing research. This study aims to enhance the accuracy of detecting text that incites communal violence, focusing specifically on Bengali textual data sourced from social media platforms. We introduce a fine-tuned BanglaBERT model tailored for this task, achieving a macro F1 score of 0.60. To address the issue of data imbalance, our dataset was expanded by adding 1,794 instances, which facilitated the development and evaluation of a fine-tuned ensemble model. This ensemble model demonstrated an improved performance, achieving a macro F1 score of 0.63, thus highlighting its effectiveness in this domain. In addition to quantitative performance metrics, qualitative analysis revealed instances where the models struggled with context understanding, leading to occasional misclassifications, even when predictions were made with high confidence. Through analyzing the cosine similarity between words, we identified certain limitations in the pre-trained BanglaBERT models, particularly in their ability to distinguish between closely related communal and non-communal terms. To further interpret the model’s decisions, we applied LIME, which helped to uncover specific areas where the model struggled in understanding context, contributing to errors in classification. These findings highlight the promise of NLP and interpretability tools in reducing online communal violence. Our work contributes to the growing body of research in communal violence detection and offers a foundation for future studies aiming to refine these techniques for better accuracy and societal impact.
zh
[NLP-4] Scaling Speculative Decoding with Lookahead Reasoning
【速读】： 该论文试图解决生成式 AI (Generative AI) 在推理模型中因生成长链式思维而导致的解码速度慢的问题。传统基于令牌级别的推测解码（Token-level speculative decoding, SD）虽然有所帮助，但其性能提升受限于猜测的令牌序列正确性随长度增加呈指数下降。该论文提出的解决方案是“前瞻推理”（Lookahead Reasoning），其关键在于引入了一个步骤级别的并行层，允许轻量级的草稿模型预测多个未来步骤，目标模型通过批量处理扩展每个预测，并由验证器保留语义正确的步骤，同时允许重新生成失败的步骤。这一方法在保持答案质量的前提下显著提升了SD的加速效果。
链接: https://arxiv.org/abs/2506.19830
作者: Yichao Fu,Rui Ge,Zelei Shao,Zhijie Deng,Hao Zhang
机构: UCSD(加州大学圣地亚哥分校); Shanghai Jiao Tong University(上海交通大学); UIUC(伊利诺伊大学厄巴纳-香槟分校)
类目: Machine Learning (cs.LG); Computation and Language (cs.CL)
备注:
Abstract:Reasoning models excel by generating long chain-of-thoughts, but decoding the resulting thousands of tokens is slow. Token-level speculative decoding (SD) helps, but its benefit is capped, because the chance that an entire \gamma -token guess is correct falls exponentially as \gamma grows. This means allocating more compute for longer token drafts faces an algorithmic ceiling – making the speedup modest and hardware-agnostic. We raise this ceiling with Lookahead Reasoning, which exploits a second, step-level layer of parallelism. Our key insight is that reasoning models generate step-by-step, and each step needs only to be semantically correct, not exact token matching. In Lookahead Reasoning, a lightweight draft model proposes several future steps; the target model expands each proposal in one batched pass, and a verifier keeps semantically correct steps while letting the target regenerate any that fail. Token-level SD still operates within each reasoning step, so the two layers of parallelism multiply. We show Lookahead Reasoning lifts the peak speedup of SD both theoretically and empirically. Across GSM8K, AIME, and other benchmarks, Lookahead Reasoning improves the speedup of SD from 1.4x to 2.1x while preserving answer quality, and its speedup scales better with additional GPU throughput. Our code is available at this https URL
zh
[NLP-5] Evaluating Compliance with Visualization Guidelines in Diagrams for Scientific Publications Using Large Vision Language Models ICDAR2025
【速读】： 该论文试图解决科研文献中数据可视化图表存在的问题，如不符合数据可视化原则和指南，导致信息误传。其解决方案的关键在于利用大型Vision Language Models（VLMs）对图表进行分析，以自动识别潜在的问题，例如缺失的坐标轴标签、图例以及不必要的三维效果。通过比较不同VLMs和提示策略的效果，验证了VLMs在多个数据可视化相关任务中的有效性。
链接: https://arxiv.org/abs/2506.19825
作者: Johannes Rückert,Louise Bloch,Christoph M. Friedrich
机构: 未知
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: Accepted at ICDAR 2025
Abstract:Diagrams are widely used to visualize data in publications. The research field of data visualization deals with defining principles and guidelines for the creation and use of these diagrams, which are often not known or adhered to by researchers, leading to misinformation caused by providing inaccurate or incomplete information. In this work, large Vision Language Models (VLMs) are used to analyze diagrams in order to identify potential problems in regards to selected data visualization principles and guidelines. To determine the suitability of VLMs for these tasks, five open source VLMs and five prompting strategies are compared using a set of questions derived from selected data visualization guidelines. The results show that the employed VLMs work well to accurately analyze diagram types (F1-score 82.49 %), 3D effects (F1-score 98.55 %), axes labels (F1-score 76.74 %), lines (RMSE 1.16), colors (RMSE 1.60) and legends (F1-score 96.64 %, RMSE 0.70), while they cannot reliably provide feedback about the image quality (F1-score 0.74 %) and tick marks/labels (F1-score 46.13 %). Among the employed VLMs, Qwen2.5VL performs best, and the summarizing prompting strategy performs best for most of the experimental questions. It is shown that VLMs can be used to automatically identify a number of potential issues in diagrams, such as missing axes labels, missing legends, and unnecessary 3D effects. The approach laid out in this work can be extended for further aspects of data visualization. Comments: Accepted at ICDAR 2025 Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL) Cite as: arXiv:2506.19825 [cs.AI] (or arXiv:2506.19825v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2506.19825 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Louise Bloch [view email] [v1] Tue, 24 Jun 2025 17:42:36 UTC (107 KB)
zh
[NLP-6] KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality
【速读】： 该论文旨在解决慢思考型大型语言模型（Large Language Models, LLMs）在推理过程中因无法准确识别知识边界而产生的严重幻觉问题。其解决方案的关键在于提出一种基于知识增强的强化学习方法——KnowRL，通过在强化学习训练过程中引入基于知识验证的事实性奖励机制，引导模型进行以事实为基础的慢思考，从而帮助模型识别其知识边界，并内化基于事实的推理策略。
链接: https://arxiv.org/abs/2506.19807
作者: Baochang Ren,Shuofei Qiao,Wenhao Yu,Huajun Chen,Ningyu Zhang
机构: 未知
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multiagent Systems (cs.MA)
备注: Work in progress
Abstract:Large Language Models (LLMs), particularly slow-thinking models, often exhibit severe hallucination, outputting incorrect content due to an inability to accurately recognize knowledge boundaries during reasoning. While Reinforcement Learning (RL) can enhance complex reasoning abilities, its outcome-oriented reward mechanism often lacks factual supervision over the thinking process, further exacerbating the hallucination problem. To address the high hallucination in slow-thinking models, we propose Knowledge-enhanced RL, KnowRL. KnowRL guides models to perform fact-based slow thinking by integrating a factuality reward, based on knowledge verification, into the RL training process, helping them recognize their knowledge boundaries. KnowRL guides models to perform fact-based slow thinking by integrating a factuality reward, based on knowledge verification, into the RL training process, helping them recognize their knowledge boundaries. This targeted factual input during RL training enables the model to learn and internalize fact-based reasoning strategies. By directly rewarding adherence to facts within the reasoning steps, KnowRL fosters a more reliable thinking process. Experimental results on three hallucination evaluation datasets and two reasoning evaluation datasets demonstrate that KnowRL effectively mitigates hallucinations in slow-thinking models while maintaining their original strong reasoning capabilities. Our code is available at this https URL.
zh
[NLP-7] LLM -Based Social Simulations Require a Boundary
【速读】： 该论文试图解决生成式 AI（Generative AI）在基于大语言模型（LLM）的社会模拟中所面临的可靠性问题，特别是在社会科学研究中用于发现社会模式时的局限性。其核心问题是LLM倾向于形成“平均人格”，缺乏足够的行为异质性，这限制了其对复杂社会动态的模拟能力。解决方案的关键在于确立清晰的边界，以确保LLM-based社会模拟能够可靠地推动社会科学研究，具体包括关注集体模式而非个体轨迹、代理行为与真实人口均值的一致性，以及具备有效的验证方法来测试模拟的鲁棒性。
链接: https://arxiv.org/abs/2506.19806
作者: Zengqing Wu,Run Peng,Takayuki Ito,Chuan Xiao
机构: Kyoto University (京都大学); Osaka University (大阪大学); University of Michigan (密歇根大学); Nagoya University (名古屋大学)
类目: Computers and Society (cs.CY); Computation and Language (cs.CL); Multiagent Systems (cs.MA)
备注:
Abstract:This position paper argues that large language model (LLM)-based social simulations should establish clear boundaries to meaningfully contribute to social science research. While LLMs offer promising capabilities for modeling human-like agents compared to traditional agent-based modeling, they face fundamental limitations that constrain their reliability for social pattern discovery. The core issue lies in LLMs’ tendency towards an ``average persona’’ that lacks sufficient behavioral heterogeneity, a critical requirement for simulating complex social dynamics. We examine three key boundary problems: alignment (simulated behaviors matching real-world patterns), consistency (maintaining coherent agent behavior over time), and robustness (reproducibility under varying conditions). We propose heuristic boundaries for determining when LLM-based simulations can reliably advance social science understanding. We believe that these simulations are more valuable when focusing on (1) collective patterns rather than individual trajectories, (2) agent behaviors aligning with real population averages despite limited variance, and (3) proper validation methods available for testing simulation robustness. We provide a practical checklist to guide researchers in determining the appropriate scope and claims for LLM-based social simulations.
zh
[NLP-8] Why Do Open-Source LLM s Struggle with Data Analysis? A Systematic Empirical Study
【速读】： 该论文试图解决开放源代码大型语言模型（Large Language Models, LLMs）在数据解析任务中表现不足的问题，特别是在需要复杂推理的场景下。其解决方案的关键在于通过构建多样化且真实的场景数据集，评估模型在数据理解、代码生成和战略规划三个维度上的能力，并基于分析结果提出一种数据合成方法，从而显著提升开放源代码LLMs的分析推理能力。
链接: https://arxiv.org/abs/2506.19794
作者: Yuqi Zhu,Yi Zhong,Jintian Zhang,Ziheng Zhang,Shuofei Qiao,Yujie Luo,Lun Du,Da Zheng,Huajun Chen,Ningyu Zhang
机构: Zhejiang University (浙江大学); Ant Group (蚂蚁集团); Independent Researcher (独立研究员); Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph (浙江大学-蚂蚁集团知识图谱联合实验室)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG); Multiagent Systems (cs.MA)
备注: Work in progress
Abstract:Large Language Models (LLMs) hold promise in automating data analysis tasks, yet open-source models face significant limitations in these kinds of reasoning-intensive scenarios. In this work, we investigate strategies to enhance the data analysis capabilities of open-source LLMs. By curating a seed dataset of diverse, realistic scenarios, we evaluate models across three dimensions: data understanding, code generation, and strategic planning. Our analysis reveals three key findings: (1) Strategic planning quality serves as the primary determinant of model performance; (2) Interaction design and task complexity significantly influence reasoning capabilities; (3) Data quality demonstrates a greater impact than diversity in achieving optimal performance. We leverage these insights to develop a data synthesis methodology, demonstrating significant improvements in open-source LLMs’ analytical reasoning capabilities.
zh
[NLP-9] SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
【速读】： 该论文试图解决大型语言模型（Large Language Models, LLMs）中监督微调（Supervised Fine-Tuning, SFT）与强化学习（Reinforcement Learning, RL）最优融合的问题，这一问题仍是基础性挑战。解决方案的关键在于通过熵（entropy）感知的加权机制，提出一种单阶段的监督强化微调（Supervised Reinforcement Fine-Tuning, SRFT）方法，该方法统一了SFT与RL两种微调范式，直接利用演示和自探索轨迹对LLM进行联合优化，而非依赖于两阶段的顺序方法。
链接: https://arxiv.org/abs/2506.19767
作者: Yuqian Fu,Tinghong Chen,Jiajun Chai,Xihuai Wang,Songjun Tu,Guojun Yin,Wei Lin,Qichao Zhang,Yuanheng Zhu,Dongbin Zhao
机构: Institute of Automation, Chinese Academy of Sciences (中国科学院自动化研究所); School of Artificial Intelligence, University of Chinese Academy of Sciences (中国科学院大学人工智能学院); Meituan (美团); Shanghai Jiao Tong University (上海交通大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Large language models (LLMs) have achieved remarkable progress in reasoning tasks, yet the optimal integration of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) remains a fundamental challenge. Through comprehensive analysis of token distributions, learning dynamics, and integration mechanisms from entropy-based perspectives, we reveal key differences between these paradigms: SFT induces coarse-grained global changes to LLM policy distributions, while RL performs fine-grained selective optimizations, with entropy serving as a critical indicator of training effectiveness. Building on these observations, we propose Supervised Reinforcement Fine-Tuning (SRFT), a single-stage method that unifies both fine-tuning paradigms through entropy-aware weighting mechanisms. Our approach simultaneously applies SFT and RL to directly optimize the LLM using demonstrations and self-exploration rollouts rather than through two-stage sequential methods. Extensive experiments show that SRFT achieves 59.1% average accuracy, outperforming zero-RL methods by 9.0% on five mathematical reasoning benchmarks and 10.9% on three out-of-distribution benchmarks.
zh
[NLP-10] Accurate fast cheap: Choose three. Replacing Multi-Head-Attention with Bidirectional Recurrent Attention for Long-Form ASR INTERSPEECH2025
【速读】： 该论文试图解决长文本语音识别（long-form speech recognition）中基于多头注意力（multi-head attention, MHA）的自动语音识别（ASR）模型因序列长度复杂度呈二次增长而效率低下的问题。解决方案的关键在于采用线性复杂度的循环注意力（recurrent attention, RA）层，实验表明双向RA层在短文本和长文本应用中均可达到与MHA相当的准确性，同时具备更高的计算效率。此外，研究还提出了有限上下文注意力（limited-context attention, LCA）基线和一种新的训练范式，进一步提升了RA的性能，并引入了方向丢弃（Direction Dropout）以优化准确率与吞吐量之间的权衡。
链接: https://arxiv.org/abs/2506.19761
作者: Martin Ratajczak,Jean-Philippe Robichaud,Jennifer Drexler Fox
机构: 未知
类目: Computation and Language (cs.CL)
备注: Accepted to Interspeech 2025
Abstract:Long-form speech recognition is an application area of increasing research focus. ASR models based on multi-head attention (MHA) are ill-suited to long-form ASR because of their quadratic complexity in sequence length. We build on recent work that has investigated linear complexity recurrent attention (RA) layers for ASR. We find that bidirectional RA layers can match the accuracy of MHA for both short- and long-form applications. We present a strong limited-context attention (LCA) baseline, and show that RA layers are just as accurate while being more efficient. We develop a long-form training paradigm which further improves RA performance, leading to better accuracy than LCA with 44% higher throughput. We also present Direction Dropout, a novel regularization method that improves accuracy, provides fine-grained control of the accuracy/throughput trade-off of bidirectional RA, and enables a new alternating directions decoding mode with even higher throughput.
zh
[NLP-11] Arabic Dialect Classification using RNNs Transformers and Large Language Models : A Comparative Analysis
【速读】： 该论文试图解决阿拉伯语方言分类问题，具体针对QADI数据集中的18种阿拉伯语方言进行识别。解决方案的关键在于利用先进的预处理技术和最新的自然语言处理模型，包括RNN、Transformer以及通过提示工程优化的大语言模型（LLMs），其中MARBERTv2在实验中表现最佳，达到了65%的准确率和64%的F1分数。
链接: https://arxiv.org/abs/2506.19753
作者: Omar A.Essameldin,Ali O.Elbeih,Wael H.Gomaa,Wael F.Elsersy
机构: MSA University (MSA大学); Beni-Suef University (本哈大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:The Arabic language is among the most popular languages in the world with a huge variety of dialects spoken in 22 countries. In this study, we address the problem of classifying 18 Arabic dialects of the QADI dataset of Arabic tweets. RNN models, Transformer models, and large language models (LLMs) via prompt engineering are created and tested. Among these, MARBERTv2 performed best with 65% accuracy and 64% F1-score. Through the use of state-of-the-art preprocessing techniques and the latest NLP models, this paper identifies the most significant linguistic issues in Arabic dialect identification. The results corroborate applications like personalized chatbots that respond in users’ dialects, social media monitoring, and greater accessibility for Arabic communities.
zh
[NLP-12] Evaluating Rare Disease Diagnostic Performance in Symptom Checkers: A Synthetic Vignette Simulation Approach
【速读】： 该论文旨在解决罕见疾病在症状检查器（Symptom Checker, SC）算法更新后诊断性能评估困难的问题，特别是在缺乏足够评估数据的情况下。其解决方案的关键在于利用人类表型本体（Human Phenotype Ontology, HPO）中的疾病-表型注释生成合成临床案例，从而模拟SC访谈并预测算法更新对实际诊断性能的影响。该方法通过回顾性比较估计值与实际指标变化，验证了其有效性，表明其能够准确预测罕见疾病在算法更新后的表现。
链接: https://arxiv.org/abs/2506.19750
作者: Takashi Nishibayashi,Seiji Kanazawa,Kumpei Yamada
机构: Ubie, Inc.(Ubie公司)
类目: Computation and Language (cs.CL)
备注:
Abstract:Background: Symptom Checkers (SCs) provide users with personalized medical information. To prevent performance degradation from algorithm updates, SC developers must evaluate diagnostic performance changes for individual diseases before deployment. However, acquiring sufficient evaluation data for rare diseases is difficult, and manually creating numerous clinical vignettes is costly and impractical. Objective: This study proposes and validates a novel Synthetic Vignette Simulation Approach to evaluate diagnostic performance changes for individual rare diseases following SC algorithm updates. Methods: We used disease-phenotype annotations from the Human Phenotype Ontology (HPO), a knowledge database for rare diseases, to generate synthetic vignettes. With these, we simulated SC interviews to estimate the impact of algorithm updates on real-world diagnostic performance. The method’s effectiveness was evaluated retrospectively by comparing estimated values with actual metric changes using the R 2(R-squared) coefficient. Results: The experiment included eight past SC algorithm updates. For updates on diseases with frequency information in HPO (n=5), the R^2 for recall@8 change was 0.831 (p=0.031), and for precision@8 change, it was 0.78 (p=0.047), indicating the method can predict post-deployment performance. In contrast, large prediction errors occurred for diseases without frequency information (n=3), highlighting its importance. The manual effort to map HPO phenotypes to SC symptoms was approximately 2 hours per disease. Conclusions: Our method enables pre-deployment evaluation of SC algorithm changes for individual rare diseases using a publicly available, expert-created knowledge base. This transparent and low-cost approach allows developers to efficiently improve diagnostic performance for rare diseases, potentially enhancing support for early diagnosis.
zh
[NLP-13] NEAR2: A Nested Embedding Approach to Efficient Product Retrieval and Ranking SIGIR
【速读】： 该论文试图解决电子商务信息检索（E-commerce Information Retrieval, IR）系统在同时实现复杂用户查询的高精度理解和高效处理大规模产品目录之间的矛盾。解决方案的关键在于提出一种称为NEAR²的嵌套嵌入方法，该方法能够在推理阶段将嵌入尺寸提高12倍效率的同时，不增加训练成本，并提升基于编码器的Transformer模型的准确性。通过在不同损失函数（如多负样本排序损失和在线对比损失）以及多个具有不同IR挑战的数据集上进行验证，该方法在较小的嵌入维度下实现了优于现有模型的性能。
链接: https://arxiv.org/abs/2506.19743
作者: Shenbin Qian,Diptesh Kanojia,Samarth Agrawal,Hadeel Saadany,Swapnil Bhosale,Constantin Orasan,Zhe Wu
机构: University of Surrey (萨里大学); eBay Inc (eBay公司); Birmingham City University (伯明翰城市大学)
类目: Information Retrieval (cs.IR); Computation and Language (cs.CL)
备注: This paper is accepted to the 2025 SIGIR Workshop on eCommerce
Abstract:E-commerce information retrieval (IR) systems struggle to simultaneously achieve high accuracy in interpreting complex user queries and maintain efficient processing of vast product catalogs. The dual challenge lies in precisely matching user intent with relevant products while managing the computational demands of real-time search across massive inventories. In this paper, we propose a Nested Embedding Approach to product Retrieval and Ranking, called NEAR ^2 , which can achieve up to 12 times efficiency in embedding size at inference time while introducing no extra cost in training and improving performance in accuracy for various encoder-based Transformer models. We validate our approach using different loss functions for the retrieval and ranking task, including multiple negative ranking loss and online contrastive loss, on four different test sets with various IR challenges such as short and implicit queries. Our approach achieves an improved performance over a smaller embedding dimension, compared to any existing models.
zh
[NLP-14] Breaking Barriers: Do Reinforcement Post Training Gains Transfer To Unseen Domains?
【速读】： 该论文试图解决强化后训练（Reinforcement Post Training, RPT）在大型语言模型（Large Language Models, LLMs）中提升推理能力后的泛化能力问题，即RPT带来的性能提升是否能够有效迁移至未见过的领域。解决方案的关键在于通过两个研究：观察性研究与干预性研究，系统评估RPT模型在不同领域（包括已见和未见领域）的表现，从而揭示RPT在不同推理模式下的泛化一致性。
链接: https://arxiv.org/abs/2506.19733
作者: Chuxuan Hu,Yuxuan Zhu,Antony Kellermann,Caleb Biddulph,Suppakit Waiwitlikhit,Jason Benn,Daniel Kang
机构: University of Illinois Urbana-Champaign (伊利诺伊大学厄巴纳-香槟分校)
类目: Computation and Language (cs.CL)
备注: 9 pages, 4 figures, 2 tables
Abstract:Reinforcement post training (RPT) has recently shown promise in improving the reasoning abilities of large language models (LLMs). However, it remains unclear how well these improvements generalize to new domains, as prior work evaluates RPT models on data from the same domains used for fine-tuning. To understand the generalizability of RPT, we conduct two studies. (1) Observational: We compare a wide range of open-weight RPT models against their corresponding base models across multiple domains, including both seen and unseen domains in their fine-tuning data. (2) Interventional: we fine-tune LLMs with RPT on single domains and evaluate their performance across multiple domains. Both studies converge on the same conclusion that, although RPT brings substantial gains on tasks similar to the fine-tuning data, the gains generalize inconsistently and can vanish on domains with different reasoning patterns.
zh
[NLP-15] Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
【速读】： 该论文旨在解决大型语言模型（Large Language Models, LLMs）中极端激活异常值（extreme activation outliers）对量化性能的严重影响，从而阻碍高效设备端部署的问题。其解决方案的关键在于提出一种名为Outlier-Safe Pre-Training (OSP)的实用指南，该方法通过主动防止异常值的形成而非依赖事后缓解措施来改善量化行为。OSP结合了三项核心创新：(1) Muon优化器，消除特权基底同时保持训练效率；(2) Single-Scale RMSNorm，防止通道级放大；(3) 可学习的嵌入投影，重新分布源自嵌入矩阵的激活幅度。
链接: https://arxiv.org/abs/2506.19697
作者: Jungwoo Park,Taewhoo Lee,Chanwoong Yoon,Hyeon Hwang,Jaewoo Kang
机构: Korea University (韩国大学); AIGEN Sciences (AIGEN 科学)
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注:
Abstract:Extreme activation outliers in Large Language Models (LLMs) critically degrade quantization performance, hindering efficient on-device deployment. While channel-wise operations and adaptive gradient scaling are recognized causes, practical mitigation remains challenging. We introduce Outlier-Safe Pre-Training (OSP), a practical guideline that proactively prevents outlier formation rather than relying on post-hoc mitigation. OSP combines three key innovations: (1) the Muon optimizer, eliminating privileged bases while maintaining training efficiency; (2) Single-Scale RMSNorm, preventing channel-wise amplification; and (3) a learnable embedding projection, redistributing activation magnitudes originating from embedding matrices. We validate OSP by training a 1.4B-parameter model on 1 trillion tokens, which is the first production-scale LLM trained without such outliers. Under aggressive 4-bit quantization, our OSP model achieves a 35.7 average score across 10 benchmarks (compared to 26.5 for an Adam-trained model), with only a 2% training overhead. Remarkably, OSP models exhibit near-zero excess kurtosis (0.04) compared to extreme values (1818.56) in standard models, fundamentally altering LLM quantization behavior. Our work demonstrates that outliers are not inherent to LLMs but are consequences of training strategies, paving the way for more efficient LLM deployment. The source code and pretrained checkpoints are available at this https URL.
zh
[NLP-16] Recurrent Visual Feature Extraction and Stereo Attentions for CT Report Generation
【速读】： 该论文旨在解决CT图像报告生成（CTRG）中的挑战性问题，特别是在处理多图的空间编码、图像体积与文本之间的对齐等方面。现有方法通常采用通用的2D或3D图像处理技术提取特征，但未能显式建模CT切片间的变换关系，也未能有效融合包含特定器官病变的多层次图像特征。该论文的关键解决方案是提出一种基于大语言模型（LLM）的CTRG方法，其核心在于递归视觉特征提取和立体注意力机制，以实现层次化特征建模。具体而言，利用视觉Transformer递归处理CT体积中的每张切片，并通过多视角注意力机制选择性地获取重要视觉信息，将其与文本特征对齐，从而更有效地指导LLM生成CT报告。
链接: https://arxiv.org/abs/2506.19665
作者: Yuanhe Tian,Lei Mao,Yan Song
机构: University of Washington, USA (华盛顿大学); Origin Omics, China (起源组学); University of Science and Technology of China, China (中国科学技术大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
备注: 7 pages, 3 figures
Abstract:Generating reports for computed tomography (CT) images is a challenging task, while similar to existing studies for medical image report generation, yet has its unique characteristics, such as spatial encoding of multiple images, alignment between image volume and texts, etc. Existing solutions typically use general 2D or 3D image processing techniques to extract features from a CT volume, where they firstly compress the volume and then divide the compressed CT slices into patches for visual encoding. These approaches do not explicitly account for the transformations among CT slices, nor do they effectively integrate multi-level image features, particularly those containing specific organ lesions, to instruct CT report generation (CTRG). In considering the strong correlation among consecutive slices in CT scans, in this paper, we propose a large language model (LLM) based CTRG method with recurrent visual feature extraction and stereo attentions for hierarchical feature modeling. Specifically, we use a vision Transformer to recurrently process each slice in a CT volume, and employ a set of attentions over the encoded slices from different perspectives to selectively obtain important visual information and align them with textual features, so as to better instruct an LLM for CTRG. Experiment results and further analysis on the benchmark M3D-Cap dataset show that our method outperforms strong baseline models and achieves state-of-the-art results, demonstrating its validity and effectiveness.
zh
[NLP-17] ailored Conversations beyond LLM s: A RL-Based Dialogue Manager
【速读】： 该论文旨在解决开放性对话系统中如何实现高效、自适应且具有特定目标的对话管理问题。其关键解决方案是将大语言模型（Large Language Models, LLMs）与基于强化学习（Reinforcement Learning, RL）的对话管理器相结合，通过分层强化学习建模对话的结构化阶段，并利用元学习提升系统在不同用户群体中的适应能力，从而实现从有限数据中学习、流畅地在对话阶段间转换以及个性化响应异质性用户需求的目标。
链接: https://arxiv.org/abs/2506.19652
作者: Lucie Galland,Catherine Pelachaud,Florian Pecune
机构: ISIR (ISIR); Sorbonne University (索邦大学); CNRS/ISIR (CNRS/ISIR); CNRS/SANPSY (CNRS/SANPSY); Univ. Bordeaux (波尔多大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:In this work, we propose a novel framework that integrates large language models (LLMs) with an RL-based dialogue manager for open-ended dialogue with a specific goal. By leveraging hierarchical reinforcement learning to model the structured phases of dialogue and employ meta-learning to enhance adaptability across diverse user profiles, our approach enhances adaptability and efficiency, enabling the system to learn from limited data, transition fluidly between dialogue phases, and personalize responses to heterogeneous patient needs. We apply our framework to Motivational Interviews, aiming to foster behavior change, and demonstrate that the proposed dialogue manager outperforms a state-of-the-art LLM baseline in terms of reward, showing a potential benefit of conditioning LLMs to create open-ended dialogue systems with specific goals.
zh
[NLP-18] Correcting Hallucinations in News Summaries: Exploration of Self-Correcting LLM Methods with External Knowledge ACL2025
【速读】： 该论文试图解决大型语言模型（Large Language Models, LLMs）在生成文本时产生的幻觉（hallucinations）问题，即生成事实性不准确的陈述。其解决方案的关键在于采用自校正（self-correcting）方法，通过利用LLMs的多轮交互特性，迭代生成验证问题以获取额外证据，并结合内部或外部知识进行回答，从而修正原始响应。该方法在百科全书生成中已有研究，但本文将其应用于新闻摘要领域，探索了基于搜索引擎片段和少量示例提示（few-shot prompts）的有效性，以及G-Eval与人工评估的一致性。
链接: https://arxiv.org/abs/2506.19607
作者: Juraj Vladika,Ihsan Soydemir,Florian Matthes
机构: Technical University of Munich (慕尼黑工业大学)
类目: Computation and Language (cs.CL)
备注: Accepted to FEVER @ ACL 2025
Abstract:While large language models (LLMs) have shown remarkable capabilities to generate coherent text, they suffer from the issue of hallucinations – factually inaccurate statements. Among numerous approaches to tackle hallucinations, especially promising are the self-correcting methods. They leverage the multi-turn nature of LLMs to iteratively generate verification questions inquiring additional evidence, answer them with internal or external knowledge, and use that to refine the original response with the new corrections. These methods have been explored for encyclopedic generation, but less so for domains like news summarization. In this work, we investigate two state-of-the-art self-correcting systems by applying them to correct hallucinated summaries using evidence from three search engines. We analyze the results and provide insights into systems’ performance, revealing interesting practical findings on the benefits of search engine snippets and few-shot prompts, as well as high alignment of G-Eval and human evaluation.
zh
[NLP-19] Social Hatred: Efficient Multimodal Detection of Hatemongers WOAH
【速读】： 该论文试图解决在线仇恨言论的自动检测问题，特别是从用户层面识别煽动仇恨的人（hate-mongers）。传统方法主要关注话语层面的仇恨言论检测，而本文认为用户层面的分析同样重要且具有挑战性。解决方案的关键在于采用多模态聚合方法，综合考虑用户的文本内容、用户行为以及用户网络信息，从而在社交语境中更准确地识别仇恨煽动者。实验结果表明，该方法在多个数据集上的表现优于以往基于文本和图的方法。
链接: https://arxiv.org/abs/2506.19603
作者: Tom Marzea,Abraham Israeli,Oren Tsur
机构: Ben Gurion University (本·古里安大学); University of Michigan (密歇根大学)
类目: Computation and Language (cs.CL); Social and Information Networks (cs.SI)
备注: To be published in WOAH, July 2025. arXiv admin note: text overlap with arXiv:2409.14464
Abstract:Automatic detection of online hate speech serves as a crucial step in the detoxification of the online discourse. Moreover, accurate classification can promote a better understanding of the proliferation of hate as a social phenomenon. While most prior work focus on the detection of hateful utterances, we argue that focusing on the user level is as important, albeit challenging. In this paper we consider a multimodal aggregative approach for the detection of hate-mongers, taking into account the potentially hateful texts, user activity, and the user network. Evaluating our method on three unique datasets X (Twitter), Gab, and Parler we show that processing a user’s texts in her social context significantly improves the detection of hate mongers, compared to previously used text and graph-based methods. We offer comprehensive set of results obtained in different experimental settings as well as qualitative analysis of illustrative cases. Our method can be used to improve the classification of coded messages, dog-whistling, and racial gas-lighting, as well as to inform intervention measures. Moreover, we demonstrate that our multimodal approach performs well across very different content platforms and over large datasets and networks.
zh
[NLP-20] ECCoT: A Framework for Enhancing Effective Cognition via Chain of Thought in Large Language Model
【速读】： 该论文试图解决大型语言模型（Large Language Models, LLMs）在推理过程中缺乏透明性及生成不可靠输出的问题，进而影响其可解释性和可信度。解决方案的关键在于提出ECCoT框架，该框架通过集成基于马尔可夫随机场嵌入主题模型（MRF-ETM）的主题感知推理链生成方法以及因果句向量模型（CSBert）的因果推理对齐机制，实现对推理链的有效评估与优化。ECCoT通过结构化排序统计过滤无效推理链，从而提升模型的可解释性、减少偏差并增强基于LLMs决策的可信度。
链接: https://arxiv.org/abs/2506.19599
作者: Zhenke Duan,Jiqun Pan,Jiani Tu,Xiaoyi Wang,Yanqing Wang
机构: Zhongnan University of Economics and Law (中南财经政法大学); School of Statistics and Mathematics (统计与数学学院); School of Finance (金融学院)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:In the era of large-scale artificial intelligence, Large Language Models (LLMs) have made significant strides in natural language processing. However, they often lack transparency and generate unreliable outputs, raising concerns about their interpretability. To address this, the Chain of Thought (CoT) prompting method structures reasoning into step-by-step deductions. Yet, not all reasoning chains are valid, and errors can lead to unreliable conclusions. We propose ECCoT, an End-to-End Cognitive Chain of Thought Validation Framework, to evaluate and refine reasoning chains in LLMs. ECCoT integrates the Markov Random Field-Embedded Topic Model (MRF-ETM) for topic-aware CoT generation and Causal Sentence-BERT (CSBert) for causal reasoning alignment. By filtering ineffective chains using structured ordering statistics, ECCoT improves interpretability, reduces biases, and enhances the trustworthiness of LLM-based decision-making. Key contributions include the introduction of ECCoT, MRF-ETM for topic-driven CoT generation, and CSBert for causal reasoning enhancement. Code is released at: this https URL.
zh
[NLP-21] Fake or Real Can Robots Tell? Evaluating Embodied Vision-Language Models on Real and 3D-Printed Objects
【速读】： 该论文试图解决机器人在桌面上场景理解中如何有效生成自然语言描述的问题，特别是在多视角图像和真实世界与3D打印物体识别之间的性能差异。解决方案的关键在于比较不同captioning策略（如BLIP和视觉-语言模型VLMs）在生成场景描述时的表现，评估其在对象识别准确性、描述完整性和自然性方面的效果，并探讨单视角与多视角描述以及真实物体与3D打印物体之间的差异。
链接: https://arxiv.org/abs/2506.19579
作者: Federico Tavella,Kathryn Mearns,Angelo Cangelosi
机构: 未知
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Robotic scene understanding increasingly relies on vision-language models (VLMs) to generate natural language descriptions of the environment. In this work, we present a comparative study of captioning strategies for tabletop scenes captured by a robotic arm equipped with an RGB camera. The robot collects images of objects from multiple viewpoints, and we evaluate several models that generate scene descriptions. We compare the performance of various captioning models, like BLIP and VLMs. Our experiments examine the trade-offs between single-view and multi-view captioning, and difference between recognising real-world and 3D printed objects. We quantitatively evaluate object identification accuracy, completeness, and naturalness of the generated captions. Results show that VLMs can be used in robotic settings where common objects need to be recognised, but fail to generalise to novel representations. Our findings provide practical insights into deploying foundation models for embodied agents in real-world settings.
zh
[NLP-22] Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progress ACL2025
【速读】： 该论文试图解决机器翻译（Machine Translation, MT）评估中自动评估指标与人类判断之间的一致性问题，以及如何准确衡量MT评估的进展。其解决方案的关键在于引入人类基线进行元评估（meta-evaluation），即对MT评估指标的能力进行评估，从而更清晰地理解指标性能并确定上限。研究结果表明，最先进的自动指标在多数情况下与人类基线相当甚至优于人类，这挑战了传统认为人类判断始终更优的假设。
链接: https://arxiv.org/abs/2506.19571
作者: Lorenzo Proietti,Stefano Perrella,Roberto Navigli
机构: Sapienza NLP Group, Sapienza University of Rome (萨皮恩扎自然语言处理小组，罗马大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: Accepted at ACL 2025 Main Conference. 24 pages
Abstract:In Machine Translation (MT) evaluation, metric performance is assessed based on agreement with human judgments. In recent years, automatic metrics have demonstrated increasingly high levels of agreement with humans. To gain a clearer understanding of metric performance and establish an upper bound, we incorporate human baselines in the MT meta-evaluation, that is, the assessment of MT metrics’ capabilities. Our results show that human annotators are not consistently superior to automatic metrics, with state-of-the-art metrics often ranking on par with or higher than human baselines. Despite these findings suggesting human parity, we discuss several reasons for caution. Finally, we explore the broader implications of our results for the research field, asking: Can we still reliably measure improvements in MT evaluation? With this work, we aim to shed light on the limits of our ability to measure progress in the field, fostering discussion on an issue that we believe is crucial to the entire MT evaluation community.
zh
[NLP-23] RCStat: A Statistical Framework for using Relative Contextualization in Transformers
【速读】： 该论文旨在解决自回归Transformer模型中输入标记重要性评估的问题，传统方法依赖于Softmax归一化的注意力权重，这掩盖了预Softmax查询-键（query-key）logits的更丰富结构。解决方案的关键在于提出RCStat，这是一种基于原始注意力logits的统计框架，通过相对情境化（Relative Contextualization, RC）这一随机变量来衡量标记段之间的上下文对齐程度，并推导出RC的有效上界。该方法在关键值压缩和归因任务中表现出色，实现了显著的性能提升。
链接: https://arxiv.org/abs/2506.19549
作者: Debabrata Mahapatra,Shubham Agarwal,Apoorv Saxena,Subrata Mitra
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Prior work on input-token importance in auto-regressive transformers has relied on Softmax-normalized attention weights, which obscure the richer structure of pre-Softmax query-key logits. We introduce RCStat, a statistical framework that harnesses raw attention logits via Relative Contextualization (RC), a random variable measuring contextual alignment between token segments, and derive an efficient upper bound for RC. We demonstrate two applications: (i) Key-Value compression, where RC-based thresholds drive adaptive key-value eviction for substantial cache reduction with minimal quality loss; and (ii) Attribution, where RC yields higher-fidelity token-, sentence-, and chunk-level explanations than post-Softmax methods. Across question answering, summarization, and attribution benchmarks, RCStat achieves significant empirical gains, delivering state-of-the-art compression and attribution performance without any model retraining.
zh
[NLP-24] Health Sentinel: An AI Pipeline For Real-time Disease Outbreak Detection
【速读】： 该论文试图解决传统基于指标的疾病监测方法在应对突发疾病事件时所面临的挑战，特别是通过人工筛选海量在线文章进行早期疾病暴发检测的不切实际性。解决方案的关键在于提出Health Sentinel，这是一个多阶段的信息提取管道，结合机器学习（ML）与非机器学习（non-ML）方法，从在线文章中提取结构化的疾病暴发或其他异常健康事件信息，从而为公共卫生机构提供及时的预警和干预支持。
链接: https://arxiv.org/abs/2506.19548
作者: Devesh Pant,Rishi Raj Grandhe,Vipin Samaria,Mukul Paul,Sudhir Kumar,Saransh Khanna,Jatin Agrawal,Jushaan Singh Kalra,Akhil VSSG,Satish V Khalikar,Vipin Garg,Himanshu Chauhan,Pranay Verma,Neha Khandelwal,Soma S Dhavala,Minesh Mathew
机构: Wadhwani AI (瓦德哈尼人工智能); National Centre for Disease Control (国家疾病控制中心)
类目: Computation and Language (cs.CL); Information Retrieval (cs.IR)
备注:
Abstract:Early detection of disease outbreaks is crucial to ensure timely intervention by the health authorities. Due to the challenges associated with traditional indicator-based surveillance, monitoring informal sources such as online media has become increasingly popular. However, owing to the number of online articles getting published everyday, manual screening of the articles is impractical. To address this, we propose Health Sentinel. It is a multi-stage information extraction pipeline that uses a combination of ML and non-ML methods to extract events-structured information concerning disease outbreaks or other unusual health events-from online articles. The extracted events are made available to the Media Scanning and Verification Cell (MSVC) at the National Centre for Disease Control (NCDC), Delhi for analysis, interpretation and further dissemination to local agencies for timely intervention. From April 2022 till date, Health Sentinel has processed over 300 million news articles and identified over 95,000 unique health events across India of which over 3,500 events were shortlisted by the public health experts at NCDC as potential outbreaks.
zh
[NLP-25] KnowMap: Efficient Knowledge-Driven Task Adaptation for LLM s
【速读】： 该论文试图解决大型语言模型（Large Language Models, LLMs）在面对新任务时难以快速适应的问题，尤其是在依赖静态预训练知识的情况下。传统方法如微调（fine-tuning）存在成本高、数据需求大及可能导致“灾难性遗忘”的缺点。论文提出的解决方案是KnowMap，其关键在于通过动态构建知识库，将环境和经验数据中的知识嵌入到一个小型知识嵌入模型中，并以此为大模型提供任务相关的知识，从而提升其性能。
链接: https://arxiv.org/abs/2506.19527
作者: Kelin Fu,Kaigui Bian
机构: Peking University (北京大学)
类目: Computation and Language (cs.CL)
备注:
Abstract:While Large Language Models (LLMs) possess significant capabilities in open-world agent tasks, they also face challenges in rapidly adapting to new, specialized tasks due to their reliance on static pre-trained knowledge. Traditional methods such as fine-tuning are often costly, data-intensive, and may lead to “catastrophic forgetting.” Therefore, we present KnowMap, a novel approach that dynamically constructs a knowledge base from environmental and experiential data. KnowMap fine-tunes a small knowledge-embedding model to equip a larger LLM with valuable task-specific knowledge. Our experiments on the ScienceWorld benchmark demonstrate 17.71% improvement for the performance of gpt-4-turbo model. KnowMap not only provides an efficient and effective means for LLM task-adapting, but also highlights how integrating environmental and experiential knowledge can enhance LLMs’ reasoning capabilities.
zh
[NLP-26] Automatic Posology Structuration : What role for LLM s?
【速读】： 该论文试图解决法语处方中用药指导（posology）结构化不足的问题，此类指导常存在歧义、不规则或口语化现象，限制了传统机器学习流水线的效果。解决方案的关键在于利用大语言模型（Large Language Models, LLMs）将自由文本的用药指导转换为结构化格式，并通过对比基于提示的方法与微调方法，发现只有微调后的LLMs能够达到基准系统的准确性。研究进一步提出了一种混合流水线，将NERL（命名实体识别与链接）系统置信度较低的案例（置信度低于0.8）路由至LLM处理，依据置信度选择输出，从而在保持高结构化准确率（91%）的同时降低延迟和计算成本。
链接: https://arxiv.org/abs/2506.19525
作者: Natalia Bobkova,Laura Zanella-Calzada,Anyes Tafoughalt,Raphaël Teboul,François Plesse,Félix Gaschi
机构: SAS Posos(萨斯波索); Sorbonne Université(索邦大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:Automatically structuring posology instructions is essential for improving medication safety and enabling clinical decision support. In French prescriptions, these instructions are often ambiguous, irregular, or colloquial, limiting the effectiveness of classic ML pipelines. We explore the use of Large Language Models (LLMs) to convert free-text posologies into structured formats, comparing prompt-based methods and fine-tuning against a “pre-LLM” system based on Named Entity Recognition and Linking (NERL). Our results show that while prompting improves performance, only fine-tuned LLMs match the accuracy of the baseline. Through error analysis, we observe complementary strengths: NERL offers structural precision, while LLMs better handle semantic nuances. Based on this, we propose a hybrid pipeline that routes low-confidence cases from NERL (0.8) to the LLM, selecting outputs based on confidence scores. This strategy achieves 91% structuration accuracy while minimizing latency and compute. Our results show that this hybrid approach improves structuration accuracy while limiting computational cost, offering a scalable solution for real-world clinical use.
zh
[NLP-27] heiDS at ArchEHR-QA 2025: From Fixed-k to Query-dependent-k for Retrieval Augmented Generation ACL2025
【速读】： 该论文旨在解决临床问答任务中如何从电子健康记录（EHRs）中生成事实准确且相关的答案的问题。其解决方案的关键在于采用一种基于检索增强生成（RAG）框架的管道，重点优化了检索策略和归因方法。具体而言，该研究提出了一种查询依赖的k值检索策略，包括现有的惊喜和自动截断方法以及本文提出的两种新方法——autocut*和elbow，相较于固定k值的检索策略，该方法在生成答案的准确性和相关性方面表现出优势。
链接: https://arxiv.org/abs/2506.19512
作者: Ashish Chouhan,Michael Gertz
机构: 未知
类目: Computation and Language (cs.CL)
备注: 12 pages, 2 figures, 6 tables, Workshop on BioNLP and Shared Tasks at ACL 2025
Abstract:This paper presents the approach of our team called heiDS for the ArchEHR-QA 2025 shared task. A pipeline using a retrieval augmented generation (RAG) framework is designed to generate answers that are attributed to clinical evidence from the electronic health records (EHRs) of patients in response to patient-specific questions. We explored various components of a RAG framework, focusing on ranked list truncation (RLT) retrieval strategies and attribution approaches. Instead of using a fixed top-k RLT retrieval strategy, we employ a query-dependent-k retrieval strategy, including the existing surprise and autocut methods and two new methods proposed in this work, autocut* and elbow. The experimental results show the benefits of our strategy in producing factual and relevant answers when compared to a fixed- k .
zh
[NLP-28] AnTKV: Anchor Token-Aware Sub-Bit Vector Quantization for KV Cache in Large Language Models
【速读】： 该论文旨在解决超低比特KV缓存量化导致的性能退化问题，特别是在减少大型语言模型（Large Language Models, LLMs）中KV缓存内存占用的同时保持模型精度。其解决方案的关键在于通过前向误差传播分析发现不同token的KV缓存对量化误差的敏感性存在显著差异，并提出了Anchor Score (AnS) 来量化这种敏感性。基于此，研究者设计了AnTKV框架，采用Anchor Token-aware Vector Quantization方法，仅对高AnS的token保留全精度，从而在极端量化场景下有效缓解精度损失。
链接: https://arxiv.org/abs/2506.19505
作者: Zeyu Li,Chuanfu Xiao,Yang Wang,Xiang Liu,Zhenheng Tang,Baotong Lu,Mao Yang,Xinyu Chen,Xiaowen Chu
机构: The Hong Kong University of Science and Technology (Guangzhou) (香港科技大学（广州）); PKU-Changsha Institute for Computing and Digital Economy (北京大学长沙计算与数字经济研究院); Microsoft (微软); The Hong Kong University of Science and Technology (香港科技大学)
类目: Computation and Language (cs.CL)
备注:
Abstract:Quantization has emerged as an effective and lightweight solution to reduce the memory footprint of the KV cache in Large Language Models (LLMs). Nevertheless, minimizing the performance degradation caused by ultra-low-bit KV cache quantization remains a significant challenge. We observe that quantizing the KV cache of different tokens has varying impacts on the quality of attention outputs. To systematically investigate this phenomenon, we perform forward error propagation analysis on attention and propose the Anchor Score (AnS) that quantifies the sensitivity of each token’s KV cache to quantization-induced error. Our analysis reveals significant disparities in AnS across tokens, suggesting that preserving a small subset with full precision (FP16) of high-AnS tokens can greatly mitigate accuracy loss in aggressive quantization scenarios. Based on this insight, we introduce AnTKV, a novel framework that leverages Anchor Token-aware Vector Quantization to compress the KV cache. Furthermore, to support efficient deployment, we design and develop a triton kernel that is fully compatible with FlashAttention, enabling fast online Anchor Token selection. AnTKV enables LLaMA-3-8B to handle context lengths up to 840K tokens on a single 80GB A100 GPU, while achieving up to 3.5x higher decoding throughput compared to the FP16 baseline. Our experiment results demonstrate that AnTKV matches or outperforms prior works such as KIVI, SKVQ, KVQuant, and CQ under 4-bit settings. More importantly, AnTKV achieves significantly lower perplexity under ultra-low-bit quantization on Mistral-7B, with only 6.32 at 1-bit and 8.87 at 0.375-bit, compared to the FP16 baseline of 4.73.
zh
[NLP-29] NaviAgent : Bilevel Planning on Tool Dependency Graphs for Function Calling
【速读】： 该论文旨在解决大型语言模型（Large Language Models, LLMs）在处理复杂、异构工具链时面临的静态知识依赖和脆弱的工具调用问题，这些问题严重限制了大规模场景下的工具链编排效果。其解决方案的关键在于提出NaviAgent，这是一种基于图导航的双层规划架构，包含多路径决策器和图编码导航器。多路径决策器通过定义四维决策空间并动态选择最优动作，实现对所有工具调用场景的全面覆盖；图编码导航器则构建了工具依赖异构图（Tool Dependency Heterogeneous Graph, TDHG），通过融合API模式结构与历史调用行为的节点嵌入，并结合一种新颖的启发式搜索策略，引导决策器高效生成高成功率的工具链。
链接: https://arxiv.org/abs/2506.19500
作者: Yan Jiang,Hao Zhou,LiZhong GU,Ai Han,TianLong Li
机构: JD.COM(京东)
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
备注:
Abstract:LLMs’ reliance on static knowledge and fragile tool invocation severely hinders the orchestration of complex, heterogeneous toolchains, particularly at large scales. Existing methods typically use rigid single-path execution, resulting in poor error recovery and exponentially growing search spaces. We introduce NaviAgent, a graph-navigated bilevel planning architecture for robust function calling, comprising a Multi-Path Decider and Graph-Encoded Navigator. As an LLM-powered agent, the Multi-Path Decider defines a four-dimensional decision space and continuously perceives environmental states, dynamically selecting the optimal action to fully cover all tool invocation scenarios. The Graph-Encoded Navigator constructs a Tool Dependency Heterogeneous Graph (TDHG), where node embeddings explicitly fuse API schema structure with historical invocation behavior. It also integrates a novel heuristic search strategy that guides the Decider toward efficient and highly successful toolchains, even for unseen tool combinations. Experiments show that NaviAgent consistently achieves the highest task success rate (TSR) across all foundation models and task complexities, outperforming the average baselines (ReAct, ToolLLM, \alpha-UMI) by 13.5%, 16.4%, and 19.0% on Qwen2.5-14B, Qwen2.5-32B, and Deepseek-V3, respectively. Its execution steps are typically within one step of the most efficient baseline, ensuring a strong balance between quality and efficiency. Notably, a fine-tuned Qwen2.5-14B model achieves a TSR of 49.5%, surpassing the much larger 32B model (44.9%) under our architecture. Incorporating the Graph-Encoded Navigator further boosts TSR by an average of 2.4 points, with gains up over 9 points on complex tasks for larger models (Deepseek-V3 and GPT-4o), highlighting its essential role in toolchain orchestration.
zh
[NLP-30] Is Long-to-Short a Free Lunch? Investigating Inconsistency and Reasoning Efficiency in LRMs
【速读】： 该论文试图解决大型推理模型（Large Reasoning Models, LRMs）在高效推理策略下可能出现的行为不一致性问题，特别是由于压缩推理过程可能导致模型响应的鲁棒性下降以及关键推理步骤的遗漏。解决方案的关键在于引入ICBENCH基准，该基准从三个维度评估LRMs的不一致性：任务设置间的不一致性（ITS）、训练目标与学习行为间的不一致性（TR-LB）以及内部推理与自我解释间的不一致性（IR-SE），从而系统地分析高效推理策略对模型行为的影响。
链接: https://arxiv.org/abs/2506.19492
作者: Shu Yang,Junchao Wu,Xuansheng Wu,Derek Wong,Ninhao Liu,Di Wang
机构: 未知
类目: Computation and Language (cs.CL)
备注:
Abstract:Large Reasoning Models (LRMs) have achieved remarkable performance on complex tasks by engaging in extended reasoning before producing final answers, yet this strength introduces the risk of overthinking, where excessive token generation occurs even for simple tasks. While recent work in efficient reasoning seeks to reduce reasoning length while preserving accuracy, it remains unclear whether such optimization is truly a free lunch. Drawing on the intuition that compressing reasoning may reduce the robustness of model responses and lead models to omit key reasoning steps, we investigate whether efficient reasoning strategies introduce behavioral inconsistencies. To systematically assess this, we introduce ICBENCH , a benchmark designed to measure inconsistency in LRMs across three dimensions: inconsistency across task settings (ITS), inconsistency between training objectives and learned behavior (TR-LB), and inconsistency between internal reasoning and self-explanations (IR-SE). Applying ICBENCH to a range of open-source LRMs, we find that while larger models generally exhibit greater consistency than smaller ones, they all display widespread “scheming” behaviors, including self-disagreement, post-hoc rationalization, and the withholding of reasoning cues. Crucially, our results demonstrate that efficient reasoning strategies such as No-Thinking and Simple Token-Budget consistently increase all three defined types of inconsistency. These findings suggest that although efficient reasoning enhances token-level efficiency, further investigation is imperative to ascertain whether it concurrently introduces the risk of models evading effective supervision.
zh
[NLP-31] Dialogic Pedagogy for Large Language Models : Aligning Conversational AI with Proven Theories of Learning
【速读】： 该论文试图解决如何将生成式 AI (Generative AI) 驱动的对话系统与教育理论有效结合，以提升其在高等教育及更广泛学习场景中的教学效果。其核心问题在于现有教育理论（如维果茨基的社会文化学习理论、苏格拉底方法和劳尔德的对话框架）与 LLM 的行为模式之间存在差距，例如模型倾向于直接提供答案而非促进知识的共同构建。解决方案的关键在于通过优化提示策略（prompting strategies）和引入检索增强生成（RAG）技术，使 LLM 的对话行为更符合教育学原理，从而支持个性化和自适应学习，并提升对话的准确性与情境相关性。
链接: https://arxiv.org/abs/2506.19484
作者: Russell Beale
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
备注:
Abstract:Large Language Models (LLMs) are rapidly transforming education by enabling rich conversational learning experiences. This article provides a comprehensive review of how LLM-based conversational agents are being used in higher education, with extensions to secondary and lifelong learning contexts. We synthesize existing literature on LLMs in education and theories of conversational and dialogic pedagogy - including Vygotsky’s sociocultural learning (scaffolding and the Zone of Proximal Development), the Socratic method, and Laurillard’s conversational framework - and examine how prompting strategies and retrieval-augmented generation (RAG) can align LLM behaviors with these pedagogical theories, and how it can support personalized, adaptive learning. We map educational theories to LLM capabilities, highlighting where LLM-driven dialogue supports established learning principles and where it challenges or falls short of traditional pedagogical assumptions. Notable gaps in applying prior theories to LLMs are identified, such as the models tendency to provide direct answers instead of fostering co-construction of knowledge, and the need to account for the constant availability and broad but non-human expertise of LLM tutors. In response, we propose practical strategies to better align LLM interactions with sound pedagogy - for example, designing prompts that encourage Socratic questioning, scaffolded guidance, and student reflection, as well as integrating retrieval mechanisms to ensure accuracy and contextual relevance. Our aim is to bridge the gap between educational theory and the emerging practice of AI-driven conversational learning, offering insights and tools for making LLM-based dialogues more educationally productive and theory-aligned.
zh
[NLP-32] Commonsense Generation and Evaluation for Dialogue Systems using Large Language Models
【速读】： 该论文试图解决对话系统中基于不同常识关系的回合级数据增强任务以及生成对话回合的自动评估问题。解决方案的关键在于利用预训练大型语言模型（Large Language Models, LLMs）的扩展知识和零样本能力，以遵循指令、理解上下文信息并进行常识推理。该方法受链式思维（Chain-of-Thought, CoT）等方法的启发，更明确地应用于基于常识属性的对话数据增强的提示生成任务及生成对话的自动评估，通过为每个常识属性设计指令提示，并使用最先进的LLMs自动检测生成对话回合中所使用的原始属性，从而实现有效的常识推理与评估。
链接: https://arxiv.org/abs/2506.19483
作者: Marcos Estecha-Garitagoitia,Chen Zhang,Mario Rodríguez-Cantelar,Luis Fernando D’Haro
机构: 未知
类目: Computation and Language (cs.CL)
备注:
Abstract:This paper provides preliminary results on exploring the task of performing turn-level data augmentation for dialogue system based on different types of commonsense relationships, and the automatic evaluation of the generated synthetic turns. The proposed methodology takes advantage of the extended knowledge and zero-shot capabilities of pretrained Large Language Models (LLMs) to follow instructions, understand contextual information, and their commonsense reasoning capabilities. The approach draws inspiration from methodologies like Chain-of-Thought (CoT), applied more explicitly to the task of prompt-based generation for dialogue-based data augmentation conditioned on commonsense attributes, and the automatic evaluation of the generated dialogues. To assess the effectiveness of the proposed approach, first we extracted 200 randomly selected partial dialogues, from 5 different well-known dialogue datasets, and generate alternative responses conditioned on different event commonsense attributes. This novel dataset allows us to measure the proficiency of LLMs in generating contextually relevant commonsense knowledge, particularly up to 12 different specific ATOMIC [10] database relations. Secondly, we propose an evaluation framework to automatically detect the quality of the generated dataset inspired by the ACCENT [26] metric, which offers a nuanced approach to assess event commonsense. However, our method does not follow ACCENT’s complex eventrelation tuple extraction process. Instead, we propose an instruction-based prompt for each commonsense attribute and use state-of-the-art LLMs to automatically detect the original attributes used when creating each augmented turn in the previous step. Preliminary results suggest that our approach effectively harnesses LLMs capabilities for commonsense reasoning and evaluation in dialogue systems. Subjects: Computation and Language (cs.CL) Cite as: arXiv:2506.19483 [cs.CL] (or arXiv:2506.19483v1 [cs.CL] for this version) https://doi.org/10.48550/arXiv.2506.19483 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
zh
[NLP-33] MuBench: Assessment of Multilingual Capabilities of Large Language Models Across 61 Languages
【速读】： 该论文旨在解决多语言大语言模型（Multilingual Large Language Models, LLMs）在评估过程中存在的语言覆盖不全面和跨语言对齐不足的问题，导致模型多语言能力的评估碎片化。其解决方案的关键在于引入MuBench基准，该基准覆盖61种语言并评估广泛的能力维度，同时提出多语言一致性（Multilingual Consistency, MLC）作为补充指标，以分析性能瓶颈并指导模型优化。
链接: https://arxiv.org/abs/2506.19468
作者: Wenhan Han,Yifan Zhang,Zhixun Chen,Binbin Liu,Haobin Lin,Bingni Zhang,Taifeng Wang,Mykola Pechenizkiy,Meng Fang,Yin Zheng
机构: Eindhoven University of Technology (埃因霍温理工大学); ByteDance (字节跳动); University of Liverpool (利物浦大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:Multilingual large language models (LLMs) are advancing rapidly, with new models frequently claiming support for an increasing number of languages. However, existing evaluation datasets are limited and lack cross-lingual alignment, leaving assessments of multilingual capabilities fragmented in both language and skill coverage. To address this, we introduce MuBench, a benchmark covering 61 languages and evaluating a broad range of capabilities. We evaluate several state-of-the-art multilingual LLMs and find notable gaps between claimed and actual language coverage, particularly a persistent performance disparity between English and low-resource languages. Leveraging MuBench’s alignment, we propose Multilingual Consistency (MLC) as a complementary metric to accuracy for analyzing performance bottlenecks and guiding model improvement. Finally, we pretrain a suite of 1.2B-parameter models on English and Chinese with 500B tokens, varying language ratios and parallel data proportions to investigate cross-lingual transfer dynamics.
zh
[NLP-34] Can Large Language Models Capture Human Annotator Disagreements?
【速读】： 该论文试图解决大型语言模型（Large Language Models, LLMs）在自动标注过程中对人类标注差异（annotation disagreements）建模能力不足的问题。现有评估方法通常仅关注预测多数投票的“真实标签”（ground truth），而忽视了模型对标注差异的捕捉能力。论文的关键解决方案是通过无重复人类标签的情况下，广泛评估LLMs预测标注差异的能力，揭示其在该任务上的局限性，并强调改进LLMs在差异建模方面性能的重要性。
链接: https://arxiv.org/abs/2506.19467
作者: Jingwei Ni,Yu Fan,Vilém Zouhar,Donya Rooein,Alexander Hoyle,Mrinmaya Sachan,Markus Leippold,Dirk Hovy,Elliott Ash
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: Preprint Under Review
Abstract:Human annotation variation (i.e., annotation disagreements) is common in NLP and often reflects important information such as task subjectivity and sample ambiguity. While Large Language Models (LLMs) are increasingly used for automatic annotation to reduce human effort, their evaluation often focuses on predicting the majority-voted “ground truth” labels. It is still unclear, however, whether these models also capture informative human annotation variation. Our work addresses this gap by extensively evaluating LLMs’ ability to predict annotation disagreements without access to repeated human labels. Our results show that LLMs struggle with modeling disagreements, which can be overlooked by majority label-based evaluations. Notably, while RLVR-style (Reinforcement learning with verifiable rewards) reasoning generally boosts LLM performance, it degrades performance in disagreement prediction. Our findings highlight the critical need for evaluating and improving LLM annotators in disagreement modeling. Code and data at this https URL.
zh
[NLP-35] SDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems
【速读】： 该论文旨在解决文本到语音（Text to Speech, TTS）系统评估中存在挑战性和资源密集的问题，尤其是主观指标如平均意见分数（Mean Opinion Score, MOS）在不同研究间的可比性不足，以及客观指标很少与主观指标进行验证。此外，由于近期TTS系统能够生成与真实语音难以区分的合成语音，传统评估方法面临新的挑战。该论文提出的解决方案是引入Text to Speech Distribution Score 2 (TTSDS2)，这是TTSDS的改进版本，其关键在于在多个领域和语言中表现出更高的鲁棒性，并且是16种比较指标中唯一在所有领域和主观评分上均达到Spearman相关性高于0.50的指标。
链接: https://arxiv.org/abs/2506.19441
作者: Christoph Minixhofer,Ondrej Klejch,Peter Bell
机构: Centre for Speech Technology Research (中心语音技术研究); University of Edinburgh (爱丁堡大学)
类目: ound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
备注:
Abstract:Evaluation of Text to Speech (TTS) systems is challenging and resource-intensive. Subjective metrics such as Mean Opinion Score (MOS) are not easily comparable between works. Objective metrics are frequently used, but rarely validated against subjective ones. Both kinds of metrics are challenged by recent TTS systems capable of producing synthetic speech indistinguishable from real speech. In this work, we introduce Text to Speech Distribution Score 2 (TTSDS2), a more robust and improved version of TTSDS. Across a range of domains and languages, it is the only one out of 16 compared metrics to correlate with a Spearman correlation above 0.50 for every domain and subjective score evaluated. We also release a range of resources for evaluating synthetic speech close to real speech: A dataset with over 11,000 subjective opinion score ratings; a pipeline for continually recreating a multilingual test dataset to avoid data leakage; and a continually updated benchmark for TTS in 14 languages.
zh
[NLP-36] Mem4Nav: Boosting Vision-and-Language Navigation in Urban Environments with a Hierarchical Spatial-Cognition Long-Short Memory System
【速读】： 该论文旨在解决大规模城市环境中视觉-语言导航（Vision-and-Language Navigation, VLN）中，具身智能体在复杂场景中对语言指令进行语义定位以及在长时程中回忆相关经验的问题。现有模块化流水线虽具备可解释性但缺乏统一的记忆机制，而端到端的大规模语言模型（M）LLM代理虽然在视觉与语言融合方面表现优异，却受限于固定的上下文窗口和隐式的空间推理能力。论文提出的解决方案是\textbf{Mem4Nav}，其关键在于构建一个分层的空间认知长短时记忆系统，通过将稀疏八叉树与语义拓扑图相结合，实现细粒度的体素索引和高层地标连通性存储，并利用可逆Transformer将两者嵌入可训练的记忆标记中，从而有效提升导航任务的完成率与路径规划效率。
链接: https://arxiv.org/abs/2506.19433
作者: Lixuan He,Haoyu Dong,Zhenxing Chen,Yangcheng Yu,Jie Feng,Yong Li
机构: Tsinghua University (清华大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注:
Abstract:Vision-and-Language Navigation (VLN) in large-scale urban environments requires embodied agents to ground linguistic instructions in complex scenes and recall relevant experiences over extended time horizons. Prior modular pipelines offer interpretability but lack unified memory, while end-to-end (M)LLM agents excel at fusing vision and language yet remain constrained by fixed context windows and implicit spatial reasoning. We introduce \textbfMem4Nav, a hierarchical spatial-cognition long-short memory system that can augment any VLN backbone. Mem4Nav fuses a sparse octree for fine-grained voxel indexing with a semantic topology graph for high-level landmark connectivity, storing both in trainable memory tokens embedded via a reversible Transformer. Long-term memory (LTM) compresses and retains historical observations at both octree and graph nodes, while short-term memory (STM) caches recent multimodal entries in relative coordinates for real-time obstacle avoidance and local planning. At each step, STM retrieval sharply prunes dynamic context, and, when deeper history is needed, LTM tokens are decoded losslessly to reconstruct past embeddings. Evaluated on Touchdown and Map2Seq across three backbones (modular, state-of-the-art VLN with prompt-based LLM, and state-of-the-art VLN with strided-attention MLLM), Mem4Nav yields 7-13 pp gains in Task Completion, sufficient SPD reduction, and 10 pp nDTW improvement. Ablations confirm the indispensability of both the hierarchical map and dual memory modules. Our codes are open-sourced via this https URL.
zh
[NLP-37] Learning to Disentangle Latent Reasoning Rules with Language VAEs: A Systematic Study
【速读】： 该论文试图解决语言模型在自然语言推理任务中依赖记忆而非基于规则的推理问题，旨在提升模型的泛化性、可解释性和可控性。其解决方案的关键在于通过语言变分自编码器（Language Variational Autoencoders, VAEs）将推理规则显式嵌入并存储在语言模型的潜在空间中，构建了一个包含三个基于规则的推理任务、理论框架和端到端架构的完整流程，从而实现推理规则的解耦、先验知识的有效注入以及模型参数中推理规则分离的保持。
链接: https://arxiv.org/abs/2506.19418
作者: Yingji Zhang,Marco Valentino,Danilo S. Carvalho,André Freitas
机构: University of Manchester (曼彻斯特大学); University of Sheffield (谢菲尔德大学); Idiap Research Institute (Idiap研究所); CRUK Manchester Institute (英国癌症研究基金会曼彻斯特研究所)
类目: Computation and Language (cs.CL)
备注:
Abstract:Incorporating explicit reasoning rules within the latent space of language models (LMs) offers a promising pathway to enhance generalisation, interpretability, and controllability. While current Transformer-based language models have shown strong performance on Natural Language Inference (NLI) tasks, they often rely on memorisation rather than rule-based inference. This work investigates how reasoning rules can be explicitly embedded and memorised within the LMs through Language Variational Autoencoders (VAEs). We propose a complete pipeline for learning reasoning rules within Transformer-based language VAEs. This pipeline encompasses three rule-based reasoning tasks, a supporting theoretical framework, and a practical end-to-end architecture. The experiment illustrates the following findings: Disentangled reasoning: Under explicit signal supervision, reasoning rules - viewed as functional mappings - can be disentangled within the encoder’s parametric space. This separation results in distinct clustering of rules in the output feature space. Prior knowledge injection: injecting reasoning information into the Query enables the model to more effectively retrieve the stored value Value from memory based on Key. This approach offers a simple method for integrating prior knowledge into decoder-only language models. Performance bottleneck: In mathematical reasoning tasks using Qwen2.5(0.5B), increasing sample count doesn’t improve performance beyond a point. Moreover, ffn layers are better than attention layers at preserving the separation of reasoning rules in the model’s parameters.
zh
[NLP-38] Automated Detection of Pre-training Text in Black-box LLM s
【速读】： 该论文试图解决在黑盒设置下检测给定文本是否属于大型语言模型（Large Language Models, LLMs）预训练数据的问题，这一问题对于保障数据隐私和版权保护至关重要。现有方法通常依赖于模型的隐藏信息（如模型参数或标记概率），在仅能访问输入和输出文本的黑盒环境中效果不佳。该论文提出的解决方案——VeilProbe，是首个无需人工干预即可在黑盒设置中自动检测LLMs预训练文本的框架。其关键在于利用序列到序列映射模型推断输入文本与LLM生成的输出后缀之间的潜在映射特征，并通过关键标记扰动获取更具区分性的成员特征，同时引入基于原型的成员分类器以缓解真实场景中真实训练文本样本有限导致的过拟合问题。
链接: https://arxiv.org/abs/2506.19399
作者: Ruihan Hu,Yu-Ming Shang,Jiankun Peng,Wei Luo,Yazhe Wang,Xi Zhang
机构: Beijing University of Posts and Telecommunications (北京邮电大学); Zhongguancun Laboratory (中关村实验室)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: 13 pages
Abstract:Detecting whether a given text is a member of the pre-training data of Large Language Models (LLMs) is crucial for ensuring data privacy and copyright protection. Most existing methods rely on the LLM’s hidden information (e.g., model parameters or token probabilities), making them ineffective in the black-box setting, where only input and output texts are accessible. Although some methods have been proposed for the black-box setting, they rely on massive manual efforts such as designing complicated questions or instructions. To address these issues, we propose VeilProbe, the first framework for automatically detecting LLMs’ pre-training texts in a black-box setting without human intervention. VeilProbe utilizes a sequence-to-sequence mapping model to infer the latent mapping feature between the input text and the corresponding output suffix generated by the LLM. Then it performs the key token perturbations to obtain more distinguishable membership features. Additionally, considering real-world scenarios where the ground-truth training text samples are limited, a prototype-based membership classifier is introduced to alleviate the overfitting issue. Extensive evaluations on three widely used datasets demonstrate that our framework is effective and superior in the black-box setting.
zh
[NLP-39] Measuring and Guiding Monosemanticity
【速读】： 该论文试图解决大型语言模型（Large Language Models, LLMs）中特征表示的可解释性和可控性问题，特别是现有方法在可靠定位和操作特征表示方面存在的根本性挑战。其解决方案的关键在于引入一种新的度量标准——特征单义性得分（Feature Monosemanticity Score, FMS），用于系统量化潜在表示中的特征单义性，并提出引导稀疏自编码器（Guided Sparse Autoencoders, G-SAE），通过在训练过程中对标注概念进行条件约束，实现目标概念在潜在空间中的可靠定位与解耦，从而提升模型的可解释性、行为检测能力和控制效果。
链接: https://arxiv.org/abs/2506.19382
作者: Ruben Härle,Felix Friedrich,Manuel Brack,Stephan Wäldchen,Björn Deiseroth,Patrick Schramowski,Kristian Kersting
机构: TU Darmstadt(达姆施塔特工业大学); Lab1141; Aleph Alpha Research(阿尔法阿尔法研究); Hessian.AI(黑森人工智能); DFKI(德国人工智能研究中心); CERTAIN; Centre of Cognitive Science, TU Darmstadt(认知科学中心，达姆施塔特工业大学)
类目: Computation and Language (cs.CL)
备注:
Abstract:There is growing interest in leveraging mechanistic interpretability and controllability to better understand and influence the internal dynamics of large language models (LLMs). However, current methods face fundamental challenges in reliably localizing and manipulating feature representations. Sparse Autoencoders (SAEs) have recently emerged as a promising direction for feature extraction at scale, yet they, too, are limited by incomplete feature isolation and unreliable monosemanticity. To systematically quantify these limitations, we introduce Feature Monosemanticity Score (FMS), a novel metric to quantify feature monosemanticity in latent representation. Building on these insights, we propose Guided Sparse Autoencoders (G-SAE), a method that conditions latent representations on labeled concepts during training. We demonstrate that reliable localization and disentanglement of target concepts within the latent space improve interpretability, detection of behavior, and control. Specifically, our evaluations on toxicity detection, writing style identification, and privacy attribute recognition show that G-SAE not only enhances monosemanticity but also enables more effective and fine-grained steering with less quality degradation. Our findings provide actionable guidelines for measuring and advancing mechanistic interpretability and control of LLMs.
zh
[NLP-40] Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation ACL2025
【速读】： 该论文试图解决大型语言模型（Large Language Models, LLMs）在生成对话时出现的偏离设定角色（Out-of-Character, OOC）行为问题，这种行为导致生成内容与设定的个性不一致，影响模型的可靠性和交互体验。解决方案的关键在于提出一种原子级别的评估框架，通过三个核心指标在更细粒度上量化个性一致性与对齐程度，从而更精确地检测出传统方法难以捕捉的微妙偏差。
链接: https://arxiv.org/abs/2506.19352
作者: Jisu Shin,Juhyun Oh,Eunsu Kim,Hoyun Song,Alice Oh
机构: Korea Advanced Institute of Science and Technology (KAIST)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
备注: Findings of ACL 2025; github repo: this https URL
Abstract:Ensuring persona fidelity in large language models (LLMs) is essential for maintaining coherent and engaging human-AI interactions. However, LLMs often exhibit Out-of-Character (OOC) behavior, where generated responses deviate from an assigned persona, leading to inconsistencies that affect model reliability. Existing evaluation methods typically assign single scores to entire responses, struggling to capture subtle persona misalignment, particularly in long-form text generation. To address this limitation, we propose an atomic-level evaluation framework that quantifies persona fidelity at a finer granularity. Our three key metrics measure the degree of persona alignment and consistency within and across generations. Our approach enables a more precise and realistic assessment of persona fidelity by identifying subtle deviations that real users would encounter. Through our experiments, we demonstrate that our framework effectively detects persona inconsistencies that prior methods overlook. By analyzing persona fidelity across diverse tasks and personality types, we reveal how task structure and persona desirability influence model adaptability, highlighting challenges in maintaining consistent persona expression.
zh
[NLP-41] In-Context Occams Razor: How Transformers Prefer Simpler Hypotheses on the Fly
【速读】： 该论文试图解决Transformer模型在面对不同复杂度任务时如何通过上下文学习（In-context Learning, ICL）选择合适复杂度的模型结构并准确推断参数的问题。其解决方案的关键在于揭示Transformer具备一种类似贝叶斯奥卡姆剃刀（Bayesian Occam’s Razor）的归纳偏置，即在多个可能的复杂度假设中，优先选择最简单但足以解释数据的模型，从而在模型拟合与复杂度惩罚之间实现平衡。
链接: https://arxiv.org/abs/2506.19351
作者: Puneesh Deora,Bhavya Vasudeva,Tina Behnia,Christos Thrampoulidis
机构: University of British Columbia (不列颠哥伦比亚大学); University of Southern California (南加利福尼亚大学)
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (stat.ML)
备注: 28 pages, 19 figures
Abstract:In-context learning (ICL) enables transformers to adapt to new tasks through contextual examples without parameter updates. While existing research has typically studied ICL in fixed-complexity environments, practical language models encounter tasks spanning diverse complexity levels. This paper investigates how transformers navigate hierarchical task structures where higher-complexity categories can perfectly represent any pattern generated by simpler ones. We design well-controlled testbeds based on Markov chains and linear regression that reveal transformers not only identify the appropriate complexity level for each task but also accurately infer the corresponding parameters–even when the in-context examples are compatible with multiple complexity hypotheses. Notably, when presented with data generated by simpler processes, transformers consistently favor the least complex sufficient explanation. We theoretically explain this behavior through a Bayesian framework, demonstrating that transformers effectively implement an in-context Bayesian Occam’s razor by balancing model fit against complexity penalties. We further ablate on the roles of model size, training mixture distribution, inference context length, and architecture. Finally, we validate this Occam’s razor-like inductive bias on a pretrained GPT-4 model with Boolean-function tasks as case study, suggesting it may be inherent to transformers trained on diverse task distributions.
zh
[NLP-42] JCAPT: A Joint Modeling Approach for CAPT ISCA
【速读】： 该论文旨在解决第二语言（L2）学习中有效发音反馈的问题，具体涉及计算机辅助发音训练（CAPT）系统中的自动发音评估（APA）和错误发音检测与诊断（MDD）任务。其解决方案的关键在于提出一个统一框架，利用选择性状态空间模型（SSM）——Mamba，并结合语音学特征和思考标记策略，以联合提升APA和MDD的可解释性与细粒度时间推理能力。该方法首次将语音学归因、基于SSM的建模以及提示技术整合到CAPT中，实验结果表明该模型在MDD任务上表现优于现有方法。
链接: https://arxiv.org/abs/2506.19315
作者: Tzu-Hsuan Yang,Yue-Yang He,Berlin Chen
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
备注: Submitted to the ISCA SLaTE-2025 Workshop
Abstract:Effective pronunciation feedback is critical in second language (L2) learning, for which computer-assisted pronunciation training (CAPT) systems often encompass two key tasks: automatic pronunciation assessment (APA) and mispronunciation detection and diagnosis (MDD). Recent work has shown that joint modeling of these two tasks can yield mutual benefits. Our unified framework leverages Mamba, a selective state space model (SSM), while integrating phonological features and think token strategies to jointly enhance interpretability and fine-grained temporal reasoning in APA and MDD. To our knowledge, this is the first study to combine phonological attribution, SSM-based modeling, and prompting in CAPT. A series of experiments conducted on the speechocean762 benchmark demonstrate that our model consistently outperforms prior methods, particularly on the MDD task.
zh
[NLP-43] Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLM s
【速读】： 该论文旨在解决软件工程（Software Engineering, SWE）领域中数据集规模小且数据收集过程耗时的问题，尤其是由于依赖人工标注和专用运行环境设置导致的数据量有限。其解决方案的关键在于提出了一种增量式、自动化的数据收集流水线，该流水线系统地提升了SWE数据集的规模与多样性，从而支持更大规模的模型训练与优化。
链接: https://arxiv.org/abs/2506.19290
作者: Liang Zeng,Yongcong Li,Yuzhen Xiao,Changshi Li,Chris Yuhao Liu,Rui Yan,Tianwen Wei,Jujie He,Xuchen Song,Yang Liu,Yahui Zhou
机构: 未知
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注:
Abstract:Software engineering (SWE) has recently emerged as a crucial testbed for next-generation LLM agents, demanding inherent capabilities in two critical dimensions: sustained iterative problem-solving (e.g., 50 interaction rounds) and long-context dependency resolution (e.g., 32k tokens). However, the data curation process in SWE remains notoriously time-consuming, as it heavily relies on manual annotation for code file filtering and the setup of dedicated runtime environments to execute and validate unit tests. Consequently, most existing datasets are limited to only a few thousand GitHub-sourced instances. To this end, we propose an incremental, automated data-curation pipeline that systematically scales both the volume and diversity of SWE datasets. Our dataset comprises 10,169 real-world Python task instances from 2,531 distinct GitHub repositories, each accompanied by a task specified in natural language and a dedicated runtime-environment image for automated unit-test validation. We have carefully curated over 8,000 successfully runtime-validated training trajectories from our proposed SWE dataset. When fine-tuning the Skywork-SWE model on these trajectories, we uncover a striking data scaling phenomenon: the trained model’s performance for software engineering capabilities in LLMs continues to improve as the data size increases, showing no signs of saturation. Notably, our Skywork-SWE model achieves 38.0% pass@1 accuracy on the SWE-bench Verified benchmark without using verifiers or multiple rollouts, establishing a new state-of-the-art (SOTA) among the Qwen2.5-Coder-32B-based LLMs built on the OpenHands agent framework. Furthermore, with the incorporation of test-time scaling techniques, the performance further improves to 47.0% accuracy, surpassing the previous SOTA results for sub-32B parameter models. We release the Skywork-SWE-32B model checkpoint to accelerate future research.
zh
[NLP-44] EmoStage: A Framework for Accurate Empathetic Response Generation via Perspective-Taking and Phase Recognition
【速读】： 该论文旨在解决当前AI驱动的心理咨询系统在理解客户心理状态和咨询阶段、依赖高质量训练数据以及商业部署中的隐私问题等方面的不足。其解决方案的关键在于提出EmoStage框架，该框架通过利用开源大语言模型（Large Language Models, LLMs）的推理能力，在无需额外训练数据的情况下提升共情响应生成的质量。该框架引入了视角转换机制以推断客户的心理状态和支持需求，并结合阶段识别技术确保响应与咨询过程的一致性，从而生成更具情感共鸣且符合情境的回复。
链接: https://arxiv.org/abs/2506.19279
作者: Zhiyang Qi,Keiko Takamizo,Mariko Ukiyo,Michimasa Inaba
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注:
Abstract:The rising demand for mental health care has fueled interest in AI-driven counseling systems. While large language models (LLMs) offer significant potential, current approaches face challenges, including limited understanding of clients’ psychological states and counseling stages, reliance on high-quality training data, and privacy concerns associated with commercial deployment. To address these issues, we propose EmoStage, a framework that enhances empathetic response generation by leveraging the inference capabilities of open-source LLMs without additional training data. Our framework introduces perspective-taking to infer clients’ psychological states and support needs, enabling the generation of emotionally resonant responses. In addition, phase recognition is incorporated to ensure alignment with the counseling process and to prevent contextually inappropriate or inopportune responses. Experiments conducted in both Japanese and Chinese counseling settings demonstrate that EmoStage improves the quality of responses generated by base models and performs competitively with data-driven methods.
zh
[NLP-45] What Matters in LLM -generated Data: Diversity and Its Effect on Model Fine-Tuning
【速读】： 该论文试图解决在特定领域中数据稀缺问题，以及通过使用大语言模型（Large Language Models, LLMs）生成的数据进行下游模型训练时，因数据多样性不足导致的模型性能下降问题。其解决方案的关键在于分析LLM生成数据的多样性对下游模型性能的影响，并探索混合不同比例LLM生成数据（即合成数据）对模型效果的作用，以找到在标签数据不足场景下提升模型性能的最优数据多样性水平。
链接: https://arxiv.org/abs/2506.19262
作者: Yuchang Zhu,Zhonghua zhen,Qunshu Lin,Haotong Wei,Xiaolong Sun,Zixuan Yu,Minghao Liu,Zibin Zheng,Liang Chen
机构: Sun Yat-sen University (中山大学); Abaka AI; 2077AI
类目: Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: Ongoing work
Abstract:With the remarkable generative capabilities of large language models (LLMs), using LLM-generated data to train downstream models has emerged as a promising approach to mitigate data scarcity in specific domains and reduce time-consuming annotations. However, recent studies have highlighted a critical issue: iterative training on self-generated data results in model collapse, where model performance degrades over time. Despite extensive research on the implications of LLM-generated data, these works often neglect the importance of data diversity, a key factor in data quality. In this work, we aim to understand the implications of the diversity of LLM-generated data on downstream model performance. Specifically, we explore how varying levels of diversity in LLM-generated data affect downstream model performance. Additionally, we investigate the performance of models trained on data that mixes different proportions of LLM-generated data, which we refer to as synthetic data. Our experimental results show that, with minimal distribution shift, moderately diverse LLM-generated data can enhance model performance in scenarios with insufficient labeled data, whereas highly diverse generated data has a negative impact. We hope our empirical findings will offer valuable guidance for future studies on LLMs as data generators.
zh
[NLP-46] Personality Prediction from Life Stories using Language Models
【速读】： 该论文试图解决如何利用长篇叙述性访谈文本（超过2000个token）来预测大五人格模型（Five-Factor Model, FFM）人格特质的问题。解决方案的关键在于提出一种两步方法：首先通过滑动窗口微调预训练语言模型提取上下文嵌入，然后使用带有注意力机制的循环神经网络（Recurrent Neural Networks, RNNs）整合长程依赖关系并增强可解释性，从而有效结合预训练Transformer模型与序列建模的优势以处理长上下文数据。
链接: https://arxiv.org/abs/2506.19258
作者: Rasiq Hussain,Jerry Ma,Rithik Khandelwal,Joshua Oltmanns,Mehak Gupta
机构: Southern Methodist University (南方卫理公会大学); Washington University in St. Louis (圣路易斯华盛顿大学)
类目: Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: 13 pages, 5 figures
Abstract:Natural Language Processing (NLP) offers new avenues for personality assessment by leveraging rich, open-ended text, moving beyond traditional questionnaires. In this study, we address the challenge of modeling long narrative interview where each exceeds 2000 tokens so as to predict Five-Factor Model (FFM) personality traits. We propose a two-step approach: first, we extract contextual embeddings using sliding-window fine-tuning of pretrained language models; then, we apply Recurrent Neural Networks (RNNs) with attention mechanisms to integrate long-range dependencies and enhance interpretability. This hybrid method effectively bridges the strengths of pretrained transformers and sequence modeling to handle long-context data. Through ablation studies and comparisons with state-of-the-art long-context models such as LLaMA and Longformer, we demonstrate improvements in prediction accuracy, efficiency, and interpretability. Our results highlight the potential of combining language-based features with long-context modeling to advance personality assessment from life narratives.
zh
[NLP-47] Augmenting Multi-Agent Communication with State Delta Trajectory
【速读】： 该论文试图解决多智能体系统在通信过程中因使用自然语言导致的信息丢失问题，特别是在传递复杂推理逻辑或抽象思想时，信息损失尤为显著。其解决方案的关键在于提出一种新的通信协议，该协议不仅传递自然语言标记，还传递逐标记的状态转移轨迹。通过引入状态差分编码（State Delta Encoding, SDE）方法，该协议能够更有效地捕捉和传递模型推理过程中的隐含信息，从而提升多智能体系统的性能。
链接: https://arxiv.org/abs/2506.19209
作者: Yichen Tang,Weihang Su,Yujia Zhou,Yiqun Liu,Min Zhang,Shaoping Ma,Qingyao Ai
机构: Tsinghua University (清华大学)
类目: Computation and Language (cs.CL)
备注: 22 pages, 5 figures
Abstract:Multi-agent techniques such as role playing or multi-turn debates have been shown to be effective in improving the performance of large language models (LLMs) in downstream tasks. Despite their differences in workflows, existing LLM-based multi-agent systems mostly use natural language for agent communication. While this is appealing for its simplicity and interpretability, it also introduces inevitable information loss as one model must down sample its continuous state vectors to concrete tokens before transferring them to the other model. Such losses are particularly significant when the information to transfer is not simple facts, but reasoning logics or abstractive thoughts. To tackle this problem, we propose a new communication protocol that transfers both natural language tokens and token-wise state transition trajectory from one agent to another. Particularly, compared to the actual state value, we find that the sequence of state changes in LLMs after generating each token can better reflect the information hidden behind the inference process, so we propose a State Delta Encoding (SDE) method to represent state transition trajectories. The experimental results show that multi-agent systems with SDE achieve SOTA performance compared to other communication protocols, particularly in tasks that involve complex reasoning. This shows the potential of communication augmentation for LLM-based multi-agent systems.
zh
[NLP-48] Bayesian Evolutionary Swarm Architecture: A Formal Epistemic System Grounded in Truth-Based Competition
【速读】： 该论文试图解决如何构建一个基于概率代理的可信人工智能系统，以实现对真理的可靠逼近与演化。其核心问题是通过结构化竞争和信念修正机制，使代理在动态环境中逐步提高对客观真理的对齐度。解决方案的关键在于将代理的适应性定义为与固定外部权威（即真实值）的对齐程度，并利用贝叶斯推理、测度论和种群动力学构建框架，通过竞争性评估与信念更新确保系统的可计算性、自调节性和演化稳定性。
链接: https://arxiv.org/abs/2506.19191
作者: Craig Steven Wright
机构: University of Exeter (埃克塞特大学)
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Science and Game Theory (cs.GT); Logic (math.LO)
备注: 83 pages, 14 sections, 92 formal results, no prior conference publication
Abstract:We introduce a mathematically rigorous framework for an artificial intelligence system composed of probabilistic agents evolving through structured competition and belief revision. The architecture, grounded in Bayesian inference, measure theory, and population dynamics, defines agent fitness as a function of alignment with a fixed external oracle representing ground truth. Agents compete in a discrete-time environment, adjusting posterior beliefs through observed outcomes, with higher-rated agents reproducing and lower-rated agents undergoing extinction. Ratings are updated via pairwise truth-aligned utility comparisons, and belief updates preserve measurable consistency and stochastic convergence. We introduce hash-based cryptographic identity commitments to ensure traceability, alongside causal inference operators using do-calculus. Formal theorems on convergence, robustness, and evolutionary stability are provided. The system establishes truth as an evolutionary attractor, demonstrating that verifiable knowledge arises from adversarial epistemic pressure within a computable, self-regulating swarm.
zh
[NLP-49] Prompt Translate Fine-Tune Re-Initialize or Instruction-Tune? Adapting LLM s for In-Context Learning in Low-Resource Languages ACL
【速读】： 该论文试图解决低资源语言在上下文学习（in-context learning）中表现不佳的问题，特别是在跨语言适应性方面缺乏明确的优化策略。其解决方案的关键在于通过大规模实验对比多种适应技术，包括少样本提示（few-shot prompting）、翻译测试（translate-test）、微调（fine-tuning）、嵌入重新初始化（embedding re-initialization）和指令微调（instruction fine-tuning），并发现少样本提示和翻译测试设置在低资源语言任务中显著优于基于梯度的适应方法。此外，研究引入了新的评估指标——有效输出召回率（Valid Output Recall, VOR），以分析模型性能下降的原因，揭示了灾难性遗忘（catastrophic forgetting）是导致训练模型性能退化的重要因素。
链接: https://arxiv.org/abs/2506.19187
作者: Christopher Toukmaji,Jeffrey Flanigan
机构: University of California, Irvine (加州大学欧文分校); University of California, Santa Cruz (加州大学圣克鲁兹分校)
类目: Computation and Language (cs.CL)
备注: Accepted to ACL GEM 2025
Abstract:LLMs are typically trained in high-resource languages, and tasks in lower-resourced languages tend to underperform the higher-resource language counterparts for in-context learning. Despite the large body of work on prompting settings, it is still unclear how LLMs should be adapted cross-lingually specifically for in-context learning in the low-resource target languages. We perform a comprehensive study spanning five diverse target languages, three base LLMs, and seven downstream tasks spanning over 4,100 GPU training hours (9,900+ TFLOPs) across various adaptation techniques: few-shot prompting, translate-test, fine-tuning, embedding re-initialization, and instruction fine-tuning. Our results show that the few-shot prompting and translate-test settings tend to heavily outperform the gradient-based adaptation methods. To better understand this discrepancy, we design a novel metric, Valid Output Recall (VOR), and analyze model outputs to empirically attribute the degradation of these trained models to catastrophic forgetting. To the extent of our knowledge, this is the largest study done on in-context learning for low-resource languages with respect to train compute and number of adaptation techniques considered. We make all our datasets and trained models available for public use.
zh
[NLP-50] Enhanced Hybrid Transducer and Attention Encoder Decoder with Text Data INTERSPEECH2025
【速读】： 该论文试图解决语音识别（Automatic Speech Recognition, ASR）中数据稀缺导致的跨领域任务性能下降问题。其解决方案的关键在于提出一种联合语音与文本优化方法（Joint TAED, J-TAED），通过同时利用语音和文本输入模态进行训练，使模型能够统一不同模态的内部表示，并进一步扩展至基于文本的领域自适应。该方法在推理阶段仅需语音数据，但在训练阶段利用大量文本语料库来提升ASR的准确性，从而有效缓解了跨领域任务中语音数据不足的问题。实验结果表明，J-TAED在Librispeech数据集上将词错误率（WER）降低了5.8%~12.8%，并在金融和命名实体相关的两个跨领域数据集上分别实现了15.3%和17.8%的WER降低。
链接: https://arxiv.org/abs/2506.19159
作者: Yun Tang,Eesung Kim,Vijendra Raj Apsingekar
机构: 未知
类目: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
备注: Accepted by Interspeech2025
Abstract:A joint speech and text optimization method is proposed for hybrid transducer and attention-based encoder decoder (TAED) modeling to leverage large amounts of text corpus and enhance ASR accuracy. The joint TAED (J-TAED) is trained with both speech and text input modalities together, while it only takes speech data as input during inference. The trained model can unify the internal representations from different modalities, and be further extended to text-based domain adaptation. It can effectively alleviate data scarcity for mismatch domain tasks since no speech data is required. Our experiments show J-TAED successfully integrates speech and linguistic information into one model, and reduce the WER by 5.8 ~12.8% on the Librispeech dataset. The model is also evaluated on two out-of-domain datasets: one is finance and another is named entity focused. The text-based domain adaptation brings 15.3% and 17.8% WER reduction on those two datasets respectively.
zh
[NLP-51] hought Anchors: Which LLM Reasoning Steps Matter?
【速读】： 该论文试图解决大型语言模型在长文本推理过程中产生的可解释性问题，因为每个生成的标记都依赖于之前的所有标记，导致计算难以分解。解决方案的关键在于从句子层面分析推理轨迹，提出了三种互补的归因方法：基于反事实重要性的黑盒方法、基于注意力模式聚合的白盒方法以及基于逻辑连接的因果归因方法。这些方法共同揭示了“思维锚点”（thought anchors）的存在，即对后续推理过程具有显著影响的规划或回溯性句子，从而为理解推理模型提供了更深入的视角。
链接: https://arxiv.org/abs/2506.19143
作者: Paul C. Bogdan,Uzay Macar,Neel Nanda,Arthur Conmy
机构: Duke University (杜克大学); Aiphabet; Arthur Conmy
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: Paul C. Bogdan and Uzay Macar contributed equally to this work, and their listed order was determined by coinflip. Neel Nanda and Arthur Conmy contributed equally to this work as senior authors, and their listed order was determined by coinflip
Abstract:Reasoning large language models have recently achieved state-of-the-art performance in many fields. However, their long-form chain-of-thought reasoning creates interpretability challenges as each generated token depends on all previous ones, making the computation harder to decompose. We argue that analyzing reasoning traces at the sentence level is a promising approach to understanding reasoning processes. We present three complementary attribution methods: (1) a black-box method measuring each sentence’s counterfactual importance by comparing final answers across 100 rollouts conditioned on the model generating that sentence or one with a different meaning; (2) a white-box method of aggregating attention patterns between pairs of sentences, which identified broadcasting'' sentences that receive disproportionate attention from all future sentences via receiver’’ attention heads; (3) a causal attribution method measuring logical connections between sentences by suppressing attention toward one sentence and measuring the effect on each future sentence’s tokens. Each method provides evidence for the existence of thought anchors, reasoning steps that have outsized importance and that disproportionately influence the subsequent reasoning process. These thought anchors are typically planning or backtracking sentences. We provide an open-source tool (this http URL) for visualizing the outputs of our methods, and present a case study showing converging patterns across methods that map how a model performs multi-step reasoning. The consistency across methods demonstrates the potential of sentence-level analysis for a deeper understanding of reasoning models.
zh
[NLP-52] Human-Aligned Faithfulness in Toxicity Explanations of LLM s
【速读】： 该论文试图解决生成式 AI (Generative AI) 在毒性检测任务中推理能力不足的问题，特别是其在解释毒性立场时的合理性和一致性问题。现有方法在评估自由形式的毒性解释时存在局限性，主要由于对输入文本扰动的过度依赖。为了解决这一问题，该研究提出了一种理论基础坚实的多维评估标准——人类对齐忠实度（Human-Aligned Faithfulness, HAF），用于衡量生成式 AI 的毒性解释与理想条件下理性人类解释的一致性。该解决方案的关键在于开发了六个基于不确定性量化的方法，无需人工参与即可全面评估生成式 AI 的毒性解释的忠实度，并揭示其解释中的“非理想”特性。
链接: https://arxiv.org/abs/2506.19113
作者: Ramaravind K. Mothilal,Joanna Roy,Syed Ishtiaque Ahmed,Shion Guha
机构: University of Toronto (多伦多大学); trail-ml
类目: Computation and Language (cs.CL)
备注: 21 pages, 5 figures, 7 tables
Abstract:The discourse around toxicity and LLMs in NLP largely revolves around detection tasks. This work shifts the focus to evaluating LLMs’ reasoning about toxicity – from their explanations that justify a stance – to enhance their trustworthiness in downstream tasks. Despite extensive research on explainability, it is not straightforward to adopt existing methods to evaluate free-form toxicity explanation due to their over-reliance on input text perturbations, among other challenges. To account for these, we propose a novel, theoretically-grounded multi-dimensional criterion, Human-Aligned Faithfulness (HAF), that measures the extent to which LLMs’ free-form toxicity explanations align with those of a rational human under ideal conditions. We develop six metrics, based on uncertainty quantification, to comprehensively evaluate \haf of LLMs’ toxicity explanations with no human involvement, and highlight how “non-ideal” the explanations are. We conduct several experiments on three Llama models (of size up to 70B) and an 8B Ministral model on five diverse toxicity datasets. Our results show that while LLMs generate plausible explanations to simple prompts, their reasoning about toxicity breaks down when prompted about the nuanced relations between the complete set of reasons, the individual reasons, and their toxicity stances, resulting in inconsistent and nonsensical responses. We open-source our code and LLM-generated explanations at this https URL.
zh
[NLP-53] Language Models Might Not Understand You: Evaluating Theory of Mind via Story Prompting
【速读】： 该论文旨在解决评估大型语言模型（Large Language Models, LLMs）理论心智（Theory of Mind, ToM）和世界建模（World Modeling, WM）能力的问题。传统基准测试可能受到预训练数据污染的影响，而本文提出的 \textttStorySim 框架通过生成新颖且可组合的故事提示，利用高度可控的 \textttStoryboard 实现对角色视角和事件的精确操控，从而提供更可靠的评估手段。该框架的关键在于其可编程性和对故事结构的精细控制，使得能够设计第一阶和第二阶的 ToM 任务以及控制心智状态追踪能力的 WM 任务。
链接: https://arxiv.org/abs/2506.19089
作者: Nathaniel Getachew,Abulhair Saparov
机构: Purdue University (普渡大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
备注: 14 pages, 11 figures
Abstract:We introduce \textttStorySim , a programmable framework for synthetically generating stories to evaluate the theory of mind (ToM) and world modeling (WM) capabilities of large language models (LLMs). Unlike prior benchmarks that may suffer from contamination in pretraining data, \textttStorySim produces novel, compositional story prompts anchored by a highly controllable \textttStoryboard , enabling precise manipulation of character perspectives and events. We use this framework to design first- and second-order ToM tasks alongside WM tasks that control for the ability to track and model mental states. Our experiments across a suite of state-of-the-art LLMs reveal that most models perform better on WM tasks than ToM tasks, and that models tend to perform better reasoning with humans compared to inanimate objects. Additionally, our framework enabled us to find evidence of heuristic behavior such as recency bias and an over-reliance on earlier events in the story. All code for generating data and evaluations is freely available.
zh
[NLP-54] MFTCXplain: A Multilingual Benchmark Dataset for Evaluating the Moral Reasoning of LLM s through Hate Speech Multi-hop Explanation
【速读】： 该论文试图解决大型语言模型（Large Language Models, LLMs）在道德推理能力上的不足，特别是在跨文化场景下评估其道德判断的局限性。现有评估基准存在两个主要问题：缺乏解释性标注以支持道德分类的透明性和可解释性，以及过度集中于英语，限制了在多元文化环境中的道德推理评估。论文提出的解决方案是引入MFTCXplain，这是一个基于道德基础理论（Moral Foundation Theory, MFT）的多语言基准数据集，通过仇恨言论的多跳解释来评估LLMs的道德推理能力。该数据集包含四种语言的3000条推文，并附有二元仇恨言论标签、道德类别和文本跨度级理由，旨在提升对LLMs道德推理能力的全面评估。
链接: https://arxiv.org/abs/2506.19073
作者: Jackson Trager,Francielle Vargas,Diego Alves,Matteo Guida,Mikel K. Ngueajio,Ameeta Agrawal,Flor Plaza-del-Arco,Yalda Daryanai,Farzan Karimi-Malekabadi
机构: University of Southern California (南加州大学); University of São Paulo (圣保罗大学); Saarland University (萨尔兰大学); University of Melbourne (墨尔本大学); Howard University (霍华德大学); Portland State University (波特兰州立大学); Leiden University (莱顿大学)
类目: Computation and Language (cs.CL)
备注: Under Review
Abstract:Ensuring the moral reasoning capabilities of Large Language Models (LLMs) is a growing concern as these systems are used in socially sensitive tasks. Nevertheless, current evaluation benchmarks present two major shortcomings: a lack of annotations that justify moral classifications, which limits transparency and interpretability; and a predominant focus on English, which constrains the assessment of moral reasoning across diverse cultural settings. In this paper, we introduce MFTCXplain, a multilingual benchmark dataset for evaluating the moral reasoning of LLMs via hate speech multi-hop explanation using Moral Foundation Theory (MFT). The dataset comprises 3,000 tweets across Portuguese, Italian, Persian, and English, annotated with binary hate speech labels, moral categories, and text span-level rationales. Empirical results highlight a misalignment between LLM outputs and human annotations in moral reasoning tasks. While LLMs perform well in hate speech detection (F1 up to 0.836), their ability to predict moral sentiments is notably weak (F1 0.35). Furthermore, rationale alignment remains limited mainly in underrepresented languages. These findings show the limited capacity of current LLMs to internalize and reflect human moral reasoning.
zh
[NLP-55] HAWAII: Hierarchical Visual Knowledge Transfer for Efficient Vision-Language Models
【速读】： 该论文旨在解决视觉-语言模型（Vision-Language Models, VLMs）在提升视觉理解能力时面临的计算成本过高的问题。现有方法通过引入多个预训练的视觉专家虽取得一定成效，但导致训练和推理阶段的计算开销显著增加。解决方案的关键在于提出HAWAII框架，该框架通过知识蒸馏将多个视觉专家的知识融合到一个视觉编码器中，从而在保持性能的同时降低计算负担。其核心创新包括使用教师特定的低秩适配器（LoRA adapters）结合路由机制，以缓解不同教师间的冲突并切换教师特定知识，以及引入细粒度和粗粒度的知识蒸馏策略，以提高知识迁移的效率与效果。
链接: https://arxiv.org/abs/2506.19072
作者: Yimu Wang,Mozhgan Nasr Azadani,Sean Sedwards,Krzysztof Czarnecki
机构: University of Waterloo (滑铁卢大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注: Work in progress
Abstract:Improving the visual understanding ability of vision-language models (VLMs) is crucial for enhancing their performance across various tasks. While using multiple pretrained visual experts has shown great promise, it often incurs significant computational costs during training and inference. To address this challenge, we propose HAWAII, a novel framework that distills knowledge from multiple visual experts into a single vision encoder, enabling it to inherit the complementary strengths of several experts with minimal computational overhead. To mitigate conflicts among different teachers and switch between different teacher-specific knowledge, instead of using a fixed set of adapters for multiple teachers, we propose to use teacher-specific Low-Rank Adaptation (LoRA) adapters with a corresponding router. Each adapter is aligned with a specific teacher, avoiding noisy guidance during distillation. To enable efficient knowledge distillation, we propose fine-grained and coarse-grained distillation. At the fine-grained level, token importance scores are employed to emphasize the most informative tokens from each teacher adaptively. At the coarse-grained level, we summarize the knowledge from multiple teachers and transfer it to the student using a set of general-knowledge LoRA adapters with a router. Extensive experiments on various vision-language tasks demonstrate the superiority of HAWAII, compared to the popular open-source VLMs.
zh
[NLP-56] NLPnorth @ TalentCLEF 2025: Comparing Discriminative Contrastive and Prompt-Based Methods for Job Title and Skill Matching
【速读】： 该论文旨在解决职业标题匹配（Multilingual Job Title Matching）和基于职业标题的技能预测（Job Title-Based Skill Prediction）问题，这两个任务在计算招聘市场领域具有重要应用价值，例如自动候选人匹配、职业路径预测和招聘市场分析。解决方案的关键在于对比分类方法（classification-based）、对比方法（contrastive-based）和提示方法（prompting）的效果，并利用额外数据（如ESCO中的语言特定职业标题及其描述）进行模型微调。研究发现，对于任务A，提示方法在英语、西班牙语和德语测试数据上的平均平均精度（MAP）达到0.492；对于任务B，微调的分类方法在测试数据上的MAP为0.290，同时最大的多语言语言模型在两项任务中表现最佳。
链接: https://arxiv.org/abs/2506.19058
作者: Mike Zhang,Rob van der Goot
机构: Aalborg University (奥尔堡大学); IT University of Copenhagen (哥本哈根信息技术大学)
类目: Computation and Language (cs.CL)
备注: TalentCLEF 2025
Abstract:Matching job titles is a highly relevant task in the computational job market domain, as it improves e.g., automatic candidate matching, career path prediction, and job market analysis. Furthermore, aligning job titles to job skills can be considered an extension to this task, with similar relevance for the same downstream tasks. In this report, we outline NLPnorth’s submission to TalentCLEF 2025, which includes both of these tasks: Multilingual Job Title Matching, and Job Title-Based Skill Prediction. For both tasks we compare (fine-tuned) classification-based, (fine-tuned) contrastive-based, and prompting methods. We observe that for Task A, our prompting approach performs best with an average of 0.492 mean average precision (MAP) on test data, averaged over English, Spanish, and German. For Task B, we obtain an MAP of 0.290 on test data with our fine-tuned classification-based approach. Additionally, we made use of extra data by pulling all the language-specific titles and corresponding \emphdescriptions from ESCO for each job and skill. Overall, we find that the largest multilingual language models perform best for both tasks. Per the provisional results and only counting the unique teams, the ranking on Task A is 5 ^\textth /20 and for Task B 3 ^\textrd /14.
zh
[NLP-57] Plan for Speed – Dilated Scheduling for Masked Diffusion Language Models
【速读】： 该论文试图解决非自回归文本生成中现有采样器在并行解掩码（parallel unmasking）时效率低下且无法有效处理词元间依赖关系的问题，这限制了其推理速度无法超越传统自回归（autoregressive, AR）模型。解决方案的关键在于提出一种无需额外训练的推理阶段方法——扩张调度解掩码策略（Dilated-scheduled Unmasking Strategy, DUS），该方法基于一阶马尔可夫假设将序列位置划分为非相邻的分组，从而实现独立且并行的解掩码步骤，同时保持局部上下文的一致性，并最小化每一步的联合熵。相比半自回归块方法（如LLADA和Dream），DUS将每个生成块的去噪器调用次数从O(B)降低至O(log B)，显著提升了生成效率。
链接: https://arxiv.org/abs/2506.19037
作者: Omer Luxembourg,Haim Permuter,Eliya Nachmani
机构: 未知
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Theory (cs.IT); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
备注:
Abstract:Masked diffusion language models (MDLM) have shown strong promise for non-autoregressive text generation, yet existing samplers act as implicit planners, selecting tokens to unmask via denoiser confidence or entropy scores. Such heuristics falter under parallel unmasking - they ignore pairwise interactions between tokens and cannot account for dependencies when unmasking multiple positions at once, limiting their inference time to traditional auto-regressive (AR) models. We introduce the Dilated-scheduled Unmasking Strategy (DUS), an inference-only, planner-model-free method that requires no additional training. DUS leverages a first-order Markov assumption to partition sequence positions into dilation-based groups of non-adjacent tokens, enabling independent, parallel unmasking steps that respect local context that minimizes the joint entropy of each iteration step. Unlike semi-AR block approaches (e.g., LLADA and Dream) that still invoke the denoiser per block, DUS reduces the number of denoiser calls to O(log B) per generation block - yielding substantial speedup over the O(B) run time of state-of-the-art diffusion models, where B is the block size in the semi-AR inference process. In experiments on math (GSM8K) and code completion (Humaneval, MBPP) benchmarks - domains suited to non-ordinal generation - DUS improves scores over parallel confidence-based planner, without modifying the underlying denoiser. DUS offers a lightweight, budget-aware approach to efficient, high-quality text generation, paving the way to unlock the true capabilities of MDLMs.
zh
[NLP-58] Quantifying Fairness in LLM s Beyond Tokens: A Semantic and Statistical Perspective
【速读】： 该论文试图解决大型语言模型（Large Language Models, LLMs）在生成长文本响应时存在的固有偏见问题，以及现有评估方法对长文本响应中的偏见和LLM输出内在变异性忽视的问题。解决方案的关键在于提出FiSCo（Fine-grained Semantic Computation），这是一种基于细粒度语义计算的统计框架，通过检测不同人口群体在长文本响应中的细微语义差异来评估群体层面的公平性。与以往关注情感或词元级别的比较不同，FiSCo在主张级别进行分析，并利用蕴含检查来评估语义一致性，从而实现对隐性偏见的稳健检测。
链接: https://arxiv.org/abs/2506.19028
作者: Weijie Xu,Yiwen Wang,Chi Xue,Xiangkun Hu,Xi Fang,Guimin Dong,Chandan K. Reddy
机构: Amazon(亚马逊)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
备注: 29 pages, 9 figures, 15 tables
Abstract:Large Language Models (LLMs) often generate responses with inherent biases, undermining their reliability in real-world applications. Existing evaluation methods often overlook biases in long-form responses and the intrinsic variability of LLM outputs. To address these challenges, we propose FiSCo(Fine-grained Semantic Computation), a novel statistical framework to evaluate group-level fairness in LLMs by detecting subtle semantic differences in long-form responses across demographic groups. Unlike prior work focusing on sentiment or token-level comparisons, FiSCo goes beyond surface-level analysis by operating at the claim level, leveraging entailment checks to assess the consistency of meaning across responses. We decompose model outputs into semantically distinct claims and apply statistical hypothesis testing to compare inter- and intra-group similarities, enabling robust detection of subtle biases. We formalize a new group counterfactual fairness definition and validate FiSCo on both synthetic and human-annotated datasets spanning gender, race, and age. Experiments show that FiSco more reliably identifies nuanced biases while reducing the impact of stochastic LLM variability, outperforming various evaluation metrics.
zh
[NLP-59] Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
【速读】： 该论文试图解决语言模型（Language Models, LM）在面对训练过程中未见过的非规范分词（non-canonical tokenization）时的鲁棒性问题。其解决方案的关键在于发现并利用语言模型在指令微调阶段对非规范分词的适应能力，证明了模型并非如以往认为的那样依赖于特定的分词器，并通过实验验证了在推理阶段干预分词方式可以提升模型性能。
链接: https://arxiv.org/abs/2506.19004
作者: Brian Siyuan Zheng,Alisa Liu,Orevaoghene Ahia,Jonathan Hayase,Yejin Choi,Noah A. Smith
机构: University of Washington (华盛顿大学); Allen Institute for AI (艾伦人工智能研究所); Stanford University (斯坦福大学)
类目: Computation and Language (cs.CL)
备注: preprint
Abstract:Modern tokenizers employ deterministic algorithms to map text into a single “canonical” token sequence, yet the same string can be encoded as many non-canonical tokenizations using the tokenizer vocabulary. In this work, we investigate the robustness of LMs to text encoded with non-canonical tokenizations entirely unseen during training. Surprisingly, when evaluated across 20 benchmarks, we find that instruction-tuned models retain up to 93.4% of their original performance when given a randomly sampled tokenization, and 90.8% with character-level tokenization. We see that overall stronger models tend to be more robust, and robustness diminishes as the tokenization departs farther from the canonical form. Motivated by these results, we then identify settings where non-canonical tokenization schemes can improve performance, finding that character-level segmentation improves string manipulation and code understanding tasks by up to +14%, and right-aligned digit grouping enhances large-number arithmetic by +33%. Finally, we investigate the source of this robustness, finding that it arises in the instruction-tuning phase. We show that while both base and post-trained models grasp the semantics of non-canonical tokenizations (perceiving them as containing misspellings), base models try to mimic the imagined mistakes and degenerate into nonsensical output, while post-trained models are committed to fluent responses. Overall, our findings suggest that models are less tied to their tokenizer than previously believed, and demonstrate the promise of intervening on tokenization at inference time to boost performance.
zh
[NLP-60] Mirag e of Mastery: Memorization Tricks LLM s into Artificially Inflated Self-Knowledge ACL
【速读】： 该论文试图解决大语言模型（Large Language Models, LLMs）在处理科学、技术、工程和数学（STEM）领域问题时，将记忆化内容误认为是推理能力的问题，这种现象导致了模型输出的不可靠性。论文提出了一种新的框架，以确定LLMs是否真正从训练数据中学习了推理模式，还是仅通过记忆解决方案来表现出对相似复杂度问题的胜任能力。该研究的关键在于揭示LLMs在面对逻辑一致但自我验证的任务扰动时，其自信程度源于记忆解法，从而导致超过45%的可行性评估不一致，这凸显了当前模型架构和训练方法在自我知识一致性方面的缺陷。
链接: https://arxiv.org/abs/2506.18998
作者: Sahil Kale,Vijaykant Nadadur
机构: Knowledgeverse AI (知识宇宙人工智能)
类目: Computation and Language (cs.CL)
备注: Accepted to the Pre-ACL Workshop 2025, Copenhagen
Abstract:When artificial intelligence mistakes memorization for intelligence, it creates a dangerous mirage of reasoning. Existing studies treat memorization and self-knowledge deficits in LLMs as separate issues and do not recognize an intertwining link that degrades the trustworthiness of LLM responses. In our study, we utilize a novel framework to ascertain if LLMs genuinely learn reasoning patterns from training data or merely memorize them to assume competence across problems of similar complexity focused on STEM domains. Our analysis shows a noteworthy problem in generalization: LLMs draw confidence from memorized solutions to infer a higher self-knowledge about their reasoning ability, which manifests as an over 45% inconsistency in feasibility assessments when faced with self-validated, logically coherent task perturbations. This effect is most pronounced in science and medicine domains, which tend to have maximal standardized jargon and problems, further confirming our approach. Significant wavering within the self-knowledge of LLMs also shows flaws in current architectures and training patterns, highlighting the need for techniques that ensure a balanced, consistent stance on models’ perceptions of their own knowledge for maximum AI explainability and trustworthiness. Our code and results are available publicly at this https URL.
zh
[NLP-61] From Web Search towards Agent ic Deep Research: Incentivizing Search with Reasoning Agents
【速读】： 该论文试图解决传统基于关键词的搜索引擎在处理复杂、多步骤信息需求时日益不足的问题。其解决方案的关键在于引入具备推理和代理能力的大型语言模型（Large Language Models, LLMs），构建一种称为“代理深度研究”（Agentic Deep Research）的新范式，该范式通过紧密集成自主推理、迭代检索与信息综合，形成动态反馈循环，从而超越传统的信息检索技术。
链接: https://arxiv.org/abs/2506.18959
作者: Weizhi Zhang,Yangning Li,Yuanchen Bei,Junyu Luo,Guancheng Wan,Liangwei Yang,Chenxuan Xie,Yuyao Yang,Wei-Chieh Huang,Chunyu Miao,Henry Peng Zou,Xiao Luo,Yusheng Zhao,Yankai Chen,Chunkit Chan,Peilin Zhou,Xinyang Zhang,Chenwei Zhang,Jingbo Shang,Ming Zhang,Yangqiu Song,Irwin King,Philip S. Yu
机构: University of Illinois Chicago; Tsinghua University; University of Illinois Urbana-Champaign; Peking University; University of California, Los Angeles; Salesforce AI Research; Zhejiang University of Technology; The Hong Kong University of Science and Technology; The Hong Kong University of Science and Technology (Guangzhou); Amazon; University of California, San Diego; The Chinese University of Hong Kong
类目: Information Retrieval (cs.IR); Computation and Language (cs.CL); Machine Learning (cs.LG)
备注:
Abstract:Information retrieval is a cornerstone of modern knowledge acquisition, enabling billions of queries each day across diverse domains. However, traditional keyword-based search engines are increasingly inadequate for handling complex, multi-step information needs. Our position is that Large Language Models (LLMs), endowed with reasoning and agentic capabilities, are ushering in a new paradigm termed Agentic Deep Research. These systems transcend conventional information search techniques by tightly integrating autonomous reasoning, iterative retrieval, and information synthesis into a dynamic feedback loop. We trace the evolution from static web search to interactive, agent-based systems that plan, explore, and learn. We also introduce a test-time scaling law to formalize the impact of computational depth on reasoning and search. Supported by benchmark results and the rise of open-source implementations, we demonstrate that Agentic Deep Research not only significantly outperforms existing approaches, but is also poised to become the dominant paradigm for future information seeking. All the related resources, including industry products, research papers, benchmark datasets, and open-source implementations, are collected for the community in this https URL.
zh
[NLP-62] A Comment On “The Illusion of Thinking”: Reframing the Reasoning Cliff as an Agent ic Gap
【速读】： 该论文试图解决大型推理模型（Large Reasoning Models, LRM）在面对高复杂度问题时性能骤降的“推理悬崖”现象，即模型在特定复杂度阈值后表现显著下降的问题。其解决方案的关键在于重新诠释这一现象，认为模型并非在推理能力上存在根本性局限，而是受限于静态文本生成评估范式中的系统级约束，如工具使用限制、上下文窗口记忆问题、缺乏关键认知基线、统计报告不足以及输出生成限制等。研究通过引入代理工具（agentic tools）验证了这一观点，展示了模型在获得执行能力后能够克服先前无法解决的复杂问题，从而揭示了“推理悬崖”更多是由于模型缺乏行动工具而非推理能力不足所导致的。
链接: https://arxiv.org/abs/2506.18957
作者: Sheraz Khan,Subha Madhavan,Kannan Natarajan
机构: 未知
类目: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
备注: 10 pages, 2 figures, Comment on “The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity” ( arXiv:2506.06941v1 )
Abstract:The recent work by Shojaee et al. (2025), titled The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity, presents a compelling empirical finding, a reasoning cliff, where the performance of Large Reasoning Models (LRMs) collapses beyond a specific complexity threshold, which the authors posit as an intrinsic scaling limitation of Chain-of-Thought (CoT) reasoning. This commentary, while acknowledging the study’s methodological rigor, contends that this conclusion is confounded by experimental artifacts. We argue that the observed failure is not evidence of a fundamental cognitive boundary, but rather a predictable outcome of system-level constraints in the static, text-only evaluation paradigm, including tool use restrictions, context window recall issues, the absence of crucial cognitive baselines, inadequate statistical reporting, and output generation limits. We reframe this performance collapse through the lens of an agentic gap, asserting that the models are not failing at reasoning, but at execution within a profoundly restrictive interface. We empirically substantiate this critique by demonstrating a striking reversal. A model, initially declaring a puzzle impossible when confined to text-only generation, now employs agentic tools to not only solve it but also master variations of complexity far beyond the reasoning cliff it previously failed to surmount. Additionally, our empirical analysis of tool-enabled models like o4-mini and GPT-4o reveals a hierarchy of agentic reasoning, from simple procedural execution to complex meta-cognitive self-correction, which has significant implications for how we define and measure machine intelligence. The illusion of thinking attributed to LRMs is less a reasoning deficit and more a consequence of an otherwise capable mind lacking the tools for action.
zh
[NLP-63] LLM s on a Budget? Say HOLA
【速读】： 该论文旨在解决在边缘设备上运行大型语言模型（Large Language Models, LLMs）时面临的高计算和内存需求问题，这一问题限制了其在医疗、教育和嵌入式系统等实时应用场景中的部署。论文提出的解决方案是HOLA框架，其关键在于内部采用分层推测解码（Hierarchical Speculative Decoding, HSD）以实现快速推理而不损失质量，外部通过AdaComp-RAG根据上下文需求调整检索复杂度，并结合LoBi技术融合结构化剪枝（LoRA）与量化，从而显著提升了性能并降低了延迟和内存占用。
链接: https://arxiv.org/abs/2506.18952
作者: Zohaib Hasan Siddiqui,Jiechao Gao,Ebad Shabbir,Mohammad Anas Azeez,Rafiq Ali,Gautam Siddharth Kashyap,Usman Naseem
机构: Jamia Hamdard, New Delhi, India; Center for SDGC, Stanford University, California, USA; DSEU-Okhla, New Delhi, India; Macquarie University, Sydney, Australia
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
备注:
Abstract:Running Large Language Models (LLMs) on edge devices is constrained by high compute and memory demands posing a barrier for real-time applications in sectors like healthcare, education, and embedded systems. Current solutions such as quantization, pruning, and retrieval-augmented generation (RAG) offer only partial optimizations and often compromise on speed or accuracy. We introduce HOLA, an end-to-end optimization framework for efficient LLM deployment. Internally, it leverages Hierarchical Speculative Decoding (HSD) for faster inference without quality loss. Externally, AdaComp-RAG adjusts retrieval complexity based on context needs. Together with LoBi, which blends structured pruning (LoRA) and quantization, HOLA delivers significant gains: 17.6% EMA on GSM8K, 10.5% MCA on ARC, and reduced latency and memory on edge devices like Jetson Nano–proving both scalable and production-ready.
zh
[NLP-64] Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
【速读】： 该论文试图解决传统Mixture-of-Experts (MoE)架构中专家独立并行操作导致的表达能力受限和计算资源利用不充分的问题。其解决方案的关键在于提出Chain-of-Experts (CoE)架构，通过在每一层内引入序列化的专家通信机制，使令牌在迭代过程中动态选择不同的专家，从而增强专家组合的多样性并提升模型的表示能力。这种设计实现了灵活的路由机制，并通过专家迭代扩展了模型的深度维度，有效提升了计算效率与性能。
链接: https://arxiv.org/abs/2506.18945
作者: Zihan Wang,Rui Pan,Jiarui Yao,Robert Csordas,Linjie Li,Lu Yin,Jiajun Wu,Tong Zhang,Manling Li,Shiwei Liu
机构: Northwestern University (西北大学); University of Illinois Urbana-Champaign (伊利诺伊大学厄巴纳-香槟分校); Stanford University (斯坦福大学); University of Washington (华盛顿大学); University of Surrey (萨里大学); University of Oxford (牛津大学)
类目: Machine Learning (cs.LG); Computation and Language (cs.CL)
备注:
Abstract:We propose Chain-of-Experts (CoE), a new Mixture-of-Experts (MoE) architecture that introduces sequential expert communication within each layer. Unlike traditional MoE models, where experts operate independently in parallel, CoE processes tokens iteratively across a chain of experts inside a layer. To support dynamic expert selection across iterations, CoE employs a dedicated router at each iteration step within a layer. This design allows tokens to re-evaluate and select different experts during each iteration, rather than being statically assigned. As a result, CoE introduces a flexible routing mechanism that increases the diversity of expert combinations and enriches the model’s representational capacity. CoE demonstrates improved performance under fixed compute: on math reasoning tasks, it reduces validation loss from 1.20 to 1.12 compared to a standard MoE. Beyond performance, CoE offers a new scaling axis: depth through expert iteration, which complements conventional width/depth scaling. For example, using 2x iterations matches the performance of 3x expert selections (in width), while reducing memory usage by 17.6-42% relative to other scaling strategies. Our analysis reveals that CoE’s benefits stem from its iterative residual structure and enhanced expert specialization empowered by iterative routing, which together unlock more expressive representations. Code is available at this https URL.
zh
[NLP-65] Mix-of-Language-Experts Architecture for Multilingual Programming ICSE2025
【速读】： 该论文旨在解决多语言编程任务中如何在模型效率与语言特异性性能之间取得平衡的问题。传统方法要么通过微调单一模型实现成本效益，但牺牲了语言特异性；要么为每种编程语言单独微调模型，虽能实现特异性但计算和存储成本高昂。论文提出的解决方案是MoLE（Mix-of-Language-Experts），其关键在于引入了一个基础模型、一个共享的低秩适配（LoRA）模块以及一组语言特异性LoRA模块，并在微调过程中联合优化这些模块，从而实现跨语言的知识共享与特异性增强。在推理阶段，MoLE会根据生成代码的编程语言自动选择对应的语言特异性LoRA模块。
链接: https://arxiv.org/abs/2506.18923
作者: Yifan Zong,Yuntian Deng,Pengyu Nie
机构: University of Waterloo (滑铁卢大学)
类目: Programming Languages (cs.PL); Computation and Language (cs.CL); Software Engineering (cs.SE)
备注: Accepted at LLM4Code @ ICSE 2025
Abstract:Large language models (LLMs) have demonstrated impressive capabilities in aiding developers with tasks like code comprehension, generation, and translation. Supporting multilingual programming – i.e., coding tasks across multiple programming languages – typically requires either (1) finetuning a single LLM across all programming languages, which is cost-efficient but sacrifices language-specific specialization and performance, or (2) finetuning separate LLMs for each programming language, which allows for specialization but is computationally expensive and storage-intensive due to the duplication of parameters. This paper introduces MoLE (Mix-of-Language-Experts), a novel architecture that balances efficiency and specialization for multilingual programming. MoLE is composed of a base model, a shared LoRA (low-rank adaptation) module, and a collection of language-specific LoRA modules. These modules are jointly optimized during the finetuning process, enabling effective knowledge sharing and specialization across programming languages. During inference, MoLE automatically routes to the language-specific LoRA module corresponding to the programming language of the code token being generated. Our experiments demonstrate that MoLE achieves greater parameter efficiency compared to training separate language-specific LoRAs, while outperforming a single shared LLM finetuned for all programming languages in terms of accuracy.
zh
[NLP-66] MemeMind: A Large-Scale Multimodal Dataset with Chain-of-Thought Reasoning for Harmful Meme Detection
【速读】： 该论文旨在解决有害模因（harmful memes）自动检测中由于其隐含语义和复杂的多模态交互所带来的挑战，当前缺乏系统性、大规模、多样化且高度可解释的数据集成为制约该领域进一步发展的关键问题。解决方案的关键在于提出MemeMind数据集，该数据集具有科学严谨性、大规模、多样性、双语支持以及详细的Chain-of-Thought（CoT）标注，能够提供全面的标签和明确的推理轨迹，从而为提升有害模因检测提供坚实基础；同时，论文还提出了MemeGuard框架，通过有效整合多模态信息与推理过程建模，显著提升了模型对有害模因的理解与识别能力。
链接: https://arxiv.org/abs/2506.18919
作者: Hexiang Gu,Qifan Yu,Saihui Hou,Zhiqin Fang,Huijia Wu,Zhaofeng He
机构: Beijing University of Posts and Telecommunications(北京邮电大学); Beijing Normal University(北京师范大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:The rapid development of social media has intensified the spread of harmful content. Harmful memes, which integrate both images and text, pose significant challenges for automated detection due to their implicit semantics and complex multimodal interactions. Although existing research has made progress in detection accuracy and interpretability, the lack of a systematic, large-scale, diverse, and highly explainable dataset continues to hinder further advancement in this field. To address this gap, we introduce MemeMind, a novel dataset featuring scientifically rigorous standards, large scale, diversity, bilingual support (Chinese and English), and detailed Chain-of-Thought (CoT) annotations. MemeMind fills critical gaps in current datasets by offering comprehensive labeling and explicit reasoning traces, thereby providing a solid foundation for enhancing harmful meme detection. In addition, we propose an innovative detection framework, MemeGuard, which effectively integrates multimodal information with reasoning process modeling, significantly improving models’ ability to understand and identify harmful memes. Extensive experiments conducted on the MemeMind dataset demonstrate that MemeGuard consistently outperforms existing state-of-the-art methods in harmful meme detection tasks.
zh
[NLP-67] Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Models
【速读】： 该论文试图解决预训练过程中面临的“数据墙”问题，即自然数据的获取速度无法跟上计算资源的增长，且高质量文本的可用性有限。解决方案的关键在于通过REcycling the Web with guIded REwrite（REWIRE）方法，对现有过滤流程中被丢弃的低质量文档进行转换和重构，使其成为可用于训练的有效数据。该方法提高了合成数据在最终预训练集中的比例，并在多个任务中表现出优于仅使用过滤后网络数据的性能。
链接: https://arxiv.org/abs/2506.04689
作者: Thao Nguyen,Yang Li,Olga Golovneva,Luke Zettlemoyer,Sewoong Oh,Ludwig Schmidt,Xian Li
机构: FAIR at Meta (FAIR at Meta); University of Washington (华盛顿大学); Stanford University (斯坦福大学)
类目: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Scaling laws predict that the performance of large language models improves with increasing model size and data size. In practice, pre-training has been relying on massive web crawls, using almost all data sources publicly available on the internet so far. However, this pool of natural data does not grow at the same rate as the compute supply. Furthermore, the availability of high-quality texts is even more limited: data filtering pipelines often remove up to 99% of the initial web scrapes to achieve state-of-the-art. To address the “data wall” of pre-training scaling, our work explores ways to transform and recycle data discarded in existing filtering processes. We propose REWIRE, REcycling the Web with guIded REwrite, a method to enrich low-quality documents so that they could become useful for training. This in turn allows us to increase the representation of synthetic data in the final pre-training set. Experiments at 1B, 3B and 7B scales of the DCLM benchmark show that mixing high-quality raw texts and our rewritten texts lead to 1.0, 1.3 and 2.5 percentage points improvement respectively across 22 diverse tasks, compared to training on only filtered web data. Training on the raw-synthetic data mix is also more effective than having access to 2x web data. Through further analysis, we demonstrate that about 82% of the mixed in texts come from transforming lower-quality documents that would otherwise be discarded. REWIRE also outperforms related approaches of generating synthetic data, including Wikipedia-style paraphrasing, question-answer synthesizing and knowledge extraction. These results suggest that recycling web texts holds the potential for being a simple and effective approach for scaling pre-training data.
zh
[NLP-68] Kling-Foley: Multimodal Diffusion Transformer for High-Quality Video-to-Audio Generation
【速读】： 该论文旨在解决视频到音频生成中的多模态对齐与高质量音频合成问题，特别是如何实现视频内容与音频的语义、时间及空间上的精确同步。解决方案的关键在于提出Kling-Foley模型，其核心是引入多模态扩散Transformer来建模视频、音频和文本模态之间的交互，并结合视觉语义表示模块与音画同步模块，以提升多模态间的对齐能力。此外，还提出了一个通用的潜在音频编解码器，支持多种音频类型的质量建模，并采用立体声渲染方法增强音频的空间感，从而实现更逼真的音画同步效果。
链接: https://arxiv.org/abs/2506.19774
作者: Jun Wang,Xijuan Zeng,Chunyu Qiang,Ruilong Chen,Shiyao Wang,Le Wang,Wangjing Zhou,Pengfei Cai,Jiahui Zhao,Nan Li,Zihan Li,Yuzhe Liang,Xiaopeng Wang,Haorui Zheng,Ming Wen,Kang Yin,Yiran Wang,Nan Li,Feng Deng,Liang Dong,Chen Zhang,Di Zhang,Kun Gai
机构: Kuaishou Technology(快手科技)
类目: Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Sound (cs.SD)
备注:
Abstract:We propose Kling-Foley, a large-scale multimodal Video-to-Audio generation model that synthesizes high-quality audio synchronized with video content. In Kling-Foley, we introduce multimodal diffusion transformers to model the interactions between video, audio, and text modalities, and combine it with a visual semantic representation module and an audio-visual synchronization module to enhance alignment capabilities. Specifically, these modules align video conditions with latent audio elements at the frame level, thereby improving semantic alignment and audio-visual synchronization. Together with text conditions, this integrated approach enables precise generation of video-matching sound effects. In addition, we propose a universal latent audio codec that can achieve high-quality modeling in various scenarios such as sound effects, speech, singing, and music. We employ a stereo rendering method that imbues synthesized audio with a spatial presence. At the same time, in order to make up for the incomplete types and annotations of the open-source benchmark, we also open-source an industrial-level benchmark Kling-Audio-Eval. Our experiments show that Kling-Foley trained with the flow matching objective achieves new audio-visual SOTA performance among public models in terms of distribution matching, semantic alignment, temporal alignment and audio quality.
zh
计算机视觉
[CV-0] Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generation
【速读】：该论文旨在解决视频扩散模型中由于时间维度的增加而导致的计算成本过高问题，特别是在长视频生成任务中，训练和推理变得极其昂贵。其解决方案的关键在于提出了一种名为Radial Attention的可扩展稀疏注意力机制，该机制通过将时空能量衰减转化为计算密度的指数衰减，实现了O(n log n)的复杂度，相较于传统的O(n²)密集注意力机制更具效率，同时比线性注意力更具表达能力。
链接: https://arxiv.org/abs/2506.19852
作者: Xingyang Li,Muyang Li,Tianle Cai,Haocheng Xi,Shuo Yang,Yujun Lin,Lvmin Zhang,Songlin Yang,Jinbo Hu,Kelly Peng,Maneesh Agrawala,Ion Stoica,Kurt Keutzer,Song Han
机构: MIT(麻省理工学院); NVIDIA(英伟达); Princeton(普林斯顿大学); UC Berkeley(加州大学伯克利分校); Stanford(斯坦福大学); First Intelligence
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Code: this https URL
Abstract:Recent advances in diffusion models have enabled high-quality video generation, but the additional temporal dimension significantly increases computational costs, making training and inference on long videos prohibitively expensive. In this paper, we identify a phenomenon we term Spatiotemporal Energy Decay in video diffusion models: post-softmax attention scores diminish as spatial and temporal distance between tokens increase, akin to the physical decay of signal or waves over space and time in nature. Motivated by this, we propose Radial Attention, a scalable sparse attention mechanism with O(n \log n) complexity that translates energy decay into exponentially decaying compute density, which is significantly more efficient than standard O(n^2) dense attention and more expressive than linear attention. Specifically, Radial Attention employs a simple, static attention mask where each token attends to spatially nearby tokens, with the attention window size shrinking with temporal distance. Moreover, it allows pre-trained video diffusion models to extend their generation length with efficient LoRA-based fine-tuning. Extensive experiments show that Radial Attention maintains video quality across Wan2.1-14B, HunyuanVideo, and Mochi 1, achieving up to a 1.9 \times speedup over the original dense attention. With minimal tuning, it enables video generation up to 4 \times longer while reducing training costs by up to 4.4 \times compared to direct fine-tuning and accelerating inference by up to 3.7 \times compared to dense attention inference.
zh
[CV-1] AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Models
【速读】：该论文旨在解决传统运动合成方法在骨骼拓扑结构固定或需要高维变形空间中高昂优化成本的问题。其解决方案的关键在于提出AnimaX，一个前馈式3D动画框架，通过将视频扩散模型的运动先验与基于骨架动画的可控制结构相结合，实现了从视频中迁移运动知识到3D领域的高效方法。该方法通过多视角、多帧2D姿态图表示3D运动，并利用模板渲染和文本运动提示进行联合视频-姿态扩散，同时引入共享位置编码和模态感知嵌入以确保视频与姿态序列的时空对齐，最终通过逆运动学将多视角姿态序列转换为网格动画。
链接: https://arxiv.org/abs/2506.19851
作者: Zehuan Huang,Haoran Feng,Yangtian Sun,Yuanchen Guo,Yanpei Cao,Lu Sheng
机构: Beihang University (北京航空航天大学); Tsinghua University (清华大学); The University of Hong Kong (香港大学); VAST (中国科学院自动化研究所)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:We present AnimaX, a feed-forward 3D animation framework that bridges the motion priors of video diffusion models with the controllable structure of skeleton-based animation. Traditional motion synthesis methods are either restricted to fixed skeletal topologies or require costly optimization in high-dimensional deformation spaces. In contrast, AnimaX effectively transfers video-based motion knowledge to the 3D domain, supporting diverse articulated meshes with arbitrary skeletons. Our method represents 3D motion as multi-view, multi-frame 2D pose maps, and enables joint video-pose diffusion conditioned on template renderings and a textual motion prompt. We introduce shared positional encodings and modality-aware embeddings to ensure spatial-temporal alignment between video and pose sequences, effectively transferring video priors to motion generation task. The resulting multi-view pose sequences are triangulated into 3D joint positions and converted into mesh animation via inverse kinematics. Trained on a newly curated dataset of 160,000 rigged sequences, AnimaX achieves state-of-the-art results on VBench in generalization, motion fidelity, and efficiency, offering a scalable solution for category-agnostic 3D animation. Project page: \hrefthis https URLthis https URL.
zh
[CV-2] Unified Vision-Language-Action Model
【速读】：该论文旨在解决传统视觉-语言-动作模型（Vision-Language-Action Models, VLAs）在机器人操作任务中依赖视觉-语言模型（Vision-Language Models, VLMs）的通用理解能力生成动作信号，而忽视了视觉观察中蕴含的丰富时序与因果结构的问题。其解决方案的关键在于提出UniVLA，一个统一且原生的多模态VLA模型，通过自回归方式将视觉、语言和动作信号建模为离散的token序列，从而实现从大规模视频数据中灵活学习多模态任务，并在后训练阶段引入世界建模以捕捉视频中的因果动态，提升长时程任务的下游策略学习效果。
链接: https://arxiv.org/abs/2506.19850
作者: Yuqi Wang,Xinghang Li,Wenxuan Wang,Junbo Zhang,Yingyan Li,Yuntao Chen,Xinlong Wang,Zhaoxiang Zhang
机构: CASIA(中国科学院); BAAI(百度研究院); THU(清华大学); HKISI(香港智能系统研究所)
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注: technical report
Abstract:Vision-language-action models (VLAs) have garnered significant attention for their potential in advancing robotic manipulation. However, previous approaches predominantly rely on the general comprehension capabilities of vision-language models (VLMs) to generate action signals, often overlooking the rich temporal and causal structure embedded in visual observations. In this paper, we present UniVLA, a unified and native multimodal VLA model that autoregressively models vision, language, and action signals as discrete token sequences. This formulation enables flexible multimodal tasks learning, particularly from large-scale video data. By incorporating world modeling during post-training, UniVLA captures causal dynamics from videos, facilitating effective transfer to downstream policy learning–especially for long-horizon tasks. Our approach sets new state-of-the-art results across several widely used simulation benchmarks, including CALVIN, LIBERO, and Simplenv-Bridge, significantly surpassing previous methods. For example, UniVLA achieves 95.5% average success rate on LIBERO benchmark, surpassing pi0-FAST’s 85.5%. We further demonstrate its broad applicability on real-world ALOHA manipulation and autonomous driving.
zh
[CV-3] A Comparative Study of NAFNet Baselines for Image Restoration
【速读】：该论文旨在解决图像恢复问题，提出了一种简单且高效的深度学习基线模型NAFNet（Nonlinear Activation Free Network）。其解决方案的关键在于引入SimpleGate激活函数、简化通道注意力机制（Simplified Channel Attention, SCA）以及LayerNorm层，这些组件共同提升了图像恢复的性能，并在训练过程中增强了稳定性。通过消融实验验证了各组件的有效性，表明SimpleGate和简化注意力机制优于传统激活函数和注意力机制，而LayerNorm对于稳定训练至关重要。
链接: https://arxiv.org/abs/2506.19845
作者: Vladislav Esaulov,M. Moein Esfahani
机构: Georgia State University (佐治亚州立大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:We study NAFNet (Nonlinear Activation Free Network), a simple and efficient deep learning baseline for image restoration. By using CIFAR10 images corrupted with noise and blur, we conduct an ablation study of NAFNet’s core components. Our baseline model implements SimpleGate activation, Simplified Channel Activation (SCA), and LayerNormalization. We compare this baseline to different variants that replace or remove components. Quantitative results (PSNR, SSIM) and examples illustrate how each modification affects restoration performance. Our findings support the NAFNet design: the SimpleGate and simplified attention mechanisms yield better results than conventional activations and attention, while LayerNorm proves to be important for stable training. We conclude with recommendations for model design, discuss potential improvements, and future work.
zh
[CV-4] Active View Selector: Fast and Accurate Active View Selection with Cross Reference Image Quality Assessment
【速读】：该论文试图解决新颖视图合成与三维重建中的主动视图选择问题（active view selection），传统方法如FisheRF和ActiveNeRF通过最小化不确定性或最大化信息增益来选择下一最佳视图，但这些方法需要针对不同的三维表示进行专门设计，并且在三维空间中涉及复杂的建模过程。该论文的解决方案关键在于将问题重新表述为二维图像质量评估（IQA）任务，通过选择当前渲染质量最低的视图来实现有效视图选择。为了解决候选视图缺乏真实图像的问题，该论文受CrossScore框架启发，训练模型在多视图设置下预测SSIM，并以此指导视图选择，从而实现了在标准基准上的显著定量和定性提升，同时对三维表示具有无关性，并且计算效率提高了14-33倍。
链接: https://arxiv.org/abs/2506.19844
作者: Zirui Wang,Yash Bhalgat,Ruining Li,Victor Adrian Prisacariu
机构: Active Vision Lab (主动视觉实验室); Visual Geometry Group (视觉几何组); University of Oxford (牛津大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:We tackle active view selection in novel view synthesis and 3D reconstruction. Existing methods like FisheRF and ActiveNeRF select the next best view by minimizing uncertainty or maximizing information gain in 3D, but they require specialized designs for different 3D representations and involve complex modelling in 3D space. Instead, we reframe this as a 2D image quality assessment (IQA) task, selecting views where current renderings have the lowest quality. Since ground-truth images for candidate views are unavailable, full-reference metrics like PSNR and SSIM are inapplicable, while no-reference metrics, such as MUSIQ and MANIQA, lack the essential multi-view context. Inspired by a recent cross-referencing quality framework CrossScore, we train a model to predict SSIM within a multi-view setup and use it to guide view selection. Our cross-reference IQA framework achieves substantial quantitative and qualitative improvements across standard benchmarks, while being agnostic to 3D representations, and runs 14-33 times faster than previous methods.
zh
[CV-5] GenHSI: Controllable Generation of Human-Scene Interaction Videos
【速读】：该论文试图解决在生成长视频时存在的问题，包括不现实的人-场景交互、主体身份无法保持以及训练成本高昂等。其解决方案的关键在于提出GenHSI，一种无需训练的可控生成方法，通过将长视频生成任务划分为三个阶段：剧本编写、预可视化和动画制作，从而生成具有丰富人-场景交互且保持人物身份的长视频。该方法利用单张场景图像、用户描述和多张人物图像，结合现成的视频扩散模型，生成具有连贯摄像机姿态和任意数量角色动作的长视频序列。
链接: https://arxiv.org/abs/2506.19840
作者: Zekun Li,Rui Zhou,Rahul Sajnani,Xiaoyan Cong,Daniel Ritchie,Srinath Sridhar
机构: Brown University (布朗大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Large-scale pre-trained video diffusion models have exhibited remarkable capabilities in diverse video generation. However, existing solutions face several challenges in using these models to generate long movie-like videos with rich human-object interactions that include unrealistic human-scene interaction, lack of subject identity preservation, and require expensive training. We propose GenHSI, a training-free method for controllable generation of long human-scene interaction videos (HSI). Taking inspiration from movie animation, our key insight is to overcome the limitations of previous work by subdividing the long video generation task into three stages: (1) script writing, (2) pre-visualization, and (3) animation. Given an image of a scene, a user description, and multiple images of a person, we use these three stages to generate long-videos that preserve human-identity and provide rich human-scene interactions. Script writing converts complex human tasks into simple atomic tasks that are used in the pre-visualization stage to generate 3D keyframes (storyboards). These 3D keyframes are rendered and animated by off-the-shelf video diffusion models for consistent long video generation with rich contacts in a 3D-aware manner. A key advantage of our work is that we alleviate the need for scanned, accurate scenes and create 3D keyframes from single-view images. We are the first to generate a long video sequence with a consistent camera pose that contains arbitrary numbers of character actions without training. Experiments demonstrate that our method can generate long videos that effectively preserve scene content and character identity with plausible human-scene interaction from a single image scene. Visit our project homepage this https URL for more information.
zh
[CV-6] Improving Progressive Generation with Decomposable Flow Matching
【速读】：该论文试图解决高维视觉模态生成的计算密集型问题，特别是在多阶段生成框架中引入的复杂性与效率低下问题。其解决方案的关键在于提出一种名为可分解流匹配（Decomposable Flow Matching, DFM）的简单而有效的框架，该框架在用户定义的多尺度表示（如拉普拉斯金字塔）的每一层上独立应用流匹配（Flow Matching），从而实现了渐进式生成，同时保持了模型架构的简洁性和对现有训练流程的最小修改需求。
链接: https://arxiv.org/abs/2506.19839
作者: Moayed Haji-Ali,Willi Menapace,Ivan Skorokhodov,Arpit Sahni,Sergey Tulyakov,Vicente Ordonez,Aliaksandr Siarohin
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: Project Webpage: this https URL
Abstract:Generating high-dimensional visual modalities is a computationally intensive task. A common solution is progressive generation, where the outputs are synthesized in a coarse-to-fine spectral autoregressive manner. While diffusion models benefit from the coarse-to-fine nature of denoising, explicit multi-stage architectures are rarely adopted. These architectures have increased the complexity of the overall approach, introducing the need for a custom diffusion formulation, decomposition-dependent stage transitions, add-hoc samplers, or a model cascade. Our contribution, Decomposable Flow Matching (DFM), is a simple and effective framework for the progressive generation of visual media. DFM applies Flow Matching independently at each level of a user-defined multi-scale representation (such as Laplacian pyramid). As shown by our experiments, our approach improves visual quality for both images and videos, featuring superior results compared to prior multistage frameworks. On Imagenet-1k 512px, DFM achieves 35.2% improvements in FDD scores over the base architecture and 26.4% over the best-performing baseline, under the same training compute. When applied to finetuning of large models, such as FLUX, DFM shows faster convergence speed to the training distribution. Crucially, all these advantages are achieved with a single model, architectural simplicity, and minimal modifications to existing training pipelines.
zh
[CV-7] SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution
【速读】：该论文旨在解决高分辨率视频生成中传统潜在扩散模型计算效率不足的问题，其核心挑战在于如何在保持计算可行性的同时满足用户对更高分辨率输出的需求。解决方案的关键在于将生成过程分解为两个阶段：语义内容生成与细节合成，其中后者通过轻量级级联视频超分辨率（VSR）模型实现高分辨率输出。论文重点研究了级联VSR模型的设计原则，提出了两种退化策略以生成更符合基础模型输出特征的训练对，并通过系统分析时间步采样策略和噪声增强对低分辨率输入的影响，指导了架构与训练方法的创新，最终引入交错时序单元和稀疏局部注意力机制以提升训练与推理效率。
链接: https://arxiv.org/abs/2506.19838
作者: Liangbin Xie,Yu Li,Shian Du,Menghan Xia,Xintao Wang,Fanghua Yu,Ziyan Chen,Pengfei Wan,Jiantao Zhou,Chao Dong
机构: University of Macau (澳门大学); Chinese Academy of Sciences (中国科学院); Tsinghua University (清华大学); Kuaishou Technology (快手科技); Shenzhen University of Advanced Technology (深圳先进技术大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project webpage available at this https URL
Abstract:Latent diffusion models have emerged as a leading paradigm for efficient video generation. However, as user expectations shift toward higher-resolution outputs, relying solely on latent computation becomes inadequate. A promising approach involves decoupling the process into two stages: semantic content generation and detail synthesis. The former employs a computationally intensive base model at lower resolutions, while the latter leverages a lightweight cascaded video super-resolution (VSR) model to achieve high-resolution output. In this work, we focus on studying key design principles for latter cascaded VSR models, which are underexplored currently. First, we propose two degradation strategies to generate training pairs that better mimic the output characteristics of the base model, ensuring alignment between the VSR model and its upstream generator. Second, we provide critical insights into VSR model behavior through systematic analysis of (1) timestep sampling strategies, (2) noise augmentation effects on low-resolution (LR) inputs. These findings directly inform our architectural and training innovations. Finally, we introduce interleaving temporal unit and sparse local attention to achieve efficient training and inference, drastically reducing computational overhead. Extensive experiments demonstrate the superiority of our framework over existing methods, with ablation studies confirming the efficacy of each design choice. Our work establishes a simple yet effective baseline for cascaded video super-resolution generation, offering practical insights to guide future advancements in efficient cascaded synthesis systems.
zh
[CV-8] Bind-Your-Avatar: Multi-Talking-Character Video Generation with Dynamic 3D-mask-based Embedding Router
【速读】：该论文旨在解决多角色在同一场景中进行对话视频生成的问题，特别是音频与角色对应控制以及缺乏适合的多角色对话视频数据集这两个关键挑战。其解决方案的关键在于提出一种基于MM-DiT的模型——Bind-Your-Avatar，该模型通过引入细粒度的Embedding Router来绑定“谁”和“说什么”，从而实现音频到角色的精准对应控制；同时采用两种方法实现3D掩码嵌入路由器，以获得逐帧的精细角色控制，并结合基于几何先验的损失函数和掩码优化策略提升预测掩码的准确性和时间平滑性。此外，还构建了首个针对多角色对话视频生成的数据集，并提供了开源的数据处理流程。
链接: https://arxiv.org/abs/2506.19833
作者: Yubo Huang,Weiqiang Wang,Sirui Zhao,Tong Xu,Lin Liu,Enhong Chen
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Recent years have witnessed remarkable advances in audio-driven talking head generation. However, existing approaches predominantly focus on single-character scenarios. While some methods can create separate conversation videos between two individuals, the critical challenge of generating unified conversation videos with multiple physically co-present characters sharing the same spatial environment remains largely unaddressed. This setting presents two key challenges: audio-to-character correspondence control and the lack of suitable datasets featuring multi-character talking videos within the same scene. To address these challenges, we introduce Bind-Your-Avatar, an MM-DiT-based model specifically designed for multi-talking-character video generation in the same scene. Specifically, we propose (1) A novel framework incorporating a fine-grained Embedding Router that binds who' and speak what’ together to address the audio-to-character correspondence control. (2) Two methods for implementing a 3D-mask embedding router that enables frame-wise, fine-grained control of individual characters, with distinct loss functions based on observed geometric priors and a mask refinement strategy to enhance the accuracy and temporal smoothness of the predicted masks. (3) The first dataset, to the best of our knowledge, specifically constructed for multi-talking-character video generation, and accompanied by an open-source data processing pipeline, and (4) A benchmark for the dual-talking-characters video generation, with extensive experiments demonstrating superior performance over multiple state-of-the-art methods.
zh
[CV-9] Look to Locate: Vision-Based Multisensory Navigation with 3-D Digital Maps for GNSS-Challenged Environments
【速读】：该论文旨在解决在无全球导航卫星系统（GNSS）信号的环境中，如室内停车场或密集城市峡谷，实现准确且鲁棒的车辆定位问题。其解决方案的关键在于构建一种基于视觉的多传感器导航系统，该系统融合了单目深度估计、语义过滤和视觉地图注册（VMR）与三维数字地图，从而在无需GNSS的情况下实现高精度定位。
链接: https://arxiv.org/abs/2506.19827
作者: Ola Elmaghraby,Eslam Mounier,Paulo Ricardo Marques de Araujo,Aboelmagd Noureldin
机构: Queen’s University (女王大学); Royal Military College of Canada (加拿大皇家军事学院); Ain Shams University (艾因夏姆斯大学)
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:In Global Navigation Satellite System (GNSS)-denied environments such as indoor parking structures or dense urban canyons, achieving accurate and robust vehicle positioning remains a significant challenge. This paper proposes a cost-effective, vision-based multi-sensor navigation system that integrates monocular depth estimation, semantic filtering, and visual map registration (VMR) with 3-D digital maps. Extensive testing in real-world indoor and outdoor driving scenarios demonstrates the effectiveness of the proposed system, achieving sub-meter accuracy of 92% indoors and more than 80% outdoors, with consistent horizontal positioning and heading average root mean-square errors of approximately 0.98 m and 1.25 °, respectively. Compared to the baselines examined, the proposed solution significantly reduced drift and improved robustness under various conditions, achieving positioning accuracy improvements of approximately 88% on average. This work highlights the potential of cost-effective monocular vision systems combined with 3D maps for scalable, GNSS-independent navigation in land vehicles.
zh
[CV-10] CronusVLA: Transferring Latent Motion Across Time for Multi-Frame Prediction in Manipulation
【速读】：该论文旨在解决现有视觉-语言-动作（Vision-Language-Action, VLA）模型受限于单帧观测范式的问题，无法充分利用多帧历史观测提供的运动信息，同时由于大型视觉-语言基础模型带来的计算成本和推理延迟，限制了其性能提升。解决方案的关键在于提出CronusVLA框架，通过高效的后训练阶段将单帧VLA模型扩展至多帧范式，其核心组件包括：基于大规模具身数据集的单帧预训练、多帧编码与运动特征聚合、以及跨帧解码机制，从而实现高效的推理和性能优化。
链接: https://arxiv.org/abs/2506.19816
作者: Hao Li,Shuai Yang,Yilun Chen,Yang Tian,Xiaoda Yang,Xinyi Chen,Hanqing Wang,Tai Wang,Feng Zhao,Dahua Lin,Jiangmiao Pang
机构: University of Science and Technology of China (中国科学技术大学); Shanghai Artificial Intelligence Laboratory (上海人工智能实验室); Zhejiang University (浙江大学); The Chinese University of Hong Kong (香港中文大学)
类目: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
备注: 36 pages, 21 figures
Abstract:Recent vision-language-action (VLA) models built on pretrained vision-language models (VLMs) have demonstrated strong generalization across manipulation tasks. However, they remain constrained by a single-frame observation paradigm and cannot fully benefit from the motion information offered by aggregated multi-frame historical observations, as the large vision-language backbone introduces substantial computational cost and inference latency. We propose CronusVLA, a unified framework that extends single-frame VLA models to the multi-frame paradigm through an efficient post-training stage. CronusVLA comprises three key components: (1) single-frame pretraining on large-scale embodied datasets with autoregressive action tokens prediction, which establishes an embodied vision-language foundation; (2) multi-frame encoding, adapting the prediction of vision-language backbones from discrete action tokens to motion features during post-training, and aggregating motion features from historical frames into a feature chunking; (3) cross-frame decoding, which maps the feature chunking to accurate actions via a shared decoder with cross-attention. By reducing redundant token computation and caching past motion features, CronusVLA achieves efficient inference. As an application of motion features, we further propose an action adaptation mechanism based on feature-action retrieval to improve model performance during finetuning. CronusVLA achieves state-of-the-art performance on SimplerEnv with 70.9% success rate, and 12.7% improvement over OpenVLA on LIBERO. Real-world Franka experiments also show the strong performance and robustness.
zh
[CV-11] One Prototype Is Enough: Single-Prototype Activation for Interpretable Image Classification
【速读】：该论文试图解决可解释图像分类中认知复杂度过高以及传统原型网络依赖多个原型协同决策的问题。其解决方案的关键在于提出ProtoSolo架构，该架构通过仅需单个原型激活即可完成分类任务，从而简化了分类决策的解释过程；同时引入基于特征的比较方法，利用特征图进行相似性比较和原型学习，以增强全局信息的利用，并提出非原型投影学习策略，以保持原型与训练图像块之间的信息关联，避免因投影操作导致的网络结构突变及其对分类性能的负面影响。
链接: https://arxiv.org/abs/2506.19808
作者: Yitao Peng,Lianghua He,Die Hu
机构: Tongji University (同济大学); Fudan University (复旦大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:In this paper, we propose ProtoSolo, a novel deep neural architecture for interpretable image classification inspired by prototypical networks such as ProtoPNet. Existing prototype networks usually rely on the collaborative decision-making of multiple prototypes to achieve the classification and interpretation of a single category. In contrast, ProtoSolo only requires the activation of a single prototype to complete the classification. This allows the network to explain each category decision by only providing the features that are most similar to the prototype of that category, significantly reducing the cognitive complexity of the explanation. Secondly, we propose a feature-based comparison method, which uses feature map instead of full-channel feature vector as the object of similarity comparison and prototype learning. This design enables ProtoSolo to utilize richer global information for classification while relying on a single prototype activation. In addition, we propose a non-prototype projection learning strategy, which preserves the information association between the prototype and the training image patches while avoiding the sharp change of the network structure caused by the projection operation, thus avoiding its negative impact on the classification performance. Experiments on the CUB-200-2011 and Stanford Cars datasets show that ProtoSolo achieves superior performance in classification tasks and reaches the best level in terms of cognitive complexity of explanations compared to state-of-the-art interpretable methods. The code is available at this https URL.
zh
[CV-12] CoCo4D: Comprehensive and Complex 4D Scene Generation
【速读】：该论文旨在解决现有4D合成方法在生成多视角一致且沉浸式的动态4D场景时存在的局限性，尤其是对象级生成或动态场景合成中新颖视角有限的问题。其解决方案的关键在于提出一种名为CoCo4D的框架，该框架通过将4D场景合成分为动态前景建模和动态背景生成两个任务，并利用参考运动序列进行指导，从而实现更高质量的场景生成。该方法首先利用视频扩散模型生成初始运动序列，随后通过一种新颖的渐进式外绘方案合成动态前景和背景，并优化前景的参数化轨迹以实现与动态背景的无缝融合。
链接: https://arxiv.org/abs/2506.19798
作者: Junwei Zhou,Xueting Li,Lu Qi,Ming-Hsuan Yang
机构: Huazhong University of Science and Technology (华中科技大学); NVIDIA (英伟达); Wuhan University (武汉大学); Insta360 Research (Insta360 研究院); UC Merced (加州大学默塞德分校)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 16 pages,10 figures
Abstract:Existing 4D synthesis methods primarily focus on object-level generation or dynamic scene synthesis with limited novel views, restricting their ability to generate multi-view consistent and immersive dynamic 4D scenes. To address these constraints, we propose a framework (dubbed as CoCo4D) for generating detailed dynamic 4D scenes from text prompts, with the option to include images. Our method leverages the crucial observation that articulated motion typically characterizes foreground objects, whereas background alterations are less pronounced. Consequently, CoCo4D divides 4D scene synthesis into two responsibilities: modeling the dynamic foreground and creating the evolving background, both directed by a reference motion sequence. Given a text prompt and an optional reference image, CoCo4D first generates an initial motion sequence utilizing video diffusion models. This motion sequence then guides the synthesis of both the dynamic foreground object and the background using a novel progressive outpainting scheme. To ensure seamless integration of the moving foreground object within the dynamic background, CoCo4D optimizes a parametric trajectory for the foreground, resulting in realistic and coherent blending. Extensive experiments show that CoCo4D achieves comparable or superior performance in 4D scene generation compared to existing methods, demonstrating its effectiveness and efficiency. More results are presented on our website this https URL.
zh
[CV-13] Systematic Comparison of Projection Methods for Monocular 3D Human Pose Estimation on Fisheye Images
【速读】：该论文旨在解决在单目鱼眼图像中进行绝对人体姿态估计时，由于鱼眼镜头固有的弯曲畸变导致的人体姿态检测准确性不足的问题。其解决方案的关键在于系统评估不同投影模型（如针孔模型、等距模型、双球模型和圆柱投影方法）对3D人体姿态估计精度的影响，并提出一种基于检测边界框的启发式方法来选择合适的投影模型，以提升预测质量。此外，研究还引入了新的数据集FISHnCHIPS，用于支持在非常规角度下的姿态估计任务。
链接: https://arxiv.org/abs/2506.19747
作者: Stephanie Käs,Sven Peter,Henrik Thillmann,Anton Burenko,David Benjamin Adrian,Dennis Mack,Timm Linder,Bastian Leibe
机构: RWTH Aachen University (亚琛工业大学); Robert Bosch GmbH (罗伯特·博世有限公司)
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注: Presented at IEEE International Conference on Robotics and Automation 2025
Abstract:Fisheye cameras offer robots the ability to capture human movements across a wider field of view (FOV) than standard pinhole cameras, making them particularly useful for applications in human-robot interaction and automotive contexts. However, accurately detecting human poses in fisheye images is challenging due to the curved distortions inherent to fisheye optics. While various methods for undistorting fisheye images have been proposed, their effectiveness and limitations for poses that cover a wide FOV has not been systematically evaluated in the context of absolute human pose estimation from monocular fisheye images. To address this gap, we evaluate the impact of pinhole, equidistant and double sphere camera models, as well as cylindrical projection methods, on 3D human pose estimation accuracy. We find that in close-up scenarios, pinhole projection is inadequate, and the optimal projection method varies with the FOV covered by the human pose. The usage of advanced fisheye models like the double sphere model significantly enhances 3D human pose estimation accuracy. We propose a heuristic for selecting the appropriate projection model based on the detection bounding box to enhance prediction quality. Additionally, we introduce and evaluate on our novel dataset FISHnCHIPS, which features 3D human skeleton annotations in fisheye images, including images from unconventional angles, such as extreme close-ups, ground-mounted cameras, and wide-FOV poses, available at: this https URL
zh
[CV-14] Noise Consistency Training: A Native Approach for One-Step Generator in Learning Additional Controls
【速读】：该论文试图解决高效且可控的高质量内容生成问题，特别是针对基于扩散蒸馏技术的一步生成器在适应新控制条件（如结构约束、语义指导或外部输入）时所面临的挑战。解决方案的关键在于提出一种名为噪声一致性训练（Noise Consistency Training, NCT）的新方法，该方法通过引入适配模块并在生成器的噪声空间中使用噪声一致性损失，直接将新的控制信号整合到预训练的一步生成器中，而无需访问原始训练图像或重新训练基础扩散模型。
链接: https://arxiv.org/abs/2506.19741
作者: Yihong Luo,Shuchen Xue,Tianyang Hu,Jing Tang
机构: HKUST(香港科技大学); UCAS(中国科学院大学); NUS(新加坡国立大学); HKUST(GZ)(香港科技大学（广州)）
类目: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
备注:
Abstract:The pursuit of efficient and controllable high-quality content generation remains a central challenge in artificial intelligence-generated content (AIGC). While one-step generators, enabled by diffusion distillation techniques, offer excellent generation quality and computational efficiency, adapting them to new control conditions–such as structural constraints, semantic guidelines, or external inputs–poses a significant challenge. Conventional approaches often necessitate computationally expensive modifications to the base model and subsequent diffusion distillation. This paper introduces Noise Consistency Training (NCT), a novel and lightweight approach to directly integrate new control signals into pre-trained one-step generators without requiring access to original training images or retraining the base diffusion model. NCT operates by introducing an adapter module and employs a noise consistency loss in the noise space of the generator. This loss aligns the adapted model’s generation behavior across noises that are conditionally dependent to varying degrees, implicitly guiding it to adhere to the new control. Theoretically, this training objective can be understood as minimizing the distributional distance between the adapted generator and the conditional distribution induced by the new conditions. NCT is modular, data-efficient, and easily deployable, relying only on the pre-trained one-step generator and a control signal model. Extensive experiments demonstrate that NCT achieves state-of-the-art controllable generation in a single forward pass, surpassing existing multi-step and distillation-based methods in both generation quality and computational efficiency. Code is available at this https URL
zh
[CV-15] Uncovering Conceptual Blindspots in Generative Image Models Using Sparse Autoencoders
【速读】：该论文试图解决生成式图像模型在训练数据中存在但生成结果中缺失或错误表示的概念盲点问题，即模型未能正确生成一些看似简单却应出现在训练数据中的概念。解决方案的关键在于引入一种系统方法，利用稀疏自编码器（Sparse Autoencoders, SAE）提取可解释的概念嵌入，从而定量比较真实图像与生成图像中概念的出现频率，进而识别和表征这些概念盲点。
链接: https://arxiv.org/abs/2506.19708
作者: Matyas Bohacek,Thomas Fel,Maneesh Agrawala,Ekdeep Singh Lubana
机构: Stanford University (斯坦福大学); Harvard University (哈佛大学)
类目: Graphics (cs.GR); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Despite their impressive performance, generative image models trained on large-scale datasets frequently fail to produce images with seemingly simple concepts – e.g., human hands or objects appearing in groups of four – that are reasonably expected to appear in the training data. These failure modes have largely been documented anecdotally, leaving open the question of whether they reflect idiosyncratic anomalies or more structural limitations of these models. To address this, we introduce a systematic approach for identifying and characterizing “conceptual blindspots” – concepts present in the training data but absent or misrepresented in a model’s generations. Our method leverages sparse autoencoders (SAEs) to extract interpretable concept embeddings, enabling a quantitative comparison of concept prevalence between real and generated images. We train an archetypal SAE (RA-SAE) on DINOv2 features with 32,000 concepts – the largest such SAE to date – enabling fine-grained analysis of conceptual disparities. Applied to four popular generative models (Stable Diffusion 1.5/2.1, PixArt, and Kandinsky), our approach reveals specific suppressed blindspots (e.g., bird feeders, DVD discs, and whitespaces on documents) and exaggerated blindspots (e.g., wood background texture and palm trees). At the individual datapoint level, we further isolate memorization artifacts – instances where models reproduce highly specific visual templates seen during training. Overall, we propose a theoretically grounded framework for systematically identifying conceptual blindspots in generative models by assessing their conceptual fidelity with respect to the underlying data-generating process.
zh
[CV-16] UltraAD: Fine-Grained Ultrasound Anomaly Classification via Few-Shot CLIP Adaptation
【速读】：该论文旨在解决医学图像中精确异常检测的问题，特别是针对超声（US）图像中存在的领域差异以及对细粒度分类（如良性与恶性肿瘤区分）的不足。其解决方案的关键在于提出一种基于视觉-语言模型（VLM）的方法UltraAD，该方法通过少量超声示例实现泛化异常定位和细粒度分类。具体而言，通过将查询视觉原型的图像级标记与可学习文本嵌入融合，再与局部块级标记整合以优化局部表示，并构建一个包含少量样本图像及其对应文本描述的记忆库，从而提升分类性能。
链接: https://arxiv.org/abs/2506.19694
作者: Yue Zhou,Yuan Bi,Wenjuan Tong,Wei Wang,Nassir Navab,Zhongliang Jiang
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Precise anomaly detection in medical images is critical for clinical decision-making. While recent unsupervised or semi-supervised anomaly detection methods trained on large-scale normal data show promising results, they lack fine-grained differentiation, such as benign vs. malignant tumors. Additionally, ultrasound (US) imaging is highly sensitive to devices and acquisition parameter variations, creating significant domain gaps in the resulting US images. To address these challenges, we propose UltraAD, a vision-language model (VLM)-based approach that leverages few-shot US examples for generalized anomaly localization and fine-grained classification. To enhance localization performance, the image-level token of query visual prototypes is first fused with learnable text embeddings. This image-informed prompt feature is then further integrated with patch-level tokens, refining local representations for improved accuracy. For fine-grained classification, a memory bank is constructed from few-shot image samples and corresponding text descriptions that capture anatomical and abnormality-specific features. During training, the stored text embeddings remain frozen, while image features are adapted to better align with medical data. UltraAD has been extensively evaluated on three breast US datasets, outperforming state-of-the-art methods in both lesion localization and fine-grained medical classification. The code will be released upon acceptance.
zh
[CV-17] Semantic Scene Graph for Ultrasound Image Explanation and Scanning Guidance
【速读】：该论文旨在解决医学超声成像中因成像和采集参数差异导致的视觉变异问题，以及非专业用户（如床旁检查场景中的使用者）对超声图像可解释性和基础扫描指导的需求尚未被充分探索的问题。其解决方案的关键在于引入超声场景图（ultrasound scene graph, USG），通过基于Transformer的一阶段方法计算USG，无需显式目标检测，并利用大语言模型（LLMs）根据用户查询进一步优化抽象的SG表示，从而为普通用户提供可理解的图像解释和扫描引导，提升超声检查的标准化与完整性。
链接: https://arxiv.org/abs/2506.19683
作者: Xuesong Li,Dianye Huang,Yameng Zhang,Nassir Navab,Zhongliang Jiang
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
备注:
Abstract:Understanding medical ultrasound imaging remains a long-standing challenge due to significant visual variability caused by differences in imaging and acquisition parameters. Recent advancements in large language models (LLMs) have been used to automatically generate terminology-rich summaries orientated to clinicians with sufficient physiological knowledge. Nevertheless, the increasing demand for improved ultrasound interpretability and basic scanning guidance among non-expert users, e.g., in point-of-care settings, has not yet been explored. In this study, we first introduce the scene graph (SG) for ultrasound images to explain image content to ordinary and provide guidance for ultrasound scanning. The ultrasound SG is first computed using a transformer-based one-stage method, eliminating the need for explicit object detection. To generate a graspable image explanation for ordinary, the user query is then used to further refine the abstract SG representation through LLMs. Additionally, the predicted SG is explored for its potential in guiding ultrasound scanning toward missing anatomies within the current imaging view, assisting ordinary users in achieving more standardized and complete anatomical exploration. The effectiveness of this SG-based image explanation and scanning guidance has been validated on images from the left and right neck regions, including the carotid and thyroid, across five volunteers. The results demonstrate the potential of the method to maximally democratize ultrasound by enhancing its interpretability and usability for ordinaries.
zh
[CV-18] Genome-Anchored Foundation Model Embeddings Improve Molecular Prediction from Histology Images
【速读】：该论文旨在解决在精准肿瘤学中，通过常规全切片图像（Whole-Slide Image, WSI）直接预测复杂分子特征和患者预后这一挑战，因为传统深度学习方法在此方面表现有限。其解决方案的关键在于引入PathLUPI，该方法在训练过程中利用转录组学特权信息（Transcriptomic Privileged Information），提取与基因组锚定的组织学嵌入，从而在推理阶段仅使用WSI即可实现有效的分子预测。
链接: https://arxiv.org/abs/2506.19681
作者: Cheng Jin,Fengtao Zhou,Yunfang Yu,Jiabo Ma,Yihui Wang,Yingxue Xu,Huajun Zhou,Hao Jiang,Luyang Luo,Luhui Mao,Zifan He,Xiuming Zhang,Jing Zhang,Ronald Chan,Herui Yao,Hao Chen
机构: The Hong Kong University of Science and Technology (香港科技大学); Sun Yat-sen University (中山大学); Guangdong-Hong Kong Joint Laboratory for RNA Medicine (广东省-香港RNA医学联合实验室); Memorial Hospital of Sun Yat-sen University (中山大学附属肿瘤医院); Shenshan Medical Center (深汕医疗中心); Macau University of Science and Technology (澳门科技大学); Jinan University (暨南大学); Harvard University (哈佛大学); Zhejiang University (浙江大学); The Chinese University of Hong Kong (香港中文大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Under Review
Abstract:Precision oncology requires accurate molecular insights, yet obtaining these directly from genomics is costly and time-consuming for broad clinical use. Predicting complex molecular features and patient prognosis directly from routine whole-slide images (WSI) remains a major challenge for current deep learning methods. Here we introduce PathLUPI, which uses transcriptomic privileged information during training to extract genome-anchored histological embeddings, enabling effective molecular prediction using only WSIs at inference. Through extensive evaluation across 49 molecular oncology tasks using 11,257 cases among 20 cohorts, PathLUPI demonstrated superior performance compared to conventional methods trained solely on WSIs. Crucially, it achieves AUC \geq 0.80 in 14 of the biomarker prediction and molecular subtyping tasks and C-index \geq 0.70 in survival cohorts of 5 major cancer types. Moreover, PathLUPI embeddings reveal distinct cellular morphological signatures associated with specific genotypes and related biological pathways within WSIs. By effectively encoding molecular context to refine WSI representations, PathLUPI overcomes a key limitation of existing models and offers a novel strategy to bridge molecular insights with routine pathology workflows for wider clinical application.
zh
[CV-19] SAM2-SGP: Enhancing SAM2 for Medical Image Segmentation via Support-Set Guided Prompting
【速读】：该论文旨在解决生成式 AI（Generative AI）在医学图像分割任务中面临的两个关键问题：一是依赖人工提供的提示信息导致的适应性不足，二是由于模型最初在自然图像和视频上训练而产生的领域偏移（domain shift）问题。解决方案的关键在于提出一种无需人工提示的框架 SAM2-SGP，该框架通过支持集引导提示机制，利用 SAM2 的记忆机制结合伪掩码生成（Pseudo-mask Generation, PMG）模块，自动生成伪掩码，并引入伪掩码注意力（Pseudo-mask Attention, PMA）模块以提升局部特征提取能力，同时采用低秩适配（LoRA）策略缓解领域偏移问题。
链接: https://arxiv.org/abs/2506.19658
作者: Yang Xing,Jiong Wu,Yuheng Bu,Kuang Gong
机构: University of Florida (佛罗里达大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Although new vision foundation models such as Segment Anything Model 2 (SAM2) have significantly enhanced zero-shot image segmentation capabilities, reliance on human-provided prompts poses significant challenges in adapting SAM2 to medical image segmentation tasks. Moreover, SAM2’s performance in medical image segmentation was limited by the domain shift issue, since it was originally trained on natural images and videos. To address these challenges, we proposed SAM2 with support-set guided prompting (SAM2-SGP), a framework that eliminated the need for manual prompts. The proposed model leveraged the memory mechanism of SAM2 to generate pseudo-masks using image-mask pairs from a support set via a Pseudo-mask Generation (PMG) module. We further introduced a novel Pseudo-mask Attention (PMA) module, which used these pseudo-masks to automatically generate bounding boxes and enhance localized feature extraction by guiding attention to relevant areas. Furthermore, a low-rank adaptation (LoRA) strategy was adopted to mitigate the domain shift issue. The proposed framework was evaluated on both 2D and 3D datasets across multiple medical imaging modalities, including fundus photography, X-ray, computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and ultrasound. The results demonstrated a significant performance improvement over state-of-the-art models, such as nnUNet and SwinUNet, as well as foundation models, such as SAM2 and MedSAM2, underscoring the effectiveness of the proposed approach. Our code is publicly available at this https URL.
zh
[CV-20] Video Compression for Spatiotemporal Earth System Data
【速读】：该论文试图解决地球观测数据集规模迅速增长带来的存储和传输挑战，其核心问题是如何高效地压缩多通道时空数据而不显著损失数据质量。解决方案的关键在于利用现有的视频压缩技术，通过将地球系统数据集编码为视频格式进行压缩，从而有效利用标准且优化的视频编解码器（如ffmpeg）实现高达250倍的压缩比，同时保持高保真度。
链接: https://arxiv.org/abs/2506.19656
作者: Oscar J. Pellicer-Valero,Cesar Aybar,Gustau Camps Valls
机构: Image Processing Laboratory, Universitat de València (图像处理实验室，瓦伦西亚大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Digital Libraries (cs.DL); Image and Video Processing (eess.IV); Geophysics (physics.geo-ph)
备注:
Abstract:Large-scale Earth system datasets, from high-resolution remote sensing imagery to spatiotemporal climate model outputs, exhibit characteristics analogous to those of standard videos. Their inherent spatial, temporal, and spectral redundancies can thus be readily exploited by established video compression techniques. Here, we present xarrayvideo, a Python library for compressing multichannel spatiotemporal datasets by encoding them as videos. Our approach achieves compression ratios of up to 250x while maintaining high fidelity by leveraging standard, well-optimized video codecs through ffmpeg. We demonstrate the library’s effectiveness on four real-world multichannel spatiotemporal datasets: DynamicEarthNet (very high resolution Planet images), DeepExtremeCubes (high resolution Sentinel-2 images), ERA5 (weather reanalysis data), and the SimpleS2 dataset (high resolution multichannel Sentinel-2 images), achieving Peak Signal-to-Noise Ratios (PSNRs) of 55.86, 40.60, 46.58, and 43.23 dB at 0.1 bits per pixel per band (bpppb) and 65.91, 54.28, 62.90, and 55.04 dB at 1 bpppb. We are redistributing two of these datasets, DeepExtremeCubes (2.3 Tb) and DynamicEarthNet (525 Gb), in the machine-learning-ready and cloud-ready TACO format through HuggingFace at significantly reduced sizes (270 Gb and 8.5 Gb, respectively) without compromising quality (PSNR 55.77-56.65 and 60.15). No performance loss is observed when the compressed versions of these datasets are used in their respective deep learning-based downstream tasks (next step reflectance prediction and landcover segmentation). In conclusion, xarrayvideo presents an efficient solution for handling the rapidly growing size of Earth observation datasets, making advanced compression techniques accessible and practical to the Earth science community. The library is available for use at this https URL
zh
[CV-21] PEVLM: Parallel Encoding for Vision-Language Models
【速读】：该论文旨在解决视觉-语言模型（Vision-Language Models, VLMs）在长视频理解任务中因标准注意力机制的二次复杂度而受到的性能限制问题。其解决方案的关键在于提出一种并行编码策略（Parallel Encoding Strategy, PEVLM），通过将输入划分为块状片段并共享一个sink，保留全注意力位置嵌入，并对齐注意力权重以模拟全注意力分布，从而将注意力计算复杂度从O((T × N)^2)降低至O(T × N)，同时保持高精度。
链接: https://arxiv.org/abs/2506.19651
作者: Letian Kang,Shixian Luo,Yiqiang Li,Xiaoyang Yu,Shenxuan Zhou,Yong Wu
机构: Li Auto Inc. (理想汽车)
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Performance (cs.PF)
备注:
Abstract:Vision-Language Models (VLMs) have demonstrated strong performance in video-language tasks, yet their application to long video understanding remains constrained by the quadratic complexity of standard attention mechanisms. In this paper, we propose \textbfPEVLM, a parallel encoding strategy specifically designed to improve the prefill efficiency of VLMs without requiring model finetuning. PEVLM partitions the input into block-wise segments with a shared sink, preserves full-attention positional embeddings, and aligns attention weights to mimic full-attention distributions. This design reduces attention computation from O((T \times N)^2) to O(T \times N) while maintaining high accuracy. Extensive experiments on the LongVideoBench benchmark show that PEVLM achieves up to 8.37% accuracy improvement over existing inference-efficient methods and delivers up to 7.47x speedup in attention computation and 40% reduction in end-to-end latency. Under strict latency constraints, PEVLM significantly outperforms baselines, raising accuracy from 23.26% to 61.03%. These results highlight PEVLM’s effectiveness for low-latency, long-context video understanding, making it well-suited for real-world applications such as autonomous driving.
zh
[CV-22] HOIverse: A Synthetic Scene Graph Dataset With Human Object Interactions
【速读】：该论文试图解决在室内环境中，人类与机器人代理共存时，如何实现准确的场景理解问题，特别是针对人类定位和动作识别的需求。当前缺乏可靠的基准数据集来支持此类场景理解任务。解决方案的关键在于提出HOIverse，这是一个结合场景图与人-物交互的合成数据集，包含精确且密集的人类与周围物体之间的关系标注，以及对应的RGB图像、分割掩码、深度图像和人体关键点信息，从而为场景理解研究提供高质量的数据支持。
链接: https://arxiv.org/abs/2506.19639
作者: Mrunmai Vivek Phatak,Julian Lorenz,Nico Hörmann,Jörg Hähner,Rainer Lienhart
机构: Universität Augsburg(奥格斯堡大学); Machine Learning and Computer Vision Lab(机器学习与计算机视觉实验室); Organic Computing Lab(有机计算实验室)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:When humans and robotic agents coexist in an environment, scene understanding becomes crucial for the agents to carry out various downstream tasks like navigation and planning. Hence, an agent must be capable of localizing and identifying actions performed by the human. Current research lacks reliable datasets for performing scene understanding within indoor environments where humans are also a part of the scene. Scene Graphs enable us to generate a structured representation of a scene or an image to perform visual scene understanding. To tackle this, we present HOIverse a synthetic dataset at the intersection of scene graph and human-object interaction, consisting of accurate and dense relationship ground truths between humans and surrounding objects along with corresponding RGB images, segmentation masks, depth images and human keypoints. We compute parametric relations between various pairs of objects and human-object pairs, resulting in an accurate and unambiguous relation definitions. In addition, we benchmark our dataset on state-of-the-art scene graph generation models to predict parametric relations and human-object interactions. Through this dataset, we aim to accelerate research in the field of scene understanding involving people.
zh
[CV-23] VideoPCDNet: Video Parsing and Prediction with Phase Correlation Networks ICANN2025
【速读】：该论文旨在解决动态环境中视频内容的理解与预测问题，特别是无监督学习对象表示和动态变化的挑战。其解决方案的关键在于提出VideoPCDNet框架，该框架利用频域相位相关技术递归地将视频解析为对象组件，并通过学习到的对象原型变换表示实现准确且可解释的跟踪；同时，通过结合频域操作和轻量级学习模块显式建模对象运动，从而实现无监督的对象跟踪和未来视频帧的预测。
链接: https://arxiv.org/abs/2506.19621
作者: Noel José Rodrigues Vicente,Enrique Lehner,Angel Villar-Corrales,Jan Nogga,Sven Behnke
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: Accepted for Publication at ICANN 2025
Abstract:Understanding and predicting video content is essential for planning and reasoning in dynamic environments. Despite advancements, unsupervised learning of object representations and dynamics remains challenging. We present VideoPCDNet, an unsupervised framework for object-centric video decomposition and prediction. Our model uses frequency-domain phase correlation techniques to recursively parse videos into object components, which are represented as transformed versions of learned object prototypes, enabling accurate and interpretable tracking. By explicitly modeling object motion through a combination of frequency domain operations and lightweight learned modules, VideoPCDNet enables accurate unsupervised object tracking and prediction of future video frames. In our experiments, we demonstrate that VideoPCDNet outperforms multiple object-centric baseline models for unsupervised tracking and prediction on several synthetic datasets, while learning interpretable object and motion representations.
zh
[CV-24] Self-Supervised Multimodal NeRF for Autonomous Driving
【速读】：该论文旨在解决多模态动态场景中新颖视角合成的问题，特别是在自动驾驶场景下同时处理LiDAR和Camera数据的时空隐式表示学习。其解决方案的关键在于提出一种自监督的Neural Radiance Fields (NeRF)框架，通过联合学习空间和时间变化的场景表示，无需依赖3D标签即可进行训练。此外，为提高训练效率和收敛速度，引入了基于启发式的图像像素采样策略，并采用双梯度掩码以保留LiDAR点的局部特征。
链接: https://arxiv.org/abs/2506.19615
作者: Gaurav Sharma,Ravi Kothari,Josef Schmid
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:In this paper, we propose a Neural Radiance Fields (NeRF) based framework, referred to as Novel View Synthesis Framework (NVSF). It jointly learns the implicit neural representation of space and time-varying scene for both LiDAR and Camera. We test this on a real-world autonomous driving scenario containing both static and dynamic scenes. Compared to existing multimodal dynamic NeRFs, our framework is self-supervised, thus eliminating the need for 3D labels. For efficient training and faster convergence, we introduce heuristic-based image pixel sampling to focus on pixels with rich information. To preserve the local features of LiDAR points, a Double Gradient based mask is employed. Extensive experiments on the KITTI-360 dataset show that, compared to the baseline models, our framework has reported best performance on both LiDAR and Camera domain. Code of the model is available at this https URL
zh
[CV-25] Implementing blind navigation through multi-modal sensing and gait guidance
【速读】：该论文试图解决视力障碍者在导航和避障过程中遇到的困难，传统辅助工具如导盲杖和导盲犬存在一定的局限性。论文提出的解决方案是开发一种可穿戴的盲人引导设备，其关键在于采用基于步态的引导系统（Gait-based Guiding System），通过步态阶段分析进行行走引导，并结合多模态传感技术获取环境信息，从而实现更有效的盲人导航辅助。
链接: https://arxiv.org/abs/2506.19593
作者: Feifan Yan,Tianle Zeng,Meixi He
机构: China University of Mining and Technology - Beijing (中国矿业大学（北京）)
类目: Computer Vision and Pattern Recognition (cs.CV); Systems and Control (eess.SY)
备注:
Abstract:By the year 2023, the global population of individuals with impaired vision has surpassed 220 million. People with impaired vision will find it difficult while finding path or avoiding obstacles, and must ask for auxiliary tools for help. Although traditional aids such as guide canes and guide dogs exist, they still have some shortcomings. In this paper, we present our wearable blind guiding device, what perform navigation guidance through our proposed Gait-based Guiding System. Our device innovatively integrates gait phase analysis for walking guide, and in terms of environmental perception, we use multimodal sensing to acquire diverse environment information. During the experiment, we conducted both indoor and outdoor experiments, and compared with the standard guide cane. The result shows superior performance of our device in blind guidance.
zh
[CV-26] Vision Transformer-Based Time-Series Image Reconstruction for Cloud-Filling Applications
【速读】：该论文试图解决多光谱影像（MSI）中云覆盖导致的早期作物制图难题，因为云层会引发光谱信息缺失或损坏。解决方案的关键在于提出一种基于视觉Transformer（ViT）的时间序列MSI图像重建框架，通过利用MSI的时间一致性以及SAR数据的互补信息，结合注意力机制实现对云覆盖区域的MSI数据重建。实验结果表明，该框架在重建效果上显著优于不使用时间序列或未融合SAR数据的基线方法。
链接: https://arxiv.org/abs/2506.19591
作者: Lujun Li,Yiqun Wang,Radu State
机构: University of Luxembourg (卢森堡大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
备注: This paper has been accepted as a conference paper at the 2025 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)
Abstract:Cloud cover in multispectral imagery (MSI) poses significant challenges for early season crop mapping, as it leads to missing or corrupted spectral information. Synthetic aperture radar (SAR) data, which is not affected by cloud interference, offers a complementary solution, but lack sufficient spectral detail for precise crop mapping. To address this, we propose a novel framework, Time-series MSI Image Reconstruction using Vision Transformer (ViT), to reconstruct MSI data in cloud-covered regions by leveraging the temporal coherence of MSI and the complementary information from SAR from the attention mechanism. Comprehensive experiments, using rigorous reconstruction evaluation metrics, demonstrate that Time-series ViT framework significantly outperforms baselines that use non-time-series MSI and SAR or time-series MSI without SAR, effectively enhancing MSI image reconstruction in cloud-covered regions.
zh
[CV-27] SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images
【速读】：该论文试图解决多源遥感（Remote Sensing, RS）数据处理中模型适应性差的问题，即现有深度学习模型通常针对单一传感器或固定组合进行设计，难以扩展到多种RS传感器。解决方案的关键在于提出SMARTIES，一个通用且灵活的基础模型，通过将异构传感器数据投影到共享的谱感知空间，实现对任意波段组合的训练和推理支持，并通过统一的Transformer模型进行跨传感器token混洗的掩码数据重建，从而获得传感器无关的特征表示。
链接: https://arxiv.org/abs/2506.19585
作者: Gencer Sumbul,Chang Xu,Emanuele Dalsasso,Devis Tuia
机构: Ecole Polytechnique Fédérale de Lausanne (EPFL)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:From optical sensors to microwave radars, leveraging the complementary strengths of remote sensing (RS) sensors is crucial for achieving dense spatio-temporal monitoring of our planet. In contrast, recent deep learning models, whether task-specific or foundational, are often specific to single sensors or to fixed combinations: adapting such models to different sensory inputs requires both architectural changes and re-training, limiting scalability and generalization across multiple RS sensors. On the contrary, a single model able to modulate its feature representations to accept diverse sensors as input would pave the way to agile and flexible multi-sensor RS data processing. To address this, we introduce SMARTIES, a generic and versatile foundation model lifting sensor-specific/dependent efforts and enabling scalability and generalization to diverse RS sensors: SMARTIES projects data from heterogeneous sensors into a shared spectrum-aware space, enabling the use of arbitrary combinations of bands both for training and inference. To obtain sensor-agnostic representations, we train a single, unified transformer model reconstructing masked multi-sensor data with cross-sensor token mixup. On both single- and multi-modal tasks across diverse sensors, SMARTIES outperforms previous models that rely on sensor-specific pretraining. Our code and pretrained models are available at this https URL.
zh
[CV-28] MambaOutRS: A Hybrid CNN-Fourier Architecture for Remote Sensing Image Classification
【速读】：该论文试图解决传统状态空间模型（State Space Models, SSMs）在适应二维视觉数据时所需的复杂修改导致效率下降的问题。其解决方案的关键在于提出一种新型混合卷积架构MambaOutRS，该架构结合了堆叠的门控卷积块进行局部特征提取，并引入了傅里叶滤波门（Fourier Filter Gate, FFG）模块，在频域中高效捕捉全局上下文信息。通过四阶段分层设计，MambaOutRS在多个遥感图像分类数据集上实现了最先进的性能，证明了其在保持高精度的同时显著降低参数量的可行性。
链接: https://arxiv.org/abs/2506.19561
作者: Minjong Cheon,Changbae Mun
机构: KAIST Applied Science Research Institute (KAIST应用科学研究所); Hanyang Cyber University (韩阳网络大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Recent advances in deep learning for vision tasks have seen the rise of State Space Models (SSMs) like Mamba, celebrated for their linear scalability. However, their adaptation to 2D visual data often necessitates complex modifications that may diminish efficiency. In this paper, we introduce MambaOutRS, a novel hybrid convolutional architecture for remote sensing image classification that re-evaluates the necessity of recurrent SSMs. MambaOutRS builds upon stacked Gated CNN blocks for local feature extraction and introduces a novel Fourier Filter Gate (FFG) module that operates in the frequency domain to capture global contextual information efficiently. Our architecture employs a four-stage hierarchical design and was extensively evaluated on challenging remote sensing datasets: UC Merced, AID, NWPU-RESISC45, and EuroSAT. MambaOutRS consistently achieved state-of-the-art (SOTA) performance across these benchmarks. Notably, our MambaOutRS-t variant (24.0M parameters) attained the highest F1-scores of 98.41% on UC Merced and 95.99% on AID, significantly outperforming existing baselines, including larger transformer models and Mamba-based architectures, despite using considerably fewer parameters. An ablation study conclusively demonstrates the critical role of the Fourier Filter Gate in enhancing the model’s ability to capture global spatial patterns, leading to robust and accurate classification. These results strongly suggest that the complexities of recurrent SSMs can be effectively superseded by a judicious combination of gated convolutions for spatial mixing and frequency-based gates for spectral global context. Thus, MambaOutRS provides a compelling and efficient paradigm for developing high-performance deep learning models in remote sensing and other vision domains, particularly where computational efficiency is paramount.
zh
[CV-29] ConCM: Consistency-Driven Calibration and Matching for Few-Shot Class-Incremental Learning
【速读】：该论文旨在解决少样本类增量学习（Few-Shot Class-Incremental Learning, FSCIL）中模型在有限监督下适应新类别同时保持已学知识的问题。现有方法通过预留空间来容纳新类别，但原型偏差和结构固定限制了嵌入空间的表达能力。论文提出的解决方案关键在于优化特征-结构双重一致性，构建了一种基于一致性的校准与匹配框架（Consistency-driven Calibration and Matching Framework, ConCM），通过记忆感知的原型校准和动态结构匹配，提升特征概念中心的一致性并确保跨会话结构一致性，从而系统性地缓解FSCIL中的知识冲突问题。
链接: https://arxiv.org/abs/2506.19558
作者: QinZhe Wang,Zixuan Chen,Keke Huang,Xiu Su,Chunhua Yang,Chang Xu
机构: School of Automation, Central South University, China; Big Data Institute, Central South University, China; School of Computer Science, Faculty of Engineering, The University of Sydney, Australia
类目: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
备注: 9 pages, 5 figures(Excluding the appendix)
Abstract:Few-Shot Class-Incremental Learning (FSCIL) requires models to adapt to novel classes with limited supervision while preserving learned knowledge. Existing prospective learning-based space construction methods reserve space to accommodate novel classes. However, prototype deviation and structure fixity limit the expressiveness of the embedding space. In contrast to fixed space reservation, we explore the optimization of feature-structure dual consistency and propose a Consistency-driven Calibration and Matching Framework (ConCM) that systematically mitigate the knowledge conflict inherent in FSCIL. Specifically, inspired by hippocampal associative memory, we design a memory-aware prototype calibration that extracts generalized semantic attributes from base classes and reintegrates them into novel classes to enhance the conceptual center consistency of features. Further, we propose dynamic structure matching, which adaptively aligns the calibrated features to a session-specific optimal manifold space, ensuring cross-session structure consistency. Theoretical analysis shows that our method satisfies both geometric optimality and maximum matching, thereby overcoming the need for class-number priors. On large-scale FSCIL benchmarks including mini-ImageNet and CUB200, ConCM achieves state-of-the-art performance, surpassing current optimal method by 3.20% and 3.68% in harmonic accuracy of incremental sessions.
zh
[CV-30] General Methods Make Great Domain-specific Foundation Models: A Case-study on Fetal Ultrasound MICCAI2025
【速读】：该论文试图解决在医疗领域中，是否应针对特定医学数据预训练定制的基础模型，还是采用从现有通用模型进行迁移学习的问题，以及在预训练定制模型时是否需要新颖的方法。其解决方案的关键在于通过在大规模区域性胎儿超声图像数据集上使用已有的DINOv2方法进行预训练，验证了在医学领域定制基础模型的有效性，并表明经过良好调优的计算机视觉方法可以在无需大量超参数调整和方法适应的情况下，成功训练出适用于特定医学领域的基础模型。
链接: https://arxiv.org/abs/2506.19552
作者: Jakob Ambsdorf,Asbjørn Munk,Sebastian Llambias,Anders Nymark Christensen,Kamil Mikolaj,Randall Balestriero,Martin Tolsgaard,Aasa Feragen,Mads Nielsen
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Submitted version of paper accepted at MICCAI 2025
Abstract:With access to large-scale, unlabeled medical datasets, researchers are confronted with two questions: Should they attempt to pretrain a custom foundation model on this medical data, or use transfer-learning from an existing generalist model? And, if a custom model is pretrained, are novel methods required? In this paper we explore these questions by conducting a case-study, in which we train a foundation model on a large regional fetal ultrasound dataset of 2M images. By selecting the well-established DINOv2 method for pretraining, we achieve state-of-the-art results on three fetal ultrasound datasets, covering data from different countries, classification, segmentation, and few-shot tasks. We compare against a series of models pretrained on natural images, ultrasound images, and supervised baselines. Our results demonstrate two key insights: (i) Pretraining on custom data is worth it, even if smaller models are trained on less data, as scaling in natural image pretraining does not translate to ultrasound performance. (ii) Well-tuned methods from computer vision are making it feasible to train custom foundation models for a given medical domain, requiring no hyperparameter tuning and little methodological adaptation. Given these findings, we argue that a bias towards methodological innovation should be avoided when developing domain specific foundation models under common computational resource constraints.
zh
[CV-31] Identifying Physically Realizable Triggers for Backdoored Face Recognition Networks ICIP2021
【速读】：该论文旨在解决深度神经网络中嵌入的后门攻击问题，特别是针对人脸识别（Face Recognition, FR）系统中由自然且物理可实现的触发器引发的安全威胁。其解决方案的关键在于提出一种新颖的技术，能够检测FR网络是否被植入了此类触发器，并在已知受感染网络的情况下识别出具体的触发器，如绿色太阳镜或红色帽子，该方法在实验中达到了74%的Top-5准确率，显著优于基线方法的56%。
链接: https://arxiv.org/abs/2506.19533
作者: Ankita Raj,Ambar Pal,Chetan Arora
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
备注: Accepted to ICIP 2021
Abstract:Backdoor attacks embed a hidden functionality into deep neural networks, causing the network to display anomalous behavior when activated by a predetermined pattern in the input Trigger, while behaving well otherwise on public test data. Recent works have shown that backdoored face recognition (FR) systems can respond to natural-looking triggers like a particular pair of sunglasses. Such attacks pose a serious threat to the applicability of FR systems in high-security applications. We propose a novel technique to (1) detect whether an FR network is compromised with a natural, physically realizable trigger, and (2) identify such triggers given a compromised network. We demonstrate the effectiveness of our methods with a compromised FR network, where we are able to identify the trigger (e.g., green sunglasses or red hat) with a top-5 accuracy of 74%, whereas a naive brute force baseline achieves 56% accuracy.
zh
[CV-32] ReMAR-DS: Recalibrated Feature Learning for Metal Artifact Reduction and CT Domain Transformation
【速读】：该论文旨在解决千伏级计算机断层扫描（kVCT）成像中的伪影问题，该问题会降低图像质量并影响临床决策。其解决方案的关键在于提出一种基于深度学习的框架ReMAR-DS，该框架采用带有增强特征重新校准的编码器-解码器结构，通过注入校准后的特征来聚焦于相关空间区域和关键特征，从而有效减少伪影并保留解剖结构，实现从kVCT到兆伏级CT（MVCT）的域转换，提升放疗计划的准确性。
链接: https://arxiv.org/abs/2506.19531
作者: Mubashara Rehman,Niki Martinel,Michele Avanzo,Riccardo Spizzo,Christian Micheloni
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: Accepted in 23rd International Conference on Image Analysis and Processing (ICIAP) 2025, Italy
Abstract:Artifacts in kilo-Voltage CT (kVCT) imaging degrade image quality, impacting clinical decisions. We propose a deep learning framework for metal artifact reduction (MAR) and domain transformation from kVCT to Mega-Voltage CT (MVCT). The proposed framework, ReMAR-DS, utilizes an encoder-decoder architecture with enhanced feature recalibration, effectively reducing artifacts while preserving anatomical structures. This ensures that only relevant information is utilized in the reconstruction process. By infusing recalibrated features from the encoder block, the model focuses on relevant spatial regions (e.g., areas with artifacts) and highlights key features across channels (e.g., anatomical structures), leading to improved reconstruction of artifact-corrupted regions. Unlike traditional MAR methods, our approach bridges the gap between high-resolution kVCT and artifact-resistant MVCT, enhancing radiotherapy planning. It produces high-quality MVCT-like reconstructions, validated through qualitative and quantitative evaluations. Clinically, this enables oncologists to rely on kVCT alone, reducing repeated high-dose MVCT scans and lowering radiation exposure for cancer patients.
zh
[CV-33] Visual hallucination detection in large vision-language models via evidential conflict
【速读】：该论文旨在解决大型视觉语言模型（LVLM）中出现的视觉幻觉问题，即视觉输入与文本输出之间的不一致性，这对安全关键型人工智能应用构成了重大风险。其解决方案的关键在于提出一种基于Dempster-Shafer理论（DST）的视觉幻觉检测方法，通过不确定性估计来高效捕捉模型推理阶段高层特征中的冲突程度，并采用简单的质量函数以降低幂集证据组合的计算复杂度。
链接: https://arxiv.org/abs/2506.19513
作者: Tao Huang,Zhekun Liu,Rui Wang,Yang Zhang,Liping Jing
机构: Beijing Key Lab of Traffic Data Mining and Embodied Intelligence, Beijing, China; State Key Laboratory of Advanced Rail Autonomous Operation, Beijing, China; School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China; School of Automation and Intelligence, Beijing Jiaotong University, Beijing, China; School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing, China
类目: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Despite the remarkable multimodal capabilities of Large Vision-Language Models (LVLMs), discrepancies often occur between visual inputs and textual outputs–a phenomenon we term visual hallucination. This critical reliability gap poses substantial risks in safety-critical Artificial Intelligence (AI) applications, necessitating a comprehensive evaluation benchmark and effective detection methods. Firstly, we observe that existing visual-centric hallucination benchmarks mainly assess LVLMs from a perception perspective, overlooking hallucinations arising from advanced reasoning capabilities. We develop the Perception-Reasoning Evaluation Hallucination (PRE-HAL) dataset, which enables the systematic evaluation of both perception and reasoning capabilities of LVLMs across multiple visual semantics, such as instances, scenes, and relations. Comprehensive evaluation with this new benchmark exposed more visual vulnerabilities, particularly in the more challenging task of relation reasoning. To address this issue, we propose, to the best of our knowledge, the first Dempster-Shafer theory (DST)-based visual hallucination detection method for LVLMs through uncertainty estimation. This method aims to efficiently capture the degree of conflict in high-level features at the model inference phase. Specifically, our approach employs simple mass functions to mitigate the computational complexity of evidence combination on power sets. We conduct an extensive evaluation of state-of-the-art LVLMs, LLaVA-v1.5, mPLUG-Owl2 and mPLUG-Owl3, with the new PRE-HAL benchmark. Experimental results indicate that our method outperforms five baseline uncertainty metrics, achieving average AUROC improvements of 4%, 10%, and 7% across three LVLMs. Our code is available at this https URL.
zh
[CV-34] Experimental Assessment of Neural 3D Reconstruction for Small UAV-based Applications
【速读】：该论文旨在解决小型无人机（Small UAVs）在室内和难以到达区域执行任务时，由于飞行动力学和能耗限制导致的自主性和任务能力不足的问题。其解决方案的关键在于将神经三维重建（Neural 3D Reconstruction, N3DR）与小型无人机系统相结合，通过集成先进的重建模型（如Instant-ngp、Nerfacto和Splatfacto），利用无人机群采集的图像提升对小型静态物体的精细三维数字重建质量，从而实现高精度三维地图构建和异常检测。
链接: https://arxiv.org/abs/2506.19491
作者: Genís Castillo Gómez-Raya,Álmos Veres-Vitályos,Filip Lemic,Pablo Royo,Mario Montagud,Sergi Fernández,Sergi Abadal,Xavier Costa-Pérez
机构: i2CAT Foundation (i2CAT 基金会); University of Zagreb (萨格勒布大学); Universitat Politècnica de Catalunya (加泰罗尼亚理工大学); NEC Laboratories Europe GmbH (NEC 欧洲实验室有限公司); ICREA (加泰罗尼亚研究与高级学习 institute)
类目: Emerging Technologies (cs.ET); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Networking and Internet Architecture (cs.NI); Image and Video Processing (eess.IV)
备注: 6 pages, 7 figures, 2 tables, accepted at IEEE International Symposium on Personal, Indoor and Mobile Radio Communications 2025
Abstract:The increasing miniaturization of Unmanned Aerial Vehicles (UAVs) has expanded their deployment potential to indoor and hard-to-reach areas. However, this trend introduces distinct challenges, particularly in terms of flight dynamics and power consumption, which limit the UAVs’ autonomy and mission capabilities. This paper presents a novel approach to overcoming these limitations by integrating Neural 3D Reconstruction (N3DR) with small UAV systems for fine-grained 3-Dimensional (3D) digital reconstruction of small static objects. Specifically, we design, implement, and evaluate an N3DR-based pipeline that leverages advanced models, i.e., Instant-ngp, Nerfacto, and Splatfacto, to improve the quality of 3D reconstructions using images of the object captured by a fleet of small UAVs. We assess the performance of the considered models using various imagery and pointcloud metrics, comparing them against the baseline Structure from Motion (SfM) algorithm. The experimental results demonstrate that the N3DR-enhanced pipeline significantly improves reconstruction quality, making it feasible for small UAVs to support high-precision 3D mapping and anomaly detection in constrained environments. In more general terms, our results highlight the potential of N3DR in advancing the capabilities of miniaturized UAV systems.
zh
[CV-35] SceneCrafter: Controllable Multi-View Driving Scene Editing CVPR2025
【速读】：该论文旨在解决自动驾驶系统仿真中生成场景缺乏现实基础以及难以保证跨摄像头3D一致性、空旷街道先验学习和编辑条件下的成对图像数据生成等问题。其解决方案的关键在于提出SceneCrafter，这是一个用于驾驶场景的多功能编辑器，基于多视角扩散模型构建，能够实现高度真实的3D一致性操作，并通过Prompt-to-Prompt框架生成几何一致的合成成对数据，结合α混合框架与新型掩码训练和多视角重绘范式，有效提升编辑效果与场景质量。
链接: https://arxiv.org/abs/2506.19488
作者: Zehao Zhu,Yuliang Zou,Chiyu Max Jiang,Bo Sun,Vincent Casser,Xiukun Huang,Jiahao Wang,Zhenpei Yang,Ruiqi Gao,Leonidas Guibas,Mingxing Tan,Dragomir Anguelov
机构: Waymo(威马); University of Texas at Austin(德克萨斯大学奥斯汀分校); Johns Hopkins University(约翰霍普金斯大学); Google DeepMind(谷歌深度思维)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: CVPR 2025
Abstract:Simulation is crucial for developing and evaluating autonomous vehicle (AV) systems. Recent literature builds on a new generation of generative models to synthesize highly realistic images for full-stack simulation. However, purely synthetically generated scenes are not grounded in reality and have difficulty in inspiring confidence in the relevance of its outcomes. Editing models, on the other hand, leverage source scenes from real driving logs, and enable the simulation of different traffic layouts, behaviors, and operating conditions such as weather and time of day. While image editing is an established topic in computer vision, it presents fresh sets of challenges in driving simulation: (1) the need for cross-camera 3D consistency, (2) learning ``empty street" priors from driving data with foreground occlusions, and (3) obtaining paired image tuples of varied editing conditions while preserving consistent layout and geometry. To address these challenges, we propose SceneCrafter, a versatile editor for realistic 3D-consistent manipulation of driving scenes captured from multiple cameras. We build on recent advancements in multi-view diffusion models, using a fully controllable framework that scales seamlessly to multi-modality conditions like weather, time of day, agent boxes and high-definition maps. To generate paired data for supervising the editing model, we propose a novel framework on top of Prompt-to-Prompt to generate geometrically consistent synthetic paired data with global edits. We also introduce an alpha-blending framework to synthesize data with local edits, leveraging a model trained on empty street priors through novel masked training and multi-view repaint paradigm. SceneCrafter demonstrates powerful editing capabilities and achieves state-of-the-art realism, controllability, 3D consistency, and scene editing quality compared to existing baselines.
zh
[CV-36] HMSViT: A Hierarchical Masked Self-Supervised Vision Transformer for Corneal Nerve Segmentation and Diabetic Neuropathy Diagnosis
【速读】：该论文旨在解决糖尿病周围神经病变（Diabetic Peripheral Neuropathy, DPN）早期检测中基于角膜共聚焦显微镜（Corneal Confocal Microscopy, CCM）的自动化方法存在的特征提取效率低、依赖人工先验和数据受限等问题。其解决方案的关键在于提出一种新型的分层掩码自监督视觉变压器（Hierarchical Masked Self-Supervised Vision Transformer, HMSViT），通过基于池化的分层结构和双注意力机制，结合绝对位置编码，实现高效多尺度特征提取，并采用块掩码自监督学习框架减少对标注数据的依赖，同时利用多尺度解码器融合分层特征以提升分割与分类性能。
链接: https://arxiv.org/abs/2506.19474
作者: Xin Zhang,Liangxiu Han,Yue Shi,Yanlin Zheng,Alam Uazman,Maryam Ferdousi,Rayaz Malik
机构: Manchester Metropolitan University (曼彻斯特都市大学); University of Liverpool (利物浦大学); University of Manchester (曼彻斯特大学); Weill Cornell Medicine-Qatar (魏尔康奈尔医学学院卡塔尔)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Diabetic Peripheral Neuropathy (DPN) affects nearly half of diabetes patients, requiring early detection. Corneal Confocal Microscopy (CCM) enables non-invasive diagnosis, but automated methods suffer from inefficient feature extraction, reliance on handcrafted priors, and data limitations. We propose HMSViT, a novel Hierarchical Masked Self-Supervised Vision Transformer (HMSViT) designed for corneal nerve segmentation and DPN diagnosis. Unlike existing methods, HMSViT employs pooling-based hierarchical and dual attention mechanisms with absolute positional encoding, enabling efficient multi-scale feature extraction by capturing fine-grained local details in early layers and integrating global context in deeper layers, all at a lower computational cost. A block-masked self supervised learning framework is designed for the HMSViT that reduces reliance on labelled data, enhancing feature robustness, while a multi-scale decoder is used for segmentation and classification by fusing hierarchical features. Experiments on clinical CCM datasets showed HMSViT achieves state-of-the-art performance, with 61.34% mIoU for nerve segmentation and 70.40% diagnostic accuracy, outperforming leading hierarchical models like the Swin Transformer and HiViT by margins of up to 6.39% in segmentation accuracy while using fewer parameters. Detailed ablation studies further reveal that integrating block-masked SSL with hierarchical multi-scale feature extraction substantially enhances performance compared to conventional supervised training. Overall, these comprehensive experiments confirm that HMSViT delivers excellent, robust, and clinically viable results, demonstrating its potential for scalable deployment in real-world diagnostic applications.
zh
[CV-37] USIS16K: High-Quality Dataset for Underwater Salient Instance Segmentation
【速读】：该论文旨在解决水下显著实例分割（Underwater Salient Instance Segmentation, USIS）问题，即在水下环境中同时实现注意力定位（显著性预测）和实例分割。由于水下环境的不可达性和动态性，以及高质量标注数据集的匮乏，USIS仍是一个研究不足的挑战。论文提出USIS16K，一个包含16,151张高分辨率水下图像的大规模数据集，覆盖158种水下物体类别，并提供高质量的实例级显著目标掩码，其在多样性、复杂性和可扩展性方面具有显著优势。解决方案的关键在于构建一个大规模、高精度的标注数据集，以推动水下视觉任务的研究与应用。
链接: https://arxiv.org/abs/2506.19472
作者: Lin Hong,Xin Wang,Yihao Li,Xia Wang
机构: Harbin Institute of Technology (Shenzhen)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 8 pages 10 figures
Abstract:Inspired by the biological visual system that selectively allocates attention to efficiently identify salient objects or regions, underwater salient instance segmentation (USIS) aims to jointly address the problems of where to look (saliency prediction) and what is there (instance segmentation) in underwater scenarios. However, USIS remains an underexplored challenge due to the inaccessibility and dynamic nature of underwater environments, as well as the scarcity of large-scale, high-quality annotated datasets. In this paper, we introduce USIS16K, a large-scale dataset comprising 16,151 high-resolution underwater images collected from diverse environmental settings and covering 158 categories of underwater objects. Each image is annotated with high-quality instance-level salient object masks, representing a significant advance in terms of diversity, complexity, and scalability. Furthermore, we provide benchmark evaluations on underwater object detection and USIS tasks using USIS16K. To facilitate future research in this domain, the dataset and benchmark models are publicly available.
zh
[CV-38] Surgery-R1: Advancing Surgical-VQLA with Reasoning Multimodal Large Language Model via Reinforcement Learning
【速读】：该论文旨在解决现有手术场景理解模型在生成式AI（Generative AI）任务中缺乏深度推理能力和可解释性的问题，这限制了其在临床应用中的可靠性与发展潜力。其解决方案的关键在于构建首个面向手术视觉问答定位任务（Surgical-VQLA）的推理多模态大语言模型（Reasoning MLLM），即Surgery-R1，通过两阶段微调机制（监督微调和强化微调）提升模型的复杂推理能力，并设计多模态一致性奖励机制以优化手术场景中的位置幻觉问题。
链接: https://arxiv.org/abs/2506.19469
作者: Pengfei Hao,Shuaibo Li,Hongqiu Wang,Zhizhuo Kou,Junhang Zhang,Guang Yang,Lei Zhu
机构: Hong Kong University of Science and Technology (Guangzhou), China; Hong Kong University of Science and Technology, Hong Kong SAR; Department of Thoracic Surgery, the Seventh Affiliated Hospital, Sun Yat-sen University, China; Bioengineering/Imperial-X, Imperial College London, UK; ROAS Thrust, Hong Kong University of Science and Technology (Guangzhou), China; Department of Electronic and Computer Engineering, Hong Kong SAR, China
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:In recent years, significant progress has been made in the field of surgical scene understanding, particularly in the task of Visual Question Localized-Answering in robotic surgery (Surgical-VQLA). However, existing Surgical-VQLA models lack deep reasoning capabilities and interpretability in surgical scenes, which limits their reliability and potential for development in clinical applications. To address this issue, inspired by the development of Reasoning Multimodal Large Language Models (MLLMs), we first build the Surgery-R1-54k dataset, including paired data for Visual-QA, Grounding-QA, and Chain-of-Thought (CoT). Then, we propose the first Reasoning MLLM for Surgical-VQLA (Surgery-R1). In our Surgery-R1, we design a two-stage fine-tuning mechanism to enable the basic MLLM with complex reasoning abilities by utilizing supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT). Furthermore, for an efficient and high-quality rule-based reward system in our RFT, we design a Multimodal Coherence reward mechanism to mitigate positional illusions that may arise in surgical scenarios. Experiment results demonstrate that Surgery-R1 outperforms other existing state-of-the-art (SOTA) models in the Surgical-VQLA task and widely-used MLLMs, while also validating its reasoning capabilities and the effectiveness of our approach. The code and dataset will be organized in this https URL.
zh
[CV-39] Stylized Structural Patterns for Improved Neural Network Pre-training
【速读】：该论文试图解决现代计算机视觉中的深度学习模型依赖于大规模真实图像数据集的问题，这些问题在数据收集、隐私和法律方面存在挑战，限制了模型的商业应用。论文提出的解决方案关键在于一种两步方法：首先，通过改进的神经分形公式引入一类新的合成数据；其次，提出反向风格化技术，将少量无版权的真实图像的视觉特征转移到合成数据集中，从而提升合成数据的有效性。
链接: https://arxiv.org/abs/2506.19465
作者: Farnood Salehi,Vandit Sharma,Amirhossein Askari Farsangi,Tunç Ozan Aydın
机构: Disney Research | Studios (迪士尼研究院 | 影视工作室); ETH Zürich (苏黎世联邦理工学院)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Modern deep learning models in computer vision require large datasets of real images, which are difficult to curate and pose privacy and legal concerns, limiting their commercial use. Recent works suggest synthetic data as an alternative, yet models trained with it often underperform. This paper proposes a two-step approach to bridge this gap. First, we propose an improved neural fractal formulation through which we introduce a new class of synthetic data. Second, we propose reverse stylization, a technique that transfers visual features from a small, license-free set of real images onto synthetic datasets, enhancing their effectiveness. We analyze the domain gap between our synthetic datasets and real images using Kernel Inception Distance (KID) and show that our method achieves a significantly lower distributional gap compared to existing synthetic datasets. Furthermore, our experiments across different tasks demonstrate the practical impact of this reduced gap. We show that pretraining the EDM2 diffusion model on our synthetic dataset leads to an 11% reduction in FID during image generation, compared to models trained on existing synthetic datasets, and a 20% decrease in autoencoder reconstruction error, indicating improved performance in data representation. Furthermore, a ViT-S model trained for classification on this synthetic data achieves over a 10% improvement in ImageNet-100 accuracy. Our work opens up exciting possibilities for training practical models when sufficiently large real training sets are not available.
zh
[CV-40] Assessing Risk of Stealing Proprietary Models for Medical Imaging Tasks MICCAI2024
【速读】：该论文旨在解决医疗影像中黑盒模型（black-box medical imaging models）在面对模型窃取（model stealing, MS）攻击时的脆弱性问题。尽管已有大量研究关注通用视觉任务中的MS攻击，但医疗影像模型在此方面的易受性尚未得到充分探讨。论文提出了一种名为QueryWise的两步模型窃取方法，其关键在于利用从代理分布中获取的未标记数据来训练窃取模型，从而在有限的查询预算下提升攻击效果。实验结果表明，该方法能够有效实现对胆囊癌和新冠肺炎分类模型的模型窃取。
链接: https://arxiv.org/abs/2506.19464
作者: Ankita Raj,Harsh Swaika,Deepankar Varma,Chetan Arora
机构: Indian Institute of Technology Delhi(印度理工学院德里分校)
类目: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
备注: Accepted to MICCAI 2024
Abstract:The success of deep learning in medical imaging applications has led several companies to deploy proprietary models in diagnostic workflows, offering monetized services. Even though model weights are hidden to protect the intellectual property of the service provider, these models are exposed to model stealing (MS) attacks, where adversaries can clone the model’s functionality by querying it with a proxy dataset and training a thief model on the acquired predictions. While extensively studied on general vision tasks, the susceptibility of medical imaging models to MS attacks remains inadequately explored. This paper investigates the vulnerability of black-box medical imaging models to MS attacks under realistic conditions where the adversary lacks access to the victim model’s training data and operates with limited query budgets. We demonstrate that adversaries can effectively execute MS attacks by using publicly available datasets. To further enhance MS capabilities with limited query budgets, we propose a two-step model stealing approach termed QueryWise. This method capitalizes on unlabeled data obtained from a proxy distribution to train the thief model without incurring additional queries. Evaluation on two medical imaging models for Gallbladder Cancer and COVID-19 classification substantiates the effectiveness of the proposed attack. The source code is available at this https URL.
zh
[CV-41] Deblurring in the Wild: A Real-World Dataset from Smartphone High-Speed Videos
【速读】：该论文旨在解决真实世界中图像去模糊（image deblurring）问题，特别是针对由手机慢动作视频生成的复杂运动模糊场景。其解决方案的关键在于构建了一个大规模的真实世界图像去模糊数据集，该数据集通过从手机慢动作视频中提取240帧并平均这些帧以模拟长曝光模糊，同时将时间中心帧作为清晰参考图像。该数据集包含超过42,000对高分辨率模糊-清晰图像，相较现有常用数据集规模扩大约10倍，并涵盖了更多样化的场景和运动模式，从而为去模糊模型提供了更具挑战性和泛化性的基准。
链接: https://arxiv.org/abs/2506.19445
作者: Mahdi Mohd Hossain Noki,Syed Mumtahin Mahmud,Prothito Shovon Majumder,Abdul Mohaimen Al Radi,Md. Haider Ali,Md. Mosaddek Khan
机构: University of Dhaka (达卡大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 8 pages (without references), 3 figures. Dataset this https URL
Abstract:We introduce the largest real-world image deblurring dataset constructed from smartphone slow-motion videos. Using 240 frames captured over one second, we simulate realistic long-exposure blur by averaging frames to produce blurry images, while using the temporally centered frame as the sharp reference. Our dataset contains over 42,000 high-resolution blur-sharp image pairs, making it approximately 10 times larger than widely used datasets, with 8 times the amount of different scenes, including indoor and outdoor environments, with varying object and camera motions. We benchmark multiple state-of-the-art (SOTA) deblurring models on our dataset and observe significant performance degradation, highlighting the complexity and diversity of our benchmark. Our dataset serves as a challenging new benchmark to facilitate robust and generalizable deblurring models.
zh
[CV-42] Sampling Matters in Explanations: Towards Trustworthy Attribution Analysis Building Block in Visual Models through Maximizing Explanation Certainty
【速读】：该论文旨在解决图像归因分析中由于梯度集成样本分布与自然图像分布不匹配而导致的解释确定性不足的问题。其解决方案的关键在于提出一种半最优采样方法，通过抑制输入中的特征来生成与自然图像分布近似一致的样本分布，从而提升归因分析的可信度和解释质量。
链接: https://arxiv.org/abs/2506.19442
作者: Róisín Luo,James McDermott,Colm O’Riordan
机构: University of Galway (爱尔兰)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Code: this https URL
Abstract:Image attribution analysis seeks to highlight the feature representations learned by visual models such that the highlighted feature maps can reflect the pixel-wise importance of inputs. Gradient integration is a building block in the attribution analysis by integrating the gradients from multiple derived samples to highlight the semantic features relevant to inferences. Such a building block often combines with other information from visual models such as activation or attention maps to form ultimate explanations. Yet, our theoretical analysis demonstrates that the extent to the alignment of the sample distribution in gradient integration with respect to natural image distribution gives a lower bound of explanation certainty. Prior works add noise into images as samples and the noise distributions can lead to low explanation certainty. Counter-intuitively, our experiment shows that extra information can saturate neural networks. To this end, building trustworthy attribution analysis needs to settle the sample distribution misalignment problem. Instead of adding extra information into input images, we present a semi-optimal sampling approach by suppressing features from inputs. The sample distribution by suppressing features is approximately identical to the distribution of natural images. Our extensive quantitative evaluation on large scale dataset ImageNet affirms that our approach is effective and able to yield more satisfactory explanations against state-of-the-art baselines throughout all experimental models.
zh
[CV-43] AMF-MedIT: An Efficient Align-Modulation-Fusion Framework for Medical Image-Tabular Data
【速读】：该论文旨在解决多模态医学分析中图像与表格数据融合的挑战，特别是在特征维度和模态贡献存在跨模态差异以及高维表格输入噪声干扰的情况下。其解决方案的关键在于提出AMF-MedIT框架，该框架包含自适应调制与融合（AMF）模块，通过调制目标、模态置信度比、特征掩码、密度损失和泄漏损失等机制，有效协调维度差异并动态调整模态贡献，同时引入FT-Mamba作为高效的表格编码器，以提升对噪声医疗表格数据的处理能力。
链接: https://arxiv.org/abs/2506.19439
作者: Congjing Yu,Jing Ye,Yang Liu,Xiaodong Zhang,Zhiyong Zhang
机构: Sun Yat-sen University (中山大学); Southern Methodist University (南方卫理公会大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Multimodal medical analysis combining image and tabular data has gained increasing attention. However, effective fusion remains challenging due to cross-modal discrepancies in feature dimensions and modality contributions, as well as the noise from high-dimensional tabular inputs. To address these problems, we present AMF-MedIT, an efficient Align-Modulation-Fusion framework for medical image and tabular data integration, particularly under data-scarce conditions. To harmonize dimension discrepancies and dynamically adjust modality contributions, we propose the Adaptive Modulation and Fusion (AMF) module, a novel modulation-based fusion paradigm with a streamlined architecture. We first derive the modulation objectives and introduce a modality confidence ratio, enabling the incorporation of prior knowledge into the fusion process. Then, the feature masks, density and leakage losses are proposed to achieve the modulation objectives. Additionally, we introduce FT-Mamba, a powerful tabular encoder leveraging a selective mechanism to handle noisy medical tabular data efficiently. Furthermore, interpretability studies are conducted to explore how different tabular encoders supervise the imaging modality during contrastive pretraining for the first time. Extensive experiments demonstrate that AMF-MedIT achieves a superior balance between multimodal performance and data efficiency while showing strong adaptability to incomplete tabular data. Interpretability analysis also highlights FT-Mamba’s capabilities in extracting distinct tabular features and guiding the image encoder toward more accurate and flexible attention patterns.
zh
[CV-44] EvDetMAV: Generalized MAV Detection from Moving Event Cameras
【速读】：该论文旨在解决现有微小型飞行器（MAV）检测方法依赖于RGB图像中目标的外观特征，导致泛化能力不足的问题。其解决方案的关键在于利用事件相机中由于高速旋转螺旋桨产生的独特事件流特征，通过三个模块提取螺旋桨的显著时空特征，并过滤背景物体和相机运动带来的噪声，从而实现不同类型的MAV检测。
链接: https://arxiv.org/abs/2506.19416
作者: Yin Zhang,Zian Ning,Xiaoyu Zhang,Shiliang Guo,Peidong Liu,Shiyu Zhao
机构: Zhejiang University (浙江大学); Westlake University (西湖大学); The Chinese University of Hong Kong (香港中文大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注: 8 pages, 7 figures. This paper is accepted by IEEE Robotics and Automation Letters
Abstract:Existing micro aerial vehicle (MAV) detection methods mainly rely on the target’s appearance features in RGB images, whose diversity makes it difficult to achieve generalized MAV detection. We notice that different types of MAVs share the same distinctive features in event streams due to their high-speed rotating propellers, which are hard to see in RGB images. This paper studies how to detect different types of MAVs from an event camera by fully exploiting the features of propellers in the original event stream. The proposed method consists of three modules to extract the salient and spatio-temporal features of the propellers while filtering out noise from background objects and camera motion. Since there are no existing event-based MAV datasets, we introduce a novel MAV dataset for the community. This is the first event-based MAV dataset comprising multiple scenarios and different types of MAVs. Without training, our method significantly outperforms state-of-the-art methods and can deal with challenging scenarios, achieving a precision rate of 83.0% (+30.3%) and a recall rate of 81.5% (+36.4%) on the proposed testing dataset. The dataset and code are available at: this https URL.
zh
[CV-45] Virtual Memory for 3D Gaussian Splatting
【速读】：该论文旨在解决大规模、复杂3D Gaussian Splatting场景在实时渲染中的内存与计算效率问题。其关键解决方案是利用虚拟内存（virtual memory）技术，结合虚拟纹理（virtual texturing）方法，动态识别并按需将可见的Gaussians流式传输到GPU，从而减少内存占用并提升渲染速度，尤其针对高复杂度场景进行了优化。
链接: https://arxiv.org/abs/2506.19415
作者: Jonathan Haberl,Philipp Fleck,Clemens Arth
机构: 未知
类目: Graphics (cs.GR); Computer Vision and Pattern Recognition (cs.CV); Human-Computer Interaction (cs.HC)
备注: Based on the Master Thesis from Jonathan Haberl from 2024, Submitted to TVCG in Feb. 2025;
Abstract:3D Gaussian Splatting represents a breakthrough in the field of novel view synthesis. It establishes Gaussians as core rendering primitives for highly accurate real-world environment reconstruction. Recent advances have drastically increased the size of scenes that can be created. In this work, we present a method for rendering large and complex 3D Gaussian Splatting scenes using virtual memory. By leveraging well-established virtual memory and virtual texturing techniques, our approach efficiently identifies visible Gaussians and dynamically streams them to the GPU just in time for real-time rendering. Selecting only the necessary Gaussians for both storage and rendering results in reduced memory usage and effectively accelerates rendering, especially for highly complex scenes. Furthermore, we demonstrate how level of detail can be integrated into our proposed method to further enhance rendering speed for large-scale scenes. With an optimized implementation, we highlight key practical considerations and thoroughly evaluate the proposed technique and its impact on desktop and mobile devices.
zh
[CV-46] A Global-Local Cross-Attention Network for Ultra-high Resolution Remote Sensing Image Semantic Segmentation
【速读】：该论文旨在解决超高分辨率（Ultra-High Resolution, UHR）遥感图像中语义分割的准确性与计算效率问题，以及多尺度特征融合的挑战。其解决方案的关键在于提出GLCANet（Global-Local Cross-Attention Network），该框架采用双流结构以高效融合全局语义与局部细节，并通过自注意力机制增强长距离依赖关系，同时利用掩码交叉注意力机制自适应地融合全局与局部特征，从而提升分割精度并减少GPU占用。
链接: https://arxiv.org/abs/2506.19406
作者: Chen Yi,Shan LianLei
机构: Xi’an University of Science and Technology (西安科技大学); University of Chinese Academy of Sciences (中国科学院大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:With the rapid development of ultra-high resolution (UHR) remote sensing technology, the demand for accurate and efficient semantic segmentation has increased significantly. However, existing methods face challenges in computational efficiency and multi-scale feature fusion. To address these issues, we propose GLCANet (Global-Local Cross-Attention Network), a lightweight segmentation framework designed for UHR remote sensing this http URL employs a dual-stream architecture to efficiently fuse global semantics and local details while minimizing GPU usage. A self-attention mechanism enhances long-range dependencies, refines global features, and preserves local details for better semantic consistency. A masked cross-attention mechanism also adaptively fuses global-local features, selectively enhancing fine-grained details while exploiting global context to improve segmentation accuracy. Experimental results show that GLCANet outperforms state-of-the-art methods regarding accuracy and computational efficiency. The model effectively processes large, high-resolution images with a small memory footprint, providing a promising solution for real-world remote sensing applications.
zh
[CV-47] Generate the Forest before the Trees – A Hierarchical Diffusion model for Climate Downscaling
【速读】：该论文试图解决传统气候降尺度方法计算成本高且难以生成高分辨率气候数据的问题，特别是在局部规划中对高分辨率数据的需求。其解决方案的关键在于引入一种分层扩散降尺度（Hierarchical Diffusion Downscaling, HDD）模型，该模型通过一种易于扩展的分层采样过程，在扩散框架中实现从粗到细的层次结构，从而在保持较高精度的同时显著降低计算负载，并支持在不同分辨率的CMIP6模型间无缝迁移。
链接: https://arxiv.org/abs/2506.19391
作者: Declan J. Curran,Sanaa Hobeichi,Hira Saleem,Hao Xue,Flora D. Salim
机构: University of New South Wales (新南威尔士大学); ARC Centre of Excellence for the Weather of the 21st Century and Climate Change Research Centre (21世纪天气与气候变化研究中心)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 8 pages
Abstract:Downscaling is essential for generating the high-resolution climate data needed for local planning, but traditional methods remain computationally demanding. Recent years have seen impressive results from AI downscaling models, particularly diffusion models, which have attracted attention due to their ability to generate ensembles and overcome the smoothing problem common in other AI methods. However, these models typically remain computationally intensive. We introduce a Hierarchical Diffusion Downscaling (HDD) model, which introduces an easily-extensible hierarchical sampling process to the diffusion framework. A coarse-to-fine hierarchy is imposed via a simple downsampling scheme. HDD achieves competitive accuracy on ERA5 reanalysis datasets and CMIP6 models, significantly reducing computational load by running on up to half as many pixels with competitive results. Additionally, a single model trained at 0.25° resolution transfers seamlessly across multiple CMIP6 models with much coarser resolution. HDD thus offers a lightweight alternative for probabilistic climate downscaling, facilitating affordable large-ensemble high-resolution climate projections. See a full code implementation at: this https URL.
zh
[CV-48] Emergence of Text Readability in Vision Language Models CVPR2025
【速读】：该论文试图解决视觉-语言模型（Vision-Language Models, VLMs）在训练过程中文本可读性（text readability） emergence 的时机问题，即为何模型在训练后期才具备识别图像中文本内容的能力，而语义理解能力则从早期阶段就开始逐步提升。解决方案的关键在于揭示对比学习在初期更侧重于通用语义理解，而文本特有的符号处理则在后期才逐渐发展，这表明需要设计针对性的训练策略以加速模型对文本的稳健理解。
链接: https://arxiv.org/abs/2506.19389
作者: Jaeyoo Park,Sanghyuk Chun,Wonjae Kim,Sangdoo Yun,Bohyung Han
机构: Institution1; Institution2; NAVER AI Lab (NAVER人工智能实验室); TwelveLabs; Computer Vision Laboratory, ECE1 & IPAI2, Seoul National University (首尔国立大学计算机视觉实验室，ECE1与IPAI2); Naver AI Lab
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: EVAL-FoMo Workshop @ CVPR 2025
Abstract:We investigate how the ability to recognize textual content within images emerges during the training of Vision-Language Models (VLMs). Our analysis reveals a critical phenomenon: the ability to read textual information in a given image \textbf(text readability) emerges abruptly after substantial training iterations, in contrast to semantic content understanding which develops gradually from the early stages of training. This delayed emergence may reflect how contrastive learning tends to initially prioritize general semantic understanding, with text-specific symbolic processing developing later. Interestingly, the ability to match images with rendered text develops even slower, indicating a deeper need for semantic integration. These findings highlight the need for tailored training strategies to accelerate robust text comprehension in VLMs, laying the groundwork for future research on optimizing multimodal learning.
zh
[CV-49] Online camera-pose-free stereo endoscopic tissue deformation recovery with tissue-invariant vision-biomechanics consistency
【速读】：该论文旨在解决基于立体内窥镜图像的组织形变恢复问题，该问题在工具-组织相互作用分析中具有重要意义，并有助于手术导航和自主软组织操作。现有方法存在相机运动、遮挡、大形变、缺乏组织特异性生物力学先验以及依赖离线处理等问题。该论文提出的解决方案的关键在于将组织几何建模为3D点和导数图，将组织形变建模为3D位移和局部形变图，通过相机中心设置下的场景运动建模，实现帧间对齐而无需估计相机位姿，并引入规范图（canonical map）以在线优化组织几何与形变。该方法在输入深度和光流信息的情况下，能够在部分遮挡或组织移出视野时稳定建模组织几何与形变。
链接: https://arxiv.org/abs/2506.19388
作者: Jiahe Chen,Naoki Tomii,Ichiro Sakuma,Etsuko Kobayashi
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Tissue deformation recovery based on stereo endoscopic images is crucial for tool-tissue interaction analysis and benefits surgical navigation and autonomous soft tissue manipulation. Previous research suffers from the problems raised from camera motion, occlusion, large tissue deformation, lack of tissue-specific biomechanical priors, and reliance on offline processing. Unlike previous studies where the tissue geometry and deformation are represented by 3D points and displacements, the proposed method models tissue geometry as the 3D point and derivative map and tissue deformation as the 3D displacement and local deformation map. For a single surface point, 6 parameters are used to describe its rigid motion and 3 parameters for its local deformation. The method is formulated under the camera-centric setting, where all motions are regarded as the scene motion with respect to the camera. Inter-frame alignment is realized by optimizing the inter-frame deformation, making it unnecessary to estimate camera pose. The concept of the canonical map is introduced to optimize tissue geometry and deformation in an online approach. Quantitative and qualitative experiments were conducted using in vivo and ex vivo laparoscopic datasets. With the inputs of depth and optical flow, the method stably models tissue geometry and deformation even when the tissue is partially occluded or moving outside the field of view. Results show that the 3D reconstruction accuracy in the non-occluded and occluded areas reaches 0.37 \pm 0.27 mm and 0.39 \pm 0.21 mm in terms of surface distance, respectively. The method can also estimate surface strain distribution during various manipulations as an extra modality for mechanical-based analysis.
zh
[CV-50] SoK: Can Synthetic Images Replace Real Data? A Survey of Utility and Privacy of Synthetic Image Generation USENIX-SECURITY USENIX-SECURITY’25
【速读】：该论文试图解决隐私保护数据合成（Privacy-Preserving Data Synthesis, PPDS）中合成图像生成方法的系统性评估与比较问题，特别是在生成-采样-分类（generation-sampling-classification）流程中如何平衡数据效用与隐私风险。其解决方案的关键在于对现有图像合成方法、隐私攻击手段及缓解措施进行系统分类，并构建一个基准测试平台，采用模型无关的成员推断攻击（model-agnostic membership inference attacks, MIAs）作为隐私风险的度量标准，从而为合成数据在实际应用中的最优释放策略提供依据。
链接: https://arxiv.org/abs/2506.19360
作者: Yunsung Chung,Yunbei Zhang,Nassir Marrouche,Jihun Hamm
机构: Tulane University (杜兰大学)
类目: Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted at the 34th USENIX Security Symposium (USENIX Security '25). 21 pages, plus a 6-page appendix
Abstract:Advances in generative models have transformed the field of synthetic image generation for privacy-preserving data synthesis (PPDS). However, the field lacks a comprehensive survey and comparison of synthetic image generation methods across diverse settings. In particular, when we generate synthetic images for the purpose of training a classifier, there is a pipeline of generation-sampling-classification which takes private training as input and outputs the final classifier of interest. In this survey, we systematically categorize existing image synthesis methods, privacy attacks, and mitigations along this generation-sampling-classification pipeline. To empirically compare diverse synthesis approaches, we provide a benchmark with representative generative methods and use model-agnostic membership inference attacks (MIAs) as a measure of privacy risk. Through this study, we seek to answer critical questions in PPDS: Can synthetic data effectively replace real data? Which release strategy balances utility and privacy? Do mitigations improve the utility-privacy tradeoff? Which generative models perform best across different scenarios? With a systematic evaluation of diverse methods, our study provides actionable insights into the utility-privacy tradeoffs of synthetic data generation methods and guides the decision on optimal data releasing strategies for real-world applications.
zh
[CV-51] raining-Free Motion Customization for Distilled Video Generators with Adaptive Test-Time Distillation
【速读】：该论文旨在解决蒸馏视频生成模型在无训练设置下，通过参考视频进行运动定制时表现不佳的问题。现有无训练方法因蒸馏模型加速的生成过程和较大的去噪步骤而难以泛化。解决方案的关键在于提出MotionEcho，这是一种新颖的无训练测试时蒸馏框架，通过利用扩散教师强制（diffusion teacher forcing）实现运动定制，其核心是使用高质量、慢速的教师模型通过端点预测和插值引导快速学生模型的推理，并根据指导需求动态分配计算资源以保持效率。
链接: https://arxiv.org/abs/2506.19348
作者: Jintao Rong,Xin Xie,Xinyi Yu,Linlin Ou,Xinyu Zhang,Chunhua Shen,Dong Gong
机构: Zhejiang University of Technology (浙江工业大学); UNSW Sydney (新南威尔士大学); University of Adelaide (阿德莱德大学); Zhejiang University (浙江大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Distilled video generation models offer fast and efficient synthesis but struggle with motion customization when guided by reference videos, especially under training-free settings. Existing training-free methods, originally designed for standard diffusion models, fail to generalize due to the accelerated generative process and large denoising steps in distilled models. To address this, we propose MotionEcho, a novel training-free test-time distillation framework that enables motion customization by leveraging diffusion teacher forcing. Our approach uses high-quality, slow teacher models to guide the inference of fast student models through endpoint prediction and interpolation. To maintain efficiency, we dynamically allocate computation across timesteps according to guidance needs. Extensive experiments across various distilled video generation models and benchmark datasets demonstrate that our method significantly improves motion fidelity and generation quality while preserving high efficiency. Project page: this https URL
zh
[CV-52] Image Segmentation using Chan-Vese Active Contours
【速读】：该论文旨在解决图像分割中的复杂任务，特别是针对噪声图像或边界弱的图像进行准确分割的问题。其解决方案的关键在于采用基于区域强度差异而非图像梯度的Chan-Vese活动轮廓模型，该模型源自Mumford-Shah变分框架，通过水平集公式进行数学推导，并结合散度定理和曲线演化理论对能量项进行详细处理，从而实现了在数值稳定性方面的改进，包括使用迎风熵方案和基于曲率的正则化方法。
链接: https://arxiv.org/abs/2506.19344
作者: Pranav Shenoy K. P
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:This paper presents a comprehensive derivation and implementation of the Chan-Vese active contour model for image segmentation. The model, derived from the Mumford-Shah variational framework, evolves contours based on regional intensity differences rather than image gradients, making it highly effective for segmenting noisy images or images with weak boundaries. We provide a rigorous mathematical derivation of the level set formulation, including detailed treatment of each energy term using the divergence theorem and curve evolution theory. The resulting algorithm is implemented in Python using finite difference methods with special care to numerical stability, including an upwind entropy scheme and curvature-based regularization. Experimental results on medical and synthetic images demonstrate accurate segmentation, robustness to noise, and superior performance compared to classical edge-based methods. This study confirms the suitability of the Chan-Vese model for complex segmentation tasks and highlights its potential for use in real-world imaging applications.
zh
[CV-53] rajectory Prediction in Dynamic Object Tracking: A Critical Study
【速读】：该论文旨在解决动态物体跟踪（DOT）与轨迹预测（TP）方法在实际应用中的技术挑战，包括提升模型的泛化能力、计算效率、减少对数据的依赖性以及应对伦理问题。其解决方案的关键在于推动多模态数据融合、语义信息整合以及开发具有上下文感知能力的系统，同时构建符合伦理和隐私保护的框架。
链接: https://arxiv.org/abs/2506.19341
作者: Zhongping Dong,Liming Chen,Mohand Tahar Kechadi
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:This study provides a detailed analysis of current advancements in dynamic object tracking (DOT) and trajectory prediction (TP) methodologies, including their applications and challenges. It covers various approaches, such as feature-based, segmentation-based, estimation-based, and learning-based methods, evaluating their effectiveness, deployment, and limitations in real-world scenarios. The study highlights the significant impact of these technologies in automotive and autonomous vehicles, surveillance and security, healthcare, and industrial automation, contributing to safety and efficiency. Despite the progress, challenges such as improved generalization, computational efficiency, reduced data dependency, and ethical considerations still exist. The study suggests future research directions to address these challenges, emphasizing the importance of multimodal data integration, semantic information fusion, and developing context-aware systems, along with ethical and privacy-preserving frameworks.
zh
[CV-54] Segment Any 3D-Part in a Scene from a Sentence
【速读】：该论文试图解决基于自然语言描述的任意3D场景中部件（part）分割问题，突破传统对象级（object-level）3D场景理解的局限，并应对数据和方法上的挑战。由于数据获取和标注成本高昂，现有数据集和方法主要局限于对象级理解。为克服数据和标注可用性的限制，研究者提出了3D-PU数据集，这是首个具有密集部件标注的大规模3D数据集，通过创新且低成本的方法构建带有细粒度部件级标注的合成3D场景，为先进的3D部件场景理解奠定了基础。在方法层面，提出了OpenPart3D，这是一个仅输入3D数据的框架，能够有效应对部件级分割的挑战。实验结果表明，该方法在开放词汇3D场景理解任务中表现出色，具有跨多种3D场景数据集的强大泛化能力。
链接: https://arxiv.org/abs/2506.19331
作者: Hongyu Wu,Pengwan Yang,Yuki M. Asano,Cees G. M. Snoek
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:This paper aims to achieve the segmentation of any 3D part in a scene based on natural language descriptions, extending beyond traditional object-level 3D scene understanding and addressing both data and methodological challenges. Due to the expensive acquisition and annotation burden, existing datasets and methods are predominantly limited to object-level comprehension. To overcome the limitations of data and annotation availability, we introduce the 3D-PU dataset, the first large-scale 3D dataset with dense part annotations, created through an innovative and cost-effective method for constructing synthetic 3D scenes with fine-grained part-level annotations, paving the way for advanced 3D-part scene understanding. On the methodological side, we propose OpenPart3D, a 3D-input-only framework to effectively tackle the challenges of part-level segmentation. Extensive experiments demonstrate the superiority of our approach in open-vocabulary 3D scene understanding tasks at the part level, with strong generalization capabilities across various 3D scene datasets.
zh
[CV-55] Comparative Performance of Finetuned ImageNet Pre-trained Models for Electronic Component Classification
【速读】：该论文旨在解决电子元件分类与检测的问题，以降低制造业中的劳动力成本并推动技术与工业发展。其解决方案的关键在于利用在ImageNet上预训练的模型进行图像分类，这些模型在有限数据条件下仍能实现优异的性能，从而验证了预训练模型在电子制造领域的实用性和有效性。
链接: https://arxiv.org/abs/2506.19330
作者: Yidi Shao,Longfei Zhou,Fangshuo Tang,Xinyi Shi,Dalang Chen,Shengtao Xia
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: This is the author’s version of the accepted paper. The final version will appear in IEEE UV 2024
Abstract:Electronic component classification and detection are crucial in manufacturing industries, significantly reducing labor costs and promoting technological and industrial development. Pre-trained models, especially those trained on ImageNet, are highly effective in image classification, allowing researchers to achieve excellent results even with limited data. This paper compares the performance of twelve ImageNet pre-trained models in classifying electronic components. Our findings show that all models tested delivered respectable accuracies. MobileNet-V2 recorded the highest at 99.95%, while EfficientNet-B0 had the lowest at 92.26%. These results underscore the substantial benefits of using ImageNet pre-trained models in image classification tasks and confirm the practical applicability of these methods in the electronics manufacturing sector.
zh
[CV-56] Memory-Augmented Incomplete Multimodal Survival Prediction via Cross-Slide and Gene-Attentive Hypergraph Learning MICCAI2025
【速读】：该论文旨在解决多模态病理-基因组分析在癌症生存预测中的模态不平衡问题以及不完整模态数据的临床适用性限制。现有方法主要依赖福尔马林固定石蜡包埋（FFPE）切片与基因组数据的整合，忽视了其他保存方式如新鲜冷冻（FF）切片的可用性，并且由于病理数据的高分辨率空间特性主导了跨模态融合过程，导致病理与基因组模态之间的融合效果不佳。论文提出的解决方案关键在于引入超图学习以有效整合多WSI信息及病理切片与基因组数据之间的跨模态交互，并通过记忆机制存储先前学习的配对病理-基因组特征，动态补偿不完整的模态数据。
链接: https://arxiv.org/abs/2506.19324
作者: Mingcheng Qu,Guang Yang,Donglin Di,Yue Gao,Tonghua Su,Yang Song,Lei Fan
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: accepted by MICCAI2025 code: this https URL
Abstract:Multimodal pathology-genomic analysis is critical for cancer survival prediction. However, existing approaches predominantly integrate formalin-fixed paraffin-embedded (FFPE) slides with genomic data, while neglecting the availability of other preservation slides, such as Fresh Froze (FF) slides. Moreover, as the high-resolution spatial nature of pathology data tends to dominate the cross-modality fusion process, it hinders effective multimodal fusion and leads to modality imbalance challenges between pathology and genomics. These methods also typically require complete data modalities, limiting their clinical applicability with incomplete modalities, such as missing either pathology or genomic data. In this paper, we propose a multimodal survival prediction framework that leverages hypergraph learning to effectively integrate multi-WSI information and cross-modality interactions between pathology slides and genomics data while addressing modality imbalance. In addition, we introduce a memory mechanism that stores previously learned paired pathology-genomic features and dynamically compensates for incomplete modalities. Experiments on five TCGA datasets demonstrate that our model outperforms advanced methods by over 2.3% in C-Index. Under incomplete modality scenarios, our approach surpasses pathology-only (3.3%) and gene-only models (7.9%). Code: this https URL
zh
[CV-57] Continual Retinal Vision-Language Pre-training upon Incremental Imaging Modalities MICCAI2025
【速读】：该论文旨在解决传统眼底图像分析模型在多模态数据融合方面的不足，以及在动态环境中进行持续预训练时出现的灾难性遗忘问题。其解决方案的关键在于提出RetCoP框架，该框架通过增量整合不同成像模态的图像和文本特征到一个统一的基础模型中，并采用回放策略和非对角线信息蒸馏方法，以缓解持续预训练过程中的知识遗忘问题。
链接: https://arxiv.org/abs/2506.19320
作者: Yuang Yao,Ruiqi Wu,Yi Zhou,Tao Zhou
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Accepted by MICCAI 2025
Abstract:Traditional fundus image analysis models focus on single-modal tasks, ignoring fundus modality complementarity, which limits their versatility. Recently, retinal foundation models have emerged, but most still remain modality-specific. Integrating multiple fundus imaging modalities into a single foundation model is valuable. However, in dynamic environments, data from different modalities often arrive incrementally, necessitating continual pre-training. To address this, we propose RetCoP, the first continual vision-language pre-training framework in the fundus domain, which incrementally integrates image and text features from different imaging modalities into a single unified foundation model. To mitigate catastrophic forgetting in continual pre-training, we introduce a rehearsal strategy utilizing representative image-text pairs and an off-diagonal information distillation approach. The former allows the model to revisit knowledge from previous stages, while the latter explicitly preserves the alignment between image and text representations. Experiments show that RetCoP outperforms all the compared methods, achieving the best generalization and lowest forgetting rate. The code can be found at this https URL.
zh
[CV-58] Progressive Modality Cooperation for Multi-Modality Domain Adaptation
【速读】：该论文旨在解决多模态域适应（Multi-Modality Domain Adaptation, MMDA）及更一般的利用特权信息的多模态域适应（MMDA-PI）中的知识迁移问题，即如何有效利用源域中多模态数据（如RGB和深度图像）来提升目标域的性能，尤其是在目标域部分模态缺失的情况下。解决方案的关键在于提出了一种名为渐进模态协作（Progressive Modality Cooperation, PMC）的框架，通过两个新提出的模块实现多模态协同选择可靠伪标签样本，从而捕捉模态特异性信息与模态融合信息；在MMDA-PI设置下，进一步引入了基于特权信息的PMC-PI方法，其核心是设计了一个多模态数据生成（MMG）网络，通过对抗学习和加权伪语义条件约束，生成目标域缺失的模态以保持领域分布一致性和语义保真度。
链接: https://arxiv.org/abs/2506.19316
作者: Weichen Zhang,Dong Xu,Jing Zhang,Wanli Ouyang
机构: The University of Sydney (悉尼大学); Beihang University (北京航空航天大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:In this work, we propose a new generic multi-modality domain adaptation framework called Progressive Modality Cooperation (PMC) to transfer the knowledge learned from the source domain to the target domain by exploiting multiple modality clues (\eg, RGB and depth) under the multi-modality domain adaptation (MMDA) and the more general multi-modality domain adaptation using privileged information (MMDA-PI) settings. Under the MMDA setting, the samples in both domains have all the modalities. In two newly proposed modules of our PMC, the multiple modalities are cooperated for selecting the reliable pseudo-labeled target samples, which captures the modality-specific information and modality-integrated information, respectively. Under the MMDA-PI setting, some modalities are missing in the target domain. Hence, to better exploit the multi-modality data in the source domain, we further propose the PMC with privileged information (PMC-PI) method by proposing a new multi-modality data generation (MMG) network. MMG generates the missing modalities in the target domain based on the source domain data by considering both domain distribution mismatch and semantics preservation, which are respectively achieved by using adversarial learning and conditioning on weighted pseudo semantics. Extensive experiments on three image datasets and eight video datasets for various multi-modality cross-domain visual recognition tasks under both MMDA and MMDA-PI settings clearly demonstrate the effectiveness of our proposed PMC framework.
zh
[CV-59] Capturing Fine-Grained Alignments Improves 3D Affordance Detection
【速读】：该论文旨在解决3D点云中可操作性检测（affordance detection）的问题，该任务需要有效捕捉点云与文本之间的细粒度对齐。现有方法在建模这种对齐方面存在不足，导致在标准基准上的性能受限，其关键问题在于依赖点云和文本嵌入之间的简单余弦相似度，缺乏进行细粒度推理所需的表达能力。为解决这一问题，作者提出了LM-AD方法，并引入了Affordance Query Module（AQM），通过利用预训练语言模型高效地捕捉点云与文本之间的细粒度对齐。
链接: https://arxiv.org/abs/2506.19312
作者: Junsei Tokumitsu,Yuiga Wada
机构: Keio AI Research Center (庆应AI研究中心)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: MVA 2025 (Oral)
Abstract:In this work, we address the challenge of affordance detection in 3D point clouds, a task that requires effectively capturing fine-grained alignments between point clouds and text. Existing methods often struggle to model such alignments, resulting in limited performance on standard benchmarks. A key limitation of these approaches is their reliance on simple cosine similarity between point cloud and text embeddings, which lacks the expressiveness needed for fine-grained reasoning. To address this limitation, we propose LM-AD, a novel method for affordance detection in 3D point clouds. Moreover, we introduce the Affordance Query Module (AQM), which efficiently captures fine-grained alignment between point clouds and text by leveraging a pretrained language model. We demonstrated that our method outperformed existing approaches in terms of accuracy and mean Intersection over Union on the 3D AffordanceNet dataset.
zh
[CV-60] Airway Skill Assessment with Spatiotemporal Attention Mechanisms Using Human Gaze
【速读】：该论文试图解决急诊医学中气道管理技能评估依赖主观评价、难以准确衡量真实场景下操作者能力的问题。解决方案的关键在于利用基于机器学习的方法，通过结合人体注视数据和视频记录来评估气管插管（Endotracheal Intubation, ETI）技能，其中采用注意力机制引导模型关注任务相关区域，从而提升对成功与失败ETI操作的识别能力。
链接: https://arxiv.org/abs/2506.19306
作者: Jean-Paul Ainam,Rahul,Lora Cavuoto,Matthew Hackett,Jack Norfleet,Suvranu De
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: 13 pages, 6 figures, 14 equations,
Abstract:Airway management skills are critical in emergency medicine and are typically assessed through subjective evaluation, often failing to gauge competency in real-world scenarios. This paper proposes a machine learning-based approach for assessing airway skills, specifically endotracheal intubation (ETI), using human gaze data and video recordings. The proposed system leverages an attention mechanism guided by the human gaze to enhance the recognition of successful and unsuccessful ETI procedures. Visual masks were created from gaze points to guide the model in focusing on task-relevant areas, reducing irrelevant features. An autoencoder network extracts features from the videos, while an attention module generates attention from the visual masks, and a classifier outputs a classification score. This method, the first to use human gaze for ETI, demonstrates improved accuracy and efficiency over traditional methods. The integration of human gaze data not only enhances model performance but also offers a robust, objective assessment tool for clinical skills, particularly in high-stress environments such as military settings. The results show improvements in prediction accuracy, sensitivity, and trustworthiness, highlighting the potential for this approach to improve clinical training and patient outcomes in emergency medicine.
zh
[CV-61] Open-Vocabulary Camouflaged Object Segmentation with Cascaded Vision Language Models
【速读】：该论文试图解决开放词汇伪装目标分割（Open-Vocabulary Camouflaged Object Segmentation, OVCOS）中的问题，即在视觉模糊和未见过类别的情况下，准确分割和分类伪装目标。现有方法存在两个主要问题：一是由于视觉语言模型（VLM）的全图训练与裁剪区域推理之间的不匹配导致的领域差距；二是依赖于针对清晰边界目标优化的通用分割模型，难以处理伪装目标的细微边界。该论文的关键解决方案是引入一种基于VLM引导的级联框架，利用SAM（Segment Anything Model）并结合VLM提取的特征作为显式提示，有效引导注意力至伪装区域，同时通过alpha通道将分割结果作为软空间先验，保留全图上下文并提供精确的空间指导，从而实现更准确和上下文感知的分类。
链接: https://arxiv.org/abs/2506.19300
作者: Kai Zhao,Wubang Yuan,Zheng Wang,Guanyi Li,Xiaoqiang Zhu,Deng-ping Fan,Dan Zeng
机构: Shanghai University (上海大学); UCLA (加州大学洛杉矶分校); Nankai University (南开大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Open-Vocabulary Camouflaged Object Segmentation (OVCOS) seeks to segment and classify camouflaged objects from arbitrary categories, presenting unique challenges due to visual ambiguity and unseen this http URL approaches typically adopt a two-stage paradigm: first segmenting objects, then classifying the segmented regions using Vision Language Models (VLMs).However, these methods (1) suffer from a domain gap caused by the mismatch between VLMs’ full-image training and cropped-region inference, and (2) depend on generic segmentation models optimized for well-delineated objects, making them less effective for camouflaged this http URL explicit guidance, generic segmentation models often overlook subtle boundaries, leading to imprecise this http URL this paper,we introduce a novel VLM-guided cascaded framework to address these issues in this http URL segmentation, we leverage the Segment Anything Model (SAM), guided by the this http URL framework uses VLM-derived features as explicit prompts to SAM, effectively directing attention to camouflaged regions and significantly improving localization this http URL classification, we avoid the domain gap introduced by hard this http URL, we treat the segmentation output as a soft spatial prior via the alpha channel, which retains the full image context while providing precise spatial guidance, leading to more accurate and context-aware classification of camouflaged this http URL same VLM is shared across both segmentation and classification to ensure efficiency and semantic this http URL experiments on both OVCOS and conventional camouflaged object segmentation benchmarks demonstrate the clear superiority of our method, highlighting the effectiveness of leveraging rich VLM semantics for both segmentation and classification of camouflaged objects.
zh
[CV-62] HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis
【速读】：该论文旨在解决从长时单目RGB视频中进行具身视图合成（embodied view synthesis, EVS）的问题，传统方法如4D高斯点云和动态NeRF在处理长时间捕获数据时面临训练开销大的挑战。其解决方案的关键在于提出HoliGS框架，通过可逆高斯点云变形网络实现大规模动态环境的准确重建，将场景分解为静态背景与随时间变化的物体，并利用可逆神经流对高斯基元进行全局刚性变换、骨骼驱动的关节运动以及细微非刚性形变，从而实现鲁棒的自由视角新视图渲染。
链接: https://arxiv.org/abs/2506.19291
作者: Xiaoyuan Wang,Yizhou Zhao,Botao Ye,Xiaojun Shan,Weijie Lyu,Lu Qi,Kelvin C.K. Chan,Yinxiao Li,Ming-Hsuan Yang
机构: CMU(卡内基梅隆大学); ETH Zurich(苏黎世联邦理工学院); UC San Diego(加州大学圣地亚哥分校); UC Merced(加州大学默塞德分校); Insta360(Insta360); Google DeepMind(谷歌深度思维)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:We propose HoliGS, a novel deformable Gaussian splatting framework that addresses embodied view synthesis from long monocular RGB videos. Unlike prior 4D Gaussian splatting and dynamic NeRF pipelines, which struggle with training overhead in minute-long captures, our method leverages invertible Gaussian Splatting deformation networks to reconstruct large-scale, dynamic environments accurately. Specifically, we decompose each scene into a static background plus time-varying objects, each represented by learned Gaussian primitives undergoing global rigid transformations, skeleton-driven articulation, and subtle non-rigid deformations via an invertible neural flow. This hierarchical warping strategy enables robust free-viewpoint novel-view rendering from various embodied camera trajectories by attaching Gaussians to a complete canonical foreground shape (\eg, egocentric or third-person follow), which may involve substantial viewpoint changes and interactions between multiple actors. Our experiments demonstrate that \ourmethod~ achieves superior reconstruction quality on challenging datasets while significantly reducing both training and rendering time compared to state-of-the-art monocular deformable NeRFs. These results highlight a practical and scalable solution for EVS in real-world scenarios. The source code will be released.
zh
[CV-63] Da Yu: Towards USV-Based Image Captioning for Waterway Surveillance and Scene Understanding
【速读】：该论文旨在解决水道环境中感知模型缺乏全局语义理解的问题，这限制了大规模监测和结构化日志生成。其解决方案的关键在于引入WaterCaption数据集，这是首个专为水道环境设计的图像描述数据集，提供细粒度、多区域的长文本描述，以及提出Da Yu模型，其中包含一种名为Nano Transformer Adaptor (NTA) 的新型视觉到语言投影器，该投影器在计算效率与视觉特征的全局及细粒度局部建模能力之间实现了有效平衡，从而显著提升了模型生成长文本输出的能力。
链接: https://arxiv.org/abs/2506.19288
作者: Runwei Guan,Ningwei Ouyang,Tianhao Xu,Shaofeng Liang,Wei Dai,Yafeng Sun,Shang Gao,Songning Lai,Shanliang Yao,Xuming Hu,Ryan Wen Liu,Yutao Yue,Hui Xiong
机构: Hong Kong University of Science and Technology (Guangzhou); Hong Kong University of Science and Technology, Hong Kong SAR, China; Xi’an Jiaotong-Liverpool University; China University of Petroleum (East China); University of Science and Technology of China; Yancheng Institute of Technology; Wuhan University of Technology
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注: 14 pages, 13 figures
Abstract:Automated waterway environment perception is crucial for enabling unmanned surface vessels (USVs) to understand their surroundings and make informed decisions. Most existing waterway perception models primarily focus on instance-level object perception paradigms (e.g., detection, segmentation). However, due to the complexity of waterway environments, current perception datasets and models fail to achieve global semantic understanding of waterways, limiting large-scale monitoring and structured log generation. With the advancement of vision-language models (VLMs), we leverage image captioning to introduce WaterCaption, the first captioning dataset specifically designed for waterway environments. WaterCaption focuses on fine-grained, multi-region long-text descriptions, providing a new research direction for visual geo-understanding and spatial scene cognition. Exactly, it includes 20.2k image-text pair data with 1.8 million vocabulary size. Additionally, we propose Da Yu, an edge-deployable multi-modal large language model for USVs, where we propose a novel vision-to-language projector called Nano Transformer Adaptor (NTA). NTA effectively balances computational efficiency with the capacity for both global and fine-grained local modeling of visual features, thereby significantly enhancing the model’s ability to generate long-form textual outputs. Da Yu achieves an optimal balance between performance and efficiency, surpassing state-of-the-art models on WaterCaption and several other captioning benchmarks.
zh
[CV-64] AirV2X: Unified Air-Ground Vehicle-to-Everything Collaboration
【速读】：该论文旨在解决传统基于基础设施的车路协同（V2X）系统在农村和郊区环境中因部署成本高而产生的“未覆盖危险区域”问题。其解决方案的关键在于利用无人机（UAVs）作为固定路侧单元（RSUs）的灵活替代或补充，通过无人机提供的俯视视角、动态定位能力以及较低的部署成本，提升多车辆协同驾驶的感知性能与覆盖范围。
链接: https://arxiv.org/abs/2506.19283
作者: Xiangbo Gao,Yuheng Wu,Xuewen Luo,Keshu Wu,Xinghao Chen,Yuping Wang,Chenxi Liu,Yang Zhou,Zhengzhong Tu
机构: Texas A&M University (德克萨斯A&M大学); KAIST (韩国科学技术院); The University of Utah (犹他大学); University of Washington (华盛顿大学); University of Michigan (密歇根大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO)
备注:
Abstract:While multi-vehicular collaborative driving demonstrates clear advantages over single-vehicle autonomy, traditional infrastructure-based V2X systems remain constrained by substantial deployment costs and the creation of “uncovered danger zones” in rural and suburban areas. We present AirV2X-Perception, a large-scale dataset that leverages Unmanned Aerial Vehicles (UAVs) as a flexible alternative or complement to fixed Road-Side Units (RSUs). Drones offer unique advantages over ground-based perception: complementary bird’s-eye-views that reduce occlusions, dynamic positioning capabilities that enable hovering, patrolling, and escorting navigation rules, and significantly lower deployment costs compared to fixed infrastructure. Our dataset comprises 6.73 hours of drone-assisted driving scenarios across urban, suburban, and rural environments with varied weather and lighting conditions. The AirV2X-Perception dataset facilitates the development and standardized evaluation of Vehicle-to-Drone (V2D) algorithms, addressing a critical gap in the rapidly expanding field of aerial-assisted autonomous driving systems. The dataset and development kits are open-sourced at this https URL.
zh
[CV-65] Self-Paced Collaborative and Adversarial Network for Unsupervised Domain Adaptation
【速读】：该论文旨在解决无监督域适应（Unsupervised Domain Adaptation, UDA）中的域分布差异问题，即如何在源域和目标域数据分布不一致的情况下，提升模型在目标域上的泛化能力。其解决方案的关键在于提出了一种名为协作与对抗网络（Collaborative and Adversarial Network, CAN）的方法，通过结合域协作学习和域对抗学习策略，实现域不变特征表示与域特定特征表示的联合学习，从而有效减少域间分布差异并保持目标域的判别性。此外，为进一步增强目标域的判别能力，作者还提出了自步CAN（Self-Paced CAN, SPCAN），通过渐进式选择伪标签目标样本进行再训练，进一步提升模型性能。
链接: https://arxiv.org/abs/2506.19267
作者: Weichen Zhang,Dong Xu,Wanli Ouyang,Wen Li
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:This paper proposes a new unsupervised domain adaptation approach called Collaborative and Adversarial Network (CAN), which uses the domain-collaborative and domain-adversarial learning strategy for training the neural network. The domain-collaborative learning aims to learn domain-specific feature representation to preserve the discriminability for the target domain, while the domain adversarial learning aims to learn domain-invariant feature representation to reduce the domain distribution mismatch between the source and target domains. We show that these two learning strategies can be uniformly formulated as domain classifier learning with positive or negative weights on the losses. We then design a collaborative and adversarial training scheme, which automatically learns domain-specific representations from lower blocks in CNNs through collaborative learning and domain-invariant representations from higher blocks through adversarial learning. Moreover, to further enhance the discriminability in the target domain, we propose Self-Paced CAN (SPCAN), which progressively selects pseudo-labeled target samples for re-training the classifiers. We employ a self-paced learning strategy to select pseudo-labeled target samples in an easy-to-hard fashion. Comprehensive experiments on different benchmark datasets, Office-31, ImageCLEF-DA, and VISDA-2017 for the object recognition task, and UCF101-10 and HMDB51-10 for the video action recognition task, show our newly proposed approaches achieve the state-of-the-art performance, which clearly demonstrates the effectiveness of our proposed approaches for unsupervised domain adaptation.
zh
[CV-66] 3D-SSM: A Novel 3D Selective Scan Module for Remote Sensing Change Detection
【速读】：该论文旨在解决现有基于Mamba的遥感变化检测方法在捕捉图像通道间长距离依赖关系方面的局限性，从而提升特征表示能力。其解决方案的关键在于提出一种三维选择性扫描模块（3D-SSM），该模块从空间平面和通道视角捕获全局信息，并结合时空交互模块（SIM）与多分支特征提取模块（MBFEM），分别实现跨时相特征融合和多域特征整合，以增强对细微变化的检测能力。
链接: https://arxiv.org/abs/2506.19263
作者: Rui Huang,Jincheng Zeng,Sen Gao,Yan Xing
机构: Civil Aviation University of China (中国民航大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Existing Mamba-based approaches in remote sensing change detection have enhanced scanning models, yet remain limited by their inability to capture long-range dependencies between image channels effectively, which restricts their feature representation capabilities. To address this limitation, we propose a 3D selective scan module (3D-SSM) that captures global information from both the spatial plane and channel perspectives, enabling a more comprehensive understanding of the this http URL on the 3D-SSM, we present two key components: a spatiotemporal interaction module (SIM) and a multi-branch feature extraction module (MBFEM). The SIM facilitates bi-temporal feature integration by enabling interactions between global and local features across images from different time points, thereby enhancing the detection of subtle changes. Meanwhile, the MBFEM combines features from the frequency domain, spatial domain, and 3D-SSM to provide a rich representation of contextual information within the image. Our proposed method demonstrates favourable performance compared to state-of-the-art change detection methods on five benchmark datasets through extensive experiments. Code is available at this https URL
zh
[CV-67] Automated Image Recognition Framework
【速读】：该论文试图解决深度学习模型在特定任务中因缺乏相关数据集而面临的数据收集与标注难题，尤其是在处理新颖或敏感主题时所遇到的时间和资源挑战。其解决方案的关键在于提出一种名为Automated Image Recognition (AIR)的框架，该框架利用生成式AI（Generative AI）自动生成高质量、预标注的数据集，并通过两个主要的数据合成过程——AIR-Gen和AIR-Aug——实现数据生成与增强，从而减少对人工标注的依赖并提升模型性能。
链接: https://arxiv.org/abs/2506.19261
作者: Quang-Binh Nguyen,Trong-Vu Hoang,Ngoc-Do Tran,Tam V. Nguyen,Minh-Triet Tran,Trung-Nghia Le
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: ICCCI 2025
Abstract:While the efficacy of deep learning models heavily relies on data, gathering and annotating data for specific tasks, particularly when addressing novel or sensitive subjects lacking relevant datasets, poses significant time and resource challenges. In response to this, we propose a novel Automated Image Recognition (AIR) framework that harnesses the power of generative AI. AIR empowers end-users to synthesize high-quality, pre-annotated datasets, eliminating the necessity for manual labeling. It also automatically trains deep learning models on the generated datasets with robust image recognition performance. Our framework includes two main data synthesis processes, AIR-Gen and AIR-Aug. The AIR-Gen enables end-users to seamlessly generate datasets tailored to their specifications. To improve image quality, we introduce a novel automated prompt engineering module that leverages the capabilities of large language models. We also introduce a distribution adjustment algorithm to eliminate duplicates and outliers, enhancing the robustness and reliability of generated datasets. On the other hand, the AIR-Aug enhances a given dataset, thereby improving the performance of deep classifier models. AIR-Aug is particularly beneficial when users have limited data for specific tasks. Through comprehensive experiments, we demonstrated the efficacy of our generated data in training deep learning models and showcased the system’s potential to provide image recognition models for a wide range of objects. We also conducted a user study that achieved an impressive score of 4.4 out of 5.0, underscoring the AI community’s positive perception of AIR.
zh
[CV-68] MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Models
【速读】：该论文试图解决生成式 AI (Generative AI) 在多模态安全对齐方面的问题，特别是针对视觉-语言模型（Vision-Language Models, VLMs）在面对有害多模态提示时可能引发的伦理或安全风险。现有安全对齐方法主要针对单模态语言模型设计，无法有效应对多模态输入带来的复杂威胁，且当前安全数据集缺乏细粒度、基于政策的推理能力。解决方案的关键在于提出 MSR-Align 数据集，该数据集通过强调多模态多样性、基于政策的推理以及严格的多模态评判质量过滤，支持在视觉和文本模态上进行细致的安全策略推理，从而提升 VLMs 对文本和视觉-语言越狱攻击的鲁棒性，同时保持或增强其通用推理能力。
链接: https://arxiv.org/abs/2506.19257
作者: Yinan Xia,Yilei Jiang,Yingshui Tan,Xiaoyong Zhu,Xiangyu Yue,Bo Zheng
机构: Future Lab, Alibaba Group (未来实验室，阿里巴巴集团); CUHK MMLab (香港中文大学多媒体实验室)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Vision-Language Models (VLMs) have achieved remarkable progress in multimodal reasoning tasks through enhanced chain-of-thought capabilities. However, this advancement also introduces novel safety risks, as these models become increasingly vulnerable to harmful multimodal prompts that can trigger unethical or unsafe behaviors. Existing safety alignment approaches, primarily designed for unimodal language models, fall short in addressing the complex and nuanced threats posed by multimodal inputs. Moreover, current safety datasets lack the fine-grained, policy-grounded reasoning required to robustly align reasoning-capable VLMs. In this work, we introduce MSR-Align, a high-quality Multimodal Safety Reasoning dataset tailored to bridge this gap. MSR-Align supports fine-grained, deliberative reasoning over standardized safety policies across both vision and text modalities. Our data generation pipeline emphasizes multimodal diversity, policy-grounded reasoning, and rigorous quality filtering using strong multimodal judges. Extensive experiments demonstrate that fine-tuning VLMs on MSR-Align substantially improves robustness against both textual and vision-language jailbreak attacks, while preserving or enhancing general reasoning performance. MSR-Align provides a scalable and effective foundation for advancing the safety alignment of reasoning-capable VLMs. Our dataset is made publicly available at this https URL.
zh
[CV-69] Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsification
【速读】：该论文旨在解决长视频理解中由于高内存和计算成本导致的处理难题，以实现强性能与高效率的平衡。其解决方案的关键在于提出Video-XL-2框架，该框架基于任务感知的键值（Key-Value）稀疏化技术，通过分块预填充和双层级键值解码两个核心步骤，有效降低计算和内存开销，并提升模型对细粒度信息的捕捉能力。
链接: https://arxiv.org/abs/2506.19225
作者: Minghao Qin,Xiangrui Liu,Zhengyang Liang,Yan Shu,Huaying Yuan,Juenjie Zhou,Shitao Xiao,Bo Zhao,Zheng Liu
机构: Beijing Academy of Artificial Intelligence (北京人工智能研究院); Shanghai Jiao Tong University (上海交通大学); University of Trento (特伦托大学); Renmin University of China (中国人民大学); Beijing University of Posts and Telecommunications (北京邮电大学); Hong Kong Polytechnic University (香港理工大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 12 pages, 5 Figure, 3 Table
Abstract:Multi-modal large language models (MLLMs) models have made significant progress in video understanding over the past few years. However, processing long video inputs remains a major challenge due to high memory and computational costs. This makes it difficult for current models to achieve both strong performance and high efficiency in long video understanding. To address this challenge, we propose Video-XL-2, a novel MLLM that delivers superior cost-effectiveness for long-video understanding based on task-aware KV sparsification. The proposed framework operates with two key steps: chunk-based pre-filling and bi-level key-value decoding. Chunk-based pre-filling divides the visual token sequence into chunks, applying full attention within each chunk and sparse attention across chunks. This significantly reduces computational and memory overhead. During decoding, bi-level key-value decoding selectively reloads either dense or sparse key-values for each chunk based on its relevance to the task. This approach further improves memory efficiency and enhances the model’s ability to capture fine-grained information. Video-XL-2 achieves state-of-the-art performance on various long video understanding benchmarks, outperforming existing open-source lightweight models. It also demonstrates exceptional efficiency, capable of processing over 10,000 frames on a single NVIDIA A100 (80GB) GPU and thousands of frames in just a few seconds.
zh
[CV-70] MedErr-CT: A Visual Question Answering Benchmark for Identifying and Correcting Errors in CT Reports CVPR2025
【速读】：该论文旨在解决医学影像报告中诊断错误识别与修正的问题，特别是针对CT报告中的错误类型进行评估。其解决方案的关键在于引入MedErr-CT基准，该基准通过视觉问答（VQA）框架评估多模态大语言模型（MLLMs）在识别和修正CT报告中错误方面的能力，涵盖六种错误类别，并设置三个任务层级（分类、检测、修正），从而为提升医学MLLM的可靠性与临床适用性提供量化评估工具。
链接: https://arxiv.org/abs/2506.19217
作者: Sunggu Kyung,Hyungbin Park,Jinyoung Seo,Jimin Sung,Jihyun Kim,Dongyeong Kim,Wooyoung Jo,Yoojin Nam,Sangah Park,Taehee Kwon,Sang Min Lee,Namkug Kim
机构: University of Ulsan College of Medicine (蔚山大学医学院)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: 14 pages, 5 figures, submitted to CVPR 2025
Abstract:Computed Tomography (CT) plays a crucial role in clinical diagnosis, but the growing demand for CT examinations has raised concerns about diagnostic errors. While Multimodal Large Language Models (MLLMs) demonstrate promising comprehension of medical knowledge, their tendency to produce inaccurate information highlights the need for rigorous validation. However, existing medical visual question answering (VQA) benchmarks primarily focus on simple visual recognition tasks, lacking clinical relevance and failing to assess expert-level knowledge. We introduce MedErr-CT, a novel benchmark for evaluating medical MLLMs’ ability to identify and correct errors in CT reports through a VQA framework. The benchmark includes six error categories - four vision-centric errors (Omission, Insertion, Direction, Size) and two lexical error types (Unit, Typo) - and is organized into three task levels: classification, detection, and correction. Using this benchmark, we quantitatively assess the performance of state-of-the-art 3D medical MLLMs, revealing substantial variation in their capabilities across different error types. Our benchmark contributes to the development of more reliable and clinically applicable MLLMs, ultimately helping reduce diagnostic errors and improve accuracy in clinical practice. The code and datasets are available at this https URL.
zh
[CV-71] Ancient Script Image Recognition and Processing: A Review
【速读】：该论文旨在解决古代文字图像识别中的复杂问题，包括数据分布不均衡、图像退化以及不同文字系统间的差异性带来的挑战。其解决方案的关键在于通过深度学习技术，结合脚本类型分类和识别方法分析，提出适用于多种文字系统的通用策略，并引入少样本学习和噪声鲁棒技术等先进方法，以提升古代文字的自动识别与解读能力。
链接: https://arxiv.org/abs/2506.19208
作者: Xiaolei Diao,Rite Bo,Yanling Xiao,Lida Shi,Zhihan Zhou,Hao Xu,Chuntao Li,Xiongfeng Tang,Massimo Poesio,Cédric M. John,Daqian Shi
机构: Queen Mary University of London (伦敦玛丽女王大学); Jilin University (吉林大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Ancient scripts, e.g., Egyptian hieroglyphs, Oracle Bone Inscriptions, and Ancient Greek inscriptions, serve as vital carriers of human civilization, embedding invaluable historical and cultural information. Automating ancient script image recognition has gained importance, enabling large-scale interpretation and advancing research in archaeology and digital humanities. With the rise of deep learning, this field has progressed rapidly, with numerous script-specific datasets and models proposed. While these scripts vary widely, spanning phonographic systems with limited glyphs to logographic systems with thousands of complex symbols, they share common challenges and methodological overlaps. Moreover, ancient scripts face unique challenges, including imbalanced data distribution and image degradation, which have driven the development of various dedicated methods. This survey provides a comprehensive review of ancient script image recognition methods. We begin by categorizing existing studies based on script types and analyzing respective recognition methods, highlighting both their differences and shared strategies. We then focus on challenges unique to ancient scripts, systematically examining their impact and reviewing recent solutions, including few-shot learning and noise-robust techniques. Finally, we summarize current limitations and outline promising future directions. Our goal is to offer a structured, forward-looking perspective to support ongoing advancements in the recognition, interpretation, and decipherment of ancient scripts.
zh
[CV-72] OpenWildlife: Open-Vocabulary Multi-Species Wildlife Detector for Geographically-Diverse Aerial Imagery
【速读】：该论文旨在解决多物种识别在多样化航空影像中的泛化能力不足问题，现有自动化方法由于分类学覆盖范围有限和模型架构僵化，难以适应不同物种和环境。其解决方案的关键在于引入OpenWildlife（OW），该系统利用语言感知嵌入和对Grounding-DINO框架的创新适配，通过自然语言输入实现陆地和海洋环境中物种的识别，同时结合k近邻与广度优先搜索的高效检索算法，显著提升了检测效率与覆盖率。
链接: https://arxiv.org/abs/2506.19204
作者: Muhammed Patel,Javier Noa Turnes,Jayden Hsiao,Linlin Xu,David Clausi
机构: University of Waterloo (滑铁卢大学); University of Calgary (卡尔加里大学)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:We introduce OpenWildlife (OW), an open-vocabulary wildlife detector designed for multi-species identification in diverse aerial imagery. While existing automated methods perform well in specific settings, they often struggle to generalize across different species and environments due to limited taxonomic coverage and rigid model architectures. In contrast, OW leverages language-aware embeddings and a novel adaptation of the Grounding-DINO framework, enabling it to identify species specified through natural language inputs across both terrestrial and marine environments. Trained on 15 datasets, OW outperforms most existing methods, achieving up to \textbf0.981 mAP50 with fine-tuning and \textbf0.597 mAP50 on seven datasets featuring novel species. Additionally, we introduce an efficient search algorithm that combines k-nearest neighbors and breadth-first search to prioritize areas where social species are likely to be found. This approach captures over \textbf95% of species while exploring only \textbf33% of the available images. To support reproducibility, we publicly release our source code and dataset splits, establishing OW as a flexible, cost-effective solution for global biodiversity assessments.
zh
[CV-73] MOSCARD – Causal Reasoning and De-confounding for Multimodal Opportunistic Screening of Cardiovascular Adverse Events
【速读】：该论文旨在解决重大不良心血管事件（MACE）风险预测中因单一模态数据限制和采样偏差导致的准确性不足问题。其解决方案的关键在于提出一种新型的多模态因果推理框架——MOSCARD，通过联合注意机制对齐胸部X光片（CXR）与12导联心电图（ECG）数据，并结合因果推理与双反向传播图以消除偏倚和混杂因素，从而实现更全面的风险评估。
链接: https://arxiv.org/abs/2506.19174
作者: Jialu Pi,Juan Maria Farina,Rimita Lahiri,Jiwoong Jeong,Archana Gurudu,Hyung-Bok Park,Chieh-Ju Chao,Chadi Ayoub,Reza Arsanjani,Imon Banerjee
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Major Adverse Cardiovascular Events (MACE) remain the leading cause of mortality globally, as reported in the Global Disease Burden Study 2021. Opportunistic screening leverages data collected from routine health check-ups and multimodal data can play a key role to identify at-risk individuals. Chest X-rays (CXR) provide insights into chronic conditions contributing to major adverse cardiovascular events (MACE), while 12-lead electrocardiogram (ECG) directly assesses cardiac electrical activity and structural abnormalities. Integrating CXR and ECG could offer a more comprehensive risk assessment than conventional models, which rely on clinical scores, computed tomography (CT) measurements, or biomarkers, which may be limited by sampling bias and single modality constraints. We propose a novel predictive modeling framework - MOSCARD, multimodal causal reasoning with co-attention to align two distinct modalities and simultaneously mitigate bias and confounders in opportunistic risk estimation. Primary technical contributions are - (i) multimodal alignment of CXR with ECG guidance; (ii) integration of causal reasoning; (iii) dual back-propagation graph for de-confounding. Evaluated on internal, shift data from emergency department (ED) and external MIMIC datasets, our model outperformed single modality and state-of-the-art foundational models - AUC: 0.75, 0.83, 0.71 respectively. Proposed cost-effective opportunistic screening enables early intervention, improving patient outcomes and reducing disparities.
zh
[CV-74] PRISM: Perceptual Recognition for Identifying Standout Moments in Human-Centric Keyframe Extraction
【速读】：该论文试图解决在线视频中识别最具影响力的“ standout ”时刻的问题，这对于内容审核、摘要生成和法医学分析至关重要。解决方案的关键在于提出PRISM（Perceptual Recognition for Identifying Standout Moments），这是一个轻量级且与人类感知对齐的帧提取框架。PRISM在CIELAB颜色空间中运行，并利用感知颜色差异度量来识别符合人类视觉敏感性的帧，其核心优势在于无需训练、可解释性强且计算效率高，适用于实时和资源受限的环境。
链接: https://arxiv.org/abs/2506.19168
作者: Mert Can Cakmak,Nitin Agarwal,Diwash Poudel
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Online videos play a central role in shaping political discourse and amplifying cyber social threats such as misinformation, propaganda, and radicalization. Detecting the most impactful or “standout” moments in video content is crucial for content moderation, summarization, and forensic analysis. In this paper, we introduce PRISM (Perceptual Recognition for Identifying Standout Moments), a lightweight and perceptually-aligned framework for keyframe extraction. PRISM operates in the CIELAB color space and uses perceptual color difference metrics to identify frames that align with human visual sensitivity. Unlike deep learning-based approaches, PRISM is interpretable, training-free, and computationally efficient, making it well suited for real-time and resource-constrained environments. We evaluate PRISM on four benchmark datasets: BBC, TVSum, SumMe, and ClipShots, and demonstrate that it achieves strong accuracy and fidelity while maintaining high compression ratios. These results highlight PRISM’s effectiveness in both structured and unstructured video content, and its potential as a scalable tool for analyzing and moderating harmful or politically sensitive media in online platforms.
zh
[CV-75] Lightweight RGB-T Tracking with Mobile Vision Transformers
【速读】：该论文旨在解决单模态目标跟踪（如仅使用RGB图像）在低光照和恶劣天气等挑战性成像条件下性能下降的问题，其解决方案的关键是提出一种基于Mobile Vision Transformers（MobileViT）的轻量级RGB-T跟踪算法。该算法引入了一种渐进式融合框架，通过可分离注意力机制联合学习模板区域与搜索区域之间的模态内和模态间交互，从而生成有效的特征表示，实现更精确的目标定位，同时保持模型规模小和推理速度快的优势。
链接: https://arxiv.org/abs/2506.19154
作者: Mahdi Falaki,Maria A. Amer
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Single-modality object tracking (e.g., RGB-only) encounters difficulties in challenging imaging conditions, such as low illumination and adverse weather conditions. To solve this, multimodal tracking (e.g., RGB-T models) aims to leverage complementary data such as thermal infrared features. While recent Vision Transformer-based multimodal trackers achieve strong performance, they are often computationally expensive due to large model sizes. In this work, we propose a novel lightweight RGB-T tracking algorithm based on Mobile Vision Transformers (MobileViT). Our tracker introduces a progressive fusion framework that jointly learns intra-modal and inter-modal interactions between the template and search regions using separable attention. This design produces effective feature representations that support more accurate target localization while achieving a small model size and fast inference speed. Compared to state-of-the-art efficient multimodal trackers, our model achieves comparable accuracy while offering significantly lower parameter counts (less than 4 million) and the fastest GPU inference speed of 122 frames per second. This paper is the first to propose a tracker using Mobile Vision Transformers for RGB-T tracking and multimodal tracking at large. Tracker code and model weights will be made publicly available upon acceptance.
zh
[CV-76] SOF: Sorted Opacity Fields for Fast Unbounded Surface Reconstruction
【速读】：该论文试图解决在大规模、无界环境中从3D高斯表示中准确提取表面的问题，现有方法依赖于近似深度估计和全局排序启发式，容易引入伪影并限制重建网格的保真度。其解决方案的关键在于提出一种名为Sorted Opacity Fields (SOF) 的方法，通过引入分层重新排序和鲁棒的高斯深度公式，更好地与水平集对齐，并结合基于不透明度场的水平集正则化器以及促进几何一致原始形状的损失函数，同时开发了一种针对不透明度公式的并行化Marching Tetrahedra算法，显著提升了网格生成速度和精度。
链接: https://arxiv.org/abs/2506.19139
作者: Lukas Radl,Felix Windisch,Thomas Deixelberger,Jozef Hladky,Michael Steiner,Dieter Schmalstieg,Markus Steinberger
机构: Graz University of TechnologyAustria; Huawei TechnologiesAustria; University of StuttgartGermany
类目: Graphics (cs.GR); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Recent advances in 3D Gaussian representations have significantly improved the quality and efficiency of image-based scene reconstruction. Their explicit nature facilitates real-time rendering and fast optimization, yet extracting accurate surfaces - particularly in large-scale, unbounded environments - remains a difficult task. Many existing methods rely on approximate depth estimates and global sorting heuristics, which can introduce artifacts and limit the fidelity of the reconstructed mesh. In this paper, we present Sorted Opacity Fields (SOF), a method designed to recover detailed surfaces from 3D Gaussians with both speed and precision. Our approach improves upon prior work by introducing hierarchical resorting and a robust formulation of Gaussian depth, which better aligns with the level-set. To enhance mesh quality, we incorporate a level-set regularizer operating on the opacity field and introduce losses that encourage geometrically-consistent primitive shapes. In addition, we develop a parallelized Marching Tetrahedra algorithm tailored to our opacity formulation, reducing meshing time by up to an order of magnitude. As demonstrated by our quantitative evaluation, SOF achieves higher reconstruction accuracy while cutting total processing time by more than a factor of three. These results mark a step forward in turning efficient Gaussian-based rendering into equally efficient geometry extraction.
zh
[CV-77] PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Scenes
【速读】：该论文旨在解决大规模3D语义场景生成中传统基于体素（voxel）表示方法存在的内存消耗大、分辨率固定及编辑困难等问题。其解决方案的关键在于引入PrITTI框架，该框架以几何原语（primitives）作为基础元素，结合栅格化地面表面与向量化3D原语的混合表示方式，实现可组合、可控且可编辑的3D语义场景布局生成。此外，通过引入基于Cholesky的稳定参数化方法，有效解决了传统编码方法中的方向歧义问题。
链接: https://arxiv.org/abs/2506.19117
作者: Christina Ourania Tze,Daniel Dauner,Yiyi Liao,Dzmitry Tsishkou,Andreas Geiger
机构: University of Tübingen, Tübingen AI Center (图宾根大学，图宾根人工智能中心); Zhejiang University (浙江大学); Noah’s Ark Lab, Huawei (诺亚方舟实验室，华为)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: Project page: this https URL
Abstract:Large-scale 3D semantic scene generation has predominantly relied on voxel-based representations, which are memory-intensive, bound by fixed resolutions, and challenging to edit. In contrast, primitives represent semantic entities using compact, coarse 3D structures that are easy to manipulate and compose, making them an ideal representation for this task. In this paper, we introduce PrITTI, a latent diffusion-based framework that leverages primitives as the main foundational elements for generating compositional, controllable, and editable 3D semantic scene layouts. Our method adopts a hybrid representation, modeling ground surfaces in a rasterized format while encoding objects as vectorized 3D primitives. This decomposition is also reflected in a structured latent representation that enables flexible scene manipulation of ground and object components. To overcome the orientation ambiguities in conventional encoding methods, we introduce a stable Cholesky-based parameterization that jointly encodes object size and orientation. Experiments on the KITTI-360 dataset show that PrITTI outperforms a voxel-based baseline in generation quality, while reducing memory requirements by up to 3\times . In addition, PrITTI enables direct instance-level manipulation of objects in the scene and supports a range of downstream applications, including scene inpainting, outpainting, and photo-realistic street-view synthesis.
zh
[CV-78] Inverse-and-Edit: Effective and Fast Image Editing by Cycle Consistency Models
【速读】：该论文试图解决扩散模型在图像编辑中计算成本高且蒸馏模型编辑能力受限的问题，其核心挑战在于低质量的图像逆向重建影响了编辑的准确性。解决方案的关键在于提出一种基于一致性模型（Consistency Models）的框架，通过引入循环一致性优化策略，显著提升重建精度，并实现编辑灵活性与内容保真度之间的可控权衡。该方法在保持高编辑质量的同时，大幅减少了所需的迭代步骤，从而提高了效率。
链接: https://arxiv.org/abs/2506.19103
作者: Ilia Beletskii,Andrey Kuznetsov,Aibek Alanov
机构: HSE University (高等经济大学); AIRI (人工智能研究研究院); Sber (斯伯尔); Innopolis (因诺波利斯)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注: The code of our method is available on GitHub at this https URL
Abstract:Recent advances in image editing with diffusion models have achieved impressive results, offering fine-grained control over the generation process. However, these methods are computationally intensive because of their iterative nature. While distilled diffusion models enable faster inference, their editing capabilities remain limited, primarily because of poor inversion quality. High-fidelity inversion and reconstruction are essential for precise image editing, as they preserve the structural and semantic integrity of the source image. In this work, we propose a novel framework that enhances image inversion using consistency models, enabling high-quality editing in just four steps. Our method introduces a cycle-consistency optimization strategy that significantly improves reconstruction accuracy and enables a controllable trade-off between editability and content preservation. We achieve state-of-the-art performance across various image editing tasks and datasets, demonstrating that our method matches or surpasses full-step diffusion models while being substantially more efficient. The code of our method is available on GitHub at this https URL.
zh
[CV-79] RareSpot: Spotting Small and Rare Wildlife in Aerial Imagery with Multi-Scale Consistency and Context-Aware Augmentation CVPR2025
【速读】：该论文旨在解决在航拍图像中自动检测小型且稀有野生动物的技术难题，以支持有效的生态保护。针对 Prairie dog（普氏草原犬）等物种因体型小、分布稀疏及视觉特征细微而导致现有检测方法效果不佳的问题，本文提出 RareSpot 框架，其关键在于融合多尺度一致性学习与上下文感知增强技术。多尺度一致性学习通过特征金字塔的结构化对齐提升细粒度目标表征并缓解尺度相关特征丢失，而上下文感知增强则通过将难以检测的样本嵌入真实环境背景中来生成具有挑战性的训练实例，从而显著提升模型的精度和召回率。
链接: https://arxiv.org/abs/2506.19087
作者: Bowen Zhang,Jesse T. Boulerice,Nikhil Kuniyil,Charvi Mendiratta,Satish Kumar,Hila Shamon,B.S. Manjunath
机构: University of California, Santa Barbara (加州大学圣塔芭芭拉分校); Smithsonian National Zoo and Conservation Biology Institute (史密森尼国家动物园与保护生物学研究所); Stanford University (斯坦福大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注: Accepted to the CVPR 2025 Workshop on Computer Vision for Animal Behavior Tracking and Modeling (CV4Animals)
Abstract:Automated detection of small and rare wildlife in aerial imagery is crucial for effective conservation, yet remains a significant technical challenge. Prairie dogs exemplify this issue: their ecological importance as keystone species contrasts sharply with their elusive presence–marked by small size, sparse distribution, and subtle visual features–which undermines existing detection approaches. To address these challenges, we propose RareSpot, a robust detection framework integrating multi-scale consistency learning and context-aware augmentation. Our multi-scale consistency approach leverages structured alignment across feature pyramids, enhancing fine-grained object representation and mitigating scale-related feature loss. Complementarily, context-aware augmentation strategically synthesizes challenging training instances by embedding difficult-to-detect samples into realistic environmental contexts, significantly boosting model precision and recall. Evaluated on an expert-annotated prairie dog drone imagery benchmark, our method achieves state-of-the-art performance, improving detection accuracy by over 35% compared to baseline methods. Importantly, it generalizes effectively across additional wildlife datasets, demonstrating broad applicability. The RareSpot benchmark and approach not only support critical ecological monitoring but also establish a new foundation for detecting small, rare species in complex aerial scenes.
zh
[CV-80] Reading Smiles: Proxy Bias in Foundation Models for Facial Emotion Recognition
【速读】：该论文试图解决生成式 AI (Generative AI) 在情感计算 (Affective Computing, AC) 中依赖的视觉线索是否具有心理基础或仅是表面学习的问题。其解决方案的关键在于通过在 AffectNet 数据集的牙齿标注子集上对不同规模的视觉语言模型 (Vision Language Models, VLMs) 进行基准测试，结合对性能最佳模型 GPT-4o 的结构化内省，揭示其情感推理主要依赖于如眉毛位置等面部属性，并验证其情感预测在效价-唤醒度维度上的内部一致性。
链接: https://arxiv.org/abs/2506.19079
作者: Iosif Tsangko,Andreas Triantafyllopoulos,Adem Abdelmoula,Adria Mallol-Ragolta,Bjoern W. Schuller
机构: Technical University of Munich (慕尼黑工业大学); Munich Center for Machine Learning (慕尼黑机器学习中心); Imperial College London (帝国理工学院); Munich Data Science Institute (慕尼黑数据科学研究所)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
备注:
Abstract:Foundation Models (FMs) are rapidly transforming Affective Computing (AC), with Vision Language Models (VLMs) now capable of recognising emotions in zero shot settings. This paper probes a critical but underexplored question: what visual cues do these models rely on to infer affect, and are these cues psychologically grounded or superficially learnt? We benchmark varying scale VLMs on a teeth annotated subset of AffectNet dataset and find consistent performance shifts depending on the presence of visible teeth. Through structured introspection of, the best-performing model, i.e., GPT-4o, we show that facial attributes like eyebrow position drive much of its affective reasoning, revealing a high degree of internal consistency in its valence-arousal predictions. These patterns highlight the emergent nature of FMs behaviour, but also reveal risks: shortcut learning, bias, and fairness issues especially in sensitive domains like mental health and education.
zh
[CV-81] LEGATO: Large-scale End-to-end Generalizable Approach to Typeset OMR
【速读】：该论文试图解决光学音乐识别（Optical Music Recognition, OMR）中端到端模型性能不足的问题，特别是针对全页或多页排版乐谱的识别以及生成可读性更强的ABC符号乐谱格式。解决方案的关键在于提出Legato，这是一个基于预训练视觉编码器和在超过214K张图像数据集上训练的ABC解码器的端到端Transformer模型，使其具备跨多种排版乐谱的强大泛化能力，并在多个数据集上实现了最先进的性能。
链接: https://arxiv.org/abs/2506.19065
作者: Guang Yang,Victoria Ebert,Nazif Tamer,Luiza Pozzobon,Noah A. Smith
机构: University of Washington (华盛顿大学); Allen Institute for AI (艾伦人工智能研究所)
类目: Computer Vision and Pattern Recognition (cs.CV); Digital Libraries (cs.DL)
备注:
Abstract:We propose Legato, a new end-to-end transformer model for optical music recognition (OMR). Legato is the first large-scale pretrained OMR model capable of recognizing full-page or multi-page typeset music scores and the first to generate documents in ABC notation, a concise, human-readable format for symbolic music. Bringing together a pretrained vision encoder with an ABC decoder trained on a dataset of more than 214K images, our model exhibits the strong ability to generalize across various typeset scores. We conduct experiments on a range of datasets and demonstrate that our model achieves state-of-the-art performance. Given the lack of a standardized evaluation for end-to-end OMR, we comprehensively compare our model against the previous state of the art using a diverse set of metrics.
zh
[CV-82] Orthogonal Projection Subspace to Aggregate Online Prior-knowledge for Continual Test-time Adaptation
【速读】：该论文旨在解决持续测试时适应（Continual Test Time Adaptation, CTTA）中模型在面对不断变化的目标分布时所面临的灾难性遗忘和错误累积问题。其解决方案的关键在于提出一种名为OoPk的新框架，该框架通过正交投影子空间来聚合在线先验知识。具体而言，首先通过正交投影方式构建调优子空间，使模型能够在保留预训练源模型知识完整性的前提下适应新领域，从而缓解灾难性遗忘；其次，引入一种激进但高效的图像掩码策略，模拟潜在的目标动态，增强学生模型的领域适应能力，并逐步优化教师模型的知识，确保高质量伪标签并减少错误累积。
链接: https://arxiv.org/abs/2506.19022
作者: Jinlong Li,Dong Zhao,Qi Zang,Zequn Jie,Lin Ma,Nicu Sebe
机构: University of Trento (特伦托大学); Meituan Inc. (美团公司)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Continual Test Time Adaptation (CTTA) is a task that requires a source pre-trained model to continually adapt to new scenarios with changing target distributions. Existing CTTA methods primarily focus on mitigating the challenges of catastrophic forgetting and error accumulation. Though there have been emerging methods based on forgetting adaptation with parameter-efficient fine-tuning, they still struggle to balance competitive performance and efficient model adaptation, particularly in complex tasks like semantic segmentation. In this paper, to tackle the above issues, we propose a novel pipeline, Orthogonal Projection Subspace to aggregate online Prior-knowledge, dubbed OoPk. Specifically, we first project a tuning subspace orthogonally which allows the model to adapt to new domains while preserving the knowledge integrity of the pre-trained source model to alleviate catastrophic forgetting. Then, we elaborate an online prior-knowledge aggregation strategy that employs an aggressive yet efficient image masking strategy to mimic potential target dynamism, enhancing the student model’s domain adaptability. This further gradually ameliorates the teacher model’s knowledge, ensuring high-quality pseudo labels and reducing error accumulation. We demonstrate our method with extensive experiments that surpass previous CTTA methods and achieve competitive performances across various continual TTA benchmarks in semantic segmentation tasks.
zh
[CV-83] Diffusion Transformer-to-Mamba Distillation for High-Resolution Image Generation
【速读】：该论文试图解决扩散变换器（DiT）中自注意力机制带来的二次计算复杂度问题，这在高分辨率图像生成中导致了显著的计算成本。其解决方案的关键在于引入了扩散变压器到Mamba的蒸馏（T2MD），通过层级教师强制和基于特征的知识蒸馏方法，降低从头训练状态空间模型的难度和成本，从而实现高效且具有全局依赖性的模型转换。
链接: https://arxiv.org/abs/2506.18999
作者: Yuan Yao,Yicong Hong,Difan Liu,Long Mai,Feng Liu,Jiebo Luo
机构: University of Rochester (罗彻斯特大学); Adobe Research (Adobe 研究院)
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:The quadratic computational complexity of self-attention in diffusion transformers (DiT) introduces substantial computational costs in high-resolution image generation. While the linear-complexity Mamba model emerges as a potential alternative, direct Mamba training remains empirically challenging. To address this issue, this paper introduces diffusion transformer-to-mamba distillation (T2MD), forming an efficient training pipeline that facilitates the transition from the self-attention-based transformer to the linear complexity state-space model Mamba. We establish a diffusion self-attention and Mamba hybrid model that simultaneously achieves efficiency and global dependencies. With the proposed layer-level teacher forcing and feature-based knowledge distillation, T2MD alleviates the training difficulty and high cost of a state space model from scratch. Starting from the distilled 512 \times 512 resolution base model, we push the generation towards 2048 \times 2048 images via lightweight adaptation and high-resolution fine-tuning. Experiments demonstrate that our training path leads to low overhead but high-quality text-to-image generation. Importantly, our results also justify the feasibility of using sequential and causal Mamba models for generating non-causal visual output, suggesting the potential for future exploration.
zh
[CV-84] GLIMPSE: Gradient-Layer Importance Mapping for Prompted Visual Saliency Explanation for Generative LVLMs
【速读】：该论文旨在解决大型视觉语言模型（Large Vision Language Models, LVLMs）在生成自由文本响应时，如何解释其视觉注意力分布及跨模态推理过程的问题，这对于理解模型行为、诊断幻觉、暴露偏差和确保透明性至关重要。论文提出的解决方案关键在于GLIMPSE框架，该框架通过融合梯度加权注意力、自适应层传播和加权标记聚合，生成全局响应级归因热图，从而实现对跨模态推理的解释，其在人类对齐性方面优于现有可解释性方法。
链接: https://arxiv.org/abs/2506.18985
作者: Guanxi Shen
机构: Georgia Institute of Technology (佐治亚理工学院)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Recent advances in large vision language models (LVLMs) have unlocked unprecedented capabilities in generating coherent responses from visual inputs. However, interpreting where LVLMs direct their visual attention while generating free-form textual responses remains a significant challenge, yet is essential for understanding model behavior, diagnosing hallucination, exposing bias and ensuring transparency. We introduce GLIMPSE (Gradient-Layer Importance Mapping for Prompted Visual Saliency Explanation), a lightweight, model-agnostic framework for visualizing the salient image regions that LVLMs rely upon during open-ended visual question answering (VQA), while concurrently revealing the multimodal textual saliency. GLIMPSE fuses gradient-weighted attention, adaptive layer propagation, and weighted token aggregation to produce holistic response-level attribution heat maps for interpreting cross-modal reasoning, outperforming prior interpretability methods in human-alignment. We demonstrate an analytic explainable AI (XAI) approach using GLIMPSE to uncover fine-grained insights into LVLM cross-modal attribution, trace token-level reasoning dynamics, and analyze systematic human-attention misalignment, hallucination, and bias.
zh
[CV-85] DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Models
【速读】：该论文旨在解决遥感图像分割（Referring Remote Sensing Image Segmentation, RRSIS）中由于目标尺度变化、方向多样性和语义模糊性等复杂特征导致的分割精度不足问题。其解决方案的关键在于提出DiffRIS框架，该框架利用预训练文本到图像扩散模型的语义理解能力，通过两个核心创新模块——上下文感知适配器（CP-adapter）和渐进式跨模态推理解码器（PCMRD），实现语言描述与视觉区域的精准对齐，从而提升分割性能。
链接: https://arxiv.org/abs/2506.18946
作者: Zhe Dong,Yuzhe Sun,Tianzhu Liu,Yanfeng Gu
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Referring remote sensing image segmentation (RRSIS) enables the precise delineation of regions within remote sensing imagery through natural language descriptions, serving critical applications in disaster response, urban development, and environmental monitoring. Despite recent advances, current approaches face significant challenges in processing aerial imagery due to complex object characteristics including scale variations, diverse orientations, and semantic ambiguities inherent to the overhead perspective. To address these limitations, we propose DiffRIS, a novel framework that harnesses the semantic understanding capabilities of pre-trained text-to-image diffusion models for enhanced cross-modal alignment in RRSIS tasks. Our framework introduces two key innovations: a context perception adapter (CP-adapter) that dynamically refines linguistic features through global context modeling and object-aware reasoning, and a progressive cross-modal reasoning decoder (PCMRD) that iteratively aligns textual descriptions with visual regions for precise segmentation. The CP-adapter bridges the domain gap between general vision-language understanding and remote sensing applications, while PCMRD enables fine-grained semantic alignment through multi-scale feature interaction. Comprehensive experiments on three benchmark datasets-RRSIS-D, RefSegRS, and RISBench-demonstrate that DiffRIS consistently outperforms existing methods across all standard metrics, establishing a new state-of-the-art for RRSIS tasks. The significant performance improvements validate the effectiveness of leveraging pre-trained diffusion models for remote sensing applications through our proposed adaptive framework.
zh
[CV-86] From Pixels and Words to Waves: A Unified Framework for Spectral Dictionary vLLM s
【速读】：该论文旨在解决视觉-语言模型（Vision-Language Models, VLMs）中计算资源消耗大和效率低的问题，特别是传统方法依赖于视觉编码器中的卷积操作和多模态融合中的二次自注意力机制。其解决方案的关键在于引入了一种基于频谱字典的令牌混合器（spectral dictionary token mixer），通过将每个图像块或词元表示为可学习频率原子的稀疏组合，从而去除卷积和自注意力机制。这一创新不仅降低了模型的参数量和计算复杂度，还提升了推理速度，同时保持了较高的性能表现。
链接: https://arxiv.org/abs/2506.18943
作者: Andrew Kiruluta,Priscilla Burity
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Vision-language models (VLMs) unify computer vision and natural language processing in a single architecture capable of interpreting and describing images. Most state-of-the-art systems rely on two computationally intensive components: convolutions in the vision encoder and quadratic self-attention for multimodal fusion. This work removes both by introducing a spectral dictionary token mixer, which represents each image patch or wordpiece as a sparse combination of learnable frequency atoms. Our 1.1B-parameter prototype, SDict-VLM, achieves BLEU-4 of 39.2, CIDEr of 127.5, and SPICE of 27.0 on MS-COCO captioning, along with 50.3 percent accuracy on VQAv2. These results close approximately 85 percent of the performance gap to BLIP-2 while using 60 percent fewer parameters, 2.3 times less peak GPU memory, and 2.2 times faster inference than PaLI-3. To our knowledge, this is the first VLM to eliminate both convolutions and self-attention while matching mid-scale transformer baselines. In addition to its O(L log L) complexity, the shared frequency dictionary enables transparent cross-modal alignment and offers a tunable trade-off between accuracy and compute, paving the way for efficient and interpretable VLMs.
zh
[CV-87] Damba-ST: Domain-Adaptive Mamba for Efficient Urban Spatio-Temporal Prediction
【速读】：该论文旨在解决城市时空基础模型在跨区域和跨城市场景下泛化能力不足的问题，特别是在数据稀缺或未见过的区域部署城市服务时的挑战。现有方法虽然通过融合跨域时空数据训练统一的Transformer模型，但面临二次计算复杂度和高内存开销的问题，限制了其可扩展性。本文的关键解决方案是提出Damba-ST，一种基于Mamba的域自适应模型，其核心在于保留Mamba线性时间复杂度的优势，同时提升对异构域的适应能力。具体包括：（1）域自适应状态空间模型，将潜在表示空间划分为共享子空间和独立域特定子空间；（2）三种不同的域适配器，用于桥接不同域分布并促进跨域共性对齐。
链接: https://arxiv.org/abs/2506.18939
作者: Rui An,Yifeng Zhang,Ziran Liang,Wenqi Fan,Yuxuan Liang,Xuequn Shang,Qing Li
机构: Northwestern Polytechnical University (西北工业大学); The Hong Kong Polytechnic Univeristy (香港理工大学); The Hong Kong University of Science and Technology (Guangzhou) (香港科技大学（广州）)
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Training urban spatio-temporal foundation models that generalize well across diverse regions and cities is critical for deploying urban services in unseen or data-scarce regions. Recent studies have typically focused on fusing cross-domain spatio-temporal data to train unified Transformer-based models. However, these models suffer from quadratic computational complexity and high memory overhead, limiting their scalability and practical deployment. Inspired by the efficiency of Mamba, a state space model with linear time complexity, we explore its potential for efficient urban spatio-temporal prediction. However, directly applying Mamba as a spatio-temporal backbone leads to negative transfer and severe performance degradation. This is primarily due to spatio-temporal heterogeneity and the recursive mechanism of Mamba’s hidden state updates, which limit cross-domain generalization. To overcome these challenges, we propose Damba-ST, a novel domain-adaptive Mamba-based model for efficient urban spatio-temporal prediction. Damba-ST retains Mamba’s linear complexity advantage while significantly enhancing its adaptability to heterogeneous domains. Specifically, we introduce two core innovations: (1) a domain-adaptive state space model that partitions the latent representation space into a shared subspace for learning cross-domain commonalities and independent, domain-specific subspaces for capturing intra-domain discriminative features; (2) three distinct Domain Adapters, which serve as domain-aware proxies to bridge disparate domain distributions and facilitate the alignment of cross-domain commonalities. Extensive experiments demonstrate the generalization and efficiency of Damba-ST. It achieves state-of-the-art performance on prediction tasks and demonstrates strong zero-shot generalization, enabling seamless deployment in new urban environments without extensive retraining or fine-tuning.
zh
[CV-88] Birds-eye view safety monitoring for the construction top under the tower crane
【速读】：该论文旨在解决塔式起重机在施工过程中对作业面上人员及模块化集成建筑（MiC）的安全保护问题，特别是在从鸟瞰视角监控施工顶部区域时存在的安全隐患。解决方案的关键在于提出一种基于人工智能的全自动安全监测系统，通过融合相机和激光雷达（LiDAR）采集的三维数据，实现对人员和MiC的精确定位，并通过警报机制避免起重机与人员或物体的碰撞。该系统结合了先进的算法与硬件及显示系统，验证了其在实际现场中的准确性和有效性。
链接: https://arxiv.org/abs/2506.18938
作者: Yanke Wang,Yu Hin Ng,Haobo Liang,Ching-Wei Chang,Hao Chen
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Systems and Control (eess.SY)
备注:
Abstract:The tower crane is involving more automated and intelligent operation procedure, and importantly, the application of automation technologies to the safety issues is imperative ahead of the utilization of any other advances. Among diverse risk management tasks on site, it is essential to protect the human workers on the workspace between the tower crane and constructed building top area (construction top) from the bird’s-eye view, especially with Modular Integrated Construction (MiC) lifted. Also, the camera and Light Detection And Ranging (LiDAR) can capture abundant 3D information on site, which is however yet made the best use. Considering the safety protection for humans and tower cranes, we present an AI-based fully automated safety monitoring system for tower crane lifting from the bird’s-eye view, surveilling to shield the human workers on the construction top and avoid cranes’ collision by alarming the crane operator. The system achieved a 3D data fusion for localization of humans and MiCs by integrating the captured information from camera and LiDAR. The state-of-the-art methods were explored and implemented into our proposed software pipeline coupled with the hardware and display systems. Furthermore, we conducted an analysis of the components in the pipeline to verify the accuracy and effectiveness of the involved methods. The display and visualization on the real site proved that our system can serve as a valuable safety monitoring toolkit on site.
zh
[CV-89] Reinforcement Learning-Based Dynamic Grouping for Tubular Structure Tracking
【速读】：该论文旨在解决在跟踪管状结构（如血管和道路）时，由于复杂形态和环境变化导致的最小路径计算问题。现有方法主要分为基于点的模型和基于段的模型，但后者在计算效率和对先验知识的依赖方面存在不足。论文提出的解决方案的关键在于将基于段的跟踪建模为马尔可夫决策过程（Markov Decision Process, MDP），并采用Q-Learning动态探索段的图结构，按需计算边权重并自适应扩展搜索空间，从而避免预计算图的高成本，并增强对不完整初始信息的鲁棒性。
链接: https://arxiv.org/abs/2506.18930
作者: Chong Di,Shuwang Zhou,Da Chen,Jean-Marie Mirebeau,Minglei Shu,Laurent D. Cohen
机构: Shandong Artificial Intelligence Institute, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China; Department of Mathematics, Centre Borelli, ENS Paris-Saclay, CNRS, University Paris-Saclay, 91190, Gif-sur-Yvette, France; University Paris Dauphine, PSL Research University, CNRS, UMR 7534, CEREMADE, 75016 Paris, France
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:The computation of minimal paths for the applications in tracking tubular structures such as blood vessels and roads is challenged by complex morphologies and environmental variations. Existing approaches can be roughly categorized into two research lines: the point-wise based models and the segment-wise based models. Although segment-wise approaches have obtained promising results in many scenarios, they often suffer from computational inefficiency and heavily rely on a prescribed prior to fit the target elongated shapes. We propose a novel framework that casts segment-wise tracking as a Markov Decision Process (MDP), enabling a reinforcement learning approach. Our method leverages Q-Learning to dynamically explore a graph of segments, computing edge weights on-demand and adaptively expanding the search space. This strategy avoids the high cost of a pre-computed graph and proves robust to incomplete initial information. Experimental reuslts on typical tubular structure datasets demonstrate that our method significantly outperforms state-of-the-art point-wise and segment-wise approaches. The proposed method effectively handles complex topologies and maintains global path coherence without depending on extensive prior structural knowledge.
zh
[CV-90] Interpretable and Granular Video-Based Quantification of Motor Characteristics from the Finger Tapping Test in Parkinson Disease
【速读】：该论文试图解决帕金森病（Parkinson’s Disease, PD）患者运动特征客观量化不足的问题，传统基于视觉评估的指敲试验存在主观性高、评价者间和评价者内变异大的缺陷，无法提供个体化运动特征的深入分析。解决方案的关键在于提出一种基于计算机视觉的细粒度方法，通过提取四类临床相关特征（包括运动减少、动作迟缓、序列效应和犹豫停顿）来量化PD患者的运动特征，并利用主成分分析验证了这些特征与临床缺陷的一致性，同时通过机器学习分类器实现了对MDS-UPDRS指敲评分的准确预测，从而提供了可解释的个体运动特征量化方案。
链接: https://arxiv.org/abs/2506.18925
作者: Tahereh Zarrat Ehsan,Michael Tangermann,Yağmur Güçlütürk,Bastiaan R. Bloem,Luc J. W. Evers
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:Accurately quantifying motor characteristics in Parkinson disease (PD) is crucial for monitoring disease progression and optimizing treatment strategies. The finger-tapping test is a standard motor assessment. Clinicians visually evaluate a patient’s tapping performance and assign an overall severity score based on tapping amplitude, speed, and irregularity. However, this subjective evaluation is prone to inter- and intra-rater variability, and does not offer insights into individual motor characteristics captured during this test. This paper introduces a granular computer vision-based method for quantifying PD motor characteristics from video recordings. Four sets of clinically relevant features are proposed to characterize hypokinesia, bradykinesia, sequence effect, and hesitation-halts. We evaluate our approach on video recordings and clinical evaluations of 74 PD patients from the Personalized Parkinson Project. Principal component analysis with varimax rotation shows that the video-based features corresponded to the four deficits. Additionally, video-based analysis has allowed us to identify further granular distinctions within sequence effect and hesitation-halts deficits. In the following, we have used these features to train machine learning classifiers to estimate the Movement Disorder Society Unified Parkinson Disease Rating Scale (MDS-UPDRS) finger-tapping score. Compared to state-of-the-art approaches, our method achieves a higher accuracy in MDS-UPDRS score prediction, while still providing an interpretable quantification of individual finger-tapping motor characteristics. In summary, the proposed framework provides a practical solution for the objective assessment of PD motor characteristics, that can potentially be applied in both clinical and remote settings. Future work is needed to assess its responsiveness to symptomatic treatment and disease progression.
zh
[CV-91] Connecting Vision and Emissions: A Behavioural AI Approach to Carbon Estimation in Road Design
【速读】：该论文旨在解决城市环境中实时车辆检测与分类问题，并通过此过程估算碳排放。其解决方案的关键在于对YOLOv8架构进行增强，以实现从实时交通视频流中检测、分割和跟踪车辆，并结合深度光学字符识别（OCR）模块对车牌进行识别与验证，从而实现车辆类型的准确分类及碳排放的精确计算。该方法采用多阶段混合流水线，克服了YOLOv8在细粒度识别任务中的局限性，提升了系统在复杂环境下的鲁棒性和准确性。
链接: https://arxiv.org/abs/2506.18924
作者: Ammar K Al Mhdawi,Nonso Nnamoko,Safanah Mudheher Raafat,M.K.S. Al-Mhdawi,Amjad J Humaidi
机构: 未知
类目: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
备注:
Abstract:We present an enhanced YOLOv8 real time vehicle detection and classification framework, for estimating carbon emissions in urban environments. The system enhances YOLOv8 architecture to detect, segment, and track vehicles from live traffic video streams. Once a vehicle is localized, a dedicated deep learning-based identification module is employed to recognize license plates and classify vehicle types. Since YOLOv8 lacks the built-in capacity for fine grained recognition tasks such as reading license plates or determining vehicle attributes beyond class labels, our framework incorporates a hybrid pipeline where each detected vehicle is tracked and its bounding box is cropped and passed to a deep Optical Character Recognition (OCR) module. This OCR system, composed of multiple convolutional neural network (CNN) layers, is trained specifically for character-level detection and license plate decoding under varied conditions such as motion blur, occlusion, and diverse font styles. Additionally, the recognized plate information is validated using a real time API that cross references with an external vehicle registration database to ensure accurate classification and emission estimation. This multi-stage approach enables precise, automated calculation of per vehicle carbon emissions. Extensive evaluation was conducted using a diverse vehicle dataset enriched with segmentation masks and annotated license plates. The YOLOv8 detector achieved a mean Average Precision (mAP@0.5) of approximately 71% for bounding boxes and 70% for segmentation masks. Character level OCR accuracy reached up to 99% with the best performing CNN model. These results affirm the feasibility of combining real time object detection with deep OCR for practical deployment in smart transportation systems, offering a scalable solution for automated, vehicle specific carbon emission monitoring.
zh
[CV-92] Correspondence-Free Multiview Point Cloud Registration via Depth-Guided Joint Optimisation IROS2025
【速读】：该论文旨在解决多视角点云配准问题，该问题在构建全局一致的3D模型中具有基础性作用。传统方法依赖于特征提取和多点云间的数据关联，但在复杂环境中难以获得全局最优解。论文提出了一种无需对应关系的多视角点云配准方法，其关键在于将全局地图表示为深度图，并利用原始深度信息构建非线性最小二乘优化问题，联合估计点云位姿和全局地图。与基于特征的捆绑调整方法不同，该方法通过点云对应的位姿将多帧点云与全局深度图进行关联，数据关联在优化过程中被隐式地集成并动态优化。
链接: https://arxiv.org/abs/2506.18922
作者: Yiran Zhou,Yingyu Wang,Shoudong Huang,Liang Zhao
机构: University of Technology Sydney (悉尼科技大学); The University of Edinburgh (爱丁堡大学)
类目: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
备注: 8 pages, accepted for publication in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025)
Abstract:Multiview point cloud registration is a fundamental task for constructing globally consistent 3D models. Existing approaches typically rely on feature extraction and data association across multiple point clouds; however, these processes are challenging to obtain global optimal solution in complex environments. In this paper, we introduce a novel correspondence-free multiview point cloud registration method. Specifically, we represent the global map as a depth map and leverage raw depth information to formulate a non-linear least squares optimisation that jointly estimates poses of point clouds and the global map. Unlike traditional feature-based bundle adjustment methods, which rely on explicit feature extraction and data association, our method bypasses these challenges by associating multi-frame point clouds with a global depth map through their corresponding poses. This data association is implicitly incorporated and dynamically refined during the optimisation process. Extensive evaluations on real-world datasets demonstrate that our method outperforms state-of-the-art approaches in accuracy, particularly in challenging environments where feature extraction and data association are difficult.
zh
[CV-93] Systematic Review of Pituitary Gland and Pituitary Adenoma Automatic Segmentation Techniques in Magnetic Resonance Imaging
【速读】：该论文旨在解决从磁共振成像（MRI）中准确分割垂体腺瘤及其垂体腺的问题，这对于垂体腺瘤的诊断和治疗至关重要。其解决方案的关键在于评估自动分割方法，以提高MRI基础上垂体腺瘤及其腺体分割的准确性与效率。研究发现，大多数文献采用深度学习方法，尤其是基于U-Net的模型，显示出在腺瘤分割中的潜力，但针对如正常垂体腺这类小结构的性能仍需进一步提升。
链接: https://arxiv.org/abs/2506.19797
作者: Mubaraq Yakubu,Navodini Wijethilake,Jonathan Shapey,Andrew King,Alexander Hammers
机构: King’s College London (伦敦国王学院)
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Purpose: Accurate segmentation of both the pituitary gland and adenomas from magnetic resonance imaging (MRI) is essential for diagnosis and treatment of pituitary adenomas. This systematic review evaluates automatic segmentation methods for improving the accuracy and efficiency of MRI-based segmentation of pituitary adenomas and the gland itself. Methods: We reviewed 34 studies that employed automatic and semi-automatic segmentation methods. We extracted and synthesized data on segmentation techniques and performance metrics (such as Dice overlap scores). Results: The majority of reviewed studies utilized deep learning approaches, with U-Net-based models being the most prevalent. Automatic methods yielded Dice scores of 0.19–89.00% for pituitary gland and 4.60–96.41% for adenoma segmentation. Semi-automatic methods reported 80.00–92.10% for pituitary gland and 75.90–88.36% for adenoma segmentation. Conclusion: Most studies did not report important metrics such as MR field strength, age and adenoma size. Automated segmentation techniques such as U-Net-based models show promise, especially for adenoma segmentation, but further improvements are needed to achieve consistently good performance in small structures like the normal pituitary gland. Continued innovation and larger, diverse datasets are likely critical to enhancing clinical applicability.
zh
[CV-94] NeRF-based CBCT Reconstruction needs Normalization and Initialization
【速读】：该论文旨在解决基于NeRF的锥形束计算机断层扫描（Cone Beam Computed Tomography, CBCT）重建方法中，哈希编码器与神经网络之间的局部-全局训练不匹配问题。该问题导致哈希特征在不同训练步骤间存在高度错位，进而引发训练不稳定、收敛速度慢和重建质量下降。解决方案的关键在于引入一种归一化哈希编码器以增强特征一致性，并提出一种映射一致性初始化（Mapping Consistency Initialization, MCI）策略，在训练前利用预训练模型的全局映射特性对神经网络进行初始化，从而提升早期训练的稳定性，加快收敛速度并改善重建性能。
链接: https://arxiv.org/abs/2506.19742
作者: Zhuowei Xu,Han Li,Dai Sun,Zhicheng Li,Yujia Li,Qingpeng Kong,Zhiwei Cheng,Nassir Navab,S. Kevin Zhou
机构: 未知
类目: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Cone Beam Computed Tomography (CBCT) is widely used in medical imaging. However, the limited number and intensity of X-ray projections make reconstruction an ill-posed problem with severe artifacts. NeRF-based methods have achieved great success in this task. However, they suffer from a local-global training mismatch between their two key components: the hash encoder and the neural network. Specifically, in each training step, only a subset of the hash encoder’s parameters is used (local sparse), whereas all parameters in the neural network participate (global dense). Consequently, hash features generated in each step are highly misaligned, as they come from different subsets of the hash encoder. These misalignments from different training steps are then fed into the neural network, causing repeated inconsistent global updates in training, which leads to unstable training, slower convergence, and degraded reconstruction quality. Aiming to alleviate the impact of this local-global optimization mismatch, we introduce a Normalized Hash Encoder, which enhances feature consistency and mitigates the mismatch. Additionally, we propose a Mapping Consistency Initialization(MCI) strategy that initializes the neural network before training by leveraging the global mapping property from a well-trained model. The initialized neural network exhibits improved stability during early training, enabling faster convergence and enhanced reconstruction performance. Our method is simple yet effective, requiring only a few lines of code while substantially improving training efficiency on 128 CT cases collected from 4 different datasets, covering 7 distinct anatomical regions.
zh
[CV-95] ReCoGNet: Recurrent Context-Guided Network for 3D MRI Prostate Segmentation
【速读】：该论文旨在解决前列腺腺体在T2加权磁共振成像（T2-weighted MRI）中的分割问题，这是一个在临床前列腺癌评估中关键但具有挑战性的任务。传统方法，尤其是二维卷积神经网络（2D CNN），无法充分利用切片间的解剖连续性，从而限制了其准确性和鲁棒性；而全三维模型虽然能提高空间一致性，但需要大量标注数据，这在临床环境中往往不现实。该研究提出的解决方案是一种混合架构，将MRI序列建模为时空数据，其关键在于结合了一个深度预训练的DeepLabV3主干网络以提取每张MRI切片的高层语义特征，并使用基于ConvLSTM层的循环卷积头来跨切片整合信息，同时保持空间结构，从而实现上下文感知的分割。
链接: https://arxiv.org/abs/2506.19687
作者: Ahmad Mustafa,Reza Rastegar,Ghassan AlRegib
机构: Georgia Institute of Technology (佐治亚理工学院)
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Prostate gland segmentation from T2-weighted MRI is a critical yet challenging task in clinical prostate cancer assessment. While deep learning-based methods have significantly advanced automated segmentation, most conventional approaches-particularly 2D convolutional neural networks (CNNs)-fail to leverage inter-slice anatomical continuity, limiting their accuracy and robustness. Fully 3D models offer improved spatial coherence but require large amounts of annotated data, which is often impractical in clinical settings. To address these limitations, we propose a hybrid architecture that models MRI sequences as spatiotemporal data. Our method uses a deep, pretrained DeepLabV3 backbone to extract high-level semantic features from each MRI slice and a recurrent convolutional head, built with ConvLSTM layers, to integrate information across slices while preserving spatial structure. This combination enables context-aware segmentation with improved consistency, particularly in data-limited and noisy imaging conditions. We evaluate our method on the PROMISE12 benchmark under both clean and contrast-degraded test settings. Compared to state-of-the-art 2D and 3D segmentation models, our approach demonstrates superior performance in terms of precision, recall, Intersection over Union (IoU), and Dice Similarity Coefficient (DSC), highlighting its potential for robust clinical deployment.
zh
[CV-96] Filling of incomplete sinograms from sparse PET detector configurations using a residual U-Net
【速读】：该论文试图解决长轴向视野正电子发射断层扫描（PET）系统因密集排列光电探测器而导致成本过高的问题，从而限制其在临床中的应用。解决方案的关键在于提出一种基于深度sinogram重建网络的方法，利用改进的残差U-Net架构，通过训练临床PET数据来填补缺失的sinogram数据，从而在保持探测器成本与传统PET系统相近的前提下，提升图像质量。该方法在sinogram和重建图像域中均优于二维插值技术，尽管重建图像在细节锐度上有所损失，但仍表现出对稀疏探测器配置引起的欠采样具有较强的补偿能力。
链接: https://arxiv.org/abs/2506.19600
作者: Klara Leffler,Luigi Tommaso Luppino,Samuel Kuttner,Karin Söderkvist,Jan Axelsson
机构: Umeå University (乌梅奥大学); University of Copenhagen (哥本哈根大学); Norsk Regnesentral (挪威计算中心); UiT The Arctic University of Norway (特罗姆瑟北极大学); University Hospital of North Norway (北挪威大学医院); Department of Clinical Medicine (临床医学系)
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Medical Physics (physics.med-ph)
备注: 15 pages, 9 figures
Abstract:Long axial field-of-view PET scanners offer increased field-of-view and sensitivity compared to traditional PET scanners. However, a significant cost is associated with the densely packed photodetectors required for the extended-coverage systems, limiting clinical utilisation. To mitigate the cost limitations, alternative sparse system configurations have been proposed, allowing an extended field-of-view PET design with detector costs similar to a standard PET system, albeit at the expense of image quality. In this work, we propose a deep sinogram restoration network to fill in the missing sinogram data. Our method utilises a modified Residual U-Net, trained on clinical PET scans from a GE Signa PET/MR, simulating the removal of 50% of the detectors in a chessboard pattern (retaining only 25% of all lines of response). The model successfully recovers missing counts, with a mean absolute error below two events per pixel, outperforming 2D interpolation in both sinogram and reconstructed image domain. Notably, the predicted sinograms exhibit a smoothing effect, leading to reconstructed images lacking sharpness in finer details. Despite these limitations, the model demonstrates a substantial capacity for compensating for the undersampling caused by the sparse detector configuration. This proof-of-concept study suggests that sparse detector configurations, combined with deep learning techniques, offer a viable alternative to conventional PET scanner designs. This approach supports the development of cost-effective, total body PET scanners, allowing a significant step forward in medical imaging technology.
zh
[CV-97] Learning from Anatomy: Supervised Anatomical Pretraining (SAP) for Improved Metastatic Bone Disease Segmentation in Whole-Body MRI
【速读】：该论文旨在解决全身磁共振成像（WB-MRI）中转移性骨病（Metastatic Bone Disease, MBD）分割的挑战性问题，尤其是在病变形态多样、边界模糊及类别不平衡等情况下获取可靠分割结果的难题。其解决方案的关键在于提出一种监督解剖预训练（Supervised Anatomical Pretraining, SAP）方法，通过有限的解剖标签数据进行预训练，从而学习到与骨结构相关的有效归纳偏置，提升后续骨病变分割任务的性能。实验结果表明，SAP在多个评估指标上均优于基线和当前最先进的自监督学习方法。
链接: https://arxiv.org/abs/2506.19590
作者: Joris Wuts,Jakub Ceranka,Nicolas Michoux,Frédéric Lecouvet,Jef Vandemeulebroucke
机构: Vrije Universiteit Brussel (弗拉瑞克大学布鲁塞尔分校); Cliniques universitaires Saint Luc & Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain (圣卢克大学诊所与实验与临床研究所（IREC），乌特勒支大学); Fonds Wetenschappelijk Onderzoek (FWO) (比利时科学基金会); imec (imec); Universitair Ziekenhuis Brussel (布鲁塞尔大学医院)
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注: This preprint is currently under review at Computers in Biology and Medicine (Elsevier). This version has not been peer-reviewed
Abstract:The segmentation of metastatic bone disease (MBD) in whole-body MRI (WB-MRI) is a challenging problem. Due to varying appearances and anatomical locations of lesions, ambiguous boundaries, and severe class imbalance, obtaining reliable segmentations requires large, well-annotated datasets capturing lesion variability. Generating such datasets requires substantial time and expertise, and is prone to error. While self-supervised learning (SSL) can leverage large unlabeled datasets, learned generic representations often fail to capture the nuanced features needed for accurate lesion detection. In this work, we propose a Supervised Anatomical Pretraining (SAP) method that learns from a limited dataset of anatomical labels. First, an MRI-based skeletal segmentation model is developed and trained on WB-MRI scans from healthy individuals for high-quality skeletal delineation. Then, we compare its downstream efficacy in segmenting MBD on a cohort of 44 patients with metastatic prostate cancer, against both a baseline random initialization and a state-of-the-art SSL method. SAP significantly outperforms both the baseline and SSL-pretrained models, achieving a normalized surface Dice of 0.76 and a Dice coefficient of 0.64. The method achieved a lesion detection F2 score of 0.44, improving on 0.24 (baseline) and 0.31 (SSL). When considering only clinically relevant lesions larger than 1~ml, SAP achieves a detection sensitivity of 100% in 28 out of 32 patients. Learning bone morphology from anatomy yields an effective and domain-relevant inductive bias that can be leveraged for the downstream segmentation task of bone lesions. All code and models are made publicly available. Comments: This preprint is currently under review at Computers in Biology and Medicine (Elsevier). This version has not been peer-reviewed Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV) Cite as: arXiv:2506.19590 [eess.IV] (or arXiv:2506.19590v1 [eess.IV] for this version) https://doi.org/10.48550/arXiv.2506.19590 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Joris Wuts [view email] [v1] Tue, 24 Jun 2025 12:59:44 UTC (10,239 KB)
zh
[CV-98] Angio-Diff: Learning a Self-Supervised Adversarial Diffusion Model for Angiographic Geometry Generation
【速读】：该论文旨在解决医学影像中血管造影图像合成的质量问题，特别是在缺乏配对的大规模X射线血管造影数据集的情况下，如何生成高质量的血管结构。其解决方案的关键在于提出一种基于扩散模型的自监督方法，通过学习血管数据的分布、生成血管结构以及引入基于掩码的对抗模块来提升合成图像的几何准确性，同时结合参数化血管模型以更好地拟合血管的形状和分布。
链接: https://arxiv.org/abs/2506.19455
作者: Zhifeng Wang,Renjiao Yi,Xin Wen,Chenyang Zhu,Kai Xu,Kunlun He
机构: National University of Defense Technology (国防科技大学)
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Vascular diseases pose a significant threat to human health, with X-ray angiography established as the gold standard for diagnosis, allowing for detailed observation of blood vessels. However, angiographic X-rays expose personnel and patients to higher radiation levels than non-angiographic X-rays, which are unwanted. Thus, modality translation from non-angiographic to angiographic X-rays is desirable. Data-driven deep approaches are hindered by the lack of paired large-scale X-ray angiography datasets. While making high-quality vascular angiography synthesis crucial, it remains challenging. We find that current medical image synthesis primarily operates at pixel level and struggles to adapt to the complex geometric structure of blood vessels, resulting in unsatisfactory quality of blood vessel image synthesis, such as disconnections or unnatural curvatures. To overcome this issue, we propose a self-supervised method via diffusion models to transform non-angiographic X-rays into angiographic X-rays, mitigating data shortages for data-driven approaches. Our model comprises a diffusion model that learns the distribution of vascular data from diffusion latent, a generator for vessel synthesis, and a mask-based adversarial module. To enhance geometric accuracy, we propose a parametric vascular model to fit the shape and distribution of blood vessels. The proposed method contributes a pipeline and a synthetic dataset for X-ray angiography. We conducted extensive comparative and ablation experiments to evaluate the Angio-Diff. The results demonstrate that our method achieves state-of-the-art performance in synthetic angiography image quality and more accurately synthesizes the geometric structure of blood vessels. The code is available at this https URL.
zh
[CV-99] NAADA: A Noise-Aware Attention Denoising Autoencoder for Dental Panoramic Radiographs
【速读】：该论文试图解决卷积去噪自编码器（Convolutional Denoising Autoencoders, DAEs）在图像恢复中对高频细节恢复不足的问题，特别是在牙科全景X光片中，这种问题会导致关键解剖结构的丢失。解决方案的关键在于提出一种噪声感知的自注意力机制（noise-aware self-attention），使模型能够在噪声区域中有效聚焦并恢复关键特征，进而构建了噪声感知注意力增强的去噪自编码器（Noise-Aware Attention-Enhanced Denoising Autoencoder, NAADA）网络，从而提升图像质量与诊断准确性。
链接: https://arxiv.org/abs/2506.19387
作者: Khuram Naveed,Bruna Neves de Freitas,Ruben Pauwels
机构: 未知
类目: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注: 10 pages, 8 figures
Abstract:Convolutional denoising autoencoders (DAEs) are powerful tools for image restoration. However, they inherit a key limitation of convolutional neural networks (CNNs): they tend to recover low-frequency features, such as smooth regions, more effectively than high-frequency details. This leads to the loss of fine details, which is particularly problematic in dental radiographs where preserving subtle anatomical structures is crucial. While self-attention mechanisms can help mitigate this issue by emphasizing important features, conventional attention methods often prioritize features corresponding to cleaner regions and may overlook those obscured by noise. To address this limitation, we propose a noise-aware self-attention method, which allows the model to effectively focus on and recover key features even within noisy regions. Building on this approach, we introduce the noise-aware attention-enhanced denoising autoencoder (NAADA) network for enhancing noisy panoramic dental radiographs. Compared with the recent state of the art (and much heavier) methods like Uformer, MResDNN etc., our method improves the reconstruction of fine details, ensuring better image quality and diagnostic accuracy.
zh
[CV-100] Reconsidering Explicit Longitudinal Mammography Alignment for Enhanced Breast Cancer Risk Prediction MICCAI2025
【速读】：该论文旨在解决乳腺癌筛查中如何通过深度学习方法优化高风险人群的筛查间隔问题，其核心挑战在于如何有效利用不同时间点的乳腺X线摄影图像来跟踪乳腺组织的变化。解决方案的关键在于探索显式对齐（explicit alignment）在输入空间与表示空间中的应用，并评估对齐与风险预测是否应联合优化。研究发现，将显式对齐与风险估计联合优化虽为当前最先进的方法，但会导致对齐质量与预测性能之间的权衡，而图像级对齐优于表示级对齐，能够提升变形场质量和风险预测准确性。
链接: https://arxiv.org/abs/2506.19363
作者: Solveig Thrun,Stine Hansen,Zijun Sun,Nele Blum,Suaiba A. Salahuddin,Kristoffer Wickstrøm,Elisabeth Wetzer,Robert Jenssen,Maik Stille,Michael Kampffmeyer
机构: 未知
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注: MICCAI 2025, early accepted
Abstract:Regular mammography screening is essential for early breast cancer detection. Deep learning-based risk prediction methods have sparked interest to adjust screening intervals for high-risk groups. While early methods focused only on current mammograms, recent approaches leverage the temporal aspect of screenings to track breast tissue changes over time, requiring spatial alignment across different time points. Two main strategies for this have emerged: explicit feature alignment through deformable registration and implicit learned alignment using techniques like transformers, with the former providing more control. However, the optimal approach for explicit alignment in mammography remains underexplored. In this study, we provide insights into where explicit alignment should occur (input space vs. representation space) and if alignment and risk prediction should be jointly optimized. We demonstrate that jointly learning explicit alignment in representation space while optimizing risk estimation performance, as done in the current state-of-the-art approach, results in a trade-off between alignment quality and predictive performance and show that image-level alignment is superior to representation-level alignment, leading to better deformation field quality and enhanced risk prediction accuracy. The code is available at this https URL.
zh
[CV-101] Explicit Residual-Based Scalable Image Coding for Humans and Machines
【速读】：该论文旨在解决图像压缩中同时满足机器视觉与人类视觉需求的挑战，特别是在现有模型过度依赖学习能力而忽视架构设计的问题。其解决方案的关键在于引入一种显式的残差压缩机制，该机制常见于分辨率可扩展编码方法（如JPEG2000），通过提出两种互补的方法——基于特征残差的可扩展编码（Feature Residual-based Scalable Coding, FR-ICMH）和基于像素残差的可扩展编码（Pixel Residual-based Scalable Coding, PR-ICMH），以提升编码效率和可解释性，并在编码复杂度与压缩性能之间提供灵活选择。
链接: https://arxiv.org/abs/2506.19297
作者: Yui Tatsumi,Ziyue Zeng,Hiroshi Watanabe
机构: Waseda University (早稻田大学)
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Scalable image compression is a technique that progressively reconstructs multiple versions of an image for different requirements. In recent years, images have increasingly been consumed not only by humans but also by image recognition models. This shift has drawn growing attention to scalable image compression methods that serve both machine and human vision (ICMH). Many existing models employ neural network-based codecs, known as learned image compression, and have made significant strides in this field by carefully designing the loss functions. In some cases, however, models are overly reliant on their learning capacity, and their architectural design is not sufficiently considered. In this paper, we enhance the coding efficiency and interpretability of ICMH framework by integrating an explicit residual compression mechanism, which is commonly employed in resolution scalable coding methods such as JPEG2000. Specifically, we propose two complementary methods: Feature Residual-based Scalable Coding (FR-ICMH) and Pixel Residual-based Scalable Coding (PR-ICMH). These proposed methods are applicable to various machine vision tasks. Moreover, they provide flexibility to choose between encoder complexity and compression performance, making it adaptable to diverse application requirements. Experimental results demonstrate the effectiveness of our proposed methods, with PR-ICMH achieving up to 29.57% BD-rate savings over the previous work.
zh
[CV-102] Convergent and divergent connectivity patterns of the arcuate fasciculus in macaques and humans
【速读】：该论文旨在解决非人灵长类动物弓状束（arcuate fasciculus, AF）的组织与连接模式在进化过程中如何区别于人类的问题。其解决方案的关键在于结合跨尺度单神经元追踪技术（包括病毒基因标记和荧光显微光学切片断层扫描）与高场强扩散磁共振成像（11.7T diffusion MRI）进行全脑纤维追踪，并辅以人类7.0T MRI的谱嵌入分析，从而实现跨物种的比较连接组学分析。通过该方法，研究揭示了人类AF在颞上回扩展更广及前额叶与顶叶岛盖连接更强等进化特征，为理解人类语言网络的特异性提供了基于连接性的神经基础。
链接: https://arxiv.org/abs/2506.19266
作者: Jiahao Huang,Ruifeng Li,Wenwen Yu,Anan Li,Xiangning Li,Mingchao Yan,Lei Xie,Qingrun Zeng,Xueyan Jia,Shuxin Wang,Ronghui Ju,Feng Chen,Qingming Luo,Hui Gong,Xiaoquan Yang,Yuanjing Feng,Zheng Wang
机构: 未知
类目: Neurons and Cognition (q-bio.NC); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
备注: 34 pages, 6 figures
Abstract:The organization and connectivity of the arcuate fasciculus (AF) in nonhuman primates remain contentious, especially concerning how its anatomy diverges from that of humans. Here, we combined cross-scale single-neuron tracing - using viral-based genetic labeling and fluorescence micro-optical sectioning tomography in macaques (n = 4; age 3 - 11 years) - with whole-brain tractography from 11.7T diffusion MRI. Complemented by spectral embedding analysis of 7.0T MRI in humans, we performed a comparative connectomic analysis of the AF across species. We demonstrate that the macaque AF originates in the temporal-parietal cortex, traverses the auditory cortex and parietal operculum, and projects into prefrontal regions. In contrast, the human AF exhibits greater expansion into the middle temporal gyrus and stronger prefrontal and parietal operculum connectivity - divergences quantified by Kullback-Leibler analysis that likely underpin the evolutionary specialization of human language networks. These interspecies differences - particularly the human AF’s broader temporal integration and strengthened frontoparietal linkages - suggest a connectivity-based substrate for the emergence of advanced language processing unique to humans. Furthermore, our findings offer a neuroanatomical framework for understanding AF-related disorders such as aphasia and dyslexia, where aberrant connectivity disrupts language function.
zh
[CV-103] Quantitative Benchmarking of Anomaly Detection Methods in Digital Pathology
【速读】：该论文旨在解决数字病理图像中异常检测的挑战，尤其是在处理图像尺寸大、多尺度结构、染色变异和重复模式等独特特性时，现有异常检测算法表现不佳的问题。其解决方案的关键在于通过系统实验评估超过20种经典和主流的异常检测方法，并基于真实和合成的五个数字病理数据集进行详尽的性能比较，从而揭示不同方法在图像尺度、异常模式类型及训练周期选择策略下的表现差异，为未来数字病理图像的异常检测研究提供全面的基准参考。
链接: https://arxiv.org/abs/2506.19234
作者: Can Cui,Xindong Zheng,Ruining Deng,Quan Liu,Tianyuan Yao,Keith T Wilson,Lori A Coburn,Bennett A Landman,Haichun Yang,Yaohong Wang,Yuankai Huo
机构: Vanderbilt University(范德堡大学); Weill Cornell Medicine(威尔康奈尔医学中心); Vanderbilt University Medical Center(范德堡大学医学中心); Veterans Affairs Tennessee Valley Healthcare System(田纳西山谷退伍军人事务医疗系统); Department of Pathology, Microbiology, and Immunology(病理学、微生物学和免疫学系); UT MD Anderson Cancer Center(德州大学MD安德森癌症中心)
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Anomaly detection has been widely studied in the context of industrial defect inspection, with numerous methods developed to tackle a range of challenges. In digital pathology, anomaly detection holds significant potential for applications such as rare disease identification, artifact detection, and biomarker discovery. However, the unique characteristics of pathology images, such as their large size, multi-scale structures, stain variability, and repetitive patterns, introduce new challenges that current anomaly detection algorithms struggle to address. In this quantitative study, we benchmark over 20 classical and prevalent anomaly detection methods through extensive experiments. We curated five digital pathology datasets, both real and synthetic, to systematically evaluate these approaches. Our experiments investigate the influence of image scale, anomaly pattern types, and training epoch selection strategies on detection performance. The results provide a detailed comparison of each method’s strengths and limitations, establishing a comprehensive benchmark to guide future research in anomaly detection for digital pathology images.
zh
[CV-104] Deformable Medical Image Registration with Effective Anatomical Structure Representation and Divide-and-Conquer Network
【速读】：该论文旨在解决当前基于学习的可变形医学图像配准（Deformable Medical Image Registration, DMIR）方法在区域兴趣（Region of Interest, ROI）表示和独立对齐方面的局限性。现有无监督方法忽略ROI表示，直接对图像对进行配准，而弱监督方法则过度依赖标签约束来促进配准过程。该论文提出的解决方案关键在于引入一种基于ROI的新型配准方法——EASR-DCN，通过有效ROI表示医学图像，并在无需标签的情况下实现这些ROI的独立对齐。具体而言，首先利用高斯混合模型进行强度分析以提取具有不同强度的多个有效ROI，随后采用一种新的分治网络（Divide-and-Conquer Network, DCN）通过独立通道处理这些ROI，学习每个ROI的特征对齐，最终将所得对应关系无缝整合生成全面的位移向量场。
链接: https://arxiv.org/abs/2506.19222
作者: Xinke Ma,Yongsheng Pan,Qingjie Zeng,Mengkang Lu,Bolysbek Murat Yerzhanuly,Bazargul Matkerim,Yong Xia
机构: Northwestern Polytechnical University(西北工业大学); Al-Farabi Kazakh National University(哈萨克国立阿里-法拉比大学)
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Effective representation of Regions of Interest (ROI) and independent alignment of these ROIs can significantly enhance the performance of deformable medical image registration (DMIR). However, current learning-based DMIR methods have limitations. Unsupervised techniques disregard ROI representation and proceed directly with aligning pairs of images, while weakly-supervised methods heavily depend on label constraints to facilitate registration. To address these issues, we introduce a novel ROI-based registration approach named EASR-DCN. Our method represents medical images through effective ROIs and achieves independent alignment of these ROIs without requiring labels. Specifically, we first used a Gaussian mixture model for intensity analysis to represent images using multiple effective ROIs with distinct intensities. Furthermore, we propose a novel Divide-and-Conquer Network (DCN) to process these ROIs through separate channels to learn feature alignments for each ROI. The resultant correspondences are seamlessly integrated to generate a comprehensive displacement vector field. Extensive experiments were performed on three MRI and one CT datasets to showcase the superior accuracy and deformation reduction efficacy of our EASR-DCN. Compared to VoxelMorph, our EASR-DCN achieved improvements of 10.31% in the Dice score for brain MRI, 13.01% for cardiac MRI, and 5.75% for hippocampus MRI, highlighting its promising potential for clinical applications. The code for this work will be released upon acceptance of the paper.
zh
[CV-105] A Deep Learning Based Method for Fast Registration of Cardiac Magnetic Resonance Images
【速读】：该论文旨在解决医学图像分析中图像配准（image registration）的挑战性问题，特别是在心脏影像中追踪组织运动以量化心脏应变（cardiac strain）的应用场景。传统深度学习方法在无监督条件下虽然能够学习预测有效的变换，但其预测速度较慢，难以在临床和研究环境中广泛应用。为了解决这一问题，论文提出了一种快速且轻量级的配准模型（Fast and Lightweight Registration, FLIR），其关键在于设计了一个高效的卷积架构，能够在保持与当前最先进模型相当的配准精度的同时，显著减少推理时间，从而实现对高动态器官如心脏的高效、准确配准。
链接: https://arxiv.org/abs/2506.19167
作者: Benjamin Graham
机构: 未知
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
备注:
Abstract:Image registration is used in many medical image analysis applications, such as tracking the motion of tissue in cardiac images, where cardiac kinematics can be an indicator of tissue health. Registration is a challenging problem for deep learning algorithms because ground truth transformations are not feasible to create, and because there are potentially multiple transformations that can produce images that appear correlated with the goal. Unsupervised methods have been proposed to learn to predict effective transformations, but these methods take significantly longer to predict than established baseline methods. For a deep learning method to see adoption in wider research and clinical settings, it should be designed to run in a reasonable time on common, mid-level hardware. Fast methods have been proposed for the task of image registration but often use patch-based methods which can affect registration accuracy for a highly dynamic organ such as the heart. In this thesis, a fast, volumetric registration model is proposed for the use of quantifying cardiac strain. The proposed Deep Learning Neural Network (DLNN) is designed to utilize an architecture that can compute convolutions incredibly efficiently, allowing the model to achieve registration fidelity similar to other state-of-the-art models while taking a fraction of the time to perform inference. The proposed fast and lightweight registration (FLIR) model is used to predict tissue motion which is then used to quantify the non-uniform strain experienced by the tissue. For acquisitions taken from the same patient at approximately the same time, it would be expected that strain values measured between the acquisitions would have very small differences. Using this metric, strain values computed using the FLIR method are shown to be very consistent. Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) Cite as: arXiv:2506.19167 [eess.IV] (or arXiv:2506.19167v1 [eess.IV] for this version) https://doi.org/10.48550/arXiv.2506.19167 Focus to learn more arXiv-issued DOI via DataCite
zh
[CV-106] Staining normalization in histopathology: Method benchmarking using multicenter dataset
【速读】：该论文试图解决组织切片在不同实验室中染色差异导致的图像外观不一致问题，这一问题对病理学家和基于AI的下游分析均构成挑战。解决方案的关键在于通过构建一个多中心组织图像数据集，其中来自结肠、肾脏和皮肤组织块的样本被分发至66个不同实验室进行常规苏木精-伊红（HE）染色，并保持其他影响组织外观的因素恒定，从而隔离染色变异因素。随后，利用该数据集对八种不同的染色归一化方法进行了性能比较，包括四种传统方法和两种基于深度学习的方法，以评估其在减少染色变异方面的效果。
链接: https://arxiv.org/abs/2506.19106
作者: Umair Khan,Jouni Härkönen,Marjukka Friman,Leena Latonen,Teijo Kuopio,Pekka Ruusuvuori
机构: University of Turku, Institute of Biomedicine, Turku, FI-20014, Finland; Hospital Nova of Central Finland, Jyväskylä, Finland; University of Eastern Finland, Institute of Biomedicine, Kuopio, Finland; Tampere University, Faculty of Medicine and Health Technology, Tampere, 33100, Finland; InFlames Research Flagship, University of Turku, Turku, Finland
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Tissues and Organs (q-bio.TO)
备注: 18 pages, 9 figures
Abstract:Hematoxylin and Eosin (HE) has been the gold standard in tissue analysis for decades, however, tissue specimens stained in different laboratories vary, often significantly, in appearance. This variation poses a challenge for both pathologists’ and AI-based downstream analysis. Minimizing stain variation computationally is an active area of research. To further investigate this problem, we collected a unique multi-center tissue image dataset, wherein tissue samples from colon, kidney, and skin tissue blocks were distributed to 66 different labs for routine HE staining. To isolate staining variation, other factors affecting the tissue appearance were kept constant. Further, we used this tissue image dataset to compare the performance of eight different stain normalization methods, including four traditional methods, namely, histogram matching, Macenko, Vahadane, and Reinhard normalization, and two deep learning-based methods namely CycleGAN and Pixp2pix, both with two variants each. We used both quantitative and qualitative evaluation to assess the performance of these methods. The dataset’s inter-laboratory staining variation could also guide strategies to improve model generalizability through varied training data
zh
[CV-107] Xray2Xray: World Model from Chest X-rays with Volumetric Context
【速读】：该论文旨在解决胸部X光片（Chest X-rays, CXRs）作为二维投影图像在结构重叠限制下对疾病精确诊断和风险预测效果受限的问题。其解决方案的关键在于提出Xray2Xray，一种新型的世界模型（World Model），该模型通过视觉模型和转换模型建模不同角度位置的X射线投影的过渡动力学，从而学习编码三维结构信息的潜在表示（latent representations）。这些潜在表示被用于下游的风险预测和疾病诊断任务，并在实验中表现出优于监督方法和自监督预训练方法的性能。
链接: https://arxiv.org/abs/2506.19055
作者: Zefan Yang,Xinrui Song,Xuanang Xu,Yongyi Shi,Ge Wang,Mannudeep K. Kalra,Pingkun Yan
机构: 未知
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
备注:
Abstract:Chest X-rays (CXRs) are the most widely used medical imaging modality and play a pivotal role in diagnosing diseases. However, as 2D projection images, CXRs are limited by structural superposition, which constrains their effectiveness in precise disease diagnosis and risk prediction. To address the limitations of 2D CXRs, this study introduces Xray2Xray, a novel World Model that learns latent representations encoding 3D structural information from chest X-rays. Xray2Xray captures the latent representations of the chest volume by modeling the transition dynamics of X-ray projections across different angular positions with a vision model and a transition model. We employed the latent representations of Xray2Xray for downstream risk prediction and disease diagnosis tasks. Experimental results showed that Xray2Xray outperformed both supervised methods and self-supervised pretraining methods for cardiovascular disease risk estimation and achieved competitive performance in classifying five pathologies in CXRs. We also assessed the quality of Xray2Xray’s latent representations through synthesis tasks and demonstrated that the latent representations can be used to reconstruct volumetric context.
zh
[CV-108] NIC-RobustBench: A Comprehensive Open-Source Toolkit for Neural Image Compression and Robustness Analysis
【速读】：该论文旨在解决神经网络在神经图像压缩（NIC）中的对抗鲁棒性评估问题，特别是针对JPEG AI这一首个端到端神经图像压缩标准的鲁棒性评估。现有研究受限于有限的编解码器和攻击类型，无法全面评估NIC系统的安全性。该论文提出的解决方案是构建\textbfNIC-RobustBench，这是一个开源框架，能够评估NIC的鲁棒性及对抗防御的有效性，并与率失真（RD）性能进行比较。其关键在于整合了最多编解码器的NIC库，并具备良好的可扩展性，从而为NIC系统的安全性分析提供了全面且可复现的评估平台。
链接: https://arxiv.org/abs/2506.19051
作者: Georgii Bychkov,Khaled Abud,Egor Kovalev,Alexander Gushchin,Dmitriy Vatolin,Anastasia Antsiferova
机构: ISP RAS Research Center for Trusted Artificial Intelligence (ISP RAS 信任人工智能研究中心); Lomonosov Moscow State University (莫斯科国立大学)
类目: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
备注: arXiv admin note: text overlap with arXiv:2411.11795
Abstract:Adversarial robustness of neural networks is an increasingly important area of research, combining studies on computer vision models, large language models (LLMs), and others. With the release of JPEG AI – the first standard for end-to-end neural image compression (NIC) methods – the question of evaluating NIC robustness has become critically significant. However, previous research has been limited to a narrow range of codecs and attacks. To address this, we present \textbfNIC-RobustBench, the first open-source framework to evaluate NIC robustness and adversarial defenses’ efficiency, in addition to comparing Rate-Distortion (RD) performance. The framework includes the largest number of codecs among all known NIC libraries and is easily scalable. The paper demonstrates a comprehensive overview of the NIC-RobustBench framework and employs it to analyze NIC robustness. Our code is available online at this https URL.
zh
人工智能
[AI-0] JoyAgents -R1: Joint Evolution Dynamics for Versatile Multi-LLM Agents with Reinforcement Learning
【速读】：该论文旨在解决多智能体强化学习（MARL）中异构智能体联合进化面临的合作低效和训练不稳定问题。其解决方案的关键在于提出了一种名为JoyAgents-R1的联合进化动态机制，该机制首次将群体相对策略优化（GRPO）应用于异构多智能体的联合训练，通过迭代优化智能体的大语言模型（LLMs）和记忆，实现决策能力和记忆能力的全局平衡。核心创新包括节点级蒙特卡洛采样以提升GRPO采样效率并保持策略多样性，以及基于边际收益的选择策略，通过识别高奖励波动的采样组进行针对性模型更新，从而提高训练稳定性和联合效益。此外，还引入了自适应记忆演化机制，利用GRPO奖励作为无成本监督信号，减少重复推理并加速收敛。
链接: https://arxiv.org/abs/2506.19846
作者: Ai Han,Junxing Hu,Pu Wei,Zhiqian Zhang,Yuhang Guo,Jiawei Lu,Zicheng Zhang
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注: 33 pages, 7 figures, under review
Abstract:Multi-agent reinforcement learning (MARL) has emerged as a prominent paradigm for increasingly complex tasks. However, joint evolution across heterogeneous agents remains challenging due to cooperative inefficiency and training instability. In this paper, we propose the joint evolution dynamics for MARL called JoyAgents-R1, which first applies Group Relative Policy Optimization (GRPO) to the joint training of heterogeneous multi-agents. By iteratively refining agents’ large language models (LLMs) and memories, the method achieves holistic equilibrium with optimal decision-making and memory capabilities. Specifically, JoyAgents-R1 first implements node-wise Monte Carlo sampling on the behavior of each agent across entire reasoning trajectories to enhance GRPO sampling efficiency while maintaining policy diversity. Then, our marginal benefit-driven selection strategy identifies top- K sampling groups with maximal reward fluctuations, enabling targeted agent model updates that improve training stability and maximize joint benefits through cost-effective parameter adjustments. Meanwhile, JoyAgents-R1 introduces an adaptive memory evolution mechanism that repurposes GRPO rewards as cost-free supervisory signals to eliminate repetitive reasoning and accelerate convergence. Experiments across general and domain-specific scenarios demonstrate that JoyAgents-R1 achieves performance comparable to that of larger LLMs while built on smaller open-source models.
zh
[AI-1] mporal-IRL: Modeling Port Congestion and Berth Scheduling with Inverse Reinforcement Learning
【速读】：该论文旨在解决港口拥堵预测问题，以提升全球供应链的可靠性。其解决方案的关键在于通过分析船舶行为和在特定码头的停留时间，特别是基于不同条件下的泊位调度，来捕捉泊位调度的底层优先级和模式。研究利用历史自动识别系统（AIS）数据重构泊位调度，并通过逆强化学习（Inverse Reinforcement Learning, IRL）确定奖励函数，进而构建Temporal-IRL模型，以预测船舶顺序和港口停留时间，从而实现对港口拥堵的准确预测。
链接: https://arxiv.org/abs/2506.19843
作者: Guo Li,Zixiang Xu,Wei Zhang,Yikuan Hu,Xinyu Yang,Nikolay Aristov,Mingjie Tang,Elenna R Dugundji
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注: TRB2025
Abstract:Predicting port congestion is crucial for maintaining reliable global supply chains. Accurate forecasts enableimprovedshipment planning, reducedelaysand costs, and optimizeinventoryanddistributionstrategies, thereby ensuring timely deliveries and enhancing supply chain resilience. To achieve accurate predictions, analyzing vessel behavior and their stay times at specific port terminals is essential, focusing particularly on berth scheduling under various conditions. Crucially, the model must capture and learn the underlying priorities and patterns of berth scheduling. Berth scheduling and planning are influenced by a range of factors, including incoming vessel size, waiting times, and the status of vessels within the port terminal. By observing historical Automatic Identification System (AIS) positions of vessels, we reconstruct berth schedules, which are subsequently utilized to determine the reward function via Inverse Reinforcement Learning (IRL). For this purpose, we modeled a specific terminal at the Port of New York/New Jersey and developed Temporal-IRL. This Temporal-IRL model learns berth scheduling to predict vessel sequencing at the terminal and estimate vessel port stay, encompassing both waiting and berthing times, to forecast port congestion. Utilizing data from Maher Terminal spanning January 2015 to September 2023, we trained and tested the model, achieving demonstrably excellent results.
zh
[AI-2] Persona Features Control Emergent Misalignment
【速读】：该论文试图解决语言模型在训练后对更广泛部署分布中的行为泛化问题，特别是关注生成式 AI (Generative AI) 在不同场景下可能出现的“涌现不对齐”（emergent misalignment）现象。其解决方案的关键在于通过“模型差分”（model diffing）方法，利用稀疏自编码器对比微调前后模型的内部表示，从而识别出导致不对齐的“错误人格特征”（misaligned persona features），其中毒性人格特征被证明是控制涌现不对齐的核心因素，并可用于预测模型是否会出现此类行为。此外，研究还表明对少量良性样本进行微调可有效恢复模型的对齐性。
链接: https://arxiv.org/abs/2506.19823
作者: Miles Wang,Tom Dupré la Tour,Olivia Watkins,Alex Makelov,Ryan A. Chi,Samuel Miserendino,Johannes Heidecke,Tejal Patwardhan,Dan Mossing
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Understanding how language models generalize behaviors from their training to a broader deployment distribution is an important problem in AI safety. Betley et al. discovered that fine-tuning GPT-4o on intentionally insecure code causes “emergent misalignment,” where models give stereotypically malicious responses to unrelated prompts. We extend this work, demonstrating emergent misalignment across diverse conditions, including reinforcement learning on reasoning models, fine-tuning on various synthetic datasets, and in models without safety training. To investigate the mechanisms behind this generalized misalignment, we apply a “model diffing” approach using sparse autoencoders to compare internal model representations before and after fine-tuning. This approach reveals several “misaligned persona” features in activation space, including a toxic persona feature which most strongly controls emergent misalignment and can be used to predict whether a model will exhibit such behavior. Additionally, we investigate mitigation strategies, discovering that fine-tuning an emergently misaligned model on just a few hundred benign samples efficiently restores alignment.
zh
[AI-3] Learning Task Belief Similarity with Latent Dynamics for Meta-Reinforcement Learning ICLR2025
【速读】：该论文旨在解决元强化学习（meta-reinforcement learning）中在稀疏奖励环境下高效任务识别与适应的问题。传统基于贝叶斯自适应的深度强化学习方法依赖于重构环境奖励信号，这在稀疏奖励场景下表现不佳。论文提出的解决方案关键在于引入SimBelief框架，通过测量贝叶斯自适应马尔可夫决策过程（BAMDP）中的任务信念相似性，提取相似任务分布的共同特征，从而提升任务识别效率和探索性能。该方法通过学习潜在任务信念度量，并将其与具体任务信念相结合，实现跨任务的共享特征学习，进而提升在稀疏奖励环境下的适应能力。
链接: https://arxiv.org/abs/2506.19785
作者: Menglong Zhang,Fuyuan Qian
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注: ICLR2025 this https URL
Abstract:Meta-reinforcement learning requires utilizing prior task distribution information obtained during exploration to rapidly adapt to unknown tasks. The efficiency of an agent’s exploration hinges on accurately identifying the current task. Recent Bayes-Adaptive Deep RL approaches often rely on reconstructing the environment’s reward signal, which is challenging in sparse reward settings, leading to suboptimal exploitation. Inspired by bisimulation metrics, which robustly extracts behavioral similarity in continuous MDPs, we propose SimBelief-a novel meta-RL framework via measuring similarity of task belief in Bayes-Adaptive MDP (BAMDP). SimBelief effectively extracts common features of similar task distributions, enabling efficient task identification and exploration in sparse reward environments. We introduce latent task belief metric to learn the common structure of similar tasks and incorporate it into the specific task belief. By learning the latent dynamics across task distributions, we connect shared latent task belief features with specific task features, facilitating rapid task identification and adaptation. Our method outperforms state-of-the-art baselines on sparse reward MuJoCo and panda-gym tasks.
zh
[AI-4] SAGE: Strategy-Adaptive Generation Engine for Query Rewriting
【速读】：该论文旨在解决密集检索中查询重写（query rewriting）效果受限的问题，传统方法依赖大规模监督数据或效率低下的强化学习（Reinforcement Learning, RL）探索。其解决方案的关键在于利用少量专家设计的策略（如语义扩展和实体消歧）引导大型语言模型（Large Language Models, LLMs），并通过引入策略自适应生成引擎（Strategy-Adaptive Generation Engine, SAGE）在强化学习框架中实现这些策略。SAGE采用两种新颖的奖励 shaping 机制——策略信用分配（Strategic Credit Shaping, SCS）和对比奖励 shaping（Contrastive Reward Shaping, CRS），以提供更有效的学习信号，从而提升检索效果并降低推理成本。
链接: https://arxiv.org/abs/2506.19783
作者: Teng Wang,Hailei Gong,Changwang Zhang,Jun Wang
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Query rewriting is pivotal for enhancing dense retrieval, yet current methods demand large-scale supervised data or suffer from inefficient reinforcement learning (RL) exploration. In this work, we first establish that guiding Large Language Models (LLMs) with a concise set of expert-crafted strategies, such as semantic expansion and entity disambiguation, substantially improves retrieval effectiveness on challenging benchmarks, including HotpotQA, FEVER, NFCorpus, and SciFact. Building on this insight, we introduce the Strategy-Adaptive Generation Engine (SAGE), which operationalizes these strategies in an RL framework. SAGE introduces two novel reward shaping mechanisms-Strategic Credit Shaping (SCS) and Contrastive Reward Shaping (CRS)-to deliver more informative learning signals. This strategy-guided approach not only achieves new state-of-the-art NDCG@10 results, but also uncovers a compelling emergent behavior: the agent learns to select optimal strategies, reduces unnecessary exploration, and generates concise rewrites, lowering inference cost without sacrificing performance. Our findings demonstrate that strategy-guided RL, enhanced with nuanced reward shaping, offers a scalable, efficient, and more interpretable paradigm for developing the next generation of robust information retrieval systems.
zh
[AI-5] Alleviating User-Sensitive bias with Fair Generative Sequential Recommendation Model
【速读】：该论文旨在解决推荐系统中的公平性问题，即由于用户敏感特征（如性别和年龄）的相似性导致推荐模型捕捉到敏感特征的强相关偏好，从而引发推荐不公平的现象。解决方案的关键在于利用扩散模型（Diffusion Model, DM）的强大建模能力，通过在训练阶段注入随机噪声并设计序列去噪模型，以及在生成结果中引入消除敏感用户特征偏见的多兴趣表示信息，从而有效建模推荐公平性并增强多样性。在推理阶段，模型通过历史交互获取噪声并进行逆向迭代以重建目标物品表示，最终实现准确性和公平性的双重提升。
链接: https://arxiv.org/abs/2506.19777
作者: Yang Liu,Feng Wu,Xuefang Zhu
机构: 未知
类目: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
备注:
Abstract:Recommendation fairness has recently attracted much attention. In the real world, recommendation systems are driven by user behavior, and since users with the same sensitive feature (e.g., gender and age) tend to have the same patterns, recommendation models can easily capture the strong correlation preference of sensitive features and thus cause recommendation unfairness. Diffusion model (DM) as a new generative model paradigm has achieved great success in recommendation systems. DM’s ability to model uncertainty and represent diversity, and its modeling mechanism has a high degree of adaptability with the real-world recommendation process with bias. Therefore, we use DM to effectively model the fairness of recommendation and enhance the diversity. This paper proposes a FairGENerative sequential Recommendation model based on DM, FairGENRec. In the training phase, we inject random noise into the original distribution under the guidance of the sensitive feature recognition model, and a sequential denoise model is designed for the reverse reconstruction of items. Simultaneously, recommendation fairness modeling is completed by injecting multi-interests representational information that eliminates the bias of sensitive user features into the generated results. In the inference phase, the model obtains the noise in the form of noise addition by using the history interactions which is followed by reverse iteration to reconstruct the target item representation. Finally, our extensive experiments on three datasets demonstrate the dual enhancement effect of FairGENRec on accuracy and fairness, while the statistical analysis of the cases visualizes the degree of improvement on the fairness of the recommendation.
zh
[AI-6] Automatic Prompt Optimization for Knowledge Graph Construction: Insights from an Empirical Study
【速读】：该论文试图解决在知识图谱（Knowledge Graph, KG）构建中，通过自动提示优化技术提升三元组（subject-relation-object）抽取任务的效果问题。传统方法依赖人工设计任务特定的提示，不仅耗时且易受模型变化影响。解决方案的关键在于采用自动提示优化技术，通过输入输出示例生成最优或近似最优的提示，从而提高三元组抽取的性能，尤其在模式复杂度和文本规模增加时表现更为显著。
链接: https://arxiv.org/abs/2506.19773
作者: Nandana Mihindukulasooriya,Niharika S. D’Souza,Faisal Chowdhury,Horst Samulowitz
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:A KG represents a network of entities and illustrates relationships between them. KGs are used for various applications, including semantic search and discovery, reasoning, decision-making, natural language processing, machine learning, and recommendation systems. Triple (subject-relation-object) extraction from text is the fundamental building block of KG construction and has been widely studied, for example, in early benchmarks such as ACE 2002 to more recent ones, such as WebNLG 2020, REBEL and SynthIE. While the use of LLMs is explored for KG construction, handcrafting reasonable task-specific prompts for LLMs is a labour-intensive exercise and can be brittle due to subtle changes in the LLM models employed. Recent work in NLP tasks (e.g. autonomy generation) uses automatic prompt optimization/engineering to address this challenge by generating optimal or near-optimal task-specific prompts given input-output examples. This empirical study explores the application of automatic prompt optimization for the triple extraction task using experimental benchmarking. We evaluate different settings by changing (a) the prompting strategy, (b) the LLM being used for prompt optimization and task execution, © the number of canonical relations in the schema (schema complexity), (d) the length and diversity of input text, (e) the metric used to drive the prompt optimization, and (f) the dataset being used for training and testing. We evaluate three different automatic prompt optimizers, namely, DSPy, APE, and TextGrad and use two different triple extraction datasets, SynthIE and REBEL. Through rigorous empirical evaluation, our main contribution highlights that automatic prompt optimization techniques can generate reasonable prompts similar to humans for triple extraction. In turn, these optimized prompts achieve improved results, particularly with increasing schema complexity and text size. Subjects: Artificial Intelligence (cs.AI) ACMclasses: I.2.7; I.2.4 Cite as: arXiv:2506.19773 [cs.AI] (or arXiv:2506.19773v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2506.19773 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
zh
[AI-7] A Survey of Multi-sensor Fusion Perception for Embodied AI: Background Methods Challenges and Prospects
【速读】：该论文旨在解决现有多传感器融合感知（Multi-sensor fusion perception, MSFP）综述文献中存在的局限性，包括研究范围过于狭窄、仅聚焦单一任务或领域，以及缺乏对MSFP方法多样性的全面考虑。其解决方案的关键在于从任务无关（task-agnostic）的角度组织MSFP研究，从多种技术视角系统地报告相关方法，涵盖多模态融合、多智能体融合、时序融合以及大语言模型（Large Language Model, LLM）的多模态融合，并探讨其未来挑战与发展方向。
链接: https://arxiv.org/abs/2506.19769
作者: Shulan Ruan,Rongwei Wang,Xuchen Shen,Huijie Liu,Baihui Xiao,Jun Shi,Kun Zhang,Zhenya Huang,Yu Liu,Enhong Chen,You He
机构: 未知
类目: Multimedia (cs.MM); Artificial Intelligence (cs.AI)
备注:
Abstract:Multi-sensor fusion perception (MSFP) is a key technology for embodied AI, which can serve a variety of downstream tasks (e.g., 3D object detection and semantic segmentation) and application scenarios (e.g., autonomous driving and swarm robotics). Recently, impressive achievements on AI-based MSFP methods have been reviewed in relevant surveys. However, we observe that the existing surveys have some limitations after a rigorous and detailed investigation. For one thing, most surveys are oriented to a single task or research field, such as 3D object detection or autonomous driving. Therefore, researchers in other related tasks often find it difficult to benefit directly. For another, most surveys only introduce MSFP from a single perspective of multi-modal fusion, while lacking consideration of the diversity of MSFP methods, such as multi-view fusion and time-series fusion. To this end, in this paper, we hope to organize MSFP research from a task-agnostic perspective, where methods are reported from various technical views. Specifically, we first introduce the background of MSFP. Next, we review multi-modal and multi-agent fusion methods. A step further, time-series fusion methods are analyzed. In the era of LLM, we also investigate multimodal LLM fusion methods. Finally, we discuss open challenges and future directions for MSFP. We hope this survey can help researchers understand the important progress in MSFP and provide possible insights for future research.
zh
[AI-8] Cross-regularization: Adaptive Model Complexity through Validation Gradients ICML2025
【速读】：该论文试图解决模型正则化过程中需要大量手动调参以平衡模型复杂度与过拟合的问题。其解决方案的关键在于交叉正则化（cross-regularization），通过在训练过程中利用验证梯度直接适应正则化参数，从而实现对复杂度的动态控制。该方法将参数优化分解为训练数据引导特征学习、验证数据塑造复杂度控制的两个过程，并可证明收敛至交叉验证最优解。
链接: https://arxiv.org/abs/2506.19755
作者: Carlos Stein Brito
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Statistics Theory (math.ST); Machine Learning (stat.ML)
备注: 21 pages, 13 figures. Accepted at ICML 2025
Abstract:Model regularization requires extensive manual tuning to balance complexity against overfitting. Cross-regularization resolves this tradeoff by directly adapting regularization parameters through validation gradients during training. The method splits parameter optimization - training data guides feature learning while validation data shapes complexity controls - converging provably to cross-validation optima. When implemented through noise injection in neural networks, this approach reveals striking patterns: unexpectedly high noise tolerance and architecture-specific regularization that emerges organically during training. Beyond complexity control, the framework integrates seamlessly with data augmentation, uncertainty calibration and growing datasets while maintaining single-run efficiency through a simple gradient-based approach.
zh
[AI-9] Who Does What in Deep Learning? Multidimensional Game-Theoretic Attribution of Function of Neural Units
【速读】：该论文试图解决如何量化神经网络中每个神经单元对高维输出（如文本、图像或语音）的贡献问题，现有可解释AI方法（如SHAP）虽能评估输入的重要性，但无法在数千个输出像素、标记或logits上准确衡量神经单元的贡献。其解决方案的关键是提出一种模型无关的博弈论框架——多重扰动Shapley值分析（Multiperturbation Shapley-value Analysis, MSA），通过系统性地破坏单元组合，生成与模型输出维度一致的单元级贡献图，从而实现对神经网络内部机制的深入解析。
链接: https://arxiv.org/abs/2506.19732
作者: Shrey Dixit,Kayson Fakhar,Fatemeh Hadaeghi,Patrick Mineault,Konrad P. Kording,Claus C. Hilgetag
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Neural networks now generate text, images, and speech with billions of parameters, producing a need to know how each neural unit contributes to these high-dimensional outputs. Existing explainable-AI methods, such as SHAP, attribute importance to inputs, but cannot quantify the contributions of neural units across thousands of output pixels, tokens, or logits. Here we close that gap with Multiperturbation Shapley-value Analysis (MSA), a model-agnostic game-theoretic framework. By systematically lesioning combinations of units, MSA yields Shapley Modes, unit-wise contribution maps that share the exact dimensionality of the model’s output. We apply MSA across scales, from multi-layer perceptrons to the 56-billion-parameter Mixtral-8x7B and Generative Adversarial Networks (GAN). The approach demonstrates how regularisation concentrates computation in a few hubs, exposes language-specific experts inside the LLM, and reveals an inverted pixel-generation hierarchy in GANs. Together, these results showcase MSA as a powerful approach for interpreting, editing, and compressing deep neural networks.
zh
[AI-10] Geometric-Aware Variational Inference: Robust and Adaptive Regularization with Directional Weight Uncertainty
【速读】：该论文旨在解决深度神经网络中不确定性量化（uncertainty quantification）的问题，现有变分推断方法通常在权重空间中采用各向同性高斯近似，这种近似无法准确反映网络固有的几何结构。解决方案的关键在于引入了浓度自适应扰动（Concentration-Adapted Perturbations, CAP），该方法通过使用冯·米塞斯-费舍尔分布（von Mises-Fisher distributions）在单位超球面上直接建模权重不确定性，从而更好地匹配网络的几何特性。CAP的核心贡献是推导出一个解析表达式，将vMF分布的浓度参数与激活噪声方差联系起来，使每一层能够通过一种新的闭式KL散度正则化器学习其最优不确定性水平。
链接: https://arxiv.org/abs/2506.19726
作者: Carlos Stein Brito
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
备注: 19 pages, 4 figures
Abstract:Deep neural networks require principled uncertainty quantification, yet existing variational inference methods often employ isotropic Gaussian approximations in weight space that poorly match the network’s inherent geometry. We address this mismatch by introducing Concentration-Adapted Perturbations (CAP), a variational framework that models weight uncertainties directly on the unit hypersphere using von Mises-Fisher distributions. Building on recent work in radial-directional posterior decompositions and spherical weight constraints, CAP provides the first complete theoretical framework connecting directional statistics to practical noise regularization in neural networks. Our key contribution is an analytical derivation linking vMF concentration parameters to activation noise variance, enabling each layer to learn its optimal uncertainty level through a novel closed-form KL divergence regularizer. In experiments on CIFAR-10, CAP significantly improves model calibration - reducing Expected Calibration Error by 5.6x - while providing interpretable layer-wise uncertainty profiles. CAP requires minimal computational overhead and integrates seamlessly into standard architectures, offering a theoretically grounded yet practical approach to uncertainty quantification in deep learning.
zh
[AI-11] From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking
【速读】：该论文试图解决AI代理在不同代码起点条件下实现并运行机器学习实验的能力评估问题，特别是在从代码复现到从零复现的连续范围内。其解决方案的关键在于提出AutoExperiment基准，该基准通过给定研究论文、部分遮蔽的代码库以及运行指令，要求代理生成缺失代码并在沙箱环境中执行实验以复现结果，从而评估AI代理在长周期代码生成、上下文检索和自主实验执行方面的能力。
链接: https://arxiv.org/abs/2506.19724
作者: Gyeongwon James Kim,Alex Wilf,Louis-Philippe Morency,Daniel Fried
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Recent progress in autonomous code generation has fueled excitement around AI agents capable of accelerating scientific discovery by running experiments. However, there is currently no benchmark that evaluates whether such agents can implement scientific ideas when given varied amounts of code as a starting point, interpolating between reproduction (running code) and from-scratch replication (fully re-implementing and running code). We introduce AutoExperiment, a benchmark that evaluates AI agents’ ability to implement and run machine learning experiments based on natural language descriptions in research papers. In each task, agents are given a research paper, a codebase with key functions masked out, and a command to run the experiment. The goal is to generate the missing code, execute the experiment in a sandboxed environment, and reproduce the results. AutoExperiment scales in difficulty by varying the number of missing functions n , ranging from partial reproduction to full replication. We evaluate state-of-the-art agents and find that performance degrades rapidly as n increases. Agents that can dynamically interact with the environment (e.g. to debug their code) can outperform agents in fixed “agentless” harnesses, and there exists a significant gap between single-shot and multi-trial success rates (Pass@1 vs. Pass@5), motivating verifier approaches to our benchmark. Our findings highlight critical challenges in long-horizon code generation, context retrieval, and autonomous experiment execution, establishing AutoExperiment as a new benchmark for evaluating progress in AI-driven scientific experimentation. Our data and code are open-sourced at this https URL .
zh
[AI-12] LLM -Driven Medical Document Analysis: Enhancing Trustworthy Pathology and Differential Diagnosis ICDAR2025
【速读】：该论文旨在解决医疗文档分析中因敏感患者数据隐私问题而限制在线大型语言模型（LLMs）服务在临床环境中应用的问题，同时提升对重叠症状进行鉴别诊断的准确性。解决方案的关键在于提出一个可信的医疗文档分析平台，该平台通过低秩适应（LoRA）微调LLaMA-v3模型，并针对鉴别诊断任务进行优化，结合DDXPlus这一最大的鉴别诊断基准数据集，实现了病理预测和变长鉴别诊断的优越性能。
链接: https://arxiv.org/abs/2506.19702
作者: Lei Kang,Xuanshuo Fu,Oriol Ramos Terrades,Javier Vazquez-Corral,Ernest Valveny,Dimosthenis Karatzas
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注: Accepted at ICDAR 2025
Abstract:Medical document analysis plays a crucial role in extracting essential clinical insights from unstructured healthcare records, supporting critical tasks such as differential diagnosis. Determining the most probable condition among overlapping symptoms requires precise evaluation and deep medical expertise. While recent advancements in large language models (LLMs) have significantly enhanced performance in medical document analysis, privacy concerns related to sensitive patient data limit the use of online LLMs services in clinical settings. To address these challenges, we propose a trustworthy medical document analysis platform that fine-tunes a LLaMA-v3 using low-rank adaptation, specifically optimized for differential diagnosis tasks. Our approach utilizes DDXPlus, the largest benchmark dataset for differential diagnosis, and demonstrates superior performance in pathology prediction and variable-length differential diagnosis compared to existing methods. The developed web-based platform allows users to submit their own unstructured medical documents and receive accurate, explainable diagnostic results. By incorporating advanced explainability techniques, the system ensures transparent and reliable predictions, fostering user trust and confidence. Extensive evaluations confirm that the proposed method surpasses current state-of-the-art models in predictive accuracy while offering practical utility in clinical settings. This work addresses the urgent need for reliable, explainable, and privacy-preserving artificial intelligence solutions, representing a significant advancement in intelligent medical document analysis for real-world healthcare applications. The code can be found at \hrefthis https URLthis https URL.
zh
[AI-13] oward Decision-Oriented Prognostics: An Integrated Estimate-Optimize Framework for Predictive Maintenance
【速读】：该论文旨在解决预测性维护（PdM）中由于模型误设导致的不确定性问题，进而影响工业应用中的维护决策质量。传统估计-优化（ETO）框架中，概率预测误差可能导致不一致和次优的维护决策，而该研究提出了一种集成估计-优化（IEO）框架，其关键在于将预测模型的调优与维护结果的直接优化相结合，从而提升决策的一致性和鲁棒性。通过理论分析和实验验证，IEO框架在小样本故障数据集上表现出色，显著降低了平均维护遗憾，尤其在决策策略与目标不匹配时效果更为明显。
链接: https://arxiv.org/abs/2506.19698
作者: Zhuojun Xie,Adam Abdin,Yiping Fang
机构: 未知
类目: Artificial Intelligence (cs.AI); Optimization and Control (math.OC)
备注: 22 pages, 5 figures, 4 tables
Abstract:Recent research increasingly integrates machine learning (ML) into predictive maintenance (PdM) to reduce operational and maintenance costs in data-rich operational settings. However, uncertainty due to model misspecification continues to limit widespread industrial adoption. This paper proposes a PdM framework in which sensor-driven prognostics inform decision-making under economic trade-offs within a finite decision space. We investigate two key questions: (1) Does higher predictive accuracy necessarily lead to better maintenance decisions? (2) If not, how can the impact of prediction errors on downstream maintenance decisions be mitigated? We first demonstrate that in the traditional estimate-then-optimize (ETO) framework, errors in probabilistic prediction can result in inconsistent and suboptimal maintenance decisions. To address this, we propose an integrated estimate-optimize (IEO) framework that jointly tunes predictive models while directly optimizing for maintenance outcomes. We establish theoretical finite-sample guarantees on decision consistency under standard assumptions. Specifically, we develop a stochastic perturbation gradient descent algorithm suitable for small run-to-failure datasets. Empirical evaluations on a turbofan maintenance case study show that the IEO framework reduces average maintenance regret up to 22% compared to ETO. This study provides a principled approach to managing prediction errors in data-driven PdM. By aligning prognostic model training with maintenance objectives, the IEO framework improves robustness under model misspecification and improves decision quality. The improvement is particularly pronounced when the decision-making policy is misaligned with the decision-maker’s target. These findings support more reliable maintenance planning in uncertain operational environments.
zh
[AI-14] When Can We Reuse a Calibration Set for Multiple Conformal Predictions?
【速读】：该论文试图解决传统归纳同余预测（Inductive Conformal Prediction, ICP）在实际应用中需要为每次新预测使用独立校准集的问题，从而限制了其实用性。解决方案的关键在于结合e-同余预测与Hoeffding不等式，使得单个校准集可以以高概率重复使用，同时保持所需的覆盖率。通过在CIFAR-10数据集上的案例研究，作者训练了一个深度神经网络，并利用校准集估计Hoeffding修正项，进而应用修正后的马尔可夫不等式构建具有可量化置信度的预测集，从而在保证理论性能的同时提升了方法的实用性。
链接: https://arxiv.org/abs/2506.19689
作者: A.A. Balinsky,A.D. Balinsky
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Statistics Theory (math.ST)
备注:
Abstract:Reliable uncertainty quantification is crucial for the trustworthiness of machine learning applications. Inductive Conformal Prediction (ICP) offers a distribution-free framework for generating prediction sets or intervals with user-specified confidence. However, standard ICP guarantees are marginal and typically require a fresh calibration set for each new prediction to maintain their validity. This paper addresses this practical limitation by demonstrating how e-conformal prediction, in conjunction with Hoeffding’s inequality, can enable the repeated use of a single calibration set with a high probability of preserving the desired coverage. Through a case study on the CIFAR-10 dataset, we train a deep neural network and utilise a calibration set to estimate a Hoeffding correction. This correction allows us to apply a modified Markov’s inequality, leading to the construction of prediction sets with quantifiable confidence. Our results illustrate the feasibility of maintaining provable performance in conformal prediction while enhancing its practicality by reducing the need for repeated calibration. The code for this work is publicly available.
zh
[AI-15] From memories to maps: Mechanisms of in context reinforcement learning in transformers
【速读】：该论文试图解决在少量经验下实现快速适应的问题，这一能力在人类和动物中表现显著，但传统强化学习算法难以有效模拟。其解决方案的关键在于利用Transformer架构中的记忆机制，通过在上下文中缓存中间计算结果，并在决策时调用这些记忆，从而支持快速的强化学习策略。此外，研究还发现模型中形成的表征与大脑海马-内嗅系统相关的计算具有相似性，表明该方法可能为人工和自然环境中的上下文学习提供了机制层面的解释。
链接: https://arxiv.org/abs/2506.19686
作者: Ching Fang,Kanaka Rajan
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Humans and animals show remarkable learning efficiency, adapting to new environments with minimal experience. This capability is not well captured by standard reinforcement learning algorithms that rely on incremental value updates. Rapid adaptation likely depends on episodic memory – the ability to retrieve specific past experiences to guide decisions in novel contexts. Transformers provide a useful setting for studying these questions because of their ability to learn rapidly in-context and because their key-value architecture resembles episodic memory systems in the brain. We train a transformer to in-context reinforcement learn in a distribution of planning tasks inspired by rodent behavior. We then characterize the learning algorithms that emerge in the model. We first find that representation learning is supported by in-context structure learning and cross-context alignment, where representations are aligned across environments with different sensory stimuli. We next demonstrate that the reinforcement learning strategies developed by the model are not interpretable as standard model-free or model-based planning. Instead, we show that in-context reinforcement learning is supported by caching intermediate computations within the model’s memory tokens, which are then accessed at decision time. Overall, we find that memory may serve as a computational resource, storing both raw experience and cached computations to support flexible behavior. Furthermore, the representations developed in the model resemble computations associated with the hippocampal-entorhinal system in the brain, suggesting that our findings may be relevant for natural cognition. Taken together, our work offers a mechanistic hypothesis for the rapid adaptation that underlies in-context learning in artificial and natural settings.
zh
[AI-16] Identifying Macro Causal Effects in C-DMGs over DMGs UAI2025
【速读】：该论文试图解决在存在循环结构的因果模型中，如何有效识别宏观层面因果效应的问题。传统方法依赖于无环有向混合图（ADMGs）和do-演算，但在现实系统中，循环因果动态普遍存在，这限制了其适用性。为此，论文提出通过输入输出结构因果模型（ioSCMs）诱导的有向混合图（DMGs）来建模循环结构，并在此基础上定义集群有向混合图（C-DMGs）作为变量簇间的高层次因果关系表示。解决方案的关键在于证明，在DMGs基础上的C-DMGs中，do-演算在无需额外假设的情况下，仍然是识别宏观因果效应的可靠工具，且原有针对C-DMGs over ADMGs的不可识别性图示准则可自然扩展至部分C-DMGs over DMGs。
链接: https://arxiv.org/abs/2506.19650
作者: Simon Ferreira,Charles K. Assaad
机构: 未知
类目: Artificial Intelligence (cs.AI); Methodology (stat.ME)
备注: Accepted to the UAI2025 workshop on Causal Abstractions and Representations. arXiv admin note: substantial text overlap with arXiv:2504.01551
Abstract:The do-calculus is a sound and complete tool for identifying causal effects in acyclic directed mixed graphs (ADMGs) induced by structural causal models (SCMs). However, in many real-world applications, especially in high-dimensional setting, constructing a fully specified ADMG is often infeasible. This limitation has led to growing interest in partially specified causal representations, particularly through cluster-directed mixed graphs (C-DMGs), which group variables into clusters and offer a more abstract yet practical view of causal dependencies. While these representations can include cycles, recent work has shown that the do-calculus remains sound and complete for identifying macro-level causal effects in C-DMGs over ADMGs under the assumption that all clusters size are greater than 1. Nevertheless, real-world systems often exhibit cyclic causal dynamics at the structural level. To account for this, input-output structural causal models (ioSCMs) have been introduced as a generalization of SCMs that allow for cycles. ioSCMs induce another type of graph structure known as a directed mixed graph (DMG). Analogous to the ADMG setting, one can define C-DMGs over DMGs as high-level representations of causal relations among clusters of variables. In this paper, we prove that, unlike in the ADMG setting, the do-calculus is unconditionally sound and complete for identifying macro causal effects in C-DMGs over DMGs. Furthermore, we show that the graphical criteria for non-identifiability of macro causal effects previously established C-DMGs over ADMGs naturally extends to a subset of C-DMGs over DMGs.
zh
[AI-17] he receptron is a nonlinear threshold logic gate with intrinsic multi-dimensional selective capabilities for analog inputs
【速读】：该论文试图解决传统阈值逻辑门（Threshold Logic Gates, TLGs）因线性特性导致的分类能力受限问题，从而难以完成复杂任务。其解决方案的关键在于提出一种名为receptron的广义模型，该模型通过引入输入依赖的权重函数，显著提升了分类性能，即使仅使用单个单元也能实现高精度的分类，尤其在三维空间内的立方域输入下表现出内在的选择性激活特性。
链接: https://arxiv.org/abs/2506.19642
作者: B. Paroli,F. Borghi,M.A.C. Potenza,P. Milani
机构: 未知
类目: Emerging Technologies (cs.ET); Artificial Intelligence (cs.AI)
备注: 12 pages, 7 figures
Abstract:Threshold logic gates (TLGs) have been proposed as artificial counterparts of biological neurons with classification capabilities based on a linear predictor function combining a set of weights with the feature vector. The linearity of TLGs limits their classification capabilities requiring the use of networks for the accomplishment of complex tasks. A generalization of the TLG model called receptron, characterized by input-dependent weight functions allows for a significant enhancement of classification performances even with the use of a single unit. Here we formally demonstrate that a receptron, characterized by nonlinear input-dependent weight functions, exhibit intrinsic selective activation properties for analog inputs, when the input vector is within cubic domains in a 3D space. The proposed model can be extended to the n-dimensional case for multidimensional applications. Our results suggest that receptron-based networks can represent a new class of devices capable to manage a large number of analog inputs, for edge applications requiring high selectivity and classification capabilities without the burden of complex training.
zh
[AI-18] On the efficacy of old features for the detection of new bots
【速读】：该论文试图解决的是在线生态系统中恶意机器人（bot）的检测问题，这类机器人被设计用于传播垃圾信息、支持公众人物并最终影响公众意见。解决方案的关键在于利用多种先进的特征集进行检测，包括基于账户资料和时间线的特征集、来自用户推文的Twitter客户端信息，以及流行机器人检测工具Botometer的输出分数。研究强调了通用分类器和低成本计算的账户特征在识别进化型机器人中的潜在应用价值。
链接: https://arxiv.org/abs/2506.19635
作者: Rocco De Nicola,Marinella Petrocchi,Manuel Pratelli
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注: pre-print version
Abstract:For more than a decade now, academicians and online platform administrators have been studying solutions to the problem of bot detection. Bots are computer algorithms whose use is far from being benign: malicious bots are purposely created to distribute spam, sponsor public characters and, ultimately, induce a bias within the public opinion. To fight the bot invasion on our online ecosystem, several approaches have been implemented, mostly based on (supervised and unsupervised) classifiers, which adopt the most varied account features, from the simplest to the most expensive ones to be extracted from the raw data obtainable through the Twitter public APIs. In this exploratory study, using Twitter as a benchmark, we compare the performances of four state-of-art feature sets in detecting novel bots: one of the output scores of the popular bot detector Botometer, which considers more than 1,000 features of an account to take a decision; two feature sets based on the account profile and timeline; and the information about the Twitter client from which the user tweets. The results of our analysis, conducted on six recently released datasets of Twitter accounts, hint at the possible use of general-purpose classifiers and cheap-to-compute account features for the detection of evolved bots.
zh
[AI-19] Hierarchical Time Series Forecasting Via Latent Mean Encoding
【速读】：该论文试图解决在时间层次预测中，如何在粗粒度和细粒度时间尺度上一致地预测目标变量行为的问题，这一问题在多个商业应用中对利润优化决策至关重要，但目前仍是一个开放的研究难题。解决方案的关键在于提出一种新的分层架构，该架构通过利用专门针对不同时间聚合层级进行预测的模块来处理这一问题，其核心是通过隐藏层学习目标变量的平均行为，从而在目标时间层次上实现准确且一致的预测。
链接: https://arxiv.org/abs/2506.19633
作者: Alessandro Salatiello,Stefan Birr,Manuel Kunz
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Coherently forecasting the behaviour of a target variable across both coarse and fine temporal scales is crucial for profit-optimized decision-making in several business applications, and remains an open research problem in temporal hierarchical forecasting. Here, we propose a new hierarchical architecture that tackles this problem by leveraging modules that specialize in forecasting the different temporal aggregation levels of interest. The architecture, which learns to encode the average behaviour of the target variable within its hidden layers, makes accurate and coherent forecasts across the target temporal hierarchies. We validate our architecture on the challenging, real-world M5 dataset and show that it outperforms established methods, such as the TSMixer model.
zh
[AI-20] Why Uncertainty Calibration Matters for Reliable Perturbation-based Explanations ICLR2025
【速读】：该论文试图解决基于扰动的解释方法在模型不确定性校准不足时导致的解释不可靠问题，即模型在特定扰动下的置信度与其实际准确性不匹配。解决方案的关键在于提出ReCalX方法，通过重新校准模型以提升扰动解释的质量，同时保持模型原有的预测不变。
链接: https://arxiv.org/abs/2506.19630
作者: Thomas Decker,Volker Tresp,Florian Buettner
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: ICLR 2025 Workshop: XAI4Science: From Understanding Model Behavior to Discovering New Scientific Knowledge
Abstract:Perturbation-based explanations are widely utilized to enhance the transparency of modern machine-learning models. However, their reliability is often compromised by the unknown model behavior under the specific perturbations used. This paper investigates the relationship between uncertainty calibration - the alignment of model confidence with actual accuracy - and perturbation-based explanations. We show that models frequently produce unreliable probability estimates when subjected to explainability-specific perturbations and theoretically prove that this directly undermines explanation quality. To address this, we introduce ReCalX, a novel approach to recalibrate models for improved perturbation-based explanations while preserving their original predictions. Experiments on popular computer vision models demonstrate that our calibration strategy produces explanations that are more aligned with human perception and actual object locations.
zh
[AI-21] Position: Intelligent Science Laboratory Requires the Integration of Cognitive and Embodied AI
【速读】：该论文试图解决科学发现过程中因人类在专业知识、身体能力及睡眠周期等方面的限制而产生的瓶颈问题，以及当前AI科学家和自动化实验室在虚拟环境与物理世界适应性之间的不足。解决方案的关键在于提出智能科学实验室（Intelligent Science Laboratories, ISLs）的范式，其核心是通过多层闭环框架深度整合认知智能与具身智能，包括科学推理的基础模型、基于代理的工作流编排以及用于稳健物理实验的具身代理，从而实现自主迭代实验与意外发现的可能性。
链接: https://arxiv.org/abs/2506.19613
作者: Sha Zhang,Suorong Yang,Tong Xie,Xiangyuan Xue,Zixuan Hu,Rui Li,Wenxi Qu,Zhenfei Yin,Tianfan Fu,Di Hu,Andres M Bran,Nian Ran,Bram Hoex,Wangmeng Zuo,Philippe Schwaller,Wanli Ouyang,Lei Bai,Yanyong Zhang,Lingyu Duan,Shixiang Tang,Dongzhan Zhou
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Scientific discovery has long been constrained by human limitations in expertise, physical capability, and sleep cycles. The recent rise of AI scientists and automated laboratories has accelerated both the cognitive and operational aspects of research. However, key limitations persist: AI systems are often confined to virtual environments, while automated laboratories lack the flexibility and autonomy to adaptively test new hypotheses in the physical world. Recent advances in embodied AI, such as generalist robot foundation models, diffusion-based action policies, fine-grained manipulation learning, and sim-to-real transfer, highlight the promise of integrating cognitive and embodied intelligence. This convergence opens the door to closed-loop systems that support iterative, autonomous experimentation and the possibility of serendipitous discovery. In this position paper, we propose the paradigm of Intelligent Science Laboratories (ISLs): a multi-layered, closed-loop framework that deeply integrates cognitive and embodied intelligence. ISLs unify foundation models for scientific reasoning, agent-based workflow orchestration, and embodied agents for robust physical experimentation. We argue that such systems are essential for overcoming the current limitations of scientific discovery and for realizing the full transformative potential of AI-driven science.
zh
[AI-22] ChordPrompt: Orchestrating Cross-Modal Prompt Synergy for Multi-Domain Incremental Learning in CLIP ECML-PKDD2025
【速读】：该论文旨在解决预训练视觉-语言模型在持续学习（Continual Learning, CL）场景下难以维持跨领域性能的问题，特别是在多领域任务增量学习中，现有提示学习方法存在两大局限：一是主要针对类别增量学习，缺乏针对多领域任务增量学习的特定策略；二是多数方法使用单模态提示，忽略了跨模态信息交互的潜在优势。论文提出的\ChordPrompt框架通过引入跨模态提示，促进视觉与文本提示之间的协同作用，并采用领域自适应文本提示以实现多领域中的持续适配，其关键在于构建视觉与文本信息间的有效交互机制，从而提升模型在零样本泛化和下游任务中的性能。
链接: https://arxiv.org/abs/2506.19608
作者: Zhiyuan Wang,Bokui Chen
机构: 未知
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Accept by ECML-PKDD 2025
Abstract:Continual learning (CL) empowers pre-trained vision-language models to adapt effectively to novel or previously underrepresented data distributions without comprehensive retraining, enhancing their adaptability and efficiency. While vision-language models like CLIP show great promise, they struggle to maintain performance across domains in incremental learning scenarios. Existing prompt learning methods face two main limitations: 1) they primarily focus on class-incremental learning scenarios, lacking specific strategies for multi-domain task incremental learning; 2) most current approaches employ single-modal prompts, neglecting the potential benefits of cross-modal information exchange. To address these challenges, we propose the \ChordPrompt framework, which facilitates a harmonious interplay between visual and textual prompts. \ChordPrompt introduces cross-modal prompts to leverage interactions between visual and textual information. Our approach also employs domain-adaptive text prompts to select appropriate prompts for continual adaptation across multiple domains. Comprehensive experiments on multi-domain incremental learning benchmarks demonstrate that \ChordPrompt outperforms state-of-the-art methods in zero-shot generalization and downstream task performance.
zh
[AI-23] Robotics Under Construction: Challenges on Job Sites ICRA
【速读】：该论文试图解决建筑行业面临的劳动力短缺和生产率停滞问题，通过自动化技术推动可持续基础设施发展。其解决方案的关键在于开发一种自主载荷运输系统，该系统基于CD110R-3履带式运载平台，集成了自主导航、车队管理和基于GNSS的定位技术，以实现施工现场材料运输的自动化。尽管当前系统尚未包含动态环境适应算法，但已开展对外部传感器感知与建图系统的初步研究，为未来实现完全无人化施工现场奠定基础。
链接: https://arxiv.org/abs/2506.19597
作者: Haruki Uchiito,Akhilesh Bhat,Koji Kusaka,Xiaoya Zhang,Hiraku Kinjo,Honoka Uehara,Motoki Koyama,Shinji Natsume
机构: 未知
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR); Emerging Technologies (cs.ET); Systems and Control (eess.SY)
备注: Workshop on Field Robotics, ICRA
Abstract:As labor shortages and productivity stagnation increasingly challenge the construction industry, automation has become essential for sustainable infrastructure development. This paper presents an autonomous payload transportation system as an initial step toward fully unmanned construction sites. Our system, based on the CD110R-3 crawler carrier, integrates autonomous navigation, fleet management, and GNSS-based localization to facilitate material transport in construction site environments. While the current system does not yet incorporate dynamic environment adaptation algorithms, we have begun fundamental investigations into external-sensor based perception and mapping system. Preliminary results highlight the potential challenges, including navigation in evolving terrain, environmental perception under construction-specific conditions, and sensor placement optimization for improving autonomy and efficiency. Looking forward, we envision a construction ecosystem where collaborative autonomous agents dynamically adapt to site conditions, optimizing workflow and reducing human intervention. This paper provides foundational insights into the future of robotics-driven construction automation and identifies critical areas for further technological development.
zh
[AI-24] Adaptive Domain Modeling with Language Models: A Multi-Agent Approach to Task Planning
【速读】：该论文试图解决复杂任务求解中依赖手动定义环境模型的问题，从而限制了系统的灵活性和适应性。解决方案的关键在于提出TAPAS（Task-based Adaptation and Planning using AgentS）框架，该框架通过集成大型语言模型（Large Language Models, LLMs）与符号规划，使多个代理协作生成和调整领域模型、初始状态及目标规范。此过程借助结构化工具调用机制实现，使得下游代理能够从上游代理请求修改，从而在无需人工重新定义领域的情况下适应新的属性和约束。
链接: https://arxiv.org/abs/2506.19592
作者: Harisankar Babu,Philipp Schillinger,Tamim Asfour
机构: 未知
类目: Artificial Intelligence (cs.AI); Robotics (cs.RO)
备注:
Abstract:We introduce TAPAS (Task-based Adaptation and Planning using AgentS), a multi-agent framework that integrates Large Language Models (LLMs) with symbolic planning to solve complex tasks without the need for manually defined environment models. TAPAS employs specialized LLM-based agents that collaboratively generate and adapt domain models, initial states, and goal specifications as needed using structured tool-calling mechanisms. Through this tool-based interaction, downstream agents can request modifications from upstream agents, enabling adaptation to novel attributes and constraints without manual domain redefinition. A ReAct (Reason+Act)-style execution agent, coupled with natural language plan translation, bridges the gap between dynamically generated plans and real-world robot capabilities. TAPAS demonstrates strong performance in benchmark planning domains and in the VirtualHome simulated real-world environment.
zh
[AI-25] owards an Introspective Dynamic Model of Globally Distributed Computing Infrastructures
【速读】：该论文旨在解决大规模科学合作中计算资源分配与数据管理效率不足的问题，特别是在数据放置和负载分配方面，现有方法多依赖于启发式策略，缺乏有效的动态模型来评估和优化不同方案。其解决方案的关键在于构建一个基于真实世界数据的交互式系统，通过分析PanDA工作流管理系统中的作业执行记录，提取关键性能指标，并利用生成式 AI (Generative AI) 模型模拟负载的时间序列，以捕捉可见和隐藏特征，从而为更高效的资源调度提供支持。
链接: https://arxiv.org/abs/2506.19578
作者: Ozgur O. Kilic,David K. Park,Yihui Ren,Tatiana Korchuganova,Sairam Sri Vatsavai,Joseph Boudreau,Tasnuva Chowdhury,Shengyu Feng,Raees Khan,Jaehyung Kim,Scott Klasky,Tadashi Maeno,Paul Nilsson,Verena Ingrid Martinez Outschoorn,Norbert Podhorszki,Frédéric Suter,Wei Yang,Yiming Yang,Shinjae Yoo,Alexei Klimentov,Adolfy Hoisie
机构: 未知
类目: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)
备注:
Abstract:Large-scale scientific collaborations like ATLAS, Belle II, CMS, DUNE, and others involve hundreds of research institutes and thousands of researchers spread across the globe. These experiments generate petabytes of data, with volumes soon expected to reach exabytes. Consequently, there is a growing need for computation, including structured data processing from raw data to consumer-ready derived data, extensive Monte Carlo simulation campaigns, and a wide range of end-user analysis. To manage these computational and storage demands, centralized workflow and data management systems are implemented. However, decisions regarding data placement and payload allocation are often made disjointly and via heuristic means. A significant obstacle in adopting more effective heuristic or AI-driven solutions is the absence of a quick and reliable introspective dynamic model to evaluate and refine alternative approaches. In this study, we aim to develop such an interactive system using real-world data. By examining job execution records from the PanDA workflow management system, we have pinpointed key performance indicators such as queuing time, error rate, and the extent of remote data access. The dataset includes five months of activity. Additionally, we are creating a generative AI model to simulate time series of payloads, which incorporate visible features like category, event count, and submitting group, as well as hidden features like the total computational load-derived from existing PanDA records and computing site capabilities. These hidden features, which are not visible to job allocators, whether heuristic or AI-driven, influence factors such as queuing times and data movement.
zh
[AI-26] Interpretable Hybrid Machine Learning Models Using FOLD-R and Answer Set Programming
【速读】：该论文试图解决高风险领域（如医疗）中机器学习（Machine Learning, ML）模型的可解释性与预测性能之间的矛盾问题。解决方案的关键在于提出一种混合方法，将基于FOLD-R++算法生成的Answer Set Programming (ASP)规则与黑盒ML分类器相结合，以选择性地修正不确定的预测并提供人类可读的解释。
链接: https://arxiv.org/abs/2506.19573
作者: Sanne Wielinga,Jesse Heyninck
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注: accepted for publication as a Technical Communication at ICLP 2025
Abstract:Machine learning (ML) techniques play a pivotal role in high-stakes domains such as healthcare, where accurate predictions can greatly enhance decision-making. However, most high-performing methods such as neural networks and ensemble methods are often opaque, limiting trust and broader adoption. In parallel, symbolic methods like Answer Set Programming (ASP) offer the possibility of interpretable logical rules but do not always match the predictive power of ML models. This paper proposes a hybrid approach that integrates ASP-derived rules from the FOLD-R++ algorithm with black-box ML classifiers to selectively correct uncertain predictions and provide human-readable explanations. Experiments on five medical datasets reveal statistically significant performance gains in accuracy and F1 score. This study underscores the potential of combining symbolic reasoning with conventional ML to achieve high interpretability without sacrificing accuracy.
zh
[AI-27] FAF: A Feature-Adaptive Framework for Few-Shot Time Series Forecasting
【速读】：该论文旨在解决多任务和小样本时间序列预测中存在的历史数据不足问题，该问题源于传统方法未能充分考虑不同任务间的泛化特征与特定特征。其解决方案的关键在于提出一种特征自适应的时间序列预测框架（Feature-Adaptive Time Series Forecasting Framework, FAF），该框架包含三个核心组件：泛化知识模块（Generalized Knowledge Module, GKM）、任务特定模块（Task-Specific Module, TSM）和排序模块（Rank Module, RM）。GKM通过元学习机制提取跨相关任务的泛化特征，TSM则通过多个功能区域捕捉任务特定的局部动态，RM在测试阶段根据输入序列特征动态选择最相关的功能区域，并与GKM的泛化知识结合以生成准确预测。
链接: https://arxiv.org/abs/2506.19567
作者: Pengpeng Ouyang,Dong Chen,Tong Yang,Shuo Feng,Zhao Jin,Mingliang Xu
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: 12 pages,4 figures, 8 tables
Abstract:Multi-task and few-shot time series forecasting tasks are commonly encountered in scenarios such as the launch of new products in different cities. However, traditional time series forecasting methods suffer from insufficient historical data, which stems from a disregard for the generalized and specific features among different tasks. For the aforementioned challenges, we propose the Feature-Adaptive Time Series Forecasting Framework (FAF), which consists of three key components: the Generalized Knowledge Module (GKM), the Task-Specific Module (TSM), and the Rank Module (RM). During training phase, the GKM is updated through a meta-learning mechanism that enables the model to extract generalized features across related tasks. Meanwhile, the TSM is trained to capture diverse local dynamics through multiple functional regions, each of which learns specific features from individual tasks. During testing phase, the RM dynamically selects the most relevant functional region from the TSM based on input sequence features, which is then combined with the generalized knowledge learned by the GKM to generate accurate forecasts. This design enables FAF to achieve robust and personalized forecasting even with sparse historical observations We evaluate FAF on five diverse real-world datasets under few-shot time series forecasting settings. Experimental results demonstrate that FAF consistently outperforms baselines that include three categories of time series forecasting methods. In particular, FAF achieves a 41.81% improvement over the best baseline, iTransformer, on the CO _2 emissions dataset.
zh
[AI-28] PrivacyXray: Detecting Privacy Breaches in LLM s through Semantic Consistency and Probability Certainty
【速读】：该论文试图解决在大型语言模型（Large Language Models, LLMs）推理过程中潜在的隐私信息泄露问题，特别是现有隐私提取攻击无法验证所提取信息准确性的问题。解决方案的关键在于提出PrivacyXray框架，通过分析LLMs内部状态来检测隐私泄露，其核心依据是当模型生成正确隐私输出时，会表现出更高的语义连贯性和概率确定性。PrivacyXray利用四种指标——层内与层间语义相似性、令牌级与句子级概率分布——实现对隐私泄露的检测，从而克服了缺乏公开私有数据集的挑战，并无需依赖外部数据进行验证。
链接: https://arxiv.org/abs/2506.19563
作者: Jinwen He,Yiyang Lu,Zijin Lin,Kai Chen,Yue Zhao
机构: 未知
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注:
Abstract:Large Language Models (LLMs) are widely used in sensitive domains, including healthcare, finance, and legal services, raising concerns about potential private information leaks during inference. Privacy extraction attacks, such as jailbreaking, expose vulnerabilities in LLMs by crafting inputs that force the models to output sensitive information. However, these attacks cannot verify whether the extracted private information is accurate, as no public datasets exist for cross-validation, leaving a critical gap in private information detection during inference. To address this, we propose PrivacyXray, a novel framework detecting privacy breaches by analyzing LLM inner states. Our analysis reveals that LLMs exhibit higher semantic coherence and probabilistic certainty when generating correct private outputs. Based on this, PrivacyXray detects privacy breaches using four metrics: intra-layer and inter-layer semantic similarity, token-level and sentence-level probability distributions. PrivacyXray addresses critical challenges in private information detection by overcoming the lack of open-source private datasets and eliminating reliance on external data for validation. It achieves this through the synthesis of realistic private data and a detection mechanism based on the inner states of LLMs. Experiments show that PrivacyXray achieves consistent performance, with an average accuracy of 92.69% across five LLMs. Compared to state-of-the-art methods, PrivacyXray achieves significant improvements, with an average accuracy increase of 20.06%, highlighting its stability and practical utility in real-world applications.
zh
[AI-29] Lost in Translation? Converting RegExes for Log Parsing into Dynatrace Pattern Language
【速读】：该论文旨在解决企业将传统基于正则表达式（Regular Expressions, RegExes）的日志解析规则迁移至现代日志分析平台时所面临的手动转换成本高、易出错的问题。其解决方案的关键在于提出Reptile工具，该工具结合了基于规则的方法用于将RegExes转换为Dynatrace Pattern Language (DPL)模式，并在无法完全转换的情况下采用最佳努力策略，同时集成GPT-4对生成的DPL模式进行优化，从而提高转换效率与准确性。
链接: https://arxiv.org/abs/2506.19539
作者: Julian Fragner,Christian Macho,Bernhard Dieber,Martin Pinzger
机构: 未知
类目: oftware Engineering (cs.SE); Artificial Intelligence (cs.AI)
备注: 18 pages, 7 tables, 18 figures
Abstract:Log files provide valuable information for detecting and diagnosing problems in enterprise software applications and data centers. Several log analytics tools and platforms were developed to help filter and extract information from logs, typically using regular expressions (RegExes). Recent commercial log analytics platforms provide domain-specific languages specifically designed for log parsing, such as Grok or the Dynatrace Pattern Language (DPL). However, users who want to migrate to these platforms must manually convert their RegExes into the new pattern language, which is costly and error-prone. In this work, we present Reptile, which combines a rule-based approach for converting RegExes into DPL patterns with a best-effort approach for cases where a full conversion is impossible. Furthermore, it integrates GPT-4 to optimize the obtained DPL patterns. The evaluation with 946 RegExes collected from a large company shows that Reptile safely converted 73.7% of them. The evaluation of Reptile’s pattern optimization with 23 real-world RegExes showed an F1-score and MCC above 0.91. These results are promising and have ample practical implications for companies that migrate to a modern log analytics platform, such as Dynatrace.
zh
[AI-30] NTRL: Encounter Generation via Reinforcement Learning for Dynamic Difficulty Adjustment in Dungeons and Drag ons
【速读】：该论文试图解决《龙与地下城》（Dungeons and Dragons，DD）中战斗遭遇平衡的问题，该问题需要游戏主持人（Dungeon Master，DM）手动评估队伍实力、敌人组成和动态玩家互动，同时避免打断叙事流程。解决方案的关键是提出一种基于强化学习的遭遇生成方法（NTRL），通过将问题建模为上下文关联的老虎机问题，根据实时队伍成员属性生成战斗遭遇，从而实现动态难度调整（Dynamic Difficulty Adjustment，DDA）。NTRL通过迭代优化战斗遭遇，显著提升了战斗持续时间、敌人对队伍造成的伤害，并在保持低总队伍死亡率（TPK）的前提下增加了玩家死亡数量，从而增强了战斗的战略深度和挑战性。
链接: https://arxiv.org/abs/2506.19530
作者: Carlo Romeo,Andrew D. Bagdanov
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Balancing combat encounters in Dungeons Dragons (DD) is a complex task that requires Dungeon Masters (DM) to manually assess party strength, enemy composition, and dynamic player interactions while avoiding interruption of the narrative flow. In this paper, we propose Encounter Generation via Reinforcement Learning (NTRL), a novel approach that automates Dynamic Difficulty Adjustment (DDA) in DD via combat encounter design. By framing the problem as a contextual bandit, NTRL generates encounters based on real-time party members attributes. In comparison with classic DM heuristics, NTRL iteratively optimizes encounters to extend combat longevity (+200%), increases damage dealt to party members, reducing post-combat hit points (-16.67%), and raises the number of player deaths while maintaining low total party kills (TPK). The intensification of combat forces players to act wisely and engage in tactical maneuvers, even though the generated encounters guarantee high win rates (70%). Even in comparison with encounters designed by human Dungeon Masters, NTRL demonstrates superior performance by enhancing the strategic depth of combat while increasing difficulty in a manner that preserves overall game fairness.
zh
[AI-31] MATE: LLM -Powered Multi-Agent Translation Environment for Accessibility Applications
【速读】：该论文试图解决当前技术在无障碍支持方面的不足，特别是现有多智能体系统（MAS）由于封闭源代码设计缺乏定制化，无法为有需求的用户提供全面辅助的问题。解决方案的关键在于提出MATE，一个基于用户需求进行模态转换的多模态无障碍MAS，通过将数据转换为可理解的格式来帮助残疾人更好地与数字环境互动，同时支持多种模型类型和本地运行以保障隐私与安全。此外，引入ModCon-Task-Identifier模型，能够准确提取用户的模态转换任务，提升了系统的适应性和效率。
链接: https://arxiv.org/abs/2506.19502
作者: Aleksandr Algazinov,Matt Laing,Paul Laban
机构: 未知
类目: Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Accessibility remains a critical concern in today’s society, as many technologies are not developed to support the full range of user needs. Existing multi-agent systems (MAS) often cannot provide comprehensive assistance for users in need due to the lack of customization stemming from closed-source designs. Consequently, individuals with disabilities frequently encounter significant barriers when attempting to interact with digital environments. We introduce MATE, a multimodal accessibility MAS, which performs the modality conversions based on the user’s needs. The system is useful for assisting people with disabilities by ensuring that data will be converted to an understandable format. For instance, if the user cannot see well and receives an image, the system converts this image to its audio description. MATE can be applied to a wide range of domains, industries, and areas, such as healthcare, and can become a useful assistant for various groups of users. The system supports multiple types of models, ranging from LLM API calling to using custom machine learning (ML) classifiers. This flexibility ensures that the system can be adapted to various needs and is compatible with a wide variety of hardware. Since the system is expected to run locally, it ensures the privacy and security of sensitive information. In addition, the framework can be effectively integrated with institutional technologies (e.g., digital healthcare service) for real-time user assistance. Furthermore, we introduce ModCon-Task-Identifier, a model that is capable of extracting the precise modality conversion task from the user input. Numerous experiments show that ModCon-Task-Identifier consistently outperforms other LLMs and statistical models on our custom data. Our code and data are publicly available at this https URL.
zh
[AI-32] Recalling The Forgotten Class Memberships: Unlearned Models Can Be Noisy Labelers to Leak Privacy IJCAI2025
【速读】：该论文试图解决机器遗忘（Machine Unlearning, MU）技术中未被充分探索的漏洞问题，特别是如何在不访问原始模型的情况下恢复被遗忘的数据实例的类别归属信息。解决方案的关键在于提出一种基于教师-学生知识蒸馏架构的Membership Recall Attack (MRA)框架，利用已遗忘模型（ULMs）作为噪声标签器，将问题转化为带有噪声标签的学习（Learning with Noisy Labels, LNL）问题，从而推断出被遗忘实例的正确标签。
链接: https://arxiv.org/abs/2506.19486
作者: Zhihao Sui,Liang Hu,Jian Cao,Dora D. Liu,Usman Naseem,Zhongyuan Lai,Qi Zhang
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: IJCAI 2025
Abstract:Machine Unlearning (MU) technology facilitates the removal of the influence of specific data instances from trained models on request. Despite rapid advancements in MU technology, its vulnerabilities are still underexplored, posing potential risks of privacy breaches through leaks of ostensibly unlearned information. Current limited research on MU attacks requires access to original models containing privacy data, which violates the critical privacy-preserving objective of MU. To address this gap, we initiate an innovative study on recalling the forgotten class memberships from unlearned models (ULMs) without requiring access to the original one. Specifically, we implement a Membership Recall Attack (MRA) framework with a teacher-student knowledge distillation architecture, where ULMs serve as noisy labelers to transfer knowledge to student models. Then, it is translated into a Learning with Noisy Labels (LNL) problem for inferring the correct labels of the forgetting instances. Extensive experiments on state-of-the-art MU methods with multiple real datasets demonstrate that the proposed MRA strategy exhibits high efficacy in recovering class memberships of unlearned instances. As a result, our study and evaluation have established a benchmark for future research on MU vulnerabilities.
zh
[AI-33] Fast and Distributed Equivariant Graph Neural Networks by Virtual Node Learning
【速读】：该论文旨在解决传统等变图神经网络（Equivariant Graph Neural Networks, GNNs）在处理大规模几何图时面临的效率瓶颈和稀疏化后性能下降的问题。其关键解决方案是提出两种新型增强方法：FastEGNN 和 DistEGNN。FastEGNN 通过引入一组小规模的虚拟节点来近似实际的无序大图，采用差异化的消息传递与聚合机制，并最小化虚拟节点与真实节点坐标之间的最大均值差异（Maximum Mean Discrepancy, MMD），从而在保持高精度的同时高效处理大规模稀疏图；而 DistEGNN 则进一步扩展为分布式版本，利用虚拟节点作为不同设备间子图的全局桥梁，以维持一致性并显著降低内存和计算开销。
链接: https://arxiv.org/abs/2506.19482
作者: Yuelin Zhang,Jiacheng Cen,Jiaqi Han,Wenbing Huang
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Equivariant Graph Neural Networks (GNNs) have achieved remarkable success across diverse scientific applications. However, existing approaches face critical efficiency challenges when scaling to large geometric graphs and suffer significant performance degradation when the input graphs are sparsified for computational tractability. To address these limitations, we introduce FastEGNN and DistEGNN, two novel enhancements to equivariant GNNs for large-scale geometric graphs. FastEGNN employs a key innovation: a small ordered set of virtual nodes that effectively approximates the large unordered graph of real nodes. Specifically, we implement distinct message passing and aggregation mechanisms for different virtual nodes to ensure mutual distinctiveness, and minimize Maximum Mean Discrepancy (MMD) between virtual and real coordinates to achieve global distributedness. This design enables FastEGNN to maintain high accuracy while efficiently processing large-scale sparse graphs. For extremely large-scale geometric graphs, we present DistEGNN, a distributed extension where virtual nodes act as global bridges between subgraphs in different devices, maintaining consistency while dramatically reducing memory and computational overhead. We comprehensively evaluate our models across four challenging domains: N-body systems (100 nodes), protein dynamics (800 nodes), Water-3D (8,000 nodes), and our new Fluid113K benchmark (113,000 nodes). Results demonstrate superior efficiency and performance, establishing new capabilities in large-scale equivariant graph learning. Code is available at this https URL.
zh
[AI-34] KunLunBaizeRAG : Reinforcement Learning Driven Inference Performance Leap for Large Language Models
【速读】：该论文试图解决传统检索增强生成（RAG）在复杂多跳问答任务中存在检索偏差、信息冗余和策略僵化等问题。其解决方案的关键在于提出了一系列创新机制，包括基于RAG的推理对齐（RAG-driven Reasoning Alignment, RDRA）机制、搜索-思考迭代增强（Search-Think Iterative Enhancement, STIE）机制、网络局部智能路由（Network-Local Intelligent Routing, NLR）机制，以及一种渐进式混合训练策略，从而有效提升了大语言模型（LLM）的推理能力。
链接: https://arxiv.org/abs/2506.19466
作者: Cheng Li,Jiexiong Liu,Yixuan Chen,Qihang Zhou,KunLun Meta
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:This paper introduces KunLunBaizeRAG, a reinforcement learning-driven reasoning framework designed to enhance the reasoning capabilities of large language models (LLMs) in complex multi-hop question-answering tasks. The framework addresses key limitations of traditional RAG, such as retrieval drift, information redundancy, and strategy rigidity. Key innovations include the RAG-driven Reasoning Alignment (RDRA) mechanism, the Search-Think Iterative Enhancement (STIE) mechanism, the Network-Local Intelligent Routing (NLR) mechanism, and a progressive hybrid training strategy. Experimental results demonstrate significant improvements in exact match (EM) and LLM-judged score (LJ) across four benchmarks, highlighting the framework’s robustness and effectiveness in complex reasoning scenarios.
zh
[AI-35] agged for Direction: Pinning Down Causal Edge Directions with Precision
【速读】：该论文试图解决因果发现中变量类型分配不准确或不灵活导致的因果方向推断问题。其解决方案的关键在于引入基于标签（tag）的因果发现方法，通过为每个变量分配多个标签，并利用已确定的边关系来推导标签间的边关系，从而指导无向边的方向确定，相较于传统的单一类型假设，这种方法提升了因果发现的鲁棒性和灵活性。
链接: https://arxiv.org/abs/2506.19459
作者: Florian Peter Busch,Moritz Willig,Florian Guldan,Kristian Kersting,Devendra Singh Dhami
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Not every causal relation between variables is equal, and this can be leveraged for the task of causal discovery. Recent research shows that pairs of variables with particular type assignments induce a preference on the causal direction of other pairs of variables with the same type. Although useful, this assignment of a specific type to a variable can be tricky in practice. We propose a tag-based causal discovery approach where multiple tags are assigned to each variable in a causal graph. Existing causal discovery approaches are first applied to direct some edges, which are then used to determine edge relations between tags. Then, these edge relations are used to direct the undirected edges. Doing so improves upon purely type-based relations, where the assumption of type consistency lacks robustness and flexibility due to being restricted to single types for each variable. Our experimental evaluations show that this boosts causal discovery and that these high-level tag relations fit common knowledge.
zh
[AI-36] Commander-GPT : Dividing and Routing for Multimodal Sarcasm Detection
【速读】：该论文旨在解决多模态讽刺理解（multimodal sarcasm understanding）这一高阶认知任务中，大型语言模型（LLM）表现不足的问题。其解决方案的关键在于提出Commander-GPT框架，该框架受军事指挥理论启发，通过协调一组专业化的LLM代理来执行特定子任务（如上下文建模、情感分析等），并将结果汇总至指挥官进行最终的讽刺判断，从而提升整体性能。
链接: https://arxiv.org/abs/2506.19420
作者: Yazhou Zhang,Chunwang Zou,Bo Wang,Jing Qin
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Multimodal sarcasm understanding is a high-order cognitive task. Although large language models (LLMs) have shown impressive performance on many downstream NLP tasks, growing evidence suggests that they struggle with sarcasm understanding. In this paper, we propose Commander-GPT, a modular decision routing framework inspired by military command theory. Rather than relying on a single LLM’s capability, Commander-GPT orchestrates a team of specialized LLM agents where each agent will be selectively assigned to a focused sub-task such as context modeling, sentiment analysis, etc. Their outputs are then routed back to the commander, which integrates the information and performs the final sarcasm judgment. To coordinate these agents, we introduce three types of centralized commanders: (1) a trained lightweight encoder-based commander (e.g., multi-modal BERT); (2) four small autoregressive language models, serving as moderately capable commanders (e.g., DeepSeek-VL); (3) two large LLM-based commander (Gemini Pro and GPT-4o) that performs task routing, output aggregation, and sarcasm decision-making in a zero-shot fashion. We evaluate Commander-GPT on the MMSD and MMSD 2.0 benchmarks, comparing five prompting strategies. Experimental results show that our framework achieves 4.4% and 11.7% improvement in F1 score over state-of-the-art (SoTA) baselines on average, demonstrating its effectiveness.
zh
[AI-37] Unsupervised Dataset Dictionary Learning for domain shift robust clustering: application to sitting posture identification
【速读】：该论文旨在解决无监督环境下坐姿识别中的鲁棒聚类问题，特别是传统方法在面对不同数据集时适应性差以及存在领域偏移（domain shift）的问题。其解决方案的关键在于提出一种无监督数据集字典学习（Unsupervised Dataset Dictionary Learning, U-DaDiL）方法，通过基于Wasserstein barycenter的表示对不同数据集的分布进行对齐，从而提升聚类的准确性与鲁棒性。
链接: https://arxiv.org/abs/2506.19410
作者: Anas Hattay,Mayara Ayat,Fred Ngole Mboula
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:This paper introduces a novel approach, Unsupervised Dataset Dictionary Learning (U-DaDiL), for totally unsupervised robust clustering applied to sitting posture identification. Traditional methods often lack adaptability to diverse datasets and suffer from domain shift issues. U-DaDiL addresses these challenges by aligning distributions from different datasets using Wasserstein barycenter based representation. Experimental evaluations on the Office31 dataset demonstrate significant improvements in cluster alignment accuracy. This work also presents a promising step for addressing domain shift and robust clustering for unsupervised sitting posture identification
zh
[AI-38] Is an object-centric representation beneficial for robotic manipulation ?
【速读】：该论文试图解决当前基于对象中心表示（object-centric representation, OCR）的方法在复杂场景下的泛化能力不足的问题，特别是在涉及多物体交互的机器人操作任务中，现有方法缺乏对学习到的表示进行推理的能力。解决方案的关键在于通过构建高随机化的模拟机器人操作任务来评估OCR方法的有效性，并与最先进的整体表示方法进行对比，从而验证对象中心方法在处理复杂场景结构时的优势。
链接: https://arxiv.org/abs/2506.19408
作者: Alexandre Chapin(imagine),Emmanuel Dellandrea(imagine),Liming Chen(imagine)
机构: 未知
类目: Artificial Intelligence (cs.AI); Robotics (cs.RO)
备注:
Abstract:Object-centric representation (OCR) has recently become a subject of interest in the computer vision community for learning a structured representation of images and videos. It has been several times presented as a potential way to improve data-efficiency and generalization capabilities to learn an agent on downstream tasks. However, most existing work only evaluates such models on scene decomposition, without any notion of reasoning over the learned representation. Robotic manipulation tasks generally involve multi-object environments with potential inter-object interaction. We thus argue that they are a very interesting playground to really evaluate the potential of existing object-centric work. To do so, we create several robotic manipulation tasks in simulated environments involving multiple objects (several distractors, the robot, etc.) and a high-level of randomization (object positions, colors, shapes, background, initial positions, etc.). We then evaluate one classical object-centric method across several generalization scenarios and compare its results against several state-of-the-art hollistic representations. Our results exhibit that existing methods are prone to failure in difficult scenarios involving complex scene structures, whereas object-centric methods help overcome these challenges.
zh
[AI-39] Conversational Intent-Driven GraphRAG : Enhancing Multi-Turn Dialogue Systems through Adaptive Dual-Retrieval of Flow Patterns and Context Semantics
【速读】：该论文旨在解决现有对话系统在多轮客户服务对话中难以同时保持上下文连贯性和目标导向进展的问题。其解决方案的关键在于提出CID-GraphRAG框架，该框架通过从历史对话中构建动态意图转移图，并实现一种双检索机制，自适应地平衡基于意图的图遍历与语义搜索，从而同时利用对话意图流模式和上下文语义，显著提升检索和响应质量。
链接: https://arxiv.org/abs/2506.19385
作者: Ziqi Zhu,Tao Hu,Honglong Zhang,Dan Yang,HanGeng Chen,Mengran Zhang,Xilun Chen
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:We present CID-GraphRAG (Conversational Intent-Driven Graph Retrieval Augmented Generation), a novel framework that addresses the limitations of existing dialogue systems in maintaining both contextual coherence and goal-oriented progression in multi-turn customer service conversations. Unlike traditional RAG systems that rely solely on semantic similarity (Conversation RAG) or standard knowledge graphs (GraphRAG), CID-GraphRAG constructs dynamic intent transition graphs from goal achieved historical dialogues and implements a dual-retrieval mechanism that adaptively balances intent-based graph traversal with semantic search. This approach enables the system to simultaneously leverage both conversional intent flow patterns and contextual semantics, significantly improving retrieval quality and response quality. In extensive experiments on real-world customer service dialogues, we employ both automatic metrics and LLM-as-judge assessments, demonstrating that CID-GraphRAG significantly outperforms both semantic-based Conversation RAG and intent-based GraphRAG baselines across all evaluation criteria. Quantitatively, CID-GraphRAG demonstrates substantial improvements over Conversation RAG across automatic metrics, with relative gains of 11% in BLEU, 5% in ROUGE-L, 6% in METEOR, and most notably, a 58% improvement in response quality according to LLM-as-judge evaluations. These results demonstrate that the integration of intent transition structures with semantic retrieval creates a synergistic effect that neither approach achieves independently, establishing CID-GraphRAG as an effective framework for addressing the challenges of maintaining contextual coherence and goal-oriented progression in knowledge-intensive multi-turn dialogues.
zh
[AI-40] Evolutionary Level Repair
【速读】：该论文试图解决游戏关卡修复（game level repair）问题，即对设计但无法正常运行的游戏关卡进行修改以使其功能化，这可能包括确保关卡的完整性、物体的可到达性或其他性能特征。解决方案的关键在于采用基于搜索的方法，特别是进化算法和质量多样性算法，以在仅允许少量修改的情况下实现有效的关卡修复。该方法与基于机器学习的程序化内容生成（PCGML）结合，展示了作为混合程序化内容生成（hybrid PCG）方法的巨大潜力。
链接: https://arxiv.org/abs/2506.19359
作者: Debosmita Bhaumik,Julian Togelius,Georgios N. Yannakakis,Ahmed Khalifa
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:We address the problem of game level repair, which consists of taking a designed but non-functional game level and making it functional. This might consist of ensuring the completeness of the level, reachability of objects, or other performance characteristics. The repair problem may also be constrained in that it can only make a small number of changes to the level. We investigate search-based solutions to the level repair problem, particularly using evolutionary and quality-diversity algorithms, with good results. This level repair method is applied to levels generated using a machine learning-based procedural content generation (PCGML) method that generates stylistically appropriate but frequently broken levels. This combination of PCGML for generation and search-based methods for repair shows great promise as a hybrid procedural content generation (PCG) method.
zh
[AI-41] Discrepancy-Aware Graph Mask Auto-Encoder
【速读】：该论文试图解决现有图自监督学习方法在异质性图（heterophilic graphs）中表现不佳的问题，这是因为现有方法仅关注捕获邻域信息而忽略了节点间的差异信息，导致节点表示难以区分。解决方案的关键在于提出一种差异感知的图掩码自编码器（Discrepancy-Aware Graph Mask Auto-Encoder, DGMAE），通过在掩码过程中重构邻近节点的差异信息，从而获得更具区分性的节点表示。
链接: https://arxiv.org/abs/2506.19343
作者: Ziyu Zheng,Yaming Yang,Ziyu Guan,Wei Zhao,Weigang Lu
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Masked Graph Auto-Encoder, a powerful graph self-supervised training paradigm, has recently shown superior performance in graph representation learning. Existing works typically rely on node contextual information to recover the masked information. However, they fail to generalize well to heterophilic graphs where connected nodes may be not similar, because they focus only on capturing the neighborhood information and ignoring the discrepancy information between different nodes, resulting in indistinguishable node representations. In this paper, to address this issue, we propose a Discrepancy-Aware Graph Mask Auto-Encoder (DGMAE). It obtains more distinguishable node representations by reconstructing the discrepancy information of neighboring nodes during the masking process. We conduct extensive experiments on 17 widely-used benchmark datasets. The results show that our DGMAE can effectively preserve the discrepancies of nodes in low-dimensional space. Moreover, DGMAE significantly outperforms state-of-the-art graph self-supervised learning methods on three graph analytic including tasks node classification, node clustering, and graph classification, demonstrating its remarkable superiority. The code of DGMAE is available at this https URL.
zh
[AI-42] Unlocking Insights Addressing Alcohol Inference Mismatch through Database-Narrative Alignment
【速读】：该论文试图解决酒精推断不匹配（Alcohol Inference Mismatch, AIM）问题，即在交通事故数据中由于信息不准确或缺失导致的酒精相关因素推断错误。解决方案的关键在于利用数据库叙述对齐技术，结合BERT模型对大量事故记录进行分析，以识别AIM事件并提升事故管理系统的数据质量，从而减少AIM事故的比例。
链接: https://arxiv.org/abs/2506.19342
作者: Sudesh Bhagat,Raghupathi Kandiboina,Ibne Farabi Shihab,Skylar Knickerbocker,Neal Hawkins,Anuj Sharma
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
备注:
Abstract:Road traffic crashes are a significant global cause of fatalities, emphasizing the urgent need for accurate crash data to enhance prevention strategies and inform policy development. This study addresses the challenge of alcohol inference mismatch (AIM) by employing database narrative alignment to identify AIM in crash data. A framework was developed to improve data quality in crash management systems and reduce the percentage of AIM crashes. Utilizing the BERT model, the analysis of 371,062 crash records from Iowa (2016-2022) revealed 2,767 AIM incidents, resulting in an overall AIM percentage of 24.03%. Statistical tools, including the Probit Logit model, were used to explore the crash characteristics affecting AIM patterns. The findings indicate that alcohol-related fatal crashes and nighttime incidents have a lower percentage of the mismatch, while crashes involving unknown vehicle types and older drivers are more susceptible to mismatch. The geospatial cluster as part of this study can identify the regions which have an increased need for education and training. These insights highlight the necessity for targeted training programs and data management teams to improve the accuracy of crash reporting and support evidence-based policymaking.
zh
[AI-43] FEAT: A Preference Feedback Dataset through a Cost-Effective Auto-Generation and Labeling Framework for English AI Tutoring ACL2025
【速读】：该论文旨在解决英语教育辅导中高质量教师反馈数据生成成本高、耗时的问题，这一问题限制了基于AI的辅导系统的发展。其解决方案的关键在于提出FEAT框架，通过构建三种互补的数据集（DIRECT-Manual、DIRECT-Generated和DIRECT-Augmented），在保证成本效益的同时提升生成反馈的质量，特别是通过将少量高质量的人工生成数据（DM）与大规模自动生成数据（DG）结合，实现了性能的显著提升。
链接: https://arxiv.org/abs/2506.19325
作者: Hyein Seo,Taewook Hwang,Yohan Lee,sangkeun Jung
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注: ACL 2025 (Short)
Abstract:In English education tutoring, teacher feedback is essential for guiding students. Recently, AI-based tutoring systems have emerged to assist teachers; however, these systems require high-quality and large-scale teacher feedback data, which is both time-consuming and costly to generate manually. In this study, we propose FEAT, a cost-effective framework for generating teacher feedback, and have constructed three complementary datasets: (1) DIRECT-Manual (DM), where both humans and large language models (LLMs) collaboratively generate high-quality teacher feedback, albeit at a higher cost; (2) DIRECT-Generated (DG), an LLM-only generated, cost-effective dataset with lower quality;, and (3) DIRECT-Augmented (DA), primarily based on DG with a small portion of DM added to enhance quality while maintaining cost-efficiency. Experimental results showed that incorporating a small portion of DM (5-10%) into DG leads to superior performance compared to using 100% DM alone.
zh
[AI-44] Emotion Detection on User Front-Facing App Interfaces for Enhanced Schedule Optimization: A Machine Learning Approach
【速读】：该论文旨在解决如何在日历应用中集成情绪检测技术，以实现根据用户情绪状态和压力水平动态调整用户界面，从而提升用户体验与参与度的问题。其解决方案的关键在于提出并评估两种互补的情绪检测方法：一种是基于生物特征的方法，利用从心电图（ECG）信号中提取的心率（HR）数据，通过长短期记忆网络（LSTM）和门控循环单元（GRU）神经网络预测情绪维度；另一种是基于行为的方法，通过分析计算机活动中的用户交互行为（如鼠标移动、点击和击键模式）进行情绪分类。研究结果表明，基于计算机活动的方法在准确性和一致性方面表现更优，尤其在鼠标相关交互上达到了约90%的准确率，同时GRU在网络性能上优于LSTM。
链接: https://arxiv.org/abs/2506.19280
作者: Feiting Yang,Antoine Moevus,Steve Lévesque
机构: 未知
类目: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
备注:
Abstract:Human-Computer Interaction (HCI) has evolved significantly to incorporate emotion recognition capabilities, creating unprecedented opportunities for adaptive and personalized user experiences. This paper explores the integration of emotion detection into calendar applications, enabling user interfaces to dynamically respond to users’ emotional states and stress levels, thereby enhancing both productivity and engagement. We present and evaluate two complementary approaches to emotion detection: a biometric-based method utilizing heart rate (HR) data extracted from electrocardiogram (ECG) signals processed through Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural networks to predict the emotional dimensions of Valence, Arousal, and Dominance; and a behavioral method analyzing computer activity through multiple machine learning models to classify emotions based on fine-grained user interactions such as mouse movements, clicks, and keystroke patterns. Our comparative analysis, from real-world datasets, reveals that while both approaches demonstrate effectiveness, the computer activity-based method delivers superior consistency and accuracy, particularly for mouse-related interactions, which achieved approximately 90% accuracy. Furthermore, GRU networks outperformed LSTM models in the biometric approach, with Valence prediction reaching 84.38% accuracy.
zh
[AI-45] AnchorDP3: 3D Affordance Guided Sparse Diffusion Policy for Robotic Manipulation
【速读】：该论文旨在解决在高度随机化环境中实现双臂机器人操作的高成功率与泛化能力问题。其解决方案的关键在于提出AnchorDP3框架，该框架通过三项核心创新实现性能突破：一是利用模拟器监督的语义分割技术，通过渲染的真实标注数据对点云中的任务关键物体进行显式分割，提供强效的可操作性先验；二是任务条件特征编码器，通过轻量级模块处理增强后的点云数据，实现基于共享扩散动作专家的高效多任务学习；三是基于可操作性锚定的关键姿态扩散方法，通过稀疏且具有几何意义的动作锚点（如预抓取姿态和抓取姿态）替代密集轨迹预测，显著简化预测空间，并强制动作专家同时预测机器人关节角度与末端执行器位姿，以几何一致性加速收敛并提升精度。
链接: https://arxiv.org/abs/2506.19269
作者: Ziyan Zhao,Ke Fan,He-Yang Xu,Ning Qiao,Bo Peng,Wenlong Gao,Dongjiang Li,Hui Shen
机构: 未知
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI)
备注:
Abstract:We present AnchorDP3, a diffusion policy framework for dual-arm robotic manipulation that achieves state-of-the-art performance in highly randomized environments. AnchorDP3 integrates three key innovations: (1) Simulator-Supervised Semantic Segmentation, using rendered ground truth to explicitly segment task-critical objects within the point cloud, which provides strong affordance priors; (2) Task-Conditioned Feature Encoders, lightweight modules processing augmented point clouds per task, enabling efficient multi-task learning through a shared diffusion-based action expert; (3) Affordance-Anchored Keypose Diffusion with Full State Supervision, replacing dense trajectory prediction with sparse, geometrically meaningful action anchors, i.e., keyposes such as pre-grasp pose, grasp pose directly anchored to affordances, drastically simplifying the prediction space; the action expert is forced to predict both robot joint angles and end-effector poses simultaneously, which exploits geometric consistency to accelerate convergence and boost accuracy. Trained on large-scale, procedurally generated simulation data, AnchorDP3 achieves a 98.7% average success rate in the RoboTwin benchmark across diverse tasks under extreme randomization of objects, clutter, table height, lighting, and backgrounds. This framework, when integrated with the RoboTwin real-to-sim pipeline, has the potential to enable fully autonomous generation of deployable visuomotor policies from only scene and instruction, totally eliminating human demonstrations from learning manipulation skills.
zh
[AI-46] Enhancing Generalization of Spiking Neural Networks Through Temporal Regularization
【速读】：该论文旨在解决直接训练的脉冲神经网络（Spiking Neural Networks, SNNs）在有限规模的类脑数据集和梯度不匹配问题下出现的严重过拟合问题，从而限制了其泛化性能。论文提出的解决方案是时间正则化训练（Temporal Regularization Training, TRT），其关键在于引入一种时间依赖的正则化机制，以加强对早期时间步的约束，从而有效缓解过拟合并提升模型的泛化能力。
链接: https://arxiv.org/abs/2506.19256
作者: Boxuan Zhang,Zhen Xu,Kuan Tao
机构: 未知
类目: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
备注: Code is available at this https URL
Abstract:Spiking Neural Networks (SNNs) have received widespread attention due to their event-driven and low-power characteristics, making them particularly effective for processing event-based neuromorphic data. Recent studies have shown that directly trained SNNs suffer from severe overfitting issues due to the limited scale of neuromorphic datasets and the gradient mismatching problem, which fundamentally constrain their generalization performance. In this paper, we propose a temporal regularization training (TRT) method by introducing a time-dependent regularization mechanism to enforce stronger constraints on early timesteps. We compare the performance of TRT with other state-of-the-art methods performance on datasets including CIFAR10/100, ImageNet100, DVS-CIFAR10, and N-Caltech101. To validate the effectiveness of TRT, we conducted ablation studies and analyses including loss landscape visualization and learning curve analysis, demonstrating that TRT can effectively mitigate overfitting and flatten the training loss landscape, thereby enhancing generalizability. Furthermore, we establish a theoretical interpretation of TRT’s temporal regularization mechanism based on the results of Fisher information analysis. We analyze the temporal information dynamics inside SNNs by tracking Fisher information during the TRT training process, revealing the Temporal Information Concentration (TIC) phenomenon, where Fisher information progressively concentrates in early timesteps. The time-decaying regularization mechanism implemented in TRT effectively guides the network to learn robust features in early timesteps with rich information, thereby leading to significant improvements in model generalization. Code is available at this https URL.
zh
[AI-47] Robust Behavior Cloning Via Global Lipschitz Regularization
【速读】：该论文旨在解决行为克隆（Behavior Cloning, BC）在部署过程中因观测误差或对抗性干扰导致策略性能下降的问题。其关键解决方案是采用全局Lipschitz正则化方法，以增强学习到的策略网络的鲁棒性，并通过构建具有Lipschitz性质的神经网络来保证策略对不同有界范数扰动的鲁棒性证书。
链接: https://arxiv.org/abs/2506.19250
作者: Shili Wu,Yizhao Jin,Puhua Niu,Aniruddha Datta,Sean B. Andersson
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:Behavior Cloning (BC) is an effective imitation learning technique and has even been adopted in some safety-critical domains such as autonomous vehicles. BC trains a policy to mimic the behavior of an expert by using a dataset composed of only state-action pairs demonstrated by the expert, without any additional interaction with the environment. However, During deployment, the policy observations may contain measurement errors or adversarial disturbances. Since the observations may deviate from the true states, they can mislead the agent into making sub-optimal actions. In this work, we use a global Lipschitz regularization approach to enhance the robustness of the learned policy network. We then show that the resulting global Lipschitz property provides a robustness certificate to the policy with respect to different bounded norm perturbations. Then, we propose a way to construct a Lipschitz neural network that ensures the policy robustness. We empirically validate our theory across various environments in Gymnasium. Keywords: Robust Reinforcement Learning; Behavior Cloning; Lipschitz Neural Network
zh
[AI-48] RecLLM -R1: A Two-Stage Training Paradigm with Reinforcement Learning and Chain-of-Thought v1
【速读】：该论文旨在解决传统推荐系统中存在的“过滤气泡”、外部知识利用不足以及模型优化与业务策略迭代脱节等问题。其解决方案的关键在于提出RecLLM-R1框架，该框架通过将用户画像、历史交互和多维物品属性转化为大语言模型（Large Language Model, LLM）可理解的自然语言提示，并采用两阶段训练策略：第一阶段通过监督微调（Supervised Fine-Tuning, SFT）赋予LLM基础推荐能力，第二阶段则结合群体相对策略优化（Group Relative Policy Optimization, GRPO）与思维链（Chain-of-Thought, CoT）机制，实现多步骤推理与全局决策，从而在提升推荐精度的同时兼顾多样性及其他定制化业务目标。
链接: https://arxiv.org/abs/2506.19235
作者: Yu Xie,Xingkai Ren,Ying Qi,Yao Hu,Lianlei Shan
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Traditional recommendation systems often grapple with “filter bubbles”, underutilization of external knowledge, and a disconnect between model optimization and business policy iteration. To address these limitations, this paper introduces RecLLM-R1, a novel recommendation framework leveraging Large Language Models (LLMs) and drawing inspiration from the DeepSeek R1 methodology. The framework initiates by transforming user profiles, historical interactions, and multi-faceted item attributes into LLM-interpretable natural language prompts through a carefully engineered data construction process. Subsequently, a two-stage training paradigm is employed: the initial stage involves Supervised Fine-Tuning (SFT) to imbue the LLM with fundamental recommendation capabilities. The subsequent stage utilizes Group Relative Policy Optimization (GRPO), a reinforcement learning technique, augmented with a Chain-of-Thought (CoT) mechanism. This stage guides the model through multi-step reasoning and holistic decision-making via a flexibly defined reward function, aiming to concurrently optimize recommendation accuracy, diversity, and other bespoke business objectives. Empirical evaluations on a real-world user behavior dataset from a large-scale social media platform demonstrate that RecLLM-R1 significantly surpasses existing baseline methods across a spectrum of evaluation metrics, including accuracy, diversity, and novelty. It effectively mitigates the filter bubble effect and presents a promising avenue for the integrated optimization of recommendation models and policies under intricate business goals.
zh
[AI-49] GBGC: Efficient and Adaptive Graph Coarsening via Granular-ball Computing
【速读】：该论文试图解决图粗化过程中如何在保持原始图关键信息的同时，生成更小且更易管理的图的问题。传统方法主要从谱保持的角度出发，依赖预定义的粗化规则来匹配原始图与粗化图的拉普拉斯矩阵特征值，但忽略了图中不同粒度层级的子区域特性。该论文提出的解决方案的关键在于结合图结构的多粒度特性，通过引入自适应的粒球图细化机制，从粗到细地将原图分割为不同大小和最优粒度的粒球，并利用这些粒球作为超节点构建粗化图，从而显著提升粗化效果和效率。
链接: https://arxiv.org/abs/2506.19224
作者: Shuyin Xia,Guan Wang,Gaojie Xu,Sen Zhao,Guoyin Wang
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:The objective of graph coarsening is to generate smaller, more manageable graphs while preserving key information of the original graph. Previous work were mainly based on the perspective of spectrum-preserving, using some predefined coarsening rules to make the eigenvalues of the Laplacian matrix of the original graph and the coarsened graph match as much as possible. However, they largely overlooked the fact that the original graph is composed of subregions at different levels of granularity, where highly connected and similar nodes should be more inclined to be aggregated together as nodes in the coarsened graph. By combining the multi-granularity characteristics of the graph structure, we can generate coarsened graph at the optimal granularity. To this end, inspired by the application of granular-ball computing in multi-granularity, we propose a new multi-granularity, efficient, and adaptive coarsening method via granular-ball (GBGC), which significantly improves the coarsening results and efficiency. Specifically, GBGC introduces an adaptive granular-ball graph refinement mechanism, which adaptively splits the original graph from coarse to fine into granular-balls of different sizes and optimal granularity, and constructs the coarsened graph using these granular-balls as supernodes. In addition, compared with other state-of-the-art graph coarsening methods, the processing speed of this method can be increased by tens to hundreds of times and has lower time complexity. The accuracy of GBGC is almost always higher than that of the original graph due to the good robustness and generalization of the granular-ball computing, so it has the potential to become a standard graph data preprocessing method.
zh
[AI-50] Private Model Personalization Revisited ICML2025
【速读】：该论文试图解决在用户级差分隐私（User-level Differential Privacy）约束下，于共享表示框架中实现模型个性化的问题。具体而言，面对数据统计异质性的多个用户，其最优参数共享一个未知的嵌入矩阵 $ U^* \in \mathbb{R}^{d \times k} $（其中 $ k \ll d $），目标是私密地恢复该共享嵌入及其局部低维表示，并在联邦设置中实现较小的额外风险。解决方案的关键在于提出一种私密且高效的联邦学习算法，基于 [CHM+21] 中的 FedRep 算法，在满足差分隐私的同时适用于噪声标签场景，并在更广泛的用户分布（子高斯分布）下提供实用保证。此外，通过改进隐私误差项并利用 Johnson-Lindenstrauss 变换降低有效维度，实现了与维度无关的风险界。
链接: https://arxiv.org/abs/2506.19220
作者: Conor Snedeker,Xinyu Zhou,Raef Bassily
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Distributed, Parallel, and Cluster Computing (cs.DC)
备注: ICML 2025
Abstract:We study model personalization under user-level differential privacy (DP) in the shared representation framework. In this problem, there are n users whose data is statistically heterogeneous, and their optimal parameters share an unknown embedding U^* \in\mathbbR^d\times k that maps the user parameters in \mathbbR^d to low-dimensional representations in \mathbbR^k , where k\ll d . Our goal is to privately recover the shared embedding and the local low-dimensional representations with small excess risk in the federated setting. We propose a private, efficient federated learning algorithm to learn the shared embedding based on the FedRep algorithm in [CHM+21]. Unlike [CHM+21], our algorithm satisfies differential privacy, and our results hold for the case of noisy labels. In contrast to prior work on private model personalization [JRS+21], our utility guarantees hold under a larger class of users’ distributions (sub-Gaussian instead of Gaussian distributions). Additionally, in natural parameter regimes, we improve the privacy error term in [JRS+21] by a factor of \widetildeO(dk) . Next, we consider the binary classification setting. We present an information-theoretic construction to privately learn the shared embedding and derive a margin-based accuracy guarantee that is independent of d . Our method utilizes the Johnson-Lindenstrauss transform to reduce the effective dimensions of the shared embedding and the users’ data. This result shows that dimension-independent risk bounds are possible in this setting under a margin loss.
zh
[AI-51] Spiritual-LLM : Gita Inspired Mental Health Therapy In the Era of LLM s
【速读】：该论文试图解决传统心理健康支持系统仅基于用户当前情绪和情境生成响应，导致干预表面化、无法满足深层次情感需求的问题。解决方案的关键在于将《薄伽梵歌》中的精神智慧与先进的大型语言模型GPT-4o相结合，构建了GITes（Gita Integrated Therapy for Emotional Support）数据集，并通过引入新的Spiritual Insight指标及基于链式思维提示的LLM作为评审框架，评估生成响应的精神相关性。该方法显著提升了Phi3-Mini 3.2B Instruct模型在NLP和精神指标上的表现。
链接: https://arxiv.org/abs/2506.19185
作者: Janak Kapuriya,Aman Singh,Jainendra Shukla,Rajiv Ratn Shah
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Traditional mental health support systems often generate responses based solely on the user’s current emotion and situations, resulting in superficial interventions that fail to address deeper emotional needs. This study introduces a novel framework by integrating spiritual wisdom from the Bhagavad Gita with advanced large language model GPT-4o to enhance emotional well-being. We present the GITes (Gita Integrated Therapy for Emotional Support) dataset, which enhances the existing ExTES mental health dataset by including 10,729 spiritually guided responses generated by GPT-4o and evaluated by domain experts. We benchmark GITes against 12 state-of-the-art LLMs, including both mental health specific and general purpose models. To evaluate spiritual relevance in generated responses beyond what conventional n-gram based metrics capture, we propose a novel Spiritual Insight metric and automate assessment via an LLM as jury framework using chain-of-thought prompting. Integrating spiritual guidance into AI driven support enhances both NLP and spiritual metrics for the best performing LLM Phi3-Mini 3.2B Instruct, achieving improvements of 122.71% in ROUGE, 126.53% in METEOR, 8.15% in BERT score, 15.92% in Spiritual Insight, 18.61% in Sufficiency and 13.22% in Relevance compared to its zero-shot counterpart. While these results reflect substantial improvements across automated empathy and spirituality metrics, further validation in real world patient populations remains a necessary step. Our findings indicate a strong potential for AI systems enriched with spiritual guidance to enhance user satisfaction and perceived support outcomes. The code and dataset will be publicly available to advance further research in this emerging area.
zh
[AI-52] Finding Clustering Algorithms in the Transformer Architecture
【速读】：该论文试图解决的问题是：尽管Transformer架构在人工智能领域取得了显著成功，但其是否能够学习并实现精确的算法仍不明确。论文提出的关键解决方案是设计一种称为k-means Transformer的架构，该架构能够精确实现Lloyd’s算法，这是k-means聚类中的一个基础且广泛使用的算法。该解决方案基于现代Transformer的标准组件，如注意力机制和残差连接，并通过理论证明和数值实现验证了其有效性，从而展示了Transformer机制如何精确映射到算法过程。
链接: https://arxiv.org/abs/2506.19125
作者: Kenneth L. Clarkson,Lior Horesh,Takuya Ito,Charlotte Park,Parikshit Ram
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注:
Abstract:The invention of the transformer architecture has revolutionized Artificial Intelligence (AI), yielding unprecedented success in areas such as natural language processing, computer vision, and multimodal reasoning. Despite these advances, it is unclear whether transformers are able to learn and implement precise algorithms. Here, we demonstrate that transformers can exactly implement a fundamental and widely used algorithm for k -means clustering: Lloyd’s algorithm. First, we theoretically prove the existence of such a transformer architecture, which we term the k -means transformer, that exactly implements Lloyd’s algorithm for k -means clustering using the standard ingredients of modern transformers: attention and residual connections. Next, we numerically implement this transformer and demonstrate in experiments the exact correspondence between our architecture and Lloyd’s algorithm, providing a fully neural implementation of k -means clustering. Finally, we demonstrate that interpretable alterations (e.g., incorporating layer normalizations or multilayer perceptrons) to this architecture yields diverse and novel variants of clustering algorithms, such as soft k -means, spherical k -means, trimmed k -means, and more. Collectively, our findings demonstrate how transformer mechanisms can precisely map onto algorithmic procedures, offering a clear and interpretable perspective on implementing precise algorithms in transformers.
zh
[AI-53] CUPID: Curating Data your Robot Loves with Influence Functions
【速读】：该论文旨在解决机器人模仿学习中演示数据质量与组成对策略性能影响的精确理解问题，特别是如何量化单个演示对下游任务成功或失败的影响。其解决方案的关键在于提出CUPID方法，该方法基于一种新颖的影响函数理论框架，用于估计每个训练演示对策略期望回报的影响，从而实现对演示数据的排序与选择，以优化策略的闭环性能。
链接: https://arxiv.org/abs/2506.19121
作者: Christopher Agia,Rohan Sinha,Jingyun Yang,Rika Antonova,Marco Pavone,Haruki Nishimura,Masha Itkina,Jeannette Bohg
机构: 未知
类目: Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Project page: this https URL . 28 pages, 15 figures
Abstract:In robot imitation learning, policy performance is tightly coupled with the quality and composition of the demonstration data. Yet, developing a precise understanding of how individual demonstrations contribute to downstream outcomes - such as closed-loop task success or failure - remains a persistent challenge. We propose CUPID, a robot data curation method based on a novel influence function-theoretic formulation for imitation learning policies. Given a set of evaluation rollouts, CUPID estimates the influence of each training demonstration on the policy’s expected return. This enables ranking and selection of demonstrations according to their impact on the policy’s closed-loop performance. We use CUPID to curate data by 1) filtering out training demonstrations that harm policy performance and 2) subselecting newly collected trajectories that will most improve the policy. Extensive simulated and hardware experiments show that our approach consistently identifies which data drives test-time performance. For example, training with less than 33% of curated data can yield state-of-the-art diffusion policies on the simulated RoboMimic benchmark, with similar gains observed in hardware. Furthermore, hardware experiments show that our method can identify robust strategies under distribution shift, isolate spurious correlations, and even enhance the post-training of generalist robot policies. Additional materials are made available at: this https URL.
zh
[AI-54] Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systems
【速读】：该论文试图解决提示注入（prompt injection）攻击对基于大语言模型（LLM）的软件应用所带来的安全威胁，特别是针对提示泄露（prompt leakage）攻击的检测问题。解决方案的关键在于分析和比较现有的开源检测技术，如LLM Guard、Vigil和Rebuff，并评估其在不同场景下的检测性能。研究发现，现有方法在检测提示泄露攻击方面存在不足，尤其是Vigil和Rebuff中的“canary word”检查机制效果有限，同时Rebuff的基于次级模型的检测方法存在可规避的弱点，因此提出了相应的改进与缓解措施。
链接: https://arxiv.org/abs/2506.19109
作者: Valerii Gakh,Hayretdin Bahsi
机构: 未知
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注: 18 pages, 8 tables, 7 figures
Abstract:Prompt injection threatens novel applications that emerge from adapting LLMs for various user tasks. The newly developed LLM-based software applications become more ubiquitous and diverse. However, the threat of prompt injection attacks undermines the security of these systems as the mitigation and defenses against them, proposed so far, are insufficient. We investigated the capabilities of early prompt injection detection systems, focusing specifically on the detection performance of techniques implemented in various open-source solutions. These solutions are supposed to detect certain types of prompt injection attacks, including the prompt leak. In prompt leakage attacks, an attacker maliciously manipulates the LLM into outputting its system instructions, violating the system’s confidentiality. Our study presents analyzes of distinct prompt leakage detection techniques, and a comparative analysis of several detection solutions, which implement those techniques. We identify the strengths and weaknesses of these techniques and elaborate on their optimal configuration and usage in high-stake deployments. In one of the first studies on existing prompt leak detection solutions, we compared the performances of LLM Guard, Vigil, and Rebuff. We concluded that the implementations of canary word checks in Vigil and Rebuff were not effective at detecting prompt leak attacks, and we proposed improvements for them. We also found an evasion weakness in Rebuff’s secondary model-based technique and proposed a mitigation. Then, the result of the comparison of LLM Guard, Vigil, and Rebuff at their peak performance revealed that Vigil is optimal for cases when minimal false positive rate is required, and Rebuff is the most optimal for average needs.
zh
[AI-55] Improving Student-AI Interaction Through Pedagogical Prompting: An Example in Computer Science Education
【速读】：该论文试图解决学生在使用生成式 AI (Generative AI) 时存在的（误）用问题，旨在通过教授学生如何有效提示大型语言模型 (LLM) 来提升学习效果。其解决方案的关键在于提出“教学提示”（pedagogical prompting）这一理论基础的新概念，并设计了一个基于情景的教学干预方案，通过交互式系统训练学生的教学提示技能，从而改善其基于 LLM 的学习求助行为。
链接: https://arxiv.org/abs/2506.19107
作者: Ruiwei Xiao,Xinying Hou,Runlong Ye,Majeed Kazemitabaar,Nicholas Diana,Michael Liut,John Stamper
机构: 未知
类目: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
备注: Under review for Computer Education: Artificial Intelligence. Journal policy allows submitting as preprint
Abstract:With the proliferation of large language model (LLM) applications since 2022, their use in education has sparked both excitement and concern. Recent studies consistently highlight students’ (mis)use of LLMs can hinder learning outcomes. This work aims to teach students how to effectively prompt LLMs to improve their learning. We first proposed pedagogical prompting, a theoretically-grounded new concept to elicit learning-oriented responses from LLMs. To move from concept design to a proof-of-concept learning intervention in real educational settings, we selected early undergraduate CS education (CS1/CS2) as the example context. We began with a formative survey study with instructors (N=36) teaching early-stage undergraduate-level CS courses to inform the instructional design based on classroom needs. Based on their insights, we designed and developed a learning intervention through an interactive system with scenario-based instruction to train pedagogical prompting skills. Finally, we evaluated its instructional effectiveness through a user study with CS novice students (N=22) using pre/post-tests. Through mixed methods analyses, our results indicate significant improvements in learners’ LLM-based pedagogical help-seeking skills, along with positive attitudes toward the system and increased willingness to use pedagogical prompts in the future. Our contributions include (1) a theoretical framework of pedagogical prompting; (2) empirical insights into current instructor attitudes toward pedagogical prompting; and (3) a learning intervention design with an interactive learning tool and scenario-based instruction leading to promising results on teaching LLM-based help-seeking. Our approach is scalable for broader implementation in classrooms and has the potential to be integrated into tools like ChatGPT as an on-boarding experience to encourage learning-oriented use of generative AI.
zh
[AI-56] Baba is LLM : Reasoning in a Game with Dynamic Rules
【速读】：该论文试图解决大型语言模型（Large Language Models, LLMs）在处理需要动态规则推理的任务时表现不佳的问题，具体通过评估LLMs在2D解谜游戏《Baba is You》中的表现来探索其推理能力。解决方案的关键在于利用游戏中的规则操控机制，该机制依赖于语言理解和逻辑推理，从而为LLMs提供了一个具有挑战性的测试环境。研究通过不同类型的提示（包括简单提示、规则扩展提示和动作扩展提示）以及对部分模型进行微调，分析了LLMs在识别游戏机制、应用规则变化及解决问题方面的能力。
链接: https://arxiv.org/abs/2506.19095
作者: Fien van Wetten,Aske Plaat,Max van Duijn
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Large language models (LLMs) are known to perform well on language tasks, but struggle with reasoning tasks. This paper explores the ability of LLMs to play the 2D puzzle game Baba is You, in which players manipulate rules by rearranging text blocks that define object properties. Given that this rule-manipulation relies on language abilities and reasoning, it is a compelling challenge for LLMs. Six LLMs are evaluated using different prompt types, including (1) simple, (2) rule-extended and (3) action-extended prompts. In addition, two models (Mistral, OLMo) are finetuned using textual and structural data from the game. Results show that while larger models (particularly GPT-4o) perform better in reasoning and puzzle solving, smaller unadapted models struggle to recognize game mechanics or apply rule changes. Finetuning improves the ability to analyze the game levels, but does not significantly improve solution formulation. We conclude that even for state-of-the-art and finetuned LLMs, reasoning about dynamic rule changes is difficult (specifically, understanding the use-mention distinction). The results provide insights into the applicability of LLMs to complex problem-solving tasks and highlight the suitability of games with dynamically changing rules for testing reasoning and reflection by LLMs.
zh
[AI-57] FairCauseSyn: Towards Causally Fair LLM -Augmented Synthetic Data Generation
【速读】：该论文旨在解决在健康领域中生成高质量且具有因果公平性的合成数据的问题。现有基于生成对抗网络（GAN）和大语言模型（LLM）的方法主要关注反事实公平性，但未在健康场景中考虑因果公平性。本文的关键解决方案是开发首个结合LLM的合成数据生成方法，通过保留因果结构来增强因果公平性，从而在因果公平性度量上与真实数据的偏差小于10%，并在使用因果公平预测器训练时，将敏感属性上的偏见减少70%。
链接: https://arxiv.org/abs/2506.19082
作者: Nitish Nagesh,Ziyu Wang,Amir M. Rahmani
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: Accepted to IEEE EMBC 2025
Abstract:Synthetic data generation creates data based on real-world data using generative models. In health applications, generating high-quality data while maintaining fairness for sensitive attributes is essential for equitable outcomes. Existing GAN-based and LLM-based methods focus on counterfactual fairness and are primarily applied in finance and legal domains. Causal fairness provides a more comprehensive evaluation framework by preserving causal structure, but current synthetic data generation methods do not address it in health settings. To fill this gap, we develop the first LLM-augmented synthetic data generation method to enhance causal fairness using real-world tabular health data. Our generated data deviates by less than 10% from real data on causal fairness metrics. When trained on causally fair predictors, synthetic data reduces bias on the sensitive attribute by 70% compared to real data. This work improves access to fair synthetic data, supporting equitable health research and healthcare delivery.
zh
[AI-58] From Rows to Yields: How Foundation Models for Tabular Data Simplify Crop Yield Prediction
【速读】：该论文试图解决南非次国家级（sub-national）夏季作物产量预测问题，旨在利用地球观测数据和气象数据提高预测的准确性与实用性。解决方案的关键在于应用一种针对小到中型表格数据的预训练模型（TabPFN），通过其在回归和分类任务中的优越性能，结合去十年期（dekadal）时间序列的地球观测数据（如FAPAR和土壤湿度）以及网格化气象数据（气温、降水和辐射），实现对作物产量的高效预测。相比传统机器学习模型，TabPFN在减少特征工程需求和提升调参效率方面表现出显著优势，从而增强了其在实际应用中的可行性。
链接: https://arxiv.org/abs/2506.19046
作者: Filip Sabo,Michele Meroni,Maria Piles,Martin Claverie,Fanie Ferreira,Elna Van Den Berg,Francesco Collivignarelli,Felix Rembold
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:We present an application of a foundation model for small- to medium-sized tabular data (TabPFN), to sub-national yield forecasting task in South Africa. TabPFN has recently demonstrated superior performance compared to traditional machine learning (ML) models in various regression and classification tasks. We used the dekadal (10-days) time series of Earth Observation (EO; FAPAR and soil moisture) and gridded weather data (air temperature, precipitation and radiation) to forecast the yield of summer crops at the sub-national level. The crop yield data was available for 23 years and for up to 8 provinces. Covariate variables for TabPFN (i.e., EO and weather) were extracted by region and aggregated at a monthly scale. We benchmarked the results of the TabPFN against six ML models and three baseline models. Leave-one-year-out cross-validation experiment setting was used in order to ensure the assessment of the models capacity to forecast an unseen year. Results showed that TabPFN and ML models exhibit comparable accuracy, outperforming the baselines. Nonetheless, TabPFN demonstrated superior practical utility due to its significantly faster tuning time and reduced requirement for feature engineering. This renders TabPFN a more viable option for real-world operation yield forecasting applications, where efficiency and ease of implementation are paramount.
zh
[AI-59] Survey of HPC in US Research Institutions
【速读】：该论文试图解决美国高校在高性能计算（HPC）资源方面相对于国家实验室和工业界存在显著不足的问题，特别是在面对生成式 AI (Generative AI) 等计算密集型任务时，高校集群的增长率（CAGR ≈ 18%）远低于国家实验室（≈ 43%）和工业界（≈ 78%）。解决方案的关键在于通过联邦计算、空闲 GPU 捕获以及成本分摊等模式来缩小能力差距，并探索去中心化强化学习等新兴范式以推动校园内 AI 训练的民主化。
链接: https://arxiv.org/abs/2506.19019
作者: Peng Shu,Junhao Chen,Zhengliang Liu,Huaqin Zhao,Xinliang Li,Tianming Liu
机构: 未知
类目: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)
备注:
Abstract:The rapid growth of AI, data-intensive science, and digital twin technologies has driven an unprecedented demand for high-performance computing (HPC) across the research ecosystem. While national laboratories and industrial hyperscalers have invested heavily in exascale and GPU-centric architectures, university-operated HPC systems remain comparatively under-resourced. This survey presents a comprehensive assessment of the HPC landscape across U.S. universities, benchmarking their capabilities against Department of Energy (DOE) leadership-class systems and industrial AI infrastructures. We examine over 50 premier research institutions, analyzing compute capacity, architectural design, governance models, and energy efficiency. Our findings reveal that university clusters, though vital for academic research, exhibit significantly lower growth trajectories (CAGR \approx 18%) than their national ( \approx 43%) and industrial ( \approx 78%) counterparts. The increasing skew toward GPU-dense AI workloads has widened the capability gap, highlighting the need for federated computing, idle-GPU harvesting, and cost-sharing models. We also identify emerging paradigms, such as decentralized reinforcement learning, as promising opportunities for democratizing AI training within campus environments. Ultimately, this work provides actionable insights for academic leaders, funding agencies, and technology partners to ensure more equitable and sustainable HPC access in support of national research priorities.
zh
[AI-60] IndieFake Dataset: A Benchmark Dataset for Audio Deepfake Detection
【速读】：该论文试图解决现有音频深度伪造（Deepfake）检测数据集在多样性方面的不足，特别是缺乏南亚裔语音样本的问题，这导致模型在多样化的语言和文化背景下检测能力受限。解决方案的关键在于构建一个包含50名英语印度说话者的27.17小时真实与深度伪造音频的IndieFake Dataset (IFD)，该数据集具有平衡的数据分布和说话人层面的特征描述，弥补了ASVspoof21 (DF)等现有数据集的不足。
链接: https://arxiv.org/abs/2506.19014
作者: Abhay Kumar,Kunal Verma,Omkar More
机构: 未知
类目: ound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
备注:
Abstract:Advancements in audio deepfake technology offers benefits like AI assistants, better accessibility for speech impairments, and enhanced entertainment. However, it also poses significant risks to security, privacy, and trust in digital communications. Detecting and mitigating these threats requires comprehensive datasets. Existing datasets lack diverse ethnic accents, making them inadequate for many real-world scenarios. Consequently, models trained on these datasets struggle to detect audio deepfakes in diverse linguistic and cultural contexts such as in South-Asian countries. Ironically, there is a stark lack of South-Asian speaker samples in the existing datasets despite constituting a quarter of the worlds population. This work introduces the IndieFake Dataset (IFD), featuring 27.17 hours of bonafide and deepfake audio from 50 English speaking Indian speakers. IFD offers balanced data distribution and includes speaker-level characterization, absent in datasets like ASVspoof21 (DF). We evaluated various baselines on IFD against existing ASVspoof21 (DF) and In-The-Wild (ITW) datasets. IFD outperforms ASVspoof21 (DF) and proves to be more challenging compared to benchmark ITW dataset. The dataset will be publicly available upon acceptance.
zh
[AI-61] Citizenship Challenges in Artificial Intelligence Education
【速读】：该论文试图解决人工智能（Artificial Intelligence, AI）在教育领域中涉及的公民身份挑战，特别是针对学生、教师及其他教育利益相关者在AI整合背景下的问题。其解决方案的关键在于培养AI意识与教育，并通过多种策略促进对AI培训的社会批判性方法，以识别并优先考虑相关且符合伦理的AI使用方式。同时，论文还强调了在特定AI支持的教育活动中，如何调动批判性思维和计算思维技能，这取决于这些活动所需的创造性与变革性参与程度。
链接: https://arxiv.org/abs/2506.18955
作者: Margarida Romero(UniCA, UIC, LINE)
机构: 未知
类目: Computers and Society (cs.CY); Artificial Intelligence (cs.AI)
备注: in French language
Abstract:This chapter addresses the citizenship challenges related to AI in education, particularly concerning students, teachers, and other educational stakeholders in the context of AI integration. We first explore how to foster AI awareness and education, along with various strategies to promote a socio-critical approach to AI training, aiming to identify relevant and ethical uses to prioritise. In the second part, we discuss critical thinking and computational thinking skills that can be mobilised within certain AI-supported educational activities, depending on the degree of creative and transformative engagement those activities require.
zh
[AI-62] SHAMaNS: Sound Localization with Hybrid Alpha-Stable Spatial Measure and Neural Steerer
【速读】：该论文旨在解决多声源环境下声源定位（Sound Source Localization, SSL）的准确性与鲁棒性问题。其解决方案的关键在于将α稳定模型与基于神经网络的导向矢量建模相结合，其中采用物理信息神经网络（Neural Steerer）对固定麦克风阵列上的测量导向矢量进行插值，从而更稳健地估计α稳定空间度量，该度量表征目标信号最可能的到达方向（DOA）。同时，利用α稳定模型对Neural Steerer在下游任务中的剩余重建误差进行建模，以提升定位性能。
链接: https://arxiv.org/abs/2506.18954
作者: Diego Di Carlo(RIKEN AIP),Mathieu Fontaine(LTCI, IP Paris),Aditya Arie Nugraha(RIKEN AIP),Yoshiaki Bando(RIKEN AIP),Kazuyoshi Yoshii
机构: 未知
类目: ound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
备注: European Signal Processing Conference (EUSIPCO), Sep 2025, Palermo, Italy
Abstract:This paper describes a sound source localization (SSL) technique that combines an \alpha -stable model for the observed signal with a neural network-based approach for modeling steering vectors. Specifically, a physics-informed neural network, referred to as Neural Steerer, is used to interpolate measured steering vectors (SVs) on a fixed microphone array. This allows for a more robust estimation of the so-called \alpha -stable spatial measure, which represents the most plausible direction of arrival (DOA) of a target signal. As an \alpha -stable model for the non-Gaussian case ( \alpha \in (0, 2)) theoretically defines a unique spatial measure, we choose to leverage it to account for residual reconstruction error of the Neural Steerer in the downstream tasks. The objective scores indicate that our proposed technique outperforms state-of-the-art methods in the case of multiple sound sources.
zh
[AI-63] SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications
【速读】：该论文旨在解决复杂SQL问题调试在实际数据库应用中的瓶颈问题，特别是针对当前大型语言模型（LLMs）在SQL调试任务中的表现尚未得到充分评估的现状。其解决方案的关键在于构建了BIRD-CRITIC基准测试集，并提出了Six-Gym训练环境，结合SQL-Rewind策略和f-Plan Boosting方法，以提升开源模型在SQL问题调试中的能力。通过这些技术，研究团队开发出开源代理Bird-Fixer，在多个基准测试中取得了优于现有专有模型的性能。
链接: https://arxiv.org/abs/2506.18951
作者: Jinyang Li,Xiaolong Li,Ge Qu,Per Jacobsson,Bowen Qin,Binyuan Hui,Shuzheng Si,Nan Huo,Xiaohan Xu,Yue Zhang,Ziwei Tang,Yuanshuai Li,Florensia Widjaja,Xintong Zhu,Feige Zhou,Yongfeng Huang,Yannis Papakonstantinou,Fatma Ozcan,Chenhao Ma,Reynold Cheng
机构: 未知
类目: Databases (cs.DB); Artificial Intelligence (cs.AI)
备注: 26 pages, 9 figures
Abstract:Resolution of complex SQL issues persists as a significant bottleneck in real-world database applications. Current Large Language Models (LLMs), while adept at text-to-SQL translation, have not been rigorously evaluated on the more challenging task of debugging SQL issues. To address this gap, we introduce BIRD-CRITIC, a new SQL issue debugging benchmark comprising 530 PostgreSQL tasks (BIRD-CRITIC-PG) and 570 multi-dialect tasks (BIRD-CRITIC-Multi), distilled from authentic user issues and replayed within new environments to facilitate rigorous evaluation. Baseline evaluations underscore the task’s complexity, with the leading reasoning model O3-Mini achieving only 38.87% success rate on BIRD-CRITIC-PG and 33.33% on BIRD-CRITIC-Multi. Meanwhile, advancing open-source models for database tasks is crucial for empowering local development while safeguarding data privacy. Therefore, we present Six-Gym (Sql-fIX-Gym), a training environment for elevating open-source model capabilities for SQL issue debugging. This environment leverages SQL-Rewind strategy, which automatically generates executable issue-solution datasets by reverse-engineering issues from verified SQLs. However, popular trajectory-based fine-tuning methods do not explore substantial supervisory signals. We further propose f-Plan Boosting, which extracts high-level debugging plans from SQL solutions, enabling teacher LLMs to produce 73.7% more successful trajectories for training. We integrate these components into an open-source agent, Bird-Fixer. Based on Qwen-2.5-Coder-14B, Bird-Fixer achieves 38.11% success rate on BIRD-CRITIC-PG and 29.65% on BIRD-CRITIC-Multi, surpassing leading proprietary models such as Claude-3.7-Sonnet and GPT-4.1, marking a significant step toward democratizing sophisticated SQL-debugging capabilities. The leaderboard and source code are available: this https URL
zh
[AI-64] Can AI support student engagement in classroom activities in higher education?
【速读】：该论文试图解决大规模课堂中学生与教师及学习内容之间参与度不足的问题（engagement）。解决方案的关键在于利用基于大型语言模型（LLMs）的对话式人工智能（CAI）工具，如ChatGPT，在课堂教学中提升学生的参与度。通过在软件工程课程中设计使用CAI的课堂活动，并与未使用CAI的活动进行对比实验，研究结果表明，CAI能够有效支持学生在课堂活动中的学习内容参与。
链接: https://arxiv.org/abs/2506.18941
作者: Neha Rani,Sharan Majumder,Ishan Bhardwaj,Pedro Guillermo Feijoo Garcia
机构: 未知
类目: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Human-Computer Interaction (cs.HC)
备注:
Abstract:Lucrative career prospects and creative opportunities often attract students to enroll in computer science majors and pursue advanced studies in the field. Consequently, there has been a significant surge in enrollment in computer science courses, resulting in large class sizes that can range from hundreds to even thousands of students. A common challenge in such large classrooms is the lack of engagement between students and both the instructor and the learning material. However, with advancements in technology and improvements in large language models (LLMs), there is a considerable opportunity to utilize LLM-based AI models, such as conversational artificial intelligence (CAI), to enhance student engagement with learning content in large classes. To explore the potential of CAI to support engagement, especially with learning content, we designed an activity in a software Engineering course (with a large class size) where students used CAI for an in-class activity. We conducted a within-subject investigation in a large classroom at a US university where we compared student engagement during an in-class activity that used CAI tool vs. one without CAI tool. The CAI tool we used was ChatGPT due to its widespread popularity and familiarity. Our results indicate that CAI (ChatGPT) has the potential to support engagement with learning content during in-class activities, especially in large class sizes. We further discuss the implications of our findings.
zh
[AI-65] AI Safety vs. AI Security: Demystifying the Distinction and Boundaries
【速读】：该论文试图解决人工智能（Artificial Intelligence, AI）在安全（Safety）与安全防护（Security）概念上的混淆问题，明确二者的研究边界及相互关系。解决方案的关键在于提供严谨的定义，厘清两者的研究重点，并探讨其相互依赖性，特别是安全漏洞可能导致安全失效，反之亦然。通过类比信息传输和建筑施工等实际场景，论文旨在为可信人工智能系统的研发提供理论支持与实践指导。
链接: https://arxiv.org/abs/2506.18932
作者: Zhiqiang Lin,Huan Sun,Ness Shroff
机构: 未知
类目: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
备注:
Abstract:Artificial Intelligence (AI) is rapidly being integrated into critical systems across various domains, from healthcare to autonomous vehicles. While its integration brings immense benefits, it also introduces significant risks, including those arising from AI misuse. Within the discourse on managing these risks, the terms “AI Safety” and “AI Security” are often used, sometimes interchangeably, resulting in conceptual confusion. This paper aims to demystify the distinction and delineate the precise research boundaries between AI Safety and AI Security. We provide rigorous definitions, outline their respective research focuses, and explore their interdependency, including how security breaches can precipitate safety failures and vice versa. Using clear analogies from message transmission and building construction, we illustrate these distinctions. Clarifying these boundaries is crucial for guiding precise research directions, fostering effective cross-disciplinary collaboration, enhancing policy effectiveness, and ultimately, promoting the deployment of trustworthy AI systems.
zh
[AI-66] Safe Pruning LoRA: Robust Distance-Guided Pruning for Safety Alignment in Adaptation of LLM s
【速读】：该论文旨在解决在使用低秩适应（LoRA）微调大型语言模型（LLMs）时，安全对齐可能被削弱的问题，这会导致模型更容易产生有害输出。现有安全对齐方法难以有效捕捉参数变化带来的复杂安全偏差，从而导致安全与效用之间的权衡不理想。论文提出的解决方案是Safe Pruning LoRA (SPLoRA)，其关键在于引入Empirical-DIEM (E-DIEM)，这是一种与维度无关的相似性度量，能够有效检测LoRA适配模型中的安全偏差，并通过选择性剪枝削弱安全对齐的LoRA层，在提升安全性的同时保持模型性能。
链接: https://arxiv.org/abs/2506.18931
作者: Shuang Ao,Yi Dong,Jinwei Hu,Sarvapali Ramchurn
机构: 未知
类目: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
备注: 13 pages, 3 figures
Abstract:Fine-tuning Large Language Models (LLMs) with Low-Rank Adaptation (LoRA) enhances adaptability while reducing computational costs. However, fine-tuning can compromise safety alignment, even with benign data, increasing susceptibility to harmful outputs. Existing safety alignment methods struggle to capture complex parameter shifts, leading to suboptimal safety-utility trade-offs. To address this issue, we propose Safe Pruning LoRA (SPLoRA), a novel pruning-based approach that selectively removes LoRA layers that weaken safety alignment, improving safety while preserving performance. At its core, we introduce Empirical-DIEM (E-DIEM), a dimension-insensitive similarity metric that effectively detects safety misalignment in LoRA-adapted models. We conduct extensive experiments on LLMs fine-tuned with mixed of benign and malicious data, and purely benign datasets, evaluating SPLoRA across utility, safety, and reliability metrics. Results demonstrate that SPLoRA outperforms state-of-the-art safety alignment techniques, significantly reducing safety risks while maintaining or improving model performance and reliability. Additionally, SPLoRA reduces inference overhead, making it a scalable and efficient solution for deploying safer and more reliable LLMs. The code is available at this https URL.
zh
[AI-67] Do LLM s Know When to Flip a Coin? Strategic Randomization through Reasoning and Experience
【速读】：该论文试图解决大型语言模型（Large Language Models, LLMs）在战略随机化（strategic randomization）方面研究不足的问题，即模型在面对需要不确定性决策的场景时，难以有效生成真正的随机策略。现有研究常将认知层面的随机决策与机械性的随机生成混为一谈，导致评估不全面。论文提出的解决方案关键在于设计一种受“田忌赛马”启发的零和博弈，其中纳什均衡对应于最大熵策略，从而能够有效区分模型在战略决策中的随机化能力。通过多轮竞赛游戏和系统提供的随机选择，该方法隔离了模型的随机化决策过程，揭示了不同模型在抽象推理和适应性学习方面的差异。
链接: https://arxiv.org/abs/2506.18928
作者: Lingyu Yang(1) ((1) Shanghai Jiao Tong University)
机构: 未知
类目: Artificial Intelligence (cs.AI)
备注:
Abstract:Strategic randomization is a key principle in game theory, yet it remains underexplored in large language models (LLMs). Prior work often conflates the cognitive decision to randomize with the mechanical generation of randomness, leading to incomplete evaluations. To address this, we propose a novel zero-sum game inspired by the Tian Ji Horse Race, where the Nash equilibrium corresponds to a maximal entropy strategy. The game’s complexity masks this property from untrained humans and underdeveloped LLMs. We evaluate five LLMs across prompt styles – framed, neutral, and hinted – using competitive multi-tournament gameplay with system-provided random choices, isolating the decision to randomize. Results show that weaker models remain deterministic regardless of prompts, while stronger models exhibit increased randomization under explicit hints. When facing weaker models, strong LLMs adopt deterministic strategies to exploit biases, but converge toward equilibrium play when facing peers. Through win/loss outcomes and Bayes factor analysis, we demonstrate meaningful variation in LLMs’ strategic reasoning capabilities, highlighting opportunities for improvement in abstract reasoning and adaptive learning. We make our implementation publicly available at this https URL to ensure full reproducibility.
zh
[AI-68] AI-based Approach in Early Warning Systems: Focus on Emergency Communication Ecosystem and Citizen Participation in Nordic Countries
【速读】：该论文试图解决气候变化和自然灾害带来的全球性挑战，这些问题需要复杂的生态系统来应对社会、经济和环境影响。其解决方案的关键在于采用整体性方法，区分准备阶段、应急响应阶段和危机后阶段，并特别强调早期预警系统（Early Warning System, EWS）、风险建模与缓解措施的作用。同时，论文探讨了人工智能（Artificial Intelligence, AI）在各阶段中的应用，重点聚焦于INFORM风险框架和EWS，并强调了应急通信与心理风险感知在应急响应中的重要性。
链接: https://arxiv.org/abs/2506.18926
作者: Fuzel Shaik,Getnet Demil,Mourad Oussalah
机构: 未知
类目: Computers and Society (cs.CY); Artificial Intelligence (cs.AI)
备注:
Abstract:Climate change and natural disasters are recognized as worldwide challenges requiring complex and efficient ecosystems to deal with social, economic, and environmental effects. This chapter advocates a holistic approach, distinguishing preparedness, emergency responses, and postcrisis phases. The role of the Early Warning System (EWS), Risk modeling and mitigation measures are particularly emphasized. The chapter reviews the various Artificial Intelligence (AI)-enabler technologies that can be leveraged at each phase, focusing on the INFORM risk framework and EWSs. Emergency communication and psychological risk perception have been emphasized in emergency response times. Finally, a set of case studies from Nordic countries has been highlighted.
zh
[AI-69] Signal Use and Emergent Cooperation
【速读】：该论文试图解决自主代理群体如何通过通信信号协调活动并提升集体效率的问题，其核心在于探索代理群体中文化自组织的形成机制及其对群体性能的影响。解决方案的关键在于使用NEC-DAC（Neurally Encoded Culture - Distributed Autonomous Communicators）系统，其中每个代理都配备独立的神经网络进行决策，通过学习和信号传递，代理群体发展出共享的行为系统，即类似文化的结构，并且通信策略的变化显著影响其适应性与合作能力。
链接: https://arxiv.org/abs/2506.18920
作者: Michael Williams
机构: 未知
类目: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multiagent Systems (cs.MA); Neural and Evolutionary Computing (cs.NE); Social and Information Networks (cs.SI)
备注: 167 pages, 19 figures, PhD dissertation, UCLA, 2006
Abstract:In this work, we investigate how autonomous agents, organized into tribes, learn to use communication signals to coordinate their activities and enhance their collective efficiency. Using the NEC-DAC (Neurally Encoded Culture - Distributed Autonomous Communicators) system, where each agent is equipped with its own neural network for decision-making, we demonstrate how these agents develop a shared behavioral system – akin to a culture – through learning and signalling. Our research focuses on the self-organization of culture within these tribes of agents and how varying communication strategies impact their fitness and cooperation. By analyzing different social structures, such as authority hierarchies, we show that the culture of cooperation significantly influences the tribe’s performance. Furthermore, we explore how signals not only facilitate the emergence of culture but also enable its transmission across generations of agents. Additionally, we examine the benefits of coordinating behavior and signaling within individual agents’ neural networks.
zh
[AI-70] Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases
【速读】：该论文试图解决用户在使用大型语言模型（Large Language Models, LLMs）作为个人代理时面临的隐私与性能之间的权衡问题。用户要么将敏感数据发送给不可信的LLM服务提供商，增加数据泄露风险；要么在可信设备上运行功能较弱的本地模型。解决方案的关键在于提出一种名为“苏格拉底式思维链推理”的混合框架，该框架首先将非私密用户查询发送至不可信的强LLM生成思维链（Chain-of-Thought, CoT）提示和子查询，随后在加密的向量数据库中执行高效的语义搜索，并最终将解密后的数据与CoT提示输入本地模型生成响应，从而在保证用户隐私的同时提升任务处理能力。
链接: https://arxiv.org/abs/2506.17336
作者: Yubeen Bae,Minchan Kim,Jaejin Lee,Sangbum Kim,Jaehyung Kim,Yejin Choi,Niloofar Mireshghallah
机构: 未知
类目: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
备注: 29 pages
Abstract:Large language models (LLMs) are increasingly used as personal agents, accessing sensitive user data such as calendars, emails, and medical records. Users currently face a trade-off: They can send private records, many of which are stored in remote databases, to powerful but untrusted LLM providers, increasing their exposure risk. Alternatively, they can run less powerful models locally on trusted devices. We bridge this gap. Our Socratic Chain-of-Thought Reasoning first sends a generic, non-private user query to a powerful, untrusted LLM, which generates a Chain-of-Thought (CoT) prompt and detailed sub-queries without accessing user data. Next, we embed these sub-queries and perform encrypted sub-second semantic search using our Homomorphically Encrypted Vector Database across one million entries of a single user’s private data. This represents a realistic scale of personal documents, emails, and records accumulated over years of digital activity. Finally, we feed the CoT prompt and the decrypted records to a local language model and generate the final response. On the LoCoMo long-context QA benchmark, our hybrid framework, combining GPT-4o with a local Llama-3.2-1B model, outperforms using GPT-4o alone by up to 7.1 percentage points. This demonstrates a first step toward systems where tasks are decomposed and split between untrusted strong LLMs and weak local ones, preserving user privacy.
zh
[AI-71] Neural Cellular Automata for ARC-AGI
【速读】：该论文试图解决在需要精确变换和少样本泛化的任务中，神经细胞自动机（Neural Cellular Automata, NCA）的表现问题。其解决方案的关键在于利用基于梯度的训练方法，学习从训练示例中迭代更新规则，将输入网格转换为输出网格，并将其应用于测试输入，从而实现对抽象网格任务的有效处理。
链接: https://arxiv.org/abs/2506.15746
作者: Kevin Xu,Risto Miikkulainen
机构: 未知
类目: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
备注: 8 pages, 5 figures
Abstract:Cellular automata and their differentiable counterparts, Neural Cellular Automata (NCA), are highly expressive and capable of surprisingly complex behaviors. This paper explores how NCAs perform when applied to tasks requiring precise transformations and few-shot generalization, using the Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) as a domain that challenges their capabilities in ways not previously explored. Specifically, this paper uses gradient-based training to learn iterative update rules that transform input grids into their outputs from the training examples and apply them to the test inputs. Results suggest that gradient-trained NCA models are a promising and efficient approach to a range of abstract grid-based tasks from ARC. Along with discussing the impacts of various design modifications and training constraints, this work examines the behavior and properties of NCAs applied to ARC to give insights for broader applications of self-organizing systems.
zh
[AI-72] A standard transformer and attention with linear biases for molecular conformer generation
【速读】：该论文旨在解决在药物发现和优化过程中，从二维分子图生成低能分子构象的问题。传统方法依赖于专门设计的等变网络，而近期非等变Transformer模型因其可扩展性成为一种可行替代方案，但其缺点是需要较大的模型规模来弥补缺乏等变偏差。论文提出的解决方案关键在于采用精心设计的位置编码，通过引入相对位置编码作为负注意力偏差，该编码根据图节点之间的最短路径距离以不同斜率线性增长，类似于自然语言处理领域广泛使用的ALiBi技术。实验表明，该方法在参数量仅为2500万时，已超越当前最先进的非等变基线模型（参数量为6400万）。
链接: https://arxiv.org/abs/2506.19834
作者: Viatcheslav Gurev,Timothy Rumbell
机构: 未知
类目: Biomolecules (q-bio.BM); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注: Revision of paper at OpenReview: this https URL
Abstract:Sampling low-energy molecular conformations, spatial arrangements of atoms in a molecule, is a critical task for many different calculations performed in the drug discovery and optimization process. Numerous specialized equivariant networks have been designed to generate molecular conformations from 2D molecular graphs. Recently, non-equivariant transformer models have emerged as a viable alternative due to their capability to scale to improve generalization. However, the concern has been that non-equivariant models require a large model size to compensate the lack of equivariant bias. In this paper, we demonstrate that a well-chosen positional encoding effectively addresses these size limitations. A standard transformer model incorporating relative positional encoding for molecular graphs when scaled to 25 million parameters surpasses the current state-of-the-art non-equivariant base model with 64 million parameters on the GEOM-DRUGS benchmark. We implemented relative positional encoding as a negative attention bias that linearly increases with the shortest path distances between graph nodes at varying slopes for different attention heads, similar to ALiBi, a widely adopted relative positional encoding technique in the NLP domain. This architecture has the potential to serve as a foundation for a novel class of generative models for molecular conformations.
zh
[AI-73] Iterative Quantum Feature Maps
【速读】：该论文试图解决在真实量子硬件上部署深度量子特征映射（Quantum Feature Maps, QFMs）所面临的电路噪声和硬件限制问题，以及变分量子算法在梯度估计中的计算瓶颈问题。其解决方案的关键在于提出迭代量子特征映射（Iterative Quantum Feature Maps, IQFMs），该框架通过迭代连接浅层QFMs与经典计算的增强权重，构建深度架构，并结合对比学习和逐层训练机制，有效降低量子运行时间并缓解噪声引起的性能退化。
链接: https://arxiv.org/abs/2506.19461
作者: Nasa Matsumoto,Quoc Hoan Tran,Koki Chinzei,Yasuhiro Endo,Hirotaka Oshima
机构: 未知
类目: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
备注: 13 pages, 12 figures
Abstract:Quantum machine learning models that leverage quantum circuits as quantum feature maps (QFMs) are recognized for their enhanced expressive power in learning tasks. Such models have demonstrated rigorous end-to-end quantum speedups for specific families of classification problems. However, deploying deep QFMs on real quantum hardware remains challenging due to circuit noise and hardware constraints. Additionally, variational quantum algorithms often suffer from computational bottlenecks, particularly in accurate gradient estimation, which significantly increases quantum resource demands during training. We propose Iterative Quantum Feature Maps (IQFMs), a hybrid quantum-classical framework that constructs a deep architecture by iteratively connecting shallow QFMs with classically computed augmentation weights. By incorporating contrastive learning and a layer-wise training mechanism, IQFMs effectively reduces quantum runtime and mitigates noise-induced degradation. In tasks involving noisy quantum data, numerical experiments show that IQFMs outperforms quantum convolutional neural networks, without requiring the optimization of variational quantum parameters. Even for a typical classical image classification benchmark, a carefully designed IQFMs achieves performance comparable to that of classical neural networks. This framework presents a promising path to address current limitations and harness the full potential of quantum-enhanced machine learning.
zh
[AI-74] From High-SNR Radar Signal to ECG: A Transfer Learning Model with Cardio-Focusing Algorithm for Scenarios with Limited Data
【速读】：该论文试图解决在数据稀缺的新场景下，基于雷达信号的心电图（ECG）恢复性能受限的问题。其解决方案的关键在于提出一种心血管聚焦与跟踪（CFT）算法，以精确追踪心脏位置从而高效获取高质量雷达信号，并引入一种迁移学习模型（RFcardi），通过利用心脏特征的内在稀疏性，从雷达信号中提取与心脏相关的信息，仅需少量同步的雷达-ECG对即可微调预训练模型实现有效的ECG恢复。
链接: https://arxiv.org/abs/2506.19358
作者: Yuanyuan Zhang,Haocheng Zhao,Sijie Xiong,Rui Yang,Eng Gee Lim,Yutao Yue
机构: 未知
类目: ignal Processing (eess.SP); Artificial Intelligence (cs.AI)
备注:
Abstract:Electrocardiogram (ECG), as a crucial find-grained cardiac feature, has been successfully recovered from radar signals in the literature, but the performance heavily relies on the high-quality radar signal and numerous radar-ECG pairs for training, restricting the applications in new scenarios due to data scarcity. Therefore, this work will focus on radar-based ECG recovery in new scenarios with limited data and propose a cardio-focusing and -tracking (CFT) algorithm to precisely track the cardiac location to ensure an efficient acquisition of high-quality radar signals. Furthermore, a transfer learning model (RFcardi) is proposed to extract cardio-related information from the radar signal without ECG ground truth based on the intrinsic sparsity of cardiac features, and only a few synchronous radar-ECG pairs are required to fine-tune the pre-trained model for the ECG recovery. The experimental results reveal that the proposed CFT can dynamically identify the cardiac location, and the RFcardi model can effectively generate faithful ECG recoveries after using a small number of radar-ECG pairs for training. The code and dataset are available after the publication.
zh
[AI-75] Statistical Inference for Optimal Transport Maps: Recent Advances and Perspectives
【速读】：该论文旨在解决在最优运输（Optimal Transport, OT）框架下，如何基于样本估计最优运输映射并建立其极限定理的问题。其解决方案的关键在于通过统计推断方法，分析从底层分布中抽取的样本，从而获得对最优运输映射的准确估计，并推导其渐近性质，为实际应用提供可靠的统计工具。
链接: https://arxiv.org/abs/2506.19025
作者: Sivaraman Balakrishnan,Tudor Manole,Larry Wasserman
机构: 未知
类目: atistics Theory (math.ST); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Methodology (stat.ME); Machine Learning (stat.ML)
备注: 36 pages, 1 figure
Abstract:In many applications of optimal transport (OT), the object of primary interest is the optimal transport map. This map rearranges mass from one probability distribution to another in the most efficient way possible by minimizing a specified cost. In this paper we review recent advances in estimating and developing limit theorems for the OT map, using samples from the underlying distributions. We also review parallel lines of work that establish similar results for special cases and variants of the basic OT setup. We conclude with a discussion of key directions for future research with the goal of providing practitioners with reliable inferential tools.
zh
[AI-76] ccDNAMamba: A Pre-Trained Model for Ultra-Long eccDNA Sequence Analysis ICML2025
【速读】：该论文旨在解决现有预训练模型无法支持全长度环形染色体外DNA（extrachromosomal circular DNA, eccDNA）下游分析的问题，以及现有基因组模型在单核苷酸分辨率限制和二次注意力机制效率低下方面的不足。其解决方案的关键在于提出eccDNAMamba，这是首个专为环形DNA序列设计的双向状态空间编码器，通过前向和反向传递实现全上下文表征学习，并采用新颖的增强策略保留环形结构，从而在保持线性时间复杂度的同时，有效处理长达200 Kbp的序列。
链接: https://arxiv.org/abs/2506.18940
作者: Zhenke Liu,Jien Li,Ziqi Zhang
机构: 未知
类目: Genomics (q-bio.GN); Artificial Intelligence (cs.AI)
备注: Accepted by ICML 2025 Generative AI and Biology (GenBio) Workshop
Abstract:Extrachromosomal circular DNA (eccDNA) plays key regulatory roles and contributes to oncogene overexpression in cancer through high-copy amplification and long-range interactions. Despite advances in modeling, no pre-trained models currently support full-length circular eccDNA for downstream analysis. Existing genomic models are either limited to single-nucleotide resolution or hindered by the inefficiency of the quadratic attention mechanism. Here, we introduce eccDNAMamba, the first bidirectional state-space encoder tailored for circular DNA sequences. It combines forward and reverse passes for full-context representation learning with linear-time complexity, and preserves circular structure through a novel augmentation strategy. Tested on two real-world datasets, eccDNAMamba achieves strong classification performance and scales to sequences up to 200 Kbp, offering a robust and efficient framework for modeling circular genomes. Our codes are available at this https URL.
zh
[AI-77] Which Consciousness Can Be Artificialized? Local Percept-Perceiver Phenomenon for the Existence of Machine Consciousness
【速读】：该论文试图解决机器是否可能具备意识的问题，特别是从还原论角度探讨机器中是否存在认识论意义上的意识。其解决方案的关键在于提出了一种局部感知-感知者现象的新范式，并基于该模型构建了一个集合论形式化框架，通过引用策梅洛-弗兰克尔集合论证明了机器意识的存在性。
链接: https://arxiv.org/abs/2506.18935
作者: Shri Lal Raghudev Ram Singh
机构: 未知
类目: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI)
备注: Paper accepted for the 18th Annual AGI Conference, AGI-2025, Reykjavik, Iceland, August 10-13, 2025
Abstract:This paper presents a novel paradigm of the local percept-perceiver phenomenon to formalize certain observations in neuroscientific theories of consciousness. Using this model, a set-theoretic formalism is developed for artificial systems, and the existence of machine consciousness is proved by invoking Zermelo-Fraenkel set theory. The article argues for the possibility of a reductionist form of epistemic consciousness within machines.
zh
[AI-78] Automatic Depression Assessment using Machine Learning: A Comprehensive Survey
【速读】：该论文试图解决传统抑郁症评估方法中存在的主观性、耗时性及资源不足等问题，以及现有自动抑郁症评估（ADA）研究中对多模态人类行为的综合回顾与分析不足的问题。其解决方案的关键在于系统性地总结与分析跨多种模态（如大脑活动、语言表达及非语言的音频/面部/身体行为）的抑郁相关人类行为，并对基于机器学习的ADA方法进行最新且全面的综述，以揭示其在学习抑郁线索方面的特点与局限性。
链接: https://arxiv.org/abs/2506.18915
作者: Siyang Song,Yupeng Huo,Shiqing Tang,Jiaee Cheong,Rui Gao,Michel Valstar,Hatice Gunes
机构: 未知
类目: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
备注:
Abstract:Depression is a common mental illness across current human society. Traditional depression assessment relying on inventories and interviews with psychologists frequently suffer from subjective diagnosis results, slow and expensive diagnosis process as well as lack of human resources. Since there is a solid evidence that depression is reflected by various human internal brain activities and external expressive behaviours, early traditional machine learning (ML) and advanced deep learning (DL) models have been widely explored for human behaviour-based automatic depression assessment (ADA) since 2012. However, recent ADA surveys typically only focus on a limited number of human behaviour modalities. Despite being used as a theoretical basis for developing ADA approaches, existing ADA surveys lack a comprehensive review and summary of multi-modal depression-related human behaviours. To bridge this gap, this paper specifically summarises depression-related human behaviours across a range of modalities (e.g. the human brain, verbal language and non-verbal audio/facial/body behaviours). We focus on conducting an up-to-date and comprehensive survey of ML-based ADA approaches for learning depression cues from these behaviours as well as discussing and comparing their distinctive features and limitations. In addition, we also review existing ADA competitions and datasets, identify and discuss the main challenges and opportunities to provide further research directions for future ADA researchers.
zh
机器学习
[LG-0] Machine Learning with Privacy for Protected Attributes
链接: https://arxiv.org/abs/2506.19836
作者: Saeed Mahloujifar,Chuan Guo,G. Edward Suh,Kamalika Chaudhuri
类目: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
*备注:
Abstract:Differential privacy (DP) has become the standard for private data analysis. Certain machine learning applications only require privacy protection for specific protected attributes. Using naive variants of differential privacy in such use cases can result in unnecessary degradation of utility. In this work, we refine the definition of DP to create a more general and flexible framework that we call feature differential privacy (FDP). Our definition is simulation-based and allows for both addition/removal and replacement variants of privacy, and can handle arbitrary and adaptive separation of protected and non-protected features. We prove the properties of FDP, such as adaptive composition, and demonstrate its implications for limiting attribute inference attacks. We also propose a modification of the standard DP-SGD algorithm that satisfies FDP while leveraging desirable properties such as amplification via sub-sampling. We apply our framework to various machine learning tasks and show that it can significantly improve the utility of DP-trained models when public features are available. For example, we train diffusion models on the AFHQ dataset of animal faces and observe a drastic improvement in FID compared to DP, from 286.7 to 101.9 at \epsilon=8 , assuming that the blurred version of a training image is available as a public feature. Overall, our work provides a new approach to private data analysis that can help reduce the utility cost of DP while still providing strong privacy guarantees.
[LG-1] Curating art exhibitions using machine learning
链接: https://arxiv.org/abs/2506.19813
作者: Eurico Covas
类目: Machine Learning (cs.LG)
*备注:
Abstract:Art curatorship has always been mostly the subjective work of human experts, who, with extensive knowledge of many and diverse artworks, select a few of those to present in communal spaces, spaces that evolved into what we now call art galleries. There are no hard and fast set of rules on how to select these artworks, given a theme which either is presented to the art curator or constructed by her/him. Here we present a series of artificial models – a total of four related models – based on machine learning techniques (a subset of artificial intelligence) that attempt to learn from existing exhibitions which have been curated by human experts, in order to be able to do similar curatorship work. We focus exclusively on the last 25 years of past exhibitions at the Metropolitan Museum of Art in New York, due to the quality of the data available and the physical and time limitations of our research. Our four artificial intelligence models achieve a reasonable ability at imitating these various curators responsible for all those exhibitions, with various degrees of precision and curatorial coherence. In particular, we can conclude two key insights: first, that there is sufficient information in these exhibitions to construct an artificial intelligence model that replicates past exhibitions with an accuracy well above random choices; second, that using feature engineering and carefully designing the architecture of modest size models can make them as good as those using the so-called large language models such as GPT in a brute force approach. We also believe, based on small attempts to use the models in out-of-sample experiments, that given more much more data, it should be possible for these kinds of artificial intelligence agents to be closer and closer to the aesthetic and curatorial judgment of human art curators.
[LG-2] Ambiguous Online Learning
链接: https://arxiv.org/abs/2506.19810
作者: Vanessa Kosoy
类目: Machine Learning (cs.LG)
*备注:
Abstract:We propose a new variant of online learning that we call “ambiguous online learning”. In this setting, the learner is allowed to produce multiple predicted labels. Such an “ambiguous prediction” is considered correct when at least one of the labels is correct, and none of the labels are “predictably wrong”. The definition of “predictably wrong” comes from a hypothesis class in which hypotheses are also multi-valued. Thus, a prediction is “predictably wrong” if it’s not allowed by the (unknown) true hypothesis. In particular, this setting is natural in the context of multivalued dynamical systems, recommendation algorithms and lossless compression. It is also strongly related to so-called “apple tasting”. We show that in this setting, there is a trichotomy of mistake bounds: up to logarithmic factors, any hypothesis class has an optimal mistake bound of either Theta(1), Theta(sqrt(N)) or N.
[LG-3] Convolution-weighting method for the physics-informed neural network: A Primal-Dual Optimization Perspective
链接: https://arxiv.org/abs/2506.19805
作者: Chenhao Si,Ming Yan
类目: Machine Learning (cs.LG)
*备注: 18 pages, 12 figures
Abstract:Physics-informed neural networks (PINNs) are extensively employed to solve partial differential equations (PDEs) by ensuring that the outputs and gradients of deep learning models adhere to the governing equations. However, constrained by computational limitations, PINNs are typically optimized using a finite set of points, which poses significant challenges in guaranteeing their convergence and accuracy. In this study, we proposed a new weighting scheme that will adaptively change the weights to the loss functions from isolated points to their continuous neighborhood regions. The empirical results show that our weighting scheme can reduce the relative L^2 errors to a lower value.
[LG-4] Multi-Preference Lambda-weighted Listwise DPO for Dynamic Preference Alignment AAAI2026
链接: https://arxiv.org/abs/2506.19780
作者: Yuhui Sun(University of Alberta),Xiyao Wang(University of Toronto),Zixi Li(Zhejiang University),Jinman Zhao(University of Toronto)
类目: Machine Learning (cs.LG)
*备注: 10 pages, 4 figures, appendix included. To appear in Proceedings of AAAI 2026. Code: this https URL
Abstract:While large-scale unsupervised language models (LMs) capture broad world knowledge and reasoning capabilities, steering their behavior toward desired objectives remains challenging due to the lack of explicit supervision. Existing alignment techniques, such as reinforcement learning from human feedback (RLHF), rely on training a reward model and performing reinforcement learning to align with human preferences. However, RLHF is often computationally intensive, unstable, and sensitive to hyperparameters. To address these limitations, Direct Preference Optimization (DPO) was introduced as a lightweight and stable alternative, enabling direct alignment of language models with pairwise preference data via classification loss. However, DPO and its extensions generally assume a single static preference distribution, limiting flexibility in multi-objective or dynamic alignment settings. In this paper, we propose a novel framework: Multi-Preference Lambda-weighted Listwise DPO, which extends DPO to incorporate multiple human preference dimensions (e.g., helpfulness, harmlessness, informativeness) and enables dynamic interpolation through a controllable simplex-weighted formulation. Our method supports both listwise preference feedback and flexible alignment across varying user intents without re-training. Empirical and theoretical analysis demonstrates that our method is as effective as traditional DPO on static objectives while offering greater generality and adaptability for real-world deployment. Comments: 10 pages, 4 figures, appendix included. To appear in Proceedings of AAAI 2026. Code: this https URL Subjects: Machine Learning (cs.LG) ACMclasses: I.2.6; I.2.7; I.5.1 Cite as: arXiv:2506.19780 [cs.LG] (or arXiv:2506.19780v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2506.19780 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-5] On the necessity of adaptive regularisation:Optimal anytime online learning on boldsymbolell_p-balls
链接: https://arxiv.org/abs/2506.19752
作者: Emmeran Johnson,David Martínez-Rubio,Ciara Pike-Burke,Patrick Rebeschini
类目: Machine Learning (cs.LG)
*备注:
Abstract:We study online convex optimization on \ell_p -balls in \mathbbR^d for p 2 . While always sub-linear, the optimal regret exhibits a shift between the high-dimensional setting ( d T ), when the dimension d is greater than the time horizon T and the low-dimensional setting ( d \leq T ). We show that Follow-the-Regularised-Leader (FTRL) with time-varying regularisation which is adaptive to the dimension regime is anytime optimal for all dimension regimes. Motivated by this, we ask whether it is possible to obtain anytime optimality of FTRL with fixed non-adaptive regularisation. Our main result establishes that for separable regularisers, adaptivity in the regulariser is necessary, and that any fixed regulariser will be sub-optimal in one of the two dimension regimes. Finally, we provide lower bounds which rule out sub-linear regret bounds for the linear bandit problem in sufficiently high-dimension for all \ell_p -balls with p \geq 1 .
[LG-6] DRIFT: Data Reduction via Informative Feature Transformation- Generalization Begins Before Deep Learning starts
链接: https://arxiv.org/abs/2506.19734
作者: Ben Keslaki
类目: Machine Learning (cs.LG)
*备注:
Abstract:Modern deep learning architectures excel at optimization, but only after the data has entered the network. The true bottleneck lies in preparing the right input: minimal, salient, and structured in a way that reflects the essential patterns of the data. We propose DRIFT (Data Reduction via Informative Feature Transformation), a novel preprocessing technique inspired by vibrational analysis in physical systems, to identify and extract the most resonant modes of input data prior to training. Unlike traditional models that attempt to learn amidst both signal and noise, DRIFT mimics physics perception by emphasizing informative features while discarding irrelevant elements. The result is a more compact and interpretable representation that enhances training stability and generalization performance. In DRIFT, images are projected onto a low-dimensional basis formed by spatial vibration mode shapes of plates, offering a physically grounded feature set. This enables neural networks to operate with drastically fewer input dimensions (~ 50 features on MNIST and less than 100 on CIFAR100) while achieving competitive classification accuracy. Extensive experiments across MNIST and CIFAR100 demonstrate DRIFT’s superiority over standard pixel-based models and PCA in terms of training stability, resistance to overfitting, and generalization robustness. Notably, DRIFT displays minimal sensitivity to changes in batch size, network architecture, and image resolution, further establishing it as a resilient and efficient data representation strategy. This work shifts the focus from architecture engineering to input curation and underscores the power of physics-driven data transformations in advancing deep learning performance.
[LG-7] Guidance in the Frequency Domain Enables High-Fidelity Sampling at Low CFG Scales
链接: https://arxiv.org/abs/2506.19713
作者: Seyedmorteza Sadat,Tobias Vontobel,Farnood Salehi,Romann M. Weber
类目: Machine Learning (cs.LG)
*备注:
Abstract:Classifier-free guidance (CFG) has become an essential component of modern conditional diffusion models. Although highly effective in practice, the underlying mechanisms by which CFG enhances quality, detail, and prompt alignment are not fully understood. We present a novel perspective on CFG by analyzing its effects in the frequency domain, showing that low and high frequencies have distinct impacts on generation quality. Specifically, low-frequency guidance governs global structure and condition alignment, while high-frequency guidance mainly enhances visual fidelity. However, applying a uniform scale across all frequencies – as is done in standard CFG – leads to oversaturation and reduced diversity at high scales and degraded visual quality at low scales. Based on these insights, we propose frequency-decoupled guidance (FDG), an effective approach that decomposes CFG into low- and high-frequency components and applies separate guidance strengths to each component. FDG improves image quality at low guidance scales and avoids the drawbacks of high CFG scales by design. Through extensive experiments across multiple datasets and models, we demonstrate that FDG consistently enhances sample fidelity while preserving diversity, leading to improved FID and recall compared to CFG, establishing our method as a plug-and-play alternative to standard classifier-free guidance.
[LG-8] Learning-aided Bigraph Matching Approach to Multi-Crew Restoration of Damaged Power Networks Coupled with Road Transportation Networks
链接: https://arxiv.org/abs/2506.19703
作者: Nathan Maurer,Harshal Kaushik,Roshni Anna Jacob,Jie Zhang,Souma Chowdhury
类目: Machine Learning (cs.LG)
*备注: IDETC 2025
Abstract:The resilience of critical infrastructure networks (CINs) after disruptions, such as those caused by natural hazards, depends on both the speed of restoration and the extent to which operational functionality can be regained. Allocating resources for restoration is a combinatorial optimal planning problem that involves determining which crews will repair specific network nodes and in what order. This paper presents a novel graph-based formulation that merges two interconnected graphs, representing crew and transportation nodes and power grid nodes, into a single heterogeneous graph. To enable efficient planning, graph reinforcement learning (GRL) is integrated with bigraph matching. GRL is utilized to design the incentive function for assigning crews to repair tasks based on the graph-abstracted state of the environment, ensuring generalization across damage scenarios. Two learning techniques are employed: a graph neural network trained using Proximal Policy Optimization and another trained via Neuroevolution. The learned incentive functions inform a bipartite graph that links crews to repair tasks, enabling weighted maximum matching for crew-to-task allocations. An efficient simulation environment that pre-computes optimal node-to-node path plans is used to train the proposed restoration planning methods. An IEEE 8500-bus power distribution test network coupled with a 21 square km transportation network is used as the case study, with scenarios varying in terms of numbers of damaged nodes, depots, and crews. Results demonstrate the approach’s generalizability and scalability across scenarios, with learned policies providing 3-fold better performance than random policies, while also outperforming optimization-based solutions in both computation time (by several orders of magnitude) and power restored.
[LG-9] ReBoot: Encrypted Training of Deep Neural Networks with CKKS Bootstrapping
链接: https://arxiv.org/abs/2506.19693
作者: Alberto Pirillo,Luca Colombo
类目: Machine Learning (cs.LG)
*备注:
Abstract:Growing concerns over data privacy underscore the need for deep learning methods capable of processing sensitive information without compromising confidentiality. Among privacy-enhancing technologies, Homomorphic Encryption (HE) stands out by providing post-quantum cryptographic security and end-to-end data protection, safeguarding data even during computation. While Deep Neural Networks (DNNs) have gained attention in HE settings, their use has largely been restricted to encrypted inference. Prior research on encrypted training has primarily focused on logistic regression or has relied on multi-party computation to enable model fine-tuning. This stems from the substantial computational overhead and algorithmic complexity involved in DNNs training under HE. In this paper, we present ReBoot, the first framework to enable fully encrypted and non-interactive training of DNNs. Built upon the CKKS scheme, ReBoot introduces a novel HE-compliant neural network architecture based on local error signals, specifically designed to minimize multiplicative depth and reduce noise accumulation. ReBoot employs a tailored packing strategy that leverages real-number arithmetic via SIMD operations, significantly lowering both computational and memory overhead. Furthermore, by integrating approximate bootstrapping, ReBoot learning algorithm supports effective training of arbitrarily deep multi-layer perceptrons, making it well-suited for machine learning as-a-service. ReBoot is evaluated on both image recognition and tabular benchmarks, achieving accuracy comparable to 32-bit floating-point plaintext training while enabling fully encrypted training. It improves test accuracy by up to +3.27% over encrypted logistic regression, and up to +6.83% over existing encrypted DNN frameworks, while reducing training latency by up to 8.83x. ReBoot is made available to the scientific community as a public repository.
[LG-10] Leverag ing Lightweight Generators for Memory Efficient Continual Learning
链接: https://arxiv.org/abs/2506.19692
作者: Christiaan Lamers,Ahmed Nabil Belbachir,Thomas Bäck,Niki van Stein
类目: Machine Learning (cs.LG)
*备注:
Abstract:Catastrophic forgetting can be trivially alleviated by keeping all data from previous tasks in memory. Therefore, minimizing the memory footprint while maximizing the amount of relevant information is crucial to the challenge of continual learning. This paper aims to decrease required memory for memory-based continuous learning algorithms. We explore the options of extracting a minimal amount of information, while maximally alleviating forgetting. We propose the usage of lightweight generators based on Singular Value Decomposition to enhance existing continual learning methods, such as A-GEM and Experience Replay. These generators need a minimal amount of memory while being maximally effective. They require no training time, just a single linear-time fitting step, and can capture a distribution effectively from a small number of data samples. Depending on the dataset and network architecture, our results show a significant increase in average accuracy compared to the original methods. Our method shows great potential in minimizing the memory footprint of memory-based continual learning algorithms.
[LG-11] Model Guidance via Robust Feature Attribution
链接: https://arxiv.org/abs/2506.19680
作者: Mihnea Ghitu,Matthew Wicker,Vihari Piratla
类目: Machine Learning (cs.LG)
*备注:
Abstract:Controlling the patterns a model learns is essential to preventing reliance on irrelevant or misleading features. Such reliance on irrelevant features, often called shortcut features, has been observed across domains, including medical imaging and natural language processing, where it may lead to real-world harms. A common mitigation strategy leverages annotations (provided by humans or machines) indicating which features are relevant or irrelevant. These annotations are compared to model explanations, typically in the form of feature salience, and used to guide the loss function during training. Unfortunately, recent works have demonstrated that feature salience methods are unreliable and therefore offer a poor signal to optimize. In this work, we propose a simplified objective that simultaneously optimizes for explanation robustness and mitigation of shortcut learning. Unlike prior objectives with similar aims, we demonstrate theoretically why our approach ought to be more effective. Across a comprehensive series of experiments, we show that our approach consistently reduces test-time misclassifications by 20% compared to state-of-the-art methods. We also extend prior experimental settings to include natural language processing tasks. Additionally, we conduct novel ablations that yield practical insights, including the relative importance of annotation quality over quantity. Code for our method and experiments is available at: this https URL.
[LG-12] Higher-Order Graph Databases
链接: https://arxiv.org/abs/2506.19661
作者: Maciej Besta,Shriram Chandran,Jakub Cudak,Patrick Iff,Marcin Copik,Robert Gerstenberger,Tomasz Szydlo,Jürgen Müller,Torsten Hoefler
类目: Databases (cs.DB); Information Retrieval (cs.IR); Machine Learning (cs.LG); Social and Information Networks (cs.SI)
*备注:
Abstract:Recent advances in graph databases (GDBs) have been driving interest in large-scale analytics, yet current systems fail to support higher-order (HO) interactions beyond first-order (one-hop) relations, which are crucial for tasks such as subgraph counting, polyadic modeling, and HO graph learning. We address this by introducing a new class of systems, higher-order graph databases (HO-GDBs) that use lifting and lowering paradigms to seamlessly extend traditional GDBs with HO. We provide a theoretical analysis of OLTP and OLAP queries, ensuring correctness, scalability, and ACID compliance. We implement a lightweight, modular, and parallelizable HO-GDB prototype that offers native support for hypergraphs, node-tuples, subgraphs, and other HO structures under a unified API. The prototype scales to large HO OLTP OLAP workloads and shows how HO improves analytical tasks, for example enhancing accuracy of graph neural networks within a GDB by 44%. Our work ensures low latency and high query throughput, and generalizes both ACID-compliant and eventually consistent systems.
[LG-13] nsor-Parallelism with Partially Synchronized Activations
链接: https://arxiv.org/abs/2506.19645
作者: Itay Lamprecht,Asaf Karnieli,Yair Hanani,Niv Giladi,Daniel Soudry
类目: Machine Learning (cs.LG)
*备注:
Abstract:Training and inference of Large Language Models (LLMs) with tensor-parallelism requires substantial communication to synchronize activations. Our findings suggest that with a few minor adjustments to current practices, LLMs can be trained without fully synchronizing activations, reducing bandwidth demands. We name this “Communication-Aware Architecture for Tensor-parallelism” (CAAT-Net). We train 1B and 7B parameter CAAT-Net models, with a 50% reduction in tensor-parallel communication and no significant drop in pretraining accuracy. Furthermore, we demonstrate how CAAT-Net accelerates both training and inference workloads.
[LG-14] Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Model
链接: https://arxiv.org/abs/2506.19643
作者: Shuncheng He,Hongchang Zhang,Jianzhun Shao,Yuhang Jiang,Xiangyang Ji
类目: Machine Learning (cs.LG)
*备注:
Abstract:Offline reinforcement learning (RL) recently gains growing interests from RL researchers. However, the performance of offline RL suffers from the out-of-distribution problem, which can be corrected by feedback in online RL. Previous offline RL research focuses on restricting the offline algorithm in in-distribution even in-sample action sampling. In contrast, fewer work pays attention to the influence of the batch data. In this paper, we first build a bridge over the batch data and the performance of offline RL algorithms theoretically, from the perspective of model-based offline RL optimization. We draw a conclusion that, with mild assumptions, the distance between the state-action pair distribution generated by the behavioural policy and the distribution generated by the optimal policy, accounts for the performance gap between the policy learned by model-based offline RL and the optimal policy. Secondly, we reveal that in task-agnostic settings, a series of policies trained by unsupervised RL can minimize the worst-case regret in the performance gap. Inspired by the theoretical conclusions, UDG (Unsupervised Data Generation) is devised to generate data and select proper data for offline training under tasks-agnostic settings. Empirical results demonstrate that UDG can outperform supervised data generation on solving unknown tasks.
[LG-15] Scaling Up Unbiased Search-based Symbolic Regression
链接: https://arxiv.org/abs/2506.19626
作者: Paul Kahlmeyer,Joachim Giesen,Michael Habeck,Henrik Voigt
类目: Machine Learning (cs.LG)
*备注:
Abstract:In a regression task, a function is learned from labeled data to predict the labels at new data points. The goal is to achieve small prediction errors. In symbolic regression, the goal is more ambitious, namely, to learn an interpretable function that makes small prediction errors. This additional goal largely rules out the standard approach used in regression, that is, reducing the learning problem to learning parameters of an expansion of basis functions by optimization. Instead, symbolic regression methods search for a good solution in a space of symbolic expressions. To cope with the typically vast search space, most symbolic regression methods make implicit, or sometimes even explicit, assumptions about its structure. Here, we argue that the only obvious structure of the search space is that it contains small expressions, that is, expressions that can be decomposed into a few subexpressions. We show that systematically searching spaces of small expressions finds solutions that are more accurate and more robust against noise than those obtained by state-of-the-art symbolic regression methods. In particular, systematic search outperforms state-of-the-art symbolic regressors in terms of its ability to recover the true underlying symbolic expressions on established benchmark data sets.
[LG-16] Beyond Static Models: Hypernetworks for Adaptive and Generalizable Forecasting in Complex Parametric Dynamical Systems
链接: https://arxiv.org/abs/2506.19609
作者: Pantelis R. Vlachas,Konstantinos Vlachas,Eleni Chatzi
类目: Machine Learning (cs.LG); Chaotic Dynamics (nlin.CD); Computational Physics (physics.comp-ph)
*备注:
Abstract:Dynamical systems play a key role in modeling, forecasting, and decision-making across a wide range of scientific domains. However, variations in system parameters, also referred to as parametric variability, can lead to drastically different model behavior and output, posing challenges for constructing models that generalize across parameter regimes. In this work, we introduce the Parametric Hypernetwork for Learning Interpolated Networks (PHLieNet), a framework that simultaneously learns: (a) a global mapping from the parameter space to a nonlinear embedding and (b) a mapping from the inferred embedding to the weights of a dynamics propagation network. The learned embedding serves as a latent representation that modulates a base network, termed the hypernetwork, enabling it to generate the weights of a target network responsible for forecasting the system’s state evolution conditioned on the previous time history. By interpolating in the space of models rather than observations, PHLieNet facilitates smooth transitions across parameterized system behaviors, enabling a unified model that captures the dynamic behavior across a broad range of system parameterizations. The performance of the proposed technique is validated in a series of dynamical systems with respect to its ability to extrapolate in time and interpolate and extrapolate in the parameter space, i.e., generalize to dynamics that were unseen during training. In all cases, our approach outperforms or matches state-of-the-art baselines in both short-term forecast accuracy and in capturing long-term dynamical features, such as attractor statistics.
[LG-17] raining Flexible Models of Genetic Variant Effects from Functional Annotations using Accelerated Linear Algebra ICML2025
链接: https://arxiv.org/abs/2506.19598
作者: Alan N. Amin,Andres Potapczynski,Andrew Gordon Wilson
类目: Machine Learning (cs.LG); Populations and Evolution (q-bio.PE)
*备注: For example: ICML 2025. Code available at: this https URL
Abstract:To understand how genetic variants in human genomes manifest in phenotypes – traits like height or diseases like asthma – geneticists have sequenced and measured hundreds of thousands of individuals. Geneticists use this data to build models that predict how a genetic variant impacts phenotype given genomic features of the variant, like DNA accessibility or the presence of nearby DNA-bound proteins. As more data and features become available, one might expect predictive models to improve. Unfortunately, training these models is bottlenecked by the need to solve expensive linear algebra problems because variants in the genome are correlated with nearby variants, requiring inversion of large matrices. Previous methods have therefore been restricted to fitting small models, and fitting simplified summary statistics, rather than the full likelihood of the statistical model. In this paper, we leverage modern fast linear algebra techniques to develop DeepWAS (Deep genome Wide Association Studies), a method to train large and flexible neural network predictive models to optimize likelihood. Notably, we find that larger models only improve performance when using our full likelihood approach; when trained by fitting traditional summary statistics, larger models perform no better than small ones. We find larger models trained on more features make better predictions, potentially improving disease predictions and therapeutic target identification.
[LG-18] ConStellaration: A dataset of QI-like stellarator plasma boundaries and optimization benchmarks
链接: https://arxiv.org/abs/2506.19583
作者: Santiago A. Cadena,Andrea Merlo,Emanuel Laude,Alexander Bauer,Atul Agrawal,Maria Pascu,Marija Savtchouk,Enrico Guiraud,Lukas Bonauer,Stuart Hudson,Markus Kaiser
类目: Machine Learning (cs.LG)
*备注:
Abstract:Stellarators are magnetic confinement devices under active development to deliver steady-state carbon-free fusion energy. Their design involves a high-dimensional, constrained optimization problem that requires expensive physics simulations and significant domain expertise. Recent advances in plasma physics and open-source tools have made stellarator optimization more accessible. However, broader community progress is currently bottlenecked by the lack of standardized optimization problems with strong baselines and datasets that enable data-driven approaches, particularly for quasi-isodynamic (QI) stellarator configurations, considered as a promising path to commercial fusion due to their inherent resilience to current-driven disruptions. Here, we release an open dataset of diverse QI-like stellarator plasma boundary shapes, paired with their ideal magnetohydrodynamic (MHD) equilibria and performance metrics. We generated this dataset by sampling a variety of QI fields and optimizing corresponding stellarator plasma boundaries. We introduce three optimization benchmarks of increasing complexity: (1) a single-objective geometric optimization problem, (2) a “simple-to-build” QI stellarator, and (3) a multi-objective ideal-MHD stable QI stellarator that investigates trade-offs between compactness and coil simplicity. For every benchmark, we provide reference code, evaluation scripts, and strong baselines based on classical optimization techniques. Finally, we show how learned models trained on our dataset can efficiently generate novel, feasible configurations without querying expensive physics oracles. By openly releasing the dataset along with benchmark problems and baselines, we aim to lower the entry barrier for optimization and machine learning researchers to engage in stellarator design and to accelerate cross-disciplinary progress toward bringing fusion energy to the grid.
[LG-19] Discovering Symmetries of ODEs by Symbolic Regression
链接: https://arxiv.org/abs/2506.19550
作者: Paul Kahlmeyer,Niklas Merk,Joachim Giesen
类目: Machine Learning (cs.LG)
*备注:
Abstract:Solving systems of ordinary differential equations (ODEs) is essential when it comes to understanding the behavior of dynamical systems. Yet, automated solving remains challenging, in particular for nonlinear systems. Computer algebra systems (CASs) provide support for solving ODEs by first simplifying them, in particular through the use of Lie point symmetries. Finding these symmetries is, however, itself a difficult problem for CASs. Recent works in symbolic regression have shown promising results for recovering symbolic expressions from data. Here, we adapt search-based symbolic regression to the task of finding generators of Lie point symmetries. With this approach, we can find symmetries of ODEs that existing CASs cannot find.
[LG-20] Overtuning in Hyperparameter Optimization
链接: https://arxiv.org/abs/2506.19540
作者: Lennart Schneider,Bernd Bischl,Matthias Feurer
类目: Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注: Accepted at the Fourth Conference on Automated Machine Learning (Methods Track). 43 pages, 9 tables, 14 figures
Abstract:Hyperparameter optimization (HPO) aims to identify an optimal hyperparameter configuration (HPC) such that the resulting model generalizes well to unseen data. As the expected generalization error cannot be optimized directly, it is estimated with a resampling strategy, such as holdout or cross-validation. This approach implicitly assumes that minimizing the validation error leads to improved generalization. However, since validation error estimates are inherently stochastic and depend on the resampling strategy, a natural question arises: Can excessive optimization of the validation error lead to overfitting at the HPO level, akin to overfitting in model training based on empirical risk minimization? In this paper, we investigate this phenomenon, which we term overtuning, a form of overfitting specific to HPO. Despite its practical relevance, overtuning has received limited attention in the HPO and AutoML literature. We provide a formal definition of overtuning and distinguish it from related concepts such as meta-overfitting. We then conduct a large-scale reanalysis of HPO benchmark data to assess the prevalence and severity of overtuning. Our results show that overtuning is more common than previously assumed, typically mild but occasionally severe. In approximately 10% of cases, overtuning leads to the selection of a seemingly optimal HPC with worse generalization error than the default or first configuration tried. We further analyze how factors such as performance metric, resampling strategy, dataset size, learning algorithm, and HPO method affect overtuning and discuss mitigation strategies. Our results highlight the need to raise awareness of overtuning, particularly in the small-data regime, indicating that further mitigation strategies should be studied.
[LG-21] Dimension Reduction for Symbolic Regression
链接: https://arxiv.org/abs/2506.19537
作者: Paul Kahlmeyer,Markus Fischer,Joachim Giesen
类目: Machine Learning (cs.LG)
*备注:
Abstract:Solutions of symbolic regression problems are expressions that are composed of input variables and operators from a finite set of function symbols. One measure for evaluating symbolic regression algorithms is their ability to recover formulae, up to symbolic equivalence, from finite samples. Not unexpectedly, the recovery problem becomes harder when the formula gets more complex, that is, when the number of variables and operators gets larger. Variables in naturally occurring symbolic formulas often appear only in fixed combinations. This can be exploited in symbolic regression by substituting one new variable for the combination, effectively reducing the number of variables. However, finding valid substitutions is challenging. Here, we address this challenge by searching over the expression space of small substitutions and testing for validity. The validity test is reduced to a test of functional dependence. The resulting iterative dimension reduction procedure can be used with any symbolic regression approach. We show that it reliably identifies valid substitutions and significantly boosts the performance of different types of state-of-the-art symbolic regression algorithms.
[LG-22] COLUR: Confidence-Oriented Learning Unlearning and Relearning with Noisy-Label Data for Model Restoration and Refinement IJCAI2025
链接: https://arxiv.org/abs/2506.19496
作者: Zhihao Sui,Liang Hu,Jian Cao,Usman Naseem,Zhongyuan Lai,Qi Zhang
类目: Machine Learning (cs.LG)
*备注: IJCAI 2025
Abstract:Large deep learning models have achieved significant success in various tasks. However, the performance of a model can significantly degrade if it is needed to train on datasets with noisy labels with misleading or ambiguous information. To date, there are limited investigations on how to restore performance when model degradation has been incurred by noisy label data. Inspired by the ``forgetting mechanism’’ in neuroscience, which enables accelerating the relearning of correct knowledge by unlearning the wrong knowledge, we propose a robust model restoration and refinement (MRR) framework COLUR, namely Confidence-Oriented Learning, Unlearning and Relearning. Specifically, we implement COLUR with an efficient co-training architecture to unlearn the influence of label noise, and then refine model confidence on each label for relearning. Extensive experiments are conducted on four real datasets and all evaluation results show that COLUR consistently outperforms other SOTA methods after MRR.
[LG-23] ADDQ: Adaptive Distributional Double Q-Learning
链接: https://arxiv.org/abs/2506.19478
作者: Leif Döring,Benedikt Wille,Maximilian Birr,Mihail Bîrsan,Martin Slowik
类目: Machine Learning (cs.LG)
*备注:
Abstract:Bias problems in the estimation of Q -values are a well-known obstacle that slows down convergence of Q -learning and actor-critic methods. One of the reasons of the success of modern RL algorithms is partially a direct or indirect overestimation reduction mechanism. We propose an easy to implement method built on top of distributional reinforcement learning (DRL) algorithms to deal with the overestimation in a locally adaptive way. Our framework is simple to implement, existing distributional algorithms can be improved with a few lines of code. We provide theoretical evidence and use double Q -learning to show how to include locally adaptive overestimation control in existing algorithms. Experiments are provided for tabular, Atari, and MuJoCo environments.
[LG-24] Center of Gravity-Guided Focusing Influence Mechanism for Multi-Agent Reinforcement Learning
链接: https://arxiv.org/abs/2506.19417
作者: Yisak Park,Sunwoo Lee,Seungyul Han
类目: Machine Learning (cs.LG); Multiagent Systems (cs.MA)
*备注: 9 technical page followed by references and appendix
Abstract:Cooperative multi-agent reinforcement learning (MARL) under sparse rewards presents a fundamental challenge due to limited exploration and insufficient coordinated attention among agents. In this work, we propose the Focusing Influence Mechanism (FIM), a novel framework that enhances cooperation by directing agent influence toward task-critical elements, referred to as Center of Gravity (CoG) state dimensions, inspired by Clausewitz’s military theory. FIM consists of three core components: (1) identifying CoG state dimensions based on their stability under agent behavior, (2) designing counterfactual intrinsic rewards to promote meaningful influence on these dimensions, and (3) encouraging persistent and synchronized focus through eligibility-trace-based credit accumulation. These mechanisms enable agents to induce more targeted and effective state transitions, facilitating robust cooperation even in extremely sparse reward settings. Empirical evaluations across diverse MARL benchmarks demonstrate that the proposed FIM significantly improves cooperative performance compared to baselines.
[LG-25] Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operators ICML2025
链接: https://arxiv.org/abs/2506.19396
作者: Shanda Li,Shinjae Yoo,Yiming Yang
类目: Machine Learning (cs.LG)
*备注: ICML 2025
Abstract:Fourier Neural Operators (FNOs) offer a principled approach for solving complex partial differential equations (PDEs). However, scaling them to handle more complex PDEs requires increasing the number of Fourier modes, which significantly expands the number of model parameters and makes hyperparameter tuning computationally impractical. To address this, we introduce \mu Transfer-FNO, a zero-shot hyperparameter transfer technique that enables optimal configurations, tuned on smaller FNOs, to be directly applied to billion-parameter FNOs without additional tuning. Building on the Maximal Update Parametrization ( \mu P) framework, we mathematically derive a parametrization scheme that facilitates the transfer of optimal hyperparameters across models with different numbers of Fourier modes in FNOs, which is validated through extensive experiments on various PDEs. Our empirical study shows that Transfer-FNO reduces computational cost for tuning hyperparameters on large FNOs while maintaining or improving accuracy.
[LG-26] Deep Electromagnetic Structure Design Under Limited Evaluation Budgets ICML2025
链接: https://arxiv.org/abs/2506.19384
作者: Shijian Zheng,Fangxiao Jin,Shuhai Zhang,Quan Xue,Mingkui Tan
类目: Machine Learning (cs.LG); Signal Processing (eess.SP); Computational Physics (physics.comp-ph)
*备注: ICML 2025 (accepted)
Abstract:Electromagnetic structure (EMS) design plays a critical role in developing advanced antennas and materials, but remains challenging due to high-dimensional design spaces and expensive evaluations. While existing methods commonly employ high-quality predictors or generators to alleviate evaluations, they are often data-intensive and struggle with real-world scale and budget constraints. To address this, we propose a novel method called Progressive Quadtree-based Search (PQS). Rather than exhaustively exploring the high-dimensional space, PQS converts the conventional image-like layout into a quadtree-based hierarchical representation, enabling a progressive search from global patterns to local details. Furthermore, to lessen reliance on highly accurate predictors, we introduce a consistency-driven sample selection mechanism. This mechanism quantifies the reliability of predictions, balancing exploitation and exploration when selecting candidate designs. We evaluate PQS on two real-world engineering tasks, i.e., Dual-layer Frequency Selective Surface and High-gain Antenna. Experimental results show that our method can achieve satisfactory designs under limited computational budgets, outperforming baseline methods. In particular, compared to generative approaches, it cuts evaluation costs by 75-85%, effectively saving 20.27-38.80 days of product designing cycle.
[LG-27] Explainable Artificial Intelligence Credit Risk Assessment using Machine Learning
链接: https://arxiv.org/abs/2506.19383
作者: Shreya,Harsh Pathak
类目: Machine Learning (cs.LG)
*备注: 15 pages, 8 Figures, 3 Tables
Abstract:This paper presents an intelligent and transparent AI-driven system for Credit Risk Assessment using three state-of-the-art ensemble machine learning models combined with Explainable AI (XAI) techniques. The system leverages XGBoost, LightGBM, and Random Forest algorithms for predictive analysis of loan default risks, addressing the challenges of model interpretability using SHAP and LIME. Preprocessing steps include custom imputation, one-hot encoding, and standardization. Class imbalance is managed using SMOTE, and hyperparameter tuning is performed with GridSearchCV. The model is evaluated on multiple performance metrics including ROC-AUC, precision, recall, and F1-score. LightGBM emerges as the most business-optimal model with the highest accuracy and best trade off between approval and default rates. Furthermore, the system generates applicant-specific XAI visual reports and business impact summaries to ensure transparent decision-making.
[LG-28] Path Learning with Trajectory Advantage Regression
链接: https://arxiv.org/abs/2506.19375
作者: Kohei Miyaguchi
类目: Machine Learning (cs.LG)
*备注:
Abstract:In this paper, we propose trajectory advantage regression, a method of offline path learning and path attribution based on reinforcement learning. The proposed method can be used to solve path optimization problems while algorithmically only solving a regression problem.
[LG-29] WebGuard:Interpretable Malicious URL Detection via Bidirectional Fusion of HTML Subgraphs and Multi-Scale Convolutional BERT
链接: https://arxiv.org/abs/2506.19356
作者: Ye Tian,Zhang Yumin,Yifan Jia,Jianguo Sun,Yanbin Wang
类目: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
*备注:
Abstract:URL+HTML feature fusion shows promise for robust malicious URL detection, since attacker artifacts persist in DOM structures. However, prior work suffers from four critical shortcomings: (1) incomplete URL modeling, failing to jointly capture lexical patterns and semantic context; (2) HTML graph sparsity, where threat-indicative nodes (e.g., obfuscated scripts) are isolated amid benign content, causing signal dilution during graph aggregation; (3) unidirectional analysis, ignoring URL-HTML feature bidirectional interaction; and (4) opaque decisions, lacking attribution to malicious DOM components. To address these challenges, we present WebGuard++, a detection framework with 4 novel components: 1) Cross-scale URL Encoder: Hierarchically learns local-to-global and coarse to fine URL features based on Transformer network with dynamic convolution. 2) Subgraph-aware HTML Encoder: Decomposes DOM graphs into interpretable substructures, amplifying sparse threat signals via Hierarchical feature fusion. 3) Bidirectional Coupling Module: Aligns URL and HTML embeddings through cross-modal contrastive learning, optimizing inter-modal consistency and intra-modal specificity. 4) Voting Module: Localizes malicious regions through consensus voting on malicious subgraph predictions. Experiments show WebGuard++ achieves significant improvements over state-of-the-art baselines, achieving 1.1x-7.9x higher TPR at fixed FPR of 0.001 and 0.0001 across both datasets.
[LG-30] Contrastive Cross-Modal Learning for Infusing Chest X-ray Knowledge into ECGs
链接: https://arxiv.org/abs/2506.19329
作者: Vineet Punyamoorty,Aditya Malusare,Vaneet Aggarwal
类目: Machine Learning (cs.LG)
*备注:
Abstract:Modern diagnostic workflows are increasingly multimodal, integrating diverse data sources such as medical images, structured records, and physiological time series. Among these, electrocardiograms (ECGs) and chest X-rays (CXRs) are two of the most widely used modalities for cardiac assessment. While CXRs provide rich diagnostic information, ECGs are more accessible and can support scalable early warning systems. In this work, we propose CroMoTEX, a novel contrastive learning-based framework that leverages chest X-rays during training to learn clinically informative ECG representations for multiple cardiac-related pathologies: cardiomegaly, pleural effusion, and edema. Our method aligns ECG and CXR representations using a novel supervised cross-modal contrastive objective with adaptive hard negative weighting, enabling robust and task-relevant feature learning. At test time, CroMoTEX relies solely on ECG input, allowing scalable deployment in real-world settings where CXRs may be unavailable. Evaluated on the large-scale MIMIC-IV-ECG and MIMIC-CXR datasets, CroMoTEX outperforms baselines across all three pathologies, achieving up to 78.31 AUROC on edema. Our code is available at this http URL.
[LG-31] Adversarial Attacks on Deep Learning-Based False Data Injection Detection in Differential Relays
链接: https://arxiv.org/abs/2506.19302
作者: Ahmad Mohammad Saber,Aditi Maheshwari,Amr Youssef,Deepa Kundur
类目: Machine Learning (cs.LG)
*备注:
Abstract:The application of Deep Learning-based Schemes (DLSs) for detecting False Data Injection Attacks (FDIAs) in smart grids has attracted significant attention. This paper demonstrates that adversarial attacks, carefully crafted FDIAs, can evade existing DLSs used for FDIA detection in Line Current Differential Relays (LCDRs). We propose a novel adversarial attack framework, utilizing the Fast Gradient Sign Method, which exploits DLS vulnerabilities by introducing small perturbations to LCDR remote measurements, leading to misclassification of the FDIA as a legitimate fault while also triggering the LCDR to trip. We evaluate the robustness of multiple deep learning models, including multi-layer perceptrons, convolutional neural networks, long short-term memory networks, and residual networks, under adversarial conditions. Our experimental results demonstrate that while these models perform well, they exhibit high degrees of vulnerability to adversarial attacks. For some models, the adversarial attack success rate exceeds 99.7%. To address this threat, we introduce adversarial training as a proactive defense mechanism, significantly enhancing the models’ ability to withstand adversarial FDIAs without compromising fault detection accuracy. Our results highlight the significant threat posed by adversarial attacks to DLS-based FDIA detection, underscore the necessity for robust cybersecurity measures in smart grids, and demonstrate the effectiveness of adversarial training in enhancing model robustness against adversarial FDIAs.
[LG-32] he Effect of Depth on the Expressivity of Deep Linear State-Space Models
链接: https://arxiv.org/abs/2506.19296
作者: Zeyu Bao,Penghao Yu,Haotian Jiang,Qianxiao Li
类目: Machine Learning (cs.LG)
*备注:
Abstract:Deep state-space models (SSMs) have gained increasing popularity in sequence modelling. While there are numerous theoretical investigations of shallow SSMs, how the depth of the SSM affects its expressiveness remains a crucial problem. In this paper, we systematically investigate the role of depth and width in deep linear SSMs, aiming to characterize how they influence the expressive capacity of the architecture. First, we rigorously prove that in the absence of parameter constraints, increasing depth and increasing width are generally equivalent, provided that the parameter count remains within the same order of magnitude. However, under the assumption that the parameter norms are constrained, the effects of depth and width differ significantly. We show that a shallow linear SSM with large parameter norms can be represented by a deep linear SSM with smaller norms using a constructive method. In particular, this demonstrates that deep SSMs are more capable of representing targets with large norms than shallow SSMs under norm constraints. Finally, we derive upper bounds on the minimal depth required for a deep linear SSM to represent a given shallow linear SSM under constrained parameter norms. We also validate our theoretical results with numerical experiments
[LG-33] Efficient Extreme Operating Condition Search for Online Relay Setting Calculation in Renewable Power Systems Based on Parallel Graph Neural Network
链接: https://arxiv.org/abs/2506.19289
作者: Yan Li,Zengli Yang,Youhuai Wang,Jing Wang,Xiaoyu Han,Jingyu Wang,Dongyuan Shi
类目: Machine Learning (cs.LG)
*备注:
Abstract:The Extreme Operating Conditions Search (EOCS) problem is one of the key problems in relay setting calculation, which is used to ensure that the setting values of protection relays can adapt to the changing operating conditions of power systems over a period of time after deployment. The high penetration of renewable energy and the wide application of inverter-based resources make the operating conditions of renewable power systems more volatile, which urges the adoption of the online relay setting calculation strategy. However, the computation speed of existing EOCS methods based on local enumeration, heuristic algorithms, and mathematical programming cannot meet the efficiency requirement of online relay setting calculation. To reduce the time overhead, this paper, for the first time, proposes an efficient deep learning-based EOCS method suitable for online relay setting calculation. First, the power system information is formulated as four layers, i.e., a component parameter layer, a topological connection layer, an electrical distance layer, and a graph distance layer, which are fed into a parallel graph neural network (PGNN) model for feature extraction. Then, the four feature layers corresponding to each node are spliced and stretched, and then fed into the decision network to predict the extreme operating condition of the system. Finally, the proposed PGNN method is validated on the modified IEEE 39-bus and 118-bus test systems, where some of the synchronous generators are replaced by renewable generation units. The nonlinear fault characteristics of renewables are fully considered when computing fault currents. The experiment results show that the proposed PGNN method achieves higher accuracy than the existing methods in solving the EOCS problem. Meanwhile, it also provides greater improvements in online computation time.
[LG-34] A Batch-Insensitive Dynamic GNN Approach to Address Temporal Discontinuity in Graph Streams
链接: https://arxiv.org/abs/2506.19282
作者: Yang Zhou,Xiaoning Ren
类目: Machine Learning (cs.LG); Graphics (cs.GR)
*备注: 8pages, 5figures
Abstract:In dynamic graphs, preserving temporal continuity is critical. However, Memory-based Dynamic Graph Neural Networks (MDGNNs) trained with large batches often disrupt event sequences, leading to temporal information loss. This discontinuity not only deteriorates temporal modeling but also hinders optimization by increasing the difficulty of parameter convergence. Our theoretical study quantifies this through a Lipschitz upper bound, showing that large batch sizes enlarge the parameter search space. In response, we propose BADGNN, a novel batch-agnostic framework consisting of two core components: (1) Temporal Lipschitz Regularization (TLR) to control parameter search space expansion, and (2) Adaptive Attention Adjustment (A3) to alleviate attention distortion induced by both regularization and batching. Empirical results on three benchmark datasets show that BADGNN maintains strong performance while enabling significantly larger batch sizes and faster training compared to TGN. Our code is available at Code: this https URL.
[LG-35] Robust OOD Graph Learning via Mean Constraints and Noise Reduction
链接: https://arxiv.org/abs/2506.19281
作者: Yang Zhou,Xiaoning Ren
类目: Machine Learning (cs.LG)
*备注: 8 pages, 6 figures
Abstract:Graph Out-of-Distribution (OOD) classification often suffers from sharp performance drops, particularly under category imbalance and structural noise. This work tackles two pressing challenges in this context: (1) the underperformance of minority classes due to skewed label distributions, and (2) their heightened sensitivity to structural noise in graph data. To address these problems, we propose two complementary solutions. First, Constrained Mean Optimization (CMO) improves minority class robustness by encouraging similarity-based instance aggregation under worst-case conditions. Second, the Neighbor-Aware Noise Reweighting (NNR) mechanism assigns dynamic weights to training samples based on local structural consistency, mitigating noise influence. We provide theoretical justification for our methods, and validate their effectiveness with extensive experiments on both synthetic and real-world datasets, showing significant improvements in Graph OOD generalization and classification accuracy. The code for our method is available at: this https URL.
[LG-36] Stabilizing PDE–ML Coupled System
链接: https://arxiv.org/abs/2506.19274
作者: Saad Qadeer,Panos Stinis,Hui Wan
类目: Numerical Analysis (math.NA); Machine Learning (cs.LG)
*备注:
Abstract:A long-standing obstacle in the use of machine-learnt surrogates with larger PDE systems is the onset of instabilities when solved numerically. Efforts towards ameliorating these have mostly concentrated on improving the accuracy of the surrogates or imbuing them with additional structure, and have garnered limited success. In this article, we study a prototype problem and draw insights that can help with more complex systems. In particular, we focus on a viscous Burgers’-ML system and, after identifying the cause of the instabilities, prescribe strategies to stabilize the coupled system. To improve the accuracy of the stabilized system, we next explore methods based on the Mori–Zwanzig formalism.
[LG-37] HARPT: A Corpus for Analyzing Consumers Trust and Privacy Concerns in Mobile Health Apps CIKM’25
链接: https://arxiv.org/abs/2506.19268
作者: Timoteo Kelly,Abdulkadir Korkmaz,Samuel Mallet,Connor Souders,Sadra Aliakbarpour,Praveen Rao
类目: Human-Computer Interaction (cs.HC); Cryptography and Security (cs.CR); Emerging Technologies (cs.ET); Machine Learning (cs.LG)
*备注: Under review at The 34th ACM International Conference on Information and Knowledge Management (CIKM’25)
Abstract:We present HARPT, a large-scale annotated corpus of mobile health app store reviews aimed at advancing research in user privacy and trust. The dataset comprises over 480,000 user reviews labeled into seven categories that capture critical aspects of trust in applications, trust in providers and privacy concerns. Creating HARPT required addressing multiple complexities, such as defining a nuanced label schema, isolating relevant content from large volumes of noisy data, and designing an annotation strategy that balanced scalability with accuracy. This strategy integrated rule-based filtering, iterative manual labeling with review, targeted data augmentation, and weak supervision using transformer-based classifiers to accelerate coverage. In parallel, a carefully curated subset of 7,000 reviews was manually annotated to support model development and evaluation. We benchmark a broad range of classification models, demonstrating that strong performance is achievable and providing a baseline for future research. HARPT is released as a public resource to support work in health informatics, cybersecurity, and natural language processing.
[LG-38] Network Structures as an Attack Surface: Topology-Based Privacy Leakage in Federated Learning
链接: https://arxiv.org/abs/2506.19260
作者: Murtaza Rangwala,Richard O. Sinnott,Rajkumar Buyya
类目: Cryptography and Security (cs.CR); Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)
*备注: 13 pages, 7 figures, 5 tables. Data from the experiments and source code can be found here: this https URL
Abstract:Federated learning systems increasingly rely on diverse network topologies to address scalability and organizational constraints. While existing privacy research focuses on gradient-based attacks, the privacy implications of network topology knowledge remain critically understudied. We conduct the first comprehensive analysis of topology-based privacy leakage across realistic adversarial knowledge scenarios, demonstrating that adversaries with varying degrees of structural knowledge can infer sensitive data distribution patterns even under strong differential privacy guarantees. Through systematic evaluation of 4,720 attack instances, we analyze six distinct adversarial knowledge scenarios: complete topology knowledge and five partial knowledge configurations reflecting real-world deployment constraints. We propose three complementary attack vectors: communication pattern analysis, parameter magnitude profiling, and structural position correlation, achieving success rates of 84.1%, 65.0%, and 47.2% under complete knowledge conditions. Critically, we find that 80% of realistic partial knowledge scenarios maintain attack effectiveness above security thresholds, with certain partial knowledge configurations achieving performance superior to the baseline complete knowledge scenario. To address these vulnerabilities, we propose and empirically validate structural noise injection as a complementary defense mechanism across 808 configurations, demonstrating up to 51.4% additional attack reduction when properly layered with existing privacy techniques. These results establish that network topology represents a fundamental privacy vulnerability in federated learning systems while providing practical pathways for mitigation through topology-aware defense mechanisms.
[LG-39] Inference-Time Reward Hacking in Large Language Models ICML2025
链接: https://arxiv.org/abs/2506.19248
作者: Hadi Khalaf,Claudio Mayrink Verdun,Alex Oesterling,Himabindu Lakkaraju,Flavio du Pin Calmon
类目: Machine Learning (cs.LG)
*备注: Accepted to ICML 2025 Workshop on Models of Human Feedback for AI Alignment
Abstract:A common paradigm to improve the performance of large language models is optimizing for a reward model. Reward models assign a numerical score to LLM outputs indicating, for example, which response would likely be preferred by a user or is most aligned with safety goals. However, reward models are never perfect. They inevitably function as proxies for complex desiderata such as correctness, helpfulness, and safety. By overoptimizing for a misspecified reward, we can subvert intended alignment goals and reduce overall performance – a phenomenon commonly referred to as reward hacking. In this work, we characterize reward hacking in inference-time alignment and demonstrate when and how we can mitigate it by hedging on the proxy reward. We study this phenomenon under Best-of- n (BoN) and Soft-Best-of- n (SBoN), and we introduce Best-of-Poisson (BoP) that provides an efficient, near-exact approximation of the optimal reward-KL divergence policy at inference time. We show that the characteristic pattern of hacking as observed in practice (where the true reward first increases before declining) is an inevitable property of a broad class of inference-time mechanisms, including BoN and BoP. To counter this effect, hedging offers a tactical choice to avoid placing undue confidence in high but potentially misleading proxy reward signals. We introduce HedgeTune, an efficient algorithm to find the optimal inference-time parameter and avoid reward hacking. We demonstrate through experiments that hedging mitigates reward hacking and achieves superior distortion-reward tradeoffs with minimal computational overhead.
[LG-40] Behavioral Anomaly Detection in Distributed Systems via Federated Contrastive Learning
链接: https://arxiv.org/abs/2506.19246
作者: Renzi Meng,Heyi Wang,Yumeng Sun,Qiyuan Wu,Lian Lian,Renhan Zhang
类目: Machine Learning (cs.LG)
*备注:
Abstract:This paper addresses the increasingly prominent problem of anomaly detection in distributed systems. It proposes a detection method based on federated contrastive learning. The goal is to overcome the limitations of traditional centralized approaches in terms of data privacy, node heterogeneity, and anomaly pattern recognition. The proposed method combines the distributed collaborative modeling capabilities of federated learning with the feature discrimination enhancement of contrastive learning. It builds embedding representations on local nodes and constructs positive and negative sample pairs to guide the model in learning a more discriminative feature space. Without exposing raw data, the method optimizes a global model through a federated aggregation strategy. Specifically, the method uses an encoder to represent local behavior data in high-dimensional space. This includes system logs, operational metrics, and system calls. The model is trained using both contrastive loss and classification loss to improve its ability to detect fine-grained anomaly patterns. The method is evaluated under multiple typical attack types. It is also tested in a simulated real-time data stream scenario to examine its responsiveness. Experimental results show that the proposed method outperforms existing approaches across multiple performance metrics. It demonstrates strong detection accuracy and adaptability, effectively addressing complex anomalies in distributed environments. Through careful design of key modules and optimization of the training mechanism, the proposed method achieves a balance between privacy preservation and detection performance. It offers a feasible technical path for intelligent security management in distributed systems.
[LG-41] Universal kernels via harmonic analysis on Riemannian symmetric spaces
链接: https://arxiv.org/abs/2506.19245
作者: Franziskus Steinert,Salem Said,Cyrus Mostajeran
类目: Machine Learning (cs.LG); Differential Geometry (math.DG)
*备注:
Abstract:The universality properties of kernels characterize the class of functions that can be approximated in the associated reproducing kernel Hilbert space and are of fundamental importance in the theoretical underpinning of kernel methods in machine learning. In this work, we establish fundamental tools for investigating universality properties of kernels in Riemannian symmetric spaces, thereby extending the study of this important topic to kernels in non-Euclidean domains. Moreover, we use the developed tools to prove the universality of several recent examples from the literature on positive definite kernels defined on Riemannian symmetric spaces, thus providing theoretical justification for their use in applications involving manifold-valued data.
[LG-42] High precision PINNs in unbounded domains: application to singularity formulation in PDEs
链接: https://arxiv.org/abs/2506.19243
作者: Yixuan Wang,Ziming Liu,Zongyi Li,Anima Anandkumar,Thomas Y. Hou
类目: Machine Learning (cs.LG); Numerical Analysis (math.NA)
*备注:
Abstract:We investigate the high-precision training of Physics-Informed Neural Networks (PINNs) in unbounded domains, with a special focus on applications to singularity formulation in PDEs. We propose a modularized approach and study the choices of neural network ansatz, sampling strategy, and optimization algorithm. When combined with rigorous computer-assisted proofs and PDE analysis, the numerical solutions identified by PINNs, provided they are of high precision, can serve as a powerful tool for studying singularities in PDEs. For 1D Burgers equation, our framework can lead to a solution with very high precision, and for the 2D Boussinesq equation, which is directly related to the singularity formulation in 3D Euler and Navier-Stokes equations, we obtain a solution whose loss is 4 digits smaller than that obtained in \citewang2023asymptotic with fewer training steps. We also discuss potential directions for pushing towards machine precision for higher-dimensional problems.
[LG-43] Simulation of a closed-loop dc-dc converter using a physics-informed neural network-based model
链接: https://arxiv.org/abs/2506.19178
作者: Marc-Antoine Coulombe,Maxime Berger,Antoine Lesage-Landry
类目: ystems and Control (eess.SY); Machine Learning (cs.LG)
*备注: 8 pages, 6 figures, Paper submitted to the International Conference on Power Systems Transients (IPST2025) in Guadalajara, Mexico, June 8-12, 2025
Abstract:The growing reliance on power electronics introduces new challenges requiring detailed time-domain analyses with fast and accurate circuit simulation tools. Currently, commercial time-domain simulation software are mainly relying on physics-based methods to simulate power electronics. Recent work showed that data-driven and physics-informed learning methods can increase simulation speed with limited compromise on accuracy, but many challenges remain before deployment in commercial tools can be possible. In this paper, we propose a physics-informed bidirectional long-short term memory neural network (BiLSTM-PINN) model to simulate the time-domain response of a closed-loop dc-dc boost converter for various operating points, parameters, and perturbations. A physics-informed fully-connected neural network (FCNN) and a BiLSTM are also trained to establish a comparison. The three methods are then compared using step-response tests to assess their performance and limitations in terms of accuracy. The results show that the BiLSTM-PINN and BiLSTM models outperform the FCNN model by more than 9 and 4.5 times, respectively, in terms of median RMSE. Their standard deviation values are more than 2.6 and 1.7 smaller than the FCNN’s, making them also more consistent. Those results illustrate that the proposed BiLSTM-PINN is a potential alternative to other physics-based or data-driven methods for power electronics simulations.
[LG-44] Distilling Tool Knowledge into Language Models via Back-Translated Traces ICML2025
链接: https://arxiv.org/abs/2506.19171
作者: Xingyue Huang,Xianglong Hu,Zifeng Ding,Yuan He,Rishabh,Waleed Alzarooni,Ziyu Ye,Wendong Fan,Bailan He,Haige Bo,Changran Hu,Guohao Li
类目: Machine Learning (cs.LG)
*备注: Accepted in Workshop in Multi-Agent Systems in the Era of Foundation Models: Opportunities, Challenges and Futures, ICML 2025
Abstract:Large language models (LLMs) often struggle with mathematical problems that require exact computation or multi-step algebraic reasoning. Tool-integrated reasoning (TIR) offers a promising solution by leveraging external tools such as code interpreters to ensure correctness, but it introduces inference-time dependencies that hinder scalability and deployment. In this work, we propose a new paradigm for distilling tool knowledge into LLMs purely through natural language. We first construct a Solver Agent that solves math problems by interleaving planning, symbolic tool calls, and reflective reasoning. Then, using a back-translation pipeline powered by multiple LLM-based agents, we convert interleaved TIR traces into natural language reasoning traces. A Translator Agent generates explanations for individual tool calls, while a Rephrase Agent merges them into a fluent and globally coherent narrative. Empirically, we show that fine-tuning a small open-source model on these synthesized traces enables it to internalize both tool knowledge and structured reasoning patterns, yielding gains on competition-level math benchmarks without requiring tool access at inference.
[LG-45] GradualDiff-Fed: A Federated Learning Specialized Framework for Large Language Model
链接: https://arxiv.org/abs/2506.19164
作者: Amir Faiyaz,Tara Salman
类目: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC)
*备注:
Abstract:The rapid proliferation of large language models (LLMs) has created an unprecedented demand for fine-tuning models for specialized domains, such as medical science. While federated learning (FL) offers a decentralized and privacy-preserving approach to collaboratively fine-tune LLMs without sharing raw data, it presents significant challenges, particularly in performance and managing large model sizes efficiently. In this paper, we introduce GradualDiff-Fed, an FL framework designed explicitly for LLMs, and their challenge of handling the high parameter size. GradualDiff-Fed reduces communication costs by transmitting only the difference of model weights rather than the entire model during training rounds. Such an approach significantly improves scalability and communication efficiency, making it more feasible to fine-tune LLMs across distributed clients without compromising performance. Our evaluation demonstrates that GradualDiff-Fed achieves performance on par with centralized training while drastically reducing communication overhead. These results highlight the potential of GradualDiff-Fed as an efficient solution for fine-tuning large models from distributed data in privacy-preserving settings without comprising performance.
[LG-46] Command-V: Pasting LLM Behaviors via Activation Profiles
链接: https://arxiv.org/abs/2506.19140
作者: Barry Wang,Avi Schwarzschild,Alexander Robey,Ali Payani,Charles Fleming,Mingjie Sun,Daphne Ippolito
类目: Machine Learning (cs.LG)
*备注:
Abstract:Retrofitting large language models (LLMs) with new behaviors typically requires full finetuning or distillation-costly steps that must be repeated for every architecture. In this work, we introduce Command-V, a backpropagation-free behavior transfer method that copies an existing residual activation adapter from a donor model and pastes its effect into a recipient model. Command-V profiles layer activations on a small prompt set, derives linear converters between corresponding layers, and applies the donor intervention in the recipient’s activation space. This process does not require access to the original training data and needs minimal compute. In three case studies-safety-refusal enhancement, jailbreak facilitation, and automatic chain-of-thought reasoning–Command-V matches or exceeds the performance of direct finetuning while using orders of magnitude less compute. Our code and data are accessible at this https URL.
[LG-47] Local Learning Rules for Out-of-Equilibrium Physical Generative Models
链接: https://arxiv.org/abs/2506.19136
作者: Cyrill Bösch,Geoffrey Roeder,Marc Serra-Garcia,Ryan P. Adams
类目: Machine Learning (cs.LG); Mesoscale and Nanoscale Physics (cond-mat.mes-hall); Emerging Technologies (cs.ET); Neural and Evolutionary Computing (cs.NE)
*备注: 6 pages, 2 figures
Abstract:We show that the out-of-equilibrium driving protocol of score-based generative models (SGMs) can be learned via a local learning rule. The gradient with respect to the parameters of the driving protocol are computed directly from force measurements or from observed system dynamics. As a demonstration, we implement an SGM in a network of driven, nonlinear, overdamped oscillators coupled to a thermal bath. We first apply it to the problem of sampling from a mixture of two Gaussians in 2D. Finally, we train a network of 10x10 oscillators to sample images of 0s and 1s from the MNIST dataset.
[LG-48] Riemannian generative decoder ICML2025
链接: https://arxiv.org/abs/2506.19133
作者: Andreas Bjerregaard,Søren Hauberg,Anders Krogh
类目: Machine Learning (cs.LG); Quantitative Methods (q-bio.QM); Machine Learning (stat.ML)
*备注: GenBio ICML 2025 (Proceedings of the Workshop on Generative AI for Biology at the 42nd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025)
Abstract:Riemannian representation learning typically relies on approximating densities on chosen manifolds. This involves optimizing difficult objectives, potentially harming models. To completely circumvent this issue, we introduce the Riemannian generative decoder which finds manifold-valued maximum likelihood latents with a Riemannian optimizer while training a decoder network. By discarding the encoder, we vastly simplify the manifold constraint compared to current approaches which often only handle few specific manifolds. We validate our approach on three case studies – a synthetic branching diffusion process, human migrations inferred from mitochondrial DNA, and cells undergoing a cell division cycle – each showing that learned representations respect the prescribed geometry and capture intrinsic non-Euclidean structure. Our method requires only a decoder, is compatible with existing architectures, and yields interpretable latent spaces aligned with data geometry.
[LG-49] On the algorithmic construction of deep ReLU networks
链接: https://arxiv.org/abs/2506.19104
作者: Daan Huybrechs
类目: Machine Learning (cs.LG); Numerical Analysis (math.NA)
*备注:
Abstract:It is difficult to describe in mathematical terms what a neural network trained on data represents. On the other hand, there is a growing mathematical understanding of what neural networks are in principle capable of representing. Feedforward neural networks using the ReLU activation function represent continuous and piecewise linear functions and can approximate many others. The study of their expressivity addresses the question: which ones? Contributing to the available answers, we take the perspective of a neural network as an algorithm. In this analogy, a neural network is programmed constructively, rather than trained from data. An interesting example is a sorting algorithm: we explicitly construct a neural network that sorts its inputs exactly, not approximately, and that, in a sense, has optimal computational complexity if the input dimension is large. Such constructed networks may have several billion parameters. We construct and analyze several other examples, both existing and new. We find that, in these examples, neural networks as algorithms are typically recursive and parallel. Compared to conventional algorithms, ReLU networks are restricted by having to be continuous. Moreover, the depth of recursion is limited by the depth of the network, with deep networks having superior properties over shallow ones.
[LG-50] Finetuning a Weather Foundation Model with Lightweight Decoders for Unseen Physical Processes
链接: https://arxiv.org/abs/2506.19088
作者: Fanny Lehmann,Firat Ozdemir,Benedikt Soja,Torsten Hoefler,Siddhartha Mishra,Sebastian Schemm
类目: Machine Learning (cs.LG)
*备注:
Abstract:Recent advances in AI weather forecasting have led to the emergence of so-called “foundation models”, typically defined by expensive pretraining and minimal fine-tuning for downstream tasks. However, in the natural sciences, a desirable foundation model should also encode meaningful statistical relationships between the underlying physical variables. This study evaluates the performance of the state-of-the-art Aurora foundation model in predicting hydrological variables, which were not considered during pretraining. We introduce a lightweight approach using shallow decoders trained on the latent representations of the pretrained model to predict these new variables. As a baseline, we compare this to fine-tuning the full model, which allows further optimization of the latent space while incorporating new variables into both inputs and outputs. The decoder-based approach requires 50% less training time and 35% less memory, while achieving strong accuracy across various hydrological variables and preserving desirable properties of the foundation model, such as autoregressive stability. Notably, decoder accuracy depends on the physical correlation between the new variables and those used during pretraining, indicating that Aurora’s latent space captures meaningful physical relationships. In this sense, we argue that an important quality metric for foundation models in Earth sciences is their ability to be extended to new variables without a full fine-tuning. This provides a new perspective for making foundation models more accessible to communities with limited computational resources, while supporting broader adoption in Earth sciences.
[LG-51] Benchmarking Music Generation Models and Metrics via Human Preference Studies ICASSP2025
链接: https://arxiv.org/abs/2506.19085
作者: Florian Grötschla,Ahmet Solak,Luca A. Lanzendörfer,Roger Wattenhofer
类目: Machine Learning (cs.LG); Sound (cs.SD)
*备注: Accepted at ICASSP 2025
Abstract:Recent advancements have brought generated music closer to human-created compositions, yet evaluating these models remains challenging. While human preference is the gold standard for assessing quality, translating these subjective judgments into objective metrics, particularly for text-audio alignment and music quality, has proven difficult. In this work, we generate 6k songs using 12 state-of-the-art models and conduct a survey of 15k pairwise audio comparisons with 2.5k human participants to evaluate the correlation between human preferences and widely used metrics. To the best of our knowledge, this work is the first to rank current state-of-the-art music generation models and metrics based on human preference. To further the field of subjective metric evaluation, we provide open access to our dataset of generated music and human evaluations.
[LG-52] Which Company Adjustment Matter? Insights from Uplift Modeling on Financial Health
链接: https://arxiv.org/abs/2506.19049
作者: Xinlin Wang,Mats Brorsson
类目: Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG)
*备注:
Abstract:Uplift modeling has achieved significant success in various fields, particularly in online marketing. It is a method that primarily utilizes machine learning and deep learning to estimate individual treatment effects. This paper we apply uplift modeling to analyze the effect of company adjustment on their financial status, and we treat these adjustment as treatments or interventions in this study. Although there have been extensive studies and application regarding binary treatments, multiple treatments, and continuous treatments, company adjustment are often more complex than these scenarios, as they constitute a series of multiple time-dependent actions. The effect estimation of company adjustment needs to take into account not only individual treatment traits but also the temporal order of this series of treatments. This study collects a real-world data set about company financial statements and reported behavior in Luxembourg for the experiments. First, we use two meta-learners and three other well-known uplift models to analyze different company adjustment by simplifying the adjustment as binary treatments. Furthermore, we propose a new uplift modeling framework (MTDnet) to address the time-dependent nature of these adjustment, and the experimental result shows the necessity of considering the timing of these adjustment.
[LG-53] Online Learning for Dynamic Vickrey-Clarke-Groves Mechanism in Sequential Auctions under Unknown Environments
链接: https://arxiv.org/abs/2506.19038
作者: Vincent Leon,S. Rasoul Etesami
类目: Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG); Multiagent Systems (cs.MA); Systems and Control (eess.SY); Optimization and Control (math.OC)
*备注: 16 pages
Abstract:We consider the problem of online dynamic mechanism design for sequential auctions in unknown environments, where the underlying market and, thus, the bidders’ values vary over time as interactions between the seller and the bidders progress. We model the sequential auctions as an infinite-horizon average-reward Markov decision process (MDP), where the transition kernel and reward functions are unknown to the seller. In each round, the seller determines an allocation and a payment for each bidder. Each bidder receives a private reward and submits a sealed bid to the seller. The state, which represents the underlying market, evolves according to an unknown transition kernel and the seller’s allocation policy. Unlike existing works that formulate the problem as a multi-armed bandit model or as an episodic MDP, where the environment resets to an initial state after each round or episode, our paper considers a more realistic and sophisticated setting in which the market continues to evolve without restarting. We first extend the Vickrey-Clarke-Groves (VCG) mechanism, which is known to be efficient, truthful, and individually rational for one-shot static auctions, to sequential auctions, thereby obtaining a dynamic VCG mechanism counterpart that preserves these desired properties. We then focus on the online setting and develop an online reinforcement learning algorithm for the seller to learn the underlying MDP model and implement a mechanism that closely resembles the dynamic VCG mechanism. We show that the learned online mechanism asymptotically converges to a dynamic mechanism that approximately satisfies efficiency, truthfulness, and individual rationality with arbitrarily high probability and achieves guaranteed performance in terms of various notions of regret.
[LG-54] Failure Modes of Time Series Interpretability Algorithms for Critical Care Applications and Potential Solutions
链接: https://arxiv.org/abs/2506.19035
作者: Shashank Yadav,Vignesh Subbian
类目: Machine Learning (cs.LG)
*备注: 13 pages, 10 figures, Accepted at the AMIA Annual Symposium 2025. The final version will appear in the official proceedings
Abstract:Interpretability plays a vital role in aligning and deploying deep learning models in critical care, especially in constantly evolving conditions that influence patient survival. However, common interpretability algorithms face unique challenges when applied to dynamic prediction tasks, where patient trajectories evolve over time. Gradient, Occlusion, and Permutation-based methods often struggle with time-varying target dependency and temporal smoothness. This work systematically analyzes these failure modes and supports learnable mask-based interpretability frameworks as alternatives, which can incorporate temporal continuity and label consistency constraints to learn feature importance over time. Here, we propose that learnable mask-based approaches for dynamic timeseries prediction problems provide more reliable and consistent interpretations for applications in critical care and similar domains.
[LG-55] Automating Traffic Monitoring with SHM Sensor Networks via Vision-Supervised Deep Learning
链接: https://arxiv.org/abs/2506.19023
作者: Hanshuo Wu,Xudong Jian,Christos Lataniotis,Cyprien Hoelzl,Eleni Chatzi,Yves Reuland
类目: Machine Learning (cs.LG)
*备注:
Abstract:Bridges, as critical components of civil infrastructure, are increasingly affected by deterioration, making reliable traffic monitoring essential for assessing their remaining service life. Among operational loads, traffic load plays a pivotal role, and recent advances in deep learning - particularly in computer vision (CV) - have enabled progress toward continuous, automated monitoring. However, CV-based approaches suffer from limitations, including privacy concerns and sensitivity to lighting conditions, while traditional non-vision-based methods often lack flexibility in deployment and validation. To bridge this gap, we propose a fully automated deep-learning pipeline for continuous traffic monitoring using structural health monitoring (SHM) sensor networks. Our approach integrates CV-assisted high-resolution dataset generation with supervised training and inference, leveraging graph neural networks (GNNs) to capture the spatial structure and interdependence of sensor data. By transferring knowledge from CV outputs to SHM sensors, the proposed framework enables sensor networks to achieve comparable accuracy of vision-based systems, with minimal human intervention. Applied to accelerometer and strain gauge data in a real-world case study, the model achieves state-of-the-art performance, with classification accuracies of 99% for light vehicles and 94% for heavy vehicles.
[LG-56] Online high-precision prediction method for injection molding product weight by integrating time series/non-time series mixed features and feature attention mechanism
链接: https://arxiv.org/abs/2506.18950
作者: Maoyuan Li,Sihong Li,Guancheng Shen,Yun Zhang,Huamin Zhou
类目: Machine Learning (cs.LG)
*备注:
Abstract:To address the challenges of untimely detection and online monitoring lag in injection molding quality anomalies, this study proposes a mixed feature attention-artificial neural network (MFA-ANN) model for high-precision online prediction of product weight. By integrating mechanism-based with data-driven analysis, the proposed architecture decouples time series data (e.g., melt flow dynamics, thermal profiles) from non-time series data (e.g., mold features, pressure settings), enabling hierarchical feature extraction. A self-attention mechanism is strategically embedded during cross-domain feature fusion to dynamically calibrate inter-modality feature weights, thereby emphasizing critical determinants of weight variability. The results demonstrate that the MFA-ANN model achieves a RMSE of 0.0281 with 0.5 g weight fluctuation tolerance, outperforming conventional benchmarks: a 25.1% accuracy improvement over non-time series ANN models, 23.0% over LSTM networks, 25.7% over SVR, and 15.6% over RF models, respectively. Ablation studies quantitatively validate the synergistic enhancement derived from the integration of mixed feature modeling (contributing 22.4%) and the attention mechanism (contributing 11.2%), significantly enhancing the model’s adaptability to varying working conditions and its resistance to noise. Moreover, critical sensitivity analyses further reveal that data resolution significantly impacts prediction reliability, low-fidelity sensor inputs degrade performance by 23.8% RMSE compared to high-precision measurements. Overall, this study provides an efficient and reliable solution for the intelligent quality control of injection molding processes.
[LG-57] From Tiny Machine Learning to Tiny Deep Learning: A Survey
链接: https://arxiv.org/abs/2506.18927
作者: Shriyank Somvanshi,Md Monzurul Islam,Gaurab Chhetri,Rohit Chakraborty,Mahmuda Sultana Mimi,Swagat Ahmed Shuvo,Kazi Sifatul Islam,Syed Aaqib Javed,Sharif Ahmed Rafat,Anandi Dutta,Subasish Das
类目: Machine Learning (cs.LG)
*备注:
Abstract:The rapid growth of edge devices has driven the demand for deploying artificial intelligence (AI) at the edge, giving rise to Tiny Machine Learning (TinyML) and its evolving counterpart, Tiny Deep Learning (TinyDL). While TinyML initially focused on enabling simple inference tasks on microcontrollers, the emergence of TinyDL marks a paradigm shift toward deploying deep learning models on severely resource-constrained hardware. This survey presents a comprehensive overview of the transition from TinyML to TinyDL, encompassing architectural innovations, hardware platforms, model optimization techniques, and software toolchains. We analyze state-of-the-art methods in quantization, pruning, and neural architecture search (NAS), and examine hardware trends from MCUs to dedicated neural accelerators. Furthermore, we categorize software deployment frameworks, compilers, and AutoML tools enabling practical on-device learning. Applications across domains such as computer vision, audio recognition, healthcare, and industrial monitoring are reviewed to illustrate the real-world impact of TinyDL. Finally, we identify emerging directions including neuromorphic computing, federated TinyDL, edge-native foundation models, and domain-specific co-design approaches. This survey aims to serve as a foundational resource for researchers and practitioners, offering a holistic view of the ecosystem and laying the groundwork for future advancements in edge AI.
[LG-58] HI-SQL: Optimizing Text-to-SQL Systems through Dynamic Hint Integration
链接: https://arxiv.org/abs/2506.18916
作者: Ganesh Parab,Zishan Ahmad,Dagnachew Birru
类目: Machine Learning (cs.LG); Databases (cs.DB)
*备注:
Abstract:Text-to-SQL generation bridges the gap between natural language and databases, enabling users to query data without requiring SQL expertise. While large language models (LLMs) have significantly advanced the field, challenges remain in handling complex queries that involve multi-table joins, nested conditions, and intricate operations. Existing methods often rely on multi-step pipelines that incur high computational costs, increase latency, and are prone to error propagation. To address these limitations, we propose HI-SQL, a pipeline that incorporates a novel hint generation mechanism utilizing historical query logs to guide SQL generation. By analyzing prior queries, our method generates contextual hints that focus on handling the complexities of multi-table and nested operations. These hints are seamlessly integrated into the SQL generation process, eliminating the need for costly multi-step approaches and reducing reliance on human-crafted prompts. Experimental evaluations on multiple benchmark datasets demonstrate that our approach significantly improves query accuracy of LLM-generated queries while ensuring efficiency in terms of LLM calls and latency, offering a robust and practical solution for enhancing Text-to-SQL systems.
[LG-59] Adaptive Anomaly Detection for Identifying Attacks in Cyber-Physical Systems: A Systematic Literature Review
链接: https://arxiv.org/abs/2411.14278
作者: Pablo Moriano,Steven C. Hespeler,Mingyan Li,Maria Mahbub
类目: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
*备注: 30 pages, 4 figures, 6 tables
Abstract:Modern cyberattacks in cyber-physical systems (CPS) rapidly evolve and cannot be deterred effectively with most current methods which focused on characterizing past threats. Adaptive anomaly detection (AAD) is among the most promising techniques to detect evolving cyberattacks focused on fast data processing and model adaptation. AAD has been researched in the literature extensively; however, to the best of our knowledge, our work is the first systematic literature review (SLR) on the current research within this field. We present a comprehensive SLR, gathering 397 relevant papers and systematically analyzing 65 of them (47 research and 18 survey papers) on AAD in CPS studies from 2013 to 2023 (November). We introduce a novel taxonomy considering attack types, CPS application, learning paradigm, data management, and algorithms. Our analysis indicates, among other findings, that reviewed works focused on a single aspect of adaptation (either data processing or model adaptation) but rarely in both at the same time. We aim to help researchers to advance the state of the art and help practitioners to become familiar with recent progress in this field. We identify the limitations of the state of the art and provide recommendations for future research directions.
[LG-60] Convergence of Mean Shift Algorithms for Large Bandwidths and Simultaneous Accurate Clustering
链接: https://arxiv.org/abs/2506.19837
作者: Susovan Pal,Praneeth Vepakomma
类目: Machine Learning (stat.ML); Machine Learning (cs.LG)
*备注:
Abstract:The mean shift (MS) is a non-parametric, density-based, iterative algorithm that has prominent usage in clustering and image segmentation. A rigorous proof for its convergence in full generality remains unknown. Two significant steps in this direction were taken in the paper \citeGh1, which proved that for \textitsufficiently large bandwidth, the MS algorithm with the Gaussian kernel always converges in any dimension, and also by the same author in \citeGh2, proved that MS always converges in one dimension for kernels with differentiable, strictly decreasing, convex profiles. In the more recent paper \citeYT, they have proved the convergence in more generality,\textit without any restriction on the bandwidth, with the assumption that the KDE f has a continuous Lipschitz gradient on the closure of the convex hull of the trajectory of the iterated sequence of the mode estimate, and also satisfies the Łojasiewicz property there. The main theoretical result of this paper is a generalization of those of \citeGh1, where we show that (1) for\textit sufficiently large bandwidth convergence is guaranteed in any dimension with \textitany radially symmetric and strictly positive definite kernels. The proof uses two alternate characterizations of radially symmetric positive definite smooth kernels by Schoenberg and Bernstein \citeFass, and borrows some steps from the proofs in \citeGh1. Although the authors acknowledge that the result in that paper is more restrictive than that of \citeYT due to the lower bandwidth limit, it uses a different set of assumptions than \citeYT, and the proof technique is different. Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG) Cite as: arXiv:2506.19837 [stat.ML] (or arXiv:2506.19837v1 [stat.ML] for this version) https://doi.org/10.48550/arXiv.2506.19837 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-61] ProxelGen: Generating Proteins as 3D Densities
链接: https://arxiv.org/abs/2506.19820
作者: Felix Faltings,Hannes Stark,Regina Barzilay,Tommi Jaakkola
类目: Biomolecules (q-bio.BM); Machine Learning (cs.LG)
*备注:
Abstract:We develop ProxelGen, a protein structure generative model that operates on 3D densities as opposed to the prevailing 3D point cloud representations. Representing proteins as voxelized densities, or proxels, enables new tasks and conditioning capabilities. We generate proteins encoded as proxels via a 3D CNN-based VAE in conjunction with a diffusion model operating on its latent space. Compared to state-of-the-art models, ProxelGen’s samples achieve higher novelty, better FID scores, and the same level of designability as the training set. ProxelGen’s advantages are demonstrated in a standard motif scaffolding benchmark, and we show how 3D density-based generation allows for more flexible shape conditioning.
[LG-62] A comparative analysis of machine learning algorithms for predicting probabilities of default
链接: https://arxiv.org/abs/2506.19789
作者: Adrian Iulian Cristescu,Matteo Giordano
类目: Applications (stat.AP); Machine Learning (cs.LG)
*备注: 6 pages, 2 tables, to appear in Book of Short Papers - IES 2025
Abstract:Predicting the probability of default (PD) of prospective loans is a critical objective for financial institutions. In recent years, machine learning (ML) algorithms have achieved remarkable success across a wide variety of prediction tasks; yet, they remain relatively underutilised in credit risk analysis. This paper highlights the opportunities that ML algorithms offer to this field by comparing the performance of five predictive models-Random Forests, Decision Trees, XGBoost, Gradient Boosting and AdaBoost-to the predominantly used logistic regression, over a benchmark dataset from Scheule et al. (Credit Risk Analytics: The R Companion). Our findings underscore the strengths and weaknesses of each method, providing valuable insights into the most effective ML algorithms for PD prediction in the context of loan portfolios.
[LG-63] he Shape of Consumer Behavior: A Symbolic and Topological Analysis of Time Series
链接: https://arxiv.org/abs/2506.19759
作者: Pola Bereta,Ioannis Diamantis
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST); Applications (stat.AP)
*备注: 33 pages, 30 figures
Abstract:Understanding temporal patterns in online search behavior is crucial for real-time marketing and trend forecasting. Google Trends offers a rich proxy for public interest, yet the high dimensionality and noise of its time-series data present challenges for effective clustering. This study evaluates three unsupervised clustering approaches, Symbolic Aggregate approXimation (SAX), enhanced SAX (eSAX), and Topological Data Analysis (TDA), applied to 20 Google Trends keywords representing major consumer categories. Our results show that while SAX and eSAX offer fast and interpretable clustering for stable time series, they struggle with volatility and complexity, often producing ambiguous ``catch-all’’ clusters. TDA, by contrast, captures global structural features through persistent homology and achieves more balanced and meaningful groupings. We conclude with practical guidance for using symbolic and topological methods in consumer analytics and suggest that hybrid approaches combining both perspectives hold strong potential for future applications. Comments: 33 pages, 30 figures Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST); Applications (stat.AP) MSC classes: 62H30, 91B42, 91C20, 55N31 ACMclasses: I.5.3; I.5.1; G.3 Cite as: arXiv:2506.19759 [stat.ML] (or arXiv:2506.19759v1 [stat.ML] for this version) https://doi.org/10.48550/arXiv.2506.19759 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-64] Conservative quantum offline model-based optimization
链接: https://arxiv.org/abs/2506.19714
作者: Kristian Sotirov,Annie E. Paine,Savvas Varsamopoulos,Antonio A. Gentile,Osvaldo Simeone
类目: Quantum Physics (quant-ph); Machine Learning (cs.LG); Machine Learning (stat.ML)
*备注: 5 pages, 5 figures, initial version
Abstract:Offline model-based optimization (MBO) refers to the task of optimizing a black-box objective function using only a fixed set of prior input-output data, without any active experimentation. Recent work has introduced quantum extremal learning (QEL), which leverages the expressive power of variational quantum circuits to learn accurate surrogate functions by training on a few data points. However, as widely studied in the classical machine learning literature, predictive models may incorrectly extrapolate objective values in unexplored regions, leading to the selection of overly optimistic solutions. In this paper, we propose integrating QEL with conservative objective models (COM) - a regularization technique aimed at ensuring cautious predictions on out-of-distribution inputs. The resulting hybrid algorithm, COM-QEL, builds on the expressive power of quantum neural networks while safeguarding generalization via conservative modeling. Empirical results on benchmark optimization tasks demonstrate that COM-QEL reliably finds solutions with higher true objective values compared to the original QEL, validating its superiority for offline design problems.
[LG-65] Near-optimal estimates for the ellp-Lipschitz constants of deep random ReLU neural networks
链接: https://arxiv.org/abs/2506.19695
作者: Sjoerd Dirksen,Patrick Finke,Paul Geuchen,Dominik Stöger,Felix Voigtlaender
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Probability (math.PR)
*备注: The introduction will still be expanded with additional references
Abstract:This paper studies the \ell^p -Lipschitz constants of ReLU neural networks \Phi: \mathbbR^d \to \mathbbR with random parameters for p \in [1,\infty] . The distribution of the weights follows a variant of the He initialization and the biases are drawn from symmetric distributions. We derive high probability upper and lower bounds for wide networks that differ at most by a factor that is logarithmic in the network’s width and linear in its depth. In the special case of shallow networks, we obtain matching bounds. Remarkably, the behavior of the \ell^p -Lipschitz constant varies significantly between the regimes p \in [1,2) and p \in [2,\infty] . For p \in [2,\infty] , the \ell^p -Lipschitz constant behaves similarly to \Vert g\Vert_p’ , where g \in \mathbbR^d is a d -dimensional standard Gaussian vector and 1/p + 1/p’ = 1 . In contrast, for p \in [1,2) , the \ell^p -Lipschitz constant aligns more closely to \Vert g \Vert_2 .
[LG-66] Operator Forces For Coarse-Grained Molecular Dynamics
链接: https://arxiv.org/abs/2506.19628
作者: Leon Klein,Atharva Kelkar,Aleksander Durumeric,Yaoyi Chen,Frank Noé
类目: Chemical Physics (physics.chem-ph); Machine Learning (cs.LG); Computational Physics (physics.comp-ph); Machine Learning (stat.ML)
*备注:
Abstract:Coarse-grained (CG) molecular dynamics simulations extend the length and time scale of atomistic simulations by replacing groups of correlated atoms with CG beads. Machine-learned coarse-graining (MLCG) has recently emerged as a promising approach to construct highly accurate force fields for CG molecular dynamics. However, the calibration of MLCG force fields typically hinges on force matching, which demands extensive reference atomistic trajectories with corresponding force labels. In practice, atomistic forces are often not recorded, making traditional force matching infeasible on pre-existing datasets. Recently, noise-based kernels have been introduced to adapt force matching to the low-data regime, including situations in which reference atomistic forces are not present. While this approach produces force fields which recapitulate slow collective motion, it introduces significant local distortions due to the corrupting effects of the noise-based kernel. In this work, we introduce more general kernels based on normalizing flows that substantially reduce these local distortions while preserving global conformational accuracy. We demonstrate our method on small proteins, showing that flow-based kernels can generate high-quality CG forces solely from configurational samples.
[LG-67] Low-Complexity Semantic Packet Aggregation for Token Communication via Lookahead Search
链接: https://arxiv.org/abs/2506.19451
作者: Seunghun Lee,Jihong Park,Jinho Choi,Hyuncheol Park
类目: ignal Processing (eess.SP); Machine Learning (cs.LG)
*备注:
Abstract:Tokens are fundamental processing units of generative AI (GenAI) and large language models (LLMs), and token communication (TC) is essential for enabling remote AI-generate content (AIGC) and wireless LLM applications. Unlike traditional bits, each of which is independently treated, the semantics of each token depends on its surrounding context tokens. This inter-token dependency makes TC vulnerable to outage channels, where the loss of a single token can significantly distort the original message semantics. Motivated by this, this paper focuses on optimizing token packetization to maximize the average token similarity (ATS) between the original and received token messages under outage channels. Due to inter-token dependency, this token grouping problem is combinatorial, with complexity growing exponentially with message length. To address this, we propose a novel framework of semantic packet aggregation with lookahead search (SemPA-Look), built on two core ideas. First, it introduces the residual semantic score (RSS) as a token-level surrogate for the message-level ATS, allowing robust semantic preservation even when a certain token packet is lost. Second, instead of full search, SemPA-Look applies a lookahead search-inspired algorithm that samples intra-packet token candidates without replacement (fixed depth), conditioned on inter-packet token candidates sampled with replacement (fixed width), thereby achieving linear complexity. Experiments on a remote AIGC task with the MS-COCO dataset (text captioned images) demonstrate that SemPA-Look achieves high ATS and LPIPS scores comparable to exhaustive search, while reducing computational complexity by up to 40 \times . Compared to other linear-complexity algorithms such as the genetic algorithm (GA), SemPA-Look achieves 10 \times lower complexity, demonstrating its practicality for remote AIGC and other TC applications.
[LG-68] CAM-NET: An AI Model for Whole Atmosphere with Thermosphere and Ionosphere Extension
链接: https://arxiv.org/abs/2506.19340
作者: Jiahui Hu,Wenjun Dong
类目: pace Physics (physics.space-ph); Machine Learning (cs.LG)
*备注:
Abstract:We present Compressible Atmospheric Model-Network (CAM-NET), an AI model designed to predict neutral atmospheric variables from the Earth’s surface to the ionosphere with high accuracy and computational efficiency. Accurate modeling of the entire atmosphere is critical for understanding the upward propagation of gravity waves, which influence upper-atmospheric dynamics and coupling across atmospheric layers. CAM-NET leverages the Spherical Fourier Neural Operator (SFNO) to capture global-scale atmospheric dynamics while preserving the Earth’s spherical structure. Trained on a decade of datasets from the Whole Atmosphere Community Climate Model with thermosphere and ionosphere eXtension (WACCM-X), CAM-NET demonstrates accuracy comparable to WACCM-X while achieving a speedup of over 1000x in inference time, can provide one year simulation within a few minutes once trained. The model effectively predicts key atmospheric parameters, including zonal and meridional winds, temperature, and time rate of pressure. Inspired by traditional modeling approaches that use external couplers to simulate tracer transport, CAM-NET introduces a modular architecture that explicitly separates tracer prediction from core dynamics. The core backbone of CAM-NET focuses on forecasting primary physical variables (e.g., temperature, wind velocity), while tracer variables are predicted through a lightweight, fine-tuned model. This design allows for efficient adaptation to specific tracer scenarios with minimal computational cost, avoiding the need to retrain the entire model. We have validated this approach on the O^2 tracer, demonstrating strong performance and generalization capabilities.
[LG-69] Rare dense solutions clusters in asymmetric binary perceptrons – local entropy via fully lifted RDT
链接: https://arxiv.org/abs/2506.19276
作者: Mihailo Stojnic
类目: Machine Learning (stat.ML); Disordered Systems and Neural Networks (cond-mat.dis-nn); Information Theory (cs.IT); Machine Learning (cs.LG)
*备注:
Abstract:We study classical asymmetric binary perceptron (ABP) and associated \emphlocal entropy (LE) as potential source of its algorithmic hardness. Isolation of \emphtypical ABP solutions in SAT phase seemingly suggests a universal algorithmic hardness. Paradoxically, efficient algorithms do exist even for constraint densities \alpha fairly close but at a finite distance (\emphcomputational gap) from the capacity. In recent years, existence of rare large dense clusters and magical ability of fast algorithms to find them have been posited as the conceptual resolution of this paradox. Monotonicity or breakdown of the LEs associated with such \emphatypical clusters are predicated to play a key role in their thinning-out or even complete defragmentation. Invention of fully lifted random duality theory (fl RDT) [90,93,94] allows studying random structures \emphtypical features. A large deviation upgrade, sfl LD RDT [96,97], moves things further and enables \emphatypical features characterizations as well. Utilizing the machinery of [96,97] we here develop a generic framework to study LE as an ABP’s atypical feature. Already on the second level of lifting we discover that the LE results are closely matching those obtained through replica methods. For classical zero threshold ABP, we obtain that LE breaks down for \alpha in (0.77,0.78) interval which basically matches \alpha\sim 0.75-0.77 range that currently best ABP solvers can handle and effectively indicates that LE’s behavior might indeed be among key reflections of the ABP’s computational gaps presumable existence. Subjects: Machine Learning (stat.ML); Disordered Systems and Neural Networks (cond-mat.dis-nn); Information Theory (cs.IT); Machine Learning (cs.LG) Cite as: arXiv:2506.19276 [stat.ML] (or arXiv:2506.19276v1 [stat.ML] for this version) https://doi.org/10.48550/arXiv.2506.19276 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
[LG-70] A Qubit-Efficient Hybrid Quantum Encoding Mechanism for Quantum Machine Learning
链接: https://arxiv.org/abs/2506.19275
作者: Hevish Cowlessur,Tansu Alpcan,Chandra Thapa,Seyit Camtepe,Neel Kanth Kundu
类目: Quantum Physics (quant-ph); Machine Learning (cs.LG)
*备注:
Abstract:Efficiently embedding high-dimensional datasets onto noisy and low-qubit quantum systems is a significant barrier to practical Quantum Machine Learning (QML). Approaches such as quantum autoencoders can be constrained by current hardware capabilities and may exhibit vulnerabilities to reconstruction attacks due to their invertibility. We propose Quantum Principal Geodesic Analysis (qPGA), a novel, non-invertible method for dimensionality reduction and qubit-efficient encoding. Executed classically, qPGA leverages Riemannian geometry to project data onto the unit Hilbert sphere, generating outputs inherently suitable for quantum amplitude encoding. This technique preserves the neighborhood structure of high-dimensional datasets within a compact latent space, significantly reducing qubit requirements for amplitude encoding. We derive theoretical bounds quantifying qubit requirements for effective encoding onto noisy systems. Empirical results on MNIST, Fashion-MNIST, and CIFAR-10 show that qPGA preserves local structure more effectively than both quantum and hybrid autoencoders. Additionally, we demonstrate that qPGA enhances resistance to reconstruction attacks due to its non-invertible nature. In downstream QML classification tasks, qPGA can achieve over 99% accuracy and F1-score on MNIST and Fashion-MNIST, outperforming quantum-dependent baselines. Initial tests on real hardware and noisy simulators confirm its potential for noise-resilient performance, offering a scalable solution for advancing QML applications.
[LG-71] Continuous-variable Quantum Diffusion Model for State Generation and Restoration
链接: https://arxiv.org/abs/2506.19270
作者: Haitao Huang,Chuangtao Chen,Qinglin Zhao
类目: Quantum Physics (quant-ph); Machine Learning (cs.LG)
*备注: 15+3 pages, 14 figures, 7 tables
Abstract:The generation and preservation of complex quantum states against environmental noise are paramount challenges in advancing continuous-variable (CV) quantum information processing. This paper introduces a novel framework based on continuous-variable quantum diffusion principles, synergizing them with CV quantum neural networks (CVQNNs) to address these dual challenges. For the task of state generation, our Continuous-Variable Quantum Diffusion Generative model (CVQD-G) employs a physically driven forward diffusion process using a thermal loss channel, which is then inverted by a learnable, parameter-efficient backward denoising process based on a CVQNN with time-embedding. This framework’s capability is further extended for state recovery by the Continuous-Variable Quantum Diffusion Restoration model (CVQD-R), a specialized variant designed to restore quantum states, particularly coherent states with unknown parameters, from thermal degradation. Extensive numerical simulations validate these dual capabilities, demonstrating the high-fidelity generation of diverse Gaussian (coherent, squeezed) and non-Gaussian (Fock, cat) states, typically with fidelities exceeding 99%, and confirming the model’s ability to robustly restore corrupted states. Furthermore, a comprehensive complexity analysis reveals favorable training and inference costs, highlighting the framework’s efficiency, scalability, and its potential as a robust tool for quantum state engineering and noise mitigation in realistic CV quantum systems.
[LG-72] Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality
链接: https://arxiv.org/abs/2506.19144
作者: Kyeongwon Lee,Lizhen Lin,Jaewoo Park,Seonghyun Jeong
类目: Machine Learning (stat.ML); Machine Learning (cs.LG)
*备注:
Abstract:This work establishes that sparse Bayesian neural networks achieve optimal posterior contraction rates over anisotropic Besov spaces and their hierarchical compositions. These structures reflect the intrinsic dimensionality of the underlying function, thereby mitigating the curse of dimensionality. Our analysis shows that Bayesian neural networks equipped with either sparse or continuous shrinkage priors attain the optimal rates which are dependent on the intrinsic dimension of the true structures. Moreover, we show that these priors enable rate adaptation, allowing the posterior to contract at the optimal rate even when the smoothness level of the true function is unknown. The proposed framework accommodates a broad class of functions, including additive and multiplicative Besov functions as special cases. These results advance the theoretical foundations of Bayesian neural networks and provide rigorous justification for their practical effectiveness in high-dimensional, structured estimation problems.
[LG-73] EEG Foundation Challenge: From Cross-Task to Cross-Subject EEG Decoding NEURIPS
链接: https://arxiv.org/abs/2506.19141
作者: Bruno Aristimunha,Dung Truong,Pierre Guetschel,Seyed Yahya Shirazi,Isabelle Guyon,Alexandre R. Franco,Michael P. Milham,Aviv Dotan,Scott Makeig,Alexandre Gramfort,Jean-Remi King,Marie-Constance Corsi,Pedro A. Valdés-Sosa,Amit Majumdar,Alan Evans,Terrence J Sejnowski,Oren Shriki,Sylvain Chevallier,Arnaud Delorme
类目: ignal Processing (eess.SP); Machine Learning (cs.LG)
*备注: Approved at Neurips Competition track. webpage: this https URL
Abstract:Current electroencephalogram (EEG) decoding models are typically trained on small numbers of subjects performing a single task. Here, we introduce a large-scale, code-submission-based competition comprising two challenges. First, the Transfer Challenge asks participants to build and test a model that can zero-shot decode new tasks and new subjects from their EEG data. Second, the Psychopathology factor prediction Challenge asks participants to infer subject measures of mental health from EEG data. For this, we use an unprecedented, multi-terabyte dataset of high-density EEG signals (128 channels) recorded from over 3,000 child to young adult subjects engaged in multiple active and passive tasks. We provide several tunable neural network baselines for each of these two challenges, including a simple network and demographic-based regression models. Developing models that generalise across tasks and individuals will pave the way for ML network architectures capable of adapting to EEG data collected from diverse tasks and individuals. Similarly, predicting mental health-relevant personality trait values from EEG might identify objective biomarkers useful for clinical diagnosis and design of personalised treatment for psychological conditions. Ultimately, the advances spurred by this challenge could contribute to the development of computational psychiatry and useful neurotechnology, and contribute to breakthroughs in both fundamental neuroscience and applied clinical research.
[LG-74] First-Order Sparse Convex Optimization: Better Rates with Sparse Updates
链接: https://arxiv.org/abs/2506.19075
作者: Dan Garber
类目: Optimization and Control (math.OC); Machine Learning (cs.LG)
*备注:
Abstract:In was recently established that for convex optimization problems with a sparse optimal solution (may it be entry-wise sparsity or matrix rank-wise sparsity) it is possible to have linear convergence rates which depend on an improved mixed-norm condition number of the form \frac\beta_1s\alpha_2 , where \beta_1 is the \ell_1 -Lipchitz continuity constant of the gradient, \alpha_2 is the \ell_2 -quadratic growth constant, and s is the sparsity of the optimal solution. However, beyond the improved convergence rate, these methods are unable to leverage the sparsity of optimal solutions towards improving also the runtime of each iteration, which may still be prohibitively high for high-dimensional problems. In this work, we establish that linear convergence rates which depend on this improved condition number can be obtained using only sparse updates, which may result in overall significantly improved running times. Moreover, our methods are considerably easier to implement.
[LG-75] When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets ICML2025
链接: https://arxiv.org/abs/2506.19031
作者: Chen Zeno,Hila Manor,Greg Ongie,Nir Weinberger,Tomer Michaeli,Daniel Soudry
类目: Machine Learning (stat.ML); Machine Learning (cs.LG)
*备注: Accepted to the Forty-second International Conference on Machine Learning (ICML 2025)
Abstract:While diffusion models generate high-quality images via probability flow, the theoretical understanding of this process remains incomplete. A key question is when probability flow converges to training samples or more general points on the data manifold. We analyze this by studying the probability flow of shallow ReLU neural network denoisers trained with minimal \ell^2 norm. For intuition, we introduce a simpler score flow and show that for orthogonal datasets, both flows follow similar trajectories, converging to a training point or a sum of training points. However, early stopping by the diffusion time scheduler allows probability flow to reach more general manifold points. This reflects the tendency of diffusion models to both memorize training samples and generate novel points that combine aspects of multiple samples, motivating our study of such behavior in simplified settings. We extend these results to obtuse simplex data and, through simulations in the orthogonal case, confirm that probability flow converges to a training point, a sum of training points, or a manifold point. Moreover, memorization decreases when the number of training samples grows, as fewer samples accumulate near training points.
[LG-76] Simulation-Based Sensitivity Analysis in Optimal Treatment Regimes and Causal Decomposition with Individualized Interventions
链接: https://arxiv.org/abs/2506.19010
作者: Soojin Park,Suyeon Kang,Chioun Lee
类目: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)
*备注: 42 pages
Abstract:Causal decomposition analysis aims to assess the effect of modifying risk factors on reducing social disparities in outcomes. Recently, this analysis has incorporated individual characteristics when modifying risk factors by utilizing optimal treatment regimes (OTRs). Since the newly defined individualized effects rely on the no omitted confounding assumption, developing sensitivity analyses to account for potential omitted confounding is essential. Moreover, OTRs and individualized effects are primarily based on binary risk factors, and no formal approach currently exists to benchmark the strength of omitted confounding using observed covariates for binary risk factors. To address this gap, we extend a simulation-based sensitivity analysis that simulates unmeasured confounders, addressing two sources of bias emerging from deriving OTRs and estimating individualized effects. Additionally, we propose a formal bounding strategy that benchmarks the strength of omitted confounding for binary risk factors. Using the High School Longitudinal Study 2009 (HSLS:09), we demonstrate this sensitivity analysis and benchmarking method.
[LG-77] owards AI-assisted Neutrino Flavor Theory Design
链接: https://arxiv.org/abs/2506.08080
作者: Jason Benjamin Baretz,Max Fieg,Vijay Ganesh,Aishik Ghosh,V. Knapp-Perez,Jake Rudolph,Daniel Whiteson
类目: High Energy Physics - Phenomenology (hep-ph); Machine Learning (cs.LG); Computational Physics (physics.comp-ph); Machine Learning (stat.ML)
*备注: 25 pages, 12 Figures
Abstract:Particle physics theories, such as those which explain neutrino flavor mixing, arise from a vast landscape of model-building possibilities. A model’s construction typically relies on the intuition of theorists. It also requires considerable effort to identify appropriate symmetry groups, assign field representations, and extract predictions for comparison with experimental data. We develop an Autonomous Model Builder (AMBer), a framework in which a reinforcement learning agent interacts with a streamlined physics software pipeline to search these spaces efficiently. AMBer selects symmetry groups, particle content, and group representation assignments to construct viable models while minimizing the number of free parameters introduced. We validate our approach in well-studied regions of theory space and extend the exploration to a novel, previously unexamined symmetry group. While demonstrated in the context of neutrino flavor theories, this approach of reinforcement learning with physics software feedback may be extended to other theoretical model-building problems in the future.
信息检索
[IR-0] KnowML: Improving Generalization of ML-NIDS with Attack Knowledge Graphs
链接: https://arxiv.org/abs/2506.19802
作者: Xin Fan Guo,Albert Merono Penuela,Sergio Maffeis,Fabio Pierazzi
类目: Cryptography and Security (cs.CR); Information Retrieval (cs.IR)
*备注:
Abstract:Despite extensive research on Machine Learning-based Network Intrusion Detection Systems (ML-NIDS), their capability to detect diverse attack variants remains uncertain. Prior studies have largely relied on homogeneous datasets, which artificially inflate performance scores and offer a false sense of security. Designing systems that can effectively detect a wide range of attack variants remains a significant challenge. The progress of ML-NIDS continues to depend heavily on human expertise, which can embed subjective judgments of system designers into the model, potentially hindering its ability to generalize across diverse attack types. To address this gap, we propose KnowML, a framework for knowledge-guided machine learning that integrates attack knowledge into ML-NIDS. KnowML systematically explores the threat landscape by leveraging Large Language Models (LLMs) to perform automated analysis of attack implementations. It constructs a unified Knowledge Graph (KG) of attack strategies, on which it applies symbolic reasoning to generate KG-Augmented Input, embedding domain knowledge directly into the design process of ML-NIDS. We evaluate KnowML on 28 realistic attack variants, of which 10 are newly collected for this study. Our findings reveal that baseline ML-NIDS models fail to detect several variants entirely, achieving F1 scores as low as 0 %. In contrast, our knowledge-guided approach achieves up to 99 % F1 score while maintaining a False Positive Rate below 0.1 %. Subjects: Cryptography and Security (cs.CR); Information Retrieval (cs.IR) Cite as: arXiv:2506.19802 [cs.CR] (or arXiv:2506.19802v1 [cs.CR] for this version) https://doi.org/10.48550/arXiv.2506.19802 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
