NeurIPS 2023 compressed

This page contains summaries of all Neurips 2023 accepted papers generated by the compressor, my personal LLM-based project.

What is Flagged in Uncertainty Quantification? Latent Density Models for Uncertainty Categorization

Hao Sun, Boris van Breugel, Jonathan Crabbé, Nabeel Seedat, Mihaela van der Schaar

https://openreview.net/forum?id=zyhxRc9bew

Keywords: Uncertainty Explaination, Uncertainty Quantification, Interpretability

Compressor summary: This paper introduces a framework to classify uncertain examples from machine learning models using the confusion density matrix, which helps compare different uncertainty quantification methods.

Don’t blame Dataset Shift! Shortcut Learning due to Gradients and Cross Entropy

Aahlad Manas Puli, Lily H Zhang, Yoav Wald, Rajesh Ranganath

https://openreview.net/forum?id=zyZkaqNnpa

Keywords: shortcut learning, spurious correlations, perfect stable feature, perception tasks, implicit bias in optimization, improving inductive biases

Compressor summary: The paragraph discusses how default-ERM models tend to use shortcuts in perception tasks due to their preference for maximizing margins, and proposes alternative loss functions called margin control (MARG-CTRL) that encourage uniform-margin solutions and mitigate shortcut learning.

Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularization

Xilie Xu, Jingfeng Zhang, Feng Liu, Masashi Sugiyama, Mohan Kankanhalli

https://openreview.net/forum?id=zuXyQsXVLF

Keywords: robust pre-training, adversarial contrastive learning

Compressor summary: Adversarial contrastive learning (ACL) enhances standard contrastive learning by using adversarial data and introduces adversarial invariant regularization (AIR) to improve robustness against style factors, as shown by causal reasoning and empirical results.

An Optimization-based Approach To Node Role Discovery in Networks: Approximating Equitable Partitions

Michael Scholkemper, Michael T Schaub

https://openreview.net/forum?id=ztDxO15N7f

Keywords: Role Extraction, Graph Learning, Node Embeddings, Weisfeiler Lehman, Equitable Partition

Compressor summary: The paragraph discusses a new method for identifying structural roles of nodes in complex networks, based on graph-isomorphism tests and the Weisfeiler-Leman algorithm, and presents a benchmark to test the approach.

Generator Identification for Linear SDEs with Additive and Multiplicative Noise

Yuanyuan Wang, Xi Geng, Wei Huang, Biwei Huang, Mingming Gong

https://openreview.net/forum?id=zsOOqjaj2z

Keywords: Linear SDE, Identification, Causal inference

Compressor summary: The paper proposes identifiability conditions for determining the generator of linear SDEs from its solution process distribution, which is useful for causal inference, and provides sufficient and necessary conditions for additive noise and sufficient ones for multiplicative noise, along with geometric interpretations and simulations.

Algorithm Selection for Deep Active Learning with Imbalanced Datasets

Jifan Zhang, Shuai Shao, saurabh verma, Robert D Nowak

https://openreview.net/forum?id=zrUEHZ6s9C

Keywords: Deep Learning, Active Learning

Compressor summary: TAILOR is a meta algorithm that adaptively selects among various active learning strategies to efficiently reduce labeling efforts for deep learning applications using class-balanced rewards.

Discover and Align Taxonomic Context Priors for Open-world Semi-Supervised Learning

Yu Wang, Zhun Zhong, Pengchong Qiao, Xuxin Cheng, Xiawu Zheng, Chang Liu, Nicu Sebe, Rongrong Ji, Jie Chen

https://openreview.net/forum?id=zrLxHYvIFL

Keywords: open-world semi-supervised learning; novel class discovery;

Compressor summary: The paper proposes TIDA, a framework for open-world semi-supervised learning that leverages multi-granularity taxonomic context priors to enhance representation learning and improve pseudo label quality.

Learning Invariant Representations of Graph Neural Networks via Cluster Generalization

Donglin Xia, Xiao Wang, Nian Liu, Chuan Shi

https://openreview.net/forum?id=zrCmeqV3Sz

Keywords: Graph neural networks, network representation learning, deep learning

Compressor summary: The paper proposes CIT, a mechanism that improves GNNs' generalization ability by transferring cluster information and preserving node diversity when the test graph structure differs from the training graph structure.

The expressive power of pooling in Graph Neural Networks

Filippo Maria Bianchi, Veronica Lachi

https://openreview.net/forum?id=zqyVjCjhYD

Keywords: Graph Neural Networks, Graph pooling, Expressive power

Compressor summary: This paper studies how graph pooling affects the expressiveness of Graph Neural Networks (GNNs) and provides a criterion for choosing or designing pooling operators based on their ability to preserve the GNN's message-passing power.

Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examples

Shaokui Wei, Mingda Zhang, Hongyuan Zha, Baoyuan Wu

https://openreview.net/forum?id=zqOcW3R9rd

Keywords: Backdoor Attack, Trustworthy AI, Backdoor Learning

Compressor summary: The paper proposes a method called Shared Adversarial Unlearning (SAU) to purify backdoored machine learning models using a small clean dataset and adversarial training techniques, achieving state-of-the-art performance.

The Crucial Role of Normalization in Sharpness-Aware Minimization

Yan Dai, Kwangjun Ahn, Suvrit Sra

https://openreview.net/forum?id=zq4vFneRiA

Keywords: Sharpness-Aware Minimization, Normalization, Deep Learning Theory

Compressor summary: The paragraph discusses a paper that explores how normalization, a key component of the SAM optimizer, improves deep neural network performance by stabilizing the algorithm and enabling it to drift along a manifold of minima.

Towards Personalized Federated Learning via Heterogeneous Model Reassembly

Jiaqi Wang, Xingyi Yang, Suhan Cui, Liwei Che, Lingjuan Lyu, Dongkuan Xu, Fenglong Ma

https://openreview.net/forum?id=zpVCITHknd

Keywords: Federated Learning

Compressor summary: The paper introduces pFedHR, a framework for personalized federated learning that handles model heterogeneity by reassembling diverse models using server-side optimization and minimizing the impact of different data distributions.

Active Learning for Semantic Segmentation with Multi-class Label Query

Sehyun Hwang, Sohyun Lee, Hoyoung Kim, Minhyeon Oh, Jungseul Ok, Suha Kwak

https://openreview.net/forum?id=znudaK78u8

Keywords: semantic segmentation; active learning; partial label learning

Compressor summary: The paper presents a new active learning method for semantic segmentation that uses multi-class labeling of local image regions and disambiguates partial labels with two loss functions and pseudo labels, achieving better performance and annotation efficiency.

Time-Reversed Dissipation Induces Duality Between Minimizing Gradient Norm and Function Value

Jaeyeon Kim, Asuman E. Ozdaglar, Chanwoo Park, Ernest K. Ryu

https://openreview.net/forum?id=znY173SCxu

Keywords: Convex Optimization, Acceleration, First-Order methods

Compressor summary: The paper introduces H-duality, a novel concept in convex optimization that reveals symmetries between different first-order optimization methods and leads to new efficient techniques for minimizing function values and gradient magnitudes.

Optimizing over trained GNNs via symmetry breaking

Shiqiang Zhang, Juan S Campos, Christian Wolfgang Feldmann, David Walz, Frederik Sandfort, Miriam Mathea, Calvin Tsay, Ruth Misener

https://openreview.net/forum?id=znW5jNIOED

Keywords: Mixed-integer optimization, Graph neural network, Symmetry-breaking, Molecular design

Compressor summary: The paper proposes methods to optimize machine learning models constrained by trained graph neural networks (GNNs) using symmetry-breaking constraints and applies them to molecular design.

An Alternating Optimization Method for Bilevel Problems under the Polyak-Łojasiewicz Condition

Quan Xiao, Songtao Lu, Tianyi Chen

https://openreview.net/forum?id=zn5ihqknGj

Keywords: Bilevel optimization, nonconvex constrained optimization, convergence analysis

Compressor summary: This paper introduces a new method (GALET) for solving bilevel optimization problems with nonconvex lower-level objectives and shows that it has the same convergence rate as gradient descent.

Scalable Fair Influence Maximization

Xiaobin Rui, Zhixiao Wang, Jiayu Zhao, Lichao Sun, Wei Chen

https://openreview.net/forum?id=zmWNe1V6jg

Keywords: influence maximization, approximation algorithm, social fairness

Compressor summary: The paper proposes an efficient algorithm for maximizing welfare fairness in community structures using a weighted maximum coverage problem approach.

Curriculum Learning With Infant Egocentric Videos

Saber Sheybani, Himanshu Hansaria, Justin Newell Wood, Linda B. Smith, Zoran Tiganj

https://openreview.net/forum?id=zkfyOkBVpz

Keywords: Curriculum learning, Self-supervised learning, Slow changes, Infant development

Compressor summary: Infant visual experiences affect the development of their visual system, and starting with data from younger infants improves learning outcomes for AI models trained on their visual inputs.

Comparing Apples to Oranges: Learning Similarity Functions for Data Produced by Different Distributions

Leonidas Tsepenekas, Ivan Brugere, Freddy Lecue, Daniele Magazzeni

https://openreview.net/forum?id=zjpjsJeVJZ

Keywords: individual fairness; similarity learning; active learning

Compressor summary: The paper proposes an efficient sampling framework to learn similarity functions between different groups using limited expert feedback, and provides theoretical and empirical evidence for its effectiveness.

Temporal Causal Mediation through a Point Process: Direct and Indirect Effects of Healthcare Interventions

Çağlar Hızlı, S. T. John, Anne Tuulikki Juuti, Tuure Tapani Saarinen, Kirsi Hannele Pietiläinen, Pekka Marttinen

https://openreview.net/forum?id=zfHCKDzzC8

Keywords: Machine learning for healthcare, Causal mediation, Gaussian process, Point Process

Compressor summary: The authors propose a new method to analyze the causal effects of an intervention on outcomes and mediators in dynamic systems using temporal point processes.

Interpreting Unsupervised Anomaly Detection in Security via Rule Extraction

Ruoyu Li, Qing Li, Yu Zhang, Dan Zhao, Yong Jiang, Yong Yang

https://openreview.net/forum?id=zfCNwRQ569

Keywords: unsupervised anomaly detection, global explanation, rule extraction

Compressor summary: The paper proposes a post-hoc method to explain black-box unsupervised anomaly detection models using distribution decomposition rules and boundary inference rules, making them more interpretable and trustworthy.

Counting Distinct Elements Under Person-Level Differential Privacy

Thomas Steinke, Alexander Knop

https://openreview.net/forum?id=zdli6OxpWd

Keywords: differential privacy, user-level privacy, person-level privacy, sensitivity

Compressor summary: The authors propose a method to count distinct elements in a dataset with person-level differential privacy, by approximating the problem with a max-flow solution and optimizing the sensitivity bound.

Optimistic Natural Policy Gradient: a Simple Efficient Policy Optimization Framework for Online RL

Qinghua Liu, Gellért Weisz, András György, Chi Jin, Csaba Szepesvari

https://openreview.net/forum?id=zaQ7wV9NOg

Keywords: Theory of reinforcement learning, policy optimization

Compressor summary: This paper introduces Optimistic NPG, a simple and efficient policy optimization framework for online RL with linear MDPs, which improves over existing algorithms in terms of computation and exploration.

Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference

Licong Lin, Mufang Ying, Suvrojit Ghosh, Koulik Khamaru, Cun-Hui Zhang

https://openreview.net/forum?id=zXckveawHa

Keywords: Adaptive linear regression, bandit algorithms, high dimensional statistics, statistical inference

Compressor summary: The paper explores how adaptive data collection affects estimation and inference in high-dimensional linear models and proposes a new estimator for single coordinate inference with better performance than OLS.

Universality and Limitations of Prompt Tuning

Yihan Wang, Jatin Chauhan, Wei Wang, Cho-Jui Hsieh

https://openreview.net/forum?id=zWxKYyW9ik

Keywords: prompt-tuning; language model; expressive power

Compressor summary: The paper analyzes the theoretical and practical aspects of prompt tuning for transformer-based models, showing its universality for approximating sequence-to-sequence functions and its limitations for limited-depth models and non-invertible datasets.

Unpaired Multi-Domain Causal Representation Learning

Nils Sturma, Chandler Squires, Mathias Drton, Caroline Uhler

https://openreview.net/forum?id=zW1uVN6Mbv

Keywords: linear structural equation models, causality, representation learning, independent component analysis, structure identifiability, multiple views, graphical model

Compressor summary: The paper proposes a method to identify a shared causal representation of unpaired data from multiple domains using linear models.

$S^3$: Increasing GPU Utilization during Generative Inference for Higher Throughput

Yunho Jin, Chun-Feng Wu, David Brooks, Gu-Yeon Wei

https://openreview.net/forum?id=zUYfbdNl1m

Keywords: Throughput, GPU utilization, Sequence length prediction

Compressor summary: The paper proposes $S^3$, a system that predicts the output sequence length of large language models to increase memory and GPU utilization, and handle mispredictions.

Beta Diffusion

Mingyuan Zhou, Tianqi Chen, Zhendong Wang, Huangjie Zheng

https://openreview.net/forum?id=zTSlm4nmlH

Keywords: Diffusion models, KL-divergence upper bounds, multiplicative transitions, scaled and shifted beta distributions

Compressor summary: Beta diffusion is a new method for generating data within limited ranges using multiplicative transitions based on beta distributions and optimized with KL-divergence upper bounds, which outperform negative ELBOs in experiments with synthetic and natural images.

Sounding Bodies: Modeling 3D Spatial Sound of Humans Using Body Pose and Audio

Xudong XU, Dejan Markovic, Jacob Sandakly, Todd Keebler, Steven Krenn, Alexander Richard

https://openreview.net/forum?id=zQTi3pziFp

Keywords: sound field, spatial audio, virtual humans, human body, body modeling

Compressor summary: The authors present a system that generates realistic 3D spatial audio for human bodies using input from microphones and body pose, and introduce a new multimodal dataset for this task.

Optimized Covariance Design for AB Test on Social Network under Interference

Qianyi Chen, Bo Li, LU DENG, Yong Wang

https://openreview.net/forum?id=zQOYGDc9pu

Keywords: AB test, interference, causal inference, optimization, social network

Compressor summary: The paper proposes a new method for designing randomized network experiments that balances bias and variance by optimizing the covariance matrix of the treatment assignment vector using projected gradient descent.

Multi-scale Diffusion Denoised Smoothing

Jongheon Jeong, Jinwoo Shin

https://openreview.net/forum?id=zQ4yraDiRe

Keywords: adversarial robustness, certified robustness, randomized smoothing, denoised smoothing, diffusion models

Compressor summary: The paper presents a method called multi-scale smoothing that improves the trade-off between robustness and accuracy in denoised smoothing by selectively applying randomized smoothing among multiple noise scales, and proposes diffusion fine-tuning to enhance the performance of diffusion models.

Conformal PID Control for Time Series Prediction

Anastasios Nikolas Angelopoulos, Emmanuel Candes, Ryan Tibshirani

https://openreview.net/forum?id=zPYeYv6YYs

Keywords: conformal prediction, time series, uncertainty quantification, distribution shift

Compressor summary: The paper presents uncertainty quantification algorithms for time series prediction that use conformal prediction and control theory, improve coverage on COVID-19 forecasts, and provide an extendable codebase.

Residual Alignment: Uncovering the Mechanisms of Residual Networks

Jianing Li, Vardan Papyan

https://openreview.net/forum?id=zOCIKYVaF5

Keywords: Deep Learning, Residual Networks, Neural Networks, Generalization, Spectral Analysis

Compressor summary: This paper investigates Residual Alignment in ResNet architectures and shows how it aligns intermediate representations linearly across layers and affects generalization.

Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints

Alistair White, Niki Kilbertus, Maximilian Gelbrecht, Niklas Boers

https://openreview.net/forum?id=zO2dAQfvHf

Keywords: neural differential equations, neural ordinary differential equations, constraints, conservation laws, stabilization, dynamical systems, dynamics, scientific machine learning, physics-informed machine learning

Compressor summary: SNDEs are a method to enforce constraints on neural differential equations by adding a stabilization term, which makes them compatible with various models and improves performance.

P-Flow: A Fast and Data-Efficient Zero-Shot TTS through Speech Prompting

Sungwon Kim, Kevin J. Shih, Rohan Badlani, Joao Felipe Santos, Evelina Bakhturina, Mikyas T. Desta, Rafael Valle, Sungroh Yoon, Bryan Catanzaro

https://openreview.net/forum?id=zNA7u7wtIN

Keywords: text-to-speech, zero-shot TTS, flow matching generative model

Compressor summary: P-Flow is a fast and data-efficient zero-shot text-to-speech model that uses speech prompts for speaker adaptation and achieves high quality and pronunciation with less training data and faster sampling speed than existing models.

FAMO: Fast Adaptive Multitask Optimization

Bo Liu, Yihao Feng, Peter Stone, qiang liu

https://openreview.net/forum?id=zMeemcUeXL

Keywords: multitask learning, multitask optimization, conflicting gradients, knowledge transfer

Compressor summary: The paper introduces FAMO, a method that balances task losses in multitask learning using minimal space and time, outperforming existing techniques.

Implicit variance regularization in non-contrastive SSL

Manu Srinath Halvagal, Axel Laborieux, Friedemann Zenke

https://openreview.net/forum?id=zMNUNd9zs1

Keywords: Self-supervised learning, Non-contrastive learning, Learning dynamics

Compressor summary: Non-contrastive self-supervised learning (SSL) methods use different losses to stabilize learning and avoid representational collapse, and a new family of loss functions called IsoLoss can further improve their performance.

New Complexity-Theoretic Frontiers of Tractability for Neural Network Training

Cornelius Brand, Robert Ganian, Mathis Rocton

https://openreview.net/forum?id=zIEaOZ0saA

Keywords: neural network training, computational complexity, ReLU networks, Linear networks

Compressor summary: The article presents new algorithms that improve the efficiency of training neural networks with linear or ReLU activation functions.

Optimal Treatment Regimes for Proximal Causal Learning

Tao Shen, Yifan Cui

https://openreview.net/forum?id=zGdH4tKtOW

Keywords: Optimal treatment regimes, Policy-making, Proximal causal inference, Unmeasured confounding, Value function

Compressor summary: The paragraph discusses a new framework for causal inference using proxy variables, and proposes an optimal individualized treatment regime based on outcome and treatment confounding bridges, with theoretical guarantees and applications shown.

Multi-Swap k-Means++

Lorenzo Beretta, Vincent Cohen-Addad, Silvio Lattanzi, Nikos Parotsidis

https://openreview.net/forum?id=zGRWp7yRqd

Keywords: Clustering, k-means, approximation algorithms

Compressor summary: The paper proposes an improved $k$-means clustering algorithm by combining $k$-means++ sampling with larger local-search neighborhoods, achieving a 9 + ε approximation ratio and practical efficiency.

Automated Classification of Model Errors on ImageNet

Momchil Peychev, Mark Niklas Mueller, Marc Fischer, Martin Vechev

https://openreview.net/forum?id=zEoP4vzFKy

Keywords: ImageNet, evaluation, error classification, error analysis

Compressor summary: The paragraph discusses how automated error classification can help evaluate computer vision models and finds that top-1 accuracy is still useful for measuring model performance despite its limitations.

(Amplified) Banded Matrix Factorization: A unified approach to private training

Christopher A. Choquette-Choo, Arun Ganesh, Ryan McKenna, Hugh Brendan McMahan, J Keith Rush, Abhradeep Guha Thakurta, Zheng Xu

https://openreview.net/forum?id=zEm6hF97Pz

Keywords: Machine Learning, Differential Privacy, Optimization, Private Machine Learning, Federated Learning, Privacy Amplification, Matrix Factorization

Compressor summary: The paper presents a method to improve privacy-utility tradeoffs for machine learning using matrix factorization with banded matrices, which can be applied in both centralized and federated settings.

Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning

Lin Guan, Karthik Valmeekam, Sarath Sreedharan, Subbarao Kambhampati

https://openreview.net/forum?id=zDbsSscmuj

Keywords: LLMs, Planning, Domain Model, LLMs for Planning, LLMs for Heuristic Guidance

Compressor summary: The authors propose a method that uses large language models to construct PDDL domain models and then plan with sound planners, reducing human involvement and improving planning efficiency.

A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergence

Carlo Alfano, Rui Yuan, Patrick Rebeschini

https://openreview.net/forum?id=zD6lXmTPPh

Keywords: Theory for Reinforcement Learning, Policy Optimization, Policy Gradient, Mirror Descent.

Compressor summary: The paper introduces a policy optimization framework based on mirror descent that allows general parameterizations and shows linear convergence, sample complexity improvements, and experimental validation.

Quasi-Monte Carlo Graph Random Features

Isaac Reid, Adrian Weller, Krzysztof Marcin Choromanski

https://openreview.net/forum?id=zCFfv49MjE

Keywords: Graph, discrete mathematics, quasi-Monte Carlo, kernel, scalability, Laplacian, clustering, random walks

Compressor summary: The paper introduces a novel method to improve graph random features by inducing negative correlations through antithetic termination, leading to lower-variance estimators and better performance on various tasks.

One-Line-of-Code Data Mollification Improves Optimization of Likelihood-based Generative Models

Ba-Hien Tran, Giulio Franzese, Pietro Michiardi, Maurizio Filippone

https://openreview.net/forum?id=zAXg8dW8ZO

Keywords: Generative Models, Normalizing Flows, Variational Autoencoders

Compressor summary: The paper introduces a simple technique to improve the quality of likelihood-based generative models by using data mollification, which enhances density estimation and low-density region handling without additional computational costs.

Decision Stacks: Flexible Reinforcement Learning via Modular Generative Models

Siyan Zhao, Aditya Grover

https://openreview.net/forum?id=zAQK5r1enm

Keywords: reinforcement learning, generative models, offline RL, sequential decision making

Compressor summary: Decision Stacks is a framework that splits goal-conditioned policy agents into three independent modules for efficient learning and flexible generative decision making in sequential decision making problems.

Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior

Shashank Subramanian, Peter Harrington, Kurt Keutzer, Wahid Bhimji, Dmitriy Morozov, Michael W. Mahoney, Amir Gholami

https://openreview.net/forum?id=zANxvzflMl

Keywords: scientific machine learning, scaling, transfer learning, neural operators, foundation models

Compressor summary: The study shows that pre-training machine learning models can significantly improve performance in scientific applications by adapting them to different tasks with fewer examples and across various physics problems.

CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation

Sihan Xu, Ziqiao Ma, Yidong Huang, Honglak Lee, Joyce Chai

https://openreview.net/forum?id=z9d9DsjAPH

Keywords: Image to image translation, latent diffusion models, conditional diffusion models

Compressor summary: Cyclenet is a method that uses cycle consistency to improve image synthesis with diffusion models, enabling consistent unpaired image-to-image translation and robust generation of high-quality images.

A Regularized Conditional GAN for Posterior Sampling in Image Recovery Problems

Matthew C Bendel, Rizwan Ahmad, Philip Schniter

https://openreview.net/forum?id=z4vKRmq7UO

Keywords: Generative adversarial network, inverse problems, posterior sampling, cGAN, GAN

Compressor summary: The paper proposes a fast method to generate multiple high-quality images from noisy measurements using a conditional Wasserstein GAN with regularization and shows its effectiveness in MRI and inpainting tasks.

Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization

Shurui Gui, Meng Liu, Xiner Li, Youzhi Luo, Shuiwang Ji

https://openreview.net/forum?id=z3HACY5CMa

Keywords: deep learning, graph neural network, out-of-distribution generalization, distribution shift

Compressor summary: The paper introduces LECI, a method to improve graph out-of-distribution generalization by using label and environment information and developing an adversarial training strategy.

Online List Labeling with Predictions

Samuel McCauley, Benjamin Moseley, Aidin Niaparast, Shikha Singh

https://openreview.net/forum?id=z37ki6nqAY

Keywords: Algorithms with Predictions, Data Structures, Learned Indices, Online List Labeling, Resource Allocation, Beyond Worst Case Analysis

Compressor summary: The paper proposes a new list labeling data structure that leverages predictions to improve performance and provides theoretical guarantees for both worst-case and stochastic error models.

Thin and deep Gaussian processes

Daniel Augusto de Souza, Alexander V Nikitin, S. T. John, Magnus Ross, Mauricio A Álvarez, Marc Peter Deisenroth, João Paulo Pordeus Gomes, Diego Mesquita, César Lincoln Mattos

https://openreview.net/forum?id=z2BHMLA8pM

Keywords: Gaussian Processes, Deep Gaussian Processes, non-stationary kernels

Compressor summary: TDGP combines the benefits of deep GPs and shallow GPs by learning interpretable low-dimensional embeddings while maintaining the flexibility to adjust kernel hyperparameters, making it better suited for uncertainty quantification than previous models.

Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers

Zixuan Jiang, Jiaqi Gu, Hanqing Zhu, David Z. Pan

https://openreview.net/forum?id=z06npyCwDq

Keywords: Transformer, Normalization, Layer Normalization, RMSNorm, Efficient Machine Learning

Compressor summary: The paragraph discusses different normalization techniques for Transformers and proposes a solution to unify and improve their efficiency without compromising performance.

SparseProp: Efficient Event-Based Simulation and Training of Sparse Recurrent Spiking Neural Networks

Rainer Engelken

https://openreview.net/forum?id=yzZbwQPkmP

Keywords: spiking networks, event-based simulation, sparse networks, backpropagation, algorithm, neuroscience

Compressor summary: SparseProp is a novel event-based algorithm that simulates and trains sparse SNNs with reduced computational cost, enabling efficient and exact simulations of large spiking networks.

What Distributions are Robust to Indiscriminate Poisoning Attacks for Linear Learners?

Fnu Suya, Xiao Zhang, Yuan Tian, David Evans

https://openreview.net/forum?id=yyLFUPNEiT

Keywords: poisoning attacks; adversarial machine learning; machine learning security

Compressor summary: The study explores how linear learners can resist indiscriminate poisoning attacks on some datasets with well-separated and low-variance data distributions.

Revisiting Adversarial Robustness Distillation from the Perspective of Robust Fairness

Xinli Yue, Ningping Mou, Qian Wang, Lingchen Zhao

https://openreview.net/forum?id=ywrPcBEXdC

Keywords: Deep Learning, Knowledge Distillation, Adversarial Training, Fairness

Compressor summary: This paper introduces Fair-ARD, a novel framework to enhance robust fairness in student models by adjusting the weights of difficult classes during adversarial robustness distillation, and shows its effectiveness in improving both overall and class-wise robustness.

Computing Optimal Equilibria and Mechanisms via Learning in Zero-Sum Extensive-Form Games

Brian Hu Zhang, Gabriele Farina, Ioannis Anagnostides, Federico Cacciamani, Stephen Marcus McAleer, Andreas Alexander Haupt, Andrea Celli, Nicola Gatti, Vincent Conitzer, Tuomas Sandholm

https://openreview.net/forum?id=yw1v4RqvPk

Keywords: extensive-form games, deep reinforcement learning, mechanism design, correlated equilibria

Compressor summary: A new method for finding optimal equilibria in extensive-form games is presented, which applies existing zero-sum game learning techniques and achieves state-of-the-art results in tabular games and auction design.

Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysis

Alexander Meulemans, Simon Schug, Seijin Kobayashi, Nathaniel Daw, Greg Wayne

https://openreview.net/forum?id=yvqqkOn9Pi

Keywords: Reinforcement learning, Long-term credit assignment, contribution analysis, hindsight credit assignment, policy gradient methods

Compressor summary: COCOA is a new model-based credit assignment algorithm that measures the contribution of actions to future rewards by answering counterfactual questions, improving sample efficiency in reinforcement learning.

Canonical normalizing flows for manifold learning

Kyriakos Flouris, Ender Konukoglu

https://openreview.net/forum?id=yubwSWol6K

Keywords: manifold learning flows, normalizing flows, optimization, orthogonalization, sparsity, sparse learning, generative modeling, Riemannian manifold, geometry, metric tensor, orthogonal basis

Compressor summary: The proposed canonical manifold learning flow method generates a sparse and orthogonal low-dimensional representation of the data that improves the efficiency and accuracy of generative modeling techniques compared to other methods.

Epidemic Learning: Boosting Decentralized Learning with Randomized Communication

Martijn De Vos, Sadegh Farhadkhani, Rachid Guerraoui, Anne-marie Kermarrec, Rafael Pires, Rishi Sharma

https://openreview.net/forum?id=ytrhsvGP0r

Keywords: Epidemic, Decentralized Learning, Randomized Communication, Peer sampling

Compressor summary: Epidemic Learning (EL) is a decentralized learning algorithm that uses changing communication topologies to converge faster than conventional approaches, achieving better performance with less communication volume.

Expressivity-Preserving GNN Simulation

Fabian Jogl, Maximilian Thiessen, Thomas Gärtner

https://openreview.net/forum?id=ytTfonl9Wd

Keywords: Graph Neural Networks, GNNs, Graphs, Message Passing, Expressiveness, Graph Transformations, Message Passing Graph Neural Networks

Compressor summary: The paragraph discusses how graph transformations enable message passing operations for simulating graph neural networks (GNNs) without losing expressivity and improving code optimization, while distinguishing between weak and strong simulation methods.

A Unifying Perspective on Multi-Calibration: Game Dynamics for Multi-Objective Learning

Nika Haghtalab, Michael Jordan, Eric Zhao

https://openreview.net/forum?id=ysqlhW0v26

Keywords: multicalibration, multi-objective learning, learning theory, calibration, fairness, games

Compressor summary: The authors present a unified framework for designing and analyzing multi-calibrated predictors by using game dynamics and achieving better guarantees than existing methods.

Emergent Correspondence from Image Diffusion

Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, Bharath Hariharan

https://openreview.net/forum?id=ypOiXjdfnU

Keywords: Correspondence, Diffusion Model

Compressor summary: The paper introduces DIFT, a method to establish correspondences between images using diffusion models without explicit supervision, and shows that it can outperform weakly-supervised methods and off-the-shelf features on various tasks.

CaMP: Causal Multi-policy Planning for Interactive Navigation in Multi-room Scenes

Xiaohan Wang, Yuehu Liu, Xinhang Song, Beibei Wang, Shuqiang Jiang

https://openreview.net/forum?id=yoZTVn0T50

Keywords: Embodied AI, Interactive Navigation, Causal Reinforcement Learning, Hierarchical Reinforcement Learning

Compressor summary: The paper introduces a causal diagram for visual navigation in complex scenes, proposes a multi-policy model to explore counterfactual interactions and reduce exploration, and presents a large-scale dataset for evaluation.

TOA: Task-oriented Active VQA

Xiaoying Xing, Mingfu Liang, Ying Wu

https://openreview.net/forum?id=yoAmURKDJi

Keywords: knowledge-based visual question answering, task-oriented, active image understanding, large language model, visual reasoning, multi-round dialogue

Compressor summary: The authors propose a new method for knowledge-based visual question answering using large language models that actively collects relevant visual evidence to verify their hypotheses, improving performance and interpretability on open-ended datasets.

Covariance-adaptive best arm identification

El Mehdi Saad, Gilles Blanchard, Nicolas Verzelen

https://openreview.net/forum?id=ymHM1qRUeb

Keywords: Multi-armed bandits, Best-arm identification, Adaptive identification

Compressor summary: The paper proposes new algorithms for identifying the best arm in a multi-armed bandit model with dependent arms and covariance, improving on standard methods in applications like clinical trials.

Model-Based Control with Sparse Neural Dynamics

Ziang Liu, Genggeng Zhou, Jeff He, Tobia Marcucci, Li Fei-Fei, Jiajun Wu, Yunzhu Li

https://openreview.net/forum?id=ymBG2xs9Zf

Keywords: model learning, model-based control, neural network sparsification, mixed-integer programming, trajectory optimization

Compressor summary: The paper proposes a new framework for learning and controlling with deep neural networks by sparsifying them and using efficient optimization algorithms, achieving better performance in tasks involving contact dynamics.

Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization

Yan Sun, Li Shen, Dacheng Tao

https://openreview.net/forum?id=ylPX5D7It7

Keywords: federated learning, local consistency, personalized initialization, excess risk

Compressor summary: The paper proposes FedInit, an efficient federated learning algorithm that uses personalized relaxed initialization to reduce the "client drift" problem and improve performance in distributed training of global models.

Deep Momentum Multi-Marginal Schrödinger Bridge

Tianrong Chen, Guan-Horng Liu, Molei Tao, Evangelos Theodorou

https://openreview.net/forum?id=ykvvv0gc4R

Keywords: Schrödinger Bridge, Trajectory Inference, Optimal Transport

Compressor summary: The paper presents DMSB, a novel computational framework for inferring high-dimensional multi-marginal trajectories from unlabeled samples at coarse time intervals, which outperforms baselines and can reconstruct velocity distributions from position snapshots.

DiffTraj: Generating GPS Trajectory with Diffusion Probabilistic Model

Yuanshao Zhu, Yongchao Ye, Shiyao Zhang, Xiangyu Zhao, James Yu

https://openreview.net/forum?id=ykMdzevPkJ

Keywords: Trajectory Generation, Diffusion Model, Urban Computing, Spatial-temporal Data Mining

Compressor summary: The authors propose a model called DiffTraj that generates realistic and privacy-preserving GPS trajectories by denoising white noise using spatial-temporal features and a deep neural network.

Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimension

Moritz Haas, David Holzmüller, Ulrike von Luxburg, Ingo Steinwart

https://openreview.net/forum?id=yjYwbZBJyl

Keywords: benign overfitting, kernels, neural tangent kernel, consistency, learning theory

Compressor summary: The key to benign overfitting is not high dimension but large derivatives in smooth estimators; this can be achieved by adding small fluctuations to the activation function of wide neural networks.

OBJECT 3DIT: Language-guided 3D-aware Image Editing

Oscar Michel, Anand Bhattad, Eli VanderBilt, Ranjay Krishna, Aniruddha Kembhavi, Tanmay Gupta

https://openreview.net/forum?id=yjWVd8Fhqt

Keywords: computer vision, image editing, generative modeling, diffusion models, 3D

Compressor summary: The authors propose a new task called language-guided 3D-aware editing, where edits are done according to language instructions while preserving the 3D scene, and release a large dataset for this task along with models that can perform well on it.

E2PNet: Event to Point Cloud Registration with Spatio-Temporal Representation Learning

Xiuhong Lin, Changjie Qiu, zhipeng cai, Siqi Shen, Yu Zang, Weiquan Liu, Xuesheng Bian, Matthias Müller, Cheng Wang

https://openreview.net/forum?id=yiehppUCO2

Keywords: event camera, 2D-3D registration, representation learning

Compressor summary: E2PNet is a novel method that registers 2D RGB images to 3D point clouds using event cameras, which improves robustness and enables other vision tasks.

Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation

GUOJUN XIONG, Jian Li

https://openreview.net/forum?id=yhNHpLWJDl

Keywords: Restless bandits, Whittle index policy, Q-learning, Two-timescale stochastic approximation, Neural network function approximation

Compressor summary: Neural-Q-Whittle is a Q-learning algorithm that uses neural networks and Whittle indices to solve intractable restless multi-armed bandits problems with an empirical convergence rate of $\mathcal{O}(1/k^{2/3})$.

Visual Programming for Step-by-Step Text-to-Image Generation and Evaluation

Jaemin Cho, Abhay Zala, Mohit Bansal

https://openreview.net/forum?id=yhBFG9Y85R

Keywords: text-to-image generation; visual programming; text-to-image evaluation; step-by-step generation; interpretability; explainability

Compressor summary: The paragraph introduces two novel visual programming frameworks, VPGen and VPEval, for text-to-image tasks that improve spatial control, handle predefined object classes, and provide interpretable and explainable evaluations with experts in different skills.

FiGURe: Simple and Efficient Unsupervised Node Representations with Filter Augmentations

Chanakya Ekbote, Ajinkya Deshpande, Arun Iyer, SUNDARARAJAN SELLAMANICKAM, Ramakrishna B Bairi

https://openreview.net/forum?id=yh0OkiUk5h

Keywords: Graph Neural Networks, Unsupervised Representation Learning, Graph Filters

Compressor summary: The paper introduces FiGURe, a method to improve node representations using filter-based augmentations and random Fourier feature projections, achieving up to 4.4% better performance on downstream tasks than existing unsupervised models.

Uncertainty Quantification over Graph with Conformalized Graph Neural Networks

Kexin Huang, Ying Jin, Emmanuel Candes, Jure Leskovec

https://openreview.net/forum?id=ygjQCOyNfh

Keywords: Graph Neural Networks, Conformal Prediction, Uncertainty Quantification

Compressor summary: CF-GNN is a graph-based model that provides guaranteed uncertainty estimates by extending conformal prediction to graph data and using a topology-aware output correction model to reduce the prediction set size.

A generative model of the hippocampal formation trained with theta driven local learning rules

Tom George, Kim Stachenfeld, Caswell Barry, Claudia Clopath, Tomoki Fukai

https://openreview.net/forum?id=yft4JlxsRf

Keywords: hippocampus, path integration, local learning, generative models, oscillations, inference, Helmholtz machine, wake-sleep

Compressor summary: The authors propose a biologically plausible model of the hippocampus that uses theta-band oscillations and a wake-sleep algorithm to learn and generate realistic sensory predictions and path integration.

PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalization

Jiancong Xiao, Ruoyu Sun, Zhi-Quan Luo

https://openreview.net/forum?id=ydKWoqWZ3t

Keywords: Pac-Bayes, Adversarial Robustness, Generalization

Compressor summary: The paper proposes a new robust generalization bound for deep neural networks that does not require additional assumptions and is tighter than existing bounds, offering insights into the causes of poor robust generalization.

Understanding the detrimental class-level effects of data augmentation

Polina Kirichenko, Mark Ibrahim, Randall Balestriero, Diane Bouchacourt, Shanmukha Ramakrishna Vedantam, Hamed Firooz, Andrew Gordon Wilson

https://openreview.net/forum?id=yageaKlk7S

Keywords: data augmentation, class-dependent bias

Compressor summary: The paper studies how data augmentation affects image classification, especially for classes with ambiguous or fine-grained distinctions, and proposes class-conditional augmentation strategies to improve performance.

Complexity of Derivative-Free Policy Optimization for Structured $\mathcal{H}_\infty$ Control

Xingang Guo, Darioush Keivan, Geir Dullerud, Peter Seiler, Bin Hu

https://openreview.net/forum?id=yaJ4vZPnHX

Keywords: Structured $\mathcal{H}_\infty$ Control, Nonsmooth Optimization, Complexity Analysis

Compressor summary: This paper presents new theoretical results on the difficulty of optimizing policies without derivatives for a class of robust control tasks, and shows how to find approximate solutions using zeroth-order oracles.

Star-Shaped Denoising Diffusion Probabilistic Models

Andrey Okhotin, Dmitry Molchanov, Arkhipkin Sergeevich Vladimir, Grigory Bartosh, Viktor Ohanesian, Aibek Alanov, Dmitry P. Vetrov

https://openreview.net/forum?id=yYUdgbmhh9

Keywords: Generative models, Diffusion, Exponential Family

Compressor summary: The paper introduces Star-Shaped Diffusion Probabilistic Models, which enable generative modeling with non-Gaussian distributions using a simpler and more efficient approach than existing methods.

Diverse Shape Completion via Style Modulated Generative Adversarial Networks

Wesley Khademi, Li Fuxin

https://openreview.net/forum?id=yVMlYSL1Bp

Keywords: multimodal shape completion, point cloud completion, 3d shape generation, generative modeling, generative adversarial networks

Compressor summary: The paper proposes a new network that can generate diverse, plausible 3D shape completions from partial observations using style modulation and multiple discriminators to prevent mode collapse.

PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance

Peiqing Yang, Shangchen Zhou, Qingyi Tao, Chen Change Loy

https://openreview.net/forum?id=yThjbzhIUP

Keywords: Face Restoration, Diffusion

Compressor summary: The paper introduces a new restoration approach called partial guidance that adapts to real-world degradations by modeling desired properties of high-quality images and applying them during the reverse diffusion process, achieving good results across various tasks.

Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions

Xiang Cheng, Bohan Wang, Jingzhao Zhang, Yusong Zhu

https://openreview.net/forum?id=yT0f93CeTw

Keywords: Sampling, MCMC, Conditional Mixing, Non-log-concave Distributions

Compressor summary: The paragraph explains that MCMC algorithms can work efficiently when a subset of the state space has Poincaré-style inequality, even if global mixing is slow, and this can help in various applications like sampling from mixtures of Gaussians or parameter estimation.

Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidance

Lisha Chen, Heshan Devaka Fernando, Yiming Ying, Tianyi Chen

https://openreview.net/forum?id=yPkbdJxQ0o

Keywords: Generalization, algorithm stability, multi-objective optimization, gradient conflict

Compressor summary: The MoDo algorithm studies the stability of dynamic weighting methods for multi-objective learning and reveals how updating along conflict-avoidant directions may hinder optimal generalization performance.

Generalized Information-theoretic Multi-view Clustering

Weitian Huang, Sirui Yang, Hongmin Cai

https://openreview.net/forum?id=yN6NHZOXkg

Keywords: information bottleneck, multi-view clustering, variational autoencoders

Compressor summary: The paper proposes an information-based multi-view clustering model that uses deep neural networks and Stochastic Gradient Variational Bayes for representation learning and clustering, achieving better results than existing methods.

Mechanism Design for Collaborative Normal Mean Estimation

Yiding Chen, Jerry Zhu, Kirthevasan Kandasamy

https://openreview.net/forum?id=yKCLfOOIL7

Keywords: Mechanism design, statistical minimax estimation, federated learning

Compressor summary: The paper proposes a novel mechanism for estimating the mean of a normal distribution collaboratively among strategic agents, using corrupted data sharing and minimax optimal estimators to incentivize truthful reporting and reduce estimation errors.

Towards a Unified Analysis of Kernel-based Methods Under Covariate Shift

Xingdong Feng, Xin HE, Caixing Wang, Chao Wang, Jingnan Zhang

https://openreview.net/forum?id=yIcCkMUCtL

Keywords: kernel methods, covariate shift, reproducing kernel Hilbert space (RKHS)

Compressor summary: The paper proposes a general analysis of nonparametric methods under covariate shift in RKHS, with applications to various learning tasks and theoretical and numerical results.

HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face

Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, Yueting Zhuang

https://openreview.net/forum?id=yHdTscY6Ci

Keywords: LLM, ChatGPT, Hugging Face, Autonomous LLM

Compressor summary: HuggingGPT is an LLM-powered agent that uses ChatGPT to connect various AI models in Hugging Face to solve complicated AI tasks across different domains and modalities, moving toward artificial general intelligence.

What Can We Learn from Unlearnable Datasets?

Pedro Sandoval-Segura, Vasu Singla, Jonas Geiping, Micah Goldblum, Tom Goldstein

https://openreview.net/forum?id=yGs9vTRjaE

Keywords: data poisoning, poisons, unlearnable dataset, data protection, imperceptible perturbations, adversarial machine learning

Compressor summary: Unlearnable dataset methods may not protect data privacy as they can enable neural networks to learn useful features, and linear separability of perturbations is not a necessary condition for this.

Augmented Memory Replay-based Continual Learning Approaches for Network Intrusion Detection

Suresh kumar Amalapuram, Sumohana S. Channappayya, Bheemarjuna Tamma

https://openreview.net/forum?id=yGLokEhdh9

Keywords: Continual learning, Class imbalance, scalability, Network intrusion detection, and Cybersecurity

Compressor summary: The paper proposes two methods to improve continual learning-based intrusion detection: one for class imbalance and another for reducing computations, which perform better than baselines on various benchmarks.

Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency

Owen Queen, Thomas Hartvigsen, Teddy Koker, Huan He, Theodoros Tsiligkaridis, Marinka Zitnik

https://openreview.net/forum?id=yEfmhgwslQ

Keywords: Explainability, Interpretability, Time Series, Explanations, Temporal patterns, Model Understanding, Latent space, Self-supervised learning

Compressor summary: TimeX is a model that trains an interpretable surrogate to mimic a pretrained time series model and provides discrete attribution maps and latent space explanations for interpreting time series predictions.

Convergence of Adam Under Relaxed Assumptions

Haochuan Li, Alexander Rakhlin, Ali Jadbabaie

https://openreview.net/forum?id=yEewbkBNzi

Keywords: Non-convex optimization, Adam, Convergence, Variance reduction

Compressor summary: The paper proves that the Adam algorithm for optimization converges to stationary points under realistic conditions and introduces a variance-reduced version with faster convergence.

Advancing Bayesian Optimization via Learning Correlated Latent Space

Seunghun Lee, Jaewon Chu, Sihyeon Kim, Juyeon Ko, Hyunwoo J. Kim

https://openreview.net/forum?id=yE62KM4qsO

Keywords: Bayesian optimization, smoothness regularization, variational autoencoder

Compressor summary: Correlated latent space Bayesian Optimization (CoBO) is a method that learns correlated latent spaces to reduce the gap between optimization in the latent space and the input space, achieving high performance on discrete data tasks with limited function evaluations.

Closing the gap between the upper bound and lower bound of Adam's iteration complexity

Bohan Wang, Jingwen Fu, Huishuai Zhang, Nanning Zheng, Wei Chen

https://openreview.net/forum?id=yDvb3mlogA

Keywords: Adam, Convergence, Upper Bound, Lower Bound

Compressor summary: This paper derives a new convergence guarantee for Adam optimization with smooth conditions and bounded noise variance, filling a gap in existing literature and providing a tight upper bound.

Credal Marginal MAP

Radu Marinescu, Debarun Bhattacharjya, Junkyu Lee, Fabio Cozman, Alexander G. Gray

https://openreview.net/forum?id=yCBqKTvYe9

Keywords: graphical models, credal networks, probabilistic inference

Compressor summary: Credal Marginal MAP inference is explored in this paper, with new exact and approximation methods proposed and evaluated on various problems.

Siamese Masked Autoencoders

Agrim Gupta, Jiajun Wu, Jia Deng, Li Fei-Fei

https://openreview.net/forum?id=yC3q7vInux

Keywords: Representation Learning, Visual Correspondence, Self-supervised learning, Videos

Compressor summary: Siamese Masked Autoencoders (SiamMAE) is a method for learning visual correspondence from videos by predicting missing patches in one frame based on another frame with masked patches, focusing on object motion and achieving competitive results on various tasks.

Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective

Yifei Wang, Liangchen Li, Jiansheng Yang, Zhouchen Lin, Yisen Wang

https://openreview.net/forum?id=yBoVwpGa5E

Keywords: Adversarial Training

Compressor summary: Adversarial Training (AT) has become arguably the state-of-the-art algorithm for extracting robust features. However, researchers recently notice that AT suffers from severe robust overfitting problems, particularly after learning rate (LR) decay. In this paper, we explain this phenomenon by viewing adversarial training as a dynamic minimax game between the model trainer and the attacker. Specifically, we analyze how LR decay breaks the balance between the minimax game by empowering the trainer with a stronger memorization ability, and show such imbalance induces robust overfitting as a result of memorizing non-robust features. We validate this understanding with extensive experiments, and provide a holistic view of robust overfitting from the dynamics of both the two game players. This understanding further inspires us to alleviate robust overfitting by rebalancing the two players by either regularizing the trainer's capacity or improving the attack strength. Experiments show that the proposed ReBalanced Adversarial Training (ReBAT) can attain good robustness and does not suffer from robust overfitting even after very long training. Code is available at https://github.com/PKU-ML/ReBAT.

Error Discovery By Clustering Influence Embeddings

Fulton Wang, Julius Adebayo, Sarah Tan, Diego Garcia-Olano, Narine Kokhlikyan

https://openreview.net/forum?id=yBVLXvJ1sb

Keywords: Debugging, interpretability, influence functions

Compressor summary: The authors introduce InfEmden, a slice discovery method that finds coherent groups of test examples where a model under-performs by using influence embeddings and K-Means clustering.

Operation-Level Early Stopping for Robustifying Differentiable NAS

Shen Jiang, Zipeng Ji, Guanghui Zhu, Chunfeng Yuan, Yihua Huang

https://openreview.net/forum?id=yAOwkf4FyL

Keywords: Differentiable neural architecture search; Image classification; Failure of DARTS

Compressor summary: The paper proposes a method called operation-level early stopping (OLES) to improve Differentiable NAS (DARTS) by addressing the overfitting issue caused by skip connections in neural architecture search.

SNEkhorn: Dimension Reduction with Symmetric Entropic Affinities

Hugues Van Assel, Titouan Vayer, Rémi Flamary, Nicolas Courty

https://openreview.net/forum?id=y9U0IJ2uFr

Keywords: Dimension Reduction, Optimal Transport, Affinities

Compressor summary: The paragraph discusses a novel way of symmetrizing entropic affinities using optimal transport, which improves clustering performance and robustness in dimensionality reduction algorithms like t-SNE.

Private Everlasting Prediction

Moni Naor, Kobbi Nissim, Uri Stemmer, Chao Yan

https://openreview.net/forum?id=y8UAQQHVTX

Keywords: Differential privacy, private learning, private prediction

Compressor summary: The paper proposes private everlasting prediction, a method to protect the privacy of both training data and queries in answering a stream of classification questions, with lower sample complexity than existing methods.

On the Importance of Exploration for Generalization in Reinforcement Learning

Yiding Jiang, J Zico Kolter, Roberta Raileanu

https://openreview.net/forum?id=y5duN2j9s6

Keywords: reinforcement learning, generalization, procgen, crafter

Compressor summary: The paper proposes EDE, a method that encourages exploration in deep RL by using an ensemble of Q-value distributions to handle epistemic uncertainty and improve generalization to new environments.

CSOT: Curriculum and Structure-Aware Optimal Transport for Learning with Noisy Labels

Wanxing Chang, Ye Shi, Jingya Wang

https://openreview.net/forum?id=y50AnAbKp1

Keywords: Learning with Noisy Labels, Optimal Transport, Curriculum Learning

Compressor summary: The paper proposes a novel optimal transport method, CSOT, to address challenges in learning with noisy labels by considering sample distribution structure and assigning reliable labels incrementally.

Learning Causal Models under Independent Changes

Sarah Mameche, David Kaltenpoth, Jilles Vreeken

https://openreview.net/forum?id=y0OlQSZsyp

Keywords: independent mechanisms, causal discovery, information theory, gaussian processes

Compressor summary: The paper proposes a method to discover causal relationships in systems with changing components using Gaussian Process models and independence assumptions.

WITRAN: Water-wave Information Transmission and Recurrent Acceleration Network for Long-range Time Series Forecasting

Yuxin Jia, Youfang Lin, Xinyan Hao, Yan Lin, Shengnan Guo, Huaiyu Wan

https://openreview.net/forum?id=y08bkEtNBK

Keywords: long-range time series forecasting, information transmission, long- and short-term repetitive patterns, global and local correlations

Compressor summary: The WITRAN framework captures semantic information for long-range time series forecasting using a bi-granular information transmission and horizontal vertical gated selective unit, while reducing time and memory complexity compared to previous methods.

Molecule Joint Auto-Encoding: Trajectory Pretraining with 2D and 3D Diffusion

weitao Du, Jiujiu Chen, Xuecang Zhang, Zhi-Ming Ma, Shengchao Liu

https://openreview.net/forum?id=xzmaFfw6oh

Keywords: Molecule Joint Auto-encoding, Molecule Joint Self-supervised Learning, Markov processes, contrastive learning, molecule representation learning

Compressor summary: The proposed method, MoleculeJAE, learns both 2D and 3D molecular information using a pretraining technique, achieving state-of-the-art results on various tasks.

Color Equivariant Convolutional Networks

Attila Lengyel, Ombretta Strafforello, Robert-Jan Bruintjes, Alexander Gielisse, Jan van Gemert

https://openreview.net/forum?id=xz8j3r3oUA

Keywords: color equivariance, equivariance, color robustness, equivariant convolutions

Compressor summary: The paper introduces CEConvs, a novel deep learning building block that allows shape feature sharing across different colors while preserving important color information, improving robustness to color changes and performance on various tasks.

Towards Hybrid-grained Feature Interaction Selection for Deep Sparse Network

Fuyuan Lyu, Xing Tang, Dugang Liu, Chen Ma, Weihong Luo, Liang Chen, xiuqiang He, Xue Liu

https://openreview.net/forum?id=xxfHMqNcum

Keywords: Feature Interaction Search, Deep Sparse Network

Compressor summary: The paper introduces a hybrid-grained feature interaction selection approach for deep sparse networks that targets both feature field and feature value, and shows its effectiveness on three benchmark datasets.

BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basis

Zelin Ni, Hang Yu, Shizhan Liu, Jianguo Li, Weiyao Lin

https://openreview.net/forum?id=xx3qRKvG0T

Keywords: time series forecasting, basis learning, self-supervised learning

Compressor summary: BasisFormer is a time series forecasting architecture that uses adaptive self-supervised learning, cross-attention, and similarity coefficients to create tailored bases for accurate predictions.

Unsupervised Polychromatic Neural Representation for CT Metal Artifact Reduction

Qing Wu, Lixuan Chen, Ce Wang, Hongjiang Wei, S Kevin Zhou, Jingyi Yu, Yuyao Zhang

https://openreview.net/forum?id=xx3QgKyghS

Keywords: Medical Image, Computed Tomography, Metal Arftiacts, Implicit Neural Representation, Unsupervised Learning

Compressor summary: The paragraph describes a novel polychromatic neural representation (Polyner) method to improve computed tomography (CT) imaging with metallic implants by modeling the metal artifact reduction problem from a nonlinear inverse perspective and achieving better performance than supervised methods.

Personalized Dictionary Learning for Heterogeneous Datasets

Geyu Liang, Naichen Shi, Raed Al Kontar, Salar Fattahi

https://openreview.net/forum?id=xw6Szwu4xz

Keywords: Dictionary Learning, Data Heterogeneity, Personalization

Compressor summary: The paper introduces PerDL, a problem where sparse linear representations are learned from heterogeneous datasets with shared and unique features, and proposes PerMA, an efficient meta-algorithm that can recover the global and local dictionaries for various learning tasks.

TIES-Merging: Resolving Interference When Merging Models

Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, Mohit Bansal

https://openreview.net/forum?id=xtaX3WyCj1

Keywords: Model Merging, Fusing, Collaborative Training, Robust Fine-tuning, Federated Learning

Compressor summary: TIES-Merging is a novel method to combine task-specific models into a multitask model by resolving parameter interference and exploiting sign information, which leads to better performance in various settings.

Faster Discrete Convex Function Minimization with Predictions: The M-Convex Case

Taihei Oki, Shinsaku Sakaue

https://openreview.net/forum?id=xtQ9IGRzIW

Keywords: algorithms with predictions, beyond the worst-case analysis of algorithms, time complexity, combinatorial optimization, discrete convex analysis, submodular functions

Compressor summary: The paper presents a framework to accelerate discrete optimization algorithms, especially laminar convex minimization, by using machine-learned predictions.

Adversarial Robustness in Graph Neural Networks: A Hamiltonian Approach

Kai Zhao, Qiyu Kang, Yang Song, Rui She, Sijie Wang, Wee Peng Tay

https://openreview.net/forum?id=xtADRDRsM2

Keywords: adversarial robustness, graph neural networks

Compressor summary: This paper explores how using conservative Hamiltonian neural flows can improve the adversarial robustness of graph neural networks, which are otherwise vulnerable to perturbations affecting both node features and graph topology.

QuIP: 2-Bit Quantization of Large Language Models With Guarantees

Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, Christopher De Sa

https://openreview.net/forum?id=xrk9g5vcXR

Keywords: Quantization, Large Language Models, Adaptive Rounding, Theoretical Guarantees

Compressor summary: The paper introduces QuIP, a new method for post-training parameter quantization in large language models that uses incoherence processing to improve results and reduce bits per weight needed.

FaceComposer: A Unified Model for Versatile Facial Content Creation

Jiayu Wang, Kang Zhao, Yifeng Ma, Shiwei Zhang, Yingya Zhang, Yujun Shen, Deli Zhao, Jingren Zhou

https://openreview.net/forum?id=xrK3QA9mLo

Keywords: diffusion model; talking face generation; face generation

Compressor summary: FaceComposer is a versatile generative model for creating facial content with text control and animation, based on the latent diffusion framework, and it includes a large-scale multi-modal face database and an easy-to-use interface.

Calibrating “Cheap Signals” in Peer Review without a Prior

Yuxuan Lu, Yuqing Kong

https://openreview.net/forum?id=xr3KAzboHY

Keywords: Peer prediction, Peer review, Calibration

Compressor summary: The paper proposes a one-shot noise calibration process for ranking papers fairly by predicting others' scores, which reduces error probability and outperforms average ratings.

Beyond Unimodal: Generalising Neural Processes for Multimodal Uncertainty Estimation

Myong Chol Jung, He Zhao, Joanna Dipnall, Lan Du

https://openreview.net/forum?id=xq1QvViDdW

Keywords: Uncertainty estimation, multimodality, neural processes

Compressor summary: MNPs are a novel approach for estimating uncertainty in multimodal data that adapts Neural Processes, providing robustness and reliability with faster computation.

StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners

Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, Dilip Krishnan

https://openreview.net/forum?id=xpjsOQtKqx

Keywords: representation learning, synthetic images, text-to-image models

Compressor summary: The paper explores using text-to-image models to generate synthetic images for learning visual representations, and shows that their method outperforms existing methods on large scale datasets.

Fitting trees to $\ell_1$-hyperbolic distances

Joon-Hyeok Yim, Anna Gilbert

https://openreview.net/forum?id=xo2lbfQE8I

Keywords: tree metric fitting, ultrametric fitting, $\ell_1$-hyperbolicity

Compressor summary: The paper proposes an algorithm to fit trees based on hyperbolicity values, which improves upon previous algorithms and reveals differences between real and synthetic data sets.

Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation

Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, BIN FU, Tao Chen, Gang YU, Shenghua Gao

https://openreview.net/forum?id=xmxgMij3LY

Keywords: Conditional 3D Shape Generation, Neural 3D Representation, 3D Reconstruction

Compressor summary: The paper proposes a new method for generating 3D shapes from 2D images or texts by aligning the representations of 3D shapes in different modalities and using two models to encode and decode them.

Text Alignment Is An Efficient Unified Model for Massive NLP Tasks

Yuheng Zha, Yichi Yang, Ruichen Li, Zhiting Hu

https://openreview.net/forum?id=xkkBFePoFn

Keywords: Text Alignment, Efficient Unified Model, NLU Tasks, Factual Consistency Evaluation, QA with Unanswerable Question

Compressor summary: The paper proposes a small and efficient text alignment model that performs well on various NLP tasks, including text entailment, similarity, question answering, and factual consistency, outperforming larger LLMs in some cases.

Asymptotics of Bayesian Uncertainty Estimation in Random Features Regression

Youngsoo Baek, Samuel Berchuck, Sayan Mukherjee

https://openreview.net/forum?id=xgzkuTGBTx

Keywords: asymptotics, random features model, Bayesian inference

Compressor summary: This paper studies how the uncertainty of Bayesian models relates to the performance of their maximum likelihood estimates in high-dimensional settings, finding agreement in certain regimes and similarity in others.

Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputs

Dmitry Chistikov, Matthias Englert, Ranko Lazic

https://openreview.net/forum?id=xgY4QcOiEZ

Keywords: implicit bias, implicit regularization, training dynamics, ReLU networks, gradient flow, theoretical analysis

Compressor summary: The paper investigates how one-hidden layer ReLU networks learn a single neuron task by gradient flow and shows that they implicitly minimize the rank of network parameters while converging to zero loss.

Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks

Steven Adriaensen, Herilalaina Rakotoarison, Samuel Müller, Frank Hutter

https://openreview.net/forum?id=xgTV6rmH6n

Keywords: learning curve extrapolation, prior-data fitted networks, transformers, Bayesian inference, uncertainty estimation, model selection

Compressor summary: The authors propose LC-PFN, a fast and accurate Bayesian method for learning curve extrapolation using prior-data fitted neural networks, which can improve model selection efficiency by enabling predictive early stopping.

Physics-Informed Bayesian Optimization of Variational Quantum Circuits

Kim Andrea Nicoli, Christopher J. Anders, Lena Funcke, Tobias Hartung, Karl Jansen, Stefan Kuhn, Klaus Robert Muller, Paolo Stornati, Pan Kessel, Shinichi Nakajima

https://openreview.net/forum?id=xfBeVGJwyL

Keywords: Bayesian optimization, Expected improvement, Quantum computing, Variational Quantum Eigensolvers

Compressor summary: The paper presents a new Bayesian optimization method for VQEs that uses a novel kernel and acquisition function to reduce uncertainty and improve performance.

Efficient Symbolic Policy Learning with Differentiable Symbolic Expression

Jiaming Guo, Rui Zhang, Shaohui Peng, Qi Yi, Xing Hu, Ruizhi Chen, Zidong Du, Xishan Zhang, Ling Li, Qi Guo, Yunji Chen

https://openreview.net/forum?id=xdQpmUPNHC

Keywords: reinforcement learning, context variables, symbolic policy

Compressor summary: The paper proposes ESPL, an efficient gradient-based method to learn simple and interpretable symbolic policies from scratch for sequential decision-making tasks using a differentiable symbolic expression and a path selector.

Towards Data-Agnostic Pruning At Initialization: What Makes a Good Sparse Mask?

Hoang Pham, The-Anh Ta, Shiwei Liu, Lichuan Xiang, Dung D. Le, Hongkai Wen, Long Tran-Thanh

https://openreview.net/forum?id=xdOoCWCYaY

Keywords: Pruning Neural Network, Sparsity, Neural Architecture Search

Compressor summary: The paper proposes a new framework and method for pruning neural networks at initialization that balances effective nodes and paths, improving accuracy and computational efficiency compared to existing methods.

Adversarial Resilience in Sequential Prediction via Abstention

Surbhi Goel, Steve Hanneke, Shay Moran, Abhishek Shetty

https://openreview.net/forum?id=xcGhx9FdxM

Keywords: Sequential prediction, adversarial examples, abstention, out-of-distribution, VC Classes

Compressor summary: The paper proposes a new model for sequential prediction that allows learners to abstain from making predictions on adversarial examples and designs learners with better error guarantees than existing methods.

On Evaluating Adversarial Robustness of Large Vision-Language Models

Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-man Cheung, Min Lin

https://openreview.net/forum?id=xbbknN9QFs

Keywords: Large Vision-Language Models, Adversarial Robustness

Compressor summary: The paragraph discusses evaluating the robustness of vision-language models (VLMs) to targeted adversarial examples and their potential security flaws before deployment.

Practical Equivariances via Relational Conditional Neural Processes

Daolang Huang, Manuel Haussmann, Ulpu Remes, S. T. John, Grégoire Clarté, Kevin Sebastian Luck, Samuel Kaski, Luigi Acerbi

https://openreview.net/forum?id=xax5eWeObb

Keywords: neural processes, equivariance, Gaussian processes

Compressor summary: Relational Conditional Neural Processes (RCNPs) are a new model that improves the performance of equivariant neural processes in higher dimensions for various machine learning tasks.

For SALE: State-Action Representation Learning for Deep Reinforcement Learning

Scott Fujimoto, Wei-Di Chang, Edward J. Smith, Shixiang Shane Gu, Doina Precup, David Meger

https://openreview.net/forum?id=xZvGrzRq17

Keywords: Deep reinforcement learning, representation learning

Compressor summary: The paper introduces SALE, a method for learning embeddings from low-level states that improve representation learning in reinforcement learning, and combines it with TD3 to create TD7, which performs better than existing continuous control algorithms on OpenAI gym benchmark tasks.

Learning Curves for Noisy Heterogeneous Feature-Subsampled Ridge Ensembles

Benjamin Samuel Ruben, Cengiz Pehlevan

https://openreview.net/forum?id=xXfDB8kJUs

Keywords: ridge regression, ensembling methods

Compressor summary: Feature bagging reduces prediction variance by combining predictions of many estimators trained on subsets or projections of features, and can mitigate double-descent in noisy least-squares ridge ensembles.

Robust Data Pruning under Label Noise via Maximizing Re-labeling Accuracy

Dongmin Park, Seola Choi, Doyoung Kim, Hwanjun Song, Jae-Gil Lee

https://openreview.net/forum?id=xWCp0uLcpG

Keywords: Data Pruning, Data Subset Selection, Noisy Labels, Relabeling, Self-training

Compressor summary: Prune4Rel is a novel data pruning algorithm that maximizes re-labeling accuracy and generalization performance by finding a subset of training examples with high prediction confidence of their neighbors in the subset.

Fair Graph Distillation

Qizhang Feng, Zhimeng Jiang, Ruiquan Li, Yicheng Wang, Na Zou, Jiang Bian, Xia Hu

https://openreview.net/forum?id=xW0ayZxPWs

Keywords: Graph Distillation, Algorithmic Fairness

Compressor summary: The paper proposes a method to create smaller graphs that are both accurate and fair for graph neural networks, by developing a new metric called coherence and using a bi-level optimization algorithm.

Rethinking Incentives in Recommender Systems: Are Monotone Rewards Always Beneficial?

Fan Yao, Chuanhao Li, Karthik Abinav Sankararaman, Yiming Liao, Yan Zhu, Qifan Wang, Hongning Wang, Haifeng Xu

https://openreview.net/forum?id=xUyBP16Q5J

Keywords: Recommender system, Mechanism design, Potential function, Optimization

Compressor summary: The text discusses how online content recommendation platforms affect media creators' choices and welfare, and proposes a new reward mechanism called Backward Rewarding Mechanisms to improve the platform's design.

Compact Neural Volumetric Video Representations with Dynamic Codebooks

Haoyu Guo, Sida Peng, Yunzhi Yan, Linzhan Mou, Yujun Shen, Hujun Bao, Xiaowei Zhou

https://openreview.net/forum?id=xTgM7XLN9P

Keywords: Computer Vision, 3D Vision, Volumetric Video

Compressor summary: The paper introduces a novel neural representation called dynamic codebook that compresses volumetric videos by merging similar features and compensating with dynamic codes, achieving better quality and storage efficiency.

Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphs

Peng Jin, Yang Wu, Yanbo Fan, Zhongqian Sun, Yang Wei, Li Yuan

https://openreview.net/forum?id=xSEhb2j3TK

Keywords: Text-driven Motion Synthesis, Diffusion Models, Graph networks

Compressor summary: The paper proposes using hierarchical semantic graphs to generate fine-grained human motion from text descriptions, improving over existing methods that may focus too much on action names and lack detail.

Robust Model Reasoning and Fitting via Dual Sparsity Pursuit

Xingyu Jiang, Jiayi Ma

https://openreview.net/forum?id=xRfTcZdQxq

Keywords: Model reasoning; Model fitting; Outliers; Sparse subspace learning; Feature matching

Compressor summary: The paper proposes a unified optimization model for outlier rejection, true model reasoning and parameter estimation, which uses sparse subspace recovery to search for independent bases in an over-embedded data space.

The Tunnel Effect: Building Data Representations in Deep Neural Networks

Wojciech Masarczyk, Mateusz Ostaszewski, Ehsan Imani, Razvan Pascanu, Piotr Miłoś, Tomasz Trzcinski

https://openreview.net/forum?id=xQOHOpe1Fv

Keywords: representation learning, continual learning, training dynamics

Compressor summary: The paper investigates how deep neural networks split into two parts during image classification training, where the initial layers create separable representations and the latter layers, called "the tunnel," compress them without significantly affecting performance.

Stability of Random Forests and Coverage of Random-Forest Prediction Intervals

Yan Wang, Huaiqing Wu, Dan Nettleton

https://openreview.net/forum?id=xPqINp0Eu1

Keywords: Stability, Prediction Intervals, Random Forests

Compressor summary: The paper shows how random forests are stable and can provide good interval predictions using out-of-bag error and weak assumptions.

Learning DAGs from Data with Few Root Causes

Panagiotis Misiakos, Chris Wendler, Markus Püschel

https://openreview.net/forum?id=xPLaXSuSvQ

Keywords: directed acyclic graph, few root causes, structural equation models, linear SEMs, additive noise

Compressor summary: The paper proposes a new algorithm for learning directed acyclic graphs (DAGs) from linear structural equation models (SEMs) with fewer root causes and noise, which improves upon previous methods.

CrossGNN: Confronting Noisy Multivariate Time Series Via Cross Interaction Refinement

Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, Yang Wang

https://openreview.net/forum?id=xOzlW2vUYc

Keywords: Time Series Forecasting;

Compressor summary: CrossGNN is a linear complexity GNN model that improves multivariate time series forecasting by refining cross-scale and cross-variable interactions using an adaptive multi-scale identifier and two specialized GNN components.

Proximity-Informed Calibration for Deep Neural Networks

Miao Xiong, Ailin Deng, Pang Wei Koh, Jiaying Wu, Shen Li, Jianqing Xu, Bryan Hooi

https://openreview.net/forum?id=xOJUmwwlJc

Keywords: Calibration, Uncertainty Estimation, Trustworthiness, Fairness, Multicalibration

Compressor summary: The text discusses a common issue in calibration algorithms called proximity bias, which causes models to be more overconfident in low proximity data, and proposes a new algorithm (ProCal) to address it.

RL-based Stateful Neural Adaptive Sampling and Denoising for Real-Time Path Tracing

Antoine Scardigli, Lukas Cavigelli, Lorenz K Muller

https://openreview.net/forum?id=xNyR7DXUzJ

Keywords: computer graphics, rendering, ray tracing, GPU acceleration, RL, spatiotemporal latent space

Compressor summary: The paper proposes a framework that uses reinforcement learning and neural networks to improve Monte-Carlo path tracing, reducing noise and rendering times for realistic image synthesis in real-time applications.

Optimal Algorithms for the Inhomogeneous Spiked Wigner Model

Alexander Pak, Justin Ko, Florent Krzakala

https://openreview.net/forum?id=xNUmTRYtV1

Keywords: Spectral Method, Community detection, Wigner Spike model, Random Matrix, BBP transition, Approximate Message Passing, Spin glasses, Statistical Physics

Compressor summary: The paper proposes an AMP algorithm for a spiked Wigner problem with inhomogeneous noise and compares it with a spectral method, showing good performance and matching information-theoretic results.

Hierarchical Multi-Agent Skill Discovery

Mingyu Yang, Yaodong Yang, Zhenbo Lu, Wengang Zhou, Houqiang Li

https://openreview.net/forum?id=xMgO04HDOS

Keywords: Multi-Agent Reinforcement Learning, Hierarchical Skill Discovery, Probabilistic Graphical Model

Compressor summary: The text describes a hierarchical algorithm called Hierarchical Multi-Agent Skill Discovery (HMASD) that learns team and individual skills for multi-agent tasks without extrinsic rewards, outperforming existing methods.

Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks

Minki Kang, Seanie Lee, Jinheon Baek, Kenji Kawaguchi, Sung Ju Hwang

https://openreview.net/forum?id=xJLEQQrFia

Keywords: language model, distillation, reasoning, knowledge augmentation

Compressor summary: KARD is a novel method that enhances small LMs with rationales from large and external sources, achieving better results on knowledge-intensive reasoning tasks.

Robust Bayesian Satisficing

Artun Saday, Y. Cahit Yıldırım, Cem Tekin

https://openreview.net/forum?id=xINPCvgULc

Keywords: robust satisficing, regret minimization, Gaussian processes

Compressor summary: The paper introduces RoBOS, a novel robust Bayesian satisficing algorithm for noisy black-box optimization that performs well under distributional shifts.

Towards Better Dynamic Graph Learning: New Architecture and Unified Library

Le Yu, Leilei Sun, Bowen Du, Weifeng Lv

https://openreview.net/forum?id=xHNzWHbklj

Keywords: dynamic graph learning, Transformer-based architecture, dynamic graph library

Compressor summary: DyGFormer is a new Transformer-based architecture for dynamic graph learning that leverages historical interactions and uses DyGLib, a unified library for rigorous evaluations.

State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understanding

Devleena Das, Sonia Chernova, Been Kim

https://openreview.net/forum?id=xGz0wAIJrS

Keywords: Concept-Based Explanations, Reinforcement Learning, Human-AI Interaction

Compressor summary: The paper proposes State2Explanation (S2E), a framework that learns to embed state-action pairs and concept-based explanations, which can improve both an RL agent's learning and users' understanding of the agent's decisions in sequential decision making tasks.

Boosting Spectral Clustering on Incomplete Data via Kernel Correction and Affinity Learning

Fangchen Yu, Runze Zhao, Zhan Shi, Yiwen Lu, Jicong Fan, Yicheng Zeng, Jianfeng Mao, Wenye Li

https://openreview.net/forum?id=xFtuNq23D5

Keywords: Spectral Clustering, Incomplete Data, Kernel Correction, Self-expressive Affinity Learning

Compressor summary: The paper proposes two new methods to improve spectral clustering on incomplete data by enhancing the kernel matrix and learning adaptive affinity matrices.

Kullback-Leibler Maillard Sampling for Multi-armed Bandits with Bounded Rewards

Hao Qin, Kwang-Sung Jun, Chicheng Zhang

https://openreview.net/forum?id=xF89MjFbWp

Keywords: multi-armed bandits, bounded rewards

Compressor summary: The paper analyzes KL-Maillard Sampling, a variant of Maillard Sampling that achieves finite-time gap-dependent regret bounds and asymptotic optimality in bandit problems with Bernoulli rewards.

Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning

Yingcong Li, Kartik Sreenivasan, Angeliki Giannou, Dimitris Papailiopoulos, Samet Oymak

https://openreview.net/forum?id=xEhKwsqxMa

Keywords: chain-of-thought, in-context learning, attention, compositional learning, approximation, length generalization

Compressor summary: The study examines how chain-of-thought (CoT) helps transformers learn compositional functions more efficiently by breaking down the learning process into two phases and reducing sample complexity.

LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching

Duy Minh Ho Nguyen, Hoang Nguyen, Nghiem Tuong Diep, Tan Ngoc Pham, Tri Cao, Binh T. Nguyen, Paul Swoboda, Nhat Ho, Shadi Albarqouni, Pengtao Xie, Daniel Sonntag, Mathias Niepert

https://openreview.net/forum?id=xE7oH5iVGK

Keywords: medical imaging; self-supervised learning; graph matching; large-vision model

Compressor summary: LVM-Med is a family of deep networks trained on large-scale medical datasets that uses a novel self-supervised contrastive learning algorithm to improve performance on various medical tasks.

Probabilistic inverse optimal control for non-linear partially observable systems disentangles perceptual uncertainty and behavioral costs

Dominik Straub, Matthias Schultheis, Heinz Koeppl, Constantin A. Rothkopf

https://openreview.net/forum?id=xDHzQQ4lnC

Keywords: inverse optimal control, probabilistic modeling, motor control, cognitive science

Compressor summary: The paper presents a probabilistic approach to inverse optimal control for partially observable stochastic non-linear systems with unobserved action signals, which can characterize behavior in sequential decision-making tasks under uncertainty and disentangle perceptual factors and behavioral costs.

SODA: Robust Training of Test-Time Data Adaptors

Zige Wang, Yonggang Zhang, Zhen Fang, Long Lan, Wenjing Yang, Bo Han

https://openreview.net/forum?id=xBqjoG0NxM

Keywords: test-time data adaptation, zeroth-order optimization, out-of-distribution generalization

Compressor summary: The paper proposes a method called SODA, which uses reliable labels and preserves data information, to improve test-time data adaptation and adapt deployed models to distribution shifts while maintaining privacy.

Do Not Marginalize Mechanisms, Rather Consolidate!

Moritz Willig, Matej Zečević, Devendra Singh Dhami, Kristian Kersting

https://openreview.net/forum?id=xBhvMu4J03

Keywords: Structural Causal Models, Marginalization, Consolidation, Compression

Compressor summary: Consolidating causal mechanisms helps simplify large-scale structural causal models while maintaining their causality and interventional behavior.

Thrust: Adaptively Propels Large Language Models with External Knowledge

Xinran Zhao, Hongming Zhang, Xiaoman Pan, Wenlin Yao, Dong Yu, Jianshu Chen

https://openreview.net/forum?id=x9FOu3W6iy

Keywords: knowledge-intensive natural language processing, pre-trained language models, instance-level adaptive knowledge usage

Compressor summary: The paper proposes IAPEK, a technique that adaptively retrieves external knowledge for PTLMs using Thrust, a metric that measures models' instance-level knowledgeability, achieving higher cost-efficiency and performance improvement on 26% of the tasks.

Recasting Continual Learning as Sequence Modeling

Soochan Lee, Jaehyeon Son, Gunhee Kim

https://openreview.net/forum?id=x816mCbWpR

Keywords: meta-continual learning, sequence modeling, Transformers, efficient Transformers

Compressor summary: The authors propose a novel approach to continual learning by framing it as a sequence modeling problem and demonstrate its effectiveness using Transformers and their efficient variants on various benchmarks.

Lending Interaction Wings to Recommender Systems with Conversational Agents

Jiarui Jin, Xianyu Chen, Fanghua Ye, Mengyue Yang, Yue Feng, Weinan Zhang, Yong Yu, Jun Wang

https://openreview.net/forum?id=x7q7w07r6Y

Keywords: Conversational Agent, Recommender System, Conversational Recommendation

Compressor summary: CORE is a framework for chatbots to improve online recommendations by minimizing uncertainty through querying attributes or items.

Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equations

Anudhyan Boral, Zhong Yi Wan, Leonardo Zepeda-Nunez, James Lottes, Qing Wang, Yi-Fan Chen, John Roberts Anderson, Fei Sha

https://openreview.net/forum?id=x6cOcxRnxG

Keywords: partial differential equations, physics, turbulence, stochastic differential equations, physical simulation, neural differential equations

Compressor summary: The authors propose a neural network approach that combines ideal large eddy simulation and stochastic differential equations to model turbulent flows, achieving better accuracy and stability than traditional methods.

FedNAR: Federated Optimization with Normalized Annealing Regularization

Junbo Li, Ang Li, Chong Tian, Qirong Ho, Eric Xing, Hongyi Wang

https://openreview.net/forum?id=x5fs7TXKDc

Keywords: Federated learning, weight decay, adaptive hyperparameters

Compressor summary: The paper proposes FedNAR, a method to improve federated learning by adjusting weight decay and gradient co-clipping, which enhances convergence speed and model accuracy.

Improving *day-ahead* Solar Irradiance Time Series Forecasting by Leveraging Spatio-Temporal Context

Oussama Boussif, Ghait Boukachab, Dan Assouline, Stefano Massaroli, Tianle Yuan, Loubna Benabbou, Yoshua Bengio

https://openreview.net/forum?id=x5ZruOa4ax

Keywords: Time series forecasting, multi-modal learning, solar irradiance, context-enriched learning

Compressor summary: The paper proposes a deep learning architecture using satellite data to improve solar irradiance forecasting accuracy and uncertainty estimation, as well as introducing a new multi-modal dataset for training and testing.

Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis

Dachao Lin, Yuze Han, Haishan Ye, Zhihua Zhang

https://openreview.net/forum?id=x5JCDCvR4b

Keywords: distributed optimization, convex optimization, second-order similarity, client sampling

Compressor summary: The paper proposes two new algorithms for distributed optimization problems with improved communication complexity and shows their effectiveness in smoothness-free and ill-conditioned scenarios.

Online Constrained Meta-Learning: Provable Guarantees for Generalization

Siyuan Xu, Minghui Zhu

https://openreview.net/forum?id=x2xQEszznV

Keywords: meta-learning; generalization

Compressor summary: The paper introduces an online meta-learning framework that learns from constrained tasks and analyzes its optimality gaps and constraint violations, with a practical algorithm and experiments on imitation learning and few-shot classification.

Taming Local Effects in Graph-based Spatiotemporal Forecasting

Andrea Cini, Ivan Marisca, Daniele Zambon, Cesare Alippi

https://openreview.net/forum?id=x2PH6q32LR

Keywords: time series forecasting, spatiotemporal forecasting, graph-based spatiotemporal forecasting, graph neural networks

Compressor summary: The paper explores the trade-off between global and local models in spatiotemporal forecasting using graph neural networks, and proposes a method to incorporate trainable node embeddings for better adaptation to different time series dynamics.

FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation

Jia Gu, Caizhi Tang, Han Yan, Qing Cui, Longfei Li, JUN ZHOU

https://openreview.net/forum?id=wzg0BsV8rQ

Keywords: Data fusion, heterogeneous treatment effects estimation, shrinkage estimation, tree-based method

Compressor summary: The paper introduces FAST, a method for estimating treatment effects that uses both trial and observational data to improve accuracy and robustness by combining optimal weighting and tree-based techniques.

Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking

Frederik Kunstner, Victor S. Portella, Mark Schmidt, Nick Harvey

https://openreview.net/forum?id=wzPcffMZ3b

Keywords: line-search, gradient descent, hypergradient, adaptive methods, smooth, convex, optimization, preconditioning

Compressor summary: The authors introduce multidimensional backtracking, a technique that finds optimal per-coordinate step-sizes for smooth convex problems by using hyper-gradients and cutting-plane methods.

Natural Actor-Critic for Robust Reinforcement Learning with Function Approximation

Ruida Zhou, Tao Liu, Min Cheng, Dileep Kalathil, Panganamala Kumar, Chao Tian

https://openreview.net/forum?id=wxkBdtDbmH

Keywords: robust reinforcement learning, policy-based approach, function approximation, actor-critic

Compressor summary: The authors propose two new uncertainty sets for large-scale robust reinforcement learning and a robust natural actor-critic algorithm that converges to the optimal policy in finite time.

Federated Learning with Bilateral Curation for Partially Class-Disjoint Data

Ziqing Fan, Ruipeng Zhang, Jiangchao Yao, Bo Han, Ya Zhang, Yanfeng Wang

https://openreview.net/forum?id=wwmKVO8bsR

Keywords: federated learning, data heterogeneity, partially class-disjoint data

Compressor summary: FedGELA is a novel approach for federated learning with partially class-disjoint data that uses a simplex ETF classifier to achieve fair and equal discrimination for all classes, improving performance and providing convergence guarantees.

Adapting Fairness Interventions to Missing Values

Raymond Feng, Flavio Calmon, Hao Wang

https://openreview.net/forum?id=wwkQUiaKbo

Keywords: algorithmic fairness, discrimination, missing values, machine learning

Compressor summary: The paper analyzes how missing values in data affect algorithmic fairness, showing that the standard "impute-then-classify" approach can worsen fairness and accuracy, and presents new algorithms that handle missing patterns while preserving their information for better fairness and accuracy.

Subject-driven Text-to-Image Generation via Apprenticeship Learning

Wenhu Chen, Hexiang Hu, YANDONG LI, Nataniel Ruiz, Xuhui Jia, Ming-Wei Chang, William W. Cohen

https://openreview.net/forum?id=wv3bHyQbX7

Keywords: Diffusion Model, Image Generation, Image Editing, In-Context Learning

Compressor summary: SuTI is a fast and high-quality text-to-image generator that learns from millions of online image clusters to produce customized images without fine-tuning for each subject.

Granger Components Analysis: Unsupervised learning of latent temporal dependencies

Jacek Dmochowski

https://openreview.net/forum?id=wqIm0Qsgy0

Keywords: components analysis, unsupervised learning, Granger Causality

Compressor summary: The paper introduces a new unsupervised learning technique for time series data that uses Granger causality to identify driving and driven signals in multivariate data sets.

ContinuAR: Continuous Autoregression For Infinite-Fidelity Fusion

WEI W. XING, Yuxin Wang, Zheng Xing

https://openreview.net/forum?id=wpfsnu5syT

Keywords: Gaussian process, autoregression, multi fidelity, nonparametric Bayesian

Compressor summary: The authors propose ContinuAR, a novel method for multi-fidelity fusion that improves efficiency and performance compared to existing techniques.

BayesDAG: Gradient-Based Posterior Inference for Causal Discovery

Yashas Annadani, Nick Pawlowski, Joel Jennings, Stefan Bauer, Cheng Zhang, Wenbo Gong

https://openreview.net/forum?id=woptnU6fh1

Keywords: Causal Discovery, Structure Learning, Bayesian Inference, Variational Inference, MCMC, Generative Model

Compressor summary: The text introduces a new Bayesian causal discovery method that samples DAGs from the posterior without regularization, using a combination of SG-MCMC and VI, and applies gradient-based MCMC sampling for the first time in this field.

Maximization of Average Precision for Deep Learning with Adversarial Ranking Robustness

Gang Li, Wei Tong, Tianbao Yang

https://openreview.net/forum?id=wm5Ane9VRO

Keywords: Adversarial Average Precision Maximization, Robust Average Precision, Adversarial Ranking Robustness, Adversarial Training

Compressor summary: The paper proposes a new method to optimize both average precision and adversarial robustness, outperforming current methods by a significant margin.

VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Models

Sheng-Yen Chou, Pin-Yu Chen, Tsung-Yi Ho

https://openreview.net/forum?id=wkIBfnGPTA

Keywords: backdoor, diffusion model, trustworthy

Compressor summary: The paper introduces a backdoor attack framework (VillanDiffusion) for diffusion models, which are generative AI models that can be vulnerable to output manipulation attacks when exposed to malicious patterns at the input.

Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Models

Lin Li, Jun Xiao, Guikun Chen, Jian Shao, Yueting Zhuang, Long Chen

https://openreview.net/forum?id=wiv21EJ0Vd

Keywords: Visual relation detection, Zero-short learning, Scene graph generation

Compressor summary: RECODE is a novel zero-shot VRD method that uses composite description prompts to improve relation detection by leveraging LLMs and spatial information.

Stein $\Pi$-Importance Sampling

Congye Wang, Wilson Ye Chen, Heishiro Kanagawa, Chris J. Oates

https://openreview.net/forum?id=wiidCRA3at

Keywords: Bayesian, discrepancy, kernel, sampling, Stein's method

Compressor summary: The paper proposes a new method for improving Monte Carlo output using Stein importance sampling and provides a novel variational construction and convergence conditions for it.

Outlier-Robust Wasserstein DRO

Sloan Nietert, Ziv Goldfeld, Soroosh Shafiee

https://openreview.net/forum?id=wg3d2FKAm8

Keywords: distributionally robust optimization, robust statistics, optimal transport, Wasserstein distance

Compressor summary: The paper proposes a new robust optimization approach that handles both geometric and non-geometric data perturbations and shows improved performance on regression and classification tasks.

Consistent Aggregation of Objectives with Diverse Time Preferences Requires Non-Markovian Rewards

Silviu Pitis

https://openreview.net/forum?id=wcdF6jR0Sp

Keywords: Normative Agency Design, Reward Design, Sequential Decision Making, Reinforcement Learning, Intertemporal Fairness, Multi-Objective Decision Making

Compressor summary: The paper shows that optimal AI agents must use non-Markovian rewards when serving multiple objectives with different discount factors, and proposes a practical aggregation scheme for this purpose.

Improved Best-of-Both-Worlds Guarantees for Multi-Armed Bandits: FTRL with General Regularizers and Multiple Optimal Arms

Tiancheng Jin, Junyan Liu, Haipeng Luo

https://openreview.net/forum?id=wbg4JEM5Jp

Keywords: multi-armed bandit, best of both worlds, Follow-the-Regularized-Leader, Tsallis entropy, Shannon entropy, Log-barrier

Compressor summary: The paper studies adaptive multi-armed bandit algorithms and improves the performance of FTRL with various regularizers and a new learning rate schedule, removing the uniqueness assumption and achieving better regret bounds.

High-dimensional Asymptotics of Denoising Autoencoders

Hugo Cui, Lenka Zdeborova

https://openreview.net/forum?id=wbbTqsiKzl

Keywords: statistical physics, replica method, autoencoder, exact asymptotics

Compressor summary: The paper proposes a two-layer non-linear autoencoder with tied weights and a skip connection for denoising high-dimensional data from a Gaussian mixture, and compares it to other methods.

Provable benefits of score matching

Chirag Pabbaraju, Dhruv Rohatgi, Anish Sevekari, Holden Lee, Ankur Moitra, Andrej Risteski

https://openreview.net/forum?id=waXoG35kbb

Keywords: theory, score matching, exponential families, sample complexity, computational hardness

Compressor summary: The paper presents a new exponential family of distributions where score matching is as easy to optimize as maximum likelihood, unlike other cases where score matching is harder or intractable.

Collaboratively Learning Linear Models with Structured Missing Data

Chen Cheng, Gary Cheng, John Duchi

https://openreview.net/forum?id=waDF0oACu2

Keywords: Collaborative Learning, Missing Data, Sensors, Linear Regression

Compressor summary: The paper proposes Collab, a distributed, semi-supervised algorithm for learning least squares estimates for m agents with different features, without communicating the labeled data, achieving near-optimal performance and generalizability.

Optimal Block-wise Asymmetric Graph Construction for Graph-based Semi-supervised Learning

Zixing Song, Yifei Zhang, Irwin King

https://openreview.net/forum?id=wYkfog48Bq

Keywords: Graph-based Semi-supervised Learning, Affinity Graph Construction

Compressor summary: This paper proposes an optimal asymmetric graph structure for label inference in graph-based semi-supervised learning that differentiates the roles of labeled and unlabeled nodes, leading to improved performance on synthetic and real datasets.

Language-driven Scene Synthesis using Multi-conditional Diffusion Model

An Dinh Vuong, Minh Nhat VU, Toan Tien Nguyen, Baoru Huang, Dzung Nguyen, Thieu Vo, Anh Nguyen

https://openreview.net/forum?id=wYKU1C77sa

Keywords: scene synthesis, language-driven, diffusion models, multi-conditional generation, 3D point cloud

Compressor summary: The paper proposes a new language-driven scene synthesis task that combines text, human motion, and objects, and presents a multi-conditional diffusion model to solve it effectively.

What can a Single Attention Layer Learn? A Study Through the Random Features Lens

Hengyu Fu, Tianyu Guo, Yu Bai, Song Mei

https://openreview.net/forum?id=wX8GuzDSJR

Keywords: transformers, attention, deep learning theory, random features

Compressor summary: The paper analyzes how attention layers in Transformers learn and generalize, showing they can efficiently learn certain functions with random features and have advantages over standard neural networks.

Transitivity Recovering Decompositions: Interpretable and Robust Fine-Grained Relationships

Abhra Chaudhuri, Massimiliano Mancini, Zeynep Akata, Anjan Dutta

https://openreview.net/forum?id=wUNPmdE273

Keywords: Interpretability, Robustness, Fine-Grained Representation Learning, Graph Theory, Information Theory

Compressor summary: The paper presents a method called TRD that converts abstract relational representations in image analysis into interpretable graphs and shows its effectiveness and robustness to noisy views.

Improving Diffusion-Based Image Synthesis with Context Prediction

Ling Yang, Jingwei Liu, Shenda Hong, Zhilong Zhang, Zhilin Huang, Zheming Cai, Wentao Zhang, Bin CUI

https://openreview.net/forum?id=wRhLd65bDt

Keywords: Diffusion Model, Image Generation

Compressor summary: ConPreDiff is a novel diffusion model that predicts neighborhood context for better image synthesis and achieves state-of-the-art results in text-to-image generation.

Critical Initialization of Wide and Deep Neural Networks using Partial Jacobians: General Theory and Applications

Darshil Doshi, Tianyu He, Andrey Gromov

https://openreview.net/forum?id=wRJqZRxDEX

Keywords: Criticality, Gaussian Process, Jacobian, LayerNorm, Residual connections, ResNet

Compressor summary: The authors propose a method to diagnose criticality in deep neural networks using partial Jacobians, which helps select optimal initialization and show that LayerNorm and residual connections lead to a critical architecture for any initialization.

Small batch deep reinforcement learning

Johan Samir Obando Ceron, Marc G Bellemare, Pablo Samuel Castro

https://openreview.net/forum?id=wPqEvmwFEh

Keywords: Reinforcement Learning, Deep Reinforcement Learning, Value based, Batch Size

Compressor summary: Reducing the batch size in value-based deep reinforcement learning can improve performance, contrary to the common belief that larger batches are better.

Progressive Ensemble Distillation: Building Ensembles for Efficient Inference

Don Dennis, Abhishek Shetty, Anish Sevekari, Kazuhito Koishida, Virginia Smith

https://openreview.net/forum?id=wNxyDofh74

Keywords: Edge computing, compression, efficient inference, distillation and inference, run-time tradeoff, inference-time tradeoff, on-device, user-side, client-side

Compressor summary: B-DISTIL is a method to create ensembles of smaller student models from a large pretrained teacher model using function composition based aggregation rules, achieving similar performance while allowing flexibility in accuracy vs. inference cost trade-offs.

Leveraging Early-Stage Robustness in Diffusion Models for Efficient and High-Quality Image Synthesis

Yulhwa Kim, Dongwon Jo, Hyesung Jeon, Taesu Kim, Daehyun Ahn, Hyungjun Kim, jae-joon kim

https://openreview.net/forum?id=wNpsGwixjG

Keywords: diffusion models, post-training quantization

Compressor summary: The paper proposes a faster noise estimation method for diffusion models by using low-bit activations in the early reverse diffusion process, while keeping high-bit activations for later stages to maintain image quality.

A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)

Weijie Tu, Weijian Deng, Tom Gedeon

https://openreview.net/forum?id=wMNpMe0vp3

Keywords: CLIP

Compressor summary: This paper investigates the safety measures of CLIP models, focusing on their resilience to visual factor variations, uncertainty estimations, and anomaly detection, by testing them on various datasets and conditions.

Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability

Maciej Falkiewicz, Naoya Takeishi, Imahn Shekhzadeh, Antoine Wehenkel, Arnaud Delaunoy, Gilles Louppe, Alexandros Kalousis

https://openreview.net/forum?id=wLiMhVJ7fx

Keywords: simulation-based inference, inverse problem, bayesian inference, uncertainty quantification, generative modeling

Compressor summary: Bayesian inference uses probability to express uncertainty, but existing methods can be overconfident; a new method adds calibration error to neural models for more accurate uncertainty quantification.

An Efficient and Robust Framework for Approximate Nearest Neighbor Search with Attribute Constraint

Mengzhao Wang, Lingwei Lv, Xiaoliang Xu, Yuxiang Wang, Qiang Yue, Jiongkang Ni

https://openreview.net/forum?id=wLFXTAWa5V

Keywords: approximate nearest neighbor search, attribute filtering, high-dimensional vector, proximity graph

Compressor summary: The paper presents a fast and accurate framework for hybrid query processing using composite indexes based on proximity graphs, improving both approximate nearest neighbor search and attribute filtering.

Learning a 1-layer conditional generative model in total variation

Ajil Jalal, Justin Kang, Ananya Uppal, Kannan Ramchandran, Eric Price

https://openreview.net/forum?id=wImYhdu4VF

Keywords: Generative models, distribution learning, maximum likelihood estimation

Compressor summary: A conditional generative model is a method to sample from a conditional distribution without assumptions on the input distribution, and it can learn deep models using few samples.

A Single-Loop Accelerated Extra-Gradient Difference Algorithm with Improved Complexity Bounds for Constrained Minimax Optimization

Yuanyuan Liu, Fanhua Shang, Weixin An, Junhao Liu, Hongying Liu, Zhouchen Lin

https://openreview.net/forum?id=wIlmx4bHrO

Keywords: Constrained Minimax Optimization; nonconvex- nonconcave

Compressor summary: The paper presents a new algorithm for solving constrained nonconvex-nonconcave problems that improves convergence rates and complexity bounds compared to existing methods.

Online Corrupted User Detection and Regret Minimization

Zhiyong Wang, Jize Xie, Tong Yu, Shuai Li, John C.S. Lui

https://openreview.net/forum?id=wHhPIv5G8Q

Keywords: online learning, online corrupted user detection, clustering of bandits

Compressor summary: The paper proposes an online learning problem and two novel algorithms to learn from user relations and detect corrupted users in web systems.

Restart Sampling for Improving Generative Processes

Yilun Xu, Mingyang Deng, Xiang Cheng, Yonglong Tian, Ziming Liu, Tommi S. Jaakkola

https://openreview.net/forum?id=wFuemocyHZ

Keywords: Generative models, diffusion models, PFGM, sampling

Compressor summary: Restart is a novel sampling algorithm that combines ODE and SDE methods to achieve faster and higher quality results in generative models.

Sharp Calibrated Gaussian Processes

Alexandre Capone, Sandra Hirche, Geoff Pleiss

https://openreview.net/forum?id=wFH5hZAwYz

Keywords: Gaussian Processes, Frequentist Statistics, Kernel Methods, Model Selection and Structure Learning, Regression

Compressor summary: The paragraph describes a new method for improving the uncertainty estimates of Gaussian processes by using a different set of hyperparameters that better satisfy calibration constraints and yield tighter predictive quantiles.

Improving Self-supervised Molecular Representation Learning using Persistent Homology

Yuankai Luo, Lei Shi, Veronika Thost

https://openreview.net/forum?id=wEiUGpcr0M

Keywords: Graph Neural Networks, Molecular Representation Learning, Persistent Homology, Contrastive Learning, Self-supervised Learning

Compressor summary: The paper proposes a self-supervised learning method for molecular representation learning using persistent homology, which offers unique features and improves predictive power, especially in small datasets.

Ambient Diffusion: Learning Clean Distributions from Corrupted Data

Giannis Daras, Kulin Shah, Yuval Dagan, Aravind Gollakota, Alex Dimakis, Adam Klivans

https://openreview.net/forum?id=wBJBLy9kBY

Keywords: corrupted data, generative models, ambient gan, inverse problems, learning from measurements

Compressor summary: The paper proposes a diffusion-based method to learn an unknown distribution using highly-corrupted samples, which can train generative models that are less likely to memorize individual samples and work on standard benchmarks with significant pixel loss or corrupted datasets.

Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approach

Yudi Zhang, Yali Du, Biwei Huang, Ziyan Wang, Jun Wang, Meng Fang, Mykola Pechenizkiy

https://openreview.net/forum?id=w7TyuWhGZP

Keywords: Reinforcement learning, sparse reward, return decomposition, causal modeling

Compressor summary: This paper proposes GRD, a framework that uses causal generative models to redistribute rewards in reinforcement learning for delayed reward scenarios and improves policy optimization by training over the most favorable state subspace.

InfoCD: A Contrastive Chamfer Distance Loss for Point Cloud Completion

Fangzhou Lin, Yun Yue, Ziming Zhang, Songlin Hou, Kazunori Yamada, Vijaya B Kolachalama, Venkatesh Saligrama

https://openreview.net/forum?id=w7LxAZfDfv

Keywords: Contrastive learning; Point cloud completion

Compressor summary: The paper introduces InfoCD, a contrastive Chamfer distance loss that improves point cloud completion and surface similarity estimation by maximizing mutual information between underlying surfaces, achieving state-of-the-art results on benchmark datasets.

Compositional Sculpting of Iterative Generative Processes

Timur Garipov, Sebastiaan De Peuter, Ge Yang, Vikas Garg, Samuel Kaski, Tommi S. Jaakkola

https://openreview.net/forum?id=w79RtqIyoM

Keywords: generative model composition, GFlowNets, diffusion models, classifier guidance, probabilistic methods

Compressor summary: The paper proposes Compositional Sculpting, a method for combining and sampling from iterative generative models like GFlowNets and diffusion models using classifier guidance.

Hyper-HMM: aligning human brains and semantic features in a common latent event space

Caroline Lee, Jane Han, Ma Feilong, Guo Jiahui, James Haxby, Christopher Baldassano

https://openreview.net/forum?id=w6krZiUa7t

Keywords: Brain Imaging, Other Cognitive Science, Other Neuroscience

Compressor summary: The Hyper-HMM is a hybrid model that aligns both temporal and spatial features across brains when processing naturalistic stimuli, allowing for mapping of individual cognitive dynamics and semantic content.

Minimax Optimal Rate for Parameter Estimation in Multivariate Deviated Models

Dat Do, Huy Nguyen, Khai Nguyen, Nhat Ho

https://openreview.net/forum?id=w3ghbKBJg4

Keywords: Mixture Model, Minimax Rate, Maximum Likelihood Estimation

Compressor summary: The paper studies how to estimate parameters in a multivariate model with a mixed density function, using a new condition called \emph{distinguishability} to deal with challenges in convergence rates.

Balanced Training for Sparse GANs

Yite Wang, Jing Wu, Naira Hovakimyan, Ruoyu Sun

https://openreview.net/forum?id=w2F8Fm6Sg3

Keywords: Dynamics sparse training; pruning; neural network pruning; empirical deep learning

Compressor summary: The paper introduces a new metric and method for balancing sparse training in generative adversarial networks (GANs) to improve efficiency without sacrificing performance.

Optimal Learners for Realizable Regression: PAC Learning and Online Learning

Idan Attias, Steve Hanneke, Alkis Kalavasis, Amin Karbasi, Grigoris Velegkas

https://openreview.net/forum?id=w116w62fxH

Keywords: Learning Theory, Regression, PAC Learning, Online Learning

Compressor summary: The authors study the complexity of learning realizable regression in different settings, introduce new dimensions to characterize learnability, and design optimal learners for these settings.

Visual Instruction Tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae Lee

https://openreview.net/forum?id=w0H2xGHlkw

Keywords: visual instruction tuning, instruction tuning, multimodal, LLM, GPT

Compressor summary: The paper introduces LLaVA, a multimodal model that uses language to generate image instructions for GPT-4, improving its performance on various tasks, including Science QA.

GAUCHE: A Library for Gaussian Processes in Chemistry

Ryan-Rhys Griffiths, Leo Klarner, Henry Moss, Aditya Ravuri, Sang T. Truong, Yuanqi Du, Samuel Don Stanton, Gary Tom, Bojana Ranković, Arian Rokkum Jamasb, Aryan Deshwal, Julius Schwartz, Austin Tripp, Gregory Kell, Simon Frieder, Anthony Bourached, Alex James Chan, Jacob Moss, Chengzhi Guo, Johannes P. Dürholt, Saudamini Chaurasia, Ji Won Park, Felix Strieth-Kalthoff, Alpha Lee, Bingqing Cheng, Alan Aspuru-Guzik, Philippe Schwaller, Jian Tang

https://openreview.net/forum?id=vzrA6uqOis

Keywords: Gaussian processes, Bayesian optimization, Chemistry, Molecular Machine Learning, Applications, Software

Compressor summary: GAUCHE is an open-source library that uses Gaussian processes to optimize molecular representations for applications like molecular discovery, chemical reaction optimization, and protein design.

Adaptive whitening with fast gain modulation and slow synaptic plasticity

Lyndon Duong, Eero P Simoncelli, Dmitri Chklovskii, David Lipshutz

https://openreview.net/forum?id=vz7SdRqWGM

Keywords: neuroscience, adaptation, whitening, efficient coding, recurrent neural network, gain modulation, synaptic plasticity, local learning rules

Compressor summary: The paragraph describes a new model that combines synaptic plasticity and gain modulation to rapidly adapt sensory neurons to changing input statistics.

Learning from Rich Semantics and Coarse Locations for Long-tailed Object Detection

Lingchen Meng, Xiyang Dai, Jianwei Yang, Dongdong Chen, Yinpeng Chen, Mengchen Liu, Yi-Ling Chen, Zuxuan Wu, Lu Yuan, Yu-Gang Jiang

https://openreview.net/forum?id=vybQs1Gbuk

Keywords: Long-tail object detection, visual semantics, soft supervision

Compressor summary: RichSem is a simple and effective method for long-tailed object detection that uses semantic branch to learn rich semantics from coarse locations as additional soft supervision without needing accurate bounding boxes.

Optimal Regret Is Achievable with Bounded Approximate Inference Error: An Enhanced Bayesian Upper Confidence Bound Framework

Ziyi Huang, Henry Lam, Amirhossein Meisami, Haofeng Zhang

https://openreview.net/forum?id=vwr4bHHsRT

Keywords: Bayesian bandits, approximate Bayesian inference, Bayesian Upper Confidence Bound, optimal regret order, bounded inference error

Compressor summary: The EBUCB framework improves on existing Bayesian bandit algorithms by achieving optimal regret bounds under constant approximate inference error using two different $\alpha$-divergences, while previous methods only had worst-case linear regret with one $\alpha$-divergence.

InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi

https://openreview.net/forum?id=vvoWPYqZJA

Keywords: Vision-Language Models, Instruction Tuning, Zero-shot

Compressor summary: The paper explores vision-language instruction tuning using pretrained BLIP-2 models and introduces a Query Transformer, achieving state-of-the-art results on various tasks.

Train Once, Get a Family: State-Adaptive Balances for Offline-to-Online Reinforcement Learning

Shenzhi Wang, Qisen Yang, Jiawei Gao, Matthieu Gaetan Lin, HAO CHEN, Liwei Wu, Ning Jia, Shiji Song, Gao Huang

https://openreview.net/forum?id=vtoY8qJjTR

Keywords: reinforcement learning, offline-to-online reinforcement learning, offline reinforcement learning, policy improvement, policy constraint

Compressor summary: FamO2O is a framework that adapts improvement and constraint balances for offline-to-online reinforcement learning based on each state, improving performance over existing methods.

The Geometry of Neural Nets' Parameter Spaces Under Reparametrization

Agustinus Kristiadi, Felix Dangel, Philipp Hennig

https://openreview.net/forum?id=vtLNwa6uX0

Keywords: neural network, invariance, equivariance, reparametrization, riemannian geometry, parameter space

Compressor summary: The paper studies how to represent the metric of neural nets under reparametrization using Riemannian geometry, which has implications for flatness analysis and optimization.

Revisiting the Minimalist Approach to Offline Reinforcement Learning

Denis Tarasov, Vladislav Kurenkov, Alexander Nikulin, Sergey Kolesnikov

https://openreview.net/forum?id=vqGWslLeEw

Keywords: Offline Reinforcement Learning

Compressor summary: The authors analyze recent offline reinforcement learning algorithms, propose ReBRAC that integrates several design elements, and show its state-of-the-art performance in various settings and datasets.

Online PCA in Converging Self-consistent Field Equations

Xihan Li, Xiang Chen, Rasul Tutunov, Haitham Bou Ammar, Lei Wang, Jun Wang

https://openreview.net/forum?id=vq11gurmUY

Keywords: Self-consistent Field Equation, Computational Science, Online PCA

Compressor summary: The authors propose a novel approach to solve the SCF equation using online PCA techniques, which improves convergence and applies to electronic structure problems.

TransHP: Image Classification with Hierarchical Prompting

Wenhao Wang, Yifan Sun, Wei Li, Yi Yang

https://openreview.net/forum?id=vpQuCsZXz2

Keywords: hierarchical image classification, hierarchical prompting, vision transformer

Compressor summary: The paper proposes a novel hierarchical image classification method that uses tokenized ancestor-class hints to improve accuracy, efficiency, and explainability of image classification models.

Combinatorial Optimization with Policy Adaptation using Latent Space Search

Felix Chalumeau, Shikha Surana, Clément Bonnet, Nathan Grinsztajn, Arnu Pretorius, Alexandre Laterre, Thomas D Barrett

https://openreview.net/forum?id=vpMBqdt9Hl

Keywords: Reinforcement Learning, Combinatorial Optimization, TSP, CVRP, JSSP

Compressor summary: COMPASS is a novel RL approach that pre-trains diverse policies in a latent space to improve search performance for combinatorial optimization problems.

Conditional score-based diffusion models for Bayesian inference in infinite dimensions

Lorenzo Baldassari, Ali Siahkoohi, Josselin Garnier, Knut Solna, Maarten V. de Hoop

https://openreview.net/forum?id=voG6nEW9BV

Keywords: score-based generative models, diffusion models, inverse problems, bayesian inference, infinite dimensions

Compressor summary: The paper proposes a new method for solving infinite-dimensional Bayesian linear inverse problems using amortized conditional score-based diffusion models, which improves upon previous heuristic approaches by being more theoretically grounded and computationally efficient.

Higher-Order Uncoupled Dynamics Do Not Lead to Nash Equilibrium - Except When They Do

Sarah Asad Toonsi, Jeff S Shamma

https://openreview.net/forum?id=vnTUuecp2v

Keywords: Learning in games, Nash equilibrium, Uncoupled Dynamics

Compressor summary: The paper introduces higher-order gradient play dynamics for multi-agent learning that can lead to Nash Equilibrium or fail depending on the game and dynamics.

On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective

Zeke Xie, zhiqiang xu, Jingzhao Zhang, Issei Sato, Masashi Sugiyama

https://openreview.net/forum?id=vnGcubtzR1

Keywords: Weight Decay, Regularization, Optimization, Deep Learning

Compressor summary: The paper proposes a new method called Scheduled Weight Decay (SWD) to reduce large gradient norms caused by weight decay, which can improve convergence and generalization in deep neural networks.

A Novel Approach for Effective Multi-View Clustering with Information-Theoretic Perspective

Chenhang Cui, Yazhou Ren, Jingyu Pu, Jiawei Li, Xiaorong Pu, Tianyi Wu, Yutao Shi, Lifang He

https://openreview.net/forum?id=vlDbqzwczj

Keywords: multi-view learning， clustering

Compressor summary: SUMVC is a new information-theoretic approach for multi-view clustering that improves performance by reducing redundancy and enhancing consistent information across views.

Structured Voronoi Sampling

Afra Amini, Li Du, Ryan Cotterell

https://openreview.net/forum?id=vf77fTbgG3

Keywords: Natural Language Processing, Text Generation, Controlled Generation, MCMC, HMC, Langevin Dynamics

Compressor summary: The paper introduces Structured Voronoi Sampling (SVS), a principled gradient-based technique for sampling from language models that generates fluent and diverse texts while adhering to control targets.

Extremal Domain Translation with Neural Optimal Transport

Milena Gazdieva, Alexander Korotin, Daniil Selikhanovych, Evgeny Burnaev

https://openreview.net/forum?id=vZRiMjo826

Keywords: optimal transport, partial optimal transport, neural networks, domain translation

Compressor summary: The paper introduces extremal transport, a method for unpaired image domain translation based on neural optimal transport and inspired by theoretical limits of possible translations.

ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning

Junguang Jiang, Baixu Chen, Junwei Pan, Ximei Wang, Dapeng Liu, jie jiang, Mingsheng Long

https://openreview.net/forum?id=vZHk1QlBQW

Keywords: Auxiliary-Task Learning, Negative Transfer

Compressor summary: ForkMerge is a novel approach that helps mitigate negative transfer in auxiliary-task learning by periodically forking the model into multiple branches and dynamically merging them based on task weights.

A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensing

Junren Chen, Jonathan Scarlett, Michael Ng, Zhaoqiang Liu

https://openreview.net/forum?id=vUXNNLatFv

Keywords: Compressed sensing, generative models, nonlinearity, uniform recovery

Compressor summary: The paper presents a unified framework for uniform recovery guarantees in nonlinear generative compressed sensing, handling discontinuous or unknown observation models, and achieves similar performance to existing non-uniform results with little additional cost.

Faster Margin Maximization Rates for Generic Optimization Methods

Guanghui Wang, Zihao Hu, Vidya Muthukumar, Jacob Abernethy

https://openreview.net/forum?id=vTug54Uunq

Keywords: Implicit bias, margin maximization, zero-sum game, online learning

Compressor summary: The paper proposes new techniques to improve the implicit bias rates of generic optimization methods like mirror descent and steepest descent in binary classification problems, using a unified framework based on regularized bilinear games.

Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity

Zhanpeng Zhou, Yongyi Yang, Xiaojiang Yang, Junchi Yan, Wei Hu

https://openreview.net/forum?id=vORUHrVEnH

Keywords: Linear Mode Connectivity, Permutation Invariance, Optimization Landscape, Science of Deep Learning

Compressor summary: This paper introduces a concept called Layerwise Linear Feature Connectivity (LLFC), which shows how different neural networks can be connected by linear paths in their feature maps, based on the Linear Mode Connectivity phenomenon.

Data Pruning via Moving-one-Sample-out

Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, XIAOJUAN QI

https://openreview.net/forum?id=vO6ZdPWaHc

Keywords: Data Valuation, Deep Learning, Data Pruning, Coreset Selection.

Compressor summary: The paper presents MoSo, a data-pruning method that uses gradient information to identify and remove uninformative samples from the training set, improving performance at high pruning ratios.

Imitation Learning from Imperfection: Theoretical Justifications and Algorithms

Ziniu Li, Tian Xu, Zeyu Qin, Yang Yu, Zhi-Quan Luo

https://openreview.net/forum?id=vO04AzsB49

Keywords: imitation learning, distribution shift, policy optimization, data selection

Compressor summary: The paper presents a framework for imitation learning with supplementary data that addresses distribution shift issues using importance sampling to improve performance in various tasks.

Suggesting Variable Order for Cylindrical Algebraic Decomposition via Reinforcement Learning

Fuqi Jia, Yuhang Dong, Minghao Liu, Pei Huang, Feifei Ma, Jian Zhang

https://openreview.net/forum?id=vNsdFwjPtL

Keywords: Reinforcement Learning, Graph Neural Network, Cylindrical Algebraic Decomposition.

Compressor summary: The paper introduces two reinforcement learning methods with graph neural networks for suggesting variable order in cylindrical algebraic decomposition, improving efficiency and generalizing well to different datasets.

Exploiting Correlated Auxiliary Feedback in Parameterized Bandits

Arun Verma, Zhongxiang Dai, Yao Shu, Bryan Kian Hsiang Low

https://openreview.net/forum?id=vM5VnNQ4n7

Keywords: Parameterized Bandits, Auxiliary Feedback, Control Variate, Regret Minimization

Compressor summary: The paper proposes a method that uses auxiliary feedback to improve reward estimation, leading to reduced regret in parameterized bandits problems.

Transformer as a hippocampal memory consolidation model based on NMDAR-inspired nonlinearity

Dong-Kyum Kim, Jea Kwon, Meeyoung Cha, C. Justin Lee

https://openreview.net/forum?id=vKpVJxplmB

Keywords: Transformer, NMDA, long-term memory, reference memory, memory consolidation

Compressor summary: The paper proposes a new activation function for deep learning models that mimics the hippocampus's NMDA receptor dynamics, improving working memory and spatial representation in transformers.

Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization

Ruichen Jiang, Aryan Mokhtari

https://openreview.net/forum?id=vIGNYQ4Alv

Keywords: convex optimization, quasi-Newton methods, Monteiro-Svaiter acceleration, Nesterov's accelerated gradient, online learning

Compressor summary: The paper presents a new optimization method that beats Nesterov's accelerated gradient in certain regimes and achieves optimal or faster convergence rates based on an online learning approach.

Accelerated Zeroth-order Method for Non-Smooth Stochastic Convex Optimization Problem with Infinite Variance

Nikita Kornilov, Ohad Shamir, Aleksandr Lobanov, Darina Dvinskikh, Alexander Gasnikov, Innokentiy Andreevich Shibaev, Eduard Gorbunov, Samuel Horváth

https://openreview.net/forum?id=vHSQTEIFkp

Keywords: stochastic optimization, gradient-free optimization, zero-order oracle, gradient clipping, infinite variance

Compressor summary: The paper presents an optimal algorithm for non-smooth stochastic convex optimization with infinite noise variance, adapted from a clipped accelerated gradient method.

Generalized Weighted Path Consistency for Mastering Atari Games

Dengwei Zhao, Shikui Tu, Lei Xu

https://openreview.net/forum?id=vHRLS8HhK1

Keywords: Monte Carlo Tree Search, Reinforcement learning, Path consistency.

Compressor summary: The paper proposes GW-PCZero, a generalized version of PCZero that uses a weighting mechanism to reduce variance and improve learning efficiency in reinforcement learning with neural-guided MCTS for real applications with non-zero immediate reward.

Self-supervised video pretraining yields robust and more human-aligned visual representations

Nikhil Parthasarathy, S. M. Ali Eslami, Joao Carreira, Olivier J Henaff

https://openreview.net/forum?id=vF8ukt5l1R

Keywords: self-supervised learning, contrastive, video pretraining, representation learning, visual representation, human alignment, robustness, shape-bias, saliency

Compressor summary: Video pretraining using VITO can learn visually general, robust, and human-like representations from complex transformations in videos.

On Class Distributions Induced by Nearest Neighbor Graphs for Node Classification of Tabular Data

Federico Errica

https://openreview.net/forum?id=vEzcRdiTkP

Keywords: Deep Graph Networks, Graph Neural Networks, Graph Representation Learning, Nearest Neighbors, Node Classification, Tabular Data

Compressor summary: The paper questions the benefits of using nearest neighbor graphs as a way to represent missing graph structures in machine learning problems and suggests exploring other techniques instead.

Neural Algorithmic Reasoning Without Intermediate Supervision

Gleb Rodionov, Liudmila Prokhorenkova

https://openreview.net/forum?id=vBwSACOB3x

Keywords: neural algorithmic reasoning, graph neural networks, self-supervised regularization

Compressor summary: This paper proposes an unsupervised method to learn neural algorithms that can generalize to larger input sizes and achieve state-of-the-art results on classic algorithmic problems like sorting.

An Exploration-by-Optimization Approach to Best of Both Worlds in Linear Bandits

Shinji Ito, Kei Takemura

https://openreview.net/forum?id=vBHKSTgcYQ

Keywords: bandit, linear bandit, best of both worlds, exploration by optimization

Compressor summary: The paper presents a linear bandit algorithm that performs near-optimally in both stochastic and adversarial settings, using exploration by optimization with specific regret bounds depending on dimension, time horizon, and sub-optimality gap.

Reflexion: language agents with verbal reinforcement learning

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, Shunyu Yao

https://openreview.net/forum?id=vAElhFcKW6

Keywords: language model, reasoning, decision making, programming

Compressor summary: Reflexion is a framework that helps language agents learn from feedback by reflecting on it verbally and storing it in memory for better decision-making in future trials, improving performance across various tasks.

MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion

Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, Yasutaka Furukawa

https://openreview.net/forum?id=vA0vj1mY77

Keywords: multiview; image generation; generative model; diffusion models

Compressor summary: MVDiffusion is a method for generating multi-view images with high resolution and rich content by concurrently interacting with all views using a correspondence-aware attention mechanism, which includes modules for generation, interpolation, and super-resolution.

Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model

Peter Súkeník, Marco Mondelli, Christoph H Lampert

https://openreview.net/forum?id=v9yC7sSXf3

Keywords: neural collapse, unconstrained features model, deep learning

Compressor summary: The paper studies deep neural collapse, a phenomenon where neural networks lose structure in earlier layers, and proposes a generalized analytical framework to explain it.

Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterion

Taehyun Cho, Seungyub Han, Heesoo Lee, Kyungjae Lee, Jungwoo Lee

https://openreview.net/forum?id=v8u3EFAyW9

Keywords: distributional reinforcement learning, risk

Compressor summary: The paper proposes a new distributional reinforcement learning method that uses randomized risk criterion for exploration and proves its convergence, optimality, and effectiveness in various environments.

End-To-End Latent Variational Diffusion Models for Inverse Problems in High Energy Physics

Alexander Shmakov, Kevin Greif, Michael James Fenton, Aishik Ghosh, Pierre Baldi, Daniel Whiteson

https://openreview.net/forum?id=v7WWesSiOu

Keywords: Diffusion, Variational, VAE, LDM, Physics, Unfolding

Compressor summary: The paragraph discusses a new deep learning method for correcting detector effects in particle physics measurements at the Large Hadron Collider, which outperforms existing methods in accuracy and efficiency.

Adaptive Algorithms for Relaxed Pareto Set Identification

Cyrille Kone, Emilie Kaufmann, Laura Richert

https://openreview.net/forum?id=v6jIxRRDyD

Keywords: bandit, pure-exploration, pareto front, pareto set

Compressor summary: The paper proposes a new sampling strategy called Adaptive Pareto Exploration that can identify a relevant subset of the Pareto optimal set in multi-objective bandit problems, and demonstrates its effectiveness on vaccination strategies against Covid-19.

Deep Equilibrium Based Neural Operators for Steady-State PDEs

Tanya Marwah, Ashwini Pokle, J Zico Kolter, Zachary Chase Lipton, Jianfeng Lu, Andrej Risteski

https://openreview.net/forum?id=v6YzxwJlQn

Keywords: Deep Equilibrium Models, Partial Differential Equations, Neural Operators

Compressor summary: The authors propose FNO-DEQ, a deep learning architecture that solves steady-state PDEs more efficiently and robustly than existing methods by using weight-tied neural networks and exploiting the fixed point nature of their solutions.

Winner Takes It All: Training Performant RL Populations for Combinatorial Optimization

Nathan Grinsztajn, Daniel Furelos-Blanco, Shikha Surana, Clément Bonnet, Thomas D Barrett

https://openreview.net/forum?id=v6VpqGcGAR

Keywords: Combinatorial Optimization, Reinforcement Learning, TSP, CVRP, JSSP

Compressor summary: The paper introduces Poppy, a training method for populations of reinforcement learning policies that induce unsupervised specialization to solve hard combinatorial optimization problems effectively.

Improving Adversarial Robustness via Information Bottleneck Distillation

Huafeng Kuang, Hong Liu, YONGJIAN WU, Shin'ichi Satoh, Rongrong Ji

https://openreview.net/forum?id=v5Aaxk4sSy

Keywords: Information Bottleneck, Adversarial training, Adversarial robustness, Knowledge distillation

Compressor summary: The study proposes an Information Bottleneck Distillation approach that uses prior knowledge from a pre-trained model to improve the information bottlenecks and robustness of deep neural networks against adversarial attacks.

UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild

Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, Stefano Ermon, Yun Fu, Ran Xu

https://openreview.net/forum?id=v54eUIayFh

Keywords: Image Generation, Multi-modal, HyperNet

Compressor summary: UniControl is a new generative foundation model that can handle various visual conditions and generate images with precise controls, while still allowing for arbitrary language prompts.

MonoUNI: A Unified Vehicle and Infrastructure-side Monocular 3D Object Detection Network with Sufficient Depth Clues

Jinrang Jia, Zhenjia Li, Yifeng Shi

https://openreview.net/forum?id=v2oGdhbKxi

Keywords: 3D detection, deep learning, autonomous driving

Compressor summary: The paper proposes a unified optimization target called normalized depth for monocular 3D detection of vehicle and infrastructure sides in autonomous driving, and introduces 3D normalized cube depth to improve accuracy.

H2RBox-v2: Incorporating Symmetry for Boosting Horizontal Box Supervised Oriented Object Detection

Yi Yu, Xue Yang, Qingyun Li, Yue Zhou, Feipeng Da, Junchi Yan

https://openreview.net/forum?id=v1VVKaMYbk

Keywords: Oriented object detection, self-supervised learning

Compressor summary: H2RBox-v2 is a new method for oriented object detection that uses reflection symmetry and self-supervision to improve performance and reduce the need for high-quality annotations.

Egocentric Planning for Scalable Embodied Task Achievement

Xiaotian Liu, Hector Palacios, Christian Muise

https://openreview.net/forum?id=v0lkbp66Uw

Keywords: Embodied AI, High-Level Actions, Symbolic Reasoning, Replanning, ALFRED Challenge, Flexible Task Achievement, User-Goal Understanding, Object Types and Actions, Perception Grounding

Compressor summary: Egocentric Planning combines symbolic planning and Object-oriented POMDPs to solve complex domestic tasks with visual perception and natural language processing, achieving a high success rate in ALFRED benchmark and winning the CVPR Embodied AI workshop challenge.

Temporal Continual Learning with Prior Compensation for Human Motion Prediction

Jianwei Tang, Jiangxin Sun, Xiaotong Lin, lifang zhang, Wei-Shi Zheng, Jian-Fang Hu

https://openreview.net/forum?id=v0GzRLvVp3

Keywords: Human Motion Prediction; Temporal Continual Learning; Prior Compensation Factor

Compressor summary: The paper proposes a new framework called Temporal Continual Learning (TCL) for human motion prediction that better preserves prior information and adapts to different models, datasets, and applications.

Online robust non-stationary estimation

Abishek Sankararaman, Murali Balakrishnan

https://openreview.net/forum?id=uzOBDerK1j

Keywords: Estimation, heavy-tails, distribution shifts, regret

Compressor summary: The text describes a new algorithm for estimating time-varying parameters from noisy and corrupted high-dimensional data-streams that is adaptive to drift, robust to heavy-tails and corruptions, requires no distributional knowledge, and can be implemented online.

Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers

Sotiris Anagnostidis, Dario Pavllo, Luca Biggio, Lorenzo Noci, Aurelien Lucchi, Thomas Hofmann

https://openreview.net/forum?id=uvdJgFFzby

Keywords: Transformers, Context-pruning, Efficient Transformer

Compressor summary: The study proposes a method that prunes uninformative tokens from context during generation, reducing computational and memory requirements while maintaining model expressiveness and interpretability.

Training Your Image Restoration Network Better with Random Weight Network as Optimization Function

Man Zhou, Naishan Zheng, Yuan Xu, Chun-Le Guo, Chongyi Li

https://openreview.net/forum?id=uv3ge0goPa

Keywords: Image restoration, low-light image enhancement, image de-noising

Compressor summary: The study proposes using random weight networks as constraints for training better image restoration networks, and explores four prototypes with different strategies and variants.

Can semi-supervised learning use all the data effectively? A lower bound perspective

Alexandru Tifrea, Gizem Yüce, Amartya Sanyal, Fanny Yang

https://openreview.net/forum?id=utreNaM1VY

Keywords: semi-supervised learning, statistical lower bound

Compressor summary: The paragraph discusses the limitations of existing theory on semi-supervised learning (SSL) algorithms, which may not always improve upon unsupervised and supervised learning algorithms, but can still perform well in real-world scenarios.

All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation

Liyao Tang, Zhe Chen, Shanshan Zhao, Chaoyue Wang, Dacheng Tao

https://openreview.net/forum?id=utQms7PPx5

Keywords: point cloud segmentation, weak supervision

Compressor summary: The paper proposes ERDA, a learning strategy that uses entropy regularization and distribution alignment to improve weakly supervised 3D segmentation tasks with pseudo-labels, achieving state-of-the-art performance with minimal annotations.

Fused Gromov-Wasserstein Graph Mixup for Graph-level Classifications

Xinyu Ma, Xu Chu, Yasha Wang, Yang Lin, Junfeng Zhao, Liantao Ma, Wenwu Zhu

https://openreview.net/forum?id=uqkUguNu40

Keywords: Graph Data Augmentation, Graph Mixup, Fused Gromov Wasserstein

Compressor summary: The paper introduces FGWMixup, a graph data augmentation method that uses optimal transport to find an optimal inter-graph node matching strategy, improving the generalizability and robustness of GNNs.

Optimal approximation using complex-valued neural networks

Paul Geuchen, Felix Voigtlaender

https://openreview.net/forum?id=uotGmrcooz

Keywords: complex-valued neural networks, approximation rates

Compressor summary: The paper analyzes the expressivity and approximation properties of complex-valued neural networks (CVNNs) with various activation functions, showing that their error scales as $m^{-k/(2n)}$ for smooth target functions and is optimal under a continuity assumption.

PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning

Neeratyoy Mallik, Eddie Bergman, Carl Hvarfner, Danny Stoll, Maciej Janowski, Marius Lindauer, Luigi Nardi, Frank Hutter

https://openreview.net/forum?id=uoiwugtpCH

Keywords: Hyperparameter Optimization, Deep Learning

Compressor summary: PriorBand is an HPO algorithm for DL that combines expert knowledge and cheap proxy tasks to efficiently optimize hyperparameters.

Minimax Forward and Backward Learning of Evolving Tasks with Performance Guarantees

Veronica Alvarez, Santiago Mazuelas, Jose A. Lozano

https://openreview.net/forum?id=uoRiO855Sj

Keywords: Concept drift, Continual learning, Minimax classification, Performance guarantees

Compressor summary: IMRCs use forward and backward learning to improve classification accuracy on evolving tasks with few samples per task.

Sample-efficient Multi-objective Molecular Optimization with GFlowNets

Yiheng Zhu, Jialu Wu, Chaowen Hu, Jiahuan Yan, Chang-Yu Hsieh, Tingjun Hou, Jian Wu

https://openreview.net/forum?id=uoG1fLIK2s

Keywords: drug discovery, multi-objective molecular optimization, Bayesian optimization, generative flow networks

Compressor summary: The paper introduces HN-GFN, a novel algorithm that uses a single hypernetwork to optimize multiple objectives and diversity in molecule design, and a hindsight-like strategy to improve learning and sharing of high-quality molecules.

Energy-Based Sliced Wasserstein Distance

Khai Nguyen, Nhat Ho

https://openreview.net/forum?id=umvV3yvo4N

Keywords: Sliced Wasserstein, Monte Carlo Methods, Point-Cloud, Optimal Transport

Compressor summary: The paper introduces a new way to choose the slicing distribution for the Wasserstein distance, called energy-based sliced Wasserstein (EBSW), which improves on previous approaches in terms of discrimination, cost, and stability.

Enhancing Knowledge Transfer for Task Incremental Learning with Data-free Subnetwork

Qiang Gao, Xiaojun Shan, Yuchen Zhang, Fan Zhou

https://openreview.net/forum?id=uj9PxVTVqq

Keywords: data-free subnetwork, task-incremental learning, knowledge transfer, mask

Compressor summary: DSN is a novel method for neuron-wise incremental learning that transfers knowledge across tasks using masks and data-free replay, avoiding forgetting and privacy issues.

Annotator: A Generic Active Learning Baseline for LiDAR Semantic Segmentation

Binhui Xie, Shuang Li, qingju guo, Chi Harold Liu, Xinjing Cheng

https://openreview.net/forum?id=uiiVSVADDc

Keywords: Active Learning, LiDAR Semantic Segmentation, Domain Adaptation

Compressor summary: Annotator is an active learning method that uses voxel-centric online selection to efficiently annotate LiDAR point clouds for semantic segmentation under various distribution shifts.

Mechanic: A Learning Rate Tuner

Ashok Cutkosky, Aaron Defazio, Harsh Mehta

https://openreview.net/forum?id=uhKtQMn21D

Keywords: optimization, deep learning, online convex optimization

Compressor summary: Mechanic is a technique for automatically adjusting the learning rate in any optimization algorithm, which performs well in various deep learning tasks and can match or surpass manual tuning.

HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution

Eric Nguyen, Michael Poli, Marjan Faizi, Armin W Thomas, Michael Wornow, Callum Birch-Sykes, Stefano Massaroli, Aman Patel, Clayton M. Rabideau, Yoshua Bengio, Stefano Ermon, Christopher Re, Stephen Baccus

https://openreview.net/forum?id=ubzNoJjOKj

Keywords: genomics, hyena, foundation models, large language models, transformers

Compressor summary: HyenaDNA is a large language model that can process up to 1 million tokens of human genome data, enabling long-range interactions and single nucleotide resolution for genomic tasks, and outperforms previous models on various benchmarks.

Uncovering Meanings of Embeddings via Partial Orthogonality

Yibo Jiang, Bryon Aragam, Victor Veitch

https://openreview.net/forum?id=ubp5s2tgXq

Keywords: embedding, representation, graphical models, partial orthogonality, Markov boundary

Compressor summary: The paper explores how semantic structure is encoded in vector embeddings using partial orthogonality and introduces embeddings that preserve conditional independence structures.

SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds

Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, Jian Ren

https://openreview.net/forum?id=ubgdInLSF9

Keywords: Text-to-Image, Diffusion model, mobile devices, distillation

Compressor summary: The authors introduce a generic approach to make text-to-image diffusion models faster and more accessible on mobile devices, improving efficiency by modifying the network architecture and step distillation, and achieving better results than Stable Diffusion v$1.5$.

Domain Agnostic Fourier Neural Operators

Ning Liu, Siavash Jafarzadeh, Yue Yu

https://openreview.net/forum?id=ubap5FKbJs

Keywords: Operator-Regression Neural Networks, Neural Operators, Data-Driven Physics Modeling, Geometrical and Topological Shape Changes

Compressor summary: The authors propose a novel neural operator architecture, DAFNO, that can learn nonlinear mappings between function spaces on irregular geometries and evolving domains using FFT, achieving state-of-the-art accuracy in material modeling and airfoil simulation, and demonstrating generalizability to unseen crack patterns.

Small Total-Cost Constraints in Contextual Bandits with Knapsacks, with Application to Fairness

Evgenii E Chzhen, Christophe Giraud, Zhen LI, Gilles Stoltz

https://openreview.net/forum?id=uZvG0HLkOB

Keywords: mutli-armed bandits, bandits with knapsacks, primal-dual approaches

Compressor summary: The paper proposes a new algorithm for contextual bandit problems with knapsacks that can handle cost constraints of the order of $\sqrt{T}$, which is more efficient and simpler than existing approaches.

CL-NeRF: Continual Learning of Neural Radiance Fields for Evolving Scene Representation

Xiuzhe Wu, Peng Dai, Weipeng DENG, Handi Chen, Yang Wu, Yan-Pei Cao, Ying Shan, XIAOJUAN QI

https://openreview.net/forum?id=uZjpSBTPik

Keywords: Neural Radiance Field; Continual Learning; Scene Representation

Compressor summary: The paper introduces CL-NeRF, a method for efficiently adapting Neural Radiance Fields to scene changes using continual learning and a new benchmark for evaluating it.

Learning Neural Implicit through Volume Rendering with Attentive Depth Fusion Priors

Pengchong Hu, Zhizhong Han

https://openreview.net/forum?id=uWNqy09dFW

Keywords: 3D Reconstruction, SDF, Neural Rendering, Implicit Representations, SLAM

Compressor summary: The paragraph introduces a novel method for 3D reconstruction from multi-view images using neural implicit representations, volume rendering with an attentive depth fusion prior, and a Truncated Signed Distance Function (TSDF).

Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift

Saurabh Garg, Amrith Setlur, Zachary Chase Lipton, Sivaraman Balakrishnan, Virginia Smith, Aditi Raghunathan

https://openreview.net/forum?id=uWGH6jDTVv

Keywords: contrastive learning; self training; distribution shift; semi supervised learning; unsupervised domain adaptation

Compressor summary: The paper investigates how combining self-training and contrastive learning techniques affects unsupervised domain adaptation and semi-supervised learning, showing improved accuracy in the former but not the latter.

Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception

Hassan Akbari, Dan Kondratyuk, Yin Cui, Rachel Hornung, Huisheng Wang, Hartwig Adam

https://openreview.net/forum?id=uTlKUAm68H

Keywords: Alternating Gradient Descent, Multimodal, Mixture of Experts, AGD, MoE, Deep Learning, Optimization

Compressor summary: IMP is a multimodal multi-task training method that uses alternating gradient descent and mixture-of-experts to efficiently improve model performance on various tasks, achieving state-of-the-art results in zero-shot video classification.

Self-Supervised Reinforcement Learning that Transfers using Random Features

Boyuan Chen, Chuning Zhu, Pulkit Agrawal, Kaiqing Zhang, Abhishek Gupta

https://openreview.net/forum?id=uRewSnLJAa

Keywords: deep reinforcement learning, self-supervised learning

Compressor summary: The paper proposes a self-supervised RL method that combines model-free and model-based approaches for transferring behaviors across tasks with different rewards in complex environments.

Sampling weights of deep neural networks

Erik Lien Bolager, Iryna Burak, Chinmay Datar, Qing Sun, Felix Dietrich

https://openreview.net/forum?id=uRHpgo6TMR

Keywords: random sampling, neural network parameters, iterative optimization

Compressor summary: The authors propose a method for quickly training fully-connected neural networks without gradient computations by using a data-dependent probability distribution and an efficient sampling algorithm.

A Logic for Expressing Log-Precision Transformers

William Merrill, Ashish Sabharwal

https://openreview.net/forum?id=uR8TtWCIsr

Keywords: transformers, logic, reasoning, circuit complexity, mechanistic interpretability

Compressor summary: The paper investigates whether log-precision transformers, which can attend universally, can be represented by a generalized first-order logic with majority-vote quantifiers.

Data Selection for Language Models via Importance Resampling

Sang Michael Xie, Shibani Santurkar, Tengyu Ma, Percy Liang

https://openreview.net/forum?id=uPSQv0leAu

Keywords: Language models, pretraining, data selection, fine-tuning

Compressor summary: DSIR is a data selection method that uses importance resampling in a reduced feature space to match a target distribution, achieving comparable or better results than expert curation or simple heuristics.

Breadcrumbs to the Goal: Supervised Goal Selection from Human-in-the-Loop Feedback

Marcel Torne Villasevil, Max Balsells I Pamies, Zihan Wang, Samedh Desai, Tao Chen, Pulkit Agrawal, Abhishek Gupta

https://openreview.net/forum?id=uOEeui0rL7

Keywords: Learning from human preferences, self-supervised learning, exploration in reinforcement learning

Compressor summary: HUGE is a technique that uses low-quality human feedback to guide exploration for reinforcement learning without requiring reward specification or exploration bonuses.

Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modeling

Zhenyu Zhu, Francesco Locatello, Volkan Cevher

https://openreview.net/forum?id=uNnPWR66b8

Keywords: Causal discovery, Score matching, Score-based generative modeling

Compressor summary: The paper studies how well deep neural networks can estimate a score function for causal discovery using score-matching methods.

Adaptive Linear Estimating Equations

Mufang Ying, Koulik Khamaru, Cun-Hui Zhang

https://openreview.net/forum?id=uNmKBZrRZC

Keywords: bandit algorithm, statistical inference, adaptively collected data, asymptotic normality

Compressor summary: The paper proposes a debiased estimator for sequential data collection that improves statistical inference by achieving both non-asymptotic performance and asymptotic normality.

The Graph Pencil Method: Mapping Subgraph Densities to Stochastic Block Models

Lee M. Gunderson, Gecia Bravo-Hermsdorff, Peter Orbanz

https://openreview.net/forum?id=uN71BdBEG8

Keywords: Stochastic block model, SBM, graphons, matrix pencil method, method of moments

Compressor summary: The paper presents a method to find the parameters of a stochastic block model from a finite set of subgraph densities using a unique translation map that is efficient and easy to compute.

Fair Allocation of Indivisible Chores: Beyond Additive Costs

Bo Li, Fangxiao Wang, Yu Zhou

https://openreview.net/forum?id=uJmsYZiu3E

Keywords: fair allocation of chores, beyond additive cost functions, bin packing, job scheduling

Compressor summary: The paper explores the limitations of achieving exact or good approximations of maximin share fairness in allocating indivisible tasks with subadditive costs and studies two specific settings, bin packing and job scheduling, where constant approximate allocations exist.

Exploring Geometry of Blind Spots in Vision models

Sriram Balasubramanian, Gaurang Sriramanan, Vinu Sankar Sadasivan, Soheil Feizi

https://openreview.net/forum?id=uJ3qNIsDGF

Keywords: Neural networks, Vision models, blind spots, undersensitivity, invariance, level set geometry, input connectivity

Compressor summary: The authors study how vision models like CNNs and Transformers are under-sensitive to input perturbations and propose a method to explore the geometry of "equi-confidence" level sets in these networks, revealing a star-like structure.

FairLISA: Fair User Modeling with Limited Sensitive Attributes Information

Zheng Zhang, Qi Liu, Hao Jiang, Fei Wang, Yan Zhuang, Le Wu, Weibo Gao, Enhong Chen

https://openreview.net/forum?id=uFpjPJMkv6

Keywords: fairness, user modeling

Compressor summary: The paper proposes a new framework called FairLISA to create user models that are fair and accurate even when some sensitive data is missing or unknown.

Focus Your Attention when Few-Shot Classification

Haoqing Wang, Shibo Jie, Zhi-Hong Deng

https://openreview.net/forum?id=uFlE0qgtRO

Keywords: few-shot image classification, fine-tuning, vision transformers

Compressor summary: The paper proposes a method to improve few-shot image classification with pre-trained vision transformers by using attention and gradient information to locate key entities in support images, which helps the model focus on class-related features and generalize better.

Static and Sequential Malicious Attacks in the Context of Selective Forgetting

CHENXU ZHAO, Wei Qian, Zhitao Ying, Mengdi Huai

https://openreview.net/forum?id=uEJfW3OtUm

Keywords: Selective forgetting, static setting, sequential setting, security and robustness

Compressor summary: The paper explores malicious data update attacks on machine unlearning systems and proposes new attack methods and algorithms to exploit them.

Efficient Robust Bayesian Optimization for Arbitrary Uncertain inputs

Lin Yang, Junlong Lyu, Wenlong Lyu, Zhitang Chen

https://openreview.net/forum?id=uDV4lA0gZ6

Keywords: bayesian optimization, robust optimization

Compressor summary: The paper proposes AIRBO, a robust Bayesian Optimization algorithm that models uncertain inputs using Maximum Mean Discrepancy and Nystrom approximation, achieving state-of-the-art performance under various input uncertainties.

(S)GD over Diagonal Linear Networks: Implicit bias, Large Stepsizes and Edge of Stability

Mathieu Even, Scott Pesme, Suriya Gunasekar, Nicolas Flammarion

https://openreview.net/forum?id=uAyElhYKxg

Keywords: SGD, GD, implicit bias, large stepsizes, edge of stability, diagonal linear networks

Compressor summary: The paper studies how randomness and big steps affect gradient descent and stochastic gradient descent in linear networks with two layers, proving their convergence and analysing how these factors influence solution quality in regression problems.

ReDS: Offline RL With Heteroskedastic Datasets via Support Constraints

Anikait Singh, Aviral Kumar, Quan Vuong, Yevgen Chebotar, Sergey Levine

https://openreview.net/forum?id=u8srPlinoj

Keywords: offline RL, support constraints, heteroskedastic data

Compressor summary: Offline reinforcement learning methods struggle to learn from data with non-uniform variability due to distribution constraints; conservative Q-learning (CQL) with reweighting improves performance by allowing the learned policy to choose how closely to follow the behavior policy per state.

Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models

Haonan Duan, Adam Dziedzic, Nicolas Papernot, Franziska Boenisch

https://openreview.net/forum?id=u6Xv3FuF8N

Keywords: differential privacy, in-context learning, trustworthy ML

Compressor summary: The authors propose a method to privately learn how to prompt large language models without exposing the data used for prompting, which can maintain high accuracy while protecting user privacy.

Real-World Image Variation by Aligning Diffusion Inversion Chain

Yuechen ZHANG, Jinbo Xing, Eric Lo, Jiaya Jia

https://openreview.net/forum?id=u6Ibs4hTJH

Keywords: image variation, diffusion model, image generation, text-driven image editing

Compressor summary: RIVAL is a novel inference pipeline that generates high-quality image variations by aligning the diffusion process to the source image's inversion chain using cross-image self-attention and step-wise distribution normalization.

MADG: Margin-based Adversarial Learning for Domain Generalization

Aveen Dayal, Vimal K B, Linga Reddy Cenkeramaddi, C Krishna Mohan, Abhinav Kumar, Vineeth N. Balasubramanian

https://openreview.net/forum?id=u6BYyPuD29

Keywords: Domain Generalization, Margin Loss, Adversarial Learning, Domain Adaptation

Compressor summary: The paper proposes a new adversarial learning method for domain generalization called MADG, which uses a margin loss-based discrepancy metric to learn domain-invariant features and achieves better generalization to unseen domains.

Graph Denoising Diffusion for Inverse Protein Folding

Kai Yi, Bingxin Zhou, Yiqing Shen, Pietro Lio, Yu Guang Wang

https://openreview.net/forum?id=u4YXKKG5dX

Keywords: inverse folding, graph neural networks, roto-translation equivariance, diffusion model

Compressor summary: The proposed graph denoising diffusion model generates diverse amino acid sequences for a given protein backbone using physiochemical properties, local environment, and prior knowledge of amino acid replacements, outperforming existing methods in inverse protein folding.

Uncovering motifs of concurrent signaling across multiple neuronal populations

Evren Gokcen, Anna Ivic Jasper, Alison Xu, Adam Kohn, Christian K. Machens, Byron M. Yu

https://openreview.net/forum?id=u39QQh5L8Q

Keywords: neuroscience, multi-population neural recordings, dimensionality reduction, latent variable models, Gaussian processes

Compressor summary: The paragraph describes a new framework for analyzing signals among multiple neuronal populations in different brain networks, which reveals how they communicate and evolve over time.

Robust Data Valuation with Weighted Banzhaf Values

Weida Li, Yaoliang Yu

https://openreview.net/forum?id=u359tNBpxF

Keywords: data valuation, robustness, weighted Banzhaf values

Compressor summary: The paper proposes a new robust data valuation method using weighted Banzhaf values and introduces Kronecker noise to analyze stochasticity.

PlanE: Representation Learning over Planar Graphs

Radoslav Dimitrov, Zeyang Zhao, Ralph Abboud, Ismail Ilkan Ceylan

https://openreview.net/forum?id=u2RJ0I3o3j

Keywords: Graph Representation Learning; Planar Graphs; Graph Property Prediction

Compressor summary: The paragraph introduces PlanE, a framework for learning complete invariants of planar graphs, which are special graph classes with efficient isomorphism testing algorithms, and reports its successful performance on benchmarks.

Knowledge Distillation Performs Partial Variance Reduction

Mher Safaryan, Alexandra Peste, Dan Alistarh

https://openreview.net/forum?id=tzxP9Rx0LV

Keywords: knowledge distillation, stochastic optimization, variance reduction

Compressor summary: The paper explores how knowledge distillation works as a stochastic variance reduction technique for enhancing student models with teacher models, and highlights the importance of careful parametrization for optimal performance.

GEX: A flexible method for approximating influence via Geometric Ensemble

SungYub Kim, Kyungsu Kim, Eunho Yang

https://openreview.net/forum?id=tz4ECtAu8e

Keywords: Influence Function, Geometric Ensemble, Loss Landscape

Compressor summary: The paper proposes a new method to improve Influence Function (IF) approximations for neural networks by addressing limitations and enhancing performance in various tasks.

InstanT: Semi-supervised Learning with Instance-dependent Thresholds

Muyang Li, Runze Wu, Haoyu Liu, Jun Yu, Xun Yang, Bo Han, Tongliang Liu

https://openreview.net/forum?id=txv7TnPvOi

Keywords: semi-supervised learning, pseudo-labeling

Compressor summary: The paper proposes a new SSL method that uses instance-dependent thresholds to select confident unlabeled instances and assign pseudo-labels with probabilistic guarantees.

Fed-FA: Theoretically Modeling Client Data Divergence for Federated Language Backdoor Defense

Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

https://openreview.net/forum?id=txPdKZrrZF

Keywords: federated learning, backdoor learning, robust federated aggregation, data divergence

Compressor summary: The paper proposes a method called Federated F-Divergence-Based Aggregation (Fed-FA) to detect and exclude suspicious clients in federated NLP systems by modeling data divergence using f-divergence indicators and synthetic datasets.

DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning

Alexander H. Liu, Heng-Jui Chang, Michael Auli, Wei-Ning Hsu, James R. Glass

https://openreview.net/forum?id=twmHKU3Ds4

Keywords: speech representation learning, self-supervised learning, self-distillation, discrete representation learning

Compressor summary: The paper presents a self-supervised speech representation learning model called DinoSR that combines masked language modeling, self-distillation, and online clustering to achieve state-of-the-art performance in various downstream tasks.

The Bayesian Stability Zoo

Shay Moran, Hilla Schefler, Jonathan Shafer

https://openreview.net/forum?id=tw4QaiiJex

Keywords: Algorithmic stability, Replicability, Differential Privacy, KL Stability, Mutual Information Stability, Global Stability, Perfect Generalization, PAC Learning, Littlestone Dimension, Clique Dimension, PAC Bayes

Compressor summary: The paper explores the connections between different definitions of stability in learning theory, including various types of privacy, generalization, and divergence-based stability notions.

Secure Out-of-Distribution Task Generalization with Energy-Based Models

Shengzhuang Chen, Long-Kai Huang, Jonathan Richard Schwarz, Yilun Du, Ying Wei

https://openreview.net/forum?id=tt7bQnTdRm

Keywords: meta-generalization, out-of-distribution tasks

Compressor summary: The EBML framework uses two neural networks to characterize any meta-training task distribution, enabling detection and adaptation of out-of-distribution tasks for safe and effective meta-learning.

Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning

Changsheng Lv, Shuai Zhang, Yapeng Tian, Mengshi Qi, Huadong Ma

https://openreview.net/forum?id=trHfuGQyyr

Keywords: Physical Audiovisual；Commonsense Reasoning

Compressor summary: The paper proposes a Disentangled Counterfactual Learning (DCL) method for inferring physical commonsense from audiovisual data, which decouples video factors and models causal relationships using a variational autoencoder and a counterfactual learning module.

NetHack is Hard to Hack

Ulyana Piterbarg, Lerrel Pinto, Rob Fergus

https://openreview.net/forum?id=tp2nEZ5zfP

Keywords: imitation learning, NetHack

Compressor summary: The paper studies why neural policy learning struggles in NetHack and proposes improvements based on a winning symbolic agent's strategy, leading to a better neural agent that still falls behind symbolic and human players.

Derandomized novelty detection with FDR control via conformal e-values

Meshi Bashari, Amir Epstein, Yaniv Romano, Matteo Sesia

https://openreview.net/forum?id=toYvRJ7Zmy

Keywords: Conformal inference, Derandomization, E-values, False discovery rate, Out-of-distribution testing, Testing for outliers, Uncertainty

Compressor summary: The authors propose a method to make conformal inference more stable for novelty detection by using conformal e-values instead of p-values and weighting them with additional data information.

SafeDICE: Offline Safe Imitation Learning with Non-Preferred Demonstrations

Youngsoo Jang, Geon-Hyeong Kim, Jongmin Lee, Sungryull Sohn, Byoungjip Kim, Honglak Lee, Moontae Lee

https://openreview.net/forum?id=toEGuA9Qfn

Keywords: Imitation learning, Preference-based learning, Safe imitation learning

Compressor summary: The paper introduces SafeDICE, an offline imitation learning algorithm that learns a safe policy from non-preferred demonstrations and unlabeled data by estimating stationary distribution corrections.

Dream the Impossible: Outlier Imagination with Diffusion Models

Xuefeng Du, Yiyou Sun, Jerry Zhu, Yixuan Li

https://openreview.net/forum?id=tnRboxQIec

Keywords: Outlier imagination, machine learning

Compressor summary: The paper introduces Dream-OOD, a framework that generates realistic outlier images for machine learning models using in-distribution data and classes, without requiring manual data collection or cleaning.

Add and Thin: Diffusion for Temporal Point Processes

David Lüdke, Marin Biloš, Oleksandr Shchur, Marten Lienen, Stephan Günnemann

https://openreview.net/forum?id=tn9Dldam9L

Keywords: Point Processes, Diffusion, Temporal Data, Generative Model, Forecasting, Density Estimation, Denoising

Compressor summary: The paper introduces ADD-THIN, a probabilistic denoising diffusion model that improves long-term forecasting for event sequences with discrete and continuous components by using temporal point process frameworks.

SPACE: Single-round Participant Amalgamation for Contribution Evaluation in Federated Learning

Yi-Chung Chen, Hsi-Wen Chen, Shun-Guei Wang, Ming-Syan Chen

https://openreview.net/forum?id=tmxjuIFSEc

Keywords: Federated Learning, Contribution Evaluation, Shapley Value, Knowledge Amalgamation

Compressor summary: The paper introduces SPACE, an efficient method for evaluating participant contributions in federated learning using Federated Knowledge Amalgamation and Prototype-based Model Evaluation.

Exponential Lower Bounds for Fictitious Play in Potential Games

Ioannis Panageas, Nikolas Patris, Stratis Skoulakis, Volkan Cevher

https://openreview.net/forum?id=tkenkPYkxj

Keywords: fictitious play, convergence rate, potential games

Compressor summary: This paper studies the convergence rate of Fictitious Play in potential games and shows that it can take exponential time even for two-agent games.

Feature learning via mean-field Langevin dynamics: classifying sparse parities and beyond

Taiji Suzuki, Denny Wu, Kazusato Oko, Atsushi Nitanda

https://openreview.net/forum?id=tj86aGVNb3

Keywords: mean-field regime, feature learning, Neural network optimization, sparse parity function, classification, sample complexity

Compressor summary: The paragraph discusses how mean-field Langevin dynamics (MFLD) can optimize and improve generalization performance of neural networks for binary classification problems by learning features, and provides a new analysis of its sample complexity and convergence rate.

No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models

Jean Kaddour, Oscar Key, Piotr Nawrot, Pasquale Minervini, Matt Kusner

https://openreview.net/forum?id=thbXgJ8gNK

Keywords: language models, transformers, efficient training

Compressor summary: The authors investigate various efficient training algorithms for Transformer-based language models, but find that a fully-decayed learning rate outperforms them all.

GlyphControl: Glyph Conditional Control for Visual Text Generation

Yukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang, Haisong Ding, Han Hu, Kai Chen

https://openreview.net/forum?id=thPI8hrA4V

Keywords: Generative Models, Visual Text Generation, Diffusion Models

Compressor summary: The paper introduces a new method called GlyphControl to improve visual text generation by using glyph instructions with an off-the-shelf Stable-Diffusion model and creates a dataset called LAION-Glyph for further research.

Bypassing spike sorting: Density-based decoding using spike localization from dense multielectrode probes

Yizi Zhang, Tianxiao He, Julien Boussard, Charlie Windolf, Olivier Winter, Eric M. Trautmann, Noam Roth, Hailey Barrel, Mark M Churchland, Nick Steinmetz, Erdem Varol, Cole Lincoln Hurwitz, Liam Paninski

https://openreview.net/forum?id=tgQRMrsxht

Keywords: neural decoding, brain-computer interfaces, spike sorting, variational inference, generative models

Compressor summary: The authors propose a new method for decoding neural activity without spike sorting, using mixture of Gaussians to model uncertainty and improve performance in brain-computer interfaces.

SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models

Hongxin Li, Jingran Su, Yuntao Chen, Qing Li, Zhaoxiang Zhang

https://openreview.net/forum?id=tfyr2zRVoK

Keywords: Large Language Model; Task Planning; Embodied AI; Robotics; Software Automation

Compressor summary: The authors propose SheetCopilot, an agent that uses natural language to control spreadsheets and complete various tasks, and present a dataset and evaluation pipeline to benchmark its performance.

Stable Diffusion is Unstable

Chengbin Du, Yanxi Li, Zhongwei Qiu, Chang Xu

https://openreview.net/forum?id=tesBViWnbx

Keywords: Adversarial Attack, Generative Model, Diffusion Model, Latent Diffusion Model, Conditional Latent Diffusion Model

Compressor summary: Auto-attack on Text-to-image Models (ATM) is a method to make text-to-image models more robust by generating perturbations that can prevent them from blending or losing primary subjects in generated images.

FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-Entropy

Zuhao Yang, Yingfang Yuan, Yang Xu, SHUO ZHAN, Huajun Bai, Kefan Chen

https://openreview.net/forum?id=tdyLryDebq

Keywords: natural language generation; evaluation metrics; cross-entropy; language model

Compressor summary: The proposed FACE metrics use Fourier Analysis of Cross-Entropy to measure similarity between model-generated and human language, effectively identifying the gap between them and reflecting various factors such as model size and sampling methods.

CQM: Curriculum Reinforcement Learning with a Quantized World Model

Seungjae Lee, Daesol Cho, Jonghae Park, H. Jin Kim

https://openreview.net/forum?id=tcotyjon2a

Keywords: Reinforcement Learning, Curriculum Learning, Goal-conditioned RL

Compressor summary: The authors propose a new method for curriculum RL that automatically defines a semantic goal space from continuous observations and suggests uncertainty and temporal distance-aware goals, improving exploration and performance on goal-reaching tasks.

Tracr: Compiled Transformers as a Laboratory for Interpretability

David Lindner, Janos Kramar, Sebastian Farquhar, Matthew Rahtz, Thomas McGrath, Vladimir Mikulik

https://openreview.net/forum?id=tbbId8u7nP

Keywords: interpretability, transformers, language models, RASP, Tracr, mechanistic interpretability

Compressor summary: The authors introduce a compiler called Tracr that generates structured transformer models from human-readable programs, enabling experiments and interpretability evaluations.

Ignorance is Bliss: Robust Control via Information Gating

Manan Tomar, Riashat Islam, Matthew E. Taylor, Sergey Levine, Philip Bachman

https://openreview.net/forum?id=tW2KSph9o8

Keywords: representation learning, mutual information

Compressor summary: Information gating is a method for learning efficient and robust representations by selectively revealing or hiding information depending on the task at hand.

Language Semantic Graph Guided Data-Efficient Learning

Wenxuan Ma, Shuang Li, Lincan Cai, Jingxuan Kang

https://openreview.net/forum?id=tUyW68cRqr

Keywords: Data-Efficient Learning, Language Semantic Graph

Compressor summary: The paragraph discusses a new approach called Language Semantic Graph (LSG) that uses semantic information from labels to improve data efficiency in machine learning tasks, and demonstrates its effectiveness across different modalities and scenarios.

ForecastPFN: Synthetically-Trained Zero-Shot Forecasting

Samuel Dooley, Gurnoor Singh Khurana, Chirag Mohapatra, Siddartha Venkat Naidu, Colin White

https://openreview.net/forum?id=tScBQRNgjk

Keywords: Forecasting, Zero-shot, Synthetic Data

Compressor summary: ForecastPFN is a new zero-shot forecasting model that uses synthetic data and Bayesian inference to make more accurate and faster predictions than existing methods, even with limited real-life observations.

Human-Guided Complexity-Controlled Abstractions

Andi Peng, Mycal Tucker, Eoin M. Kenny, Noga Zaslavsky, Pulkit Agrawal, Julie Shah

https://openreview.net/forum?id=tSEeRl7ACo

Keywords: human-in-the-loop, representation learning, interpretability

Compressor summary: The authors propose training neural networks to generate a spectrum of discrete representations and control their complexity based on the task, which leads to better generalization and can be informed by human intuition.

Modeling Human Visual Motion Processing with Trainable Motion Energy Sensing and a Self-attention Network

Zitang Sun, Yen-Ju Chen, Yung-Hao Yang, Shin'ya Nishida

https://openreview.net/forum?id=tRKimbAk5D

Keywords: motion perception, optical flow estimation, attention mechanism, psychophysics, In silico neurophysiology, human vision

Compressor summary: The paper proposes a new image-computable model that combines biological and computer vision approaches to simulate human motion perception in complex scenes.

D$^2$CSG: Unsupervised Learning of Compact CSG Trees with Dual Complements and Dropouts

Fenggen Yu, Qimin Chen, Maham Tanveer, Ali Mahdavi Amiri, Hao Zhang

https://openreview.net/forum?id=tQYGjnxPOm

Keywords: 3D reconstruction, constructive solid geometry, unsupervised learning, compact shape assembly

Compressor summary: D$^2$CSG is a neural model that learns to reconstruct 3D CAD shapes using quadric primitives and a dedicated residual branch for complex shape complements.

Non-Stationary Bandits with Auto-Regressive Temporal Dependency

Qinyi Chen, Negin Golrezaei, Djallel Bouneffouf

https://openreview.net/forum?id=tP50lLiZIo

Keywords: non-stationary bandits; autoregressive model; low-regret policy; online learning algorithms

Compressor summary: The paper introduces a new non-stationary MAB framework that considers temporal dynamics in real-world applications like recommendation systems, and proposes an algorithm with two mechanisms for exploration-exploitation and discarding outdated information.

Optimistic Active Exploration of Dynamical Systems

Bhavya Sukhija, Lenart Treven, Cansu Sancaktar, Sebastian Blaes, Stelian Coros, Andreas Krause

https://openreview.net/forum?id=tLrkjK128n

Keywords: Active Exploration, Reinforcement Learning, Dynamical Systems

Compressor summary: OPAX is an algorithm for active exploration in reinforcement learning that uses probabilistic models to quantify uncertainty and maximizes information gain, enabling zero-shot solutions for multiple tasks.

Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts

Emanuele Marconato, Stefano Teso, Antonio Vergari, Andrea Passerini

https://openreview.net/forum?id=tLTtqySDFb

Keywords: Neuro-Symbolic Integration, Trustworthy AI, Concept Learning, Learning Shortcuts, Mitigation Strategies

Compressor summary: The paragraph discusses how neuro-symbolic predictive models can suffer from reasoning shortcuts that compromise their advantages, and proposes mitigation strategies to address this issue.

Emergent Communication in Interactive Sketch Question Answering

Zixing Lei, Yiming Zhang, Yuxin Xiong, Siheng Chen

https://openreview.net/forum?id=tLEDsaKuDh

Keywords: Emergent communication, Interactive, Question Answering

Compressor summary: The authors propose a new task (ISQA) and system for vision-based emergent communication, which uses sketches to answer questions about images in multiple rounds, improving accuracy, complexity, and interpretability.

Parallel-mentoring for Offline Model-based Optimization

Can Chen, Christopher Beckham, Zixuan Liu, Xue Liu, Christopher Pal

https://openreview.net/forum?id=tJwyg9Zg9G

Keywords: offline model-based optimization, bi-level optimization

Compressor summary: The authors propose parallel-mentoring, a novel method that uses majority voting and adaptive soft-labeling to improve model-based optimization with multiple proxies, addressing out-of-distribution issues in black-box objective function maximization.

Offline RL with Discrete Proxy Representations for Generalizability in POMDPs

Pengjie Gu, Xinyu Cai, Dong Xing, Xinrun Wang, Mengchen Zhao, Bo An

https://openreview.net/forum?id=tJN664ZNVG

Keywords: Offline RL, POMDP

Compressor summary: ORDER is a probabilistic framework for offline RL that uses discrete state representations and proxy representations to improve robustness against diverse masked observabilities.

Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data

Boris van Breugel, Nabeel Seedat, Fergus Imrie, Mihaela van der Schaar

https://openreview.net/forum?id=tJ88RBqupo

Keywords: model evaluation, tabular, synthetic data

Compressor summary: The paper proposes 3S Testing, a method that uses generative models to create synthetic test data for small subgroups and simulate distributional shifts, improving the evaluation of machine learning model performance on diverse and underrepresented groups.

Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Prediction

Zuobai Zhang, Minghao Xu, Aurelie Lozano, Vijil Chenthamarakshan, Payel Das, Jian Tang

https://openreview.net/forum?id=tIzbNQko3c

Keywords: Protein representation learning, diffusion models, self-supervised learning

Compressor summary: The paper proposes a pre-training method for protein encoders using joint sequence-structure diffusion models and enhances it with a technique to capture conformational variations among protein conformers.

Latent Graph Inference with Limited Supervision

Jianglin Lu, Yi Xu, Huan Wang, Yue Bai, Yun Fu

https://openreview.net/forum?id=tGuMwFnRZX

Keywords: Latent Graph Inference, CUR Matrix Decomposition, Graph Neural Networks

Compressor summary: The paper proposes a method to improve latent graph inference by restoring corrupted affinities and replenishing missed supervision for better node representation learning.

Distribution-Free Model-Agnostic Regression Calibration via Nonparametric Methods

Shang Liu, Zhongze Cai, Xiaocheng Li

https://openreview.net/forum?id=tGPx7HdBr4

Keywords: Regression calibration, model recalibration, conditional quantile, nonparametric method

Compressor summary: The paper proposes nonparametric methods for individual calibration of regression models that are efficient, consistent, and provide theoretical guarantees, addressing limitations of existing heuristic methods.

Maximum State Entropy Exploration using Predecessor and Successor Representations

Arnav Kumar Jain, Lucas Lehnert, Irina Rish, Glen Berseth

https://openreview.net/forum?id=tFsxtqGmkn

Keywords: Reinforcement Learning, Maximum state entropy exploration, Non-Markovian exploration, Successor Representation

Compressor summary: The authors propose a new exploration algorithm, $\eta\psi$-Learning, that learns efficient policies by using past experience to predict state visitation entropy and maximize coverage.

Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditions

Sayantan Choudhury, Eduard Gorbunov, Nicolas Loizou

https://openreview.net/forum?id=tFeaLw9AWn

Keywords: Optimization, Machine Learning, Extragradient Methods, Min-Max Optimization

Compressor summary: The paper studies single-call stochastic extragradient methods for solving large-scale min-max optimization and variational inequalities problems, improves their convergence guarantees, and applies them to two classes of structured non-monotone VIPs using the expected residual condition.

Federated Compositional Deep AUC Maximization

Xinwen Zhang, Yihan Zhang, Tianbao Yang, Richard Souvenir, Hongchang Gao

https://openreview.net/forum?id=tF7W8ai8J3

Keywords: federated learning, compositional optimization, minimax optimization, AUC maximization

Compressor summary: The paragraph discusses a novel federated learning method for imbalanced data that optimizes the AUC score and provides theoretical and empirical evidence of its effectiveness.

PPi: Pretraining Brain Signal Model for Patient-independent Seizure Detection

Zhizhang Yuan, Daoze Zhang, Yang Yang, Junru Chen, Yafeng Li

https://openreview.net/forum?id=tEmFyqjaJh

Keywords: Brain signal, Seizure detection, Pretraining, Domain generalization

Compressor summary: The study proposes a patient-independent seizure detection model using stereoelectroencephalography (SEEG) data that overcomes challenges like domain shift and pattern evolution with novel self-supervised tasks and domain generalization techniques.

Safety Verification of Decision-Tree Policies in Continuous Time

Christian Schilling, Anna Lukina, Emir Demirović, Kim Guldstrand Larsen

https://openreview.net/forum?id=tEKBU5XOTw

Keywords: safety verification, decision tree, reinforcement learning, controller, continuous time

Compressor summary: The paper introduces an algorithm to verify the safety of decision tree controlled systems in continuous time, overcoming the challenges of existing methods.

Doubly Constrained Fair Clustering

John P Dickerson, Seyed A. Esmaeili, Jamie Heather Morgenstern, Claire Jie Zhang

https://openreview.net/forum?id=tECyQO1QOp

Keywords: Fairness, Clustering, Approximation Algorithms

Compressor summary: The paper investigates how two common fairness criteria for clustering, group fairness and diversity in center selection, can be simultaneously satisfied using a constant approximation algorithm, and shows that they are compatible with some but not all distance-based fairness notions.

A Robust and Opponent-Aware League Training Method for StarCraft II

Ruozi Huang, Xipeng Wu, Hongsheng Yu, Zhong Fan, Haobo Fu, QIANG FU, Yang Wei

https://openreview.net/forum?id=tDAu3FPJn9

Keywords: StarCraft II, league training, AlphaStar, opponent-modeling, reinforcement learning

Compressor summary: The paper presents improvements to AlphaStar's league training for StarCraft II, enabling better and more resource-efficient AI agents with advanced exploitation and opponent modeling abilities.

On the Power of SVD in the Stochastic Block Model

Xinyu Mao, Jiapeng Zhang

https://openreview.net/forum?id=tC0r8duG9z

Keywords: Clustering Algorithms, Stochastic Block Model, Spectral Algorithms

Compressor summary: The paper investigates how spectral methods like SVD can improve clustering algorithms and shows that it works well for a specific type of network model called the stochastic block model.

Replicable Reinforcement Learning

ERIC EATON, Marcel Hussing, Michael Kearns, Jessica Sorrell

https://openreview.net/forum?id=tBwRbgsol1

Keywords: Reinforcement Learning, Learning Theory, Replicability, Reproducibility

Compressor summary: The text discusses the development of algorithm frameworks for replicability in various fields and introduces provably replicable algorithms for parallel value iteration and R-Max in episodic reinforcement learning.

Understanding Deep Gradient Leakage via Inversion Influence Functions

Haobo Zhang, Junyuan Hong, Yuyang Deng, Mehrdad Mahdavi, Jiayu Zhou

https://openreview.net/forum?id=tBib2fWr3r

Keywords: Deep Learning, Privacy, Federated Learning, Influence Function

Compressor summary: The paper proposes a method called Inversion Influence Function (I$^2$F) to analyze and understand the privacy leakage caused by Deep Gradient Leakage (DGL) attack in distributed learning settings, and show its effectiveness on various scenarios.

A Bounded Ability Estimation for Computerized Adaptive Testing

Yan Zhuang, Qi Liu, GuanHao Zhao, Zhenya Huang, Weizhe Huang, Zachary Pardos, Enhong Chen, Jinze Wu, Xin Li

https://openreview.net/forum?id=tAwjG5bM7H

Keywords: adaptive learning, computerized adaptive testing, educational measurement, cognitive diagnosis

Compressor summary: The paper proposes a new method (BECAT) for computerized adaptive testing that selects questions to estimate student's ability more accurately and efficiently by matching the gradient of full responses.

Uncertainty Estimation for Safety-critical Scene Segmentation via Fine-grained Reward Maximization

Hongzheng Yang, Cheng Chen, Yueyao Chen, Markus Scheppach, Hon Chi Yip, Qi Dou

https://openreview.net/forum?id=t9Swbo82dB

Keywords: uncertainty estimation, semantic segmentation, medical application

Compressor summary: The paper proposes a novel framework that uses reinforcement learning and reward optimization to improve uncertainty estimation in deep segmentation models for safety-critical applications like medical imaging.

Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models

Gen Luo, Yiyi Zhou, Tianhe Ren, Shengxin Chen, Xiaoshuai Sun, Rongrong Ji

https://openreview.net/forum?id=t877958UGZ

Keywords: vision-language instruction tuning, multimodal LLM, efficient training

Compressor summary: The paper proposes a cost-effective method called Mixture-of-Modality Adaptation (MMA) to improve the multimodal capabilities of large language models for vision-language tasks, achieving competitive performance and efficiency with low expenditure.

Rewiring Neurons in Non-Stationary Environments

Zhicheng Sun, Yadong MU

https://openreview.net/forum?id=t7ozN4AXd0

Keywords: continual learning, reinforcement learning, brain-inspired learning

Compressor summary: The paragraph describes a novel rewiring approach for the human brain in continual reinforcement learning that promotes adaptivity and exploration, with a focus on stability and plasticity-stability tradeoffs.

Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network

Tristan Deleu, Mizu Nishikawa-Toomey, Jithendaraa Subramanian, Nikolay Malkin, Laurent Charlin, Yoshua Bengio

https://openreview.net/forum?id=t7lnhhi7De

Keywords: bayesian network, bayesian, structure learning, causal discovery, gflownet

Compressor summary: The paper proposes JSP-GFN, a method that uses GFlowNets to approximate the joint posterior over Bayesian Network structure and parameters, achieving good results on various datasets.

Achieving Cross Modal Generalization with Multimodal Unified Representation

Yan Xia, Hai Huang, Jieming Zhu, Zhou Zhao

https://openreview.net/forum?id=t7ZowrDWVw

Keywords: multi-modal, discrete representation, mutual information estimation

Compressor summary: The paper proposes a novel task called Cross Modal Generalization (CMG) that learns a unified discrete representation from paired multimodal data and introduces Uni-Code, which contains two key contributions for fine-grained unified representation of multimodal sequences.

Trans-Dimensional Generative Modeling via Jump Diffusion Models

Andrew Campbell, William Harvey, Christian Dietrich Weilbach, Valentin De Bortoli, Tom Rainforth, Arnaud Doucet

https://openreview.net/forum?id=t6nA7x3GAC

Keywords: diffusion, score-based, score, markov chain, jump diffusion, poisson

Compressor summary: The proposed generative model can handle data of different dimensions by switching between them during the generation process, leading to better performance on tasks like imputation and interpolation.

QuadAttac$K$: A Quadratic Programming Approach to Learning Ordered Top-$K$ Adversarial Attacks

Thomas Paniagua, Ryan Grainger, Tianfu Wu

https://openreview.net/forum?id=t3vPEjgNtj

Keywords: Ordered Top-K Clear-Box Targeted Adversarial Attack, Deep Neural Networks, Quadratic Programming, Robustness

Compressor summary: The paper introduces QuadAttac$K$, a method to learn more aggressive targeted attacks on DNNs by solving quadratic programming in the feature embedding space.

Scalable Membership Inference Attacks via Quantile Regression

Martin Andres Bertran, Shuai Tang, Aaron Roth, Michael Kearns, Jamie Heather Morgenstern, Steven Wu

https://openreview.net/forum?id=t3WCiGjHqd

Keywords: machine learning, privacy, membership inference

Compressor summary: Our proposed attack estimates the distribution of confidence scores on non-training points using quantile regression, and it is more efficient and black-box than shadow model attacks.

Tailoring Self-Attention for Graph via Rooted Subtrees

Siyuan Huang, Yunchong Song, Jiayue Zhou, Zhouhan Lin

https://openreview.net/forum?id=t2hEZadBBk

Keywords: Graph Based Learning

Compressor summary: The paper introduces Subtree Attention, a novel graph attention mechanism that addresses the limitations of local and global attention by allowing computation of attention weights among multi-hop neighbors and has linear time complexity.

Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities

Andrii Zadaianchuk, Maximilian Seitzer, Georg Martius

https://openreview.net/forum?id=t1jLRFvBqm

Keywords: object-centric learning, video, representation learning, self-supervised learning, unsupervised learning

Compressor summary: The paper proposes a new method to learn object-centric representations from videos using pre-trained features and temporal feature similarity loss, achieving state-of-the-art results on synthetic and unconstrained video datasets.

A unified framework for information-theoretic generalization bounds

Yifeng Chu, Maxim Raginsky

https://openreview.net/forum?id=t0fkjO4aZj

Keywords: generalization bounds, information theory, chaining, PAC-Bayes, couplings

Compressor summary: The paper proposes a method to derive generalization bounds for learning algorithms using a probabilistic decorrelation lemma and other techniques.

Riemannian Projection-free Online Learning

Zihao Hu, Guanghui Wang, Jacob Abernethy

https://openreview.net/forum?id=szFqlNRxeS

Keywords: Online learning, Riemannian optimization, projection-free optimization

Compressor summary: The paper proposes projection-free optimization methods for online geodesically convex problems on curved spaces with separation or linear optimization oracles, achieving sub-linear regret guarantees in different settings.

A Neural Collapse Perspective on Feature Evolution in Graph Neural Networks

Vignesh Kothapalli, Tom Tirer, Joan Bruna

https://openreview.net/forum?id=sxao2udWXi

Keywords: Neural collapse, Graph neural networks, Community detection

Compressor summary: The paper investigates how graph topology affects feature evolution in node-wise classification using graph neural networks (GNNs) and compares it to the Neural Collapse phenomenon in instance-wise deep classifiers.

Is RLHF More Difficult than Standard RL? A Theoretical Perspective

Yuanhao Wang, Qinghua Liu, Chi Jin

https://openreview.net/forum?id=sxZLrBqg50

Keywords: reinforcement learning theory, reinforcement learning from human feedback, preference-based reinforcement learning

Compressor summary: The paper shows that existing reward-based RL algorithms can be used to solve preference-based RL with minimal modifications, and applies the theory to various models with preferences.

Meta-in-context learning in large language models

Julian Coda-Forno, Marcel Binz, Zeynep Akata, Matthew Botvinick, Jane X Wang, Eric Schulz

https://openreview.net/forum?id=sx0xpaO0za

Keywords: Large language models, in-context learning, meta-learning, GPT-3

Compressor summary: The paper introduces meta-in-context learning, a method to improve large language models' in-context learning abilities by recursively applying them to different tasks, and shows its effectiveness on various domains and benchmarks.

The noise level in linear regression with dependent data

Ingvar Ziemann, Stephen Tu, George J. Pappas, Nikolai Matni

https://openreview.net/forum?id=swNtr6vGqg

Keywords: Learning Theory, Learning with dependent data, Time-Series

Compressor summary: We find upper limits for random linear regression with dependent data without assuming realizability, and our analysis shows how the error improves as we reduce misspecification.

A Unified, Scalable Framework for Neural Population Decoding

Mehdi Azabou, Vinam Arora, Venkataramana Ganesh, Ximeng Mao, Santosh B Nachimuthu, Michael Jacob Mendelson, Blake Aaron Richards, Matthew G Perich, Guillaume Lajoie, Eva L Dyer

https://openreview.net/forum?id=sw2Y0sirtM

Keywords: neural population, brain decoder, transformer, tokenization, sequence-to-sequence, electrophysiology, brain-computer interfaces

Compressor summary: The paper introduces a method to model neural activity across diverse recordings using tokenization, cross-attention, and PerceiverIO, enabling few-shot performance with minimal labels in new sessions.

CEIL: Generalized Contextual Imitation Learning

Jinxin Liu, Li He, Yachen Kang, Zifeng Zhuang, Donglin Wang, Huazhe Xu

https://openreview.net/forum?id=suzMI2P1rT

Keywords: imitation learning, reinforcement learning, offline imitation learning

Compressor summary: The paper introduces ContExtual Imitation Learning (CEIL), a versatile algorithm for imitation learning that learns from expert behaviors using hindsight embeddings, and demonstrates its effectiveness on various settings and benchmarks.

Riemannian SAM: Sharpness-Aware Minimization on Riemannian Manifolds

Jihun Yun, Eunho Yang

https://openreview.net/forum?id=strvrjSi3C

Keywords: optimization, riemannian, manifolds, sharpness-aware

Compressor summary: The paper introduces Riemannian SAM, a novel optimization algorithm for training geometric deep learning models, and shows its benefits on knowledge graph completion and machine translation tasks.

The Simplicity Bias in Multi-Task RNNs: Shared Attractors, Reuse of Dynamics, and Geometric Representation

Elia Turner, Omri Barak

https://openreview.net/forum?id=stDm3S0CV7

Keywords: Computational Neural Models; Recurrent Neural Networks; Multiple Tasks; Geometry;Dynamical Systems;Attractors;Neuroscience

Compressor summary: The authors study how Recurrent Neural Networks (RNNs) handle multiple tasks and find that they tend to reuse existing dynamics and opt for simple solutions, which they call the "simplicity bias".

Stable and low-precision training for large-scale vision-language models

Mitchell Wortsman, Tim Dettmers, Luke Zettlemoyer, Ari S. Morcos, Ali Farhadi, Ludwig Schmidt

https://openreview.net/forum?id=sqqASmpA2R

Keywords: CLIP, int8, stability

Compressor summary: The authors propose SwitchBack, a fast linear layer for int8 training, and recommend a hybrid of AdamW and Adafactor to prevent loss spikes in large language-vision models.

HASSOD: Hierarchical Adaptive Self-Supervised Object Detection

Shengcao Cao, Dhiraj Joshi, Liangyan Gui, Yu-Xiong Wang

https://openreview.net/forum?id=sqkGJjIRfG

Keywords: self-supervised learning, object detection

Compressor summary: HASSOD is a novel self-supervised object detection method that learns to detect objects and understand their compositions without human supervision, achieving improved performance and interpretability over existing methods.

Sparsity-Preserving Differentially Private Training of Large Embedding Models

Badih Ghazi, Yangsibo Huang, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Amer Sinha, Chiyuan Zhang

https://openreview.net/forum?id=sqTcCXkG4P

Keywords: Differential Privacy, Recommendation Systems, Embedding Models, Efficient Machine Learning

Compressor summary: The paper introduces DP-FEST and DP-AdaFEST, two new privacy-preserving algorithms for training large embedding models that reduce gradient size and maintain accuracy.

What’s Left? Concept Grounding with Logic-Enhanced Foundation Models

Joy Hsu, Jiayuan Mao, Joshua B. Tenenbaum, Jiajun Wu

https://openreview.net/forum?id=sq4o3tjWaj

Keywords: concept learning, visual reasoning, large language models, neuro-symbolic learning

Compressor summary: The paper proposes LEFT, a framework that learns to ground and reason with concepts across domains using a logic-based program executor and an LLM interpreter, outperforming previous methods in reasoning tasks.

Belief Projection-Based Reinforcement Learning for Environments with Delayed Feedback

Jangwon Kim, Hangyeol Kim, Jiwook Kang, Jongchan Baek, Soohee Han

https://openreview.net/forum?id=sq0m11cUMV

Keywords: time-delay system, reinforcement learning

Compressor summary: The paragraph describes a new actor-critic algorithm called BPQL that solves delayed feedback problems in reinforcement learning by reducing the state space size and outperforming traditional methods in continuous control tasks.

Unsupervised Semantic Correspondence Using Stable Diffusion

Eric Hedlin, Gopal Sharma, Shweta Mahajan, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi

https://openreview.net/forum?id=sovxUzPzLN

Keywords: Semantic Correspondence, Stable Diffusion, Optimization-based Inference

Compressor summary: The authors propose a method to find semantic correspondences between images using diffusion models' semantic knowledge without any training, achieving results comparable to strongly supervised methods.

Dynamically Masked Discriminator for GANs

Wentian Zhang, Haozhe Liu, Bing Li, Jinheng Xie, Yawen Huang, Yuexiang Li, Yefeng Zheng, Bernard Ghanem

https://openreview.net/forum?id=sodl2c3aTM

Keywords: Generative model, Generative Adversarial Network

Compressor summary: The paper proposes a new GAN method that helps the discriminator adapt faster to changes in generated data, improving the quality of generated results.

AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesis

Susan Liang, Chao Huang, Yapeng Tian, Anurag Kumar, Chenliang Xu

https://openreview.net/forum?id=snY3FOnlQi

Keywords: Scene synthesis, audio-visual, NeRF

Compressor summary: The authors propose a novel approach to synthesize realistic audio-visual scenes using NeRF, integrating knowledge of sound propagation and source-centric acoustic fields, and present a new dataset and demo videos.

Beyond MLE: Convex Learning for Text Generation

Chenze Shao, Zhengrui Ma, Min Zhang, Yang Feng

https://openreview.net/forum?id=sla7V80uWA

Keywords: Maximum likelihood estimation, Convex function, Text generation

Compressor summary: The paper proposes convex functions as an alternative to maximum likelihood estimation (MLE) for training text generation models, which can improve performance and efficiency on various tasks.

Survival Instinct in Offline Reinforcement Learning

Anqi Li, Dipendra Misra, Andrey Kolobov, Ching-An Cheng

https://openreview.net/forum?id=shePL2nbwl

Keywords: Offline RL, safe RL

Compressor summary: Offline reinforcement learning can produce good and safe policies even with wrong reward labels due to a survival instinct from pessimism and implicit biases in data collection.

FD-Align: Feature Discrimination Alignment for Fine-tuning Pre-Trained Models in Few-Shot Learning

Kun Song, Huimin Ma, Bochao Zou, Huishuai Zhang, Weiran Huang

https://openreview.net/forum?id=shXnfALjuH

Keywords: few-shot learning, CLIP, fine-tuning

Compressor summary: The paper introduces FD-Align, a method that improves few-shot learning by preserving spurious feature consistency and generalizing better to ID and OOD tasks.

Data Market Design through Deep Learning

Sai Srivatsa Ravindranath, Yanchen Jiang, David C. Parkes

https://openreview.net/forum?id=sgCrNMOuXp

Keywords: Data Markets, Information Design, Differentiable Economics, Economics, Deep Learning, Mechanism Design, Algorithmic Game Theory

Compressor summary: The paper proposes a deep learning approach to design revenue-optimal data markets by learning signaling schemes and handling obedience and incentive constraints.

On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling and Beyond

Thanh Nguyen-Tang, Raman Arora

https://openreview.net/forum?id=sdlh4gVOj8

Keywords: reinforcement learning, offline reinforcement learning

Compressor summary: The paper investigates what makes offline reinforcement learning sample-efficient and unifies three classes of algorithms using a new concept of data diversity.

Retrieval-Augmented Multiple Instance Learning

Yufei CUI, Ziquan Liu, Yixin CHEN, Yuchen Lu, Xinyue Yu, Xue Liu, Tei-Wei Kuo, Miguel R. D. Rodrigues, Chun Jason Xue, Antoni B. Chan

https://openreview.net/forum?id=scaKiAtbI3

Keywords: Multiple Instance Learning, Whole Slide Imaging, Nearest Neighbor Retrieval

Compressor summary: The paper introduces RAM-MIL, a framework that uses Optimal Transport to improve MIL performance on out-of-domain data in medical diagnosis tasks based on whole slide images.

VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset

Sihan Chen, Handong Li, Qunbo Wang, Zijia Zhao, Mingzhen Sun, Xinxin Zhu, Jing Liu

https://openreview.net/forum?id=scYa9DYUAy

Keywords: Cross-Modality Foundation Model, Cross-Modality Pretraining Dataset

Compressor summary: This paper introduces a large dataset (VAST-27M) for multi-modal video captioning and a model (VAST) that can process vision, audio, and subtitle modalities for various tasks.

Unleashing the Full Potential of Product Quantization for Large-Scale Image Retrieval

Yu Liang, Shiliang Zhang, Kenli Li, Xiaoyu Wang

https://openreview.net/forum?id=scG0cwftEe

Keywords: Deep Hash, Image Retrieval, Product Quantization

Compressor summary: The paragraph introduces a novel deep hashing framework using product quantization that addresses the limitations of current methods on large-scale datasets and achieves better performance.

Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing

Yelysei Bondarenko, Markus Nagel, Tijmen Blankevoort

https://openreview.net/forum?id=sbusw6LD41

Keywords: transformers, LLM, softmax, attention, outliers, quantization, post-training quantization

Compressor summary: The paper proposes two modifications to the attention mechanism in transformer models to reduce outliers and enable efficient quantization without compromising performance.

DreamSparse: Escaping from Plato’s Cave with 2D Diffusion Model Given Sparse Views

Paul Yoo, Jiaxian Guo, Yutaka Matsuo, Shixiang Shane Gu

https://openreview.net/forum?id=sZNBYvunEr

Keywords: Novel View Synthesis, Diffusion Model

Compressor summary: DreamSparse is a framework that uses a pre-trained diffusion model to generate high-quality novel view images from sparse views by incorporating 3D geometry information as a prior.

DiffPack: A Torsional Diffusion Model for Autoregressive Protein Side-Chain Packing

Yangtian Zhang, Zuobai Zhang, Bozitao Zhong, Sanchit Misra, Jian Tang

https://openreview.net/forum?id=sXMQPKbLXf

Keywords: protein side-chain packing, diffusion models, autoregressive models, geometric deep learning

Compressor summary: The authors propose DiffPack, a torsional diffusion model that predicts protein side-chain packing by learning joint distributions of side-chain angles and improves accuracy while reducing model size.

DISCOVER: Making Vision Networks Interpretable via Competition and Dissection

Konstantinos P. Panousis, Sotirios Chatzis

https://openreview.net/forum?id=sWNOvNXGLP

Keywords: Interpretability, Explainability, Network Dissection, Competitive Networks, Sparsity, Multimodal Models

Compressor summary: The authors present a method to generate textual descriptions of individual neuron functions in deep vision networks, enabling easier interpretation and exploration of their decision processes.

Ordering-based Conditions for Global Convergence of Policy Gradient Methods

Jincheng Mei, Bo Dai, Alekh Agarwal, Mohammad Ghavamzadeh, Csaba Szepesvari, Dale Schuurmans

https://openreview.net/forum?id=sW8yGZ4uVJ

Keywords: reinforcement learning, policy gradient, policy optimization, function approximation, global convergence

Compressor summary: The paper shows that policy gradient methods can converge globally for finite-arm bandits with linear function approximation, depending on properties between the policy update and the representation, without relying heavily on approximation error as a key quantity.

Weakly-Supervised Audio-Visual Segmentation

Shentong Mo, Bhiksha Raj

https://openreview.net/forum?id=sUqG96QqZM

Keywords: audio-visual learning, visual sound localization, audio-visual segmentation

Compressor summary: The paragraph describes a novel framework called WS-AVS that simplifies the supervision for audio-visual segmentation by using instance-level annotations and multi-scale contrastive learning, and shows its effectiveness on various scenarios.

D-Separation for Causal Self-Explanation

Wei Liu, Jun Wang, Haozhao Wang, Ruixuan Li, Zhiying Deng, YuanKai Zhang, Yang Qiu

https://openreview.net/forum?id=sUFGPYS25Q

Keywords: interpretability, causal inference, rationalization, self-explaining

Compressor summary: The Minimum Conditional Dependence (MCD) criterion is a novel way to uncover the causal rationale of NLP models by minimizing the dependence between non-selected input parts and the target label, which improves interpretability and performance compared to previous MMI-based methods.

Let the Flows Tell: Solving Graph Combinatorial Problems with GFlowNets

Dinghuai Zhang, Hanjun Dai, Nikolay Malkin, Aaron Courville, Yoshua Bengio, Ling Pan

https://openreview.net/forum?id=sTjW3JHs2V

Keywords: graph; combinatorial optimization; sampling; gflownets

Compressor summary: The paper presents a method to train conditional GFlowNets on MDPs for solving combinatorial optimization problems by sampling from the solution space and demonstrates their effectiveness in finding high-quality solutions.

Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learning

Hongyu Zang, Xin Li, Leiji Zhang, Yang Liu, Baigui Sun, Riashat Islam, Remi Tachet des Combes, Romain Laroche

https://openreview.net/forum?id=sQyRQjun46

Keywords: Bisimulation metrics, Reinforcement Learning, Representation Learning, Offline RL

Compressor summary: The paper analyzes why bisimulation methods work well in online RL but not in offline RL, and proposes using the expectile operator and reward scaling to improve performance on two benchmarks.

Cocktail: Mixing Multi-Modality Control for Text-Conditional Image Generation

Minghui Hu, Jianbin Zheng, Daqing Liu, Chuanxia Zheng, Chaoyue Wang, Dacheng Tao, Tat-Jen Cham

https://openreview.net/forum?id=sQBGVw5qH9

Keywords: Multi-modality, Image Generation, Diffusion

Compressor summary: Cocktail is a pipeline that mixes different modalities into one embedding and uses a hyper-network called gControlNet to align and infuse control signals from various sources into a pre-trained diffusion model, resulting in high-quality image synthesis with diverse contents.

A Measure-Theoretic Axiomatisation of Causality

Junhyung Park, Simon Buchholz, Bernhard Schölkopf, Krikamol Muandet

https://openreview.net/forum?id=sPLTQSf6GI

Keywords: Causality, probability theory, causal models

Compressor summary: The text introduces a new concept of causal space that combines probability theory with causality to overcome some limitations of current approaches.

Contrast Everything: A Hierarchical Contrastive Framework for Medical Time-Series

Yihe Wang, Yu Han, Haishuai Wang, Xiang Zhang

https://openreview.net/forum?id=sOQBHlCmzp

Keywords: Deep Learning, Contrastive Learning, Self-supervised Learning, Time Series, Healthcare

Compressor summary: COMET is a hierarchical framework that leverages data consistencies at multiple levels in medical time series to improve contrastive representation learning and outperforms existing methods on diverse datasets.

Projection-Free Online Convex Optimization via Efficient Newton Iterations

Khashayar Gatmiry, Zakaria Mhammedi

https://openreview.net/forum?id=sOOg1xJADA

Keywords: Online Learning, Online convex optimization, projection-free, Newton method

Compressor summary: The paper proposes a new online convex optimization algorithm that uses self-concordant barriers to avoid projections and achieve low regret without computing matrix inverses often.

Error Bounds for Learning with Vector-Valued Random Features

Samuel Lanthaler, Nicholas H. Nelsen

https://openreview.net/forum?id=sLr1sohnmo

Keywords: random features, random feature model, operator learning, vector-valued

Compressor summary: The paper analyzes vector-valued random features ridge regression in an infinite-dimensional setting, providing strong consistency and minimax optimal convergence rates without relying on random matrix theory.

On skip connections and normalisation layers in deep optimisation

Lachlan Ewen MacDonald, Jack Valmadre, Hemanth Saratchandran, Simon Lucey

https://openreview.net/forum?id=sLhXMkI0kx

Keywords: optimisation, optimization, skip, connection, normalisation, normalization, deep, learning, polyak, lojasiewicz, lipschitz

Compressor summary: The paper presents a theory for gradient optimisation of deep neural networks that explains the roles of normalisation layers and skip connections, and shows how they enable training to global optima and acceleration of training.

ELDEN: Exploration via Local Dependencies

Zizhao Wang, Jiaheng Hu, Peter Stone, Roberto Martín-Martín

https://openreview.net/forum?id=sL4pJBXkxu

Keywords: reinforcement learning; intrinsic motivation; exploration

Compressor summary: ELDEN is a novel intrinsic reward function that encourages reinforcement learning agents to explore environments with complex dependencies by exploiting uncertainty in the learned dynamics.

Cross-links Matter for Link Prediction: Rethinking the Debiased GNN from a Data Perspective

Zihan Luo, Hong Huang, Jianxun Lian, Xiran Song, Xing Xie, Hai Jin

https://openreview.net/forum?id=sJDkwMVqb9

Keywords: Cross-links, Debias, Graph Neural Networks, Link Prediction

Compressor summary: The paper proposes a twin-structure framework to reduce data bias between internal-links and cross-links in GNN-based link prediction, enhancing both bias mitigation and utility.

VOCE: Variational Optimization with Conservative Estimation for Offline Safe Reinforcement Learning

Jiayi Guan, Guang Chen, Jiaming Ji, Long Yang, Ao Zhou, Zhijun Li, changjun jiang

https://openreview.net/forum?id=sIU3WujeSl

Keywords: Offline safe reinforcement learning, Pessimistic conservative estimation, Variational optimization, Reinforcement Learning

Compressor summary: VOCE is a new offline safe RL algorithm that uses probabilistic inference and pessimistic estimation to learn policies that satisfy safety constraints while optimizing rewards.

Q-DM: An Efficient Low-bit Quantized Diffusion Model

Yanjing Li, Sheng Xu, Xianbin Cao, Xiao Sun, Baochang Zhang

https://openreview.net/forum?id=sFGkL5BsPi

Keywords: network quantization, diffusion model, image synthesize

Compressor summary: The paper proposes TaQ and NeM methods to improve low-bit quantized diffusion models, reducing computation and memory costs while maintaining high-quality data generation.

SutraNets: Sub-series Autoregressive Networks for Long-Sequence, Probabilistic Forecasting

Shane Bergsma, Tim Zeyl, Lei Guo

https://openreview.net/forum?id=sC4RbbVKbu

Keywords: time series, probabilistic forecasting, autoregressive generative models, neural networks

Compressor summary: SutraNets are a new method for predicting long time series by factoring likelihood into conditional probabilities and generating multivariate outputs to reduce error accumulation.

Doubly Robust Augmented Transfer for Meta-Reinforcement Learning

Yuankun Jiang, Nuowen Kan, Chenglin Li, Wenrui Dai, Junni Zou, Hongkai Xiong

https://openreview.net/forum?id=sABYNWKcwK

Keywords: Meta-reinforcement learning, doubly robust (DR), sample transfer

Compressor summary: The paper proposes a new Meta-RL method called DRaT that addresses dynamics and reward function variations across tasks by using a doubly robust estimator to transfer informative samples from other tasks.

RH-BrainFS: Regional Heterogeneous Multimodal Brain Networks Fusion Strategy

Hongting Ye, Yalu Zheng, Yueying Li, Ke Zhang, Youyong Kong, Yonggui Yuan

https://openreview.net/forum?id=s97ezbqoDZ

Keywords: Multimodal, Neuroscience, Subgraph, Transformer

Compressor summary: The authors present RH-BrainFS, a novel multimodal brain network fusion strategy that addresses regional heterogeneity between structural and functional connectivity, and shows its effectiveness in various neuroscience tasks.

AND: Adversarial Neural Degradation for Learning Blind Image Super-Resolution

Fangzhou Luo, Xiaolin Wu, Yanhui Guo

https://openreview.net/forum?id=s8QsYV1VZ2

Keywords: Blind Image Super-Resolution

Compressor summary: The paper proposes an adversarial neural degradation (AND) model that improves image super-resolution by generating a wide range of complex degradation effects without explicit supervision, resulting in better generalization and performance on real-world images.

Soft-Unification in Deep Probabilistic Logic

Jaron Maene, Luc De Raedt

https://openreview.net/forum?id=s86M8naPSv

Keywords: neuro-symbolic AI, probabilistic logic, embeddings

Compressor summary: The paragraph discusses soft-unification, a technique for combining logic and neural concepts in AI, and proposes a new framework called DeepSoftLog that improves upon previous systems by introducing probabilistic semantics and achieving better performance on neuro-symbolic benchmarks.

Don’t Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner

Zhengxiang Shi, Aldo Lipani

https://openreview.net/forum?id=s7xWeJQACI

Keywords: Continued Pre-training, Prompt-based Fine-tuning, Language Models

Compressor summary: The study proposes Prompt-based Continued Pre-training (PCP), which improves prompt-based fine-tuning in natural language processing tasks by combining instruction tuning with conventional continued pre-training using unsupervised objectives.

On Certified Generalization in Structured Prediction

Bastian Boll, Christoph Schnoerr

https://openreview.net/forum?id=s1jQ91yFAb

Keywords: Structured Prediction, PAC-Bayes, Concentration Inequalities, Statistical Learning Theory, Knothe-Rosenblatt Rearrangement

Compressor summary: The paper proposes a new way to measure how well structured prediction models generalize using generative models and Wasserstein dependency matrices.

Focused Transformer: Contrastive Training for Context Scaling

Szymon Tworkowski, Konrad Staniszewski, Mikołaj Pacek, Yuhuai Wu, Henryk Michalewski, Piotr Miłoś

https://openreview.net/forum?id=s1FjXzJ0jy

Keywords: Transformers, Language Models, Natural Language Processing

Compressor summary: The Focused Transformer (FoT) technique addresses the distraction issue in large language models by enhancing the structure of the (key, value) space and extending the effective context length through contrastive learning.

GALOPA: Graph Transport Learning with Optimal Plan Alignment

Yejiang Wang, Yuhai Zhao, Daniel Zhengkui Wang, Ling Li

https://openreview.net/forum?id=rzlqOVExUA

Keywords: graph neural network; self-supervised learning; optimal transport;

Compressor summary: The paper proposes a self-supervised graph learning method that preserves both structural and matching information between graphs using optimal transport plans, and shows its advantages over existing methods in different scenarios.

Private Federated Frequency Estimation: Adapting to the Hardness of the Instance

Jingfeng Wu, Wennan Zhu, Peter Kairouz, Vladimir Braverman

https://openreview.net/forum?id=rzDBoh1tBh

Keywords: sketch, federated analytics, privacy

Compressor summary: The paper proposes an improved sketch algorithm for federated frequency estimation with multiple communication rounds, and a two-phase approach that adapts the sketch size based on problem complexity and ensures differential privacy.

EgoEnv: Human-centric environment representations from egocentric video

Tushar Nagarajan, Santhosh Kumar Ramakrishnan, Ruta Desai, James Hillis, Kristen Grauman

https://openreview.net/forum?id=rybsHQ4DXy

Keywords: egocentric video, 3D environment, sim2real, sim-to-real, episodic memory

Compressor summary: The paper proposes an approach to understand human activities in their environment using egocentric video and 3D simulation, and shows its effectiveness on real-world videos.

Matrix Compression via Randomized Low Rank and Low Precision Factorization

Rajarshi Saha, Varun Srivastava, Mert Pilanci

https://openreview.net/forum?id=rxsCTtkqA9

Keywords: Matrix compression, Randomized low rank factorization, Randomized SVD, Sketching, Quantized embeddings, Random matrices

Compressor summary: The paper proposes an algorithm to decompose large matrices into low rank factors and quantize them for efficient storage and processing.

Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors

Paul Steven Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Cohen Ethan, Aidan James Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, Kenneth Norman, Tanishq Mathew Abraham

https://openreview.net/forum?id=rwrblCYb2A

Keywords: fMRI, computational neuroscience, mind reading, diffusion models

Compressor summary: MindEye is a novel method to reconstruct and retrieve images from brain activity using fMRI and specialized submodules for retrieval and reconstruction.

Break It Down: Evidence for Structural Compositionality in Neural Networks

Michael A. Lepori, Thomas Serre, Ellie Pavlick

https://openreview.net/forum?id=rwbzMiuFQl

Keywords: Deep Learning, Compositionality, Cognitive Science

Compressor summary: The authors investigate whether neural networks use modular subnetworks or template matching to solve complex tasks, and find evidence that they can learn structural compositionality without relying on symbolic mechanisms.

DäRF: Boosting Radiance Fields from Sparse Input Views with Monocular Depth Adaptation

Jiuhn Song, Seonghoon Park, Honggyu An, Seokju Cho, Min-Seop Kwak, Sungjin Cho, Seungryong Kim

https://openreview.net/forum?id=rsrfEIdawr

Keywords: Neural Radiance Fields, 3D Reconstruction, Few-shot NeRF, Monocular Priors

Compressor summary: The DäRF framework combines NeRF and monocular depth estimation to improve novel view synthesis and 3D geometry reconstruction with a small number of real-world images.

PointGPT: Auto-regressively Generative Pre-training from Point Clouds

Guangyan Chen, Meiling Wang, Yi Yang, Kai Yu, Li Yuan, Yufeng Yue

https://openreview.net/forum?id=rqE0fEQDqs

Keywords: Generative Pre-training Transformer; GPT; Auto-regressively Generative Pre-training; Self-supervised Learning; Point clouds

Compressor summary: PointGPT extends the GPT language model to point clouds, using a novel auto-regressive generation task and a dual masking strategy, achieving state-of-the-art results on various downstream tasks.

On the Role of Noise in the Sample Complexity of Learning Recurrent Neural Networks: Exponential Gaps for Long Sequences

Alireza Fathollah Pour, Hassan Ashtiani

https://openreview.net/forum?id=rpuEARqB54

Keywords: PAC Learning, Recurrent Neural Networks, Noise, Sample Complexity

Compressor summary: The paper studies how adding noise to multi-layered sigmoid networks affects their sample complexity for classifying sequences, and shows an exponential gap between noisy and non-noisy networks.

ConRad: Image Constrained Radiance Fields for 3D Generation from a Single Image

Senthil Purushwalkam, Nikhil Naik

https://openreview.net/forum?id=roGYQvarnC

Keywords: 3D, generation, diffusion, viewpoint

Compressor summary: The paper introduces ConRad, a novel method that uses pretrained image generation models to reconstruct 3D objects from a single RGB image, capturing the appearance and preserving details while producing realistic 3D reconstructions.

AdaPlanner: Adaptive Planning from Feedback with Language Models

Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai, Chao Zhang

https://openreview.net/forum?id=rnKgbKmelt

Keywords: Large language models, decision making, closed-loop planning

Compressor summary: AdaPlanner is a closed-loop approach that allows large language models to adaptively refine their plans in response to environmental feedback and outperforms state-of-the-art methods with fewer samples.

Managing Temporal Resolution in Continuous Value Estimation: A Fundamental Trade-off

Zichen Zhang, Johannes Kirschner, Junxi Zhang, Francesco Zanini, Alex Ayoub, Masood Dehghan, Dale Schuurmans

https://openreview.net/forum?id=rmQgQCZWiP

Keywords: Reinforcement Learning, Policy Evaluation, Temporal Discretization, Continuous Time, LQR

Compressor summary: The paper analyzes how time discretization affects Monte-Carlo policy evaluation for LQR systems and finds a trade-off between approximation and statistical error, leading to an optimal choice of temporal resolution for data efficiency.

False Discovery Proportion control for aggregated Knockoffs

Alexandre Blain, Bertrand Thirion, Olivier Grisel, Pierre Neuvial

https://openreview.net/forum?id=rlPUJ60bwM

Keywords: Knockoffs, Derandomization of Knockoffs, False Discoveries Proportion control, Controlled variable selection, Statistical inference, High-dimensional inference

Compressor summary: KOPI is a new method that improves Knockoff-based inference by controlling the actual proportion of false discoveries and using a novel aggregation technique.

Transformed Low-Rank Parameterization Can Help Robust Generalization for Tensor Neural Networks

Andong Wang, Chao Li, Mingyuan Bai, Zhong Jin, Guoxu Zhou, Qibin Zhao

https://openreview.net/forum?id=rih3hsSWx8

Keywords: Tensor SVD; Tensor Neural Networks; Transformed Low-rankness; Adversarial Generalization; Implicit Bias.

Compressor summary: The paper analyzes the theoretical generalization error of neural networks with t-product layers (t-NNs) and shows that transformed low-rank parameterization can improve their adversarial robustness.

DreamHuman: Animatable 3D Avatars from Text

Nikos Kolotouros, Thiemo Alldieck, Andrei Zanfir, Eduard Gabriel Bazavan, Mihai Fieraru, Cristian Sminchisescu

https://openreview.net/forum?id=rheCTpRrxI

Keywords: text to 3d; 3d avatars

Compressor summary: DreamHuman is a method that creates realistic 3D human avatars from text descriptions by combining different models and optimization techniques, resulting in diverse and high-quality animatable models.

Counterfactual Evaluation of Peer-Review Assignment Policies

Martin Saveski, Steven Jecmen, Nihar B Shah, Johan Ugander

https://openreview.net/forum?id=rhIfzCZoXG

Keywords: peer review, causal inference, counterfactual policy evaluation

Compressor summary: This paper proposes and applies novel methods to evaluate how changes in peer-review assignment algorithms affect review quality using randomness and data from two computer science venues.

Policy Optimization for Continuous Reinforcement Learning

Hanyang Zhao, Wenpin Tang, David Yao

https://openreview.net/forum?id=rfcak9EV99

Keywords: exploratory stochastic control, occupation time, performance difference, policy optimization

Compressor summary: The authors develop a new concept called occupation time for continuous reinforcement learning and show how it can improve performance and optimization methods in this setting.

Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies

Wei Fang, Zhaofei Yu, Zhaokun Zhou, Ding Chen, Yanqi Chen, Zhengyu Ma, Timothée Masquelier, Yonghong Tian

https://openreview.net/forum?id=rfTFJvTkr2

Keywords: Spiking Neural Network, SNN, deep learning, spiking neuron, neuromorphic computing

Compressor summary: The authors propose Parallel Spiking Neurons, which improve simulation speed and accuracy in spiking neural networks by reformulating neuronal dynamics without reset and enabling parallel processing of inputs.

Why think step by step? Reasoning emerges from the locality of experience

Ben Prystawski, Michael Y. Li, Noah Goodman

https://openreview.net/forum?id=rcXXNFVlEn

Keywords: chain-of-thought; language models; reasoning

Compressor summary: The text discusses how chain-of-thought reasoning helps language models make better inferences when trained on locally structured data with strong variable dependencies, reducing bias and improving efficiency.

Adaptive Test-Time Personalization for Federated Learning

Wenxuan Bao, Tianxin Wei, Haohan Wang, Jingrui He

https://openreview.net/forum?id=rbw9xCU6Ci

Keywords: federated learning, personalized federated learning, test-time adaptation

Compressor summary: The paper introduces ATP, a novel test-time personalized federated learning algorithm that adapts global models in an unsupervised way without labeled data from multiple clients with different distributions, achieving strong generalization and outperforming existing methods.

Swarm Reinforcement Learning for Adaptive Mesh Refinement

Niklas Freymuth, Philipp Dahlinger, Tobias Daniel Würth, Simon Reisch, Luise Kärger, Gerhard Neumann

https://openreview.net/forum?id=rZqRu8e4uc

Keywords: Adaptive Mesh Refinement, Finite Element Method, Swarm Reinforcement Learning, Graph Neural Networks

Compressor summary: The paper proposes ASMR, a novel Adaptive Swarm Markov Decision Process for adaptive mesh refinement in engineering simulations, which achieves significant speedup and quality improvements compared to existing methods.

On the impact of activation and normalization in obtaining isometric embeddings at initialization

Amir Joudaki, Hadi Daneshmand, Francis Bach

https://openreview.net/forum?id=rY4sA9qYKy

Keywords: dynamical isometry, Lyapunov analysis, random neural networks

Compressor summary: In this paper, we explore the structure of the penultimate Gram matrix in deep neural networks, which contains the pairwise inner products of outputs corresponding to a batch of inputs. In several architectures it has been observed that this Gram matrix becomes degenerate with depth at initialization, which dramatically slows training. Normalization layers, such as batch or layer normalization, play a pivotal role in preventing the rank collapse issue. Despite promising advances, the existing theoretical results do not extend to layer normalization, which is widely used in transformers, and can not quantitatively characterize the role of non-linear activations. To bridge this gap, we prove that layer normalization, in conjunction with activation layers, biases the Gram matrix of a multilayer perceptron towards the identity matrix at an exponential rate with depth at initialization. We quantify this rate using the Hermite expansion of the activation function.

Wasserstein Gradient Flows for Optimizing Gaussian Mixture Policies

Hanna Ziesche, Leonel Rozo

https://openreview.net/forum?id=rW4mNcDxpS

Keywords: Policy optimization, robot learning, reinforcement learning, Gaussian mixture models, optimal transport, robotics

Compressor summary: The paper presents a novel policy optimization method for robots based on optimal transport and Riemannian optimization of GMMs, which enhances performance in different robot tasks.

PRED: Pre-training via Semantic Rendering on LiDAR Point Clouds

Hao Yang, Haiyang Wang, Di Dai, Liwei Wang

https://openreview.net/forum?id=rUldfB4SPT

Keywords: Pre-Train, Autonomous Driving, LiDAR, 3D Object Detection

Compressor summary: The paper proposes a new method, PRED, that uses images and neural rendering to help train point cloud encoders for 3D perception tasks in autonomous driving, addressing the challenges of incompleteness and occlusions.

Locality Sensitive Hashing in Fourier Frequency Domain For Soft Set Containment Search

Indradyumna Roy, Rishi Agarwal, Soumen Chakrabarti, Anirban Dasgupta, Abir De

https://openreview.net/forum?id=rUf0GV5CuU

Keywords: Locality sensitive hashing, Fourier transform, Order embeddings

Compressor summary: The paper proposes FourierHashNet, a data-sensitive and trainable index for fast retrieval of relevant documents using hinge distance transformed into dominance similarity in the frequency domain.

Homotopy-based training of NeuralODEs for accurate dynamics discovery

Joon-Hyuk Ko, Hankyul Koh, Nojun Park, Wonho Jhe

https://openreview.net/forum?id=rUFckPrzXR

Keywords: neural ordinary differential equations, synchronization, homotopy optimization, loss landscape, dynamical systems

Compressor summary: This paper introduces a new training method for NeuralODEs that improves their performance and speed by using synchronization and homotopy optimization without changing the model architecture.

Efficient Learning of Linear Graph Neural Networks via Node Subsampling

Seiyun Shin, Ilan Shomorony, Han Zhao

https://openreview.net/forum?id=rQI3FOzo1f

Keywords: Graph neural networks, Random sampling, Regression

Compressor summary: Graph Neural Networks (GNNs) aim to learn from large graphs, but their computational cost can be high; this paper proposes an efficient training method for a specific type of GNN by subsampling nodes and using leverage score sampling.

Continuous-time Analysis of Anchor Acceleration

Jaewook J. Suh, Jisun Park, Ernest K. Ryu

https://openreview.net/forum?id=rN99gLCBe4

Keywords: acceleration, convex optimization, continuous-time analysis, monotone operator, monotone inclusion, minimax optimization, fixed-point problem, anchor acceleration

Compressor summary: This paper analyzes the continuous-time models of anchor acceleration, a minimization technique with an unclear mechanism, and proposes an adaptive method based on the findings.

Preconditioning Matters: Fast Global Convergence of Non-convex Matrix Factorization via Scaled Gradient Descent

Xixi Jia, Hailin Wang, Jiangjun Peng, Xiangchu Feng, Deyu Meng

https://openreview.net/forum?id=rLpLjCBW4J

Keywords: Non-convex optimization, matrix factorization, low rank, scaled gradient descent

Compressor summary: The paper proposes two preconditioned gradient descent methods for low-rank matrix factorization that converge faster and do not require small learning rates or initializations.

ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections

Chun-Han Yao, Amit Raj, Wei-Chih Hung, Michael Rubinstein, Yuanzhen Li, Ming-Hsuan Yang, Varun Jampani

https://openreview.net/forum?id=rJc5Lsn5QU

Keywords: 3D articulated shape, animal body estimation, diffusion for 3D

Compressor summary: ARTIC3D is a self-supervised framework that reconstructs 3D shapes from sparse images using skeleton-based surface representation and 2D diffusion priors, achieving high-quality results even with noisy and occluded inputs.

Adversarial Counterfactual Environment Model Learning

Xiong-Hui Chen, Yang Yu, Zhengmao Zhu, ZhiHua Yu, Chen Zhenjun, Chenghe Wang, Yinan Wu, Rong-Jun Qin, Hongqiu Wu, Ruijin Ding, Huang Fangsheng

https://openreview.net/forum?id=rHAX0LRwk8

Keywords: environment model learning, offline reinforcement learning, off-policy evaluation, individual treatment effects estimation, causal inference, adversarial learning

Compressor summary: The paper proposes a new model-learning objective called AWRM to address the selection bias of behavior policies in environment dynamics models and shows that it improves counterfactual prediction and downstream tasks using the GALILEO algorithm.

Formalizing locality for normative synaptic plasticity models

Colin Bredenberg, Ezekiel Williams, Cristina Savin, Blake Aaron Richards, Guillaume Lajoie

https://openreview.net/forum?id=rGN3X9jnEg

Keywords: synaptic plasticity, computational neuroscience

Compressor summary: The authors propose formal definitions of locality for machine learning algorithms that aim to be biologically plausible, in order to make clear what quantities cannot be included in a learning rule and test their predictions against them.

FLuID: Mitigating Stragglers in Federated Learning using Invariant Dropout

Irene Wang, Prashant J. Nair, Divya Mahajan

https://openreview.net/forum?id=rG1M3kOVba

Keywords: Federated Learning

Compressor summary: FLuID is a method to balance the training load in Federated Learning by creating sub-models for low-performance devices without sacrificing accuracy.

Learning better with Dale’s Law: A Spectral Perspective

Pingsheng Li, Jonathan Cornford, Arna Ghosh, Blake Aaron Richards

https://openreview.net/forum?id=rDiMgZulwi

Keywords: Dale's Law, RNNs, brain-inspired neural networks, DANNs, computational neuroscience, spectral properties, inhibition

Compressor summary: The authors extend Dale's ANNs to RNNs, showing that good performance is possible while respecting Dale's Law, which states that neurons must be excitatory or inhibitory. They also find that the spectral properties of the recurrent weight matrix at initialisation are more impactful on network performance than sign constraints.

Learning Invariant Molecular Representation in Latent Discrete Space

Xiang Zhuang, Qiang Zhang, Keyan Ding, Yatao Bian, Xiao Wang, Jingsong Lv, Hongyang Chen, Huajun Chen

https://openreview.net/forum?id=r9fzp8eyhZ

Keywords: molecular representation learning, out-of-distribution

Compressor summary: The authors propose a new molecular representation learning framework that improves generalization and robustness against distribution shifts, using a novel strategy of first encoding and then separating features in the latent space.

Learning to Group Auxiliary Datasets for Molecule

Tinglin Huang, Ziniu Hu, Zhitao Ying

https://openreview.net/forum?id=r9eZH6WNm2

Keywords: molecule, routing mechanism, meta gradient

Compressor summary: MolGroup is a method that uses graph structure similarity and task similarity to select the best auxiliary datasets for small molecule machine learning models, improving their performance.

Grounding Neural Inference with Satisfiability Modulo Theories

Zifan Wang, Saranya Vijayakumar, Kaiji Lu, Vijay Ganesh, Somesh Jha, Matt Fredrikson

https://openreview.net/forum?id=r8snfquzs3

Keywords: Satisfiability Modulo Theories, Solver Layer, Combinatorial Problem, MAXSAT, SAT

Compressor summary: The paper proposes SMTLayer, a technique that integrates SMT solvers into DNNs to encode domain knowledge as mathematical formulas, enabling more efficient, robust, and interpretable learning.

TexQ: Zero-shot Network Quantization with Texture Feature Distribution Calibration

Xinrui Chen, Yizhi Wang, Renao Yan, Yiqing Liu, Tian Guan, Yonghong He

https://openreview.net/forum?id=r8LYNleLf9

Keywords: Zero-shot quantization, Texture feature calibration, Post-training quantization, low bit width, Neural network compression

Compressor summary: The paper proposes TexQ, a novel zero-shot quantization method for neural networks that synthesizes calibration images and uses mixup knowledge distillation to generate diverse samples, improving performance in ultra-low bit width quantization.

Learning Adaptive Tensorial Density Fields for Clean Cryo-ET Reconstruction

YUANHAO WANG, Ramzi Idoughi, Wolfgang Heidrich

https://openreview.net/forum?id=r7g9nFsulw

Keywords: Neural density fields, Coordinate-based representations, Quadtree structure, Cryo-electron microscope

Compressor summary: The authors propose a new learning-based method for improving 3D structure reconstruction from cryo-ET data, which addresses challenges such as missing data and high noise levels by using an adaptive tensor representation and a novel loss function.

Meta-Learning Adversarial Bandit Algorithms

Mikhail Khodak, Ilya Osadchiy, Keegan Harris, Nina Balcan, Kfir Yehuda Levy, Ron Meir, Steven Wu

https://openreview.net/forum?id=r6xGZ0XL2g

Keywords: online learning, multi-armed bandits, meta-learning, multi-task learning, bandit linear optimization

Compressor summary: The authors propose meta-algorithms that improve performance across multiple similar tasks using bandit feedback, by tuning hyperparameters of inner learners in online-within-online settings for MAB and BLO.

Global Update Tracking: A Decentralized Learning Algorithm for Heterogeneous Data

Sai Aparna Aketi, Abolfazl Hashemi, Kaushik Roy

https://openreview.net/forum?id=qyixBZl8Ph

Keywords: Federated Learning, Decentralized Learning, Non-IID Data, Heterogeneous data distribution, Peer-to-peer connectivity

Compressor summary: The paper proposes Global Update Tracking (GUT), a decentralized learning algorithm that reduces the impact of heterogeneous data distribution and achieves state-of-the-art performance on various Computer Vision datasets.

Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information

Arman Zharmagambetov, Brandon Amos, Aaron M Ferber, Taoan Huang, Bistra Dilkina, Yuandong Tian

https://openreview.net/forum?id=qyEm4tF2p1

Keywords: learning surrogates, predict+optimize framework, combinatorial nonlinear optimization, argmin differentiation

Compressor summary: The paper proposes a learnable landscape surrogate to improve learning-integrated optimization by addressing challenges such as problem uncertainty and sparse gradients.

Diffusion Schrödinger Bridge Matching

Yuyang Shi, Valentin De Bortoli, Andrew Campbell, Arnaud Doucet

https://openreview.net/forum?id=qy07OHsJT5

Keywords: diffusion Schrödinger bridge, bridge matching, optimal transport

Compressor summary: The paragraph discusses a new method, Iterative Markovian Fitting (IMF), for solving Schrödinger bridges (SBs) problems and a novel algorithm, Diffusion Schrödinger Bridge Matching (DSBM), which significantly improves SB numerics and recovers various recent transport methods.

Reinforcement Learning with Simple Sequence Priors

Tankred Saanum, Noemi Elteto, Peter Dayan, Marcel Binz, Eric Schulz

https://openreview.net/forum?id=qxF8Pge6vM

Keywords: Deep Reinforcement Learning, Compression, Sequence learning, Information bottleneck, Mutual information

Compressor summary: The paper proposes an RL algorithm that learns simple action sequences, either learned by autoregressive models or compressed by data algorithms, and shows it outperforms model-free approaches in continuous control tasks.

Multimodal Deep Learning Model Unveils Behavioral Dynamics of V1 Activity in Freely Moving Mice

Aiwen Xu, Yuchen Hou, Cris M. Niell, Michael Beyeler

https://openreview.net/forum?id=qv5UZJTNda

Keywords: neuroscience, cognitive science, multimodal learning, representation learning, network architecture, computational biology, visual perception

Compressor summary: The authors introduce a multimodal recurrent neural network that can predict visual cortex activity in free-moving mice by integrating visual input, behavior, and temporal dynamics, revealing new insights into cortical function.

On the Robustness of Mechanism Design under Total Variation Distance

Anuran Makur, Marios Mertzanidis, Alexandros Psomas, Athina Terzoglou

https://openreview.net/forum?id=qumBHr77ht

Keywords: mechanism design, revenue maximization, correlated distributions, total variation distance

Compressor summary: The paper studies how to design truthful mechanisms for agents with unknown and correlated valuation functions, showing that some existing mechanisms are robust to variations in the distribution, and providing various applications and extensions.

Performance Scaling via Optimal Transport: Enabling Data Selection from Partially Revealed Sources

Feiyang Kang, Hoang Anh Just, Anit Kumar Sahu, Ruoxi Jia

https://openreview.net/forum?id=quMBEd27x9

Keywords: data-centric AI, data acquisition, data valuation, performance prediction, data markets, optimal transport, scaling laws

Compressor summary: The paper introduces , a framework that predicts model performance and supports data selection decisions using partial samples of prospective data sources, improving over existing methods in accuracy and computation costs.

TabMT: Generating tabular data with masked transformers

Manbir S Gulati, Paul F Roysdon

https://openreview.net/forum?id=qs4swxtIAQ

Keywords: Tabular Data, Deep Learning, Generative Modeling, Transformers, Masked Transformers, Synthetic data

Compressor summary: TabMT is a new Masked Transformer model that generates synthetic tabular data with strong performance, handling missing data and various data types, and providing good privacy tradeoffs.

PHOTOSWAP: Personalized Subject Swapping in Images

Jing Gu, Yilin Wang, Nanxuan Zhao, Tsu-Jui Fu, Wei Xiong, Qing Liu, Zhifei Zhang, HE Zhang, Jianming Zhang, HyunJoon Jung, Xin Eric Wang

https://openreview.net/forum?id=qqcIM8NiiB

Keywords: image editing, diffusion model, text to image generation

Compressor summary: The text introduces Photoswap, a novel approach for personalized subject swapping in images without training, which preserves the pose and coherence of the image while swapping subjects.

Lower Bounds on Adaptive Sensing for Matrix Recovery

Praneeth Kacham, David Woodruff

https://openreview.net/forum?id=qptO6YDZEP

Keywords: Compressed Sensing, Matrix Recovery, Low rank approximation

Compressor summary: The authors study the lower bounds on adaptive algorithms for recovering low rank matrices using linear measurements, showing that any such algorithm must perform a certain number of rounds to achieve a satisfactory approximation.

Aligning Synthetic Medical Images with Clinical Knowledge using Human Feedback

Shenghuan Sun, Gregory Goldgof, Atul Butte, Ahmed Alaa

https://openreview.net/forum?id=qlnlamFQEa

Keywords: Synthetic clinical data, Machine learning for healthcare

Compressor summary: The paper introduces a pathologist-in-the-loop framework that uses human feedback to improve the quality and plausibility of synthetic medical images generated by conditional diffusion models, addressing challenges in assessing clinical sensibility and incorporating domain knowledge.

Bayesian nonparametric (non-)renewal processes for analyzing neural spike train variability

David Liu, Máté Lengyel

https://openreview.net/forum?id=qlJoo2y3gY

Keywords: Gaussian processes, renewal processes, point processes, neural data analysis, Bayesian machine learning, non-stationary time series

Compressor summary: The authors propose a Bayesian approach to model neural spiking activity that captures instantaneous variability in response to covariates, and apply it to two datasets of animal navigation, showing improved predictive power and richer patterns of modulation.

Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm

Miaoxi Zhu, Li Shen, Bo Du, Dacheng Tao

https://openreview.net/forum?id=ql6LVyi2Dg

Keywords: decentralized algorithm, minimax problem, algorithmic stability, generalization analysis

Compressor summary: The paper studies how decentralized algorithms like D-SGDA can generalize well despite their structure, by analyzing their primal-dual generalization bound using algorithmic stability.

Rank-1 Matrix Completion with Gradient Descent and Small Random Initialization

Daesung Kim, Hye Won Chung

https://openreview.net/forum?id=qjqJL2lfkH

Keywords: Matrix completion, gradient descent, random initialization

Compressor summary: The paper analyzes gradient descent algorithm's convergence behavior on rank-1 symmetric matrix completion problem with small random initialization, and shows that it converges to the ground truth when more samples are available.

High-Fidelity Audio Compression with Improved RVQGAN

Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, Kundan Kumar

https://openreview.net/forum?id=qjnl1QUnFA

Keywords: audio generation, audio compression, GAN, audio, speech

Compressor summary: The authors introduce a universal neural audio compression algorithm that achieves 90x compression of various audio domains with high fidelity, outperforming existing methods and providing open-source code and weights.

Transformers learn through gradual rank increase

Emmanuel Abbe, Samy Bengio, Enric Boix-Adserà, Etai Littwin, Joshua M. Susskind

https://openreview.net/forum?id=qieeNlO3C7

Keywords: transformers, low-rank bias, incremental learning

Compressor summary: The text describes how transformers learn incrementally, leading to a growing difference between trained and initial weights, and provides theoretical and empirical evidence for this behavior.

Diffusion Self-Guidance for Controllable Image Generation

Dave Epstein, Allan Jabri, Ben Poole, Alexei A Efros, Aleksander Holynski

https://openreview.net/forum?id=qgv56R2YJ7

Keywords: generative models, image editing, diffusion, guidance

Compressor summary: Self-guidance is a technique that uses internal representations of diffusion models to control various aspects of image generation and manipulation without additional training or models.

Convolutional Visual Prompt for Robust Visual Perception

Yun-Yun Tsai, Chengzhi Mao, Junfeng Yang

https://openreview.net/forum?id=qgmrC8jhCo

Keywords: self-supervised learning, representation learning, visual prompts, domain generalization, input adaptation

Compressor summary: Convolutional visual prompts (CVP) are a lightweight method for adapting vision models in test-time settings without labels, improving their robustness against out-of-distribution samples.

Affinity-Aware Graph Networks

Ameya Velingker, Ali Kemal Sinop, Ira Ktena, Petar Veličković, Sreenivas Gollapudi

https://openreview.net/forum?id=qgiG7WZohZ

Keywords: graph neural networks, message passing, effective resistance, hitting time

Compressor summary: The paper explores using random walk measures as features in graph neural networks to improve performance on various node and graph property prediction tasks with low computational complexity.

ANTN: Bridging Autoregressive Neural Networks and Tensor Networks for Quantum Many-Body Simulation

Zhuo Chen, Laker Newhouse, Eddie Chen, Di Luo, Marin Soljacic

https://openreview.net/forum?id=qdsDy0zbn4

Keywords: Autoregressive neural network, tensor network, quantum many-body physics, variational Monte Carlo

Compressor summary: The Autoregressive Neural TensorNet (ANTN) is a novel architecture that combines tensor networks and autoregressive neural networks to improve quantum many-body physics simulation, outperforming existing methods.

Cross-Domain Policy Adaptation via Value-Guided Data Filtering

Kang Xu, Chenjia Bai, Xiaoteng Ma, Dong Wang, Bin Zhao, Zhen Wang, Xuelong Li, Wei Li

https://openreview.net/forum?id=qdM260dXsa

Keywords: Reinforcement Learning; Domain Adaptation; Online Dynamics Adaptation

Compressor summary: The paper proposes a new reinforcement learning method that adapts policies across domains with different dynamics by using value consistency to selectively share data, and shows its superior performance.

Parsel🐍: Algorithmic Reasoning with Language Models by Composing Decompositions

Eric Zelikman, Qian Huang, Gabriel Poesia, Noah Goodman, Nick Haber

https://openreview.net/forum?id=qd9qcbVAwQ

Keywords: reasoning, language models, code synthesis, decomposition

Compressor summary: Parsel is a framework that helps large language models generate complex algorithms with code by breaking down tasks into hierarchical natural language function descriptions and searching over combinations of possible function implementations using tests.

Multi-Head Adapter Routing for Cross-Task Generalization

Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni

https://openreview.net/forum?id=qcQhBli5Ho

Keywords: Parameter Efficient Finetuning, Multitask Learning, Transfer Learning, Natural Language Processing

Compressor summary: The paper explores how adapter routing affects PEFT and proposes $\texttt{MHR}$, a new routing method that improves performance by combining subsets of adapter parameters, and $\texttt{MHR}$-$\mu$, a variant that discards routing for single-adapter fine-tuning and zero-shot transfer.

One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learning

Marc Rigter, Bruno Lacerda, Nick Hawes

https://openreview.net/forum?id=qZjl2TKvUY

Keywords: offline reinforcement learning, model-based reinforcement learning, risk, uncertainty

Compressor summary: The paper proposes a model-based offline reinforcement learning algorithm that combines risk-aversion to aleatoric and epistemic uncertainty to address distributional shift and safety in safety-critical domains.

Multi-Objective Intrinsic Reward Learning for Conversational Recommender Systems

Zhendong Chu, Nan Wang, Hongning Wang

https://openreview.net/forum?id=qYAp31KwU2

Keywords: Conversational Recommendation, Reinforcement Learning, Meta Learning

Compressor summary: The paper proposes a new method to improve Conversational Recommender Systems by learning user preferences and designing task-specific rewards from user interactions, leading to better recommendations with fewer turns.

Are GATs Out of Balance?

Nimrah Mustafa, Aleksandar Bojchevski, Rebekka Burkholz

https://openreview.net/forum?id=qY7UqLoora

Keywords: graph attention networks, gradient flow, conservation law

Compressor summary: The paper explores the optimization and learning dynamics of Graph Attention Networks (GATs), derives a conservation law for their gradient flow dynamics, proposes an initialization scheme to improve trainability and speedup, and serves as a foundation for future studies on positive homogeneous models with attention.

CoPriv: Network/Protocol Co-Optimization for Communication-Efficient Private Inference

Wenxuan Zeng, Meng Li, Haichuan Yang, Wen-jie Lu, Runsheng Wang, Ru Huang

https://openreview.net/forum?id=qVeDwgYsho

Keywords: Private Inference, Network/Protocol Co-Optimization, Winograd Convolution, Structural Re-parameterization

Compressor summary: CoPriv is a framework that optimizes both the DNN architecture and the 2PC inference protocol to significantly reduce communication overhead while maintaining accuracy for secure privacy protection in DNN inference.

LambdaBeam: Neural Program Search with Higher-Order Functions and Lambdas

Kensen Shi, Hanjun Dai, Wen-Ding Li, Kevin Ellis, Charles Sutton

https://openreview.net/forum?id=qVMPXrX4FR

Keywords: Program Synthesis, Programming By Example, Lambdas, Functional Programming

Compressor summary: LambdaBeam is a search algorithm that uses neural networks to synthesize lambda functions for longer and more general programs, overcoming the limitations of prior approaches in handling iterative loops and higher-order functions.

Multi-resolution Spectral Coherence for Graph Generation with Score-based Diffusion

Hyuna Cho, Minjae Jeong, Sooyeon Jeon, Sungsoo Ahn, Won Hwa Kim

https://openreview.net/forum?id=qUlpDjYnsp

Keywords: graph wavelet transform, multi-scale wavelet filtering, graph generation, diffusion model

Compressor summary: The paper introduces a new method called Wavelet Graph Diffusion Model (Wave-GD) that generates realistic graphs with accurate node and edge frequency characteristics by modeling their joint distribution in the spectral space.

Alleviating the Semantic Gap for Generalized fMRI-to-Image Reconstruction

Tao Fang, Qian Zheng, Gang Pan

https://openreview.net/forum?id=qSS9izTOpo

Keywords: fMRI, image reconstruction, brain decoding

Compressor summary: The paper proposes a method (GESS) to improve fMRI-to-image reconstruction by using CLIP features to bridge the semantic gap between training and testing data, as well as leveraging structural information and uncertainty estimation.

The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks

Spencer Frei, Gal Vardi, Peter Bartlett, Nathan Srebro

https://openreview.net/forum?id=qSCziWQBPD

Keywords: adversarial robustness, neural networks, implicit bias, generalization

Compressor summary: The paper investigates how the biased gradient flow in two-layer ReLU networks affects generalization and adversarial robustness, showing that it leads to solutions that are good at fitting data but vulnerable to small perturbations.

Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability

Hanlin Zhu, Amy Zhang

https://openreview.net/forum?id=qS9aHF8bXz

Keywords: offline goal-conditioned RL, provably efficient algorithm, single-policy concentrability, general function approximation

Compressor summary: The paper provides a rigorous theoretical analysis of an existing offline goal-conditioned reinforcement learning algorithm, showing its efficiency, stability, and effectiveness in various real-world environments.

Complex Query Answering on Eventuality Knowledge Graph with Implicit Logical Constraints

Jiaxin Bai, Xin Liu, Weiqi Wang, Chen Luo, Yangqiu Song

https://openreview.net/forum?id=qQnO1HLQHe

Keywords: Knowledge Graph, Complex Query Answering, Eventuality Graph

Compressor summary: The paper proposes a new framework, CEQA, that uses implicit logical constraints and theorem provers to answer complex queries on eventuality-centric knowledge graphs, improving neural query encoders' performance.

Statistical Insights into HSIC in High Dimensions

Tao Zhang, Yaowu Zhang, Tingyou Zhou

https://openreview.net/forum?id=qPyvuFT0U9

Keywords: High dimensionality; Independence test; Kernel method; Nonlinear dependency.

Compressor summary: The paper studies how a popular measure of dependence, HSIC, behaves when the dimensions of the variables it measures grow at different rates and provides conditions for its performance in high-dimensional settings.

Monitor-Guided Decoding of Code LMs with Static Analysis of Repository Context

Lakshya Agrawal, Aditya Kanade, Navin Goyal, Shuvendu K Lahiri, Sriram Rajamani

https://openreview.net/forum?id=qPUbKxKvXq

Keywords: Language models, code generation, correctness, program analysis

Compressor summary: Monitor-guided decoding (MGD) helps language models of code achieve better compilation rates and agreement with ground truth by using static analysis to provide context, especially when dealing with global information.

Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals

Yue Wu, Yewen Fan, Paul Pu Liang, Amos Azaria, Yuanzhi Li, Tom Mitchell

https://openreview.net/forum?id=qP0Drg2HuH

Keywords: Games, Instruction Manual, Atari Games, Large Language Models, Language Models, Zero-shot, In-context prompting

Compressor summary: The Read and Reward framework uses instruction manuals to enhance the performance and efficiency of reinforcement learning agents on Atari games.

TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion

Preetha Vijayan, Prashant Shivaram Bhat, Bahram Zonooz, Elahe Arani

https://openreview.net/forum?id=qL3zPoWJda

Keywords: Continual Learning, Catastrophic Forgetting, Experience Replay, Lifelong Learning, Bio-Inspired, Active Forgetting, Scalable Neurogenesis

Compressor summary: The paragraph introduces TriRE, a novel continual learning paradigm that combines multiple neurophysiological processes to reduce catastrophic forgetting and promote knowledge transfer in deep neural networks.

VaRT: Variational Regression Trees

Sebastian Salazar

https://openreview.net/forum?id=qJRlz3SucN

Keywords: Probabilistic Machine Learning, Variational Inference, Bayesian Inference, Bayesian Nonparametrics

Compressor summary: The paper presents a new Bayesian model for decision trees that uses variational inference and shows its effectiveness in regression and causal inference problems, with a PyTorch implementation available.

Is Heterogeneity Notorious? Taming Heterogeneity to Handle Test-Time Shift in Federated Learning

Yue Tan, Chen Chen, Weiming Zhuang, Xin Dong, Lingjuan Lyu, Guodong Long

https://openreview.net/forum?id=qJJmu4qsLO

Keywords: Federated Learning, Test-Time Shift, Contrastive Learning

Compressor summary: FedICON is a framework that uses contrastive learning to capture and adapt to heterogeneous data in federated learning, improving performance under test-time shifts.

Sketching Algorithms for Sparse Dictionary Learning: PTAS and Turnstile Streaming

Gregory Dexter, Petros Drineas, David Woodruff, Taisuke Yasuda

https://openreview.net/forum?id=qHzEFxtheD

Keywords: dictionary learning, k means clustering, sketching, ptas, streaming

Compressor summary: The paper develops new techniques to extend sketching-based approaches to sparse dictionary learning and Euclidean k-means clustering problems, and obtains new upper and lower bounds for these problems in various settings.

Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models

Leonardo Galli, Holger Rauhut, Mark Schmidt

https://openreview.net/forum?id=qHrZszJSXj

Keywords: line search, nonmonotone, stochastic gradient descent, over-parametrized models, Polyak step size, optimization

Compressor summary: Nonmonotone line search methods can improve the speed and generalization of SGD/Adam by allowing larger step sizes without compromising convergence rates, as demonstrated by the PoNoS method and a new resetting technique.

Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective

Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, Liwei Wang

https://openreview.net/forum?id=qHrADgAdYu

Keywords: Chain-of-Thought Prompting, Large Language Models, Theory, Circuit Complexity, Dynamic Programming

Compressor summary: The paper investigates how Chain-of-Thought prompting enhances Large Language Models' performance in solving mathematical and decision-making problems, using circuit complexity theory and experiments.

Gradient Descent with Linearly Correlated Noise: Theory and Applications to Differential Privacy

Anastasia Koloskova, Ryan McKenna, Zachary Charles, J Keith Rush, Hugh Brendan McMahan

https://openreview.net/forum?id=qCglMj6A4z

Keywords: optimization, machine learning, differential privacy

Compressor summary: The paper analyzes how gradient descent works with linearly correlated noise, which arises in some privacy-preserving optimization methods, and proposes improved matrix factorizations for better performance.

VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models

Ziyi Yin, Muchao Ye, Tianrong Zhang, Tianyu Du, Jinguo Zhu, Han Liu, Jinghui Chen, Ting Wang, Fenglong Ma

https://openreview.net/forum?id=qBAED3u1XZ

Keywords: vision-language, adversarial attacks, pre-trained model, fine-tuned model

Compressor summary: This paper proposes VLATTACK, a method to craft image and text perturbations that can attack black-box fine-tuned Vision-Language models on different tasks by using block-wise similarity attacks for images and existing methods for texts.

Complex-valued Neurons Can Learn More but Slower than Real-valued Neurons via Gradient Descent

Jin-Hui Wu, Shao-Qun Zhang, Yuan Jiang, Zhi-Hua Zhou

https://openreview.net/forum?id=qA0uHmaVKk

Keywords: Complex-valued Neural Networks; Learning Neurons; Real-valued Neural Networks; Convergence Rate

Compressor summary: Complex-valued neural networks can efficiently learn functions of real-valued neurons and complex-valued neurons faster than real-valued neurons can learn each other or be learned by complex-valued neurons.

Trust Region-Based Safe Distributional Reinforcement Learning for Multiple Constraints

Dohyeong Kim, Kyungjae Lee, Songhwai Oh

https://openreview.net/forum?id=q9WMXjUxxT

Keywords: Reinforcement learning, Safety, Multiple Constraints, Distributional Critic

Compressor summary: The paper proposes a trust region-based safe reinforcement learning algorithm for robotic tasks with multiple constraints, and shows its effectiveness in experiments.

Deep Contract Design via Discontinuous Networks

Tonghan Wang, Paul Duetting, Dmitry Ivanov, Inbal Talgam-Cohen, David C. Parkes

https://openreview.net/forum?id=q8mH2d6uw2

Keywords: Automated contract design, discontinuous neural networks

Compressor summary: The paper proposes using deep learning and a new type of network (DeLU) to automate the design of optimal contracts, considering both agent's incentives and principal's utility.

Learning from Active Human Involvement through Proxy Value Propagation

Zhenghao Peng, Wenjie Mo, Chenda Duan, Quanyi Li, Bolei Zhou

https://openreview.net/forum?id=q8SukwaEBy

Keywords: Machine Learning, Human-in-the-loop Reinforcement Learning, Safety, Sample Efficiency, Reward-free

Compressor summary: The proposed method uses a proxy value function to express human intents and label state-action pairs for policy optimization, allowing the AI agent to learn from active human involvement in various control tasks.

Triple Eagle: Simple, Fast and Practical Budget-Feasible Mechanisms

Kai Han, You Wu, He Huang, Shuang Cui

https://openreview.net/forum?id=q6bVqOgGxP

Keywords: mechanism design, budget-feasible, truthful

Compressor summary: The paper introduces TripleEagle, a new algorithmic framework for designing Budget-Feasible Mechanisms with better approximation ratios, linear complexities, and strategyproofness for submodular valuation functions.

Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting

Marcel Kollovieh, Abdul Fatir Ansari, Michael Bohlke-Schneider, Jasper Zschiegner, Hao Wang, Bernie Wang

https://openreview.net/forum?id=q6X038vKgU

Keywords: diffusion models, time series forecasting, generative modeling, deep learning

Compressor summary: The authors propose a task-agnostic diffusion model for time series applications and show its effectiveness in forecasting, refinement, and synthetic data generation.

Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Models

Simian Luo, Chuanhao Yan, Chenxu Hu, Hang Zhao

https://openreview.net/forum?id=q5FAZAIooz

Keywords: Video-to-Audio Generation; Contrastive Audio-Visual Pretraining; Latent Diffusion Model;

Compressor summary: Diff-Foley is a new method for generating high-quality audio from silent videos with improved synchronization and relevance, using a latent diffusion model and contrastive pretraining.

Discriminative Feature Attributions: Bridging Post Hoc Explainability and Inherent Interpretability

Usha Bhalla, Suraj Srinivas, Himabindu Lakkaraju

https://openreview.net/forum?id=q4HlFS7B7Y

Keywords: Machine Learning Explainability, Machine Learning Interpretability

Compressor summary: The paragraph discusses a method called Distractor Erasure Tuning (DiET), which improves the faithfulness of post hoc explanations for machine learning models by making them robust to distractor erasure in inputs.

Efficient Data Subset Selection to Generalize Training Across Models: Transductive and Inductive Networks

Eeshaan Jain, Tushar Nandy, Gaurav Aggarwal, Ashish V. Tendulkar, Rishabh K Iyer, Abir De

https://openreview.net/forum?id=q3fCWoC9l0

Keywords: Data Subset Selection, Efficient Learning

Compressor summary: The paragraph describes $\texttt{SubSelNet}$, a non-adaptive framework for subset selection in efficient learning that uses an attention-based neural gadget and can be either transductive or inductive, achieving better results than other methods on various real datasets.

Brain Dissection: fMRI-trained Networks Reveal Spatial Selectivity in the Processing of Natural Images

Gabriel Herbert Sarch, Michael J. Tarr, Katerina Fragkiadaki, Leila Wehbe

https://openreview.net/forum?id=q3fA5tTod3

Keywords: Computational Neuroscience, Deep Neural Networks, Visual Neuroscience, Visual Streams, Scene Perception, Brain Imaging

Compressor summary: The paper trains neural networks to predict brain responses to images and uses "network dissection" to explore how different regions of the visual cortex interpret various features of natural scenes.

Learning to Search Feasible and Infeasible Regions of Routing Problems with Flexible Neural k-Opt

Yining Ma, Zhiguang Cao, Yeow Meng Chee

https://openreview.net/forum?id=q1JukwH2yP

Keywords: learning to optimize, vehicle routing problem, combinatorial optimization

Compressor summary: The paper introduces NeuOpt, a learning-to-search solver for routing problems that learns flexible k-opt exchanges and uses GIRE, D2A, and reward shaping to explore both feasible and infeasible regions effectively, achieving superior performance on TSP and CVRP compared to existing methods.

Learning Linear Causal Representations from Interventions under General Nonlinear Mixing

Simon Buchholz, Goutham Rajendran, Elan Rosenfeld, Bryon Aragam, Bernhard Schölkopf, Pradeep Kumar Ravikumar

https://openreview.net/forum?id=q131tA7HCT

Keywords: Causal Representation Learning, Interventional data, Gaussian Structural Causal models

Compressor summary: The authors study how to learn causal representations from unknown interventions using Gaussian latent distributions and a general mixing function, and they prove identifiability results without access to intervention targets.

SPQR: Controlling Q-ensemble Independence with Spiked Random Model for Reinforcement Learning

Dohyeok Lee, Seungyub Han, Taehyun Cho, Jungwoo Lee

https://openreview.net/forum?id=q0sdoFIfNg

Keywords: Deep Reinforcement Learning, Ensemble Q-learning

Compressor summary: The paper proposes a new method, spiked Wishart Q-ensemble independence regularization (SPQR), to improve deep reinforcement learning by ensuring diversity of multiple Q-functions using random matrix theory.

On the Consistency of Maximum Likelihood Estimation of Probabilistic Principal Component Analysis

Arghya Datta, Sayak Chakrabarty

https://openreview.net/forum?id=q0RfX96un8

Keywords: maximum likelihood estimate, non-identifiability, Redner approach, quotient topological spaces, consistency

Compressor summary: The text describes PPCA, a widely used statistical tool with applications in various fields, and proposes a novel approach to resolve the identifiability issue of its maximum likelihood solution using quotient topological spaces.

StateMask: Explaining Deep Reinforcement Learning through State Mask

Zelei Cheng, Xian Wu, Jiahao Yu, Wenhai Sun, Wenbo Guo, Xinyu Xing

https://openreview.net/forum?id=pzc6LnUxYN

Keywords: deep reinforcement learning, interpretation, explanation

Compressor summary: StateMask is a novel method to identify critical states for an agent's final reward by learning a mask net that temporarily blinds the agent without affecting its performance.

Unified Enhancement of Privacy Bounds for Mixture Mechanisms via $f$-Differential Privacy

Chendi Wang, Buxin Su, Jiayuan Ye, Reza Shokri, Weijie J Su

https://openreview.net/forum?id=pw5hEuEroL

Keywords: Differential privacy, $f$-DP, mixture mechanisms, shuffling, differentially private gradient descent

Compressor summary: The paper proposes an improved analysis of differential privacy for shuffling models and one-iteration DP-GD using $f$-DP, with applications to random initialization and mixture mechanisms.

Flow-Attention-based Spatio-Temporal Aggregation Network for 3D Mask Detection

Yuxin Cao, Yian Li, Yumeng Zhu, Derui Wang, Minhui Xue

https://openreview.net/forum?id=pvSKVt3EsM

Keywords: 3D mask detection, spatio-temporal aggregation, optical flow, deep learning

Compressor summary: FASTEN is a novel 3D mask detection framework that uses flow attention and spatio-temporal aggregation to quickly and accurately detect spoofing attacks in face recognition systems with low computational overhead and only five frames of input.

Most Neural Networks Are Almost Learnable

Amit Daniely, Nathan Srebro, Gal Vardi

https://openreview.net/forum?id=pvPujuvjQd

Keywords: learning neural networks, computational complexity, random networks

Compressor summary: The paper proposes a polynomial-time approximation scheme (PTAS) for learning random constant-depth neural networks with an additive error of epsilon under certain conditions on the network size and activation functions.

GraphPatcher: Mitigating Degree Bias for Graph Neural Networks via Test-time Augmentation

Mingxuan Ju, Tong Zhao, Wenhao Yu, Neil Shah, Yanfang Ye

https://openreview.net/forum?id=puupdGOWUp

Keywords: Graph neural network, Test-time Augmentation

Compressor summary: The paragraph discusses a test-time augmentation framework, GraphPatcher, that improves the performance of graph neural networks on low-degree nodes without compromising their ability to handle high-degree nodes.

Self-Predictive Universal AI

Elliot Catt, Jordi Grau-Moya, Marcus Hutter, Matthew Aitchison, Tim Genewein, Gregoire Deletang, Li Kevin Wenliang, Joel Veness

https://openreview.net/forum?id=psXVkKO9No

Keywords: General Reinforcement Learning, Reinforcement Learning, Self-Modeling, Bayes-optimality, Policy Distillation, Uncertainty, Universal AI

Compressor summary: Self-AIXI is a universal agent that uses learning to obtain good policies by self-predicting its own action data, converging to AIXI and having maximal intelligence and self-optimization properties.

Label Poisoning is All You Need

Rishi Dev Jha, Jonathan Hayase, Sewoong Oh

https://openreview.net/forum?id=prftZp6mDH

Keywords: security, backdoor attack

Compressor summary: The paper introduces FLIP, a novel label-only backdoor attack method that can successfully manipulate image predictions with corrupted training labels, while maintaining high clean test accuracy.

Distributional Pareto-Optimal Multi-Objective Reinforcement Learning

Xin-Qiang Cai, Pushi Zhang, Li Zhao, Jiang Bian, Masashi Sugiyama, Ashley Juan Llorens

https://openreview.net/forum?id=prIwYTU9PV

Keywords: Multi-Objective Reinforcement Learning

Compressor summary: DPMORL is a novel approach that learns policies balancing multiple objectives while considering return uncertainty and satisfying diverse distributional preferences.

ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation

Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, Jun Zhu

https://openreview.net/forum?id=ppJuFSOAnM

Keywords: diffusion models, text to 3D

Compressor summary: VSD is a particle-based variational framework that improves text-to-3D generation by modeling the 3D parameter as a random variable and addressing over-saturation, over-smoothing, and low-diversity issues.

Wasserstein Quantum Monte Carlo: A Novel Approach for Solving the Quantum Many-Body Schrödinger Equation

Kirill Neklyudov, Jannes Nys, Luca Thiede, Juan Felipe Carrasquilla Alvarez, qiang liu, Max Welling, Alireza Makhzani

https://openreview.net/forum?id=pjSzKhSrfs

Keywords: Quantum Monte Carlo, Schrödinger equation, Wasserstein Fisher-Rao gradient flow

Compressor summary: The paper introduces Wasserstein Quantum Monte Carlo, a new method for solving quantum many-body problems that uses transportation-based gradients and converges faster than traditional methods.

Kernelized Reinforcement Learning with Order Optimal Regret Bounds

Sattar Vakili, Julia Olkhovskaya

https://openreview.net/forum?id=pirH9ycaNg

Keywords: Reinforcement Learning, Kernel ridge regression, Gaussian processes, LSVI

Compressor summary: The paper proposes $\pi$-KRVI, an optimistic modification of least-squares value iteration for reinforcement learning with nonlinear function approximation using kernel ridge regression, and proves its sublinear regret bound under a general setting.

Learning Probabilistic Symmetrization for Architecture Agnostic Equivariance

Jinwoo Kim, Dat Tien Nguyen, Ayhan Suleymanzade, Hyeokjun An, Seunghoon Hong

https://openreview.net/forum?id=phnN1eu5AX

Keywords: equivariant machine learning, transformers, graphs, general-purpose architectures

Compressor summary: The authors propose a framework that uses a non-equivariant backbone and an equivariant network to learn functions with group symmetries, achieving competitive results against tailored equivariant architectures.

FedFed: Feature Distillation against Data Heterogeneity in Federated Learning

Zhiqin Yang, Yonggang Zhang, Yu Zheng, Xinmei Tian, Hao Peng, Tongliang Liu, Bo Han

https://openreview.net/forum?id=phnGilhPH8

Keywords: Federated Learning, Data Heterogeneity

Compressor summary: The paper proposes a method called FedFed that shares some features of data to improve federated learning while preserving privacy.

Recurrent Hypernetworks are Surprisingly Strong in Meta-RL

Jacob Beck, Risto Vuorio, Zheng Xiong, Shimon Whiteson

https://openreview.net/forum?id=pefAAzu8an

Keywords: meta-RL, RL, reinforcement learning, memory, rnn, recurrent, hypernetwork, few-shot

Compressor summary: The paper investigates how using hypernetworks can improve the performance of simple recurrent networks in meta-RL tasks.

Reducing Blackwell and Average Optimality to Discounted MDPs via the Blackwell Discount Factor

Julien Grand-Clément, Marek Petrik

https://openreview.net/forum?id=pcuC65JWAa

Keywords: Markov Decision Process, Blackwell optimality, average optimality, robust optimization

Compressor summary: The paper introduces a discount factor for Markov Decision Processes called the Blackwell discount factor and shows how it can be used to find optimal policies without strict assumptions on the MDP structure.

Initialization-Dependent Sample Complexity of Linear Predictors and Neural Networks

Roey Magen, Ohad Shamir

https://openreview.net/forum?id=pcpjtYNJCH

Keywords: sample complexity; learning theory; neural networks; linear predictors

Compressor summary: The text discusses new findings on how many examples are needed to learn vector-valued linear predictors and neural networks, showing different sample complexity behavior compared to scalar-valued linear predictors.

Binarized Spectral Compressive Imaging

Yuanhao Cai, Yuxin Zheng, Jing Lin, Xin Yuan, Yulun Zhang, Haoqian Wang

https://openreview.net/forum?id=pcKwgdVAlq

Keywords: Applications, Computer Vision, Low-level Vision, Image Restoration, Snapshot Compressive Imaging, Hyperspectral Image Reconstruction

Compressor summary: The paper proposes a novel method, BiSRNet, for efficient HSI restoration from compressed measurements in SCI systems using binarized convolutional modules that outperform existing binarization algorithms.

Learning Generalizable Agents via Saliency-guided Features Decorrelation

Sili Huang, Yanchao Sun, Jifeng Hu, Siyuan Guo, Hechang Chen, Yi Chang, Lichao Sun, Bo Yang

https://openreview.net/forum?id=pb1OwZNgr2

Keywords: reinforcement learning, generalization

Compressor summary: SGFD is a method to improve generalization in visual RL by decorrelating features using saliency maps and random Fourier functions.

Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator

Hanzhuo Huang, Yufan Feng, Cheng Shi, Lan Xu, Jingyi Yu, Sibei Yang

https://openreview.net/forum?id=paa2OU5jN8

Keywords: Text-to-Video, Zero-Shot Generation, Large Language Model, Latent Diffusion Models

Compressor summary: Text-to-video generation using large language models and latent diffusion models to create vivid and high-quality videos without video data or training.

Kernel-Based Tests for Likelihood-Free Hypothesis Testing

Patrik Robert Gerber, Tianze Jiang, Yury Polyanskiy, Rui Sun

https://openreview.net/forum?id=paTESG8iSE

Keywords: Kernel methods, Maximum mean discrepancy, Likelihood-free inference, Hypothesis testing, Minimax statistics

Compressor summary: The paper studies the trade-off between labeling more inputs and using less training data in two applications: detecting the Higgs boson and identifying planted images in CIFAR-10 dataset.

Enhancing Robot Program Synthesis Through Environmental Context

Tianyi Chen, Qidi Wang, Zhen Dong, Liwei Shen, Xin Peng

https://openreview.net/forum?id=pZ2Ww45GkL

Keywords: program synthesis, partial envrionment, robotic programming, domain-specific language

Compressor summary: The paper presents a framework for synthesizing programs for robots in partially observed environments by learning an embedding space and using graph structure to rectify errors.

The Best of Both Worlds in Network Population Games: Reaching Consensus and Convergence to Equilibrium

Shuyue Hu, Harold Soh, Georgios Piliouras

https://openreview.net/forum?id=pXtVyj4R33

Keywords: Multi-Agent Learning, Consensus Formation, Smooth Fictitious Play, Network Game, Population Game

Compressor summary: The paper explores how consensus and equilibrium are related in multi-agent systems with multiple interacting sub-populations and shows that smooth fictitious play can achieve both goals.

Principled Weight Initialisation for Input-Convex Neural Networks

Pieter-Jan Hoedt, Günter Klambauer

https://openreview.net/forum?id=pWZ97hUQtQ

Keywords: initialization, signal propagation, input-convex networks

Compressor summary: ICNNs are networks with guaranteed convexity that require a new weight initialization method for better learning and generalization, which can also be trained without skip-connections and applied to drug discovery tasks.

RETVec: Resilient and Efficient Text Vectorizer

Elie Bursztein, Marina Zhang, Owen Skipper Vallis, Xinyu Jia, Alexey Kurakin

https://openreview.net/forum?id=pVlC0reMKq

Keywords: language modeling, text embedding, adversarial text attack, text vectorization

Compressor summary: RETVec is a robust, multilingual text vectorizer with a character encoding and an optional embedding model that can handle typos and adversarial attacks.

CorresNeRF: Image Correspondence Priors for Neural Radiance Fields

Yixing Lao, Xiaogang Xu, zhipeng cai, Xihui Liu, Hengshuang Zhao

https://openreview.net/forum?id=pTCZWSDltG

Keywords: Neural Radiance Fields, 3D Reconstruction, Few View

Compressor summary: CorresNeRF uses image correspondence priors to supervise NeRF training, improving performance under sparse-view settings for novel view synthesis and surface reconstruction tasks.

Parameter-efficient Tuning of Large-scale Multimodal Foundation Model

Haixin Wang, Xinlong Yang, Jianlong Chang, Dian Jin, Jinan Sun, Shikun Zhang, Xiao Luo, Qi Tian

https://openreview.net/forum?id=pT8DIhsJCw

Keywords: parameter-efficient transfer learning; multi-modal learning; prompt learning

Compressor summary: The paper introduces AURORA, a framework for multimodal transfer learning that uses mode approximation and Informative Context Enhancement to achieve high performance with few parameters.

Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models

Yichao Cao, Qingfei Tang, Xiu Su, Song Chen, Shan You, Xiaobo Lu, Chang Xu

https://openreview.net/forum?id=pQvAL40Cdj

Keywords: Human-object interaction, Commonsense Knowledge, Foundation Models

Compressor summary: UniHOI is a method for recognizing complex human-object interactions in open world settings using vision-language foundation models and large language models to guide decoding, interpret interactions, and handle various input types.

Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detection

Linyan Huang, Zhiqi Li, Chonghao Sima, Wenhai Wang, Jingdong Wang, Yu Qiao, Hongyang Li

https://openreview.net/forum?id=pQF9kbM8Ea

Keywords: camera-only detection, multi-modal distillation, multi-view object detection

Compressor summary: The paper proposes VCD, a framework that improves camera-only 3D object detection by using a multi-modal expert with LiDAR input and trajectory-based distillation supervision, achieving state-of-the-art results on nuScenes.

ESSEN: Improving Evolution State Estimation for Temporal Networks using Von Neumann Entropy

Qiyao Huang, Yingyue Zhang, Zhihong Zhang, Edwin Hancock

https://openreview.net/forum?id=pO7d6iFdnc

Keywords: Temporal Network, Graph Neural Network, Von Neumann Entropy

Compressor summary: ESSEN is a novel framework that uses von Neumann entropy and thermodynamic temperature to measure temporal network evolution and improve link prediction performance.

Statistical Knowledge Assessment for Large Language Models

Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Zhifang Sui, Lei Li

https://openreview.net/forum?id=pNtG6NAmx0

Keywords: Large Language Models, Knowledge Assessment, Evaluation

Compressor summary: The paper introduces KaRR, a statistical approach that quantifies how well large language models (LLMs) can generate factually correct answers for given facts, based on the ratio of LLM text to random chances. The paper tests 20 LLMs and finds strong agreement with human judgments.

PRODIGY: Enabling In-context Learning Over Graphs

Qian Huang, Hongyu Ren, Peng Chen, Gregor Kržmanc, Daniel Zeng, Percy Liang, Jure Leskovec

https://openreview.net/forum?id=pLwYhNNnoR

Keywords: Graph Neural Network, in-context learning, pretraining

Compressor summary: PRODIGY is a novel framework that enables in-context learning over graphs using a prompt graph representation and a graph neural network architecture, achieving strong performance on citation networks and knowledge graphs tasks.

Kernelized Cumulants: Beyond Kernel Mean Embeddings

Patric Bonnier, Harald Oberhauser, Zoltán Szabó

https://openreview.net/forum?id=pLsPFxqn7J

Keywords: kernel, cumulant, mean embedding, Hilbert-Schmidt independence criterion, maximum mean discrepancy

Compressor summary: The paper introduces kernelized cumulants for reproducing kernel Hilbert spaces, which are computationally efficient and provide new all-purpose statistics for data analysis.

An active learning framework for multi-group mean estimation

Abdellah Aznag, Rachel Cummings, Adam N. Elmachtoub

https://openreview.net/forum?id=pLcSrn8NpJ

Keywords: Active learning, mean estimation, bandit feedback, data acquisition

Compressor summary: The paper proposes an active learning algorithm called Variance-UCB to learn the means of multiple unknown groups by minimizing the $p$-norm of variances, with optimal regret bounds for finite and infinite $p$.

Non-Smooth Weakly-Convex Finite-sum Coupled Compositional Optimization

Quanqi Hu, Dixian Zhu, Tianbao Yang

https://openreview.net/forum?id=pLOWV1UGF6

Keywords: non-smooth optimization, weakly-convex optimization, compositional optimization, AUC maximization

Compressor summary: The paper studies non-smooth weakly-convex finite-sum coupled compositional optimization problems and proposes algorithms for solving them in machine learning applications.

PID-Inspired Inductive Biases for Deep Reinforcement Learning in Partially Observable Control Tasks

Ian Char, Jeff Schneider

https://openreview.net/forum?id=pKnhUWqZTJ

Keywords: Reinforcement Learning, Control, POMDP

Compressor summary: The paragraph discusses using PID features for history encoding in deep reinforcement learning to improve robustness and performance on various control tasks.

Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Linear Subspaces

Odelia Melamed, Gilad Yehudai, Gal Vardi

https://openreview.net/forum?id=pJbEXBBN88

Keywords: Adversarial Examples, Robustness, Neural Networks, Classification

Compressor summary: This paper studies how gradient methods affect the robustness of two-layer neural networks trained on low dimensional linear subspaces and suggests ways to improve their resistance to adversarial examples in certain directions.

Optimal Exploration for Model-Based RL in Nonlinear Systems

Andrew Wagenmaker, Guanya Shi, Kevin Jamieson

https://openreview.net/forum?id=pJQu0zpKCS

Keywords: reinforcement learning, control theory, system identification, experiment design, active learning

Compressor summary: The paper proposes an algorithm for exploring nonlinear dynamical systems to learn relevant parameters for controller optimization more efficiently.

What Makes Good Examples for Visual In-Context Learning?

Yuanhan Zhang, Kaiyang Zhou, Ziwei Liu

https://openreview.net/forum?id=pIXTMrBe7f

Keywords: computer vision, visual in-context learning, prompt learning

Compressor summary: This paper proposes prompt retrieval frameworks for large vision models to improve their performance in in-context learning by selecting appropriate input-output pairs without updating any internal model parameters.

Causal Effect Identification in Uncertain Causal Networks

Sina Akbari, Fateme Jamshidi, Ehsan Mokhtarian, Matthew James Vowels, Jalal Etesami, Negar Kiyavash

https://openreview.net/forum?id=pH4Fv7C3yC

Keywords: Causal effect, identifiability, causal DAGs, probabilistic graphs

Compressor summary: The paper studies how to find the most plausible probabilistic causal graph that can identify a specific causal effect, using algorithms that solve an NP-hard optimization problem called edge ID.

FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning

Zhuo Huang, Li Shen, Jun Yu, Bo Han, Tongliang Liu

https://openreview.net/forum?id=pE3yaP0Eqg

Keywords: Semi-Supervised Learning

Compressor summary: FlatMatch is a new SSL method that balances learning on labeled and unlabeled data by penalizing the prediction difference between worst-case and original models, improving generalization performance.

Latent Space Translation via Semantic Alignment

Valentino Maiorca, Luca Moschella, Antonio Norelli, Marco Fumero, Francesco Locatello, Emanuele Rodolà

https://openreview.net/forum?id=pBa70rGHlr

Keywords: latent space translation, relative representation, Procrustes analysis, zero-shot, stitching, latent communication, representation learning, manifold alignment, multimodal

Compressor summary: The paper proposes a method to translate and stitch representations learned from different neural networks, enabling better understanding of their intrinsic similarity and improving performance in various tasks and domains.

Demystifying the Optimal Performance of Multi-Class Classification

Minoh Jeong, Martina Cardone, Alex Dytso

https://openreview.net/forum?id=p9k5MS0JAL

Keywords: Bayes error, estimation, classification, minimum error probability

Compressor summary: The authors propose a new method to estimate the Bayes error rate for multi-class classification problems and analyze its properties and performance on synthetic and real data.

From Trainable Negative Depth to Edge Heterophily in Graphs

Yuchen Yan, Yuzhong Chen, Huiyuan Chen, Minghua Xu, Mahashweta Das, Hao Yang, Hanghang Tong

https://openreview.net/forum?id=p8lowHbuv8

Keywords: graph convolutional network

Compressor summary: The paper proposes a new GCN model that can adjust its depth continuously to handle both homophilic and heterophilic graph topologies, and shows improved performance on node classification tasks.

Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parser

Yung-Hsuan Lai, Yen-Chun Chen, Yu-Chiang Frank Wang

https://openreview.net/forum?id=p8gTWkFIvx

Keywords: Audio-Visual Video Parsing, Audio-Visual Learning

Compressor summary: The paper introduces VALOR, a method that uses pre-trained models as teachers to help learn audio and visual labels from weak labels in an unaligned setting, improving performance on the Look, Listen, and Parse dataset and Audio-Visual Event Localization tasks.

DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction

Mohammadreza Pourreza, Davood Rafiei

https://openreview.net/forum?id=p53QDxSIc5

Keywords: In-Context Learning, Text-to-SQL, Task Decomposition, Spider Challenge, Natural Language Interfaces to Databases

Compressor summary: The paper proposes decomposing the text-to-SQL task into smaller sub-tasks and using LLMs to solve them, leading to improved performance and setting new benchmarks.

Riemannian stochastic optimization methods avoid strict saddle points

Ya-Ping Hsieh, Mohammad Reza Karimi Jaghargh, Andreas Krause, Panayotis Mertikopoulos

https://openreview.net/forum?id=p4SjKPchJy

Keywords: Riemannian optimization, saddle points, stochastic approximation

Compressor summary: The paper investigates if retraction-based stochastic Riemannian optimization algorithms can avoid saddle points with certain assumptions, ensuring the convergence to a local minimizer.

How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

Michael Hanna, Ollie Liu, Alexandre Variengien

https://openreview.net/forum?id=p4PckNQR8k

Keywords: interpretability, language models, NLP

Compressor summary: The paper examines how pre-trained language models like GPT-2 small use basic math skills and identifies the specific components and mechanisms involved in solving a simple mathematical task.

Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision

Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Daniel Cox, Yiming Yang, Chuang Gan

https://openreview.net/forum?id=p40XRfBX96

Keywords: AI Alignment, Large Language Models, In Context Learning, Neural Symbolics

Compressor summary: The proposed SELF-ALIGN approach enables AI assistant agents to align themselves with human intentions using minimal human supervision by combining LLMs and principle-driven reasoning.

FlowPG: Action-constrained Policy Gradient with Normalizing Flows

Janaka Chathuranga Brahmanage, Jiajing Ling, Akshat Kumar

https://openreview.net/forum?id=p1gzxzJ4Y5

Keywords: action-constrained reinforcement learning, decision making

Compressor summary: ACRL uses a normalizing flow model to learn an invertible mapping between feasible actions and latent variables, enabling faster and more accurate action sampling and constraint satisfaction in DDPG-based RL problems.

Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training

Yefan Zhou, Tianyu Pang, Keqin Liu, charles h martin, Michael W. Mahoney, Yaoqing Yang

https://openreview.net/forum?id=oyV9FslE3j

Keywords: Heavy-tail self-regularization, learning rate schedule

Compressor summary: The paper introduces TempBalance, a layer-wise learning rate method for neural network training, which uses Heavy-Tailed Self-Regularization Theory to balance the temperature across layers and improve performance on various datasets.

Language-based Action Concept Spaces Improve Video Self-Supervised Learning

Kanchana Ranasinghe, Michael S Ryoo

https://openreview.net/forum?id=oyFyOPZUCs

Keywords: self-supervised learning for videos, zero-shot action recognition

Compressor summary: The paper proposes a method to adapt image CLIP models to the video domain using language tied self-supervised learning, which improves action recognition performance on benchmarks.

MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition

Nicolas Menet, Michael Hersche, Geethan Karunaratne, Luca Benini, Abu Sebastian, Abbas Rahimi

https://openreview.net/forum?id=ox7aynitoW

Keywords: Computation in superposition, Vector-symbolic architectures, Convolutional neural networks, Transformers

Compressor summary: The authors propose MIMONets, which can process multiple inputs at once using variable binding and unbinding. This enables computation in superposition, improving inference efficiency and performance-accuracy trade-offs for CNN and Transformer models. They also provide theoretical bounds on the interference between superposition channels.

On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions

Yufeng Zhang, Jialu Pan, Kenli Li, Wanwei Liu, Zhenbang Chen, Xinwang Liu, J Wang

https://openreview.net/forum?id=ouLe91yibj

Keywords: Kullback-Leibler divergence, statistical divergence, multivariate Gaussian distribution, mathematical optimization, Lambert $W$ function, machine learning, flow-based model, reinforcement learning

Compressor summary: The paper studies properties of KL divergence between multivariate Gaussian distributions, deriving bounds for different scenarios and showing that it follows a relaxed triangle inequality.

Linear Time Algorithms for k-means with Multi-Swap Local Search

Junyu Huang, Qilong Feng, Ziyun Huang, Jinhui Xu, Jianxin Wang

https://openreview.net/forum?id=oss2jXD1Zs

Keywords: Approximation Algorithms, k-means Clustering, Local Search

Compressor summary: The paper proposes a multi-swap local search algorithm for the $k$-means problem with linear running time and improved approximation ratio, and also introduces a sampling-based method and a recombination mechanism to enhance the performance.

Model and Feature Diversity for Bayesian Neural Networks in Mutual Learning

Cuong Pham, Cuong C. Nguyen, Trung Le, Dinh Phung, Gustavo Carneiro, Thanh-Toan Do

https://openreview.net/forum?id=os2BdbiGwX

Keywords: Bayesian Neural Networks, Deep Mutual Learning

Compressor summary: The paper proposes a deep mutual learning approach that increases diversity in parameter and feature distributions of Bayesian Neural Networks, leading to significant performance improvements in classification accuracy, negative log-likelihood, and expected calibration error.

Bypassing the Simulator: Near-Optimal Adversarial Linear Contextual Bandits

Haolin Liu, Chen-Yu Wei, Julian Zimmert

https://openreview.net/forum?id=orh4e0AO9R

Keywords: adversarial linear contextual bandits, log-determinant barrier

Compressor summary: The paper proposes a new method for solving the adversarial linear contextual bandit problem that achieves lower regret and maintains computational efficiency in the presence of stochastic arm availability and additive misspecification errors.

Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds

Ziqiao Wang, Yongyi Mao

https://openreview.net/forum?id=oqDSDKLd3S

Keywords: generalization, information-theoretic bounds, stability

Compressor summary: The authors propose a new way to measure generalization in machine learning using a matrix and a stability concept that improve existing bounds, especially for stochastic convex optimization problems.

Explore In-Context Learning for 3D Point Cloud Understanding

Zhongbin Fang, Xiangtai Li, Xia Li, Joachim M. Buhmann, Chen Change Loy, Mengyuan Liu

https://openreview.net/forum?id=ooXpTZYwXa

Keywords: In-context learning, Point cloud, Prompt tuning

Compressor summary: The paper introduces Point-In-Context, a novel framework for in-context learning in 3D point clouds, which addresses technical challenges and outperforms individual models.

Multi-Fidelity Multi-Armed Bandits Revisited

Xuchuang Wang, Qingyun Wu, Wei Chen, John C.S. Lui

https://openreview.net/forum?id=oi45JlpSOT

Keywords: Multi-fidelity, multi-armed bandits

Compressor summary: The paper investigates the multi-fidelity multi-armed bandit problem, where each arm has different costs and observation accuracies, and proposes algorithms for best arm identification and regret minimization.

Successor-Predecessor Intrinsic Exploration

Changmin Yu, Neil Burgess, Maneesh Sahani, Samuel Gershman

https://openreview.net/forum?id=ohKbQp0jIY

Keywords: Exploration, reinforcement learning

Compressor summary: The paper introduces SPIE, an exploration algorithm for reinforcement learning that combines prospective and retrospective information to generate more efficient and ethologically plausible behavior in environments with sparse rewards and bottleneck states.

Disentangling Cognitive Diagnosis with Limited Exercise Labels

Xiangzhi Chen, Le Wu, Fei Liu, Lei Chen, Kun Zhang, Richang Hong, Meng Wang

https://openreview.net/forum?id=ogPBujRhiN

Keywords: Intelligent Education System, Cognitive Diagnosis, Disentangled Representation Learning, Interpretability

Compressor summary: The paper proposes a new method called Disentanglement based Cognitive Diagnosis (DCD) for measuring students' proficiency in specific knowledge concepts with limited exercise labels, using response records and two novel modules.

Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations

Thomas Edward Yerxa, Yilun Kuang, Eero P Simoncelli, SueYeon Chung

https://openreview.net/forum?id=og9V7NgOrQ

Keywords: computational neuroscience, theoretical neuroscience, efficient coding, representation geometry, neural manifolds, self-supervised learning, statistical physics of learning

Compressor summary: The efficient coding hypothesis suggests sensory systems adapt to their inputs for maximal information capture, but measuring this is difficult; researchers have developed Maximum Manifold Capacity Representations (MMCR) which use a novel efficiency metric and are competitive with self-supervised learning methods.

Online (Multinomial) Logistic Bandit: Improved Regret and Constant Computation Cost

Yu-Jie Zhang, Masashi Sugiyama

https://openreview.net/forum?id=ofa1U5BJVJ

Keywords: Logistic Bandit, Generalized Linear Bandit, Regret Bound, Computation Cost

Compressor summary: The paper proposes a logistic bandit algorithm that is statistically efficient, computationally fast, and adaptable to complex decision-making problems, improving on existing binary and multinomial methods.

Demystifying Structural Disparity in Graph Neural Networks: Can One Size Fit All?

Haitao Mao, Zhikai Chen, Wei Jin, Haoyu Han, Yao Ma, Tong Zhao, Neil Shah, Jiliang Tang

https://openreview.net/forum?id=oef30oScVB

Keywords: Graph Neural Network

Compressor summary: The study shows that Graph Neural Networks perform well on nodes with similar structures in homophilic graphs and heterophilic nodes in heterophilic graphs, but struggle with nodes having different structures in the opposite graph type.

FedGCN: Convergence-Communication Tradeoffs in Federated Training of Graph Convolutional Networks

Yuhang Yao, Weizhao Jin, Srivatsan Ravi, Carlee Joe-Wong

https://openreview.net/forum?id=ody3RBUuJS

Keywords: Federated Graph Learning;

Compressor summary: FedGCN is a novel algorithm that trains GCN models on graphs distributed among clients using federated learning, achieving fast convergence, high accuracy, and low communication.

SQ Lower Bounds for Learning Mixtures of Linear Classifiers

Ilias Diakonikolas, Daniel Kane, Yuxin Sun

https://openreview.net/forum?id=obCNIzeSrg

Keywords: mixtures models, linear classifier, Statistical Query model, spherical designs

Compressor summary: The paper studies how to learn mixtures of linear classifiers with Gaussian features and shows that current algorithms are nearly optimal using a new construction of spherical designs.

FGPrompt: Fine-grained Goal Prompting for Image-goal Navigation

Xinyu Sun, Peihao Chen, Jugang Fan, Jian Chen, Thomas H. Li, Mingkui Tan

https://openreview.net/forum?id=oaJEB5Qcia

Keywords: Visual Navigation, Image-Goal Navigation, Embodied AI

Compressor summary: The paper proposes a Fine-grained Goal Prompting method to improve image-goal navigation for autonomous systems by using high-resolution feature maps in the goal image as prompts.

TFLEX: Temporal Feature-Logic Embedding Framework for Complex Reasoning over Temporal Knowledge Graph

Xueyuan Lin, Haihong E, Chengjin Xu, Gengxian Zhou, Haoran Luo, Tianyi Hu, Fenglong Su, Ningyuan Li, Mingzhi Sun

https://openreview.net/forum?id=oaGdsgB18L

Keywords: Temporal Knowledge Graph Reasoning, Temporal Knowledge Graph Embedding, Temporal Knowledge Graph, Temporal Logic, Knowledge Graph Reasoning, Knowledge Graph Embedding, Knowledge Graph, Machine Learning

Compressor summary: TFLEX is a novel framework for multi-hop logical reasoning over temporal knowledge graphs that uses fuzzy logic to handle complex queries involving entities, timestamps, and set operations.

Follow-ups Also Matter: Improving Contextual Bandits via Post-serving Contexts

Chaoqi Wang, Ziyu Ye, Zhe Feng, Ashwinkumar Badanidiyuru, Haifeng Xu

https://openreview.net/forum?id=oaCDiKoJ2w

Keywords: linear stochastic bandits, online learning, partial information, contextual bandits

Compressor summary: The paragraph discusses a novel contextual bandit problem with post-serving contexts, where additional valuable information about user rewards can be observed after arm selection, and presents poLinUCB, a new algorithm that achieves tight regret under standard assumptions using a robustified version of the Elliptical Potential Lemma.

Learning Cuts via Enumeration Oracles

Daniel Thuerck, Boro Sofranac, Marc Pfetsch, Sebastian Pokutta

https://openreview.net/forum?id=oU4QHdcIWW

Keywords: Integer Programming, Cutting Planes, Optimization

Compressor summary: The paper presents a novel method for learning facets of polyhedra by embedding an enumeration oracle in a Frank-Wolfe algorithm to generate strong cutting planes.

Softmax Output Approximation for Activation Memory-Efficient Training of Attention-based Networks

Changhyeon Lee, Seulki Lee

https://openreview.net/forum?id=oScaeIibRx

Keywords: Memory efficient, Activation saving memory, NLP, Transformer

Compressor summary: The paper proposes a method to reduce memory usage in attention-based networks by approximating the softmax output during training and shows that it improves performance with less memory.

Neural Harmonics: Bridging Spectral Embedding and Matrix Completion in Self-Supervised Learning

Marina Munkhoeva, Ivan Oseledets

https://openreview.net/forum?id=oSYjkJKHZx

Keywords: unsupervised learning, self-supervised learning, representation learning, matrix completion

Compressor summary: The paper explores how self-supervised learning methods use a Laplace operator and low-rank matrix completion to learn representations without labels, and analyzes their convergence and performance.

Analyzing the Sample Complexity of Self-Supervised Image Reconstruction Methods

Tobit Klug, Dogukan Atik, Reinhard Heckel

https://openreview.net/forum?id=oRn953uhFq

Keywords: image reconstruction, denoising, accelerated MRI, self-supervised, sample complexity

Compressor summary: The paper compares the performance and sample complexity of supervised and self-supervised methods for image reconstruction tasks, finding that self-supervised methods require more training data but eventually achieve similar results.

Activity Grammars for Temporal Action Segmentation

Dayoung Gong, Joonseok Lee, Deunsol Jung, Suha Kwak, Minsu Cho

https://openreview.net/forum?id=oOXZ5JEjPb

Keywords: neuro-symbolic approach, Temporal action segmentation, grammar

Compressor summary: The paper introduces KARI, a grammar induction algorithm for temporal action segmentation, and BEP, a parser that transforms frame-level probabilities into actions according to the induced grammar.

Neural Fields with Hard Constraints of Arbitrary Differential Order

Fangcheng Zhong, Kyle Thomas Fogarty, Param Hanji, Tianhao Walter Wu, Alejandro Sztrajman, Andrew Everett Spielberg, Andrea Tagliasacchi, Petra Bosilj, Cengiz Oztireli

https://openreview.net/forum?id=oO1IreC6Sd

Keywords: neural fields, constrained optimization

Compressor summary: The authors present Constrained Neural Fields (CNF), a method for imposing hard constraints on neural networks during optimization, inspired by meshless interpolation and spectral collocation techniques in scientific computing.

Particle-based Variational Inference with Generalized Wasserstein Gradient Flow

Ziheng Cheng, Shiyue Zhang, Longlin Yu, Cheng Zhang

https://openreview.net/forum?id=oNuam8eFz2

Keywords: Particle-based VI, generalized Wasserstein gradient flow

Compressor summary: The paper introduces GWG, a novel ParVI framework that improves on ParVIs by using a generalized Wasserstein gradient flow of the KL divergence, which allows for more flexible kernel design and stronger convergence guarantees.

Unbiased constrained sampling with Self-Concordant Barrier Hamiltonian Monte Carlo

Maxence Noble, Valentin De Bortoli, Alain Durmus

https://openreview.net/forum?id=oMm1dfo3tK

Keywords: Hamiltonian Monte Carlo, Riemannian manifold, self-concordant barrier, constrained sampling

Compressor summary: The paper introduces Barrier Hamiltonian Monte Carlo, a method for sampling from constrained Gibbs distributions using Hamiltonian dynamics and a new filter step, which overcomes bias issues in existing Riemannian HMC methods.

When Demonstrations meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning

Siliang Zeng, Chenliang Li, Alfredo Garcia, Mingyi Hong

https://openreview.net/forum?id=oML3v2cFg2

Keywords: Inverse Reinforcement Learning, Model-based Offline Inverse Reinforcement Learning

Compressor summary: The paragraph introduces a bi-level optimization approach to improve offline inverse reinforcement learning by incorporating uncertainty in the expert's model of the world.

Worst-case Performance of Popular Approximate Nearest Neighbor Search Implementations: Guarantees and Limitations

Piotr Indyk, Haike Xu

https://openreview.net/forum?id=oKqaWlEfjY

Keywords: Nearest neighbor search; graph-based algorithms; worst-case analysis

Compressor summary: The paper analyzes the theoretical guarantees of graph-based approximate nearest neighbor search algorithms, finding that some variants have slow preprocessing but good performance, while others have fast preprocessing but high query time.

UniT: A Unified Look at Certified Robust Training against Text Adversarial Perturbation

Muchao Ye, Ziyi Yin, Tianrong Zhang, Tianyu Du, Jinghui Chen, Ting Wang, Fenglong Ma

https://openreview.net/forum?id=oGxE2Nvlda

Keywords: certified robust training, text adversarial defense

Compressor summary: The paragraph describes a new framework called UniT that unifies and improves existing certified robust training pipelines for text classification by working in the word embedding space and using decoupled regularization loss to enhance the base model's robustness.

Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion

Zhengyi Yang, Jiancan Wu, Zhicai Wang, Xiang Wang, Yancheng Yuan, Xiangnan He

https://openreview.net/forum?id=oFpBnt6bgC

Keywords: Sequential Recommendation, Recommendation System, Generative Model, Diffusion Model

Compressor summary: The text describes a new sequential recommendation method, DreamRec, that generates an oracle item based on the user's historical interactions, rather than relying on negative sampling and classification.

Gradient Informed Proximal Policy Optimization

Sanghyun Son, Laura Yu Zheng, Ryan Sullivan, Yi-Ling Qiao, Ming Lin

https://openreview.net/forum?id=oFaLc6fHSt

Keywords: Reinforcement Learning, Analytic Gradient-Based Policy Learning, Proximal Policy Optimization, Differentiable Programming

Compressor summary: The paper introduces a new policy learning method that combines analytical gradients with PPO, using an α-policy to adjust the influence of these gradients and improve performance in different environments.

Directional diffusion models for graph representation learning

Run Yang, Yuling Yang, Fan Zhou, Qiang Sun

https://openreview.net/forum?id=oDtyJt5JLk

Keywords: diffusion models, graph representation learning, unsupervised learning

Compressor summary: The paper proposes directional diffusion models for unsupervised graph representation learning, which use data-dependent, anisotropic, and directional noises to better capture graph structures and outperform existing methods on 12 datasets.

Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective

Zeyu Zhang, Yi Su, Hui Yuan, Yiran Wu, Rishab Balasubramanian, Qingyun Wu, Huazheng Wang, Mengdi Wang

https://openreview.net/forum?id=oDcWnfZyZW

Keywords: learning to rank, off-policy learning, reinforcement learning, click model

Compressor summary: The paper proposes a method (CUOLR) for off-policy Learning to Rank that adapts to various click models using offline reinforcement learning without complex debiasing techniques or prior knowledge.

Distributional Policy Evaluation: a Maximum Entropy approach to Representation Learning

Riccardo Zamboni, Alberto Maria Metelli, Marcello Restelli

https://openreview.net/forum?id=o91in9tDEs

Keywords: reinforcement learning, distributional reinforcement learning, maximum entropy estimation, representation learning

Compressor summary: The paper proposes a new Max-Ent framework for evaluating policies in distributional RL, which can consider the state representation complexity and guide state space learning using progressive factorization.

Recovering Unbalanced Communities in the Stochastic Block Model with Application to Clustering with a Faulty Oracle

Chandra Sekhar Mukherjee, Pan Peng, Jiapeng Zhang

https://openreview.net/forum?id=o7W0Zet6p3

Keywords: SBM, Unbalanced SBM, Spectral algorithms, Small cluster barrier

Compressor summary: The paper proposes an SVD-based algorithm to detect communities of varying sizes in the stochastic block model and improves upon previous results, also providing an efficient clustering algorithm with sublinear query complexity.

Regression with Cost-based Rejection

Xin Cheng, Yuzhou Cao, Haobo Wang, Hongxin Wei, Bo An, Lei Feng

https://openreview.net/forum?id=o7HckkxOZH

Keywords: regression, rejection costs, surrogate loss

Compressor summary: The paper proposes a regression problem with cost-based rejection, where models reject examples based on their variance and mean squared error, and shows that this approach can improve prediction performance.

Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels

Jian Chen, Ruiyi Zhang, Tong Yu, Rohan Sharma, zhiqiang xu, Tong Sun, Changyou Chen

https://openreview.net/forum?id=o778eWSr1S

Keywords: diffusion model, label noise, retrieval augmented learning

Compressor summary: The paper proposes a new diffusion model that uses generative uncertainty and neighbor consistency to learn from noisy labels and achieve state-of-the-art results on real-world datasets.

Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability

Jishnu Ray Chowdhury, Cornelia Caragea

https://openreview.net/forum?id=o6yTKfdnbA

Keywords: Recursive Neural Networks, Long Range Arena, RvNN, Long Range Sequence Modeling, Length Generalization, LRA, Structured Encoding, Inductive Bias, Hierarchical Model, Recursive Models

Compressor summary: RIR combines a balanced tree model with a Beam Tree RvNN inner recursion to achieve length-generalization on ListOps and scalability on long sequence tasks like LRA, outperforming Transformers.

Arbitrarily Scalable Environment Generators via Neural Cellular Automata

Yulun Zhang, Matthew Christopher Fontaine, Varun Bhatt, Stefanos Nikolaidis, Jiaoyang Li

https://openreview.net/forum?id=o6Dnt1uEyZ

Keywords: Multi-robot systems, quality diversity, automatic environment generation, neural cellular automata

Compressor summary: The paper proposes optimizing Neural Cellular Automata (NCA) environment generators via Quality Diversity (QD) algorithms to create arbitrarily large environments for multi-robot systems, improving their scalability and throughput.

Certifiably Robust Graph Contrastive Learning

Minhua Lin, Teng Xiao, Enyan Dai, Xiang Zhang, Suhang Wang

https://openreview.net/forum?id=o50nH0sV9x

Keywords: Certifiable Robustness, Graph Contrastive Learning

Compressor summary: The paper proposes a unified criterion and a novel technique called RES to certify and enhance the robustness of Graph Contrastive Learning models against adversarial attacks, which can be proven to be effective in downstream tasks.

Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Data

Siyuan Guo, Viktor Tóth, Bernhard Schölkopf, Ferenc Huszár

https://openreview.net/forum?id=o4RtDFMSNL

Keywords: Independent Causal Mechanism, Causal Discovery, Exchangeable, Bayesian Statistics

Compressor summary: The text discusses how exchangeable data can help constraint-based causal discovery methods identify more accurate causal structures than i.i.d. data by using independent causal mechanism generative processes.

Identifiability Guarantees for Causal Disentanglement from Soft Interventions

Jiaqi Zhang, Kristjan Greenewald, Chandler Squires, Akash Srivastava, Karthikeyan Shanmugam, Caroline Uhler

https://openreview.net/forum?id=o16sYKHk3S

Keywords: Causality, Identifiability, Disentanglement

Compressor summary: The paper proposes a method for causal disentanglement using latent variables, which can identify the causal model from unpaired observational and interventional data even when some causal variables are unobserved, and demonstrates its application to predicting combinatorial perturbation effects in genomics.

Active Observing in Continuous-time Control

Samuel Holt, Alihan Hüyük, Mihaela van der Schaar

https://openreview.net/forum?id=o0ggjFD24U

Keywords: Sensing, Model-Based Reinforcement Learning

Compressor summary: The paper introduces a new continuous-time control problem with costly observations, where irregular observation policies can improve expected utility, and presents a simple initial method to solve it.

Individual Arbitrariness and Group Fairness

Carol Xuan Long, Hsiang Hsu, Wael Alghamdi, Flavio Calmon

https://openreview.net/forum?id=nzkWhoXUpv

Keywords: predictive multiplicity, fairness in machine learning, Rashomon effect

Compressor summary: The paper discusses how group fairness and accuracy optimizations can increase predictive multiplicity, a phenomenon where similar models produce different outputs for individual samples, and proposes an ensemble algorithm to improve consistency in individual-level decision-making.

Variational Gaussian Processes with Decoupled Conditionals

Xinran Zhu, Kaiwen Wu, Natalie Maus, Jacob R. Gardner, David Bindel

https://openreview.net/forum?id=nwK8UkK3uB

Keywords: Gaussian processes, variational inference, variational Gaussian processes, Bayesian optimization

Compressor summary: The paper proposes a new way to improve variational Gaussian processes by making the predictive mean and covariance more flexible, leading to better performance on regression tasks and Bayesian optimization.

State-Action Similarity-Based Representations for Off-Policy Evaluation

Brahma S Pavse, Josiah P. Hanna

https://openreview.net/forum?id=nvX3MiQM0G

Keywords: reinforcement learning, off-policy evaluation, off-policy RL, representation learning, behavioral similarity metrics

Compressor summary: The paper proposes a new method to improve off-policy evaluation in reinforcement learning by using a learned encoder to transform the dataset before applying fitted q-evaluation, and introduces an OPE-specific state-action similarity metric to learn the encoder.

A Scale-Invariant Sorting Criterion to Find a Causal Order in Additive Noise Models

Alexander Gilbert Reisach, Myriam Tami, Christof Seiler, Antoine Chambaz, Sebastian Weichwald

https://openreview.net/forum?id=nrbR2F29vU

Keywords: Causal Discovery, Directed Acyclic Graph, Varsortability, Additive Noise Model, Structural Causal Model, Simulation, Benchmark

Compressor summary: The paper studies how different parameter choices in additive noise models affect causal discovery and introduces a new method called $R^2$-SortnRegress that leverages high $R^2$ values to find causal orders.

Leave No Stone Unturned: Mine Extra Knowledge for Imbalanced Facial Expression Recognition

Yuhang Zhang, Yaqi Li, lixiong Qin, Xuannan Liu, Weihong Deng

https://openreview.net/forum?id=nrQif5tH7O

Keywords: Facial expression recognition, imbalanced learning

Compressor summary: The paper proposes a novel method for facial expression recognition that extracts extra knowledge from both major and minor class samples using re-balanced attention maps and smooth labels to handle the imbalance problem.

CLIP4HOI: Towards Adapting CLIP for Practical Zero-Shot HOI Detection

Yunyao Mao, Jiajun Deng, Wengang Zhou, Li Li, Yao Fang, Houqiang Li

https://openreview.net/forum?id=nqIIWnwe73

Keywords: human-object interaction detection, zero-shot learning, CLIP model adaptatiion

Compressor summary: CLIP4HOI is a novel framework for zero-shot human-object interaction detection that uses CLIP to independently identify humans and objects and adapts the model into a fine-grained classifier for proposal discrimination, improving transferability and performance on rare and unseen categories.

Truncated Affinity Maximization: One-class Homophily Modeling for Graph Anomaly Detection

Hezhe Qiao, Guansong Pang

https://openreview.net/forum?id=nq4OhifyEe

Keywords: Anomaly Detection, Graph Neural Network, Graph Anomaly Detection, One-Class Homophily, Local Node Affinity

Compressor summary: The paper introduces a novel anomaly scoring method for graph anomaly detection, based on local node affinity, and a tailored representation learning approach called Truncated Affinity Maximization (TAM) that leverages the one-class homophily phenomenon.

Marginal Density Ratio for Off-Policy Evaluation in Contextual Bandits

Muhammad Faaiz Taufiq, Arnaud Doucet, Rob Cornish, Jean-Francois Ton

https://openreview.net/forum?id=noyleECBam

Keywords: contextual bandits, variance reduction, off-policy evaluation

Compressor summary: The Marginal Ratio (MR) estimator is a new off-policy evaluation method for contextual bandits that reduces variance compared to existing methods like Inverse Probability Weighting and Doubly Robust, and performs well in causal inference settings.

Joint Feature and Differentiable $ k $-NN Graph Learning using Dirichlet Energy

Lei Xu, Lei Chen, Rong Wang, Feiping Nie, Xuelong Li

https://openreview.net/forum?id=noMktb4ait

Keywords: Feature Selection, Differential k-NN Graph, Dirichlet Energy

Compressor summary: The paper proposes a deep feature selection method that uses differentiable k-NN graph learning based on Dirichlet Energy to identify important features and learn dynamic graphs, and applies Optimal Transport theory to address non-differentiability issues.

Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima

Dongkuk Si, Chulhee Yun

https://openreview.net/forum?id=nijJN0LHqM

Keywords: Sharpness-Aware Minimization, convex optimization

Compressor summary: The paper studies how varying the parameters in an optimizer called SAM affects its convergence properties on smooth and non-smooth functions.

Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense

Zunzhi You, Daochang Liu, Bohyung Han, Chang Xu

https://openreview.net/forum?id=niHkj9ixUZ

Keywords: self-supervised learning, adversarial robustness

Compressor summary: The paper proposes De^3, a method that uses noisy image modeling to improve adversarial robustness of deep neural networks by exploiting the pretrained decoder for denoising and sampling the noise scale hyperparameter from random distributions.

Post-processing Private Synthetic Data for Improving Utility on Selected Measures

Hao Wang, Shivchander Sudalairaj, John Henning, Kristjan Greenewald, Akash Srivastava

https://openreview.net/forum?id=neu9JlNweE

Keywords: differential privacy, synthetic data

Compressor summary: The paper introduces a post-processing technique to improve the utility of synthetic data for specific end-user requirements while maintaining privacy and quality, by resampling and optimizing weights using efficient algorithms.

Symbolic Discovery of Optimization Algorithms

Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, Quoc V Le

https://openreview.net/forum?id=ne6zeqLFCZ

Keywords: AutoML

Compressor summary: The authors propose a method to discover optimization algorithms for deep neural network training and introduce Lion, an efficient and effective optimizer that performs well on various tasks.

Learning Curves for Deep Structured Gaussian Feature Models

Jacob A Zavatone-Veth, Cengiz Pehlevan

https://openreview.net/forum?id=nbG6zfJtIe

Keywords: random feature models, generalization, deep networks, ridge regression

Compressor summary: The study examines how weight anisotropy and structured Gaussian features impact generalization in deep learning models using the replica trick from statistical physics.

Generating Behaviorally Diverse Policies with Latent Diffusion Models

Shashank Hegde, Sumeet Batra, K.R. Zentner, Gaurav S. Sukhatme

https://openreview.net/forum?id=nafgeYknRT

Keywords: Latent Diffusion, Quality Diversity, Reinforcement Learning, Graph Neural Networks

Compressor summary: The paper proposes using diffusion models to compress a collection of diverse policies into one generative model that can select and sequence behaviors with language.

Uncertainty Quantification via Neural Posterior Principal Components

Elias Nehme, Omer Yair, Tomer Michaeli

https://openreview.net/forum?id=nZ0jnXizyR

Keywords: Uncertainty Quantification, Inverse Problems, Probabilistic Modelling, Principal Components Analysis, Deep Learning

Compressor summary: This paper proposes a fast method to predict the principal components of the posterior distribution for image restoration models, enabling better uncertainty quantification and visualization.

Regret Matching+: (In)Stability and Fast Convergence in Games

Gabriele Farina, Julien Grand-Clément, Christian Kroer, Chung-Wei Lee, Haipeng Luo

https://openreview.net/forum?id=nYgs0qZJ97

Keywords: Regret Matching, Predictive algorithms, Extensive-Form Games

Compressor summary: The paper investigates the instability of regret matching algorithms, proposes fixes, and demonstrates their effectiveness in various game scenarios using theory and experiments.

Mitigating Source Bias for Fairer Weak Supervision

Changho Shin, Sonia Cromp, Dyah Adila, Frederic Sala

https://openreview.net/forum?id=nXPqMyWUnx

Keywords: Weak supervision, fairness

Compressor summary: Weak supervision can produce biased pseudolabels, and a fairness-based technique can improve accuracy and reduce bias in weak supervision without tradeoffs.

Aggregating Capacity in FL through Successive Layer Training for Computationally-Constrained Devices

Kilian Pfeiffer, Ramin Khalili, Joerg Henkel

https://openreview.net/forum?id=nXNsqB4Yr1

Keywords: Federated Learning, Memory, Resource Constraints

Compressor summary: The paper proposes a new method for federated learning on edge devices that reduces resource requirements and improves model accuracy by enabling successive freezing and training of model parameters.

FedGame: A Game-Theoretic Defense against Backdoor Attacks in Federated Learning

Jinyuan Jia, Zhuowen Yuan, Dinuka Sahabandu, Luyao Niu, Arezoo Rajabi, Bhaskar Ramasubramanian, Bo Li, Radha Poovendran

https://openreview.net/forum?id=nX0zYBGEka

Keywords: backdoor defense, federated learning, game theory

Compressor summary: Federated learning faces backdoor attacks from dynamic attackers, and the paper proposes FedGame, a minimax game-based interactive defense mechanism that enhances robustness against such attacks.

RegBN: Batch Normalization of Multimodal Data with Regularization

MORTEZA GHAHREMANI, Christian Wachinger

https://openreview.net/forum?id=nUbdkXqC8R

Keywords: Multimodal Data, Multimodality, Batch Normalization, Heterogeneous data, Regularization, Confounder, Confounding Effect Removal, Data Dependency

Compressor summary: The paper introduces RegBN, a novel approach for multimodal Batch Normalization with REGularization, which uses the Frobenius norm as a regularizer term to address confounders and dependencies among different data sources, simplifying training and inference, and improving performance across diverse modalities and architectures.

Robust Matrix Sensing in the Semi-Random Model

Xing Gao, Yu Cheng

https://openreview.net/forum?id=nSr2epejn2

Keywords: Matrix sensing, Optimization, Low-rank matrix recovery, Semi-random, Adversarial input, Robustness

Compressor summary: The paper proposes a new non-convex optimization algorithm for semi-random matrix sensing that avoids bad local optima by reweighting the input matrices based on the current solution and takes weighted gradient steps.

Shape Non-rigid Kinematics (SNK): A Zero-Shot Method for Non-Rigid Shape Matching via Unsupervised Functional Map Regularized Reconstruction

Souhaib Attaiki, Maks Ovsjanikov

https://openreview.net/forum?id=nSgMh5v5Ne

Keywords: shape matching

Compressor summary: Shape Non-rigid Kinematics (SNK) is a new method for matching shapes that requires no training or ground truth data, and uses an encoder-decoder architecture to deform the source shape to match the target shape.

Differentiable Clustering with Perturbed Spanning Forests

Lawrence Stewart, Francis Bach, Felipe Llinares-López, Quentin Berthet

https://openreview.net/forum?id=nRfcVBsF9n

Keywords: Structured learning, Clustering, Differentiable, weakly supervised, semi-supervised, representation learning

Compressor summary: A new method for clustering data that can be trained end-to-end with efficient gradients and works well in noisy and complex settings is presented, along with a custom loss function for learning from partial clustering data.

Interpretability at Scale: Identifying Causal Mechanisms in Alpaca

Zhengxuan Wu, Atticus Geiger, Thomas Icard, Christopher Potts, Noah Goodman

https://openreview.net/forum?id=nRfClnMhVX

Keywords: Mechanistic Interpretability

Compressor summary: The paper introduces Boundless DAS, an improved method to find interpretable causal structures in large language models, and applies it to the Alpaca model, revealing its use of two boolean variables for numerical reasoning.

Multiclass Boosting: Simple and Intuitive Weak Learning Criteria

Nataly Brukhim, Amit Daniely, Yishay Mansour, Shay Moran

https://openreview.net/forum?id=nQ84YY9Iut

Keywords: Boosting, Multiclass classification, PAC Learning, List PAC Learning

Compressor summary: The paper presents a new boosting method for multiclass classification that generalizes weak learnability, has low sample complexity, and simplifies theoretical applications in list learning.

Eliminating Domain Bias for Federated Learning in Representation Space

Jianqing Zhang, Yang Hua, Jian Cao, Hao Wang, Tao Song, Zhengui XUE, Ruhui Ma, Haibing Guan

https://openreview.net/forum?id=nO5i1XdUS0

Keywords: Federated Learning, Personalized Federated Learning, Representation, Knowledge Transfer

Compressor summary: The paper introduces a framework called Domain Bias Eliminator (DBE) to address representation degeneration in federated learning and improve generalization and personalization abilities.

Memory Efficient Optimizers with 4-bit States

Bingrui Li, Jianfei Chen, Jun Zhu

https://openreview.net/forum?id=nN8TnHB5nw

Keywords: memory efficiency, optimizer, Adam, quantization

Compressor summary: The paper proposes 4-bit quantization of optimizer states for neural networks, achieving memory efficiency without sacrificing accuracy on various tasks.

GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks

Zhonghang Li, Lianghao Xia, Yong Xu, Chao Huang

https://openreview.net/forum?id=nMH5cUaSj8

Keywords: Spatial Temporal Prediction, Deep Neural Networks, Pre-training Model

Compressor summary: The authors propose a spatio-temporal pre-training framework that enhances the performance of downstream models for traffic management and travel planning by learning customized representations and adaptive mask strategies.

Provably Efficient Algorithm for Nonstationary Low-Rank MDPs

Yuan Cheng, Jing Yang, Yingbin Liang

https://openreview.net/forum?id=nMB41QjLDY

Keywords: Reinforcement Learning, Nonstationary Environment, Representation Learning, Policy Optimization, Statistical Complexity

Compressor summary: The paper proposes two algorithms, PORTAL and Ada-PORTAL, for reinforcement learning in nonstationary low-rank Markov Decision Processes, with theoretical guarantees on their sample efficiency and suboptimality gap.

A Framework for Fast and Stable Representations of Multiparameter Persistent Homology Decompositions

David Loiseaux, Mathieu Carrière, Andrew Blumberg

https://openreview.net/forum?id=nKCUDd9GYu

Keywords: Topological Data Analysis, Multiparameter Persistent Homology, Kernel Methods, Convergence Rate, Statistical Learning

Compressor summary: The article proposes a new representation framework for multiparameter persistent homology in topological data analysis, which is stable, efficient, and informative, and shows its effectiveness on geometric and point cloud data.

Evaluating Robustness and Uncertainty of Graph Models Under Structural Distributional Shifts

Gleb Bazhenov, Denis Kuznedelev, Andrey Malinin, Artem Babenko, Liudmila Prokhorenkova

https://openreview.net/forum?id=nJFJcgjnGo

Keywords: graph, distributional shift, structural shift, uncertainty, robustness, graph neural networks

Compressor summary: The authors propose a method to create diverse distributional shifts in graph learning based on node structure, evaluate its difficulty, and show that simple models often perform better than complex ones.

PromptRestorer: A Prompting Image Restoration Method with Degradation Perception

Cong Wang, Jinshan Pan, Wei Wang, Jiangxin Dong, Mengzhu Wang, Yakun Ju, Junyang Chen

https://openreview.net/forum?id=nIaNgaQvsV

Keywords: Degradation Vanishing, Prompting Learning, Image Restoration

Compressor summary: PromptRestorer is a novel image restoration model that uses degradation features to guide the restoration process and achieves state-of-the-art results on 4 tasks.

$H$-Consistency Bounds: Characterization and Extensions

Anqi Mao, Mehryar Mohri, Yutao Zhong

https://openreview.net/forum?id=nI7EmXq2PL

Keywords: consistency, H-consistency, characterization, learning theory

Compressor summary: The paper introduces general characterizations and extensions of $H$-consistency bounds for surrogate losses in multi-class classification, covering constrained and comp-sum losses, and providing tighter bounds than previous studies.

What Truly Matters in Trajectory Prediction for Autonomous Driving?

Tran Phong, Haoran Wu, Cunjun Yu, Panpan Cai, Sifa Zheng, David Hsu

https://openreview.net/forum?id=nG35q8pNL9

Keywords: trajectory prediction; autonomous driving

Compressor summary: The paper studies the dynamics gap between trajectory prediction accuracy on fixed datasets and real-world driving performance, and emphasizes the trade-off between computational efficiency and prediction accuracy for autonomous driving systems.

Regret-Optimal Model-Free Reinforcement Learning for Discounted MDPs with Short Burn-In Time

Xiang Ji, Gen Li

https://openreview.net/forum?id=nFsbQHFmj2

Keywords: reinforcement learning theory, regret minimization, minimax optimality

Compressor summary: The paper presents a new model-free reinforcement learning algorithm that achieves optimal performance with low memory and computational cost, and fast sample efficiency.

Label Correction of Crowdsourced Noisy Annotations with an Instance-Dependent Noise Transition Model

Hui Guo, Boyu Wang, Grace Yi

https://openreview.net/forum?id=nFEQNYsjQO

Keywords: Noisy Label, Instance-Dependent Transition Matrix, Label Correction, Crowdsourcing

Compressor summary: The paper proposes a new label correction algorithm using a Bayesian network to model instance-dependent noise transitions in crowdsourced annotations, with theoretical guarantees and experimental validation.

Contrastive Training of Complex-Valued Autoencoders for Object Discovery

Aleksandar Stanić, Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber

https://openreview.net/forum?id=nF6X3u0FaA

Keywords: object-centric learning, complex-valued networks, unsupervised learning, temporal correlation hypothesis

Compressor summary: The authors propose a new method for synchrony-based object-centric models that improves their ability to learn from multi-object color datasets and store more than three objects.

Private Distribution Learning with Public Data: The View from Sample Compression

Shai Ben-David, Alex Bie, Clement Louis Canonne, Gautam Kamath, Vikrant Singhal

https://openreview.net/forum?id=nDIrJmKPd5

Keywords: differential privacy, distribution learning, gaussians, mixture of gaussians, compression schemes, robust compression schemes, privacy

Compressor summary: The paper studies how to privately learn a distribution from public and private data, and shows that this depends on compressing the data and learning from lists of examples.

FouriDown: Factoring Down-Sampling into Shuffling and Superposing

Qi Zhu, Man Zhou, Jie Huang, Naishan Zheng, Hongzhi Gao, Chongyi Li, Yuan Xu, Feng Zhao

https://openreview.net/forum?id=nCwStXFDQu

Keywords: Image restoration, Down-Sampling, Fourier transform

Compressor summary: The authors propose a new down-sampling method called FouriDown, which uses a learnable and context-adaptive Fourier function to improve image restoration tasks like de-blurring and low-light enhancement.

The Equivalence of Dynamic and Strategic Stability under Regularized Learning in Games

Victor Boone, Panayotis Mertikopoulos

https://openreview.net/forum?id=nCLdsEzZBV

Keywords: Regularized learning, dynamic stability, strategic stability, Nash equilibrium

Compressor summary: The paper studies how no-regret learning in multiplayer games leads to setwise rationality properties and closedness under better replies, and explores the convergence rate of different regularization methods.

Predicting Global Label Relationship Matrix for Graph Neural Networks under Heterophily

Langzhang Liang, Xiangjing Hu, Zenglin Xu, Zixing Song, Irwin King

https://openreview.net/forum?id=nBFMCyEi0j

Keywords: graph neural networks, heterophily problem, global label relationship matrix

Compressor summary: LRGNN is a generic GNN that works on both homophilous and heterophilous graphs by predicting the low-rank label relationship matrix using robust low-rank approximation, which has two advantages for graph modeling.

Training Transitive and Commutative Multimodal Transformers with LoReTTa

Manuel Tran, Yashin Dicente Cid, Amal Lahiani, Fabian J Theis, Tingying Peng, Eldad Klaiman

https://openreview.net/forum?id=nArzDm353Y

Keywords: generative pre-training, causal modeling, masked modeling, commutative modeling, transitive modeling, multimodal learning

Compressor summary: LoReTTa is a self-supervised framework that links modalities with causal modeling and masked modeling to handle multimodal datasets with missing modalities.

Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses

Gon Buzaglo, Niv Haim, Gilad Yehudai, Gal Vardi, Yakir Oz, Yaniv Nikankin, michal Irani

https://openreview.net/forum?id=nA9Fh3HFHJ

Keywords: memorization, data reconstruction, implicit bias

Compressor summary: This paper extends a previous study on reconstructing training data from neural networks and explores factors affecting this process, such as weight decay and neuron count.

Environment-Aware Dynamic Graph Learning for Out-of-Distribution Generalization

Haonan Yuan, Qingyun Sun, Xingcheng Fu, Ziwei Zhang, Cheng Ji, Hao Peng, Jianxin Li

https://openreview.net/forum?id=n8JWIzYPRz

Keywords: dynamic graph learning, out-of-distribution generalization, invariant learning, link prediction

Compressor summary: The EAGLE framework models and exploits spatio-temporal environments for out-of-distribution generalization in dynamic graph neural networks, achieving superior performance against existing methods under distribution shifts.

Clifford Group Equivariant Neural Networks

David Ruhe, Johannes Brandstetter, Patrick Forré

https://openreview.net/forum?id=n84bzMrGUD

Keywords: Clifford algebras, geometric deep dearning, Clifford group equivariance, E(n)-equivariant neural networks, O(n)-equivariant neural networks

Compressor summary: The paragraph introduces a new method for building equivariant neural networks using the Clifford group, which has desirable properties and shows state-of-the-art performance on various tasks.

Learning with Explanation Constraints

Rattana Pukdee, Dylan Sam, J Zico Kolter, Nina Balcan, Pradeep Kumar Ravikumar

https://openreview.net/forum?id=n6ztJ3Lrdj

Keywords: Interpretable ML, Semi-supervised learning, Learning theory

Compressor summary: The paper proposes a learning framework that uses explanation constraints from prior knowledge to improve the performance and interpretability of deep learning models, especially when using gradient information as explanations.

Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneity

Youssef Allouah, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, Geovani Rizk

https://openreview.net/forum?id=n3fPDW87is

Keywords: Optimization, Byzantine resilience, Distributed machine learning, federated learning

Compressor summary: The paper proposes a more realistic heterogeneity model for robust distributed learning algorithms, showing lower breakdown points and better matching with empirical observations.

DRAUC: An Instance-wise Distributionally Robust AUC Optimization Framework

Siran Dai, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, Qingming Huang

https://openreview.net/forum?id=n3ZVdny7OH

Keywords: Robust Learning AUC

Compressor summary: The paper proposes a method to optimize AUC in long-tailed classification scenarios using distributionally robust optimization and a surrogate loss, while addressing label bias.

Fast Optimal Transport through Sliced Generalized Wasserstein Geodesics

Guillaume Mahey, Laetitia Chapel, Gilles Gasso, Clément Bonet, Nicolas Courty

https://openreview.net/forum?id=n3XuYdvhNW

Keywords: Optimal Transport, Wasserstein distance, Generalized Geodesics, Sliced Wasserstein

Compressor summary: The paper introduces a new proxy for Wasserstein distance called min-SWGG, which is based on one-dimensional projections and has benefits in various applications such as gradient flows, shape matching, and image colorization.

2Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression

Alexander Tyurin, Peter Richtárik

https://openreview.net/forum?id=n18MhTsSGb

Keywords: convex optimization, accelerated method, communication compression, bidirectional compression, distributed optimization

Compressor summary: The 2Direction method speeds up distributed convex optimization by using bidirectional compressed communication and a new error-feedback mechanism, outperforming previous methods in both communication complexity and acceleration.

Optimality in Mean Estimation: Beyond Worst-Case, Beyond Sub-Gaussian, and Beyond $1+\alpha$ Moments

Trung Dang, Jasper C.H. Lee, Maoyuan Song, Paul Valiant

https://openreview.net/forum?id=mvSDs51eqQ

Keywords: mean estimation, instance optimality

Compressor summary: The paper explores if mean estimation algorithms can benefit from knowing specific features of the input distribution and finds that it's possible in some cases, but generally not, while introducing a new framework for analyzing algorithm optimality.

Thinker: Learning to Plan and Act

Stephen Chung, Ivan Anokhin, David Krueger

https://openreview.net/forum?id=mumEBl0arj

Keywords: Reinforcement learning, model-based reinforcement learning, planning, Monte Carlo Tree Search, Markov Decision Process

Compressor summary: The Thinker algorithm is a novel reinforcement learning approach that allows agents to interact with and utilize a learned world model for autonomous planning, achieving state-of-the-art performance in Sokoban and competitive results in Atari 2600.

Reliable Off-Policy Learning for Dosage Combinations

Jonas Schweisthal, Dennis Frauen, Valentyn Melnychuk, Stefan Feuerriegel

https://openreview.net/forum?id=muVKSb8gi5

Keywords: off-policy learning, causal inference, reliable machine learning, medicine, dosaging, normalizing flows

Compressor summary: The paper proposes a novel method for personalized medicine that estimates individualized dose-response functions and optimal dosage combinations using neural networks, conditional normalizing flows, and gradient-based learning.

Efficient Sampling of Stochastic Differential Equations with Positive Semi-Definite Models

Anant Raj, Umut Simsekli, Alessandro Rudi

https://openreview.net/forum?id=mookk2nLO9

Keywords: Kernel Methods, Sampling, Fokker-Planck Equation, Fractional Fokker-Planck Equation, Stochastic Differential Equations, Partial Differential Equations

Compressor summary: The paper proposes an efficient method to sample from a stochastic differential equation using a recent probabilistic model and shows that it works well for smooth solutions.

Transient Neural Radiance Fields for Lidar View Synthesis and 3D Reconstruction

Anagh Malik, Parsa Mirdehghan, Sotiris Nousias, Kyros Kutulakos, David B. Lindell

https://openreview.net/forum?id=mmmd2vp0n0

Keywords: neural radiance fields, 3D reconstruction, single-photon lidar, computational imaging

Compressor summary: The paper proposes a novel method to render transient NeRFs using single-photon lidar data and captures light transport phenomena at picosecond timescales, showing improved geometry and appearance compared to point cloud-based supervision.

Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Method

Constantine Caramanis, Dimitris Fotakis, Alkis Kalavasis, Vasilis Kontonis, Christos Tzamos

https://openreview.net/forum?id=mmTy1iyU5G

Keywords: Policy Gradient, Combinatorial Optimization, Gradient Descent

Compressor summary: The authors develop a theoretical framework to analyze effective deep neural network and reinforcement learning methods for solving combinatorial problems, showing they can find near-optimal solutions with a limited number of parameters and without getting stuck in sub-optimal states. They also propose a novel regularization process for gradient descent that improves performance.

A Causal Framework for Decomposing Spurious Variations

Drago Plecko, Elias Bareinboim

https://openreview.net/forum?id=mm9svgvwvk

Keywords: Causal Inference, Confounding, Fair and Explainable AI

Compressor summary: The authors develop formal tools for decomposing spurious associations in causal mechanisms using Markovian and Semi-Markovian models, providing applications in various fields and demonstrating their approach on a real dataset.

Language Quantized AutoEncoders: Towards Unsupervised Text-Image Alignment

Hao Liu, Wilson Yan, Pieter Abbeel

https://openreview.net/forum?id=mlxRLIy7kc

Keywords: Large Language Model, VQVAE, Vector Quantization, Multimodal

Compressor summary: The paper proposes LQAE, a method to align images and texts without supervision, using pretrained language models, which improves few-shot learning in vision tasks like image classification and VQA.

Unleashing the Power of Randomization in Auditing Differentially Private ML

Krishna Pillutla, Galen Andrew, Peter Kairouz, Hugh Brendan McMahan, Alina Oprea, Sewoong Oh

https://openreview.net/forum?id=mlbes5TAAg

Keywords: Differential privacy auditing, multiple canaries, randomization, lifting, adaptive confidence intervals

Compressor summary: The authors propose a method to audit differentially private machine learning by adding multiple randomized examples (canaries) and use lifted differential privacy, statistical tests, and confidence intervals to improve sample complexity.

Robust Mean Estimation Without Moments for Symmetric Distributions

Gleb Novikov, David Steurer, Stefan Tiegel

https://openreview.net/forum?id=mkve1raJUc

Keywords: Robust Mean Estimation, Unbounded First Moment, Symmetric Distributions (Spherical, Elliptical, Product), Filtering Algorithm, Huber Loss

Compressor summary: The paper proposes efficient and robust algorithms for estimating the mean of symmetric distributions, including product Cauchy and elliptical distributions, without moment assumptions or strong distributional assumptions.

Memory-Constrained Algorithms for Convex Optimization

Moise Blanchard, Junhui Zhang, Patrick Jaillet

https://openreview.net/forum?id=mkKQr56xdB

Keywords: Convex optimization, feasibility problem, first-order methods, memory constraints, cutting planes, oracle complexity

Compressor summary: The paper introduces recursive cutting-plane algorithms for solving feasibility problems and convex optimization with constrained memory, achieving sub-polynomial complexity and optimal memory trade-offs in some regimes.

Beyond Average Return in Markov Decision Processes

Alexandre Marthe, Aurélien Garivier, Claire Vernade

https://openreview.net/forum?id=mgNu8nDFwa

Keywords: Markov Decision Process, Dynamic Programming, statistical functionnals, Distributionnal Reinforcement Learning, Policy Evaluation, Planning

Compressor summary: The paragraph discusses how Dynamic Programming can handle certain operations efficiently in Markov Decision Processes, but only for specific statistics classes, and explores the use of Distributional Reinforcement Learning to evaluate functionals approximately.

Large Language Models Are Zero-Shot Time Series Forecasters

Nate Gruver, Marc Anton Finzi, Shikai Qiu, Andrew Gordon Wilson

https://openreview.net/forum?id=md68e8iZK1

Keywords: large language models, time series, probabilistic forecasting

Compressor summary: The authors propose using large language models to predict future values in time series by encoding them as text and show their effectiveness in handling various aspects of time series data.

Neural Lighting Simulation for Urban Scenes

Ava Pun, Gary Sun, Jingkang Wang, Yun Chen, Ze Yang, Sivabalan Manivasagam, Wei-Chiu Ma, Raquel Urtasun

https://openreview.net/forum?id=mcx8IGneYw

Keywords: Scene Relighting, Lighting Estimation, Camera Simulation, Self-Driving, Lighting Simulation, Scene Editing

Compressor summary: LightSim is a neural lighting camera simulation system that creates realistic and diverse images under different illumination conditions for training image-based robot perception models.

SEENN: Towards Temporal Spiking Early Exit Neural Networks

Yuhang Li, Tamar Geller, Youngeun Kim, Priyadarshini Panda

https://openreview.net/forum?id=mbaN0Y0QTw

Keywords: Spiking Neural Networks, ANN-SNN Conversion, Conditional Computing

Compressor summary: This paper introduces Spiking Early-Exit Neural Networks (SEENNs), which adjust the number of timesteps in spiking neural networks based on input samples, achieving better accuracy and efficiency tradeoffs.

Bridging Discrete and Backpropagation: Straight-Through and Beyond

Liyuan Liu, Chengyu Dong, Xiaodong Liu, Bin Yu, Jianfeng Gao

https://openreview.net/forum?id=mayAyPrhJI

Keywords: discrete random variables, back-propagation, straight through

Compressor summary: ReinMax is a novel method that approximates gradients for discrete latent variables in deep learning using Heun's method, achieving second-order accuracy with negligible computation overheads and outperforming existing approaches.

Towards the Difficulty for a Deep Neural Network to Learn Concepts of Different Complexities

Dongrui Liu, Huiqi Deng, Xu Cheng, Qihan Ren, Kangrui Wang, Quanshi Zhang

https://openreview.net/forum?id=mZ3hnyL9bS

Keywords: representation complexity, deep learning

Compressor summary: The paper explores why deep neural networks find it harder to learn complex concepts involving many input variables, and identifies the specific factor that increases learning difficulty.

Class-Conditional Conformal Prediction with Many Classes

Tiffany Ding, Anastasios Nikolas Angelopoulos, Stephen Bates, Michael Jordan, Ryan Tibshirani

https://openreview.net/forum?id=mYz6ApeU4J

Keywords: conformal prediction, uncertainty quantification, class imbalance

Compressor summary: Clustered conformal prediction is a method that groups similar classes together and improves the probability of correctly predicting their labels when there is limited labeled data per class.

Reining Generalization in Offline Reinforcement Learning via Representation Distinction

Yi Ma, Hongyao Tang, Dong Li, Zhaopeng Meng

https://openreview.net/forum?id=mVywRIDNIl

Keywords: Offline Reinforcement Learning

Compressor summary: The paper proposes Representation Distinction (RD), a technique that improves offline reinforcement learning by differentiating between in-sample and out-of-distribution data representations to reduce overgeneralization.

Hierarchical Gaussian Mixture based Task Generative Model for Robust Meta-Learning

Yizhou Zhang, Jingchao Ni, Wei Cheng, Zhengzhang Chen, Liang Tong, Haifeng Chen, Yan Liu

https://openreview.net/forum?id=mVTyeQIiE4

Keywords: Few-Shot Learning, Meta Learning, Task Representation

Compressor summary: The paper proposes a meta-learning framework that models the density of task instances using a Hierarchical Gaussian Mixture based Task Generative Model (HTGM), which helps adapt to new tasks from different or unseen distributions.

InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding

Junda Wu, Tong Yu, Rui Wang, Zhao Song, Ruiyi Zhang, Handong Zhao, Chaochao Lu, Shuai Li, Ricardo Henao

https://openreview.net/forum?id=mSNfjOcDUv

Keywords: soft prompt tuning

Compressor summary: The paragraph describes a new method called InfoPrompt that improves soft prompt tuning by maximizing the mutual information between prompts and other model parameters, leading to faster convergence and better performance.

The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification

Linhao Qu, xiaoyuan Luo, Kexue Fu, Manning Wang, Zhijian Song

https://openreview.net/forum?id=mSDfBXr8Py

Keywords: multiple instance learning, whole slide image classification, prompt learning, vision-language model, few-shot learning

Compressor summary: The paper proposes a novel few-shot weakly supervised learning method for pathology Whole Slide Image (WSI) classification using prompt learning and GPT-4 to handle challenges posed by the weak bag labels within the Multiple Instance Learning framework.

Scaling Open-Vocabulary Object Detection

Matthias Minderer, Alexey A. Gritsenko, Neil Houlsby

https://openreview.net/forum?id=mQPNcBWjGc

Keywords: object detection, open-vocabulary object detection, vision transformers, vision-language models, scaling, self-training

Compressor summary: OWLv2 is a model that uses self-training with pseudo-annotations to scale up object detection data, achieving significant improvements in performance on rare classes.

Slimmed Asymmetrical Contrastive Learning and Cross Distillation for Lightweight Model Training

Jian Meng, Li Yang, Kyungmin Lee, Jinwoo Shin, Deliang Fan, Jae-sun Seo

https://openreview.net/forum?id=mOVEJletyD

Keywords: Contrastive Learning, Self-supervised Learning, Energy-efficient contrastive learning

Compressor summary: SACL-XD is a new self-supervised contrastive learning scheme that improves the performance of lightweight models and reduces energy consumption in AI applications.

(Provable) Adversarial Robustness for Group Equivariant Tasks: Graphs, Point Clouds, Molecules, and More

Jan Schuchardt, Yan Scholten, Stephan Günnemann

https://openreview.net/forum?id=mLe63bAYc7

Keywords: Adversarial robustness, Geometric machine learning, Equivariances, Robustness Certification, Graph neural networks

Compressor summary: The paragraph discusses a new approach to adversarial robustness that considers task equivariance in real-world tasks like molecular property prediction, and proposes methods to achieve provable robustness for various models and architectures.

Efficient Testable Learning of Halfspaces with Adversarial Label Noise

Ilias Diakonikolas, Daniel Kane, Vasilis Kontonis, Sihan Liu, Nikos Zarifis

https://openreview.net/forum?id=mIm0hsUUt1

Keywords: Machine Learning

Compressor summary: The paper presents a fast algorithm for learning halfspaces under the Gaussian distribution with adversarial label noise, using iterative soft localization and testers to ensure data quality.

Theoretically Guaranteed Bidirectional Data Rectification for Robust Sequential Recommendation

yatong sun, Bin Wang, Zhu Sun, Xiaochun Yang, Yan Wang

https://openreview.net/forum?id=mHsxsrLl0y

Keywords: recommender systems, sequential recommendation

Compressor summary: The paper proposes BirDRec, a framework that rectifies unreliable data in sequential recommender systems by adjusting both input and target items, and reduces its complexity with sampling and self-ensemble methods.

Provably Fast Convergence of Independent Natural Policy Gradient for Markov Potential Games

Youbang Sun, Tao Liu, Ruida Zhou, Panganamala Kumar, Shahin Shahrampour

https://openreview.net/forum?id=mA7nTGXjD3

Keywords: Multi Agent Reinforcement Learning, Markov Potential Games, Natural Policy Gradient, Nash Equilibrium

Compressor summary: The paper proposes an efficient algorithm for multi-agent reinforcement learning that finds approximate Nash Equilibria using policy gradients, and demonstrates its effectiveness on two examples.

Learning Motion Refinement for Unsupervised Face Animation

Jiale Tao, Shuhang Gu, Wen Li, Lixin Duan

https://openreview.net/forum?id=m9uHv1Pxq7

Keywords: Face animation, Motion refinement, Structure correlation

Compressor summary: The paper presents a new unsupervised face animation approach that learns both coarse and fine facial motions using a local affine motion model and a novel motion refinement module based on dense correlation between source and driving images.

Characterizing Graph Datasets for Node Classification: Homophily-Heterophily Dichotomy and Beyond

Oleg Platonov, Denis Kuznedelev, Artem Babenko, Liudmila Prokhorenkova

https://openreview.net/forum?id=m7PIJWOdlY

Keywords: graph characteristics, homophily, heterophily, label informativeness, constant baseline, GNN

Compressor summary: The paper discusses the drawbacks of common homophily measures in graph neural networks and proposes new measures called adjusted homophily and label informativeness.

Equivariant Adaptation of Large Pretrained Models

Arnab Kumar Mondal, Siba Smarak Panigrahi, Sékou-Oumar Kaba, Sai Rajeswar, Siamak Ravanbakhsh

https://openreview.net/forum?id=m6dRQJw280

Keywords: deep learning, large pretrained models, symmetry, equivariance, group theory, computer vision, point clouds, foundation models

Compressor summary: Equivariant networks can be made from large pretrained models by using a canonicalization network that transforms input to a canonical form, improving robustness to deterministic data transformations.

Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks

Jiarong Xu, Renhong Huang, XIN JIANG, Yuxuan Cao, Carl Yang, Chunping Wang, Yang Yang

https://openreview.net/forum?id=m2WR1yJ8N9

Keywords: graph neural networks, pre-training

Compressor summary: The paper proposes a graph pre-training framework (APT) that uses fewer, carefully selected data points to improve downstream tasks, by choosing representative and instructive data based on graph properties and predictive uncertainty.

Learning List-Level Domain-Invariant Representations for Ranking

Ruicheng Xian, Honglei Zhuang, Zhen Qin, Hamed Zamani, Jing Lu, Ji Ma, Kai Hui, Han Zhao, Xuanhui Wang, Michael Bendersky

https://openreview.net/forum?id=m21rQusNgb

Keywords: learning to rank, domain adaptation, text ranking

Compressor summary: This paper introduces list-level alignment, a new invariant representation learning method for ranking problems that leverages the list structure of data and provides theoretical and empirical improvements over existing methods.

Efficient Hyper-parameter Optimization with Cubic Regularization

Zhenqian Shen, Hansi Yang, Yong Li, James Kwok, quanming yao

https://openreview.net/forum?id=m11TbsaQQI

Keywords: hyper-parameter optimization, cubic regularization

Compressor summary: The paper proposes a new hyper-parameter optimization method that uses cubic regularization and stochastic relaxation to avoid local optima and work without hyper-gradients.

Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discovery

Sarah Rastegar, Hazel Doughty, Cees G. M. Snoek

https://openreview.net/forum?id=m0vfXMrLwF

Keywords: Generalized category discovery, Open world learning, Open-set recognition

Compressor summary: The paper proposes a self-supervised method for discovering unknown categories at test time by viewing them as optimal solutions to well-defined problems and using minimal length codes to represent the hierarchy in real-world data.

StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models

Yinghao Aaron Li, Cong Han, Vinay S Raghavan, Gavin Mischler, Nima Mesgarani

https://openreview.net/forum?id=m0RbqrUM26

Keywords: Speech Processing, Text-to-Speech, Diffusion Model, Large Language Model, Self-Supervised Speech Model, WavLM

Compressor summary: StyleTTS 2 is a text-to-speech model that uses style diffusion, adversarial training, and large speech language models to generate high-quality, natural-sounding speech with different styles.

DiffComplete: Diffusion-based Generative 3D Shape Completion

Ruihang Chu, Enze Xie, Shentong Mo, Zhenguo Li, Matthias Nießner, Chi-Wing Fu, Jiaya Jia

https://openreview.net/forum?id=lzqaQRsITh

Keywords: 3d shape completion, conditional generation, diffusion models

Compressor summary: The authors present a new diffusion-based method called DiffComplete that balances realism, multi-modality, and high fidelity in 3D shape completion by using hierarchical feature aggregation and occupancy-aware fusion.

Spontaneous symmetry breaking in generative diffusion models

Gabriel Raya, Luca Ambrogioni

https://openreview.net/forum?id=lxGFGMMSVl

Keywords: generative models;diffusion models;score-based generative models; symmetry-breaking

Compressor summary: The paper presents a new understanding of diffusion models' dynamics, leading to improved performance and diversity in generating high-dimensional data.

CARE: Modeling Interacting Dynamics Under Temporal Environmental Variation

Xiao Luo, Haixin Wang, Zijie Huang, Huiyu Jiang, Abhijeet Sadashiv Gangan, Song Jiang, Yizhou Sun

https://openreview.net/forum?id=lwg3ohkFRv

Keywords: Dynamical System, Distribution Shift, Neural ODE, Graph Neural Network

Compressor summary: The paper proposes a probabilistic model called Context-attended Graph ODE (CARE) for time-varying interacting dynamical systems and shows its effectiveness on four datasets.

H-InDex: Visual Reinforcement Learning with Hand-Informed Representations for Dexterous Manipulation

Yanjie Ze, Yuyao Liu, Ruizhe Shi, Jiaxin Qin, Zhecheng Yuan, Jiashun Wang, Huazhe Xu

https://openreview.net/forum?id=lvvaNwnP6M

Keywords: Visual Reinforcement Learning, Representation Learning, Dexterous Manipulation

Compressor summary: The paper proposes a framework called H-InDex that uses human hand poses to improve robotic dexterous manipulation tasks using reinforcement learning.

K-Nearest-Neighbor Local Sampling Based Conditional Independence Testing

Shuai Li, Yingjie Zhang, Hongtu Zhu, Christina Dan Wang, Hai Shu, Ziqi Chen, Zhuoran Sun, Yanfeng Yang

https://openreview.net/forum?id=luyXPdkNSN

Keywords: Conditional Independence testing, causal inference, conditional mutual information, k-nearest neighbor, conditional randomization test, conditional permutation test

Compressor summary: The article introduces a novel conditional independence testing method that is efficient, powerful, and robust, without assumptions about distributions or dependencies.

SPA: A Graph Spectral Alignment Perspective for Domain Adaptation

Zhiqing Xiao, Haobo Wang, Ying Jin, Lei Feng, Gang Chen, Fei Huang, Junbo Zhao

https://openreview.net/forum?id=lpx9LZPVtZ

Keywords: Domain Adaptation, Self-training, Graph Spectra

Compressor summary: The paper proposes a new graph-based method for unsupervised domain adaptation, called SPA, which improves the performance of machine learning models in different target domains by aligning their structures and enhancing discriminability.

ProtoDiff: Learning to Learn Prototypical Networks by Task-Guided Diffusion

Yingjun Du, Zehao Xiao, Shengcai Liao, Cees G. M. Snoek

https://openreview.net/forum?id=lp9GR2t3hn

Keywords: Meta-learning, few-shot learning, diffusion model, prototype

Compressor summary: ProtoDiff is a novel framework that uses task-guided diffusion to generate overfitted prototypes for few-shot learning challenges, improving classification performance.

A Computation and Communication Efficient Method for Distributed Nonconvex Problems in the Partial Participation Setting

Alexander Tyurin, Peter Richtárik

https://openreview.net/forum?id=loxinzXlCx

Keywords: Nonconvex Optimization, Partial Participation, Variance Reduction, Compressed Communication, Distributed Optimization

Compressor summary: The new method optimizes distributed optimization and federated learning by combining variance reduction, partial participation, and compressed communication with optimal oracle and communication complexities.

Robust Learning for Smoothed Online Convex Optimization with Feedback Delay

Pengfei Li, Jianyi Yang, Adam Wierman, Shaolei Ren

https://openreview.net/forum?id=loixpHDZKj

Keywords: Online optimization, competitive algorithm, switching cost

Compressor summary: The paper proposes an online optimization method called Robustness-Constrained Learning that uses machine learning predictions to improve performance while accounting for multi-step costs and feedback delay.

Sample Complexity for Quadratic Bandits: Hessian Dependent Bounds and Optimal Algorithms

Qian Yu, Yining Wang, Baihe Huang, Qi Lei, Jason D. Lee

https://openreview.net/forum?id=lnTpBUge5G

Keywords: optimization, quadratic bandits, sample complexity, optimality

Compressor summary: The paper studies how to efficiently optimize quadratic functions using stochastic zeroth-order methods and introduces the concept of energy allocation to analyze the information-theoretic limitations and proposes a Hessian-independent algorithm that works well for all cases.

Flexible Attention-Based Multi-Policy Fusion for Efficient Deep Reinforcement Learning

Zih-Yun Chiu, Yi-Lin Tuan, William Yang Wang, Michael C. Yip

https://openreview.net/forum?id=lmXNcKhj4c

Keywords: Reinforcement Learning, Deep Reinforcement Learning, Sample Efficiency, Generalizability, Multi-Policy Decision Making, Multi-Policy Continuous Control

Compressor summary: The paper introduces Knowledge-Grounded RL (KGRL), a reinforcement learning paradigm that fuses multiple knowledge policies for efficient and flexible learning, and proposes Knowledge-Inclusive Attention Network (KIAN) as a new actor architecture.

A General Framework for Robust G-Invariance in G-Equivariant Networks

Sophia Sanborn, Nina Miolane

https://openreview.net/forum?id=llP6lmMiXE

Keywords: equivariance, group-equivariant cnns, invariance, pooling, convolutional neural networks

Compressor summary: The $G$-TC layer is a robust group-invariance method for $G$-CNNs that uses triple correlation theory to preserve signal structure and improve classification accuracy, working for both commutative and non-commutative groups.

Single-Pass Pivot Algorithm for Correlation Clustering. Keep it simple!

Konstantin Makarychev, Sayak Chakrabarty

https://openreview.net/forum?id=lkEiOZlmPm

Keywords: correlation clustering, Pivot algorithm, streaming

Compressor summary: The paper proposes a simplified and efficient algorithm for correlation clustering with improved memory usage compared to previous approaches.

Do SSL Models Have Déjà Vu? A Case of Unintended Memorization in Self-supervised Learning

Casey Meehan, Florian Bordes, Pascal Vincent, Kamalika Chaudhuri, Chuan Guo

https://openreview.net/forum?id=lkBygTc0SI

Keywords: self-supervised learning, privacy, data reconstruction, memorization

Compressor summary: SSL models can accidentally remember specific parts of images they've seen before, which could expose private information and be a security risk.

A Theoretical Analysis of the Test Error of Finite-Rank Kernel Ridge Regression

Tin Sum Cheng, Aurelien Lucchi, Anastasis Kratsios, Ivan Dokmanić, David Belius

https://openreview.net/forum?id=lk6KDG6qI7

Keywords: Kernel, regression, bias-variance, generalization

Compressor summary: Existing statistical learning guarantees for general kernel regressors often yield loose bounds when used with finite-rank kernels. Yet, finite-rank kernels naturally appear in a number of machine learning problems, e.g. when fine-tuning a pre-trained deep neural network's last layer to adapt it to a novel task when performing transfer learning. We address this gap for finite-rank kernel ridge regression (KRR) by deriving sharp non-asymptotic upper and lower bounds for the KRR test error of any finite-rank KRR. Our bounds are tighter than previously derived bounds on finite-rank KRR and, unlike comparable results, they also remain valid for any regularization parameters.

Nominality Score Conditioned Time Series Anomaly Detection by Point/Sequential Reconstruction

Chih-Yu Lai, Fan-Keng Sun, Zhengqi Gao, Jeffrey Lang, Duane S Boning

https://openreview.net/forum?id=ljgM3vNqfQ

Keywords: time series, anomaly detection, point anomalies, contextual anomalies, nominality score, induced anomaly score

Compressor summary: The paper presents a framework for unsupervised time series anomaly detection using point-based and sequence-based models, and shows it beats most existing methods.

Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection

Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, Song Mei

https://openreview.net/forum?id=liMSqUuVg9

Keywords: in-context learning, transformers, deep learning theory, learning theory

Compressor summary: The paragraph discusses how neural sequence models based on the transformer architecture can perform in-context learning and adaptively select different algorithms or tasks without explicit prompting, using a comprehensive statistical theory and experimental evidence.

Toward Re-Identifying Any Animal

Bingliang Jiao, Lingqiao Liu, Liying Gao, Ruiqi Wu, Guosheng Lin, PENG WANG, Yanning Zhang

https://openreview.net/forum?id=leS8668NJm

Keywords: Re-identification, Category-generalizable

Compressor summary: The paper introduces a new task called Re-identify Any Animal in the Wild (ReID-AW) and proposes UniReID, a universal re-identification model that can handle any unseen wildlife category using dynamic prompts and semantic knowledge from GPT-4.

A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence

Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Polania Cabrera, Varun Jampani, Deqing Sun, Ming-Hsuan Yang

https://openreview.net/forum?id=lds9D17HRd

Keywords: Semantic Correspondence, Diffusion Models, Vision Transformer, Representation

Compressor summary: The text describes how diffusion models can be used for image processing tasks and explores their properties, performance, and potential applications.

A Single 2D Pose with Context is Worth Hundreds for 3D Human Pose Estimation

Qitao Zhao, Ce Zheng, Mengyuan Liu, Chen Chen

https://openreview.net/forum?id=lclQ2RvWYu

Keywords: Human Pose Estimation; 2D-to-3D Lifting; Context-Aware

Compressor summary: The proposed method improves 3D human pose estimation by using spatial context from 2D pose detectors without relying on temporal clues or additional computation.

Lift Yourself Up: Retrieval-augmented Text Generation with Self-Memory

Xin Cheng, Di Luo, Xiuying Chen, Lemao Liu, Dongyan Zhao, Rui Yan

https://openreview.net/forum?id=lYNSvp51a7

Keywords: natural language processing, retrieval-augmented text generation, self memory

Compressor summary: The paper proposes a novel framework called selfmem that uses its own output as memory to improve text generation tasks like translation, summarization, and dialogue generation.

DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining

Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc V Le, Tengyu Ma, Adams Wei Yu

https://openreview.net/forum?id=lXuByUeHhd

Keywords: language models, pretraining, domain reweighting, data curation

Compressor summary: DoReMi is a method that uses a small proxy model to optimize mixture proportions of pretraining data domains and improve language model performance without knowing downstream tasks.

Entropy-based Training Methods for Scalable Neural Implicit Samplers

Weijian Luo, Boya Zhang, Zhihua Zhang

https://openreview.net/forum?id=lXOoR4KYcJ

Keywords: implicit sampler, learning to sample, generative models

Compressor summary: The paper introduces a neural implicit sampler that efficiently generates samples from un-normalized target distributions using two novel training methods, and shows its effectiveness, efficiency, and scalability on three sampling benchmarks.

Not All Out-of-Distribution Data Are Harmful to Open-Set Active Learning

Yang Yang, Yuxuan Zhang, XIN SONG, Yi Xu

https://openreview.net/forum?id=lV3LIGlc1w

Keywords: Out-of-Distribution, Active Learning

Compressor summary: Progressive Active Learning (PAL) is a sampling scheme that selects valuable out-of-distribution instances and balances pseudo-ID and pseudo-OOD instances to improve both ID classifier and OOD detector performance in open-set active learning.

Unconstrained Dynamic Regret via Sparse Coding

Zhiyu Zhang, Ashok Cutkosky, Ioannis Paschalidis

https://openreview.net/forum?id=lT9n36RH1w

Keywords: Dynamic online learning, parameter-free online learning, time series forecasting, wavelet

Compressor summary: The paper proposes adaptive regret bounds for online convex optimization under nonstationary comparator sequences by using sparse coding and energy-sparse complexity measures.

Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards

Alexandre Rame, Guillaume Couairon, Corentin Dancette, Jean-Baptiste Gaya, Mustafa Shukor, Laure Soulier, Matthieu Cord

https://openreview.net/forum?id=lSbbC2VyCu

Keywords: Deep learning, Foundation models, Fine-tuning, Reward optimization, Linear mode connectivity, Weight averaging, Model soups, Robustness, Generalization, Alignment, Multi objective learning.

Compressor summary: The paper introduces rewarded soup, a multi-policy approach that aims for Pareto-optimal generalization across different rewards and preferences, by interpolating specialized networks' weights linearly.

Masked Space-Time Hash Encoding for Efficient Dynamic Scene Reconstruction

Feng Wang, Zilong Chen, Guokang Wang, Yafei Song, Huaping Liu

https://openreview.net/forum?id=lSLYXuLqRQ

Keywords: NeRF, Dynamic Scenes

Compressor summary: MSTH is a novel method that efficiently reconstructs dynamic 3D scenes from videos by using a learnable mask to guide a weighted combination of 3D and 4D hash encodings, reducing redundancy and achieving better results than previous methods with less training time and memory storage.

Extensible Prompts for Language Models on Zero-shot Language Style Customization

Tao Ge, Jing Hu, Li Dong, Shaoguang Mao, Yan Xia, Xun Wang, Si-Qing Chen, Furu Wei

https://openreview.net/forum?id=lRxpVfDMzz

Keywords: large language model, prompt, imaginary words, OOD robustness, natural language, zero-shot

Compressor summary: X-Prompt uses imaginary words that can be understood by large language models to help them comprehend complex concepts and enable better interactions with humans.

Social Motion Prediction with Cognitive Hierarchies

Wentao Zhu, Jason Qin, Yuke Lou, Hang Ye, Xiaoxuan Ma, Hai Ci, Yizhou Wang

https://openreview.net/forum?id=lRu0dN7BY6

Keywords: multi-person motion prediction

Compressor summary: The study introduces a new benchmark, formulation, and framework for predicting 3D multi-person motions in team sports using behavioral cloning, generative adversarial imitation learning, and cognitive hierarchy.

Functional-Group-Based Diffusion for Pocket-Specific Molecule Generation and Elaboration

Haitao Lin, Yufei Huang, Odin Zhang, Yunfan Liu, Lirong Wu, Siyuan Li, Zhiyuan Chen, Stan Z. Li

https://openreview.net/forum?id=lRG11M91dx

Keywords: sturcture-based drug design; molecule generation; diffusion model

Compressor summary: \textsc{D3FG} is a functional-group-based diffusion model that generates realistic and interacting molecules for protein pocket-specific drug design using graph neural networks.

Energy-Based Cross Attention for Bayesian Context Update in Text-to-Image Diffusion Models

Geon Yeong Park, Jeongsol Kim, Beomsu Kim, Sang Wan Lee, Jong Chul Ye

https://openreview.net/forum?id=lOCHMGO6ow

Keywords: Diffusion model, Energy-based model, Text-to-image generation

Compressor summary: The paper proposes a novel energy-based model framework that adapts context control for text-to-image diffusion models, improving semantic alignment and enabling zero-shot compositional generation for various image generation tasks.

Hypothesis Selection with Memory Constraints

Maryam Aliakbarpour, Mark Bun, Adam Smith

https://openreview.net/forum?id=lM1UnEssuX

Keywords: Hypothesis selection, memory constrained algorithms, density estimation, limited space

Compressor summary: The paper studies how to select the best distribution from a set of candidates using limited memory and queries, achieving nearly optimal tradeoffs between memory usage and sample complexity.

On the Interplay between Social Welfare and Tractability of Equilibria

Ioannis Anagnostides, Tuomas Sandholm

https://openreview.net/forum?id=lM0xyViO90

Keywords: learning in games, optimistic gradient descent, Nash equilibrium, price of anarchy, smooth games, social welfare

Compressor summary: The paper connects efficiency, smoothness, and no-regret learning algorithms to study equilibrium computation in different game types and improve welfare bounds.

PDP: Parameter-free Differentiable Pruning is All You Need

Minsik Cho, Saurabh Adya, Devang Naik

https://openreview.net/forum?id=lLztVBaBVU

Keywords: pruning, cnn, transformers

Compressor summary: PDP is an efficient and effective train-time pruning scheme for DNNs that works on various tasks, architectures, and pruning constraints, achieving state-of-the-art results in model size, accuracy, and training cost.

Unlimiformer: Long-Range Transformers with Unlimited Length Input

Amanda Bertsch, Uri Alon, Graham Neubig, Matthew R. Gormley

https://openreview.net/forum?id=lJWUJWLCJo

Keywords: retrieval augmentation, summarization, long-context, generation, long-input, encoder-decoder, transformers, language models, natural language generation, natural language processing, deep learning, neural networks

Compressor summary: The paper introduces Unlimiformer, a method that allows transformers to handle long inputs by using a kNN index for attention instead of attending to every token.

Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism

Chengcheng Wang, Wei He, Ying Nie, Jianyuan Guo, Chuanjian Liu, Yunhe Wang, Kai Han

https://openreview.net/forum?id=lJDoPAjkCV

Keywords: YOLO, object detection, computer vision

Compressor summary: The paper introduces Gold-YOLO, a new real-time object detection model that uses convolution and self-attention to improve feature fusion and achieves state-of-the-art performance while maintaining low latency.

The CLIP Model is Secretly an Image-to-Prompt Converter

Yuxuan Ding, Chunna Tian, Haoxuan Ding, Lingqiao Liu

https://openreview.net/forum?id=lHa7gFbmvS

Keywords: Diffusion Model, CLIP model, Image Variation, Customized Generation

Compressor summary: The paper shows how to use CLIP to convert images into text prompts for text-to-image generation, improving this ability with some training data or online steps, and enabling better image-text interactions for tasks like variation and editing.

AutoGO: Automated Computation Graph Optimization for Neural Network Evolution

Mohammad Salameh, Keith G Mills, Negar Hassanpour, Fred X. Han, Shuting Zhang, Wei Lu, SHANGLING JUI, CHUNHUA ZHOU, Fengyu Sun, Di Niu

https://openreview.net/forum?id=lDI3ZuyzM9

Keywords: Neural Architecture Search, Optimization Framework, Performance Prediction

Compressor summary: AutoGO is a framework that optimizes neural networks by evolving their low-level computation graphs, improving performance and hardware compatibility for various computer vision tasks.

An information-theoretic quantification of the content of communication between brain regions

Marco Celotto, Jan Bím, Alejandro Tlaie, Vito De Feo, Alessandro Toso, Stefan M Lemke, Daniel Chicharro, Hamed Nili, Malte Bieler, Ileana Livia Hanganu-Opatz, Tobias H. Donner, Andrea Brovelli, Stefano Panzeri

https://openreview.net/forum?id=lD8xaUWw24

Keywords: Information transmission; Brain data analysis; Sensory processing; Partial information decomposition

Compressor summary: The authors develop a new method called Feature-specific Information Transfer (FIT) that can measure how much information about specific features is transmitted between brain regions, beyond what traditional methods can reveal.

Team-PSRO for Learning Approximate TMECor in Large Team Games via Cooperative Reinforcement Learning

Stephen Marcus McAleer, Gabriele Farina, Gaoyue Zhou, Mingzhi Wang, Yaodong Yang, Tuomas Sandholm

https://openreview.net/forum?id=lCThtrJxoH

Keywords: PSRO, team games, TMECor, populations, equilibrium, game theory, RL

Compressor summary: The paper introduces two algorithms for multi-player team games that balance game-theoretic guarantees with scalability and outperform existing methods on some experiments.

Adversarial Learning for Feature Shift Detection and Correction

Míriam Barrabés, Daniel Mas Montserrat, Margarita Geleta, Xavier Giró-i-Nieto, Alexander G Ioannidis

https://openreview.net/forum?id=lBhRTO2uWf

Keywords: feature shift detection, distribution shift, shift, data-centric AI

Compressor summary: The paper proposes a method to detect and fix feature shifts in data using adversarial learning and simple iterative heuristics, outperforming existing approaches.

Normalization Layers Are All That Sharpness-Aware Minimization Needs

Maximilian Mueller, Tiffany Joyce Vlaar, David Rolnick, Matthias Hein

https://openreview.net/forum?id=lArwl3y9x6

Keywords: sharpness-aware minimization, flatness, generalization, normalization layers

Compressor summary: The paper explores how perturbing only the normalization parameters in the adversarial step of Sharpness-aware minimization (SAM) can improve generalization performance compared to perturbing all parameters or using alternative sparse perturbation methods.

DrugCLIP: Contrasive Protein-Molecule Representation Learning for Virtual Screening

Bowen Gao, Bo Qiang, Haichuan Tan, Yinjun Jia, Minsi Ren, Minsi Lu, Jingjing Liu, Wei-Ying Ma, Yanyan Lan

https://openreview.net/forum?id=lAbCgNcxm7

Keywords: Application, Drug Discovery, Representation Learning, Dataset Augmentation

Compressor summary: The paper proposes a new contrastive learning method called DrugCLIP for faster and better virtual screening of potential drugs using protein pocket and molecule representations without explicit binding-affinity scores.

Unsupervised Learning for Solving the Travelling Salesman Problem

Yimeng Min, Yiwei Bai, Carla P Gomes

https://openreview.net/forum?id=lAEc7aIW20

Keywords: Combinatorial Optimization, Graph Neural Network, Travelling Salesman Problem

Compressor summary: UTSP is an unsupervised learning framework that uses a graph neural network and local search to solve the TSP, achieving better results than existing data-driven heuristics while being more efficient in terms of parameters and training samples.

Globally solving the Gromov-Wasserstein problem for point clouds in low dimensional Euclidean spaces

Martin Ryner, Jan Kronqvist, Johan Karlsson

https://openreview.net/forum?id=l9MbuqzlZt

Keywords: Gromov-Wasserstein problem, QAP, Global optimization

Compressor summary: The paper proposes an efficient method for solving the Gromov-Wasserstein problem, which finds the best assignment between two sets of points preserving distances, using low-dimensional optimization.

Visual Instruction Inversion: Image Editing via Image Prompting

Thao Nguyen, Yuheng Li, Utkarsh Ojha, Yong Jae Lee

https://openreview.net/forum?id=l9BsCh8ikK

Keywords: image editing, diffusion models, visual prompting

Compressor summary: The paper proposes a method for image editing using visual prompts and text-to-image diffusion models to generate editing directions from examples.

Generative Category-level Object Pose Estimation via Diffusion Models

Jiyao Zhang, Mingdong Wu, Hao Dong

https://openreview.net/forum?id=l6ypbj6Nv5

Keywords: Category-Level Object Pose Estimation, Diffusion Model

Compressor summary: The authors propose a novel solution for category-level object pose estimation using conditional generative modeling with score-based diffusion models, achieving state-of-the-art results on the REAL275 dataset and generalizing well to new categories.

Practical Contextual Bandits with Feedback Graphs

Mengxiao Zhang, Yuheng Zhang, Olga Vrousgou, Haipeng Luo, Paul Mineiro

https://openreview.net/forum?id=l6pYRbuHpO

Keywords: Online learning with feedback graphs, Contextual Bandits, Practical algorithms

Compressor summary: The paper proposes a method for contextual bandits with feedback graphs that simplifies learning by reducing it to regression and achieving minimal performance loss.

Fine-Grained Visual Prompting

Lingfeng Yang, Yueze Wang, Xiang Li, Xinlong Wang, Jian Yang

https://openreview.net/forum?id=l6R4Go3noz

Keywords: visual prompting, zero-shot, visual language model, referring expression comprehension

Compressor summary: The authors propose a fine-grained visual prompting method that uses pixel-level masks to improve zero-shot instance-level recognition of objects in images, achieving significant improvements over existing methods on several benchmarks.

Relative Entropic Optimal Transport: a (Prior-aware) Matching Perspective to (Unbalanced) Classification

Liangliang Shi, Haoyu Zhen, Gu Zhang, Junchi Yan

https://openreview.net/forum?id=l61Kp1zBwC

Keywords: Optimal Transport; Unbalanced Classification

Compressor summary: The paper proposes a new optimal transport method for classification, which connects optimal transport and machine learning concepts like barycentric projection and transfer learning.

FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space

Shengzhong Liu, Tomoyoshi Kimura, Dongxin Liu, Ruijie Wang, Jinyang Li, Suhas Diggavi, Mani Srivastava, Tarek Abdelzaher

https://openreview.net/forum?id=l4CZCKXoSn

Keywords: Multimodal Time Series; Contrastive Learning; Factorized Latent Space

Compressor summary: FOCAL is a novel contrastive learning framework for multimodal time series that considers modality-specific features and temporal information locality, improving performance in downstream tasks.

BIRD: Generalizable Backdoor Detection and Removal for Deep Reinforcement Learning

Xuan Chen, Wenbo Guo, Guanhong Tao, Xiangyu Zhang, Dawn Song

https://openreview.net/forum?id=l3yxZS3QdT

Keywords: Backdoor Defense, Deep Reinforcement Learning

Compressor summary: BIRD is a technique that detects and removes backdoors from pretrained deep reinforcement learning policies without needing information about the attack or access to its training process.

Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer

Yuandong Tian, Yiping Wang, Beidi Chen, Simon Shaolei Du

https://openreview.net/forum?id=l3HUgVHqGQ

Keywords: transformer, training dynamics, theoretical analysis, self-attention, interpretability, neural network understanding

Compressor summary: The paper analyzes how self-attention in a 1-layer transformer works during training for next token prediction, showing that it acts as a discriminative scanning algorithm with a scan and snap dynamics.

Feature Likelihood Score: Evaluating the Generalization of Generative Models Using Samples

Marco Jiralerspong, Joey Bose, Ian Gemp, Chongli Qin, Yoram Bachrach, Gauthier Gidel

https://openreview.net/forum?id=l2VKZkolT7

Keywords: Generative model, FID, Evaluation, Precision, Recall, Likelihood

Compressor summary: FLS is a new metric that assesses the novelty, fidelity, and diversity of generated data by deep generative models, addressing limitations of existing methods.

Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL

Peng Cheng, Xianyuan Zhan, Zhihao Wu, Wenjia Zhang, Youfang Lin, Shou cheng Song, Han Wang, Li Jiang

https://openreview.net/forum?id=kyXMU3H7RB

Keywords: sample efficiency; offline reinforcement learning; fundamental symmetry

Compressor summary: The paper proposes a new offline reinforcement learning algorithm (TSRL) that leverages time-reversal symmetry of system dynamics to improve performance on small datasets and enhance data efficiency and generalizability.

Minimum-Risk Recalibration of Classifiers

Zeyu Sun, Dogyoon Song, Alfred Hero

https://openreview.net/forum?id=kvXcHfBghm

Keywords: probability calibration, optimal number of bins, label shift adaptation

Compressor summary: The paper introduces minimum-risk recalibration using MSE decomposition, analyzes UMB recalibration, and proposes a two-stage approach for label shift adaptation.

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning.

Mingjia Shi, Yuhao Zhou, Kai Wang, Huaizheng Zhang, Shudong Huang, Qing Ye, Jiancheng Lv

https://openreview.net/forum?id=kuxu4lCRr5

Keywords: Federated Learning, Personalized Federated Learning, Expectation Maximization, Relaxed Mirror Descent

Compressor summary: The paper proposes a new method, pFedBreD, for personalized federated learning that injects prior knowledge into the global model to improve performance on various datasets.

Computing Optimal Nash Equilibria in Multiplayer Games

Youzhi Zhang, Bo An, Venkatramanan Siva Subrahmanian

https://openreview.net/forum?id=kupNhxLc6k

Keywords: Algorithmic game theory, Optimal Nash equilibrium

Compressor summary: The paper proposes a new algorithm, CRM, to efficiently compute a Nash Equilibrium in multiplayer games by using correlation plans to reduce the solution space and improve the speed.

DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models

Ge Zheng, Bin Yang, Jiajin Tang, Hong-Yu Zhou, Sibei Yang

https://openreview.net/forum?id=ktYjrgOENR

Keywords: Chain-of-Thought Reasoning, Multimodal Science Question Answering, Vision and Langauge

Compressor summary: The text discusses challenges in applying chain of thought reasoning to multimodal contexts and proposes a new method called DDCoT that improves reasoning abilities, generalizability, and explainability of AI models.

Multi-task learning with summary statistics

Parker Knight, Rui Duan

https://openreview.net/forum?id=ktTSji9ZIs

Keywords: multi-task learning, genetic risk prediction, summary statistics

Compressor summary: The paper proposes a multi-task learning framework using summary statistics and adaptive parameter selection for applications where data-sharing is restricted, such as healthcare, and analyzes its performance theoretically and empirically.

Towards Last-layer Retraining for Group Robustness with Fewer Annotations

Tyler LaBonte, Vidya Muthukumar, Abhishek Kumar

https://openreview.net/forum?id=kshC3NOP6h

Keywords: spurious correlations, group robustness, last-layer retraining, distribution shift

Compressor summary: The paper proposes a method called SELF that improves group robustness of neural networks by retuning the last layer using disagreements or misclassifications, without needing group annotations.

SpatialRank: Urban Event Ranking with NDCG Optimization on Spatiotemporal Data

BANG AN, Xun Zhou, Yongjian Zhong, Tianbao Yang

https://openreview.net/forum?id=ks7Mf5lzSx

Keywords: urban event, NDCG optimization, ranking, traffic accident, crime, spatiotemporal data

Compressor summary: SpatialRank is a novel method for urban event ranking that uses adaptive graph convolution layers and a hybrid NDCG loss to predict the top-k most risky locations based on spatiotemporal dependencies and neighboring spatial proximity.

Easy Learning from Label Proportions

Robert Istvan Busa-Fekete, Heejin Choi, Travis Dick, Claudio Gentile, Andres Munoz medina

https://openreview.net/forum?id=kqBUgrkm1c

Keywords: learning with partial information, unbiased loss, classification, proportion matching

Compressor summary: EASYLLP is a debiasing technique for weakly supervised classification that uses aggregate labels and works with various loss functions, improving instance level performance on Learning from Label Proportions.

Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks

Mingze Wang, Chao Ma

https://openreview.net/forum?id=konBXvt2iS

Keywords: non-convex optimization, training dynamics, neural network

Compressor summary: The paper analyzes the training process of a two-layer ReLU network using Gradient Flow and identifies four phases and specific nonlinear behaviors that occur during learning.

Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examples

Hao Sun, Alihan Hüyük, Daniel Jarrett, Mihaela van der Schaar

https://openreview.net/forum?id=kmbG9iBRIb

Keywords: Accountability, Reinforcement Learning, Batched Control, Accountable Decision-Making, Offline RL, Interpretability in RL

Compressor summary: The paper presents AOC, a controller that uses an offline dataset as the Decision Corpus and selects a subset of examples for accountable control in low-data scenarios, evaluating it in simulated and real healthcare settings.

Offline Minimax Soft-Q-learning Under Realizability and Partial Coverage

Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun

https://openreview.net/forum?id=kjkLJ7NJJZ

Keywords: Reinforcement learning theory, PAC RL, Offline Reinforcement learning

Compressor summary: The paper proposes value-based algorithms for offline reinforcement learning that can achieve PAC guarantees even with partial coverage of data, using novel minimax loss functions derived from Lagrange functions in nonlinear convex optimization problems.

Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitation

David Brandfonbrener, Ofir Nachum, Joan Bruna

https://openreview.net/forum?id=kjMGHTo8Cs

Keywords: representation learning, imitation learning

Compressor summary: This paper explores how large datasets with multitask demonstrations can be pretrained using inverse dynamics modeling to learn low dimensional representations for imitation learning in unknown environments.

On permutation symmetries in Bayesian neural network posteriors: a variational perspective

Simone Rossi, Ankit Singh, Thomas Hannagan

https://openreview.net/forum?id=kj33zJ9Vue

Keywords: Bayesian deep learning, approximate inference, permutation symmetries

Compressor summary: The authors extend the formalism of marginalized loss barrier and solution interpolation to Bayesian neural networks, propose an algorithm to search for linearly connected solutions, and experiment with different architectures and datasets.

End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes

Alexandre Max Maraval, Matthieu Zimmer, Antoine Grosnit, Haitham Bou Ammar

https://openreview.net/forum?id=kfWzpZvEUh

Keywords: meta-learning, bayesian optimisation, neural process, transformer, end-to-end, reinforcement learning

Compressor summary: The paper presents a differentiable meta-BO framework that uses neural processes and transformers to learn acquisition functions with RL and an auxiliary task to handle sparse rewards.

TextDiffuser: Diffusion Models as Text Painters

Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, Furu Wei

https://openreview.net/forum?id=ke3RgcDmfO

Keywords: Diffusion Model; Text Rendering

Compressor summary: TextDiffuser is a new method for generating images with readable and coherent text that works by first generating keyword layouts with a Transformer model and then using diffusion models to create images based on the layout and text prompts. It also introduces a large-scale dataset and evaluation tool for text rendering quality.

Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition

Shuhuai Ren, Aston Zhang, Yi Zhu, Shuai Zhang, Shuai Zheng, Mu Li, Alex Smola, Xu Sun

https://openreview.net/forum?id=kdFR6IUEW6

Keywords: Prompt Pre-Training, CLIP, Open-Vocabulary Visual Recognition

Compressor summary: POMP is a prompt pre-training method for vision-language models that improves visual recognition tasks with zero-shot learning and achieves state-of-the-art performances on 21 datasets.

DESSERT: An Efficient Algorithm for Vector Set Search with Vector Set Queries

Joshua Engels, Benjamin Coleman, Vihan Lakshman, Anshumali Shrivastava

https://openreview.net/forum?id=kXfrlWXLwH

Keywords: Embedding Based Retrieval, Passage Ranking, Locality Sensitive Hashing, Randomized Algorithms

Compressor summary: The paper introduces DESSERT, a fast approximate search algorithm for vector set queries, which can be integrated into semantic search models like ColBERT to improve their speed and performance.

DaTaSeg: Taming a Universal Multi-Dataset Multi-Task Segmentation Model

Xiuye Gu, Yin Cui, Jonathan Huang, Abdullah Rashwan, Xuan Yang, Xingyi Zhou, Golnaz Ghiasi, Weicheng Kuo, Huizhong Chen, Liang-Chieh Chen, David A Ross

https://openreview.net/forum?id=kXOXrVnwbb

Keywords: universal segmentation, multi-task segmentation, multi-dataset segmentation, panoptic segmentation, semantic segmentation, instance segmentation, weakly-supervised segmentation

Compressor summary: The paper proposes DaTaSeg, a universal multi-dataset multi-task segmentation model that leverages weak-supervision, text embeddings, and merge operations to improve performance across different tasks and datasets, including open-vocabulary segmentation.

Towards Efficient Pre-Trained Language Model via Feature Correlation Distillation

Kun Huang, Xin Guo, Meng Wang

https://openreview.net/forum?id=kVfHQV668B

Keywords: Knowledge Distillation; Pre-Trained Language Model

Compressor summary: FCD is a new method for compressing PLMs that uses output feature relationships to transfer knowledge more effectively and achieves better results than previous KD approaches.

Bayesian Active Causal Discovery with Multi-Fidelity Experiments

Zeyu Zhang, Chaozhuo Li, Xu Chen, Xing Xie

https://openreview.net/forum?id=kS8rIH43Zc

Keywords: Causal Discovery, Active Learning, Multi-fidelity

Compressor summary: The paper presents a probabilistic model for multi-fidelity active causal discovery, with a mutual-information based acquisition function and a cascading model to capture correlations between different fidelity oracles, and extends it to batch intervention scenarios.

A Fractional Graph Laplacian Approach to Oversmoothing

Sohir Maskey, Raffaele Paolino, Aras Bacho, Gitta Kutyniok

https://openreview.net/forum?id=kS7ED7eE74

Keywords: Graph Neural Networks, Graph Neural ODE, Fractional Laplacian, Oversmoothing

Compressor summary: The paper introduces fractional graph Laplacian neural ODEs for directed graphs to capture long-range dependencies and mitigate oversmoothing in graph neural networks.

Inferring Hybrid Neural Fluid Fields from Videos

Hong-Xing Yu, Yang Zheng, Yuan Gao, Yitong Deng, Bo Zhu, Jiajun Wu

https://openreview.net/forum?id=kRdaTkaBwC

Keywords: neural scene representations, fluid dynamics, flow reconstruction, physics-based learning

Compressor summary: The paper presents HyFluid, a neural method to jointly estimate fluid density and velocity from sparse multiview videos, using physics-based losses and a hybrid neural velocity representation that handles visual ambiguities and turbulence in fluid flows.

NuTrea: Neural Tree Search for Context-guided Multi-hop KGQA

Hyeong Kyu Choi, Seunghun Lee, Jaewon Chu, Hyunwoo J. Kim

https://openreview.net/forum?id=kR5ycmBclj

Keywords: Knowledge Graph Question Answering, Knowledge Graph, Graph Neural Networks

Compressor summary: NuTrea is a tree search-based GNN model that improves multi-hop KGQA by incorporating broader KG context and handling ambiguous nodes.

Subclass-Dominant Label Noise: A Counterexample for the Success of Early Stopping

Yingbin Bai, Zhongyi Han, Erkun Yang, Jun Yu, Bo Han, Dadong Wang, Tongliang Liu

https://openreview.net/forum?id=kR21XsZeAr

Keywords: learning with noisy labels, weakly supervised learning

Compressor summary: The paper studies a type of label noise called subclass-dominant label noise (SDN) and proposes a method called NoiseCluster that uses long-trained representations to identify and correct it, achieving better results than existing methods on synthetic and real data.

Online Ad Allocation with Predictions

Fabian Christian Spaeh, Alina Ene

https://openreview.net/forum?id=kPfd3pcwHV

Keywords: Learning Augmented Algorithms, Display Ads, Generalized Assignment Problem

Compressor summary: The paper presents a learning-augmented algorithm for online packing problems that uses machine-learned predictions to improve performance beyond worst-case algorithms, and shows its effectiveness on synthetic and real data.

Abide by the law and follow the flow: conservation laws for gradient flows

Sibylle Marcotte, Rémi Gribonval, Gabriel Peyré

https://openreview.net/forum?id=kMueEV8Eyy

Keywords: Implicit bias, conservation laws, gradient flow, linear neural network, matrix factorization

Compressor summary: The article explores how to identify and count conservation laws, which describe quantities conserved during gradient flows of a given model, and how these laws relate to optimization initialization properties in over-parameterized models.

Point Cloud Completion with Pretrained Text-to-Image Diffusion Models

Yoni Kasten, Ohad Rahamim, Gal Chechik

https://openreview.net/forum?id=kMmAYbT0VL

Keywords: Point Cloud, Text, 3D

Compressor summary: SDS-Complete uses a text-to-image diffusion model to complete incomplete point clouds of various objects without needing extensive 3D data, improving performance on out-of-distribution objects.

On the choice of Perception Loss Function for Learned Video Compression

Sadaf Salehkalaibar, Truong Buu Phan, Jun Chen, Wei Yu, Ashish J Khisti

https://openreview.net/forum?id=kLIieSS2P3

Keywords: Video Compression, Information Theory, Neural Compression

Compressor summary: The choice of perception loss function affects video compression quality, especially at low bit rates, but encoded representations can be near universal and work for either choice of function.

ConDaFormer: Disassembled Transformer with Local Structure Enhancement for 3D Point Cloud Understanding

Lunhao Duan, Shanshan Zhao, Nan Xue, Mingming Gong, Gui-Song Xia, Dacheng Tao

https://openreview.net/forum?id=kKXJkiniOx

Keywords: Point Cloud, Transformer, 3D Segmentation, 3D object detection

Compressor summary: The paper proposes ConDaFormer, a new transformer block for 3D point cloud understanding that reduces costs and models local geometry prior by disassembling cubic windows into three orthogonal 2D planes and using depth-wise convolution to enhance local structure.

On Learning Necessary and Sufficient Causal Graphs

Hengrui Cai, Yixin Wang, Michael Jordan, Rui Song

https://openreview.net/forum?id=kKFDMtpeDW

Keywords: Causal structural learning, Necessity and sufficiency, Natural causal effects, Probabilities of causation, Variable selection

Compressor summary: The paper proposes a method to learn a subset of causally relevant variables from a complex graph using probabilities of causation, which can improve causal estimation in various fields.

When Do Graph Neural Networks Help with Node Classification? Investigating the Homophily Principle on Node Distinguishability

Sitao Luan, Chenqing Hua, Minkai Xu, Qincheng Lu, Jiaqi Zhu, Xiao-Wen Chang, Jie Fu, Jure Leskovec, Doina Precup

https://openreview.net/forum?id=kJmYu3Ti2z

Keywords: Graph Neural Networks, Homophily, Heterophily, Low-pass filter, High-pass filter, Node Distinguishability, Metrics

Compressor summary: The paper studies how the similarity of node neighborhood patterns (intra-class Node Distinguishability) and differences between classes (inter-class ND) affect Graph Neural Networks' performance, proposing new metrics to measure them and a hypothesis-testing based performance metric that goes beyond homophily.

On the Identifiability of Sparse ICA without Assuming Non-Gaussianity

Ignavier Ng, Yujia Zheng, Xinshuai Dong, Kun Zhang

https://openreview.net/forum?id=kJIibP5bq2

Keywords: independent component analysis, second-order statistics, sparsity

Compressor summary: The paper presents a new method for identifying Gaussian sources using independent component analysis that does not require non-Gaussian assumptions or restrictive connective structures, and provides experimental validation.

Limits, approximation and size transferability for GNNs on sparse graphs via graphops

Thien Le, Stefanie Jegelka

https://openreview.net/forum?id=kDQwossJuI

Keywords: graph neural networks, convolution, graph limits, size transferability

Compressor summary: The paper investigates how well graph neural networks (GNNs) can generalize to different graphs, especially sparse ones, using a theoretical approach based on graphops.

Can Pre-Trained Text-to-Image Models Generate Visual Goals for Reinforcement Learning?

Jialu Gao, Kaizhe Hu, Guowei Xu, Huazhe Xu

https://openreview.net/forum?id=kChEBODIx9

Keywords: Visual Reinforcement Learning, Large Generative Models, Image Editing, Robotics

Compressor summary: LfVoid is a method that uses text-to-image models and image editing to guide robots based on natural language instructions, even without in-domain training or true goal observations.

Coherent Soft Imitation Learning

Joe Watson, Sandy Huang, Nicolas Heess

https://openreview.net/forum?id=kCCD8d2aEu

Keywords: imitation learning, inverse reinforcement learning, behavioral cloning, learning from demonstration

Compressor summary: The authors propose a hybrid imitation learning method that combines behavioral cloning and reinforcement learning to learn from expert demonstrations and achieve complex tasks with minimal hyperparameter tuning.

SAME: Uncovering GNN Black Box with Structure-aware Shapley-based Multipiece Explanations

Ziyuan Ye, Rihan Huang, Qilin Wu, Quanying Liu

https://openreview.net/forum?id=kBBsj9KRgh

Keywords: GNN explainability, Shapley value, Monte Carlo tree search, structure awareness, multi-grained explanation

Compressor summary: SAME is a new method for explaining graph neural networks that uses Monte Carlo tree search to explore structure-aware substructures and improves explanation fidelity on various benchmarks.

Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design

Matthew Thomas Jackson, Minqi Jiang, Jack Parker-Holder, Risto Vuorio, Chris Lu, Gregory Farquhar, Shimon Whiteson, Jakob Nicolaus Foerster

https://openreview.net/forum?id=kAU6Cdq1gV

Keywords: Reinforcement Learning, Meta-Learning, Meta-RL, Meta-Optimization, Policy Meta-Optimization, Environment Design, Unsupervised Environment Design, Auto-Curricula

Compressor summary: This paper explores how meta-learning update rules for deep reinforcement learning can improve generalization and proposes GROOVE, a novel method that generates curricula to maximize the regret of a meta-learned optimizer.

Open Compound Domain Adaptation with Object Style Compensation for Semantic Segmentation

Tingliang Feng, Hao Shi, Xueyang Liu, Wei Feng, Liang Wan, Yanlin Zhou, Di Lin

https://openreview.net/forum?id=k9zSU3pdi4

Keywords: Object Style Compensation, Open Compound Domain Adaptation, Semantic Segmentation

Compressor summary: The paper introduces Object Style Compensation, a method that adapts object styles across domains for semantic image segmentation by using an Object-Level Discrepancy Memory.

PDF: Point Diffusion Implicit Function for Large-scale Scene Neural Representation

Yuhan Ding, Fukun Yin, Jiayuan Fan, Hui Li, Xin Chen, Wen Liu, Chongshan Lu, Gang YU, Tao Chen

https://openreview.net/forum?id=k8U8ZijXHh

Keywords: implicit neural representation; diffusion; point cloud; volume rendering

Compressor summary: The authors propose a new method called Point Diffusion implicit Function (PDF) that uses point cloud super-resolution and background modeling to represent and synthesize large-scale outdoor scenes for novel view synthesis, achieving better results than existing methods.

L2T-DLN: Learning to Teach with Dynamic Loss Network

Zhaoyang Hai, Liyuan Pan, Xiabi Liu, Zhengzheng Liu, Mirna Yunita

https://openreview.net/forum?id=k6yNi6DEqK

Keywords: Learning to teach, dynamic loss function, optimization

Compressor summary: The paper proposes a teacher model with memory units and a Dynamic Loss Network to adjust loss functions adaptively based on both the states of the student model and the loss function, leading to improved deep learning performance on various tasks.

Streaming Algorithms and Lower Bounds for Estimating Correlation Clustering Cost

Sepehr Assadi, Vihan Shah, Chen Wang

https://openreview.net/forum?id=k4ZCORSFEd

Keywords: Correlation Clustering, Graph Streaming Algorithms, Large-scale Clustering, Graph Learning

Compressor summary: This paper studies streaming correlation clustering with very limited memory, proposing two novel algorithms that estimate the optimal cost up to a constant factor and some additional error.

L-C2ST: Local Diagnostics for Posterior Approximations in Simulation-Based Inference

Julia Linhart, Alexandre Gramfort, Pedro L. C. Rodrigues

https://openreview.net/forum?id=k2UVKezeWn

Keywords: machine learning, calibration, simulation-based inference, neuroscience, normalizing flows, classifier two-sample tests

Compressor summary: The paper introduces a new method, $\ell$-C2ST, for evaluating deep generative models' approximations of complex posterior distributions at specific observations, improving interpretability and performance over existing approaches.

Lookaround Optimizer: $k$ steps around, 1 step average

Jiangtao Zhang, Shunyu Liu, Jie Song, Tongtian Zhu, Zhengqi Xu, Mingli Song

https://openreview.net/forum?id=k1Xy5zCNOJ

Keywords: Deep Learning, Computer Vision, Mode Connectivity, Weight Average

Compressor summary: Lookaround is a new optimizer that improves diversity and generalization in deep networks by training multiple networks on transformed data and averaging them, leading to flatter minima.

Metropolis Sampling for Constrained Diffusion Models

Nic Fishman, Leo Klarner, Emile Mathieu, Michael John Hutchinson, Valentin De Bortoli

https://openreview.net/forum?id=jzseUq55eP

Keywords: diffusion model, generative modelling, manifold, constraints, proteins, robotics

Compressor summary: The paper proposes a simple noising scheme based on Metropolis sampling for diffusion models with domain-informed constraints, improving efficiency and performance over existing methods.

Generalized test utilities for long-tail performance in extreme multi-label classification

Erik Schultheis, Marek Wydmuch, Wojciech Kotlowski, Rohit Babbar, Krzysztof Dembczynski

https://openreview.net/forum?id=jze2r6RDFz

Keywords: extreme multi-label classification, long-tail labels performance, complex performance measures

Compressor summary: The paper proposes a new metric for extreme multi-label classification that captures the importance of rare labels and optimizes expected test utility using efficient prediction rules.

Ess-InfoGAIL: Semi-supervised Imitation Learning from Imbalanced Demonstrations

Huiqiao Fu, Kaiqiang Tang, Yuanyang Lu, Yiming Qi, Guizhou Deng, Flood Sung, Chunlin Chen

https://openreview.net/forum?id=jxhUNLoi4m

Keywords: Generative adversarial imitation learning, semi-supervised learning, multi-modal behaviors, imbalanced data

Compressor summary: Imitation learning can reproduce expert behaviors without explicit rewards, but real-world demos are challenging; a new semi-supervised method learns disentangled behavior representations from imbalanced data using generative adversarial networks and information maximization.

Auxiliary Losses for Learning Generalizable Concept-based Models

Ivaxi Sheth, Samira Ebrahimi Kahou

https://openreview.net/forum?id=jvYXln6Gzn

Keywords: Interpretability, concept bottleneck models, explainability

Compressor summary: The paper introduces coop-CBM, a transparent neural network model that preserves concept representations and improves performance under different data distributions.

Improving Language Plasticity via Pretraining with Active Forgetting

Yihong Chen, Kelly Marchisio, Roberta Raileanu, David Ifeoluwa Adelani, Pontus Stenetorp, Sebastian Riedel, Mikel Artetxe

https://openreview.net/forum?id=jvEbQBxd8X

Keywords: plasticity, continual learning, meta-learning, embeddings, cross-lingual transfer, forgetting

Compressor summary: The paper proposes using active forgetting during pretraining to create PLMs that can adapt quickly and effectively to new languages, especially when there is little data available.

Deep Reinforcement Learning with Plasticity Injection

Evgenii Nikishin, Junhyuk Oh, Georg Ostrovski, Clare Lyle, Razvan Pascanu, Will Dabney, Andre Barreto

https://openreview.net/forum?id=jucDLW6G9l

Keywords: deep reinforcement learning, continual learning, loss of plasticity

Compressor summary: The paper proposes plasticity injection, a method to increase neural network plasticity in deep reinforcement learning without changing parameters or predictions, and demonstrates its usefulness as a diagnostic and improvement tool for RL agents.

Simple and Controllable Music Generation

Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, Alexandre Défossez

https://openreview.net/forum?id=jtiQ26sCJi

Keywords: Music generation, Generative AI, Transformer, Language Models

Compressor summary: MusicGen is a single transformer model that generates high-quality music based on text or melodic features, outperforming previous methods.

Fine-grained Expressivity of Graph Neural Networks

Jan Böker, Ron Levie, Ningyuan Teresa Huang, Soledad Villar, Christopher Morris

https://openreview.net/forum?id=jt10uWlEbc

Keywords: graphons, universal approximation, weisfeiler-leman, graph metric, tree homomorphisms, tree distance, optimal transport, GNNs

Compressor summary: The paper presents a continuous extension of the Weisfeiler--Leman test for analyzing the expressive power of message-passing graph neural networks on graphons, providing a theoretical and empirical framework to understand their limitations and performance.

Domain Re-Modulation for Few-Shot Generative Domain Adaptation

Yi Wu, Ziqiang Li, Chaoyue Wang, Heliang Zheng, Shanshan Zhao, Bin Li, Dacheng Tao

https://openreview.net/forum?id=jown9RvYn7

Keywords: StyleGAN, Few-Shot Generative Domain Adaptation

Compressor summary: The study proposes a new generator structure called Domain Re-Modulation (DoRM) for few-shot Generative Domain Adaptation, which incorporates memory and domain association to achieve high quality, diversity, and consistency across domains.

Implicit Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis

Zhu Wang, Sourav Medya, Sathya N. Ravi

https://openreview.net/forum?id=jooPcatnVF

Keywords: Implicit layer, Out-of-distribution detection, multimodal learning

Compressor summary: The authors propose a method to combine deep neural networks and semantic knowledge for vision and language tasks, using an implicit out-of-distribution detector to filter irrelevant information.

Learning Reliable Logical Rules with SATNet

Zhaoyu Li, Jinpei Guo, Yuhe Jiang, Xujie Si

https://openreview.net/forum?id=jnIBiP2di1

Keywords: Logical Reasoning, Rule Learning, Interpretation, SATNet

Compressor summary: The paper proposes a new framework that generates interpretable and verifiable logical rules for advanced AI systems using differentiable learning and MaxSAT solver.

D-CIPHER: Discovery of Closed-form Partial Differential Equations

Krzysztof Kacprzyk, Zhaozhi Qian, Mihaela van der Schaar

https://openreview.net/forum?id=jnCPN1vpSR

Keywords: differential equations, symbolic regression

Compressor summary: D-CIPHER is a new method for finding differential equations from data that works with noisy and infrequent observations and discovers more phenomena than existing approaches.

Energy-based learning algorithms for analog computing: a comparative study

Benjamin Scellier, Maxence Ernoult, Jack Kendall, Suhas Kumar

https://openreview.net/forum?id=jl5a3t78Uh

Keywords: energy-based learning algorithm, contrastive learning, equilibrium propagation, coupled learning, convolutional Hopfield network

Compressor summary: The paragraph compares seven energy-based learning algorithms on deep convolutional Hopfield networks for five vision tasks and finds that negative perturbations perform better than positive ones.

Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theory

Sokhna Diarra Mbacke, Florence Clerc, Pascal Germain

https://openreview.net/forum?id=jkPDRHff3s

Keywords: Variational Autoencoders, PAC-Bayes, Statistical Learning Theory

Compressor summary: The authors use PAC-Bayesian theory to provide statistical guarantees for Variational Autoencoders (VAEs) and bounds on their performance metrics.

Adaptive Online Replanning with Diffusion Models

Siyuan Zhou, Yilun Du, Shun Zhang, Mengdi Xu, Yikang Shen, Wei Xiao, Dit-Yan Yeung, Chuang Gan

https://openreview.net/forum?id=jhs8F63xI6

Keywords: Decision making, Robotics, Planning-based

Compressor summary: The paper proposes a method to improve diffusion planning by deciding when to replan and adjusting trajectories, leading to better performance in robotic control tasks.

Finite-Time Analysis of Single-Timescale Actor-Critic

Xuyang Chen, Lin Zhao

https://openreview.net/forum?id=jh3UNSQK0l

Keywords: Finite-time analysis, single-timescale actor-critic

Compressor summary: The paper analyzes the convergence of online single-timescale actor-critic with linear function approximation on continuous state space, achieving an $\epsilon$-approximate stationary point with improved sample complexity.

Debiasing Scores and Prompts of 2D Diffusion for View-consistent Text-to-3D Generation

Susung Hong, Donghoon Ahn, Seungryong Kim

https://openreview.net/forum?id=jgIrJeHHlz

Keywords: Text-to-3D, Diffusion Models

Compressor summary: The paper proposes two methods to improve view consistency in text-to-3D generation by debiasing scores from 2D diffusion models and aligning user prompts with object views.

From ViT Features to Training-free Video Object Segmentation via Streaming-data Mixture Models

Roy Uziel, Or Dinari, Oren Freifeld

https://openreview.net/forum?id=jfsjKBDB1z

Keywords: Unsupervised, video segmentation, clustering

Compressor summary: The paper presents a training-free video object segmentation method that uses pre-trained deep features and classical clustering, achieving state-of-the-art results with low memory and fast inference.

MKOR: Momentum-Enabled Kronecker-Factor-Based Optimizer Using Rank-1 Updates

Mohammad Mozaffari, Sikan Li, Zhao Zhang, Maryam Mehri Dehnavi

https://openreview.net/forum?id=jcnvDO96N5

Keywords: machine learning, deep learning, optimizers, distributed training;second-order optimization;

Compressor summary: MKOR is a new optimizer that improves training speed and convergence of DNNs by reducing complexity and increasing second-order updates frequency.

Interaction Measures, Partition Lattices and Kernel Tests for High-Order Interactions

Zhaolu Liu, Robert Peach, Pedro A. M. Mediano, Mauricio Barahona

https://openreview.net/forum?id=jcRB6xHdJ2

Keywords: High-order interactions; Lattice theory; Kernel tests

Compressor summary: The text introduces a method to measure and test interactions between groups of more than two variables in complex systems using kernel-based tests and lattice theory connections.

Generator Born from Classifier

Runpeng Yu, Xinchao Wang

https://openreview.net/forum?id=jcJVgIFY2r

Keywords: Generative Model

Compressor summary: The paper presents a new method to reconstruct an image generator using a pre-trained classifier, by training the generator to satisfy convergence conditions based on maximum-margin bias theory.

Design from Policies: Conservative Test-Time Adaptation for Offline Policy Optimization

Jinxin Liu, Hongyin Zhang, Zifeng Zhuang, Yachen Kang, Donglin Wang, Bin Wang

https://openreview.net/forum?id=jZYf1GxH1V

Keywords: offline reinforcement learning, test-time adaptation

Compressor summary: The paper proposes a non-iterative bi-level offline RL method called DROP that learns an MBO score model and a behavior embedding for safe policy extraction during testing.

Moral Responsibility for AI Systems

Sander Beckers

https://openreview.net/forum?id=jYIknUIgkd

Keywords: responsibility, causation, causal models

Compressor summary: The paper defines moral responsibility for AI systems using causal models, compares its approach to existing ones, and extends the concept to a degree of responsibility.

Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift

Florian Seligmann, Philipp Becker, Michael Volpp, Gerhard Neumann

https://openreview.net/forum?id=jX49iKr6vb

Keywords: bayesian deep learning, distribution shift, calibration

Compressor summary: The paper surveys recent BDL methods on realistic tasks, focusing on how they generalize and calibrate under distribution shift. It explores different aspects of BDL, such as signed error, fine-tuning, and ensembles, and compares various algorithms in terms of accuracy and calibration.

RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths

Zeyue Xue, Guanglu Song, Qiushan Guo, Boxiao Liu, Zhuofan Zong, Yu Liu, Ping Luo

https://openreview.net/forum?id=jUdZCcoOu3

Keywords: Diffusion Model, Text-to-Image Generation

Compressor summary: RAPHAEL is a text-conditional image diffusion model that generates highly artistic images by stacking tens of mixture-of-experts layers, outperforming other models in image quality and style switching, with a zero-shot FID score of 6.61 on the COCO dataset.

SPRING: Studying Papers and Reasoning to play Games

Yue Wu, So Yeon Min, Shrimai Prabhumoye, Yonatan Bisk, Ruslan Salakhutdinov, Amos Azaria, Tom Mitchell, Yuanzhi Li

https://openreview.net/forum?id=jU9qiRMDtR

Keywords: Games, Instruction Manual, Crafter, Open-world games, Large Language Models, Language Models, Zero-shot, In-context prompting

Compressor summary: SPRING is a novel approach that uses a large language model to reason and play open-world survival games like Crafter, outperforming state-of-the-art RL baselines without any training.

Spuriosity Rankings: Sorting Data to Measure and Mitigate Biases

Mazda Moayeri, Wenxiao Wang, Sahil Singla, Soheil Feizi

https://openreview.net/forum?id=jSuhnO9QJv

Keywords: spurious correlations, interpretability, bias, distributional robustness

Compressor summary: The paper proposes a simple and effective way to reduce model biases due to spurious features by ranking images based on how much they rely on such cues, using deep neural features of an interpretable network. The method works on any data and model without major modifications, and shows that class-wise biases are strongly related across models.

When are ensembles really effective?

Ryan Theisen, Hyunsuk Kim, Yaoqing Yang, Liam Hodgkinson, Michael W. Mahoney

https://openreview.net/forum?id=jS4DUGOtBD

Keywords: Ensembling, theory, deep learning

Compressor summary: The paper investigates when ensembling improves classification performance, finding that it depends on the disagreement rate and the average error rate of the models involved, with non-interpolating models benefiting more from ensembling than interpolating ones.

Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning

Guozheng Ma, Linrui Zhang, Haoyu Wang, Lu Li, Zilin Wang, Zhen Wang, Li Shen, Xueqian Wang, Dacheng Tao

https://openreview.net/forum?id=jRL6ErxMVB

Keywords: Data Augmentation, Visual Reinforcement Learning, Sample Efficiency

Compressor summary: This work investigates the attributes of data augmentation (DA) in visual reinforcement learning, introducing new DA operations and fusion schemes to improve sample efficiency.

Is Learning in Games Good for the Learners?

William Brown, Jon Schneider, Kiran Vodrahalli

https://openreview.net/forum?id=jR2FkqW6GB

Keywords: learning in games, correlated equilibria, Stackelberg equilibria, swap regret, dynamic regret

Compressor summary: This text discusses tradeoffs between reward and regret in repeated games with two agents using generalized equilibrium, and shows that different algorithms can lead to different outcomes depending on the opponent's algorithm choice.

IEBins: Iterative Elastic Bins for Monocular Depth Estimation

Shuwei Shao, Zhongcai Pei, Xingming Wu, Zhong Liu, Weihai Chen, Zhengguo Li

https://openreview.net/forum?id=jOuxQGRVoQ

Keywords: Monocular depth estimation, Iterative refinement, Deep learning

Compressor summary: The paper introduces a novel iterative elastic bins method for monocular depth estimation, which progressively optimizes the search range and utilizes temporal context modeling with GRU-based architecture to achieve state-of-the-art results.

Fast Partitioned Learned Bloom Filter

Atsuki Sato, Yusuke Matsui

https://openreview.net/forum?id=jL2eJxPK88

Keywords: optimization, data structures, algorithms, theory, learned algorithms

Compressor summary: This paper presents two methods to construct a learned Bloom filter faster and with similar memory efficiency, while one method can achieve the same data structure as the original.

CluB: Cluster Meets BEV for LiDAR-Based 3D Object Detection

Yingjie Wang, Jiajun Deng, Yuenan Hou, Yao Li, Yu Zhang, Jianmin Ji, Wanli Ouyang, Yanyong Zhang

https://openreview.net/forum?id=jIhX7SpfCz

Keywords: 3D object detection ; Point clouds

Compressor summary: The paper proposes CluB, a 3D object detection framework that combines BEV-based and cluster-based detectors by enriching object representation at feature and query levels.

Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobians

Rainer Engelken

https://openreview.net/forum?id=jEQRoJzDx8

Keywords: exploding/vanishing gradients, Lyapunov exponents, Lyapunov spectrum, chaos, RNN, condition number, Jacobian

Compressor summary: Gradient flossing is a new method to stabilize recurrent neural networks by regularizing Lyapunov exponents during learning, leading to improved convergence speed and success rate for tasks with long time horizons.

The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation

Saurabh Saxena, Charles Herrmann, Junhwa Hur, Abhishek Kar, Mohammad Norouzi, Deqing Sun, David J. Fleet

https://openreview.net/forum?id=jDIlzSU8wJ

Keywords: Monocular depth, optical flow, diffusion, depth, flow

Compressor summary: Denoising diffusion probabilistic models can estimate optical flow and monocular depth effectively without specialized architectures, providing uncertainty and multimodality information.

Learning Repeatable Speech Embeddings Using An Intra-class Correlation Regularizer

Jianwei Zhang, Suren Jayasuriya, Visar Berisha

https://openreview.net/forum?id=jCPRG3FuHV

Keywords: repeatability, embeddings, metric learning, intra-class correlation, intra-class variance

Compressor summary: The authors propose a method to measure and improve the repeatability of embeddings for machine learning tasks using intra-class correlation coefficients (ICC) and show its effectiveness on speech tasks.

Hierarchical Adaptive Value Estimation for Multi-modal Visual Reinforcement Learning

Yangru Huang, Peixi Peng, Yifan Zhao, Haoran Xu, Mengyue Geng, Yonghong Tian

https://openreview.net/forum?id=jB4wsc1DQW

Keywords: vision-based reinforcement learning, multi-modal, event camera

Compressor summary: The paper proposes a local value estimation framework for vision-based reinforcement learning that adapts to different modalities and improves performance in autonomous driving.

Jailbroken: How Does LLM Safety Training Fail?

Alexander Wei, Nika Haghtalab, Jacob Steinhardt

https://openreview.net/forum?id=jA235JGM09

Keywords: red teaming, safety, RLHF, large language models

Compressor summary: The text discusses how large language models trained for safety still face adversarial misuse, investigates why this happens, and evaluates new attacks on state-of-the-art models like GPT-4 and Claude v1.3.

On Masked Pre-training and the Marginal Likelihood

Pablo Moreno-Muñoz, Pol G. Recasens, Søren Hauberg

https://openreview.net/forum?id=j9wGUcS30B

Keywords: Marginal likelihood, masked pre-training, Bayesian inference

Compressor summary: The paper explains how masked pre-training works by maximizing a model's generalization measure and explores its potential for Bayesian models.

Learning from Both Structural and Textual Knowledge for Inductive Knowledge Graph Completion

Kunxun Qi, Jianfeng Du, Hai Wan

https://openreview.net/forum?id=j7x9wW3tCf

Keywords: Knowledge graph completion; Neural approximate rule learning; Neural rule-based system

Compressor summary: The paper proposes a two-stage framework for knowledge graph completion using both structural and textual knowledge, which improves performance over existing rule-based systems.

DynPoint: Dynamic Neural Point For View Synthesis

Kaichen Zhou, Jia-Xing Zhong, Sangyun Shin, Kai Lu, Yiyuan Yang, Andrew Markham, Niki Trigoni

https://openreview.net/forum?id=j7U4pFkCYB

Keywords: View Synthesis, Monocular Video

Compressor summary: DynPoint is an algorithm that accelerates view synthesis for unconstrained monocular videos by predicting 3D correspondence between frames and using hierarchical neural point clouds.

Scaling Data-Constrained Language Models

Niklas Muennighoff, Alexander M Rush, Boaz Barak, Teven Le Scao, Nouamane Tazi, Aleksandra Piktus, Sampo Pyysalo, Thomas Wolf, Colin Raffel

https://openreview.net/forum?id=j5BuTrEj35

Keywords: large language models, scaling laws, data engineering

Compressor summary: The paper explores how to scale language models when there is limited text data and proposes a scaling law for compute optimality that takes into account repeated tokens and excess parameters.

What You See is What You Read? Improving Text-Image Alignment Evaluation

Michal Yarom, Yonatan Bitton, Soravit Changpinyo, Roee Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, Idan Szpektor

https://openreview.net/forum?id=j5AoleAIru

Keywords: Vision-and-language, Image-text alignment, Text-to-image generation, Image-to-text generation, Multi-modal models, Synthetic images, Meta-evaluation, Visual-question-answering

Compressor summary: The authors propose two methods for automatic evaluation of text-image alignment and introduce SeeTRUE, a dataset with human judgements for semantically aligned text-image pairs.

Resolving the Tug-of-War: A Separation of Communication and Learning in Federated Learning

Junyi Li, Heng Huang

https://openreview.net/forum?id=j4QVhftpYM

Keywords: Federated Learning

Compressor summary: FedSep is a novel two-layer federated learning framework that separates communication and learning layers for better privacy and efficiency, and outperforms existing methods in communication-efficient and heterogeneous-model FL tasks.

Active Vision Reinforcement Learning under Limited Visual Observability

Jinghuan Shang, Michael S Ryoo

https://openreview.net/forum?id=j2oYaFpbrB

Keywords: Reinforcement Learning, Active Reinforcement Learning, Visual Reinforcement Learning, Active Vision, Active Perception, Partial Observability, Sensorimotor

Compressor summary: The paper introduces SUGARL, a framework that learns motor and sensory policies together using an intrinsic sensorimotor reward, which enables agents to control their visual observations in partially observable environments.

Versatile Energy-Based Probabilistic Models for High Energy Physics

Taoli Cheng, Aaron Courville

https://openreview.net/forum?id=j0U6XJubbP

Keywords: Generative modeling, Energy-based models, Out-of-distribution detection, Sciences, Application, Physics

Compressor summary: The authors propose a flexible energy-based probabilistic model for High Energy Physics events that can generate simulations, detect anomalies, and identify particles.

Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimation

Wei-Ning Chen, Dan Song, Ayfer Ozgur, Peter Kairouz

https://openreview.net/forum?id=izNfcaHJk0

Keywords: Differential Privacy, Federated Learning, Communication

Compressor summary: The paper studies how to achieve the best balance of privacy, communication, and accuracy in federated learning and analytics, and proposes two methods that use compression and random data selection to improve this trade-off.

Near-Optimal Algorithms for Gaussians with Huber Contamination: Mean Estimation and Linear Regression

Ilias Diakonikolas, Daniel Kane, Ankit Pensia, Thanasis Pittas

https://openreview.net/forum?id=iyweRIXAeH

Keywords: robust statistics, high-dimensional inference, regression, nearly linear time algorithms

Compressor summary: The paper presents novel algorithms for Gaussian mean estimation and linear regression with Huber contamination, achieving near-optimal sample and time complexity with optimal error guarantees.

GPEX, A Framework For Interpreting Artificial Neural Networks

Amir Akbarnejad, Gilbert Bigras, Nilanjan Ray

https://openreview.net/forum?id=iy4Of0w8ML

Keywords: Gaussian processes, Explainable AI

Compressor summary: The authors propose a method that uses Gaussian processes to match and explain the outputs of deep neural networks without strict assumptions, and demonstrate its effectiveness on various datasets using computational techniques for scalability and GPU acceleration.

Non-adversarial training of Neural SDEs with signature kernel scores

Zacharia Issa, Blanka Horvath, Maud Lemercier, Cristopher Salvi

https://openreview.net/forum?id=ixcsBZw5pl

Keywords: Neural SDEs, score-based generative models, signature kernels, time series

Compressor summary: This paper proposes a new way to train Neural SDEs non-adversarially using kernel scores, which improves stability, avoids mode collapse, and allows for generating spatiotemporal data.

Open Visual Knowledge Extraction via Relation-Oriented Multimodality Model Prompting

Hejie Cui, Xinyu Fang, Zihan Zhang, Ran Xu, Xuan Kan, Xin Liu, Yue Yu, Manling Li, Yangqiu Song, Carl Yang

https://openreview.net/forum?id=ixVAXsdtJO

Keywords: Visual Knowledge Extraction, Multimodality, Large Model Prompting

Compressor summary: OpenVik is a new method for extracting open visual knowledge from images using an open relational region detector and a multimodal model, with enhanced data techniques to diversify the results.

Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models

Andy Zhou, Jindong Wang, Yu-Xiong Wang, Haohan Wang

https://openreview.net/forum?id=iwp3H8uSeK

Keywords: robustness, knowledge distillation, adversarial training, data augmentation, generalization

Compressor summary: The paper introduces a framework that improves vision models' robustness by combining knowledge distillation and data augmentation using a robust teacher and Discrete Adversarial Distillation (DAD).

Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models

Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, Mingyuan Zhou

https://openreview.net/forum?id=iv2sTQtbst

Keywords: diffusion models, training efficiency, data efficiency

Compressor summary: Patch Diffusion is a faster and more efficient way to train diffusion models that uses patches with random sizes and coordinates, improving the quality of generated images even on small datasets.

Saddle-to-Saddle Dynamics in Diagonal Linear Networks

Scott Pesme, Nicolas Flammarion

https://openreview.net/forum?id=iuqCXg1Gng

Keywords: gradient flow, saddle-to-saddle, diagonal linear network, incremental learning

Compressor summary: The paper describes how gradient flow in two-layer linear networks for regression finds the minimum l1-norm solution by jumping between saddles using a recursive algorithm and arc-length time-reparametrization, with no assumptions on data or network size.

Improving Compositional Generalization using Iterated Learning and Simplicial Embeddings

Yi Ren, Samuel Lavoie, Mikhail Galkin, Danica J. Sutherland, Aaron Courville

https://openreview.net/forum?id=irRHgjePdR

Keywords: compositional generalization, systematic generalization, iterated learning, representation learning, graph neural networks

Compressor summary: The authors propose iterated learning on models with simplicial embeddings to improve deep networks' compositional generalization, inspired by human language development and Kolmogorov complexity analysis.

Punctuation-level Attack: Single-shot and Single Punctuation Can Fool Text Models

Wenqiang Wang, Chongyang Du, Tao Wang, Kaihao Zhang, Wenhan Luo, Lin Ma, Wei Liu, Xiaochun Cao

https://openreview.net/forum?id=ir6WWkFR80

Keywords: Punctuation-level Attack, Textual Adversarial attack, Natural Language Processing

Compressor summary: The paper introduces a new way of textual attack using single punctuation changes and proposes a search method to find the best position for the attack without exhaustive search.

Adaptive recurrent vision performs zero-shot computation scaling to unseen difficulty levels

Vijay Veerabadran, Srinivas Ravishankar, Yuan Tang, Ritik Raina, Virginia R. de Sa

https://openreview.net/forum?id=iqezE0EyXq

Keywords: cognitive science, recurrent neural networks, adaptive computation time, visual reasoning

Compressor summary: The study investigates how adaptive recurrent neural networks can help vision models solve difficult visual reasoning problems by adjusting their computational resources based on the input's needs.

Generalized Belief Transport

Junqi Wang, PEI WANG, Patrick Shafto

https://openreview.net/forum?id=iohoef1bfM

Keywords: Bayesian theory, Belief transport, Unbalanced optimal transport, parametrization, asymptotic behavior, environment drift detection

Compressor summary: The paragraph discusses a new mathematical framework called Generalized Belief Transport that unifies different learning models by considering them as points in a broader space of possibilities, and explores its properties and behavior.

History Filtering in Imperfect Information Games: Algorithms and Complexity

Christopher Solinas, Doug Rebstock, Nathan R. Sturtevant, Michael Buro

https://openreview.net/forum?id=inIONNg8Sq

Keywords: search, game theory, multi-agent, learning, markov chain monte carlo, complexity

Compressor summary: The authors analyze the computational aspects of filtering histories for subgame decomposition, a key technique for AI in imperfect information games, and introduce a novel Markov Chain Monte Carlo-based generation algorithm for trick-taking card games.

AIMS: All-Inclusive Multi-Level Segmentation for Anything

Lu Qi, Jason Kuen, Weidong Guo, Jiuxiang Gu, Zhe Lin, Bo Du, Yu Xu, Ming-Hsuan Yang

https://openreview.net/forum?id=ikkdTD3hQJ

Keywords: Image Segmentation

Compressor summary: The paper introduces AIMS, a new task for image segmentation that divides visual regions into three levels, and proposes a unified model to tackle challenges like annotation inconsistency and task correlation.

Semi-Supervised Contrastive Learning for Deep Regression with Ordinal Rankings from Spectral Seriation

Weihang Dai, Yao DU, Hanru Bai, Kwang-Ting Cheng, Xiaomeng Li

https://openreview.net/forum?id=ij3svnPLzG

Keywords: Semi-supervised learning, deep regression, contrastive learning

Compressor summary: The authors propose a new semi-supervised contrastive learning method for deep regression that uses unlabeled data and achieves better results than existing methods.

Frequency-domain MLPs are More Effective Learners in Time Series Forecasting

Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, Zhendong Niu

https://openreview.net/forum?id=iif9mGCTfy

Keywords: time series forecasting, multi-layer perceptrons, frequency domain

Compressor summary: The paper proposes FreTS, a time series forecasting method that uses frequency-domain MLPs to learn global dependencies and concentrate on key parts of frequency components, outperforming existing methods.

GNNEvaluator: Evaluating GNN Performance On Unseen Graphs Without Labels

Xin Zheng, Miao Zhang, Chunyang Chen, Soheila Molaei, Chuan Zhou, Shirui Pan

https://openreview.net/forum?id=ihlT8yvQ2I

Keywords: graph neural networks, GNN model evaluation, node classification accuracy

Compressor summary: The paper proposes a two-stage framework to evaluate graph neural network (GNN) models by estimating their performance on unseen graphs without labels using a DiscGraph set and a GNNEvaluator.

Maximum Independent Set: Self-Training through Dynamic Programming

Lorenzo Brusca, Lars C.P.M. Quaedvlieg, Stratis Skoulakis, Grigorios Chrysos, Volkan Cevher

https://openreview.net/forum?id=igE3Zbxvws

Keywords: Maximum Independent Set, Combinatorial Optimization, Graph Neural Networks, Dynamic Programming

Compressor summary: The paper introduces a GNN-based algorithm for solving the MIS problem using a DP-like recursive approach and shows its effectiveness on various datasets.

EvoPrompting: Language Models for Code-Level Neural Architecture Search

Angelica Chen, David Dohan, David So

https://openreview.net/forum?id=ifbF4WdT8f

Keywords: language models, evolution, prompting, neural architecture search, code generation

Compressor summary: The paper explores using language models as mutation and crossover operators for evolutionary neural architecture search (NAS), proposing EvoPrompting which combines prompt engineering with soft prompt-tuning, and shows its effectiveness in finding diverse and high performing models on various tasks.

Interpretable Prototype-based Graph Information Bottleneck

Sangwoo Seo, Sungwon Kim, Chanyoung Park

https://openreview.net/forum?id=icWwBKyVMs

Keywords: Graph neural network, Explainable AI, Interpretability

Compressor summary: The paper introduces interpretable Prototype-based Graph Information Bottleneck (PGIB), a novel framework for explainable GNNs that learns prototypes from key subgraphs important for predictions, improving both performance and interpretability.

DELIFFAS: Deformable Light Fields for Fast Avatar Synthesis

YoungJoong Kwon, Lingjie Liu, Henry Fuchs, Marc Habermann, Christian Theobalt

https://openreview.net/forum?id=iajxrSgOSX

Keywords: DELIFFAS: Avatar Modeling, Avatar Synthesis, Animatable Human, Light Fields, Human Performance Capture

Compressor summary: The DELIFFAS method creates realistic human avatars with fast inference speed by using a surface light field attached to a deformable mesh model, achieving state-of-the-art results in appearance synthesis and speed.

Similarity-based cooperative equilibrium

Caspar Oesterheld, Johannes Treutlein, Roger Baker Grosse, Vincent Conitzer, Jakob Nicolaus Foerster

https://openreview.net/forum?id=ia4AL3QnOv

Keywords: Program equilibrium, multi-agent learning, game theory, opponent shaping, superrationality, decision theory, cooperative AI, Newcomb's problem

Compressor summary: The paper proposes a realistic setting in which machine learning agents can achieve cooperative outcomes in social dilemmas by only observing a single number indicating their similarity, and shows that this can be learned using simple ML methods.

PCF-GAN: generating sequential data via the characteristic function of measures on the path space

Hang Lou, Siran Li, Hao Ni

https://openreview.net/forum?id=iWWLgcUTZU

Keywords: Generative adversarial networks, time series generation, rough path theory, Lie group

Compressor summary: The paragraph introduces PCF-GAN, a novel GAN that uses path characteristic functions to represent time series distributions and improve their generation and reconstruction quality.

Provable benefits of annealing for estimating normalizing constants: Importance Sampling, Noise-Contrastive Estimation, and beyond

Omar Chehab, Aapo Hyvarinen, Andrej Risteski

https://openreview.net/forum?id=iWGC0Nsq9i

Keywords: noise-contrastive estimation, annealed importance sampling

Compressor summary: The paragraph discusses different design choices for Monte Carlo methods that estimate the normalization constant using annealing and their impact on estimation error, showing that some choices are more efficient than others depending on the context.

On the Identifiability and Interpretability of Gaussian Process Models

Jiawen Chen, Wancen Mu, Yun Li, Didong Li

https://openreview.net/forum?id=iVYInarGXg

Keywords: Gaussian process, Identifiability, Interpretability, Mixture kernel, Separable kernel

Compressor summary: The paper analyzes additive and multiplicative mixtures of Mat\'ern kernels in single- and multi-output Gaussian process models, showing that the latter are more suitable for multi-output tasks due to better identifiability properties.

Adversarial Training from Mean Field Perspective

Soichiro Kumano, Hiroshi Kera, Toshihiko Yamasaki

https://openreview.net/forum?id=iT9MOAZqsb

Keywords: adversarial training; mean field theory

Compressor summary: The study analyzes adversarial training in random deep neural networks using a new mean field theory framework, showing its effects on network capacity, width, and dimensions.

A Trichotomy for Transductive Online Learning

Steve Hanneke, Shay Moran, Jonathan Shafer

https://openreview.net/forum?id=iSd8g75QvP

Keywords: Online Learning, Transductive Online Learning, Offline Learning, Mistake Bound

Compressor summary: The paper presents new bounds on learner mistakes in transductive online learning, showing a trichotomy of possible mistake rates depending on VC and Littlestone dimensions.

Uncertainty-Aware Alignment Network for Cross-Domain Video-Text Retrieval

Xiaoshuai Hao, Wanqian Zhang

https://openreview.net/forum?id=iQlK3VJxV7

Keywords: video-text retrieval; cross-domain;Unsupervised Domain Adaptation Video-text Retrieval;

Compressor summary: The paper presents UAN, a novel method to improve unsupervised domain adaptation video-text retrieval by addressing multimodal mutual information and uncertainty-aware alignment.

Learning To Dive In Branch And Bound

Max B. Paulus, Andreas Krause

https://openreview.net/forum?id=iPTF2hON1C

Keywords: Generative Modeling, Combinatorial Optimization, Mixed Integer Programming, Graph Neural Networks, Diving Heuristics

Compressor summary: L2Dive is a diving heuristic that uses graph neural networks to predict variable assignments and improve the performance of mixed integer linear programs on various combinatorial optimization problems.

Parallel Submodular Function Minimization

Deeparnab Chakrabarty, Andrei Graur, Haotian Jiang, Aaron Sidford

https://openreview.net/forum?id=iMfPFPMsZo

Keywords: parallel computation, convex optimization, submodular function minimization

Compressor summary: The paragraph discusses two new methods for parallel submodular function minimization with different trade-offs between depth and query complexity, and also introduces a highly-parallel algorithm for minimizing an $\ell_\infty$-Lipschitz function over the hypercube.

A Unified Model and Dimension for Interactive Estimation

Nataly Brukhim, Miroslav Dudík, Aldo Pacchiano, Robert E. Schapire

https://openreview.net/forum?id=iM0MWWBr4W

Keywords: Interactive learning, bandits, statistical queries

Compressor summary: The paper proposes a new interactive learning framework with a combinatorial measure called Dissimilarity dimension that captures learnability and unifies two classic models, with improved analyses.

Bicriteria Approximation Algorithms for the Submodular Cover Problem

Wenjing Chen, Victoria G. Crawford

https://openreview.net/forum?id=iKarSI2a73

Keywords: submodular, combinatorial optimization, approximation algorithms

Compressor summary: The paper presents scalable algorithms for solving submodular cover problems with different variants and applications in data summarization and graph cut.

Joint Prompt Optimization of Stacked LLMs using Variational Inference

Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux

https://openreview.net/forum?id=iImnbUVhok

Keywords: deep prompt optimization, llm, variational inference, graphical model, chaining

Compressor summary: The text describes how to optimize prompts for two-layer deep language networks (DLNs) using latent variable inference and shows their potential to match or surpass GPT-4's performance with smaller models.

Simplifying Neural Network Training Under Class Imbalance

Ravid Shwartz-Ziv, Micah Goldblum, Yucen Lily Li, C. Bayan Bruss, Andrew Gordon Wilson

https://openreview.net/forum?id=iGmDQn4CRj

Keywords: Class Imbalance, Hyperparameters, Long-Tailed Distributions

Compressor summary: The authors show that by adjusting some standard deep learning components, one can achieve excellent results on class-imbalanced datasets without using specialized techniques.

Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks

Feng Chen, Daniel Kunin, Atsushi Yamamura, Surya Ganguli

https://openreview.net/forum?id=iFxWrxDekd

Keywords: Implicit Bias, SGD Dynamics, Implicit regularization, Learning rate schedule, Stochastic Gradient Descent, Invariant set, Attractive saddle points, Stochastic collapse, Permutation invariance, Simplicity bias, Teacher-student

Compressor summary: The paper shows that stochastic gradient descent (SGD) often simplifies neural networks during training by converging to simpler subnetworks, improving generalization.

MultiMoDN—Multimodal, Multi-Task, Interpretable Modular Networks

Vinitra Swamy, Malika Satayeva, Jibril Frej, Thierry Bossy, Thijs Vogels, Martin Jaggi, Tanja Käser, Mary-Anne Hartley

https://openreview.net/forum?id=iB3Ew6z4WL

Keywords: Deep Learning, Multimodal Learning, Multi-task learning, Missingness, Interpretability

Compressor summary: MultiModN is a multimodal network that fuses latent representations in a sequence of any modality type, providing interpretable feedback on real-world tasks while being resistant to missing not-at-random data.

Sharp Bounds for Generalized Causal Sensitivity Analysis

Dennis Frauen, Valentyn Melnychuk, Stefan Feuerriegel

https://openreview.net/forum?id=iAcEmyhwk2

Keywords: Causal machine learning, treatment effect estimation, sensitivity analysis, unobserved confounding, uncertainty estimation

Compressor summary: The paper presents a general framework for causal sensitivity analysis under unobserved confounding, extending the marginal sensitivity model to various types of treatments and outcomes, and providing sharp bounds for different causal effects using observational data.

CAST: Cross-Attention in Space and Time for Video Action Recognition

Dongho Lee, Jongseo Lee, Jinwoo Choi

https://openreview.net/forum?id=iATY9W5Xw7

Keywords: action recognition, video understanding, cross-attention, balanced spatio-temporal understanding

Compressor summary: Recognizing human actions in videos requires spatial and temporal understanding. Most existing action recognition models lack a balanced spatio-temporal understanding of videos. In this work, we propose a novel two-stream architecture, called Cross-Attention in Space and Time (CAST), that achieves a balanced spatio-temporal understanding of videos using only RGB input. Our proposed bottleneck cross-attention mechanism enables the spatial and temporal expert models to exchange information and make synergistic predictions, leading to improved performance. We validate the proposed method with extensive experiments on public benchmarks with different characteristics: EPIC-Kitchens-100, Something-Something-V2, and Kinetics-400. Our method consistently shows favorable performance across these datasets, while the performance of existing methods fluctuates depending on the dataset characteristics. The code is available at https://github.com/KHU-VLL/CAST.

Geodesic Multi-Modal Mixup for Robust Fine-Tuning

Changdae Oh, Junhyuk So, Hoyoon Byun, YongTaek Lim, Minchul Shin, Jong-June Jeon, Kyungwoo Song

https://openreview.net/forum?id=iAAXq60Bw1

Keywords: multi-modal learning, robustness, fine-tuning, contrastive learning, CLIP, Mixup

Compressor summary: The authors study CLIP's multi-modal embeddings and propose a fine-tuning method to improve alignment and uniformity for better transferability and robustness.

Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activity

Zijiao Chen, Jiaxin Qing, Juan Helen Zhou

https://openreview.net/forum?id=i913TUOvTK

Keywords: Video Reconstruction from Brain Activities, Diffusion Model, Contrastive Learning

Compressor summary: The paragraph describes a new method called Mind-Video that can reconstruct high-quality videos from brain activities using various techniques and outperforms previous methods in accuracy and interpretability.

ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer

Haoran You, Huihong Shi, Yipin Guo, Yingyan Celine Lin

https://openreview.net/forum?id=i6mMWNcTfu

Keywords: Efficient Vision Transformer; Multiplication-reduced networks; Hardware acceleration

Compressor summary: The authors propose a new multiplication-reduced model called ShiftAddViT, which uses bitwise shifts and additions instead of dense multiplications to speed up ViTs on GPUs without losing much accuracy.

Decentralized Matrix Sensing: Statistical Guarantees and Fast Convergence

Marie Maros, Gesualdo Scutari

https://openreview.net/forum?id=i5sSWKbF3b

Keywords: Distributed non-convex optimization, Low-rank matrix recovery

Compressor summary: The paper studies how agents in a network can estimate a low-rank matrix using a decentralized gradient algorithm and shows that it converges to the correct solution with good performance guarantees.

Segment Any Point Cloud Sequences by Distilling Vision Foundation Models