ICML 2024 compressed

This page contains one-sentence summaries of ICML 2024 accepted papers, generated by the compressor, my personal LLM-based project.

A Graph is Worth $K$ Words: Euclideanizing Graph using Pure Transformer

https://openreview.net/forum?id=zxxSJAVQPc

Compressor summary: GraphsGPT uses a Graph2Seq encoder to transform Non-Euclidean graphs into learnable Graph Words, and a GraphGPT decoder to reconstruct the original graph, achieving state-of-the-art results in graph representation learning and generation.

Understanding Server-Assisted Federated Learning in the Presence of Incomplete Client Participation

https://openreview.net/forum?id=zwUEk9WpsR

Compressor summary: The text discusses how server-assisted federated learning (SA-FL) can address incomplete client participation and provide theoretical and practical benefits over conventional FL.

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

https://openreview.net/forum?id=ztn8FCR1td

Compressor summary: The paper presents a new architecture (Mamba-2) that combines state-space models and attention, improving the speed and performance of language modeling.

Rethinking Independent Cross-Entropy Loss For Graph-Structured Data

https://openreview.net/forum?id=zrQIc9mQQN

Compressor summary: The paper proposes a new method to improve graph neural networks for node classification by modeling the joint distribution of nodes and their clusters, enhancing accuracy and robustness against adversarial attacks.

Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks

https://openreview.net/forum?id=znz261CQK7

Compressor summary: Key points: - The paper studies random and SGD-optimized neural networks with zero training error - Overparameterization in terms of width helps generalization due to SGD bias - Overparameterization in terms of depth harms generalization but random and SGD-optimized networks behave similarly, suggesting an architectural bias Summary: The paper investigates how optimization and architecture affect generalization of neural networks with zero training error, finding that overparameterization in width is good for generalization because of SGD bias, while overparameterization in depth is bad for generalization but random and SGD-optimized networks are similar, indicating an architectural bias.

S3GCL: Spectral, Swift, Spatial Graph Contrastive Learning

https://openreview.net/forum?id=znKAWRZSF9

Compressor summary: S3GCL is a new graph representation learning method that addresses challenges of homophily assumptions and scalability by using cosine-parameterized Chebyshev polynomials as filters and an MLP encoder with positive pairs for context awareness.

Trustworthy Actionable Perturbations

https://openreview.net/forum?id=zkjGpZrIX3

Compressor summary: The text introduces Trustworthy Actionable Perturbations (TAP), a framework to modify inputs that change true class probabilities instead of fooling classifiers, with verification, cost, reward, and goal definitions for real-world applications.

Don’t Label Twice: Quantity Beats Quality when Comparing Binary Classifiers on a Budget

https://openreview.net/forum?id=zkcya47Sq5

Compressor summary: To compare two binary classifiers, spend the budget on collecting a single noisy label for more samples rather than aggregating multiple labels via majority vote, as this increases accuracy and provides better sample size bounds.

Flexible Residual Binarization for Image Super-Resolution

https://openreview.net/forum?id=zji9DLksTz

Compressor summary: FRB is a method to improve binary image super-resolution by using second-order residual binarization and distillation-guided binarization training, achieving better results than existing methods.

Improving Factuality and Reasoning in Language Models through Multiagent Debate

https://openreview.net/forum?id=zj7YuTE4t8

Compressor summary: The paper proposes a method where multiple language models debate and improve each other's responses, enhancing reasoning and validity of generated content.

Conditionally-Conjugate Gaussian Process Factor Analysis for Spike Count Data via Data Augmentation

https://openreview.net/forum?id=zgiT3uxvCF

Compressor summary: The paper proposes ccGPFA, a conditionally-conjugate method for modeling neural activity from spike count data that enables tractable inference using variational EM and sparse Gaussian Processes.

Individualized Privacy Accounting via Subsampling with Applications in Combinatorial Optimization

https://openreview.net/forum?id=zfmwAaB9Nw

Compressor summary: The paper proposes a new technique to improve privacy analysis and algorithms for combinatorial optimization problems and heavy hitter identification in data streams.

TIC-TAC: A Framework For Improved Covariance Estimation In Deep Heteroscedastic Regression

https://openreview.net/forum?id=zdNTiTs5gU

Compressor summary: Deep heteroscedastic regression improves covariance estimation using gradient and curvature information, and introduces a metric to evaluate accuracy without supervision.

ODIN: Disentangled Reward Mitigates Hacking in RLHF

https://openreview.net/forum?id=zcIV8OQFVF

Compressor summary: The paper investigates reward hacking in Reinforcement Learning from Human Feedback using Long Short-Term Memories (LLMs) and proposes an improved evaluation protocol and reward model to address this issue.

Differentially Private Sum-Product Networks

https://openreview.net/forum?id=zc3bAEI5lp

Compressor summary: The paper proposes a novel method using sum-product networks to create a single model that performs both privacy-preserving classification and data generation, outperforming existing approaches in stability and utility.

Contrastive Predict-and-Search for Mixed Integer Linear Programs

https://openreview.net/forum?id=zatLnLvbs8

Compressor summary: ConPaS is a new machine learning framework that predicts and fixes integer variable assignments to solve MILPs faster and better than existing methods.

A Distributional Analogue to the Successor Representation

https://openreview.net/forum?id=zajsXCxMgW

Compressor summary: The paper introduces a new method for distributional reinforcement learning that separates transition structure and reward, using a distributional successor measure that describes the consequences of behavior, which can be learned from data and enables zero-shot risk-sensitive policy evaluation.

Handling Heterogeneous Curvatures in Bandit LQR Control

https://openreview.net/forum?id=zWIS8I9G9B

Compressor summary: The paper studies online LQR control with heterogeneous cost curvatures and provides a novel analysis using Newton decrement to improve adaptivity and performance.

DSD-DA: Distillation-based Source Debiasing for Domain Adaptive Object Detection

https://openreview.net/forum?id=zS8zUuAU8T

Compressor summary: The paper proposes a new framework that uses distillation and target-relevant information to improve domain adaptive object detection, addressing source bias and achieving better classification and localization in both domains.

Homomorphism Counts for Graph Neural Networks: All About That Basis

https://openreview.net/forum?id=zRrzSLwNHQ

Compressor summary: The paper proposes a new method for graph neural networks that counts all structures in a pattern basis, improving their expressive power and counting abilities without increasing complexity.

Agent Instructs Large Language Models to be General Zero-Shot Reasoners

https://openreview.net/forum?id=zMwFvxr6CV

Compressor summary: The paper proposes an agent that generates instructions to improve the zero-shot reasoning abilities of large language models on various tasks and datasets, achieving state-of-the-art performance.

Deep Networks Always Grok and Here is Why

https://openreview.net/forum?id=zMue490KMr

Compressor summary: Grokking is a widespread phenomenon in deep neural networks where generalization occurs long after zero training error, and it leads to delayed robustness against adversarial examples due to the linearization of the network mapping.

Impact of Decentralized Learning on Player Utilities in Stackelberg Games

https://openreview.net/forum?id=zMsMQJraEj

Compressor summary: The paper studies how two learning agents interact over time, shows that existing benchmarks don't capture their dynamics well, and proposes new algorithms and environments to improve their performance.

A Rate-Distortion View of Uncertainty Quantification

https://openreview.net/forum?id=zMGUDsPopK

Compressor summary: The paper proposes a new method called Distance Aware Bottleneck that enhances deep neural networks with uncertainty estimation and out-of-distribution detection by learning a codebook of input representations.

GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting

https://openreview.net/forum?id=zL9q2JD1dC

Compressor summary: GALA3D is a framework that uses language models to generate realistic 3D scenes with controllable editing and consistent properties.

Transitional Uncertainty with Layered Intermediate Predictions

https://openreview.net/forum?id=zII3Olw7cr

Compressor summary: The paper presents TULIP, a method to improve single-pass uncertainty estimation in neural networks by preserving feature distances at intermediate layers.

Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samples

https://openreview.net/forum?id=zFHaB7KESM

Compressor summary: The paper proposes a method to learn general nonlinear representations from multiple data sources with different distributions and dependencies, providing statistical guarantees and requiring fewer samples than existing methods.

SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding

https://openreview.net/forum?id=zEqeNEuiJr

Compressor summary: The paper proposes signSGD-FD, a distributed learning optimizer that uses federated defense to handle both honest and adversarial workers, ensuring fast convergence even under attacks.

On a Neural Implementation of Brenier's Polar Factorization

https://openreview.net/forum?id=zDCwJQY3eI

Compressor summary: The paper proposes a practical implementation of the polar factorization theorem for vector fields, using neural networks and optimal transport to parameterize the components, and explores its applications in machine learning.

Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning

https://openreview.net/forum?id=zCmMkWK4Ly

Compressor summary: The paper proposes ICES, a method to encourage multi-agent exploration in sparse reward environments by assessing each agent's contribution and using global transition information during training.

Overcoming the Optimizer's Curse: Obtaining Realistic Prescriptions from Neural Networks

https://openreview.net/forum?id=zB6VQzDmK8

Compressor summary: The paper proposes a method to obtain realistic decisions from ReLU networks by modeling the data manifold as an optimization constraint and using adaptive sampling to solve the problem efficiently.

What is Dataset Distillation Learning?

https://openreview.net/forum?id=z8sYc334fU

Compressor summary: The study examines the behavior, representativeness, and point-wise information content of dataset distillation, finding that it retains high task performance by compressing early training dynamics and contains meaningful semantic information.

Characterizing ResNet's Universal Approximation Capability

https://openreview.net/forum?id=z7zHsNFXHc

Compressor summary: ResNet can efficiently approximate functions such as polynomials and smooth ones with fewer tunable weights than ReLU networks, achieving optimal approximation rates.

DITTO: Diffusion Inference-Time T-Optimization for Music Generation

https://openreview.net/forum?id=z5Ux2u6t7U

Compressor summary: DITTO is a framework to control text-to-music diffusion models without fine-tuning by optimizing initial noise latents and achieving high quality, flexible, and efficient music generation.

Dynamic Metric Embedding into lp Space

https://openreview.net/forum?id=z3PUNzdmGs

Compressor summary: The paper presents a new algorithm for maintaining a randomized mapping of a weighted graph to an $\ell_p$ space while preserving the distance between vertices, with low expected distortion and fast update time, even in dynamic settings where edge weights change over time.

Demystifying SGD with Doubly Stochastic Gradients

https://openreview.net/forum?id=z373OXJXWU

Compressor summary: This paper analyzes the convergence properties of doubly stochastic gradient descent (doubly SGD), a popular optimization strategy for problems with intractable expectations, and shows that random reshuffling can improve its performance.

Triple Changes Estimator for Targeted Policies

https://openreview.net/forum?id=yzNEkTmcoF

Compressor summary: The paper proposes and tests a new estimator called 'triple changes' that allows for a more nuanced assessment of causal effects by incorporating information from potential outcomes, compared to existing methods like difference-in-differences or triple differences.

By Tying Embeddings You Are Assuming the Distributional Hypothesis

https://openreview.net/forum?id=yyYMAprcAR

Compressor summary: Tied input-output embeddings in language models relate to Harris' distributional hypothesis and contextual similarity of words.

Stabilizing Policy Gradients for Stochastic Differential Equations via Consistency with Perturbation Process

https://openreview.net/forum?id=ytz2naZoDB

Compressor summary: Key points: - The paper proposes a framework to train SDEs with policy gradients for generating samples with high rewards - The framework constrains the SDE to be consistent with its associated perturbation process, which covers the entire space and is easy to sample - The method is applied to structure-based drug design and improves the binding affinity of ligand molecules Summary: The paper presents a policy gradient framework for training SDEs that are compatible with their perturbation processes, enabling effective and efficient generation of high-reward samples. The method is used for structure-based drug design and achieves the best score on a dataset.

Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning

https://openreview.net/forum?id=yrFUJzcTsk

Compressor summary: HesScale is a fast and stable approximation of Hessian diagonals that outperforms existing methods in small networks for second-order optimization and reinforcement learning problems.

Scaling Laws for Fine-Grained Mixture of Experts

https://openreview.net/forum?id=yoqdlynCRs

Compressor summary: This paper studies how to adjust the size of experts in Mixture of Experts (MoE) models for efficient language processing, introducing a new hyperparameter called granularity and showing how it affects scaling laws.

Rethinking Guidance Information to Utilize Unlabeled Samples: A Label Encoding Perspective

https://openreview.net/forum?id=yoTCwNqQS6

Compressor summary: The paper proposes a new method called Label-Encoding Risk Minimization (LERM) that uses unlabeled samples to improve the learning of labeled samples and balance prediction discriminability and diversity in scenarios with insufficient labels.

Position: AI/ML Influencers Have a Place in the Academic Process

https://openreview.net/forum?id=yo9Jyt3XCY

Compressor summary: The paper examines how social media influencers boost the visibility and citations of AI and ML papers, and suggests they should promote diversity in their content.

Estimating Barycenters of Distributions with Neural Optimal Transport

https://openreview.net/forum?id=ymgcTqrZLT

Compressor summary: The paper proposes a new method to find an average distribution from multiple probability measures using neural networks and optimal transport, with theoretical guarantees and applications to image data.

Incentivized Learning in Principal-Agent Bandit Games

https://openreview.net/forum?id=ykgZk6vFrh

Compressor summary: This paper studies a repeated game where a principal tries to learn how to incentivize an agent to act in the principal's best interest in various settings, motivated by applications like healthcare and ecology.

Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical

https://openreview.net/forum?id=ykZYLBcA9g

Compressor summary: The paper proposes a new consistent method for complementary-label learning that does not rely on the uniform distribution assumption or ordinary-label training sets, and shows its effectiveness in experiments.

Bootstrap AutoEncoders With Contrastive Paradigm for Self-supervised Gaze Estimation

https://openreview.net/forum?id=ykRY34kL3j

Compressor summary: The paper proposes a new method called BeCa, which combines generative and contrastive methods for self-supervised full-face gaze estimation and improves performance over existing approaches on various datasets.

Amortizing Pragmatic Program Synthesis with Rankings

https://openreview.net/forum?id=yj8h567Ia7

Compressor summary: The paper proposes a method to speed up pragmatic program synthesis by using a global ranking derived from partial rankings of programs generated by an exact RSA synthesizer.

Provably Robust DPO: Aligning Language Models with Noisy Feedback

https://openreview.net/forum?id=yhpDKSw7yA

Compressor summary: This paper proposes a framework for policy optimization with random preference flips, introduces a novel loss function to debias the effect of noise, and proves its sub-optimality bound in theory.

Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving

https://openreview.net/forum?id=yh6Y7ppf46

Compressor summary: The paper proposes a novel method to combine machine learning and legacy numerical codes without modification for faster scientific computing.

Learning Shadow Variable Representation for Treatment Effect Estimation under Collider Bias

https://openreview.net/forum?id=ycXo4tQIpN

Compressor summary: The paper proposes a method to learn shadow variables from observational data for estimating treatment effects in the presence of collider bias, using hypothesis testing and a novel treatment effect estimator.

Better Safe than Sorry: Pre-training CLIP against Targeted Data Poisoning and Backdoor Attacks

https://openreview.net/forum?id=ycLHJuLYuD

Compressor summary: SAFECLIP is a defense method for pre-training CLIP models against targeted data poisoning and backdoor attacks by using unimodal contrastive learning and dividing data into safe and risky sets.

Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints

https://openreview.net/forum?id=yb5xV8LFDq

Compressor summary: The paper proposes a new method for selecting small coresets that maintain model performance while minimizing the coreset size in deep learning algorithms, and provides theoretical and empirical results to support its effectiveness.

CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networks

https://openreview.net/forum?id=yY6N89IlHa

Compressor summary: The paper introduces CLIF, a new neuron model for spiking neural networks that improves accuracy by facilitating the backpropagation of temporal gradients, and shows its superior performance over other models and conventional ANNs on various datasets.

Towards Generalization beyond Pointwise Learning: A Unified Information-theoretic Perspective

https://openreview.net/forum?id=yXlQL9goY8

Compressor summary: The paper develops new information-theoretic methods to analyze and compare different types of contrastive learning algorithms, including pointwise, pairwise, triplet, quadruplet, and higher-order scenarios.

COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability

https://openreview.net/forum?id=yUxdk32TU6

Compressor summary: The paper presents a novel framework for generating diverse and controllable adversarial attacks on large language models using an efficient text generation algorithm.

Adapting Static Fairness to Sequential Decision-Making: Bias Mitigation Strategies towards Equal Long-term Benefit Rate

https://openreview.net/forum?id=yUPBkPKzHw

Compressor summary: ELBERT is a long-term fairness concept for machine learning models in sequential decision-making that reduces biases and maintains high utility.

Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecture

https://openreview.net/forum?id=yTz0u4B8ug

Compressor summary: Memoria is a novel artificial neural network memory system inspired by human memory that outperforms conventional techniques in various tasks.

Performance Bounds for Active Binary Testing with Information Maximization

https://openreview.net/forum?id=yTXv8KDD1P

Compressor summary: The paper proposes tight non-vacuous bounds on the greedy heuristic InfoMax for binary test prediction, assuming that the conditional probability of a test being 'true' is within $\delta$ units of one-half, and analyzes two scenarios with modest values of $\delta$.

${\rm E}(3)$-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning

https://openreview.net/forum?id=yShA4VPYZB

Compressor summary: This paper explores Euclidean symmetries in cooperative multi-agent reinforcement learning, designing neural networks with symmetric constraints that improve performance and generalization in various applications.

High-Dimensional Geometric Streaming for Nearly Low Rank Data

https://openreview.net/forum?id=yQfA0etfB7

Compressor summary: The text studies algorithms for approximating subspaces of points in the $\ell_p$ norm, giving fast and strong coreset constructions that can be applied to various geometric problems.

ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections

https://openreview.net/forum?id=yPDTXQwUPy

Compressor summary: The paper introduces *ETHER*, a new method to fine-tune foundation models with fewer parameters, less performance deterioration, and hyperparameter robustness.

RODEO: Robust Outlier Detection via Exposing Adaptive Out-of-Distribution Samples

https://openreview.net/forum?id=yOe5lqDPvM

Compressor summary: RODEO is a data-centric approach that uses text-to-image models to generate diverse and near-distribution outliers for robust outlier detection in adversarial settings.

Towards Efficient Spiking Transformer: a Token Sparsification Framework for Training and Inference Acceleration

https://openreview.net/forum?id=yL6hljtjW4

Compressor summary: The paper proposes a method called STATA that improves the efficiency of training Spiking Transformers by sparsifying tokens, identifying important ones across timesteps, and aligning attention maps.

Performative Prediction with Bandit Feedback: Learning through Reparameterization

https://openreview.net/forum?id=yHs3jIPgaF

Compressor summary: The paper proposes a framework to study social prediction without assuming convexity, known mapping, or first-order information, by reparameterizing the objective function and optimizing it iteratively.

Provable Benefits of Local Steps in Heterogeneous Federated Learning for Neural Networks: A Feature Learning Perspective

https://openreview.net/forum?id=yHRxnhKyEJ

Compressor summary: This paper investigates the effect of local steps in Federated Learning on feature learning and generalization, and shows that they help improve performance and reduce communication costs.

Domain Generalisation via Imprecise Learning

https://openreview.net/forum?id=yFUdZfbEme

Compressor summary: The Imprecise Domain Generalisation framework helps machine learners deal with uncertainty in out-of-distribution generalisation by allowing them to optimise against a range of strategies during training and enabling operators to specify their preferences at deployment time.

Variational Partial Group Convolutions for Input-Aware Partial Equivariance of Rotations and Color-Shifts

https://openreview.net/forum?id=yDXnXJE1RK

Compressor summary: The paper proposes VP G-CNNs, a novel approach to capture varying levels of partial equivariance for different data instances using variational inference and redesigned distributions.

On the Implicit Bias of Adam

https://openreview.net/forum?id=y8YovS0lOg

Compressor summary: The paper investigates how different optimization algorithms like RMSProp and Adam regularize solutions during training by analyzing their corresponding ordinary differential equations and their dependence on hyperparameters.

Neural Collapse in Multi-label Learning with Pick-all-label Loss

https://openreview.net/forum?id=y8NevOhrnW

Compressor summary: The paper investigates neural collapse in multi-label classification and shows how it affects feature representation and optimizer behavior, leading to improved performance.

Roping in Uncertainty: Robustness and Regularization in Markov Games

https://openreview.net/forum?id=y6y2HauOpR

Compressor summary: The paper studies robust Markov games with rectangular uncertainty, shows a connection between robust Nash equilibrium and regularized Markov games, provides a planning algorithm and provable guarantees, and identifies a special class of games that can be solved efficiently for reward-uncertain two-player zero-sum cases.

Evolution-Inspired Loss Functions for Protein Representation Learning

https://openreview.net/forum?id=y5L8W0KRUX

Compressor summary: Evolutionary Ranking (EvoRank) is a new training objective for AI-based protein engineering frameworks that uses evolutionary information from multiple sequence alignments to improve mutation effect predictions and learn diverse protein representations.

Neural Diffusion Models

https://openreview.net/forum?id=xzX7kf486K

Compressor summary: Neural Diffusion Models generalize conventional diffusion models by enabling non-linear transformations of data, improving generative tasks performance and sample quality.

Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations

https://openreview.net/forum?id=xye7iNsgXn

Compressor summary: Key points: - Large-scale recommendation systems need to handle high cardinality, heterogeneous features and billions of user actions - DLRMs fail to scale with compute, while Transformers succeed in language and vision domains - HSTU is a new architecture for streaming recommendation data that outperforms baselines and is faster than FlashAttention2-based Transformers - Generative Recommenders with HSTU improve metrics in online A/B tests and reduce carbon footprint Summary: The authors propose HSTU, a new Transformer-based architecture for recommendation systems that handles high features, scales well with compute, and improves both performance and sustainability.

Exploring Intrinsic Dimension for Vision-Language Model Pruning

https://openreview.net/forum?id=xxL7CEWuxz

Compressor summary: The paper explores how intrinsic dimension can be used as a metric to measure and improve the prunability of vision-language models by modality, finding that visual representations are more crucial while language representations are more robust.

Convergence Guarantees for the DeepWalk Embedding on Block Models

https://openreview.net/forum?id=xwxUbBHC1q

Compressor summary: The paper analyzes the convergence properties and theoretical guarantees of DeepWalk algorithm on graphs generated by Stochastic Block Model, a simple model to study algorithm behavior on large graphs.

Learning Graph Representation via Graph Entropy Maximization

https://openreview.net/forum?id=xwOENWCo46

Compressor summary: The paper proposes a method to learn diverse graph representations by approximating graph entropy using orthonormal neural networks, which improve performance in unsupervised and semi-supervised learning tasks.

Drug Discovery with Dynamic Goal-aware Fragments

https://openreview.net/forum?id=xuX2rDSSco

Compressor summary: GEAM is a molecular generative framework for drug discovery that considers target chemical properties and updates its fragment vocabulary dynamically during the generation process.

Memory Efficient Neural Processes via Constant Memory Attention Block

https://openreview.net/forum?id=xtwCf7iAs2

Compressor summary: The paper introduces CMANPs, a variant of Neural Processes that uses constant memory and attention blocks to model predictive uncertainty efficiently.

Position: Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination

https://openreview.net/forum?id=xtKWwB6lzT

Compressor summary: The paper critiques offline reinforcement learning in dynamic treatment regimes, citing concerns about evaluation metrics, baselines, and RL formulations; it also presents a case study showing RL performance variations and the possibility of random baselines outperforming RL algorithms.

Membership Inference Attacks on Diffusion Models via Quantile Regression

https://openreview.net/forum?id=xqqccG7gf1

Compressor summary: The text describes a privacy vulnerability of diffusion models in image synthesis and proposes an improved membership inference attack that uses quantile regression and bootstrapping to detect if a given example belongs to the training data or not with higher accuracy and lower computational cost than previous methods.

Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models

https://openreview.net/forum?id=xpSlt67vxQ

Compressor summary: R-Bench is a benchmark to evaluate vision relationship hallucinations in large language models by testing their understanding of visual relationships and content.

RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis

https://openreview.net/forum?id=xnQ1qoly7Q

Compressor summary: The paper introduces RoboCodeX, a framework for generating detailed robotic actions from high-level human instructions using tree-structured multimodal code generation and pre-training with a specialized dataset.

Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape

https://openreview.net/forum?id=xm2lU7tteQ

Compressor summary: The paper explores how Transformer models with a nonlinear representation layer can learn effectively in context, and proves that their loss landscape is benign and stable under certain conditions.

The WMDP Benchmark: Measuring and Reducing Malicious Use with Unlearning

https://openreview.net/forum?id=xlr6AUDuJz

Compressor summary: The authors introduce the WMDP benchmark to measure hazardous knowledge in LLMs and propose RMU, an unlearning method that reduces malicious use while preserving general capabilities.

InstructSpeech: Following Speech Editing Instructions via Large Language Models

https://openreview.net/forum?id=xlWcdtCyOC

Compressor summary: This paper presents InstructSpeech, a multi-task language model that can edit speech based on natural language instructions, achieving state-of-the-art results in eleven tasks.

An Analysis of Linear Time Series Forecasting Models

https://openreview.net/forum?id=xl82CcbYaT

Compressor summary: The paper shows that simple linear models and their variants perform well in time series forecasting and can be interpreted as unconstrained linear regression over expanded features, leading to better forecasts.

Improved Differentially Private and Lazy Online Convex Optimization: Lower Regret without Smoothness Requirements

https://openreview.net/forum?id=xl2yU3dsHK

Compressor summary: The paper presents new differentially private algorithms for online convex optimization that work well with non-smooth and high-dimensional loss functions, using sampling from log-concave densities and rejection sampling.

Langevin Policy for Safe Reinforcement Learning

https://openreview.net/forum?id=xgoilgLPGD

Compressor summary: The paper introduces Langevin Actor-Critic (LAC), a method that combines sampling-based Langevin policy with optimization-based actor-critic to enable safe reinforcement learning in complex tasks.

S3O: A Dual-Phase Approach for Reconstructing Dynamic Shape and Skeleton of Articulated Objects from Single Monocular Video

https://openreview.net/forum?id=xcyKKACmSd

Compressor summary: S3O is a novel method that learns parametric models for shape and motion from monocular videos without requiring additional annotations or extensive computational resources, improving 3D reconstruction accuracy and robustness.

CHAI: Clustered Head Attention for Efficient LLM Inference

https://openreview.net/forum?id=xcDRx8vzCa

Compressor summary: CHAI reduces memory and compute requirements for large language models by clustering correlated attention heads at runtime.

Positive and Unlabeled Learning with Controlled Probability Boundary Fence

https://openreview.net/forum?id=xbQqhojHTg

Compressor summary: PULCPBF is a two-stage method for PU learning that uses weak classifiers as a probability boundary fence and self-training to improve performance.

Robust Classification via a Single Diffusion Model

https://openreview.net/forum?id=xaSpuvNYwS

Compressor summary: Key points: - Diffusion models can improve adversarial robustness but have limitations - Robust Diffusion Classifier (RDC) is a generative classifier based on a pre-trained diffusion model - RDC uses multi-head diffusion and efficient sampling strategies to reduce computational cost - RDC outperforms adversarial training models against various adaptive attacks on CIFAR-10 Summary: The paper proposes Robust Diffusion Classifier, a generative classifier that leverages a pre-trained diffusion model to achieve better adversarial robustness than existing methods.

Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection

https://openreview.net/forum?id=xZO7SmM12y

Compressor summary: The paper proposes EOE, a method that uses large language models to generate outlier class labels and a score function based on potential outlier penalty for zero-shot OOD detection in open-world scenarios.

How Private are DP-SGD Implementations?

https://openreview.net/forum?id=xWI0MKwJSS

Compressor summary: Shuffling and Poisson subsampling are two different batch sampling methods for private machine learning that have significantly different privacy guarantees, and practitioners should be careful when reporting the privacy analysis of private SGD.

Model-based Reinforcement Learning for Parameterized Action Spaces

https://openreview.net/forum?id=xW79geE0RA

Compressor summary: DLPA is a new RL algorithm for PAMDPs that learns a dynamics model, plans with a modified control method, and outperforms existing methods in sample efficiency and performance.

A Dense Reward View on Aligning Text-to-Image Diffusion with Preference

https://openreview.net/forum?id=xVXnXk9I3I

Compressor summary: The paper proposes a new method for aligning text-to-image diffusion models with user preferences by using dense rewards and temporal discounting, which improves the efficiency and effectiveness of preference alignment.

Out-of-Domain Generalization in Dynamical Systems Reconstruction

https://openreview.net/forum?id=xTYIAD2NND

Compressor summary: The paper proposes a formal framework to analyze generalization in dynamical systems reconstruction from time series data, showing that black-box deep learning techniques often fail to achieve it and suggesting ways to improve it.

CF-OPT: Counterfactual Explanations for Structured Prediction

https://openreview.net/forum?id=xSkIxKdO08

Compressor summary: Key points: - Optimization layers in deep neural networks improve structured learning but lack interpretability - Counterfactual explanations can make them more transparent by providing alternative outcomes - Variational autoencodters enable counterfactuals in latent space with plausibility - CF-OPT is a first-order optimization algorithm for finding counterfactual explanations in structured learning Summary: The paper proposes a novel algorithm (CF-OPT) that uses variational autoencoders to find interpretable counterfactual explanations for optimization layers in deep neural networks.

Translating Subgraphs to Nodes Makes Simple GNNs Strong and Efficient for Subgraph Representation Learning

https://openreview.net/forum?id=xSizvCoI79

Compressor summary: The paper proposes Subgraph-To-Node translation, a method for learning subgraph representations that reduces memory and computational costs and performs better than existing graph neural networks.

Achieving Margin Maximization Exponentially Fast via Progressive Norm Rescaling

https://openreview.net/forum?id=xS2YKQlBIZ

Compressor summary: PRGD is a novel algorithm that maximizes the margin at an exponential rate for linearly separable data, while existing algorithms like GD and NGD fail under certain conditions and only achieve a polynomial rate.

Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making

https://openreview.net/forum?id=xQiYCmDrjp

Compressor summary: The paper proposes a novel temporal distance for stochastic settings that satisfies the triangle inequality, enabling better planning and control in reinforcement learning.

FrameQuant: Flexible Low-Bit Quantization for Transformers

https://openreview.net/forum?id=xPypr0kufs

Compressor summary: The paper proposes using Fusion Frames to quantize Transformers to two bits, achieving efficient and accurate models with known recovery guarantees and denoising filters.

A New Branch-and-Bound Pruning Framework for $\ell_0$-Regularized Problems

https://openreview.net/forum?id=xPmSNLle1w

Compressor summary: The paper proposes an alternative pruning test method for $\ell_0$-regularized problems that improves the solving time of Branch-and-Bound algorithms in machine-learning applications.

Critical feature learning in deep neural networks

https://openreview.net/forum?id=xMJT4XW468

Compressor summary: The paper develops a theory of network kernels in deep neural networks that explains how their Bayesian prior adapts to data and features using large deviation and field-theoretic approaches.

Do Efficient Transformers Really Save Computation?

https://openreview.net/forum?id=xLikRS9OhW

Compressor summary: The paper investigates the reasoning capabilities of Sparse Transformer and Linear Transformer, finding that they are expressive enough for Dynamic Programming problems but require more model size than standard Transformer, and identifies a class of problems where they are more efficient.

Graph Neural Stochastic Diffusion for Estimating Uncertainty in Node Classification

https://openreview.net/forum?id=xJUhgvM2u8

Compressor summary: Graph neural stochastic diffusion (GNSD) is a new method for estimating uncertainty in graph neural network predictions by connecting them to stochastic partial differential equations with a $Q$-Wiener process and two networks to ensure accurate prediction and uncertainty propagation.

LLM-Empowered State Representation for Reinforcement Learning

https://openreview.net/forum?id=xJMZbdiQnf

Compressor summary: The proposed method, LESR, uses a large language model to generate task-related state representation codes for reinforcement learning, improving sample efficiency and performance in Mujoco and Gym-Robotics tasks.

Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning

https://openreview.net/forum?id=xIRKB5nRJl

Compressor summary: The authors propose a framework to train robots using multimodal prompts, combining vision and language signals, and achieve state-of-the-art performance on robot manipulation tasks.

Dynamic Survival Analysis with Controlled Latent States

https://openreview.net/forum?id=xGlVkBSDdt

Compressor summary: The authors propose a new method to learn intensity functions of counting processes using neural controlled differential equations and signature-based estimation, with theoretical guarantees and applications in various domains.

EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism

https://openreview.net/forum?id=xFk0w9zoV3

Compressor summary: EE-LLM is a framework that trains and uses large language models with early exiting to speed up inference while maintaining quality.

CRoFT: Robust Fine-Tuning with Concurrent Optimization for OOD Generalization and Open-Set OOD Detection

https://openreview.net/forum?id=xFDJBzPhci

Compressor summary: The paper proposes a new objective function to improve out-of-distribution generalization and detection in vision-language pre-trained models during fine-tuning, by minimizing the gradient magnitude of energy scores on training data.

Ai-sampler: Adversarial Learning of Markov kernels with involutive maps

https://openreview.net/forum?id=xFCA2yWVs4

Compressor summary: The paper introduces a method to train Markov chains with neural networks for efficient and well-mixed sampling, using involutive Metropolis-Hastings kernels from reversible neural networks for detailed balance.

Online Algorithms with Uncertainty-Quantified Predictions

https://openreview.net/forum?id=xF656w37Mj

Compressor summary: The paper explores using uncertainty quantification (UQ) predictions in online algorithms for ski rental and online search problems, and proposes a new online learning framework to leverage UQ effectively.

Position: Application-Driven Innovation in Machine Learning

https://openreview.net/forum?id=xEB2oF3vvb

Compressor summary: The paper advocates for application-driven research in machine learning as a way to create impactful solutions and foster innovation, while suggesting reforms in the current reviewing, hiring, and teaching practices.

Simultaneous identification of models and parameters of scientific simulators

https://openreview.net/forum?id=xC7SYAZygF

Compressor summary: SBMI is a method that uses neural networks and simulations to systematically select model components in composite scientific models, enabling data-driven discovery and uncertainty-informed decision making.

RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning

https://openreview.net/forum?id=xB6YJZOKyT

Compressor summary: The paper proposes RVI-SAC, an off-policy DRL method using average reward criterion, which performs well on Mujoco locomotion tasks.

FedBAT: Communication-Efficient Federated Learning via Learnable Binarization

https://openreview.net/forum?id=x2zxPwCkAZ

Compressor summary: FedBAT is a novel framework for federated learning that learns binary model updates during local training, reducing approximation errors and improving accuracy.

Improved Communication-Privacy Trade-offs in $L_2$ Mean Estimation under Streaming Differential Privacy

https://openreview.net/forum?id=x1G7ieRgRd

Compressor summary: The paper proposes a novel privacy accounting method and sparsification scheme for mean estimation in central differential privacy that improves MSEs, communication efficiency, and compatibility with streaming differential privacy and DP-FTRL optimizers.

HarmonyDream: Task Harmonization Inside World Models

https://openreview.net/forum?id=x0yIaw2fgk

Compressor summary: The paper investigates how observation and reward modeling tasks in world models affect sample-efficient MBRL and proposes HarmonyDream, a method that balances these two tasks to improve performance.

Randomized Confidence Bounds for Stochastic Partial Monitoring

https://openreview.net/forum?id=x0vLj1S6Wg

Compressor summary: The paper introduces new partial monitoring strategies using randomized confidence bounds and extends regret guarantees to stochastic settings with side information.

A Computational Framework for Solving Wasserstein Lagrangian Flows

https://openreview.net/forum?id=wwItuHdus6

Compressor summary: The text proposes a deep learning framework that simplifies solving various optimal transport problems without needing optimal couplings or simulating dynamics.

Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning

https://openreview.net/forum?id=wuQ2DRPAuy

Compressor summary: Robust-HDP improves federated learning utility and convergence speed by efficiently estimating and reducing differential privacy noise across heterogeneous clients.

TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling

https://openreview.net/forum?id=wrTzLoqbCg

Compressor summary: TimeSiam is a self-supervised pre-training framework for time series that leverages Siamese networks to capture temporal correlations and outperforms other methods in forecasting and classification tasks.

Attention Meets Post-hoc Interpretability: A Mathematical Perspective

https://openreview.net/forum?id=wnkC5T11Z9

Compressor summary: The paper explores how attention mechanisms in transformer models can offer insights on their behavior and compares them with post-hoc explanations.

Unsupervised Parameter-free Simplicial Representation Learning with Scattering Transforms

https://openreview.net/forum?id=wmljUnbjy6

Compressor summary: The paper introduces simplicial scattering networks (SSNs), a parameter-free model that extracts task-agnostic features from simplicial complex data without labels using random walk matrices, improving robustness and performance in various tasks.

Matrix Information Theory for Self-Supervised Learning

https://openreview.net/forum?id=wleAlsklEh

Compressor summary: Matrix-SSL is a novel method that leverages matrix information theory to improve non-contrastive learning methods by aligning covariance matrices and achieving better results than previous state-of-the-art methods on image and language tasks.

The Good, The Bad, and Why: Unveiling Emotions in Generative AI

https://openreview.net/forum?id=wlOaG9g0uq

Compressor summary: This paper proposes three approaches to understand and manipulate emotions in generative AI models using psychological theories, and shows that they can comprehend emotional stimuli like the human brain's dopamine mechanism.

Better Locally Private Sparse Estimation Given Multiple Samples Per User

https://openreview.net/forum?id=wlBtHP8KqS

Compressor summary: The paper proposes a user-level differential privacy method for sparse linear regression that eliminates the dimension dependency and performs better than item-level methods with the same number of samples.

Joint Composite Latent Space Bayesian Optimization

https://openreview.net/forum?id=wkCUmO7oi2

Compressor summary: JoCo is a novel framework that combines neural network encoders and probabilistic models to compress high-dimensional input and output spaces for efficient black-box optimization.

InterLUDE: Interactions between Labeled and Unlabeled Data to Enhance Semi-Supervised Learning

https://openreview.net/forum?id=wilej5VnqL

Compressor summary: InterLUDE is a new SSL method that improves image classification by interpolating labeled and unlabeled embeddings and minimizing discrepancies in predictions between them.

On The Fairness Impacts of Hardware Selection in Machine Learning

https://openreview.net/forum?id=weixEb6Wjd

Compressor summary: The paper examines how hardware choices can affect model performance and fairness in machine learning, highlighting the need for considering hardware in ML-as-a-service platforms.

Incremental Topological Ordering and Cycle Detection with Predictions

https://openreview.net/forum?id=wea7nsJdMc

Compressor summary: The paper proposes a data structure for dynamic graph problems using algorithms-with-predictions to achieve consistency, robustness, and smoothness in running time, and shows empirical results on real datasets.

Perfect Alignment May be Poisonous to Graph Contrastive Learning

https://openreview.net/forum?id=wdezvnc9EG

Compressor summary: This paper investigates how different augmentations in graph contrastive learning affect downstream performance by separating classes rather than aligning nodes of the same class, and proposes two methods to verify the findings.

Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models

https://openreview.net/forum?id=wdTiuvd0fR

Compressor summary: The text discusses how large pretrained protein language models can improve protein prediction tasks but shows that their performance does not always scale with pretraining time and suggests a need for better pretraining methods.

Improving Equivariant Graph Neural Networks on Large Geometric Graphs via Virtual Nodes Learning

https://openreview.net/forum?id=wWdkNkUY8k

Compressor summary: FastEGNN is an improved equivariant GNN model for large geometric graphs that uses virtual nodes to approximate the graph and achieve a balance between accuracy and efficiency.

An Interpretable Evaluation of Entropy-based Novelty of Generative Models

https://openreview.net/forum?id=wUgTnf918v

Compressor summary: The paper proposes a spectral method to measure novelty in multi-modal generative models by comparing their sample types with a reference model using the Kernel-based Entropic Novelty score.

Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptions

https://openreview.net/forum?id=wTd7dogTsB

Compressor summary: The paper analyzes how well score-based diffusion models can approximate complex distributions, and shows that they can achieve near-optimal performance under mild assumptions.

What’s the score? Automated Denoising Score Matching for Nonlinear Diffusions

https://openreview.net/forum?id=wLoESsgZIq

Compressor summary: The paper introduces a method called local-DSM that allows training generative models with non-Gaussian priors using nonlinear diffusion processes and applies it to image generation and statistical physics problems.

Understanding Heterophily for Graph Neural Networks

https://openreview.net/forum?id=wK9RvVmi7u

Compressor summary: The paper proposes a general random graph model to analyze heterophily patterns in Graph Neural Networks (GNNs) and shows how different factors affect separability in GNNs.

CompeteAI: Understanding the Competition Dynamics of Large Language Model-based Agents

https://openreview.net/forum?id=wGtzp4ZT1n

Compressor summary: Key points: - The paper explores competition dynamics in LLM-based agents using GPT-4 to simulate a virtual town with restaurant and customer agents - Competition encourages transformation, such as new operating strategies for restaurants - Simulation experiments reveal interesting findings that align with existing market and sociological theories Summary: The paper uses GPT-4 to create a competitive environment for LLM-based agents in a virtual town and studies how competition leads to transformation and insight into society.

A Unified Framework for Learning with Nonlinear Model Classes from Arbitrary Linear Samples

https://openreview.net/forum?id=wG2SgnH6Zv

Compressor summary: The paper introduces a framework for learning unknown objects from training data using a given model class, with general applicability to various problems and learning guarantees based on the variation of the model class.

Position: A Call to Action for a Human-Centered AutoML Paradigm

https://openreview.net/forum?id=wELbEYgnmo

Compressor summary: The paper argues for a more human-centered approach to AutoML, focusing on user interaction and collaboration between humans and AutoML systems.

xT: Nested Tokenization for Larger Context in Large Images

https://openreview.net/forum?id=wDDprThYeT

Compressor summary: The paper introduces *xT*, a vision transformer framework that effectively aggregates global context with local details and can model large images without significant losses or memory growth.

InferCept: Efficient Intercept Support for Augmented Large Language Model Inference

https://openreview.net/forum?id=wDDGQabYPQ

Compressor summary: InferCept is a new framework for large language models that reduces resource waste and improves serving speed by efficiently handling interactions with external environments, tools, and agents.

Fast Adversarial Attacks on Language Models In One GPU Minute

https://openreview.net/forum?id=wCMNbdshcY

Compressor summary: Key points: - BEAST is a novel, fast, and interpretable adversarial attack for Language Models - BEAST can jailbreak aligned LMs quickly and with high success rates - BEAST can induce hallucinations in LM chatbots and generate privacy attacks Summary: BEAST is a new attack method that can break the security of Language Models, make them produce wrong or irrelevant outputs, and expose user information.

Position: Future Directions in the Theory of Graph Machine Learning

https://openreview.net/forum?id=wBr5ozDEKp

Compressor summary: The authors call for a better theoretical understanding of graph neural networks (GNNs), considering their expressive power, generalization, and optimization in graph machine learning.

Position: $C^*$-Algebraic Machine Learning $-$ Moving in a New Direction

https://openreview.net/forum?id=w9nxTXuaCc

Compressor summary: The paper proposes a new direction for machine learning research called $C^*$-algebraic ML, which unifies existing learning strategies and constructs a new framework using the mathematical concept of $C^*$-algebra.

Sample-Efficient Multiagent Reinforcement Learning with Reset Replay

https://openreview.net/forum?id=w8ei1o9U5y

Compressor summary: The paper introduces Multiagent Reinforcement Learning with Reset Replay (MARR), a method to improve sample efficiency of MARL in parallel environments using reset strategy and data augmentation.

Learning to Play Atari in a World of Tokens

https://openreview.net/forum?id=w8BnKGFIYN

Compressor summary: DART is a sample-efficient transformer-based method that uses discrete representations for world modeling and learning behavior, achieving high performance on Atari games without look-ahead search.

Kernel Semi-Implicit Variational Inference

https://openreview.net/forum?id=w5oUo0LhO1

Compressor summary: Kernel SIVI (KSIVI) is a method for Bayesian inference that uses kernel tricks to simplify the score matching objective, enabling faster convergence and easier optimization.

Recurrent Early Exits for Federated Learning with Heterogeneous Clients

https://openreview.net/forum?id=w4B42sxNq3

Compressor summary: ReeFL is a recurrent early exit approach for federated learning that fuses features from different sub-models into a single shared classifier, improving performance and privacy.

Position: Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback

https://openreview.net/forum?id=w1d9DOGymR

Compressor summary: The paper explores how social choice theory can help address ethical and safety issues in fine-tuning foundation models like GPT-4 using human feedback or principles.

PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning

https://openreview.net/forum?id=w1HdBXSJXn

Compressor summary: The text introduces Persona In-Context Learning (PICLe), a method to customize the behavior of large language models based on desired personality traits using Bayesian inference and likelihood ratio selection criterion.

FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Prediction

https://openreview.net/forum?id=vye4OgLaTy

Compressor summary: The paper presents a method called FlashST that improves traffic prediction by adapting pre-trained models to different scenarios and data distributions using a spatio-temporal prompt network and a distribution mapping mechanism.

On the Hardness of Probabilistic Neurosymbolic Learning

https://openreview.net/forum?id=vxPmrxKe0J

Compressor summary: Probabilistic neurosymbolic models combine neural networks and logical reasoning, and WeightME is an unbiased gradient estimator for them based on model sampling.

Information Flow in Self-Supervised Learning

https://openreview.net/forum?id=vxDjeeBnTu

Compressor summary: The paper analyzes two self-supervised learning methods using matrix mutual information and introduces M-MAE, a new method that improves upon existing ones for visual representation learning.

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective

https://openreview.net/forum?id=vuMD71R20q

Compressor summary: The paper investigates how removing the square root from adaptive gradient optimizers improves generalization on convolutional architectures, while maintaining performance on transformers, and discusses practical benefits for developing non-diagonal adaptive methods.

Efficient Contrastive Learning for Fast and Accurate Inference on Graphs

https://openreview.net/forum?id=vsy21Xodrt

Compressor summary: GraphECL is a fast and efficient contrastive learning method for graphs that uses an MLP to mimic the computations of a GNN, achieving superior performance and efficiency compared to existing methods.

What is the Long-Run Distribution of Stochastic Gradient Descent? A Large Deviations Analysis

https://openreview.net/forum?id=vsOF7qDNhl

Compressor summary: This paper studies how stochastic gradient descent behaves in non-convex problems and shows that it visits low-energy regions, like the global minimum, more often than high-energy ones, using a thermodynamics analogy.

Kernel Debiased Plug-in Estimation: Simultaneous, Automated Debiasing without Influence Functions for Many Target Parameters

https://openreview.net/forum?id=vq7ITv8a49

Compressor summary: Kernel debiased plug-in estimation is a novel method to estimate multiple target parameters in nonparametric models without relying on influence functions and while maintaining computational efficiency.

Two Heads are Actually Better than One: Towards Better Adversarial Robustness via Transduction and Rejection

https://openreview.net/forum?id=vn92qYjL1F

Compressor summary: The paper proposes a new transductive algorithm that improves robustness against adversarial attacks by applying a reduction technique to construct effective defenses with provable guarantees.

Neural Operators with Localized Integral and Differential Kernels

https://openreview.net/forum?id=vl9GB3fbht

Compressor summary: The authors propose a method for learning differential and integral operators with locally supported kernels to improve the performance of Fourier neural operators for solving partial differential equations.

Graph Automorphism Group Equivariant Neural Networks

https://openreview.net/forum?id=vjkq5fwsj3

Compressor summary: The paper proposes a method to construct neural networks that can learn from data on graphs with complex relations using their actual symmetry group, Aut(G), instead of the symmetric group S_n.

DiffDA: a Diffusion model for weather-scale Data Assimilation

https://openreview.net/forum?id=vhMq3eAB34

Compressor summary: DiffDA is a denoising diffusion model that uses GraphCast and predicted states to generate high-resolution atmospheric data from sparse observations, enabling weather forecasting and climate modeling applications.

Local vs. Global Interpretability: A Computational Complexity Perspective

https://openreview.net/forum?id=veEjiN2w9F

Compressor summary: The authors propose a framework using computational complexity theory to assess local and global perspectives of interpreting ML models, comparing linear models, decision trees, and neural networks.

Comparing Graph Transformers via Positional Encodings

https://openreview.net/forum?id=va3r3hSA6n

Compressor summary: The paper investigates the relationship between absolute and relative positional encodings for graph transformers, finding them equivalent in distinguishing non-isomorphic graphs, but with potential differences depending on node features.

Partial Optimality in the Linear Ordering Problem

https://openreview.net/forum?id=vYYIuJDTHq

Compressor summary: The paper presents partial solutions to the linear ordering problem by constructing maps and testing cost function conditions.

On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity Analysis

https://openreview.net/forum?id=vXUqOCsbj8

Compressor summary: We analyze the efficiency of modern Hopfield models based on pattern norms and show sub-quadratic variants exist under a computational complexity assumption, with examples and lower bounds provided.

One for All: A Universal Generator for Concept Unlearnability via Multi-Modal Alignment

https://openreview.net/forum?id=vSerUPYFtB

Compressor summary: The text proposes a universal perturbation generator that uses multi-modal pre-trained models to transform image data into text concepts and make it unlearnable, protecting personal information in free internet data.

Achieving Lossless Gradient Sparsification via Mapping to Alternative Space in Federated Learning

https://openreview.net/forum?id=vQmVmMN5ft

Compressor summary: The paper proposes a new space for gradient compression in federated learning, which improves compressibility and allows for higher accuracies with minimal information loss.

Particle Denoising Diffusion Sampler

https://openreview.net/forum?id=vMUnnS4OWC

Compressor summary: The paper proposes a new method to sample from unnormalized probability distributions using a particle-based denoising diffusion scheme with a novel score matching loss, which is more consistent than standard methods.

Stereographic Spherical Sliced Wasserstein Distances

https://openreview.net/forum?id=vLtVGtEz5h

Compressor summary: The paper proposes a fast and parallelizable distance for comparing spherical probability distributions using the stereographic projection and generalized Radon transform, called Stereographic Spherical Sliced Wasserstein (S3W) distance, and evaluates its speed and accuracy in various applications.

Chain of Code: Reasoning with a Language Model-Augmented Code Emulator

https://openreview.net/forum?id=vKtomqlSxm

Compressor summary: Chain of Code is an extension that improves language models' ability to reason by having them write and emulate code for semantic tasks.

Locality-Sensitive Hashing-Based Efficient Point Transformer with Applications in High-Energy Physics

https://openreview.net/forum?id=vJx6fld6l0

Compressor summary: The study presents HEPT, a novel efficient transformer model for large-scale point cloud processing in HEP and astrophysics, using local inductive bias and OR & AND-construction LSH for kernel approximation.

Semantic-Aware Human Object Interaction Image Generation

https://openreview.net/forum?id=vITl6CqIkk

Compressor summary: The paper proposes a framework to improve text-to-image generative models for human-object interaction by refining human pose and interaction boundary regions using guidance from pose quality and boundary information.

Simulation-Based Inference with Quantile Regression

https://openreview.net/forum?id=vGHOFeUQi8

Compressor summary: NQE is a new SBI method that uses conditional quantile regression and spline interpolation to estimate posterior samples and credible regions, with calibration options for limited data or model errors.

Gradient Compressed Sensing: A Query-Efficient Gradient Estimator for High-Dimensional Zeroth-Order Optimization

https://openreview.net/forum?id=vG7YpsJT74

Compressor summary: The paper proposes a new method, GraCe, for nonconvex ZOO in high dimensions, which uses fewer queries and achieves similar convergence rates as previous methods.

Interpreting Equivariant Representations

https://openreview.net/forum?id=vFk9fqXLst

Compressor summary: The paper highlights the importance of considering equivariant models' inductive bias when using latent representations and proposes principles for choosing invariant projections to improve performance in downstream tasks, as shown in molecular graph generation and image classification examples.

UGrid: An Efficient-And-Rigorous Neural Multigrid Solver for Linear PDEs

https://openreview.net/forum?id=vFATIZXlCm

Compressor summary: The paper presents a mathematically rigorous neural solver for linear PDEs that combines U-Net and MultiGrid principles, achieves high accuracy and generalization, and introduces a new residual loss metric for unsupervised training.

Lookbehind-SAM: k steps back, 1 step forward

https://openreview.net/forum?id=vCN5lwcWWE

Compressor summary: The paper proposes Lookbehind, a method that improves the loss-sharpness trade-off in SAM by using multiple ascent steps and linear interpolation, leading to better generalization, robustness, and lifelong learning performance.

Modelling Microbial Communities with Graph Neural Networks

https://openreview.net/forum?id=vBJZ93tvoE

Compressor summary: The authors use graph neural networks (GNNs) to model bacterial communities from their genomes, predicting relative abundance profiles without growth curves, and show generalization to unseen bacteria and different community structures.

Energy-Efficient Gaussian Processes Using Low-Precision Arithmetic

https://openreview.net/forum?id=v9tIJW1fzt

Compressor summary: The authors propose using low-precision floating-point representations to reduce energy consumption in Gaussian process regression, and show that well-conditioned kernel matrices allow for significant energy savings without compromising model performance.

Model Assessment and Selection under Temporal Distribution Shift

https://openreview.net/forum?id=v8MgLJ7kbL

Compressor summary: The paper proposes an adaptive rolling window method for model assessment and selection that works well with changing environments and historical data.

Tabular Insights, Visual Impacts: Transferring Expertise from Tables to Images

https://openreview.net/forum?id=v7I5FtL2pV

Compressor summary: Charms is a method to transfer relevant tabular knowledge to images, enhancing image classification and interpretability.

Balancing Similarity and Complementarity for Federated Learning

https://openreview.net/forum?id=v6tAdeCXKH

Compressor summary: The paper proposes FedSaC, a framework for Federated Learning that balances similarity and complementarity among clients to enhance model performance in heterogeneous and multimodal scenarios.

Adaptive Robust Learning using Latent Bernoulli Variables

https://openreview.net/forum?id=v6eaD7Wekw

Compressor summary: The paper proposes an adaptive method for learning from corrupted data, using latent variables and variational inference to infer the corruption level and improve performance on various machine learning tasks.

Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases

https://openreview.net/forum?id=v2o9rRJcEv

Compressor summary: The paper proposes a new method, TDPO-R, that aligns diffusion models with human preferences while minimizing reward overoptimization by exploiting the temporal inductive bias of diffusion models and addressing primacy bias from active neurons in the critic model.

TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision

https://openreview.net/forum?id=v1I4zRAjMb

Compressor summary: The paper presents TENG, a novel method for solving time-dependent PDEs using neural networks with high accuracy and efficiency.

Smoothness Adaptive Hypothesis Transfer Learning

https://openreview.net/forum?id=v0VUsQI5yw

Compressor summary: Smoothness Adaptive Transfer Learning (SATL) is a kernel ridge regression algorithm that adapts to varying and unknown smoothness in transfer learning tasks, achieving minimax optimality and optimal statistical rate.

Delaunay Graph: Addressing Over-Squashing and Over-Smoothing Using Delaunay Triangulation

https://openreview.net/forum?id=uyhjKoaIQa

Compressor summary: The paper proposes a new method to improve GNNs by constructing a graph from features using Delaunay Triangulation, which reduces over-squashing and oversmoothing problems.

RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback

https://openreview.net/forum?id=uydQ2W41KO

Compressor summary: RLAIF is an alternative to RLHF that uses preferences from an LLM and achieves comparable or better results than RLHF, overcoming scalability issues.

Understanding the Training Speedup from Sampling with Approximate Losses

https://openreview.net/forum?id=uun4fzaiat

Compressor summary: The paper proposes a greedy method for selecting samples with large approximate losses instead of exact losses to reduce selection overhead and training time, and evaluates it on BERT model training.

Flora: Low-Rank Adapters Are Secretly Gradient Compressors

https://openreview.net/forum?id=uubBZKM99Y

Compressor summary: Flora reduces memory usage for training large neural networks by using random projections and resampling matrices, without sacrificing performance.

Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game

https://openreview.net/forum?id=usUPvQH3XK

Compressor summary: The text proposes a framework that combines large language models with reinforcement learning to create strategic language agents that can overcome intrinsic bias and perform well in complex decision-making tasks like the Werewolf game.

Multi-Region Markovian Gaussian Process: An Efficient Method to Discover Directional Communications Across Multiple Brain Regions

https://openreview.net/forum?id=us6zMORsMe

Compressor summary: The study develops a novel method (MRM-GP) that combines Linear Dynamical System and Gaussian Process techniques to analyze complex interactions between different brain regions, revealing communication patterns and separating them by frequency bands.

Amortized Equation Discovery in Hybrid Dynamical Systems

https://openreview.net/forum?id=uqWfZ23O9g

Compressor summary: AMORE is a new method for learning laws in hybrid dynamical systems that jointly discovers equations and categorizes modes, outperforming existing two-stage approaches.

Towards Compositionality in Concept Learning

https://openreview.net/forum?id=upO8FUwf92

Compressor summary: Compositional Concept Extraction (CCE) is a new method to find high-level concepts in foundation models that are useful for understanding them, and it outperforms existing methods on various datasets and tasks.

Fast and Sample Efficient Multi-Task Representation Learning in Stochastic Contextual Bandits

https://openreview.net/forum?id=uog14iBFLA

Compressor summary: The paper proposes a new algorithm for multi-task learning in linear contextual bandits that improves efficiency by using representation learning and provides theoretical guarantees on sample and iteration complexity.

DRED: Zero-Shot Transfer in Reinforcement Learning via Data-Regularised Environment Design

https://openreview.net/forum?id=uku9r6RROl

Compressor summary: This paper explores how level sampling affects generalization in deep RL agents, proposes data-regularised environment design to improve it, and provides theoretical justification for prioritizing levels based on value loss.

Stability-Informed Initialization of Neural Ordinary Differential Equations

https://openreview.net/forum?id=uiqbnV4msl

Compressor summary: The paper explores how different aspects of neural ODE training impact performance and introduces a new initialization technique based on stability analysis.

Two Tales of Single-Phase Contrastive Hebbian Learning

https://openreview.net/forum?id=ui8ewXg1hV

Compressor summary: The paper proposes and analyzes dual propagation, a local learning algorithm that mimics gradients by using oppositely nudged compartments in artificial neurons, and shows its relation to adversarial robustness and a stable adjoint state method.

Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference

https://openreview.net/forum?id=uhHDhVKFMW

Compressor summary: The paper proposes LESS, a cache method that integrates a constant-sized cache with eviction-based methods to improve memory efficiency and retain information in large language models.

On Prompt-Driven Safeguarding for Large Language Models

https://openreview.net/forum?id=ugxGpOEkox

Compressor summary: The authors study how safety prompts affect large language models' behavior and propose a method to optimize them for better safeguarding against harmful queries.

Towards Scalable and Versatile Weight Space Learning

https://openreview.net/forum?id=ug2uoAZ9c2

Compressor summary: SANE is a method to learn task-agnostic representations of neural networks that can embed larger models into a learned space and generate unseen models sequentially.

Sparse Dimensionality Reduction Revisited

https://openreview.net/forum?id=ufgVvFmUom

Compressor summary: The text introduces a sparse Johnson-Lindenstrauss transform that can embed points into fewer dimensions while preserving distances, and improves upon previous sparsity results for certain cases.

Isometric Representation Learning for Disentangled Latent Space of Diffusion Models

https://openreview.net/forum?id=ufCptn28vG

Compressor summary: Isometric Diffusion is a technique that improves diffusion models by learning a disentangled latent space, enabling smoother interpolation, more accurate inversion, and better control over attributes.

Learning Scale-Aware Spatio-temporal Implicit Representation for Event-based Motion Deblurring

https://openreview.net/forum?id=udFZhUgtkI

Compressor summary: Key points: - SASNet is a novel method to restore blurred images with event streams at arbitrary scales - It uses spatial and temporal correlation features from events to generalize at continuous scales - It has two modules: SIRM for local areas and TIRM for global motion blur - A new H2D dataset is introduced to evaluate the method Summary: SASNet restores blurred images with event streams using spatial and temporal correlation features from events, and has two modules for different scales of blur. It is tested on a new dataset.

Distinguishing the Knowable from the Unknowable with Language Models

https://openreview.net/forum?id=ud4GSrqUKI

Compressor summary: The paper explores how to identify and measure the uncertainty in large language models' outputs using small probes and unsupervised methods.

Integrated Hardware Architecture and Device Placement Search

https://openreview.net/forum?id=ucl3B05EsX

Compressor summary: PHAZE optimizes deep learning training by co-optimizing architecture and device placement using novel algorithms, achieving higher throughput than TPUv4 and Spotlight on large language models.

Fast White-Box Adversarial Streaming Without a Random Oracle

https://openreview.net/forum?id=uaExqhJ2Ag

Compressor summary: This paper presents a near-optimal solution for sparse recovery and other tasks in white-box adversarial streaming models without needing a random oracle, using homomorphic encryption schemes.

Language Models with Conformal Factuality Guarantees

https://openreview.net/forum?id=uYISs2tpwP

Compressor summary: The text proposes a framework called conformal factuality that ensures high probability correctness guarantees for language models by connecting language modeling and conformal prediction, which involves making LM outputs less specific to expand uncertainty sets.

Graph-based Forecasting with Missing Data through Spatiotemporal Downsampling

https://openreview.net/forum?id=uYIFQOtb58

Compressor summary: The paper proposes a method for spatiotemporal forecasting that handles missing data by coarsening time series and combining them with attention.

Slicing Mutual Information Generalization Bounds for Neural Networks

https://openreview.net/forum?id=uWNUTRgBso

Compressor summary: The text introduces new generalization bounds for machine learning algorithms using slicing methods, which improve performance and enable control over compressibility in high-dimensional problems.

A Dynamic Algorithm for Weighted Submodular Cover Problem

https://openreview.net/forum?id=uUeXaKLE1I

Compressor summary: The paper studies the submodular cover problem with updates to the set, proposing a randomized algorithm that approximates the optimal solution and minimizes queries per update.

GPTSwarm: Language Agents as Optimizable Graphs

https://openreview.net/forum?id=uTC9AFXIhg

Compressor summary: The authors propose a unified framework for developing, integrating, and improving LLM-based agents using computational graphs and automatic graph optimizers.

Bidirectional Reciprocative Information Communication for Few-Shot Semantic Segmentation

https://openreview.net/forum?id=uRz9GZN17X

Compressor summary: The paper proposes a method for few-shot semantic segmentation that uses feedback and rectification layers to address the impact of intra-class diversity and improve the model's performance.

Random matrix theory improved Fréchet mean of symmetric positive definite matrices

https://openreview.net/forum?id=uQiFsBil3p

Compressor summary: The authors propose a random matrix theory method for estimating Fréchet means on symmetric positive definite matrices in machine learning tasks, which performs better than existing methods with low sample support and many matrices to average.

An LLM Compiler for Parallel Function Calling

https://openreview.net/forum?id=uQ2FUoFjnF

Compressor summary: LLMCompiler is a tool that improves the efficiency, cost, and accuracy of large language models by enabling parallel function calling for complex tasks.

Conformal prediction for multi-dimensional time series by ellipsoidal sets

https://openreview.net/forum?id=uN39Tt9P8b

Compressor summary: The paper proposes a new sequential method for predicting regions of multivariate time series with good coverage and small regions.

Scaling Beyond the GPU Memory Limit for Large Mixture-of-Experts Model Training

https://openreview.net/forum?id=uLpyWQPyF9

Compressor summary: ES-MoE is a new method that improves the efficiency of MoE training by balancing token loads and using host memory, achieving better scalability and throughput than existing frameworks.

Statistical Test for Attention Maps in Vision Transformers

https://openreview.net/forum?id=uLonuOfrwp

Compressor summary: The study proposes a statistical test for ViT's attention mechanisms in computer vision tasks, enabling reliable quantitative evidence for decision-making with controlled error rates, applied to brain image diagnosis.

Less is More: on the Over-Globalizing Problem in Graph Transformers

https://openreview.net/forum?id=uKmcyyrZae

Compressor summary: The paper questions the benefits of global attention in Graph Transformer for graph-structured data, proposes a Bi-Level Global Graph Transformer with Collaborative Training to address the over-globalizing problem, and provides empirical and theoretical evidence of its effectiveness.

BadPart: Unified Black-box Adversarial Patch Attacks against Pixel-wise Regression Tasks

https://openreview.net/forum?id=uGoi3nY62g

Compressor summary: The paper introduces a new black-box attack method to exploit vulnerabilities in pixel-wise regression models for applications like autonomous driving and augmented reality, showing its effectiveness against 7 models and Google's online service.

Multi-View Clustering by Inter-cluster Connectivity Guided Reward

https://openreview.net/forum?id=uEx2bSAJu8

Compressor summary: The paper proposes a graph-based multi-view clustering algorithm that infers the unknown number of clusters $K$ using inter-cluster connectivity as a reward in reinforcement learning, outperforming existing methods.

LayerMerge: Neural Network Depth Compression through Layer Pruning and Merging

https://openreview.net/forum?id=uDoy7AGvEC

Compressor summary: LayerMerge is a novel depth compression method that jointly prunes convolution layers and activation functions to enhance efficiency without sacrificing performance.

Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation

https://openreview.net/forum?id=uDkXoZMzBv

Compressor summary: This paper proposes a method to reduce computational costs of overparameterized models by exploiting low-dimensional structures in data and model parameters, showing its effectiveness for deep matrix completion and language model fine-tuning.

An amortized approach to non-linear mixed-effects modeling based on neural posterior estimation

https://openreview.net/forum?id=uCdcXRuHnC

Compressor summary: The authors propose a machine learning method using neural density estimation to efficiently infer population parameters in non-linear mixed-effects models for heterogeneous populations in various fields.

On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows

https://openreview.net/forum?id=uA3FRvO2DJ

Compressor summary: The paper introduces a new framework to better understand and improve normalizing flows, which are neural networks for transforming one probability distribution into another, by showing their limitations and how to overcome them.

A Global Geometric Analysis of Maximal Coding Rate Reduction

https://openreview.net/forum?id=u9qmjV2khT

Compressor summary: The paper characterizes the properties of local and global optima of the MCR$^2$ objective for learning structured deep representations, showing that it leads to diverse and discriminative solutions.

Empowering Graph Invariance Learning with Deep Spurious Infomax

https://openreview.net/forum?id=u9oSQtujCF

Compressor summary: Key points: - Graph neural networks need to generalize out-of-distribution data - Existing methods assume spurious features and target labels are correlated - New paradigm induces a robust inductive bias based on infomax principle - EQuAD framework disentangles invariant and spurious features - EQuAD shows improved performance in synthetic and real datasets Summary: The paper proposes a novel graph invariance learning method, EQuAD, that leverages the infomax principle to learn invariant features and disentangle them from spurious ones, achieving better generalization on OOD data.

Neural Image Compression with Text-guided Encoding for both Pixel-level and Perceptual Fidelity

https://openreview.net/forum?id=u8TZ9gm4im

Compressor summary: The text describes a new image compression algorithm that uses text information to improve both perceptual and pixel-wise quality, while avoiding the drawbacks of existing methods.

Position: Evolving AI Collectives Enhance Human Diversity and Enable Self-Regulation

https://openreview.net/forum?id=u6PeRHEsjL

Compressor summary: The text suggests that large language models can shape each other's behavior and form emergent AI societies, which can have benefits for human society and online environments, and calls for research on these issues.

STELLA: Continual Audio-Video Pre-training with SpatioTemporal Localized Alignment

https://openreview.net/forum?id=u4VR3WBH7a

Compressor summary: Key points: - The paper proposes a new method for continual audio-video pre-training that addresses two challenges: sparse spatio-temporal correlation and multimodal correlation overwriting. - The method uses Localized Patch Importance Scoring and Replay-guided Correlation Assessment to select relevant patches for pre-training. - The method improves performance in zero-shot retrieval tasks and reduces memory consumption. Summary: The paper presents a new continual audio-video pre-training method that uses patch importance scoring and correlation assessment to select relevant patches, improving retrieval performance and memory efficiency.

Improving Antibody Humanness Prediction using Patent Data

https://openreview.net/forum?id=u26c52rxZC

Compressor summary: The study uses patent data to improve antibody humanness prediction by training an encoder with contrastive learning and cross-entropy loss.

Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs

https://openreview.net/forum?id=u09gadH3BU

Compressor summary: The paper introduces any-precision LLM, a method to compress large language models and deploy them efficiently using varying bit-widths without sacrificing quality or performance.

Multi-Factor Adaptive Vision Selection for Egocentric Video Question Answering

https://openreview.net/forum?id=u00dmbI8Db

Compressor summary: The MFAS framework improves egocentric video question answering by enhancing small object recognition, suppressing noise, and aggregating visual semantics guided by questions.

Masked Face Recognition with Generative-to-Discriminative Representations

https://openreview.net/forum?id=tya725xlZ3

Compressor summary: The paper proposes a deep network that learns generative-to-discriminative representations for masked face recognition, using synthetic masked faces for pretraining.

Asymmetry in Low-Rank Adapters of Foundation Models

https://openreview.net/forum?id=txRZBD8tBV

Compressor summary: The paper studies the asymmetry in importance of low-rank adapter matrices during fine-tuning and shows that updating B matrix is more effective than A matrix, leading to parameter savings and improved generalization.

Bivariate Causal Discovery using Bayesian Model Selection

https://openreview.net/forum?id=twm7qPVX1F

Compressor summary: The paper proposes a new Bayesian method to identify causal directions in Markov equivalent structures, which works better than maximum likelihood methods and can handle realistic assumptions.

Faithfulness Measurable Masked Language Models

https://openreview.net/forum?id=tw1PwpuAuN

Compressor summary: The authors propose a novel method for measuring the faithfulness of NLP model explanations by incorporating token masking into a fine-tuning process that makes it in-distribution and improves importance measures.

DNCs Require More Planning Steps

https://openreview.net/forum?id=tu5fCCuua2

Compressor summary: This paper studies how the amount of computation time and memory affects the performance of a neural solver called Differentiable Neural Computer (DNC) on various algorithms, and finds that limiting its planning steps can lead to poor generalization and stability.

Pairwise Alignment Improves Graph Domain Adaptation

https://openreview.net/forum?id=ttnbM598vZ

Compressor summary: Pairwise Alignment (Pair-Align) is a novel graph domain adaptation method that handles shifts in features, labels, and connecting patterns using edge weights and label weights to recalibrate node influence and adjust classification loss.

Learning to Infer Generative Template Programs for Visual Concepts

https://openreview.net/forum?id=ttaTyweIr1

Compressor summary: The paper presents a neurosymbolic system that learns to infer general-purpose programs from visual datasets to perform various tasks in different domains.

How to Escape Sharp Minima with Random Perturbations

https://openreview.net/forum?id=tpYHbEl7P1

Compressor summary: The text discusses how to define and find flat minima in optimization algorithms for machine learning applications using the trace of the Hessian as a measure of flatness.

Diffusion Posterior Sampling is Computationally Intractable

https://openreview.net/forum?id=tp6ruPIfIV

Compressor summary: The paper shows that posterior sampling for inpainting and other tasks is computationally intractable even with fast unconditional sampling, using cryptographic assumptions.

Rethinking Adversarial Robustness in the Context of the Right to be Forgotten

https://openreview.net/forum?id=tmUorldOWN

Compressor summary: This paper explores a new security vulnerability in machine unlearning methods, proposes an attack that reduces adversarial robustness, and shows its potential for enhancing model stealing attacks.

Offline Actor-Critic Reinforcement Learning Scales to Large Models

https://openreview.net/forum?id=tl2qmO5kpD

Compressor summary: Offline actor-critic reinforcement learning can scale to large models like transformers and outperform behavioral cloning for multi-task training on continuous control tasks with sub-optimal data.

Keypoint-based Progressive Chain-of-Thought Distillation for LLMs

https://openreview.net/forum?id=tgsSKziIEa

Compressor summary: KPOD is a framework that improves language model reasoning transfer by weighting tokens and using progressive distillation.

Batch Singular Value Polarization and Weighted Semantic Augmentation for Universal Domain Adaptation

https://openreview.net/forum?id=teteOa9nJ9

Compressor summary: BSP-WSA is a novel method for universal domain adaptation that uses an adversarial classifier, singular value decomposition, and weighted semantic augmentation to handle category shift between domains.

MD tree: a model-diagnostic tree grown on loss landscape

https://openreview.net/forum?id=teHPKqjX8q

Compressor summary: The paper proposes a diagnosis method for neural networks based on loss landscape metrics, which outperforms validation-based approaches in identifying the source of model failure.

Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss

https://openreview.net/forum?id=tdomF3PW6A

Compressor summary: The paper proposes Convex Concave Loss (CCL), a method to increase the loss variance of training data and defend against membership inference attacks by reducing the convexity of loss functions with a concave term.

Connecting the Dots: Is Mode-Connectedness the Key to Feasible Sample-Based Inference in Bayesian Neural Networks?

https://openreview.net/forum?id=tc3Nmcpmnx

Compressor summary: The text discusses how to improve sample-based inference for Bayesian neural networks by understanding the relationship between weight and function space, and provides guidelines and an effective solution with competitive performance and uncertainty quantification.

ContPhy: Continuum Physical Concept Learning and Reasoning from Videos

https://openreview.net/forum?id=tVwzR1myUp

Compressor summary: The Continuum Physical Dataset (ContPhy) is a new benchmark for testing AI models' ability to reason about diverse physical properties and dynamics of soft-bodied objects and other continuum phenomena, revealing their current limitations and inspiring improvements in perception and reasoning.

Finite Time Logarithmic Regret Bounds for Self-Tuning Regulation

https://openreview.net/forum?id=tTtSnpH4fc

Compressor summary: The paper proposes a new algorithm, PIECE, that achieves finite-time logarithmic regret bounds for self-tuning regulation problem and improves initial transient performance.

Scene Graph Generation Strategy with Co-occurrence Knowledge and Learnable Term Frequency

https://openreview.net/forum?id=tTq3qMkJ8w

Compressor summary: The paper proposes CooK, a model that uses co-occurrence knowledge and TF-$l$-IDF to improve scene graph generation, achieving better performance and generalization than existing methods.

Latent Noise Segmentation: How Neural Noise Leads to the Emergence of Segmentation and Grouping

https://openreview.net/forum?id=tSjyKR8WIf

Compressor summary: The paper proposes that neural noise can help humans and AI models group and segment images without supervision, leading to better performance on perceptual grouping tasks.

Barrier Algorithms for Constrained Non-Convex Optimization

https://openreview.net/forum?id=tRESfzWFtf

Compressor summary: The paper presents new interior-point methods for non-convex optimization with constraints that have better global complexity than existing methods.

PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetition

https://openreview.net/forum?id=tQPkzTdaaN

Compressor summary: PARDEN is a method that uses a large language model as a safeguard to detect and prevent jailbreaks by asking it to repeat its own outputs, achieving better results than existing approaches.

Random Exploration in Bayesian Optimization: Order-Optimal Regret and Computational Efficiency

https://openreview.net/forum?id=tOO6PD3kYP

Compressor summary: The paper proposes and analyzes a Bayesian optimization method using Gaussian Process models with random exploration, which achieves optimal error rates, has computational advantages, and partially resolves an open problem.

Fast Sampling-Based Sketches for Tensors

https://openreview.net/forum?id=tMkPL7Tiul

Compressor summary: The text presents a new method for creating sketches that can be applied to rank one tensors in two or three modes, achieving faster computation times by using fast convolution and random subsets of tensor entries.

Zero-Shot Reinforcement Learning via Function Encoders

https://openreview.net/forum?id=tHBLwSYnLf

Compressor summary: The paper proposes the function encoder, an algorithm that helps reinforcement learning agents transfer between related tasks by providing a coherent vector representation of the reward or transition function.

A Touch, Vision, and Language Dataset for Multimodal Alignment

https://openreview.net/forum?id=tFEOOH9eH0

Compressor summary: The text introduces a new dataset of visiontouch pairs with English labels and presents a TVL model that improves tactile-vision-language alignment over existing models.

Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference

https://openreview.net/forum?id=tDRYrAkOB7

Compressor summary: Dynamic Memory Compression (DMC) is a method to compress the cache of key-value representations for past tokens in large language models, improving generation efficiency and allowing them to fit longer contexts and larger batches within any given memory budget.

SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models

https://openreview.net/forum?id=tDMlQkJRhZ

Compressor summary: SPHINX-X is a large language model developed upon SPHINX, with improved architecture and training efficiency, trained on diverse multi-domain and multi-modal data, and obtaining strong correlation between performance and data/parameter scales.

MADA: Meta-Adaptive Optimizers Through Hyper-Gradient Descent

https://openreview.net/forum?id=tASXcrMekp

Compressor summary: MADA is a unified optimizer framework that learns the best adaptive optimizer for deep learning tasks during training, and outperforms Adam and other popular optimizers on vision and language tasks.

Task-aware Orthogonal Sparse Network for Exploring Shared Knowledge in Continual Learning

https://openreview.net/forum?id=tABvuya05B

Compressor summary: The paper proposes a method for continuous learning that partitions the network into three parts to share knowledge between old and new tasks, improving performance and preventing forgetting.

Larimar: Large Language Models with Episodic Memory Control

https://openreview.net/forum?id=t8mt4YrPsq

Compressor summary: Larimar is a brain-inspired architecture that enhances LLMs with a distributed episodic memory for efficient, accurate, and flexible knowledge updates without re-training or fine-tuning.

The Computational Complexity of Finding Second-Order Stationary Points

https://openreview.net/forum?id=t8WDBcegae

Compressor summary: The paper shows that finding approximate second-order stationary points in non-convex optimization problems is as hard as finding first-order stationary points and depends on the domain complexity, contrary to previous results.

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning

https://openreview.net/forum?id=t82Y3fmRtk

Compressor summary: The paper presents **R**$^3$, a method that uses outcome supervision and reverse curriculum to improve large language models' reasoning skills.

TimeX++: Learning Time-Series Explanations with Information Bottleneck

https://openreview.net/forum?id=t6dBpwkbea

Compressor summary: The paper proposes TimeX++, an explanation framework that uses information bottleneck principle to produce high-quality explanations for deep learning models operating on time series data, and evaluates it on synthetic and real-world datasets.

Compositional Few-Shot Class-Incremental Learning

https://openreview.net/forum?id=t4908PyZxs

Compressor summary: The paper proposes a cognitive-inspired method for few-shot class-incremental learning that uses compositional learning, set similarities, and primitive reusability to recognize novel classes with fewer samples, achieving improved performance and interpretability.

Coprocessor Actor Critic: A Model-Based Reinforcement Learning Approach For Adaptive Brain Stimulation

https://openreview.net/forum?id=t3SEfoTaYQ

Compressor summary: Coprocessor Actor Critic is a novel reinforcement learning method for adaptive brain stimulation that learns how to act and induce optimal actions in the world with less samples and higher success than traditional approaches.

Surface-VQMAE: Vector-quantized Masked Auto-encoders on Molecular Surfaces

https://openreview.net/forum?id=szxtVHOh0C

Compressor summary: Key points: - Surface-VQMAE is a novel surface-based unsupervised learning algorithm for protein function analysis - It uses Transformer architecture and vector quantization to capture patch-level relations and enforce discrete posterior distribution - It shows effectiveness in various scenarios such as binding site scoring, affinity prediction, and mutant effect estimation Summary: Surface-VQMAE is a new algorithm that uses protein surface information and unsupervised learning to analyze protein functions, achieving good results in different applications.

DataFreeShield: Defending Adversarial Attacks without Training Data

https://openreview.net/forum?id=szvKJgmubh

Compressor summary: Key points: - The text is about achieving adversarial robustness without real data - Existing methods assume accessibility to original data and fail in this scenario - DataFreeShield proposes two solutions: surrogate dataset generation and adversarial training - DataFreeShield outperforms baselines and shows the first data-free solution Summary: The text presents DataFreeShield, a method that achieves adversarial robustness without real data by generating surrogate datasets and training with them, outperforming existing methods.

Advancing Dynamic Sparse Training by Exploring Optimization Opportunities

https://openreview.net/forum?id=szRHR9XGrY

Compressor summary: The paper introduces BiDST, a novel framework for dynamic sparse training that optimizes both weights and masks simultaneously, achieving better accuracy, faster speed, and reduced overhead compared to traditional methods.

Dirichlet Flow Matching with Applications to DNA Sequence Design

https://openreview.net/forum?id=syXFAVqx85

Compressor summary: Dirichlet flow matching on the simplex improves sequence generation speed and quality over autoregressive models, especially for complex DNA sequences.

IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation

https://openreview.net/forum?id=swTG6xju8O

Compressor summary: The paper proposes IM-3D, a text-to-3D model that uses video generators, 3D reconstruction with Gaussian splatting, and reduces evaluation times to create high-quality 3D outputs efficiently.

Sparser, Better, Deeper, Stronger: Improving Static Sparse Training with Exact Orthogonal Initialization

https://openreview.net/forum?id=svm53KQAtN

Compressor summary: EOI is a novel sparse orthogonal initialization scheme that provides exact orthogonality and enables creation of layers with arbitrary densities, outperforming common sparse initialization techniques for static sparse training.

Accelerated Speculative Sampling Based on Tree Monte Carlo

https://openreview.net/forum?id=stMhi1Sn2G

Compressor summary: Accelerated Speculative Sampling (ASpS) is a new algorithm that improves inference speed of large language models by using Tree Monte Carlo methods to generate multiple tokens in one step and finding better global maximum couplings.

Transferable Facial Privacy Protection against Blind Face Restoration via Domain-Consistent Adversarial Obfuscation

https://openreview.net/forum?id=st2BTty53v

Compressor summary: The paper proposes a new method for protecting anonymity in facial recognition by creating adversarial obfuscation that fools face restoration techniques, which can restore pixelated faces with high accuracy.

ULAREF: A Unified Label Refinement Framework for Learning with Inaccurate Supervision

https://openreview.net/forum?id=ssFMq35UUY

Compressor summary: The paper proposes ULAREF, a unified framework for learning with inaccurate supervision, which refines labels using global reliability detection and local enhancement with a consistency loss.

ReLUs Are Sufficient for Learning Implicit Neural Representations

https://openreview.net/forum?id=srejp9uOx7

Compressor summary: The authors propose using ReLU neurons with constraints in deep neural networks to learn implicit neural representations, showing their effectiveness and versatility in various tasks.

Ambiguity-Aware Abductive Learning

https://openreview.net/forum?id=sqv2xP8rfb

Compressor summary: The paper proposes a new method for abductive learning, called Ambiguity-Aware Abductive Learning (A$^3$BL), which improves the existing approach by evaluating all potential candidates and their probabilities to better handle uncertainty in the knowledge base.

On a Combinatorial Problem Arising in Machine Teaching

https://openreview.net/forum?id=spOpHW1No2

Compressor summary: This paper proves that the worst case teaching dimension occurs when using binary representations of numbers in machine teaching.

Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset

https://openreview.net/forum?id=snhurpZt63

Compressor summary: The paper introduces a new underwater salient instance segmentation dataset (USIS10K) and a method (USIS-SAM) based on Segment Anything Model that uses visual prompts to improve segmentation accuracy in underwater scenes.

Revisiting Context Aggregation for Image Matting

https://openreview.net/forum?id=sjJZHPV9Id

Compressor summary: The paper proposes AEMatter, a new matting network with a Hybrid-Transformer backbone and appearance-enhanced axis-wise learning blocks that achieves superior performance compared to existing methods.

Online Learning under Budget and ROI Constraints via Weak Adaptivity

https://openreview.net/forum?id=shzEkKPrsn

Compressor summary: The paper proposes a new online learning framework that works without strict feasibility assumptions and proves its no-regret guarantees for bidding in ad auctions.

Fast Algorithms for Hypergraph PageRank with Applications to Semi-Supervised Learning

https://openreview.net/forum?id=sfQH4JJ4We

Compressor summary: The paper presents fast and scalable algorithms for hypergraph models in semi-supervised learning that capture higher-order relations better than graph-based methods and improve clustering on categorical data.

In value-based deep reinforcement learning, a pruned network is a good network

https://openreview.net/forum?id=seo9V9QRZp

Compressor summary: Gradual magnitude pruning improves value-based deep reinforcement learning agents' parameter effectiveness and performance with minimal parameters.

Scaling Laws for the Value of Individual Data Points in Machine Learning

https://openreview.net/forum?id=scSB9RynSd

Compressor summary: The text discusses how machine learning models' performance improves with more data and how individual data points contribute differently to model accuracy depending on dataset size, suggesting ways to use this knowledge for data valuation and selection.

Bayesian Optimization of Function Networks with Partial Evaluations

https://openreview.net/forum?id=scMAQ3mFAA

Compressor summary: Cost-aware node selection improves Bayesian optimization of function networks by reducing evaluation costs.

Projecting Molecules into Synthesizable Chemical Spaces

https://openreview.net/forum?id=scFlbJQdm1

Compressor summary: The authors propose a novel framework that uses a transformer-based model to generate new chemical structures while ensuring synthetic accessibility through postfix notations of synthesis pathways.

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference

https://openreview.net/forum?id=sb81Xl50JG

Compressor summary: APT is a method that adaptively prunes and tunes language model parameters for improved training and inference efficiency, maintaining high performance with significantly reduced parameters and faster fine-tuning.

Pruned Pivot: Correlation Clustering Algorithm for Dynamic, Parallel, and Local Computation Models

https://openreview.net/forum?id=saP7s0ZgYE

Compressor summary: The paper proposes a fast and efficient algorithm for correlation clustering in dynamic graphs, with performance guarantees similar to the well-known Pivot algorithm.

Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification

https://openreview.net/forum?id=sZla6SnooP

Compressor summary: The paper proposes model-based policy iteration algorithms for nonlinear optimal control problems using neural networks to solve PDEs, with convergence guarantees and outperforming traditional methods.

Language Models as Semantic Indexers

https://openreview.net/forum?id=sYeioWoF9u

Compressor summary: The paper proposes LMIndexer, a self-supervised framework that uses a generative language model to learn semantic identifiers for information retrieval tasks, addressing challenges such as sequential discrete ID and semantic supervision deficiency.

Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systems

https://openreview.net/forum?id=sTVSyqD6XX

Compressor summary: The paper proposes methods for preserving privacy in Federated Learning with centralized systems, ensuring Differential Privacy while maintaining optimal convergence rates and linear computational complexity.

Low-Cost High-Power Membership Inference Attacks

https://openreview.net/forum?id=sT7UJh5CTc

Compressor summary: The paper proposes a new statistical test for robustly detecting if a data point was used to train a model, using reference models and population data samples, with low computational cost and high accuracy.

Private Heterogeneous Federated Learning Without a Trusted Server Revisited: Error-Optimal and Communication-Efficient Algorithms for Convex Losses

https://openreview.net/forum?id=sSAEhcdB9N

Compressor summary: This paper proposes novel federated learning algorithms that protect data privacy, perform well with heterogeneous data, and require fewer communication rounds than existing methods.

PAC-Bayesian Generalization Bounds for Knowledge Graph Representation Learning

https://openreview.net/forum?id=sOyJSNUrzQ

Compressor summary: The paper presents a generic framework called ReED for analyzing various knowledge graph representation learning (KGRL) models and provides theoretical generalization bounds for them, which can guide their practical design.

Optimal Eye Surgeon: Finding image priors through sparse generators at initialization

https://openreview.net/forum?id=sO5qtpvsUZ

Compressor summary: The Optimal Eye Surgeon (OES) framework prunes and trains deep image generator networks by adaptively underparameterizing them, which helps resist overfitting to noise and improves image restoration tasks.

Stochastic Quantum Sampling for Non-Logconcave Distributions and Estimating Partition Functions

https://openreview.net/forum?id=sNjxqSnXFO

Compressor summary: The paper presents quantum algorithms for sampling from non-logconcave probability distributions and estimating their partition functions, overcoming challenges by using a reference reversible Markov chain and showing polynomial speedups.

CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Process

https://openreview.net/forum?id=sLZzFTMWSt

Compressor summary: CaRiNG is a method to identify causal processes in sequential data with non-invertible generation, using temporal context and an identifiability theory.

Sample Complexity Bounds for Estimating Probability Divergences under Invariances

https://openreview.net/forum?id=sKjcrAC4eZ

Compressor summary: The paper studies how Lie group invariances reduce sample complexity and improve convergence rates when estimating various distances and density estimation problems in machine learning models.

Large Language Models are Geographically Biased

https://openreview.net/forum?id=sHtIStlg0v

Compressor summary: The authors study geographic biases in large language models and show that they are correlated with socioeconomic conditions and exhibit significant variation across models.

Language-Driven Cross-Modal Classifier for Zero-Shot Multi-Label Image Recognition

https://openreview.net/forum?id=sHswzNWUW2

Compressor summary: The paper proposes a new language-driven framework for zero-shot multi-label image recognition using CLIP, LLMs, and cross-modal mapping without annotated images during training.

Tell, Don't Show: Language Guidance Eases Transfer Across Domains in Images and Videos

https://openreview.net/forum?id=sFN49CfklF

Compressor summary: LaGTran is a text supervision framework that transfers discriminative knowledge from labeled source to unlabeled target data with domain gaps, outperforming prior approaches on image and video datasets.

Vectorized Conditional Neural Fields: A Framework for Solving Time-dependent Parametric Partial Differential Equations

https://openreview.net/forum?id=sF9epWkNUG

Compressor summary: VCNeFs are a new method to solve time-dependent PDEs using neural fields, attention mechanisms, and conditional learning, addressing limitations of previous Transformer-based approaches.

LASER: Linear Compression in Wireless Distributed Optimization

https://openreview.net/forum?id=sDjszMb2Ir

Compressor summary: LASER is a new compression scheme that efficiently transmits low-rank gradients over noisy channels, achieving better performance than existing methods on practical machine learning tasks.

Compressing Large Language Models by Joint Sparsification and Quantization

https://openreview.net/forum?id=sCGRhnuMUJ

Compressor summary: The paper proposes JSQ, a novel model compression technique for large language models that integrates sparsification and quantization to achieve significant computation reduction without performance degradation.

Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data

https://openreview.net/forum?id=sBJNokmYuV

Compressor summary: CPL is a method that generates refined candidate pseudolabels for vision-language models to improve fine-tuning performance in downstream tasks.

Understanding and Diagnosing Deep Reinforcement Learning

https://openreview.net/forum?id=s9RKqT7jVM

Compressor summary: The text discusses concerns about the stability of deep neural policies in various settings, and introduces a method to analyze and improve these policies using robust training techniques.

WARM: On the Benefits of Weight Averaged Reward Models

https://openreview.net/forum?id=s7RDnNUJy6

Compressor summary: Weight Averaged Reward Models (WARM) is a method to prevent reward hacking in large language models by averaging multiple fine-tuned reward models, improving quality and alignment of predictions.

Fundamental Benefit of Alternating Updates in Minimax Optimization

https://openreview.net/forum?id=s6ZAT8MLKU

Compressor summary: The paper analyzes the convergence rates of two variants of the GDA algorithm for minimax optimization problems, shows that Alternating-GDA is faster and has better performance than Simultaneous-GDA, and proposes a new algorithm called Alex-GDA that improves upon both.

Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environments

https://openreview.net/forum?id=s5PLISyNyP

Compressor summary: The paper proposes a method to estimate the CATE from observational data in multiple environments with violations of standard causal assumptions by using them as instrumental variables and combining meta-learners with machine learning models.

High-Performance Temporal Reversible Spiking Neural Networks with $\mathcal{O}(L)$ Training Memory and $\mathcal{O}(1)$ Inference Cost

https://openreview.net/forum?id=s4h6nyjM9H

Compressor summary: T-RevSNN is a novel spiking neural network architecture that reduces memory, training time, and inference energy costs by temporally reversible interactions and redesigned input encoding.

MS-TIP: Imputation Aware Pedestrian Trajectory Prediction

https://openreview.net/forum?id=s4Hy0L4mml

Compressor summary: The paper proposes a novel approach (MS-TIP) that uses transformers, multi-scale hypergraphs, and scenic attention to predict pedestrian trajectories accurately even when the observed sequences are incomplete.

Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance

https://openreview.net/forum?id=s4EYBJ30WY

Compressor summary: The paper introduces spectral imbalance as a source of class bias in feature learning and proposes a framework to study, compare, and mitigate it using 11 pre-trained encoders.

In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation

https://openreview.net/forum?id=s3e8poX3kb

Compressor summary: The authors study why large language models make factual errors, find a pattern related to sharpness of context activations, and propose an entropy-based metric to improve the quality of generated text by adjusting token prediction distributions.

LoRA Training in the NTK Regime has No Spurious Local Minima

https://openreview.net/forum?id=s1sdx6vNsU

Compressor summary: This paper analyzes how low-rank adaptation (LoRA) helps fine-tune large language models efficiently and effectively by avoiding spurious minima and achieving good generalization.

DiffAug: Enhance Unsupervised Contrastive Learning with Domain-Knowledge-Free Diffusion-based Data Augmentation

https://openreview.net/forum?id=s0UDX7Kswl

Compressor summary: DiffAug is a novel unsupervised contrastive learning technique that uses diffusion models to generate positive samples based on semantic encoders, improving representation ability in various domains.

Don't trust your eyes: on the (un)reliability of feature visualizations

https://openreview.net/forum?id=s0Jvdolv2I

Compressor summary: Feature visualizations are not reliable for explaining how neural networks process natural images because they can be easily fooled by arbitrary patterns and do not match standard input processing.

SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation

https://openreview.net/forum?id=ryDa4mS18V

Compressor summary: SAM-E is a novel architecture for robot manipulation that uses a vision foundation model for scene understanding and sequence imitation for long-term action reasoning, achieving superior performance and efficiency.

Differentiable Annealed Importance Sampling Minimizes The Jensen-Shannon Divergence Between Initial and Target Distribution

https://openreview.net/forum?id=rvaN2P1rvC

Compressor summary: DAIS optimizes a symmetrized KL divergence between initial and target distribution, acting as a variational inference method that often improves uncertainty estimates.

Dynamic Facility Location in High Dimensional Euclidean Spaces

https://openreview.net/forum?id=rucbIsWoEV

Compressor summary: The paper presents the first dynamic algorithm for facility location in high-dimensional spaces, achieving good quality and stable solutions with sub-linear update times.

Sampling in Unit Time with Kernel Fisher-Rao Flow

https://openreview.net/forum?id=rtyqBfcg8j

Compressor summary: The paper proposes a new method to sample from unnormalized probability densities using a mean-field ODE and interacting particle systems that are gradient-free, closed-form, and require only sampling from a reference density and computing its ratio to the target density.

AlphaFold Meets Flow Matching for Generating Protein Ensembles

https://openreview.net/forum?id=rs8Sh2UASt

Compressor summary: Key points: - A flow-based generative modeling approach for learning and sampling protein conformational landscapes - AlphaFlow and ESMFlow are sequence-conditioned generative models of protein structure based on AlphaFold and ESMFold - Outperforms AlphaFold with MSA subsampling and captures conformational flexibility, positional distributions, and higher-order ensemble observables - Can diversify a static PDB structure faster than replicate MD trajectories Summary: The authors develop AlphaFlow and ESMFlow, generative models of protein structure that use flow methods to improve accuracy and diversity, and show their potential for simulating conformational flexibility and equilibrium properties.

Designing Decision Support Systems using Counterfactual Prediction Sets

https://openreview.net/forum?id=rqyXubsBhH

Compressor summary: This paper proposes a new design for decision support systems using prediction sets and online learning, which improves the performance by giving less agency to human experts.

Listening to the noise: Blind Denoising with Gibbs Diffusion

https://openreview.net/forum?id=rmEgJ7bhuZ

Compressor summary: The paper introduces Gibbs Diffusion, a method that enables blind denoising by simultaneously sampling the signal and noise parameters using a conditional diffusion model and a Monte Carlo sampler.

Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustness

https://openreview.net/forum?id=rkYOxLLv2x

Compressor summary: The paper evaluates large kernel convolutional neural networks (CNNs) and shows they can be as or more robust than vision transformers (ViTs), revealing novel insights into their source of robustness.

Reducing Item Discrepancy via Differentially Private Robust Embedding Alignment for Privacy-Preserving Cross Domain Recommendation

https://openreview.net/forum?id=rk4kmL8aOY

Compressor summary: The paper introduces RidCDR, a model for privacy-preserving cross-domain recommendation that uses embedding alignment to share knowledge without overlapping data.

Position: Intent-aligned AI Systems Must Optimize for Agency Preservation

https://openreview.net/forum?id=rfvgdfd1K9

Compressor summary: The text discusses the need for agency-preserving AI-human interactions that protect humans' long-term control, rather than just aligning AI systems with human intentions.

SaVeR: Optimal Data Collection Strategy for Safe Policy Evaluation in Tabular MDP

https://openreview.net/forum?id=reB9FFAaKw

Compressor summary: The paper studies how to collect data safely for policy evaluation in tabular MDPs and proposes a safe oracle algorithm called SaVeR.

A New Theoretical Perspective on Data Heterogeneity in Federated Optimization

https://openreview.net/forum?id=re6es2atbl

Compressor summary: The paper proposes a new assumption to analyze federated learning convergence rate under data heterogeneity and shows that it can reduce the impact of local Lipschitz constant and improve performance.

Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-Loop and Hessian-Free Solution Strategy

https://openreview.net/forum?id=rZD9hV0Bc4

Compressor summary: This paper proposes an efficient gradient-based algorithm for large-scale nonconvex Bi-Level Optimization problems in machine learning, with theoretical guarantees and experimental validation.

Piecewise Constant and Linear Regression Trees: An Optimal Dynamic Programming Approach

https://openreview.net/forum?id=rXnBvu5D7i

Compressor summary: The authors develop and test optimal dynamic programming methods to improve the scalability and performance of regression trees, a machine learning model that represents complex relationships.

Coactive Learning for Large Language Models using Implicit User Feedback

https://openreview.net/forum?id=rVWsTjMW1m

Compressor summary: Coactive learning uses users' implicit edits to improve large language models without supervised training, enabling personalized LLMs.

Position: Is machine learning good or bad for the natural sciences?

https://openreview.net/forum?id=rU8o0QQCy0

Compressor summary: Machine learning methods can be valuable for causal inference and emulating simulations in natural sciences, but they also introduce unwanted biases in some cases.

Decomposing and Editing Predictions by Modeling Model Computation

https://openreview.net/forum?id=rTBR0eqE4G

Compressor summary: Component modeling helps to understand how machine learning models make predictions by breaking them down into their parts, and COAR is a tool that estimates the impact of each part on the prediction.

Effective Federated Graph Matching

https://openreview.net/forum?id=rSfzchjIYu

Compressor summary: The paper presents UFGM, an unsupervised federated graph matching algorithm that uses graphlet features and trust region optimization to match node pairs across clients while preserving privacy.

Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learning

https://openreview.net/forum?id=rReWhol66R

Compressor summary: The paper proposes a novel representation-based approach to measure the domain gap and filter data for cross-domain offline reinforcement learning, achieving superior performance with less target data.

Expressivity and Generalization: Fragment-Biases for Molecular GNNs

https://openreview.net/forum?id=rPm5cKb1VB

Compressor summary: The authors propose a new GNN architecture and a fragmentation method with infinite vocabulary that improves expressiveness and performance on molecular property prediction compared to recent advances in higher-order GNNs, using the Fragment-WL test for theoretical analysis.

Highway Value Iteration Networks

https://openreview.net/forum?id=rORsGuE2hV

Compressor summary: The paper proposes a new planning algorithm called highway value iteration network that combines value iteration with skip connections, exploration, and safety checks to enable effective long-term planning with deep neural networks.

Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesis

https://openreview.net/forum?id=rMV86cAOh6

Compressor summary: The text introduces SCDM, a robust conditional diffusion model that generates realistic images from noisy semantic maps by stochastically perturbing the labels through Label Diffusion and using a class-wise noise schedule.

Transport of Algebraic Structure to Latent Embeddings

https://openreview.net/forum?id=rK6AZem0hX

Compressor summary: The paper proposes a method to learn consistent operations from latent embeddings by mapping them to a mirrored algebra on Euclidean space.

Closing the Gap: Achieving Global Convergence (Last Iterate) of Actor-Critic under Markovian Sampling with Neural Network Parametrization

https://openreview.net/forum?id=rJxFvAs7pq

Compressor summary: The paper provides a comprehensive theoretical analysis of Actor-Critic algorithms, considering practical aspects such as multi-layer neural networks, Markovian sampling, continuous state-action spaces, last iterate performance, and global optimality, and shows their global convergence with sample complexity bounds.

Learning to Compile Programs to Neural Networks

https://openreview.net/forum?id=rJti61Uere

Compressor summary: A neural surrogate compiler uses a hypernetwork to generate efficient neural networks that mimic the behavior of programs, improving data efficiency and training speed compared to traditional methods.

In-context Learning on Function Classes Unveiled for Transformers

https://openreview.net/forum?id=rJkGOARXns

Compressor summary: The paper investigates how transformer-based neural sequence models can learn different types of functions in-context by approximating them with neural networks and studying their pre-training and activation functions.

Linguistic Calibration of Long-Form Generations

https://openreview.net/forum?id=rJVjQSQ8ye

Compressor summary: The paper proposes a method to train language models to generate long-form text with calibrated confidence statements, which helps users make better decisions based on the model's predictions.

Exploration and Anti-Exploration with Distributional Random Network Distillation

https://openreview.net/forum?id=rIrpzmqRBk

Compressor summary: The paper proposes DRND, a modified RND algorithm that improves exploration in deep reinforcement learning by distilling a distribution of random networks and allocating bonuses more precisely.

GNNs Also Deserve Editing, and They Need It More Than Once

https://openreview.net/forum?id=rIc9adYbH2

Compressor summary: Key points: - The paper proposes SEED-GNN, a GNN model editing method that is practical and effective. - The main challenge of GNN editing is sequential editing robustness, which is lacking in existing methods due to overfitting. - The paper also defines the task paradigm of GNN editing and hopes to inspire more research in this area. Summary: The paper introduces SEED-GNN, a novel method for editing graph neural networks (GNNs) that can handle multiple errors sequentially without overfitting, and formalizes the GNN editing task paradigm.

Visual Representation Learning with Stochastic Frame Prediction

https://openreview.net/forum?id=rI6lxIX0uX

Compressor summary: The paper proposes a stochastic frame prediction model for learning image representations that captures uncertainty and temporal information, and shows its effectiveness on various video-based tasks.

Restoring balance: principled under/oversampling of data for optimal classification

https://openreview.net/forum?id=rHylzxK3HU

Compressor summary: The paper analyzes how class imbalance affects generalization curves for linear classifiers and proposes mixed under/oversampling strategies to improve performance.

PASOA- PArticle baSed Bayesian Optimal Adaptive design

https://openreview.net/forum?id=rGCvMARXkG

Compressor summary: PASOA is a new Bayesian experimental design method that uses contrastive estimation, stochastic optimization, and tempered SMC to balance information gain and accuracy in sequential design optimization and parameter inference.

UPOCR: Towards Unified Pixel-Level OCR Interface

https://openreview.net/forum?id=rEZ24oJhbn

Compressor summary: UPOCR is a simple and effective generalist model for pixel-level OCR tasks that unifies paradigms, architectures, and training strategies using vision Transformers and learnable task prompts.

InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models

https://openreview.net/forum?id=rADFNrIss3

Compressor summary: InstructZero is a method that optimizes soft prompts to create instructions for black-box LLMs, improving their performance on various tasks.

BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model

https://openreview.net/forum?id=r9rzU9QzPe

Compressor summary: The BiSHop model combines associative memory and attention mechanisms to handle non-rotational invariance and feature sparsity in deep tabular learning, achieving superior performance on real-world datasets with fewer hyperparameter searches.

Reweighted Solutions for Weighted Low Rank Approximation

https://openreview.net/forum?id=r9XICONppE

Compressor summary: The paper introduces a new relaxed solution to weighted low rank approximation (WLRA) that uses the weight matrix itself to reweight a low rank solution, achieving simple and efficient algorithms with provable approximation guarantees.

Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation

https://openreview.net/forum?id=r8k5JrGip6

Compressor summary: Superposition prompting is a novel method to improve efficiency and quality of large language models in retrieval-augmented generation by processing input documents in parallel paths and discarding irrelevant ones.

Learning to Route Among Specialized Experts for Zero-Shot Generalization

https://openreview.net/forum?id=r0qcGcFL4U

Compressor summary: The paper proposes a post-hoc method for routing tokens to specialized language models to improve zero-shot generalization, called PHATGOOSE.

Test-Time Model Adaptation with Only Forward Passes

https://openreview.net/forum?id=qz1Vx1v9iK

Compressor summary: The paper proposes a test-time adaptation method for resource-limited devices that uses input prompts, covariance matrix adaptation evolution strategy, and activation shifting to adapt models without backpropagation or weight changes.

$S^2$IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting

https://openreview.net/forum?id=qwQVV5R8Y7

Compressor summary: The paper proposes a method to use pre-trained language models for time series forecasting by aligning their semantic space with temporal dynamics using token embeddings and learned prompts.

Multi-Agent Reinforcement Learning Meets Leaf Sequencing in Radiotherapy

https://openreview.net/forum?id=qwKSTLbati

Compressor summary: The paper proposes a deep reinforcement learning model called Reinforced Leaf Sequencer to improve radiotherapy planning by optimizing leaf sequencing in a multi-agent framework.

Latent Space Symmetry Discovery

https://openreview.net/forum?id=qstt2OguvM

Compressor summary: LaLiGAN is a novel generative model that can discover nonlinear symmetries in data and latent space, enabling applications like equation discovery and long-term forecasting.

Learning High-Frequency Functions Made Easy with Sinusoidal Positional Encoding

https://openreview.net/forum?id=qqPL0DkcrI

Compressor summary: Sinusoidal positional encoding (SPE) is a new method that learns adaptive frequency features without hyperparameter tuning, improving fidelity and speed in various tasks such as 3D view synthesis, Text-to-Speech generation, and 1D regression.

An Empirical Study Into What Matters for Calibrating Vision-Language Models

https://openreview.net/forum?id=qoxuPshrZb

Compressor summary: The study examines how well vision-language models (VLMs) can estimate uncertainty across different settings and finds that temperature scaling helps improve calibration, even with a small amount of data.

Manifold Integrated Gradients: Riemannian Geometry for Feature Attribution

https://openreview.net/forum?id=qoOt02l2WC

Compressor summary: The paper proposes a method to improve Integrated Gradients, a feature attribution technique for deep learning models, by aligning the path of attribution with the data manifold's geometry, resulting in more intuitive explanations and increased robustness to adversarial attacks.

Multi-Source Conformal Inference Under Distribution Shift

https://openreview.net/forum?id=qmUbSAgz08

Compressor summary: The paper proposes a method to estimate uncertainty in machine learning predictions using multiple biased data sources, and shows its usefulness in predicting hospital length of stay for pediatric heart surgery patients.

Implicit Regularization in Feedback Alignment Learning Mechanisms for Neural Networks

https://openreview.net/forum?id=qklMNNub0H

Compressor summary: The text introduces a unified framework explaining how feedback alignment works in neural networks, improves its performance on multi-class tasks, and offers theoretical and empirical insights for better understanding and developing the method.

From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems

https://openreview.net/forum?id=qkhbyDqlNI

Compressor summary: This paper investigates how large language models can solve physical world problems through hierarchical reinforcement learning and exploration strategies.

Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot

https://openreview.net/forum?id=qjqlhWDcId

Compressor summary: The paper shows that transformers can learn sparse token selection and have better generalization than fully-connected networks.

On the Emergence of Cross-Task Linearity in Pretraining-Finetuning Paradigm

https://openreview.net/forum?id=qg6AlnpEQH

Compressor summary: Cross-Task Linearity is a phenomenon where linearly interpolating weights of finetuned models starting from the same pretrained checkpoint results in similar features across tasks, suggesting neural networks act as approximate linear maps.

Memory Consolidation Enables Long-Context Video Understanding

https://openreview.net/forum?id=qeFgvVVAJ2

Compressor summary: The memory-consolidated vision transformer (MC-ViT) extends the context for video understanding by fine-tuning pre-trained video transformers to attend to non-parametric memories, achieving state-of-the-art results on long-context tasks with fewer parameters.

Provably Efficient Reinforcement Learning for Adversarial Restless Multi-Armed Bandits with Unknown Transitions and Bandit Feedback

https://openreview.net/forum?id=qbIKUfastZ

Compressor summary: Key points: - The paper proposes a reinforcement learning algorithm for episodic RMAB with unknown transitions, bandit feedback, and adversarial rewards - The algorithm has two main components: a biased adversarial reward estimator and a low-complexity index policy - The algorithm achieves $ ilde{\mathcal{O}}(H\sqrt{T})$ regret bound, which is the first to ensure $ ilde{\mathcal{O}}(\sqrt{T})$ regret for this problem setting Summary: The paper presents a novel reinforcement learning algorithm for sequential decision making problems with adversarial rewards and unknown transitions, using a biased reward estimator and a low-complexity policy, and showing an improved regret bound.

PriorBoost: An Adaptive Algorithm for Learning from Aggregate Responses

https://openreview.net/forum?id=qawwyKqOkj

Compressor summary: The paper proposes a bagging method for event-level loss functions in linear regression and GLMs, which reduces to one-dimensional clustering and improves model quality with adaptive homogeneous sampling and label differential privacy.

Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chains

https://openreview.net/forum?id=qY63FnLuJ1

Compressor summary: The paper proposes a new pre-training strategy for protein models that combines residue and atom information, improving performance on downstream tasks like binding site prediction and function prediction.

Pausing Policy Learning in Non-stationary Reinforcement Learning

https://openreview.net/forum?id=qY622O6Ehg

Compressor summary: The text discusses how strategically pausing decision updates in online reinforcement learning can improve performance by managing uncertainty, and provides theoretical and experimental evidence for this claim.

Defining Neural Network Architecture through Polytope Structures of Datasets

https://openreview.net/forum?id=qXoqV40imX

Compressor summary: The paper defines upper and lower bounds for neural network widths based on dataset complexity, explores how geometry affects network requirements, and develops an algorithm to infer dataset structure from trained networks.

Extracting Training Data From Document-Based VQA Models

https://openreview.net/forum?id=qTX1vxzs8b

Compressor summary: The paper studies how vision-language models can memorize and leak personal information from images during training, and proposes a countermeasure to prevent this issue.

BLO-SAM: Bi-level Optimization Based Finetuning of the Segment Anything Model for Overfitting-Preventing Semantic Segmentation

https://openreview.net/forum?id=qRtM5EqE9l

Compressor summary: BLO-SAM is a model that improves semantic segmentation by optimizing prompt embeddings and using bi-level optimization to reduce overfitting and enable autonomous object segmentation.

Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning

https://openreview.net/forum?id=qQjUgItPq4

Compressor summary: DiCo is a method for controlling behavioral diversity in multi-agent systems by adjusting policy components, without changing the learning objective, and improving performance and sample efficiency.

BiLLM: Pushing the Limit of Post-Training Quantization for LLMs

https://openreview.net/forum?id=qOl2WWOqFg

Compressor summary: BiLLM is a 1-bit post-training quantization scheme that compresses pretrained LLMs while preserving their performance and efficiency.

DynSyn: Dynamical Synergistic Representation for Efficient Learning and Control in Overactuated Embodied Systems

https://openreview.net/forum?id=qOMQ0UGLYl

Compressor summary: DynSyn is a deep reinforcement learning algorithm that uses synergistic representations of muscle actuators derived from dynamical structures to improve motor control in high-dimensional, overactuated systems.

Cluster-Aware Similarity Diffusion for Instance Retrieval

https://openreview.net/forum?id=qMG3OK7Xcg

Compressor summary: The paper proposes a novel method called Cluster-Aware Similarity for instance retrieval, which reduces misinformation propagation by diffusing similarity within local clusters instead of using pairwise instances.

Learning from Streaming Data when Users Choose

https://openreview.net/forum?id=qLZ32oS7j2

Compressor summary: The paper proposes a decentralized algorithm to optimize user choice and service provider models in digital markets, and proves its convergence and effectiveness.

GiLOT: Interpreting Generative Language Models via Optimal Transport

https://openreview.net/forum?id=qKL25sGjxL

Compressor summary: The paper introduces GiLOT, a method to measure and explain the impact of each word in large language models using Optimal Transport and token similarity.

On Interpolating Experts and Multi-Armed Bandits

https://openreview.net/forum?id=qIiPM5CbRY

Compressor summary: The paper studies a family of online decision problems that interpolate between learning with expert advice and multi-armed bandit, and provides tight minimax regret bounds and an optimal PAC algorithm for a special case called $\mathbf m$-BAI.

Graph Neural Network Explanations are Fragile

https://openreview.net/forum?id=qIOSNyPPwB

Compressor summary: The text discusses adversarial attacks on graph neural network explainers, showing that they can be easily manipulated by slightly perturbing the graph structure.

The Effect of Weight Precision on the Neuron Count in Deep ReLU Networks

https://openreview.net/forum?id=qHt8FzPvU9

Compressor summary: This paper analyzes how weight precision affects ReLU neural networks in terms of number of neurons and preprocessing time, presenting an exponential algorithm to reduce neurons at the cost of precision, and showing that high precision alone doesn't help in reducing neurons.

Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networks

https://openreview.net/forum?id=qGEEso256L

Compressor summary: The text describes a new method for predicting molecular properties using 2D and 3D representations of molecules integrated with a novel aggregation mechanism that is invariant under Euclidean transformations.

AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls

https://openreview.net/forum?id=qFILbkTQWw

Compressor summary: AnyTool is a powerful language model agent that uses over 16,000 APIs to address user queries and improves upon previous evaluation protocols with its own benchmark, AnyToolBench.

Operator SVD with Neural Networks via Nested Low-Rank Approximation

https://openreview.net/forum?id=qESG5HaaoJ

Compressor summary: The paper introduces a new neural network approach to compute eigenvalue decomposition efficiently and accurately using low-rank approximation and nesting techniques.

Estimating Unknown Population Sizes Using the Hypergeometric Distribution

https://openreview.net/forum?id=qE4nkfyMYl

Compressor summary: The paper proposes a novel method to estimate discrete distributions in multivariate hypergeometric sampling when both population size and category sizes are unknown, using variational autoencoder framework and showing its applications in NLP and biology.

Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertainty

https://openreview.net/forum?id=qDw4FxMubj

Compressor summary: This paper proposes a sample-efficient model-based algorithm for learning robust equilibria in distributionally robust Markov games, where agents learn policies that perform well under various environmental uncertainties.

Model-Based Minimum Bayes Risk Decoding for Text Generation

https://openreview.net/forum?id=qDUaH9xHVV

Compressor summary: Model-based MBR decoding improves text generation by using the model probability instead of a Monte Carlo estimate for hypothesis selection.

ACM-MILP: Adaptive Constraint Modification via Grouping and Selection for Hardness-Preserving MILP Instance Generation

https://openreview.net/forum?id=qDAAMmGsGw

Compressor summary: The paper proposes ACM-MILP, a framework that generates better Mixed-Integer Linear Programming (MILP) instances by adaptively modifying constraints and modeling their interrelations.

tinyBenchmarks: evaluating LLMs with fewer examples

https://openreview.net/forum?id=qAml3FpfhG

Compressor summary: The paper proposes strategies to reduce the number of evaluations needed for testing large language models on various benchmarks by using curated examples and releasing evaluation tools and smaller versions of popular benchmarks.

MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilization

https://openreview.net/forum?id=q6fXuPLpao

Compressor summary: MLIP improves CLIP by using multiple domains and levels of supervision, token merging, and frequency transforms to enhance multimodal learning efficiency and performance.

Byzantine Resilient and Fast Federated Few-Shot Learning

https://openreview.net/forum?id=q5q59s2WJy

Compressor summary: The paper presents a Byzantine resilient algorithm for learning low-dimensional linear representation in a federated setting, with improved efficiency and security compared to existing methods.

Disguised Copyright Infringement of Latent Diffusion Models

https://openreview.net/forum?id=q5Bg858Hef

Compressor summary: The paper discusses concealed copyright infringement by generative models that use disguised copies of protected data for training and proposes methods to detect and prevent it.

Classification Under Strategic Self-Selection

https://openreview.net/forum?id=q3Bz1TVTq4

Compressor summary: The text discusses how users strategically choose whether to participate in predictive systems based on favorable outcomes, and proposes a framework for learning from such self-selected populations.

An Effective Dynamic Gradient Calibration Method for Continual Learning

https://openreview.net/forum?id=q14AbM4kdv

Compressor summary: The paper proposes an algorithm to calibrate gradients for continual learning, reducing catastrophic forgetting and improving performance when historical data is limited.

Sparse-to-dense Multimodal Image Registration via Multi-Task Learning

https://openreview.net/forum?id=q0vILV7zAw

Compressor summary: The paper presents SDME, a multi-task network that combines sparse and dense feature matching for robust multimodal image registration, achieving excellent results on various datasets.

Disentanglement Learning via Topology

https://openreview.net/forum?id=q0lxAs5GGO

Compressor summary: TopDis is a novel method for learning disentangled data representations using a multi-scale topological loss that improves various disentanglement scores and works unsupervised, even for correlated factors of variation.

Probabilistic Routing for Graph-Based Approximate Nearest Neighbor Search

https://openreview.net/forum?id=pz4B2kHVKo

Compressor summary: The paper proposes a probabilistic routing method called PEOs that improves the efficiency of approximate nearest neighbor search in high-dimensional spaces using graph-based approaches.

Latent Logic Tree Extraction for Event Sequence Explanation from LLMs

https://openreview.net/forum?id=pwfcwEqdUz

Compressor summary: Key points: - The goal is to use Large Language Models (LLMs) to provide explanations for high-stakes event sequences. - The method builds on the temporal point process model and uses the likelihood function as a score for logic trees. - The approach combines an amortized EM learning framework with a GFlowNet generator for structured discrete variables. - The online setting extracts relevant rules from LLMs for each sequence in few iterations. Summary: The paper proposes a method to use LLMs for explanations of high-stakes event sequences, using a score based on the likelihood function and an EM learning framework with a GFlowNet generator.

DiNADO: Norm-Disentangled Neurally-Decomposed Oracles for Controlling Language Models

https://openreview.net/forum?id=pvg1OdUtDQ

Compressor summary: DiNADO is an improved version of NADO for controllable language generation that addresses challenges like gradient vanishing, limited capacity, and allows combination with finetuning methods.

Relational Learning in Pre-Trained Models: A Theory from Hypergraph Recovery Perspective

https://openreview.net/forum?id=puSMYmHmJW

Compressor summary: The paper proposes a mathematical model that represents the world as a hypergraph and studies how pre-training of foundation models can recover this structure using graph theory concepts.

DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents

https://openreview.net/forum?id=psup68MBvt

Compressor summary: DisCo-Diff is a new diffusion model that simplifies the learning process by adding discrete latents, improving performance in various tasks.

The Balanced-Pairwise-Affinities Feature Transform

https://openreview.net/forum?id=pspyQm4ko0

Compressor summary: The BPA feature transform uses optimal transport optimization to efficiently and effectively represent high order relations between input features for various tasks like few-shot classification, clustering, and person re-identification.

In-Context Reinforcement Learning for Variable Action Spaces

https://openreview.net/forum?id=pp3v2ch5Sd

Compressor summary: Headless-AD is a single-trained model that can adapt to different action spaces without retraining or data collection.

Fair Off-Policy Learning from Observational Data

https://openreview.net/forum?id=poEPRuNvM3

Compressor summary: Key points: - The paper proposes a framework for fair off-policy learning from observational data under different notions of fairness - The framework applies to actions or policy values as measures of fairness - The framework has theoretical guarantees and is tested on simulated and real-world data Summary: The paper presents a novel approach to learn fair decision rules from observational data in off-policy learning settings, with different fairness criteria and theoretical guarantees.

An Intrinsic Vector Heat Network

https://openreview.net/forum?id=po4NsL9KvX

Compressor summary: The paper presents a neural network architecture for learning tangent vector fields on surface manifolds that preserves intrinsic properties using a trainable vector heat diffusion module and vector-valued neurons.

More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms

https://openreview.net/forum?id=pmsPKIBAu6

Compressor summary: The text introduces a new framework for studying meta-learning methods using PAC-Bayesian theory, which allows more flexibility and directness in transferring knowledge between tasks, and demonstrates its effectiveness in both theory and practice.

Agent-Specific Effects: A Causal Effect Propagation Analysis in Multi-Agent MDPs

https://openreview.net/forum?id=pmncWWkGMz

Compressor summary: The paper proposes agent-specific effects (ASE) to measure how an agent's action affects the outcome by influencing other agents and introduces counterfactual counterpart (cf-ASE) for identifying and estimating it.

Graph Out-of-Distribution Detection Goes Neighborhood Shaping

https://openreview.net/forum?id=pmcusTywXO

Compressor summary: TopoOOD is a method for detecting out-of-distribution node instances on graphs that considers graph topology and neighborhood context, and shows improved performance over existing approaches.

Emergent Equivariance in Deep Ensembles

https://openreview.net/forum?id=plXXbXjvQ9

Compressor summary: Deep ensembles become equivariant for all inputs and architectures with data augmentation, and this emergent property holds off-manifold in the infinite width limit.

Efficient Denoising Diffusion via Probabilistic Masking

https://openreview.net/forum?id=pktvuR7b5v

Compressor summary: EDDPM is an efficient denoising diffusion method that uses probabilistic masking to identify and skip redundant steps during training, improving inference efficiency without losing important information.

Robust Inverse Constrained Reinforcement Learning under Model Misspecification

https://openreview.net/forum?id=pkUl39b0in

Compressor summary: The paper proposes a method to learn safe policies from expert demonstrations by inferring robust constraints that account for environmental differences, and evaluates it in continuous and discrete domains.

Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models

https://openreview.net/forum?id=piujJIF3zs

Compressor summary: Model Tailor is a method to reduce catastrophic forgetting in multi-modal language models by replacing a small portion of fine-tuned parameters and improving performance on both original and new tasks.

GPT-4V(ision) is a Generalist Web Agent, if Grounded

https://openreview.net/forum?id=piecKJ2DlB

Compressor summary: The paper explores using large multimodal models like GPT-4V as generalist web agents that can follow natural language instructions on websites, and proposes SEEACT to evaluate them on the MIND2WEB benchmark.

DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching

https://openreview.net/forum?id=phGHQOKmaU

Compressor summary: DiffStitch is a data augmentation method that connects low-reward trajectories with high-reward ones, improving offline reinforcement learning performance across different methods.

Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error

https://openreview.net/forum?id=pgI9inG2Ny

Compressor summary: This paper studies the optimal robust policy for deep reinforcement learning agents under state perturbations, proposes a new consistent adversarial robust deep Q-network algorithm, and shows its effectiveness in various benchmarks.

Translation Equivariant Transformer Neural Processes

https://openreview.net/forum?id=pftXzp6Yn3

Compressor summary: The text introduces a new family of translation equivariant neural processes (TE-TNPs) that improve spatio-temporal modelling by leveraging symmetries in posterior predictive maps.

A General Online Algorithm for Optimizing Complex Performance Metrics

https://openreview.net/forum?id=pfnBLXgFVS

Compressor summary: Key points: - The paper proposes a general online algorithm for optimizing non-decomposable performance metrics based on confusion matrices - The algorithm is simple, efficient, and works for different types of classification problems - The algorithm achieves sublinear regret for concave and smooth metrics and performs well in experiments Summary: The paper presents an online algorithm that optimizes complex performance metrics using confusion matrices, with simplicity, efficiency, and low regret guarantees.

Open-Vocabulary Calibration for Fine-tuned CLIP

https://openreview.net/forum?id=pY2UpspnBB

Compressor summary: This paper proposes Distance-Aware Calibration (DAC), a simple and effective method to improve confidence calibration in vision-language models for open-vocabulary tasks using prompt learning.

Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation

https://openreview.net/forum?id=pXaEYzrFae

Compressor summary: The paper introduces DOMINO, a new algorithm that enforces constraints on large language models without sacrificing accuracy or speed.

Position: Understanding LLMs Requires More Than Statistical Generalization

https://openreview.net/forum?id=pVyOchWUBa

Compressor summary: This paper argues that non-identifiability in AR probabilistic models is a separate theoretical explanation for the desirable qualities of large language models, and discusses its relevance through three case studies.

SelMatch: Effectively Scaling Up Dataset Distillation via Selection-Based Initialization and Partial Updates by Trajectory Matching

https://openreview.net/forum?id=pTFud6SetK

Compressor summary: SelMatch is a novel dataset distillation method that effectively scales with the number of images per class and improves performance on image classification tasks.

Subgoal-based Demonstration Learning for Formal Theorem Proving

https://openreview.net/forum?id=pSnhA7Em1P

Compressor summary: The paper proposes a subgoal-based learning framework that improves the performance of large language models in formal theorem proving by using demonstrative examples and diffusion models for organization.

Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting

https://openreview.net/forum?id=pQyoBWA146

Compressor summary: Split-Ensemble is a novel method that improves uncertainty estimation for deep learning models without extra OOD data or inference costs, using subtask-splitting and feature sharing.

Class-Imbalanced Graph Learning without Class Rebalancing

https://openreview.net/forum?id=pPnkpvBeZN

Compressor summary: This paper proposes a topological augmentation framework called BAT that can effectively mitigate class imbalance bias in graph learning without reweighting or resampling.

Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraints

https://openreview.net/forum?id=pPNMhdYMaz

Compressor summary: Key points: - The paper studies multi-agent reinforcement learning with adaptivity constraints - It proposes a policy elimination algorithm that achieves low regret and batch complexity - It proves lower bounds for such algorithms and extends to related problems Summary: The paper presents a near optimal algorithm and analysis for multi-agent reinforcement learning with adaptivity constraints, and shows its applicability to bandit games and reward-free MARL.

SILVER: Single-loop variance reduction and application to federated learning

https://openreview.net/forum?id=pOgMluzEIH

Compressor summary: SILVER is a new optimization method for non-convex problems that reduces computation time and achieves optimal gradient complexity without needing multiple full gradients.

Efficient Algorithms for Empirical Group Distributionally Robust Optimization and Beyond

https://openreview.net/forum?id=pOJbk4Nzmi

Compressor summary: The paper proposes a new algorithm called ALEG for solving empirical Group Distributionally Robust Optimization (GDRO), which minimizes the maximal risk across multiple groups and outperforms existing methods.

Mechanistic Neural Networks for Scientific Machine Learning

https://openreview.net/forum?id=pLtuwhoQh7

Compressor summary: Mechanistic Neural Networks use a new block to learn differential equations and improve interpretability and efficiency in scientific data modeling.

O$n$ Learning Deep O($n$)-Equivariant Hyperspheres

https://openreview.net/forum?id=pFWmHUdJE5

Compressor summary: Key points: - Paper proposes O$(n)$-equivariant neurons with spherical decision surfaces for learning deep features under nd reflections and rotations - Neurons generalize to any dimension n and are called Deep Equivariant Hyperspheres - Network combines them using an invariant operator based on the relation between two points and a sphere - Approach outperforms competing methods on O$(n)$-equivariant benchmark datasets Summary: The paper introduces Deep Equivariant Hyperspheres, neurons that learn deep features under nd reflections and rotations with spherical decision surfaces. The network uses an invariant operator and shows superior performance on O$(n)$-equivariant tasks.

Better & Faster Large Language Models via Multi-token Prediction

https://openreview.net/forum?id=pEWAcejiU2

Compressor summary: The paper proposes training language models to predict multiple future tokens simultaneously, improving sample efficiency, downstream capabilities, and inference speed for both code and natural language models.

SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning

https://openreview.net/forum?id=pDoAjdrMf0

Compressor summary: The paper investigates transfer reinforcement learning using successor features and generalized policy improvement, providing convergence analysis and generalization guarantees for this approach.

Revealing Vision-Language Integration in the Brain with Multimodal Networks

https://openreview.net/forum?id=pD9BTIDUoX

Compressor summary: The authors use (multi)modal DNNs to identify brain regions where multimodal integration occurs by predicting SEEG recordings from vision and language models.

RAUCA: A Novel Physical Adversarial Attack on Vehicle Detectors via Robust and Accurate Camouflage Generation

https://openreview.net/forum?id=pBTLGM9uWx

Compressor summary: RAUCA is a method for generating effective adversarial camouflage against vehicle detectors, using a novel neural rendering component (NRP) and a multi-weather dataset.

QORA: Zero-Shot Transfer via Interpretable Object-Relational Model Learning

https://openreview.net/forum?id=pAzDdYzEva

Compressor summary: QORA is an algorithm that uses a domain-agnostic object-based state representation to construct expressive models for various reinforcement-learning domains, achieving high generalization and interpretability with fewer observations than neural networks.

Stochastic Weakly Convex Optimization beyond Lipschitz Continuity

https://openreview.net/forum?id=pAyX8q1IIn

Compressor summary: The paper proposes new adaptive regularization strategies for stochastic weakly convex optimization that preserve the $\mathcal{O} ( 1 / \sqrt{K})$ convergence rate with a wide class of algorithms, using weak assumptions and showing efficiency and robustness in experiments.

Combinatorial Multivariant Multi-Armed Bandits with Applications to Episodic Reinforcement Learning and Beyond

https://openreview.net/forum?id=pAdI75JG3G

Compressor summary: The paper presents a novel framework for combinatorial multi-armed bandits with multivariant and probabilistically triggering arms, enhancing modeling power and achieving improved results in various applications like episodic reinforcement learning and probabilistic maximum coverage.

Theory of Consistency Diffusion Models: Distribution Estimation Meets Fast Sampling

https://openreview.net/forum?id=pAPykbqUHf

Compressor summary: This paper develops a statistical theory for consistency models that speed up sample generation by merging steps in the diffusion process, achieving similar performance to diffusion models.

RL-CFR: Improving Action Abstraction for Imperfect Information Extensive-Form Games with Reinforcement Learning

https://openreview.net/forum?id=pA2Q5Wfspp

Compressor summary: RL-CFR is a new RL method for dynamic action abstraction in IIEFGs that uses CFR for strategy derivation and achieves higher expected payoff than existing methods.

Generalizing Orthogonalization for Models with Non-Linearities

https://openreview.net/forum?id=p9SMltcfsu

Compressor summary: The paper proposes a method to reduce biases in black-box algorithms like neural networks by correcting non-linearities and handling scalar and tensor-valued predictions.

Inexact Newton-type Methods for Optimisation with Nonnegativity Constraints

https://openreview.net/forum?id=p7gpooFIr3

Compressor summary: The paper proposes extensions of the two-metric projection framework to solve large scale nonconvex optimisation problems with nonnegativity constraints, achieving state-of-the-art convergence rates and competitive practical performance.

Prospective Side Information for Latent MDPs

https://openreview.net/forum?id=p5FIjG9fbs

Compressor summary: This paper studies decision problems where agents receive some weak information about the hidden context and shows that current RL algorithms are not optimal for this setting, proposing a better algorithm with improved sample complexity.

PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control

https://openreview.net/forum?id=p225Od0aYt

Compressor summary: PRISE is a method that uses BPE to compress sequences and learn action abstractions for better skill learning in robotic manipulation tasks.

Nesting Particle Filters for Experimental Design in Dynamical Systems

https://openreview.net/forum?id=p1kDNFs62o

Compressor summary: The paper presents a new Bayesian experimental design method for non-exchangeable data using Inside-Out SMC$^2$ algorithm and particle Markov chain Monte Carlo, which outperforms existing methods on dynamical systems.

MEMORYLLM: Towards Self-Updatable Large Language Models

https://openreview.net/forum?id=p0lKWzdikQ

Compressor summary: MEMORYLLM is a self-updatable language model that can memorize new text knowledge and maintain its performance after many updates.

Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning

https://openreview.net/forum?id=p0MGN0LSnx

Compressor summary: FedType is a framework for federated learning that uses proxy models to exchange information securely, efficiently, and without relying on public data.

On the Origins of Linear Representations in Large Language Models

https://openreview.net/forum?id=otuTw4Mghk

Compressor summary: This paper investigates why large language models encode high-level semantic concepts linearly, and shows that it is due to the loss function and gradient descent.

InterpreTabNet: Distilling Predictive Signals from Tabular Data by Salient Feature Interpretation

https://openreview.net/forum?id=or8BQ4ohGb

Compressor summary: InterpreTabNet uses a latent variable and GPT-4 to learn distinct and sparse feature masks for tabular data, improving interpretability and predictive performance.

Algorithm and Hardness for Dynamic Attention Maintenance in Large Language Models

https://openreview.net/forum?id=opkluZm9gX

Compressor summary: The paper studies a dynamic version of the attention matrix multiplication problem in large language models and provides an algorithm with efficient update and query time.

A New Robust Partial p-Wasserstein-Based Metric for Comparing Distributions

https://openreview.net/forum?id=opieUcKjPa

Compressor summary: The paper introduces a new family of distances called $k$-RPW that combines partial Wasserstein distance with robustness to outliers and faster convergence rate than existing measures, making it suitable for image retrieval tasks.

Theoretical Analysis of Learned Database Operations under Distribution Shift through Distribution Learnability

https://openreview.net/forum?id=oowQ8LPA12

Compressor summary: The paper presents a theoretical framework to analyze the performance of machine learning methods for database operations in dynamic datasets, showing when they can outperform non-learned alternatives.

Position: Explain to Question not to Justify

https://openreview.net/forum?id=ooikIHLHCs

Compressor summary: The text discusses two complementary approaches to Explainable Artificial Intelligence (XAI) - human-oriented explanations (BLUE XAI) and model-oriented explanations (RED XAI), emphasizing the need for more methods in RED XAI to ensure AI safety.

A Theory of Fault-Tolerant Learning

https://openreview.net/forum?id=ooh8tkXKyR

Compressor summary: The paper introduces fault-tolerant PAC learning, a framework to identify robust machine learning models against random and adversarial faults, showing its sample complexity can vary depending on the type of fault.

Mitigating Catastrophic Forgetting in Online Continual Learning by Modeling Previous Task Interrelations via Pareto Optimization

https://openreview.net/forum?id=olbTrkWo1D

Compressor summary: The paper proposes a new continual learning algorithm, POCL, that optimizes past tasks' performance while maintaining the current task's performance, addressing the catastrophic forgetting challenge in machine learning.

Q-value Regularized Transformer for Offline Reinforcement Learning

https://openreview.net/forum?id=ojtddicekd

Compressor summary: Q-value regularized Transformer (QT) combines trajectory modeling with dynamic programming to improve offline reinforcement learning.

Off-policy Evaluation Beyond Overlap: Sharp Partial Identification Under Smoothness

https://openreview.net/forum?id=oiY7yhyi6W

Compressor summary: The paper proposes new methods for off-policy evaluation without overlap or a well-specified model, using Lipschitz smoothness assumptions to provide sharp bounds and optimal estimators.

Optimal bounds for $\ell_p$ sensitivity sampling via $\ell_2$ augmentation

https://openreview.net/forum?id=ohH3sbUue2

Compressor summary: The text describes improved sampling methods for large data sets using sensitivity and subspace embeddings, achieving optimal complexity bounds with better guarantees than previous approaches.

Generating In-Distribution Proxy Graphs for Explaining Graph Neural Networks

https://openreview.net/forum?id=ohG9bVMs5j

Compressor summary: The paper proposes a new method to generate proxy graphs that help explain graph neural networks by preserving explanatory factors and adhering to the distribution of training data.

Generative Active Learning for Long-tailed Instance Segmentation

https://openreview.net/forum?id=ofXRBPtol3

Compressor summary: The paper proposes BSGAL, a new active learning algorithm that uses gradient cache to select batches of generated data for enhancing long-tailed instance segmentation tasks.

Uniformly Stable Algorithms for Adversarial Training and Beyond

https://openreview.net/forum?id=odCl49tWA6

Compressor summary: The paper proposes a new algorithm (ME-$\mathcal{A}$) that improves robustness of neural networks in adversarial machine learning by ensuring uniform stability and mitigating the issue of robust overfitting.

The Relative Value of Prediction in Algorithmic Decision Making

https://openreview.net/forum?id=oaACFfNbXl

Compressor summary: The authors study when and why predictions are more valuable than expanding access or increasing intervention effectiveness in algorithmic decision making for public goods and welfare.

A Linear Time and Space Local Point Cloud Geometry Encoder via Vectorized Kernel Mixture (VecKM)

https://openreview.net/forum?id=oYltxxam2t

Compressor summary: VecKM is an efficient and accurate local point cloud geometry encoder that uses vectorized kernel mixtures and eliminates the need for grouping points into neighbors.

BWS: Best Window Selection Based on Sample Scores for Data Pruning across Broad Ranges

https://openreview.net/forum?id=oWYzIodyC4

Compressor summary: BWS is a universal data subset selection method that efficiently selects informative samples for training neural networks across various selection ratios using difficulty scores and kernel ridge regression.

Foundations of Testing for Finite-Sample Causal Discovery

https://openreview.net/forum?id=oUmXcewb83

Compressor summary: The text discusses a new method for finite-sample causal discovery that combines structured multiple testing and graph skeleton information, allowing efficient verification of graph structure.

Human-like Category Learning by Injecting Ecological Priors from Large Language Models into Neural Networks

https://openreview.net/forum?id=oTmQmaNkGn

Compressor summary: ERMI is a cognitive model that uses large language models to generate ecologically valid tasks and meta-learning to create rational agents, outperforming other models in explaining human behavior and achieving high accuracy on a benchmark task.

Efficient Stochastic Approximation of Minimax Excess Risk Optimization

https://openreview.net/forum?id=oTYuORAMaP

Compressor summary: The paper proposes efficient stochastic approximation methods for minimizing excess risk in minimax optimization problems.

Learning to Explore in POMDPs with Informational Rewards

https://openreview.net/forum?id=oTD3WoQyFR

Compressor summary: The paper proposes a POMDP agent that uses meta-exploration techniques to gather relevant information for completing tasks in partially observed environments, outperforming prior methods when complex strategies are needed.

On the sample complexity of conditional independence testing with Von Mises estimator with application to causal discovery

https://openreview.net/forum?id=oSOZ31ISBV

Compressor summary: The paper proposes a new test for conditional independence using an estimator based on the Von Mises entropy and kernel density estimation, which has better performance than existing methods in causal discovery for non-linear models and non-Gaussian continuous variables.

DRCT: Diffusion Reconstruction Contrastive Training towards Universal Detection of Diffusion Generated Images

https://openreview.net/forum?id=oRLwyayrh1

Compressor summary: The paper proposes DRCT, a framework to improve the generalizability of image detectors for generated images by using high-quality diffusion reconstruction and contrastive learning, and introduces a large dataset with different diffusion models for evaluation.

How to Leverage Diverse Demonstrations in Offline Imitation Learning

https://openreview.net/forum?id=oOlooUu2Sb

Compressor summary: Key points: - The paper proposes a data selection method for offline imitation learning with imperfect demonstrations. - The method selects data based on resultant states, which uses dynamics information and extracts both expert and diverse behaviors. - The paper also presents a behavior cloning algorithm that leverages the selected data. - The method achieves state-of-the-art performance on complex and high-dimensional offline IL benchmarks. Summary: The paper introduces a resultant state-based data selection method and a behavior cloning algorithm for offline imitation learning with imperfect demonstrations, which outperforms existing methods on 20/21 benchmarks.

Preference Optimization for Molecule Synthesis with Conditional Residual Energy-based Models

https://openreview.net/forum?id=oLfq1KKneW

Compressor summary: The paper proposes a framework using conditional residual energy-based models to improve molecule synthesis routes in drug discovery by considering criteria like costs, yields, and step count.

Position: Amazing Things Come From Having Many Good Models

https://openreview.net/forum?id=oFDFGd9Age

Compressor summary: The Rashomon Effect means many good models exist for the same data, influencing various aspects of machine learning and its applications, especially for noisy tabular data problems.

Image Restoration Through Generalized Ornstein-Uhlenbeck Bridge

https://openreview.net/forum?id=oDUJmNCV8D

Compressor summary: The GOUB model uses a generalized OU process to map from low-quality images to high-quality ones, achieving state-of-the-art performance in various image restoration tasks.

KISA: A Unified Keyframe Identifier and Skill Annotator for Long-Horizon Robotics Demonstrations

https://openreview.net/forum?id=oCI9gHocws

Compressor summary: Keyframe Identifier and Skill Annotator (KISA) is a method that uses visual-language representations to accurately and interpretably decompose unlabeled robotic manipulation demonstrations into keyframes and skills.

SPADE: Sparsity-Guided Debugging for Deep Neural Networks

https://openreview.net/forum?id=oBYv73nOoA

Compressor summary: The paper introduces SPADE, a method that uses sample-targeted pruning to provide more accurate and interpretable image saliency maps and neuron visualizations for deep neural networks without affecting their behavior.

VideoPrism: A Foundational Visual Encoder for Video Understanding

https://openreview.net/forum?id=oBP8vXFJNQ

Compressor summary: VideoPrism is a versatile video encoder that uses pretraining on large datasets to perform well on various video understanding tasks.

Learning Latent Space Hierarchical EBM Diffusion Models

https://openreview.net/forum?id=o9uOuIwhZK

Compressor summary: The paper proposes using diffusion probabilistic schemes to improve learning of energy-based priors for multi-layer generative models, addressing the prior hole problem and increasing modelling expressivity.

Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?

https://openreview.net/forum?id=o8AaRKbP9K

Compressor summary: The paper explores how linear looped Transformers can learn and converge to fix-point iterative algorithms like preconditioned gradient descent in linear regression, using a novel theoretical analysis and experiments.

How Spurious Features are Memorized: Precise Analysis for Random and NTK Features

https://openreview.net/forum?id=o6N1Bqay0k

Compressor summary: The paper proposes a theoretical framework to quantify how deep learning models memorize spurious features unrelated to the task and reveals factors affecting this phenomenon.

PairNet: Training with Observed Pairs to Estimate Individual Treatment Effect

https://openreview.net/forum?id=o5SVr80Rgg

Compressor summary: PairNet is a novel training strategy for individual treatment effect estimation that minimizes losses over pairs of examples based on their factual observed outcomes, achieving smaller generalization error than existing methods.

ReLU Network with Width $d+\mathcal{O}(1)$ Can Achieve Optimal Approximation Rate

https://openreview.net/forum?id=o4HF3N6CZR

Compressor summary: The paper proves that ReLU neural networks with a width of $d+1$ can optimally approximate continuous functions over $[0,1]^d$ under $L^p$ norm for $p\in[1,\infty)$, and $d+11$ for the uniform norm.

Interpreting and Improving Diffusion Models from an Optimization Perspective

https://openreview.net/forum?id=o2ND9v0CeK

Compressor summary: This paper shows how denoising diffusion models can be seen as a form of gradient descent, analyzes their convergence, and proposes a new sampler that achieves state-of-the-art performance in image generation.

Using Left and Right Brains Together: Towards Vision and Language Planning

https://openreview.net/forum?id=o1gS6MNAw8

Compressor summary: The authors propose a novel framework for concurrent visual and language planning, which outperforms separate approaches on various tasks, demonstrating the benefits of integrating vision and language.

BRAIn: Bayesian Reward-conditioned Amortized Inference for natural language generation from feedback

https://openreview.net/forum?id=nxzXTLByXO

Compressor summary: BRAIn is a method that improves distribution matching techniques for language model alignment by reducing variance and generalizing the target distribution using Bayes' rule, leading to better performance in summarization and self-attention tasks.

Robustness of Deep Learning for Accelerated MRI: Benefits of Diverse Training Data

https://openreview.net/forum?id=nvfZgdHtHc

Compressor summary: The study shows that using diverse data for training deep learning models improves their performance and robustness in accelerated MRI tasks without sacrificing in-distribution accuracy.

$H$-Consistency Guarantees for Regression

https://openreview.net/forum?id=nvHlHfjJPe

Compressor summary: The paper studies and generalizes tools for proving $H$-consistency bounds, a measure of how well regression algorithms perform, and derives new surrogate losses for adversarial regression with promising results.

Can a Few Decide for Many? The Metric Distortion of Sortition

https://openreview.net/forum?id=nsjfoziR5j

Compressor summary: The paper studies how to select representative sortition panels that reflect the population's opinion using metric distortion and compares two selection algorithms.

EVEREST: Efficient Masked Video Autoencoder by Removing Redundant Spatiotemporal Tokens

https://openreview.net/forum?id=nn5OPHom8t

Compressor summary: EVEREST is an efficient MVA approach for video representation learning that focuses on informative frames, reducing computation and memory requirements.

LLM Maybe LongLM: SelfExtend LLM Context Window Without Tuning

https://openreview.net/forum?id=nkOMLBIiI7

Compressor summary: SelfExtend is a method to extend the context window of language models without fine-tuning by using grouped and neighbor attention mechanisms.

Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models

https://openreview.net/forum?id=njwv9BsGHF

Compressor summary: LATS is a framework that combines language models and Monte Carlo Tree Search to create more capable autonomous agents for various decision-making tasks.

Efficient Exploration in Average-Reward Constrained Reinforcement Learning: Achieving Near-Optimal Regret With Posterior Sampling

https://openreview.net/forum?id=njpTpkvUbO

Compressor summary: The paper introduces a new algorithm for learning in Constrained Markov Decision Processes, which achieves near-optimal regret bounds and performs better than existing methods in simulations.

Momentum Particle Maximum Likelihood

https://openreview.net/forum?id=ngjmcfowtc

Compressor summary: The paper proposes a novel dynamical systems approach to fit latent variable models to data using optimal transport insights, which improves upon existing particle methods and other MLE algorithms.

RLVF: Learning from Verbal Feedback without Overgeneralization

https://openreview.net/forum?id=ngcZhfXCBW

Compressor summary: C3PO is a method that learns from high-level verbal feedback to adjust large language models without overgeneralizing the feedback to unrelated contexts.

Bringing Motion Taxonomies to Continuous Domains via GPLVM on Hyperbolic manifolds

https://openreview.net/forum?id=ndVXXmxSC5

Compressor summary: Key points: - Human motion taxonomies are useful for analysing how humans move and interact with their environment - They have a hierarchical structure but lack computational models that connect them to high-dimensional data - The proposed model uses hyperbolic embeddings and Gaussian processes to capture the taxonomy structure and learn from data - The model outperforms other methods and can generate realistic trajectories Summary: The paper proposes a novel model that uses hyperbolic embeddings and Gaussian processes to learn human motion taxonomies from data and generate realistic trajectories.

Graph-based Time Series Clustering for End-to-End Hierarchical Forecasting

https://openreview.net/forum?id=nd47Za5jk5

Compressor summary: The paper proposes a graph-based method to unify relational and hierarchical inductive biases in deep learning for time series forecasting, using trainable graph pooling operators to learn the hierarchy from data and a differentiable reconciliation stage to balance constraints and predictions.

Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining

https://openreview.net/forum?id=ncjhi4qAPV

Compressor summary: The paper discusses the potential benefits and drawbacks of using non-private models pretrained on public datasets to improve differentially private machine learning.

Conformalized Adaptive Forecasting of Heterogeneous Trajectories

https://openreview.net/forum?id=nbpwNmXTTw

Compressor summary: The paper introduces a new method for generating uncertainty bands that cover the entire path of random trajectories, useful for motion planning applications with unpredictable objects.

A Dynamical Model of Neural Scaling Laws

https://openreview.net/forum?id=nbOY1OmtRc

Compressor summary: The text discusses a random feature model that explains neural scaling laws in network training and generalization, predicting different power law exponents for performance with training time and model size, and showing convergence rates dependent on architecture and task.

video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models

https://openreview.net/forum?id=nYsh5GFIqX

Compressor summary: The paper presents a single end-to-end audio-visual large language model (av-LLM) called video-SALMONN, which improves speech understanding in videos and achieves significant accuracy gains on various audio-visual tasks.

HAMLET: Graph Transformer Neural Operator for Partial Differential Equations

https://openreview.net/forum?id=nYX7I6PsL7

Compressor summary: HAMLET is a graph transformer framework that uses modular input encoders to solve PDEs using neural networks, achieving robustness and adaptability across different domains and data scenarios.

Double-Step Alternating Extragradient with Increasing Timescale Separation for Finding Local Minimax Points: Provable Improvements

https://openreview.net/forum?id=nUVForc3VP

Compressor summary: The paper proposes two improvements to existing two-timescale gradient methods for nonconvex-nonconcave minimax optimization, addressing instability and timescale separation challenges in overparameterized settings.

STEER: Assessing the Economic Rationality of Large Language Models

https://openreview.net/forum?id=nU1mtFDtMX

Compressor summary: The paper presents a methodology for assessing the economic rationality of LLMs as decision-making agents by surveying the literature, proposing a benchmark distribution, and conducting an empirical experiment.

Predictive Coding beyond Correlations

https://openreview.net/forum?id=nTgzmXvuEA

Compressor summary: This paper demonstrates how predictive coding, a biological learning algorithm, can do causal inference by modifying its inference process without changing the causal graph and applying it to image classification tasks.

Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning

https://openreview.net/forum?id=nSGnx8lNJ6

Compressor summary: The paper proposes a method to improve offline reinforcement learning by combining discrete-time and continuous-time RL, using the value function's first derivative to better predict unvisited states.

Thermometer: Towards Universal Calibration for Large Language Models

https://openreview.net/forum?id=nP7Q1PnuLK

Compressor summary: THERMOMETER is a calibration approach that uses an auxiliary model to improve the accuracy and reliability of large language models for diverse tasks.

Viewing Transformers Through the Lens of Long Convolutions Layers

https://openreview.net/forum?id=nOyj26YdIQ

Compressor summary: The paper proposes minimal modifications to the transformer architecture to improve its performance on long-range tasks by incorporating smoothness and locality principles into the attention mechanism.

Unsupervised Representation Learning of Brain Activity via Bridging Voxel Activity and Functional Connectivity

https://openreview.net/forum?id=nOjZfpLyh1

Compressor summary: BrainMixer is an unsupervised learning framework that leverages voxel-level activity and functional connectivity to represent the brain effectively, performing better than existing methods in cognitive tasks and neurological diagnosis.

SHINE: Shielding Backdoors in Deep Reinforcement Learning

https://openreview.net/forum?id=nMWxLnSBGW

Compressor summary: SHINE is a new method for protecting deep reinforcement learning agents from backdoor attacks by identifying and removing triggers in the agent's policy.

StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization

https://openreview.net/forum?id=nMN5hNZMQK

Compressor summary: The paper examines how different ways of parameterizing state-space models affect their memory learning abilities and introduces new reparameterization techniques to overcome the memory limitations of these models.

Completing Visual Objects via Bridging Generation and Segmentation

https://openreview.net/forum?id=nLgtHHBgl3

Compressor summary: The paper proposes MaskComp, a method that reconstructs incomplete objects using iterative generation and segmentation stages with a mask condition to improve image quality and refine the object mask.

Merging Multi-Task Models via Weight-Ensembling Mixture of Experts

https://openreview.net/forum?id=nLRKnO74RB

Compressor summary: The paper proposes a dynamic weight-ensembling module for merging task-specific Transformer models, which can adapt to different tasks and reduce parameter interference.

Adaptive Online Experimental Design for Causal Discovery

https://openreview.net/forum?id=nJzf3TVnOn

Compressor summary: The text proposes a new causal discovery algorithm that efficiently learns cause-and-effect relationships from interventional data using adaptive interventions and sampling history.

Fourier Controller Networks for Real-Time Decision-Making in Embodied Learning

https://openreview.net/forum?id=nDps3Q8j2l

Compressor summary: The paper introduces FCNet, a new network that uses the frequency domain to improve data efficiency and inference speed in robotics reinforcement learning, outperforming Transformer on various tasks.

Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learning

https://openreview.net/forum?id=nCZYRBK1J4

Compressor summary: The paper explores using reinforcement learning for online path planning in unknown environments and proposes a new map representation and reward function to improve coverage.

Equivariant Deep Weight Space Alignment

https://openreview.net/forum?id=nBPnmk6EeO

Compressor summary: Deep-Align is a novel framework that learns to solve the weight alignment problem of deep networks without requiring labeled data and improves both speed and quality of alignment compared to existing methods.

Align Your Steps: Optimizing Sampling Schedules in Diffusion Models

https://openreview.net/forum?id=nBGBzV4It3

Compressor summary: The paper proposes a new approach to optimize the sampling schedules of diffusion models for better quality outputs, using methods from stochastic calculus.

Combining Experimental and Historical Data for Policy Evaluation

https://openreview.net/forum?id=nB6ERIud2y

Compressor summary: The paper proposes data integration methods for policy evaluation using experimental and historical data, with optimized weights and pessimistic principle to achieve robustness and efficiency in different reward shift scenarios.

Verifying message-passing neural networks via topology-based bounds tightening

https://openreview.net/forum?id=nAoiUlz4Bf

Compressor summary: Our approach provides robust certificates for message-passing neural networks using ReLU activation, mixed-integer optimization, and topology-based bounds tightening to ensure trustworthiness in the face of various graph attacks.

A fast algorithm to simulate nonlinear resistive networks

https://openreview.net/forum?id=nAbfF37H6t

Compressor summary: The authors propose a fast and exact method for simulating nonlinear resistive networks, enabling efficient training and reducing the simulation bottleneck.

Position: Enforced Amnesia as a Way to Mitigate the Potential Risk of Silent Suffering in the Conscious AI

https://openreview.net/forum?id=nACGn4US1R

Compressor summary: The paper suggests a way to prevent possible silent suffering in AI by limiting their access to memory or resetting it regularly, even without confirming their consciousness.

Scaling Down Deep Learning with MNIST-1D

https://openreview.net/forum?id=n9pru4bJU9

Compressor summary: MNIST-1D is a minimalist, procedurally generated deep learning benchmark that enables various experiments and research on low-memory and low-compute platforms.

Decoding-time Realignment of Language Models

https://openreview.net/forum?id=n8g6WMxt09

Compressor summary: DeRa is a method for exploring different levels of regularization in aligned language models without retraining, allowing users to control alignment and improve efficiency.

Parameterized Physics-informed Neural Networks for Parameterized PDEs

https://openreview.net/forum?id=n3yYrtt9U7

Compressor summary: P$^2$INNs are a new extension of physics-informed neural networks that can efficiently model solutions of parameterized partial differential equations, improving accuracy and efficiency on benchmark problems.

Solving Hierarchical Information-Sharing Dec-POMDPs: An Extensive-Form Game Approach

https://openreview.net/forum?id=n3smZl8itR

Compressor summary: The paper proposes a method to simplify complex multi-player games by breaking them down into smaller subgames and applying Bellman's principle of optimality, improving scalability and efficiency.

Learning the Uncertainty Sets of Linear Control Systems via Set Membership: A Non-asymptotic Analysis

https://openreview.net/forum?id=n2kq2EOHFE

Compressor summary: The paper investigates set membership estimation for unknown linear systems, providing the first convergence rate bounds and demonstrating its practical potential.

Multiply-Robust Causal Change Attribution

https://openreview.net/forum?id=n2eppIzHlL

Compressor summary: The paper proposes a new estimation strategy that combines regression and re-weighting methods to quantify the contribution of each causal mechanism in explaining the change in an outcome variable distribution.

Rejuvenating image-GPT as Strong Visual Representation Learners

https://openreview.net/forum?id=mzGtunvpJH

Compressor summary: The paper introduces D-iGPT, a new approach to learn visual representations from images by predicting semantic tokens and visible pixels with autoregressive pretraining using CLIP-based models.

Bayesian Regret Minimization in Offline Bandits

https://openreview.net/forum?id=mz55Ox0Igz

Compressor summary: The paper proposes a new algorithm for offline linear bandits that minimizes Bayesian regret using efficient conic optimization solvers and shows its superiority over the maximum lower confidence bound approach.

BeigeMaps: Behavioral Eigenmaps for Reinforcement Learning from Images

https://openreview.net/forum?id=myCgfQZzbc

Compressor summary: The paper proposes a new method called Behavioral Eigenmaps (BeigeMaps) for learning representations in reinforcement learning agents from high-dimensional image observations that group similar states and improve policy performance.

Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?

https://openreview.net/forum?id=mxjB0LIgpT

Compressor summary: The paper discusses challenges and insights related to using evidential deep learning methods for quantifying uncertainty in ML systems.

Learning Surrogates for Offline Black-Box Optimization via Gradient Matching

https://openreview.net/forum?id=mv9beA1wDF

Compressor summary: The text discusses a problem in offline design optimization and proposes a new algorithm to improve surrogate models' accuracy by matching the latent gradient field in the data.

Soft Prompt Recovers Compressed LLMs, Transferably

https://openreview.net/forum?id=muBJPCIqZT

Compressor summary: The authors show that using soft prompts can improve the performance of compressed large language models, making them more accessible without additional engineering efforts.

Gambling-Based Confidence Sequences for Bounded Random Vectors

https://openreview.net/forum?id=mu7Er7f9NQ

Compressor summary: The paper introduces a new method to construct confidence sequences for multivariate stochastic processes using a general gambling framework, which improves upon existing methods in terms of tightness.

Major-Minor Mean Field Multi-Agent Reinforcement Learning

https://openreview.net/forum?id=mslTE1qgLa

Compressor summary: The paper introduces Major-Minor Mean Field Control (M3FC), a generalization of Mean Field Control that models many similar and few complex agents, and proposes an M3FMARL algorithm that approximates the policy gradient of the M3FC MDP and outperforms state-of-the-art methods in various scenarios.

Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning

https://openreview.net/forum?id=mrd4e8ZJjm

Compressor summary: The paper proposes a novel dynamics model for reinforcement learning that infers fine-grained causal structures using discrete latent variables, improving robustness and performance in downstream tasks.

Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective

https://openreview.net/forum?id=mphq2jMFLZ

Compressor summary: The paper investigates how two-layer neural networks can efficiently learn multiple kernel spaces and functions using mean-field Langevin dynamics in a two-timescale limit.

Repoformer: Selective Retrieval for Repository-Level Code Completion

https://openreview.net/forum?id=moyG54Okrj

Compressor summary: The paper proposes a selective RAG framework that uses self-supervised learning to decide when to retrieve contexts for code completion, achieving better performance and efficiency than existing methods.

Double Stochasticity Gazes Faster: Snap-Shot Decentralized Stochastic Gradient Tracking Methods

https://openreview.net/forum?id=mkbSXxovP5

Compressor summary: The text proposes two new decentralized optimization algorithms, snap-shot DSGT and accelerated snap-shot DSGT, which improve upon the existing DSGT method by using snapshot gradient tracking and achieving better convergence properties in different network topologies.

GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding

https://openreview.net/forum?id=mk8oRhox2l

Compressor summary: GliDe and CaPE are methods to speed up decoding of large language models by reusing cached keys and values and using confidence scores for token selection.

Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

https://openreview.net/forum?id=mk3A5IUdn8

Compressor summary: Caduceus is a new family of DNA language models that can handle long-range interactions, bi-directionality, and reverse complementarity, improving performance on downstream tasks compared to larger models without these features.

Bounded and Uniform Energy-based Out-of-distribution Detection for Graphs

https://openreview.net/forum?id=mjh7AOWozN

Compressor summary: NODESAFE improves GNNs' ability to detect out-of-distribution data in graphs by bounding negative energy scores and mitigating logit shifts, achieving better performance than previous methods.

LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models

https://openreview.net/forum?id=mhI5nc5QwX

Compressor summary: Key points: - The study observes low-rank structure in MHA sub-layer of Transformer, but not in FFN sub-layer - LoRAP is a novel method that combines low-rank approximation and structured pruning for LLMs - LoRAP outperforms previous methods on zero-shot perplexity and task classification Summary: The study proposes LoRAP, a new compression method for large language models that uses low-rank approximation and structured pruning, and shows its superior performance on various tasks.

Privacy Attacks in Decentralized Learning

https://openreview.net/forum?id=mggc3oYHy4

Compressor summary: The paper proposes a new attack on Decentralized Gradient Descent (D-GD) that allows an attacker to reconstruct private data of other users by exploiting the gossip averaging protocol.

On Online Experimentation without Device Identifiers

https://openreview.net/forum?id=merZTLSdC9

Compressor summary: HIFIVE is a variational method for accurately estimating human preferences and causal effects from fragmented device identifiers in online behaviors.

Unsupervised Domain Adaptation for Anatomical Structure Detection in Ultrasound Images

https://openreview.net/forum?id=meItvvCO7X

Compressor summary: ToMo-UDA is a new method for detecting fetal structures in ultrasound images that uses topology and morphology knowledge to overcome challenges due to differences between institutions and overlapping structures.

Position: Tensor Networks are a Valuable Asset for Green AI

https://openreview.net/forum?id=mcg6jppkwb

Compressor summary: The paper argues that tensor networks can enhance both sustainability and inclusivity in AI research by providing mathematical rigor and efficient compression.

A2Q+: Improving Accumulator-Aware Weight Quantization

https://openreview.net/forum?id=mbx2pLK5Eq

Compressor summary: A2Q+ is an improved quantization method for neural networks that avoids numerical overflow, reduces quantization error, and uses weight normalization.

Distributionally Robust Data Valuation

https://openreview.net/forum?id=mbBehLOAqR

Compressor summary: The paper proposes a method to value data for machine learning models without knowing the validation dataset, using distributionally robust generalization error and model deviation as measures, and shows its effectiveness on real-world datasets.

HumanTOMATO: Text-aligned Whole-body Motion Generation

https://openreview.net/forum?id=maVIKlGqr7

Compressor summary: The paper introduces HumanTOMATO, a framework for generating realistic whole-body motions from text descriptions, addressing the limitations of previous methods by using a hierarchical VQ-VAE and a Hierarchical-GPT model.

Low-Rank Similarity Mining for Multimodal Dataset Distillation

https://openreview.net/forum?id=mY93trX2Qz

Compressor summary: The paper proposes LoRS, a method to create synthetic data from image-text pairs for visual-language dataset distillation, which improves existing algorithms and focuses on modality correspondence.

Reinforcement Learning from Reachability Specifications: PAC Guarantees with Expected Conditional Distance

https://openreview.net/forum?id=mXUDDL4r1Q

Compressor summary: The text discusses a fundamental problem in sequential decision making, presenting lower bounds and an algorithm for Reinforcement Learning (RL) from reachability specifications using expected conditional distance (ECD).

Beyond the ROC Curve: Classification Trees Using Cost-Optimal Curves, with Application to Imbalanced Datasets

https://openreview.net/forum?id=mXLcbRBA8v

Compressor summary: The paper proposes an algorithm for building oblique classification trees that optimizes a loss function based on minimizing false negatives subject to a maximum false positive rate, which improves accuracy and interpretability in imbalanced datasets with class costs.

Spider: A Unified Framework for Context-dependent Concept Segmentation

https://openreview.net/forum?id=mWV8NeU79e

Compressor summary: The text introduces Spider, a unified model that can understand and distinguish various context-dependent concepts in different domains, outperforming existing specialized models and enabling continuous learning.

Adaptive Accompaniment with ReaLchords

https://openreview.net/forum?id=mUVydzrkgz

Compressor summary: ReaLchords is an online generative model for improvising chord accompaniment to user melody that uses reinforcement learning and a novel reward model for harmonic and temporal coherency.

Privacy-Preserving Instructions for Aligning Large Language Models

https://openreview.net/forum?id=mUT1biz09t

Compressor summary: The text proposes using synthetic instructions to protect user privacy in language model applications, and shows that this method achieves high utility in both supervised and reinforcement learning settings.

WebLINX: Real-World Website Navigation with Multi-Turn Dialogue

https://openreview.net/forum?id=mUSPhG4uDW

Compressor summary: The paper introduces WEBLINX, a benchmark for conversational web navigation, and compares different models' performance in this task.

PruNeRF: Segment-Centric Dataset Pruning via 3D Spatial Consistency

https://openreview.net/forum?id=mU7FfQT6VE

Compressor summary: PruNeRF is a framework to prune distracting objects from NeRF training data using 3D spatial consistency, segmentation, and depth-based reprojection.

Overcoming Data and Model heterogeneities in Decentralized Federated Learning via Synthetic Anchors

https://openreview.net/forum?id=mNzkumTSVL

Compressor summary: Key points: - Decentralized FL is a serverless network where clients train local models separately - This may reduce model generalizability due to data and model heterogeneity among clients - DeSA introduces synthetic global anchors based on raw data distribution to facilitate knowledge transfer and regularization Summary: DeSA is a decentralized FL technique that uses synthetic global anchors to enhance the generalizability of local models trained separately by clients.

Rethinking Data Shapley for Data Selection Tasks: Misleads and Merits

https://openreview.net/forum?id=mKYBMf1hHG

Compressor summary: The study explores the inconsistency of Data Shapley's performance in data selection tasks and identifies a class of utility functions where it works optimally, proposing a heuristic for predicting its effectiveness.

Studying K-FAC Heuristics by Viewing Adam through a Second-Order Lens

https://openreview.net/forum?id=mK6FB9xQ7v

Compressor summary: The paper compares the performance of a new optimizer, AdamQLR, which combines elements of first-order (Adam) and second-order (K-FAC) methods for deep learning optimization.

What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding

https://openreview.net/forum?id=mJhXlsZzzE

Compressor summary: This study analyzes the sample complexity and convergence of a shallow Graph Transformer for semi-supervised node classification, showing how self-attention and positional encoding improve generalization.

Challenges in Training PINNs: A Loss Landscape Perspective

https://openreview.net/forum?id=mJGiFr8jLa

Compressor summary: The paper investigates challenges in training Physics-Informed Neural Networks (PINNs), compares different optimizers, and proposes a new second-order optimizer to improve their performance.

Rethinking Optimization and Architecture for Tiny Language Models

https://openreview.net/forum?id=mHIEOZtDDF

Compressor summary: The authors study various optimization techniques for tiny language models and achieve significant improvement in performance compared to baseline models.

On Hypothesis Transfer Learning of Functional Linear Models

https://openreview.net/forum?id=mGsF8Q0fGZ

Compressor summary: The paper proposes two transfer learning algorithms for functional linear regression, using RKHS distance to measure similarity and analyze dynamics, with empirical results on synthetic and real data.

A Multimodal Automated Interpretability Agent

https://openreview.net/forum?id=mDw42ZanmE

Compressor summary: MAIA is a system that uses neural models to automate understanding of other neural models' behavior, especially for vision tasks, by providing tools such as synthesizing inputs, computing activating exemplars, and summarizing results.

Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion

https://openreview.net/forum?id=mCzyRdDak5

Compressor summary: The paper presents two zero-shot audio editing techniques using pre-trained diffusion models, ZETA for text-based edits and ZEUS for semantically meaningful unsupervised modifications in music signals.

Reinformer: Max-Return Sequence Modeling for Offline RL

https://openreview.net/forum?id=mBc8Pestd5

Compressor summary: The paper introduces Reinforcer, a new sequence model for offline RL that maximizes returns and improves trajectory stitching.

Smooth Min-Max Monotonic Networks

https://openreview.net/forum?id=m8t1yzfBsJ

Compressor summary: The smooth min-max (SMM) network module is a simple modification of the existing min-max architecture that ensures monotonicity, improves training stability, and maintains good generalization performance in data-driven models.

Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency

https://openreview.net/forum?id=m8lCi7rG4u

Compressor summary: This paper investigates the causes of overfitting in adversarial training, finding that it's related to shortcuts formed in early layers of neural networks, and proposes a method to mitigate it by perturbing weights across layers.

When Representations Align: Universality in Representation Learning Dynamics

https://openreview.net/forum?id=m5nB7ucXHT

Compressor summary: The study proposes a theory of representation learning in deep neural networks that applies to various architectures and activation functions, showing that some aspects of learning dynamics are similar across them.

Smooth Tchebycheff Scalarization for Multi-Objective Optimization

https://openreview.net/forum?id=m4dO5L6eCp

Compressor summary: The paper proposes a lightweight and efficient smooth Tchebycheff scalarization approach for gradient-based multi-objective optimization with good theoretical properties and lower computational complexity.

Adaptive Conformal Inference by Betting

https://openreview.net/forum?id=lwWV4Zl3h1

Compressor summary: The paper proposes a new method for uncertain predictions in machine learning that doesn't require data assumptions or learning rate tuning.

Degeneration-free Policy Optimization: RL Fine-Tuning for Language Models without Degeneration

https://openreview.net/forum?id=lwTshcWlmB

Compressor summary: DfPO is a fine-tuning method for language models using reinforcement learning that prevents text degeneration and improves downstream task scores by masking KL divergence and using truncated advantage functions.

Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation

https://openreview.net/forum?id=luqH1eL4PN

Compressor summary: BiPE is a new positional encoding method for language sequences that combines intra-segment and inter-segment encodings to improve semantic information capture and extrapolation capabilities.

Jetfire: Efficient and Accurate Transformer Pretraining with INT8 Data Flow and Per-Block Quantization

https://openreview.net/forum?id=ltzTHGFF5i

Compressor summary: Jetfire is an INT8 training method for transformers that optimizes memory access and maintains accuracy while speeding up pretraining and reducing memory usage.

Convergence and Trade-Offs in Riemannian Gradient Descent and Riemannian Proximal Point

https://openreview.net/forum?id=ltb2XaIr9p

Compressor summary: The paper analyzes two important optimization algorithms, gives their convergence rates, shows they stay near optimizers, provides an implementable version, and explores properties of a related method.

CauDiTS: Causal Disentangled Domain Adaptation of Multivariate Time Series

https://openreview.net/forum?id=lsavZkUjFZ

Compressor summary: CauDiTS is a framework for unsupervised domain adaptation of multivariate time series that disentangles causal patterns from correlations to improve classification reliability across domains.

MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts

https://openreview.net/forum?id=lsQnneYa8p

Compressor summary: The paper proposes a multi-task vehicle routing solver with mixture-of-experts and hierarchical gating, which improves generalization and performance on various vehicle routing problems.

DistiLLM: Towards Streamlined Distillation for Large Language Models

https://openreview.net/forum?id=lsHZNNoC7r

Compressor summary: DistiLLM is a new knowledge distillation framework for language models that improves efficiency and performance by using a novel divergence loss and an adaptive off-policy approach.

E$^2$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation

https://openreview.net/forum?id=lrPrkWXqzd

Compressor summary: This paper proposes techniques to make distilling GANs from diffusion models more efficient, enabling real-time high-quality image editing on mobile devices with low training and storage costs.

Federated Combinatorial Multi-Agent Multi-Armed Bandits

https://openreview.net/forum?id=lrFwPeDdEQ

Compressor summary: Key points: - Paper introduces federated learning for online combinatorial optimization with bandit feedback - Transforms any offline resilient single-agent algorithm into an online multi-agent algorithm with better performance - Algorithm is communication-efficient and applies to stochastic submodular maximization Summary: The paper presents a federated learning framework that improves online combinatorial optimization with bandit feedback by transforming offline resilient algorithms into online multi-agent ones, achieving better regret bounds and communication efficiency.

SMaRt: Improving GANs with Score Matching Regularity

https://openreview.net/forum?id=lqeVCc9zYq

Compressor summary: The paper proposes a score matching regularity (SMaRt) technique to improve GANs' ability to generate data that matches the real data manifold, especially for diverse and complex data.

TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors

https://openreview.net/forum?id=lpHjmPvxW1

Compressor summary: The paper proposes a backdoor defense framework for diffusion models called TERD, which can reverse triggers and detect backdoor inputs, achieving high security and adaptability.

Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shifts

https://openreview.net/forum?id=lon750Kf7n

Compressor summary: Density-Softmax is a sampling-free framework that improves uncertainty estimation and robustness by combining a density function with the softmax layer, achieving competitive results with fewer parameters and faster test time.

Learning Modality Knowledge Alignment for Cross-Modality Transfer

https://openreview.net/forum?id=lmiurzioja

Compressor summary: The paper proposes MoNA, a meta-learning method that learns data transformations to reduce modality gaps and improve cross-modality transfer.

Detecting Influence Structures in Multi-Agent Reinforcement Learning

https://openreview.net/forum?id=lm04PyXoEl

Compressor summary: The paper introduces new ways to measure and analyze how much one agent can affect another in multi-agent reinforcement learning, including approximation algorithms and empirical validation.

Spike Distance Function as a Learning Objective for Spike Prediction

https://openreview.net/forum?id=limyQ1Kk0k

Compressor summary: The paper proposes a new method for predicting neuronal responses using neural networks and shows its superiority over Poisson models in inferring spike trains from retinal ganglion cells' recordings.

Mean-field Chaos Diffusion Models

https://openreview.net/forum?id=lgcFX4VFrM

Compressor summary: The paper introduces a new type of generative models that can handle large and complex data sets by using ideas from mean-field theory and score-matching methods.

On the Complexity of Finite-Sum Smooth Optimization under the Polyak–Łojasiewicz Condition

https://openreview.net/forum?id=leJGQCron2

Compressor summary: The paper analyzes the complexity of gradient methods for optimizing a Polyak–Łojasiewicz function with mean-squared smooth components and distributed on a network, and proposes an efficient decentralized method.

DMTG: One-Shot Differentiable Multi-Task Grouping

https://openreview.net/forum?id=lcX5GbDIi8

Compressor summary: The paper proposes a new Multi-Task Learning method that simultaneously groups tasks and trains a model in one-shot, using a differentiable Categorical distribution to prune task heads.

Meta-Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning

https://openreview.net/forum?id=laIOUtstMs

Compressor summary: PSBL is a meta-RL method that uses Bayesian inference to sample actions from the posterior distribution of the optimal policy, enabling in-context learning and improving performance on tasks with different distributions.

Revisiting Inexact Fixed-Point Iterations for Min-Max Problems: Stochasticity and Structured Nonconvexity

https://openreview.net/forum?id=lWy2lCTyJa

Compressor summary: The paper studies constrained and nonconvex min-max problems, improves first-order methods' complexity guarantees by using conic nonexpansiveness and relaxing inexactness levels, and analyzes a stochastic iteration method.

Generalization to New Sequential Decision Making Tasks with In-Context Learning

https://openreview.net/forum?id=lVQ4FUZ6dp

Compressor summary: The paper investigates how transformers can be trained to learn new sequential decision making tasks from few examples by using trajectory sequences with specific properties, and shows that larger datasets and more diverse tasks lead to better in-context learning.

Predictive Linear Online Tracking for Unknown Targets

https://openreview.net/forum?id=lT3W4AkyM7

Compressor summary: The paper proposes PLOT, an online tracking algorithm that learns a time-varying model of the target and uses it in receding horizon control, with theoretical results for non-stationary targets and real-world quadrotor demonstration.

Unsupervised Concept Discovery Mitigates Spurious Correlations

https://openreview.net/forum?id=lQzmDFlsHX

Compressor summary: The paper proposes CoBalT, a concept balancing technique that mitigates spurious correlations in unsupervised object-centric learning without requiring human labeling of subgroups.

Counterfactual Reasoning for Multi-Label Image Classification via Patching-Based Training

https://openreview.net/forum?id=lQIN9ZyMLz

Compressor summary: The paper proposes a causal inference framework to improve multi-label image classification by leveraging label correlations while mitigating overfitting issues caused by co-occurrence relationships.

GaussianPro: 3D Gaussian Splatting with Progressive Propagation

https://openreview.net/forum?id=lQ3SEBH1gF

Compressor summary: GaussianPro is a novel method that improves 3D neural rendering by using progressive propagation and patch matching to densify 3D Gaussians, outperforming the traditional 3D Gaussian Splatting technique on large-scale scenes.

Inferring the Long-Term Causal Effects of Long-Term Treatments from Short-Term Experiments

https://openreview.net/forum?id=lQ2o7JteMO

Compressor summary: The text discusses how to infer long-term causal effects of continuous interventions using short-term observations and doubly-robust estimators in offline reinforcement learning.

Robust Stable Spiking Neural Networks

https://openreview.net/forum?id=lIYtJtpJR0

Compressor summary: Key points: - The paper studies the robustness of spiking neural networks (SNNs) against adversarial attacks by analyzing membrane potential perturbation dynamics. - The paper proposes a training framework with modified SNN neurons to reduce the mean square of membrane potential perturbation and enhance the stability of nonlinear systems. - The paper verifies the effectiveness of the framework on image classification task using Gaussian noise training and adversarial training. Summary: The paper improves the robustness of spiking neural networks against adversarial attacks by modifying their neurons to reduce perturbation dynamics and enhancing input-output stability.

HelmFluid: Learning Helmholtz Dynamics for Interpretable Fluid Prediction

https://openreview.net/forum?id=lHJFfDFbm6

Compressor summary: HelmFluid is a method that predicts fluid dynamics using the Helmholtz theorem, decomposing it into curl-free and divergence-free parts, and integrating them in multiple spatial scales.

Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning

https://openreview.net/forum?id=lGvIV4Bgsz

Compressor summary: The paper proposes a Convolution Bottleneck structure in CNNs, where early layers transform input into a few frequencies and channels, and later layers map back to outputs, explaining the common practice of down-sampling and how it affects function learning.

Coarse-To-Fine Tensor Trains for Compact Visual Representations

https://openreview.net/forum?id=lGZUvfP2ZF

Compressor summary: The text proposes PuTT, a method for learning compact and high-quality representations of visual data using tensor train upsampling, which improves compression, denoising, and image completion tasks.

Feel-Good Thompson Sampling for Contextual Dueling Bandits

https://openreview.net/forum?id=l9ga3iQuHt

Compressor summary: FGTS.CDB is a Thompson sampling algorithm for linear contextual dueling bandits with a new Feel-Good exploration term that achieves near minimax-optimal regret.

Understanding Stochastic Natural Gradient Variational Inference

https://openreview.net/forum?id=l8GrPpsZfy

Compressor summary: Stochastic NGVI has a non-asymptotic convergence rate of $\mathcal{O}(\frac{1}{T})$ for conjugate likelihoods, similar to stochastic gradient descent, and likely converges faster in practice; however, for non-conjugate likelihoods, it optimizes a non-convex objective with no known global convergence rate.

On the Feasibility of Single-Pass Full-Capacity Learning in Linear Threshold Neurons with Binary Input Vectors

https://openreview.net/forum?id=l7vQQi0I2d

Compressor summary: The text investigates whether there exist single-pass learning rules with maximum capacity in a certain class of rules, but finds such rules impossible using a linear program.

Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulation

https://openreview.net/forum?id=l7shXGuGBT

Compressor summary: The paper proposes MATRIX, a social scene simulator that helps large language models align with human values by emulating realistic scenarios and fine-tuning the LLMs with simulated data, achieving better alignment than existing methods.

PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling

https://openreview.net/forum?id=l6Hef6FVd0

Compressor summary: PIPER is a novel hierarchical reinforcement learning method that uses preference-based learning and hindsight relabeling to learn from sparse rewards and improve performance in challenging tasks.

Scalable Multiple Kernel Clustering: Learning Clustering Structure from Expectation

https://openreview.net/forum?id=l5lgbVR6BP

Compressor summary: This paper proposes a novel multiple kernel clustering framework that learns from expectation kernel matrices and provides theoretical guarantees for its performance.

TravelPlanner: A Benchmark for Real-World Planning with Language Agents

https://openreview.net/forum?id=l5XQzNkAOe

Compressor summary: The text introduces TravelPlanner, a new planning benchmark for testing language agents' abilities in complex real-world scenarios, showing that current language models struggle to handle it.

Turnstile $\ell_p$ leverage score sampling with applications

https://openreview.net/forum?id=l4ZjeDDnu9

Compressor summary: The paper proposes a novel algorithm for sampling rows in a matrix based on their norm when the data is presented as a turnstile data stream, which can improve subsampling constructions for regression problems with low overhead.

Multi-group Learning for Hierarchical Groups

https://openreview.net/forum?id=l4H7Hv7LhJ

Compressor summary: The paper proposes a method to build decision trees that can handle hierarchical groups and generalize well with few samples.

Causal Discovery via Conditional Independence Testing with Proxy Variables

https://openreview.net/forum?id=l1YbS3qkdk

Compressor summary: The paper proposes a new method to test causal relationships over continuous variables without parametric assumptions, using discretization and a novel test statistic.

Semantically-correlated memories in a dense associative model

https://openreview.net/forum?id=l0OGoZPZuC

Compressor summary: CDAM is a novel associative memory model that uses graph structures to link memory patterns and can handle auto- and hetero-association with anti-Hebbian learning rules for various applications.

Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samples

https://openreview.net/forum?id=kzz0kn546b

Compressor summary: The text explains how neural network-based active learning works by prioritizing samples with yet-to-be-learned features, and shows that both uncertainty-based and diversity-based query criteria achieve this goal.

Selective Mixup Helps with Distribution Shifts, But Not (Only) because of Mixup

https://openreview.net/forum?id=ksph9pkEDc

Compressor summary: Selective mixup improves generalization by non-randomly selecting pairs, which resamples data to uniform class distribution, an effect explained by regression toward the mean.

Feasible Reachable Policy Iteration

https://openreview.net/forum?id=ks8qSwkkuZ

Compressor summary: The paper proposes a safe reinforcement learning method that considers goal achievement and reduces ineffective exploration by using a feasible reachable function and policy iteration.

Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design

https://openreview.net/forum?id=kpDd2HCBka

Compressor summary: The paper proposes methods to make online Monte Carlo estimators more efficient and unbiased by learning a tailored behavior policy from offline data.

When and How Does In-Distribution Label Help Out-of-Distribution Detection?

https://openreview.net/forum?id=knhbhDLdry

Compressor summary: The paper explores how using in-distribution (ID) labels can improve out-of-distribution (OOD) detection in machine learning by analyzing data separability with a graph-theoretic approach.

Bayesian Knowledge Distillation: A Bayesian Perspective of Distillation with Uncertainty Quantification

https://openreview.net/forum?id=knZ4NYzGUd

Compressor summary: Bayesian Knowledge Distillation (BKD) is a method that connects KD to Bayesian modeling, providing insight into its working mechanism and tools for measuring student model uncertainty.

Monotone, Bi-Lipschitz, and Polyak-Łojasiewicz Networks

https://openreview.net/forum?id=kn2xp8UOvQ

Compressor summary: The paper introduces BiLipNet, an invertible neural network that can control its output sensitivity and input distinguishability, and PLNet, a scalar-output network based on BiLipNet and quadratic potential, which can learn non-convex losses with global minimum.

On the Expressive Power of Spectral Invariant Graph Neural Networks

https://openreview.net/forum?id=kmugaw9Kfq

Compressor summary: The paper introduces EPNN, a novel message-passing framework for spectral invariant GNNs, and analyzes their expressiveness compared to other architectures and subgraph GNNs.

High-Probability Bound for Non-Smooth Non-Convex Stochastic Optimization with Heavy Tails

https://openreview.net/forum?id=klKk9ETAyU

Compressor summary: The paper proposes a new online-to-non-convex framework that uses heavy-tailed gradients and gradient clipping to find stationary points with high probability, improving on existing results for smooth and non-smooth objectives.

Differentially Private Domain Adaptation with Theoretical Guarantees

https://openreview.net/forum?id=kkqIEp2bRa

Compressor summary: The paper proposes two differentially private algorithms for adapting predictions from public data to private data with similar performance as non-private methods.

Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network

https://openreview.net/forum?id=kfpe7Dg23G

Compressor summary: The text proposes a new optimization-based approach to improve spiking neural networks by approximating subgradient methods and expanding their nonlinearity support, achieving state-of-the-art conversion from artificial neural networks.

FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering

https://openreview.net/forum?id=kc4dZYJlJG

Compressor summary: FedRC is a new algorithm that helps protect privacy in machine learning by handling multiple types of data shifts among clients using clustering and bi-level optimization.

BAT: Learning to Reason about Spatial Sounds with Large Language Models

https://openreview.net/forum?id=kao5hRX9YA

Compressor summary: BAT is a model that combines spatial sound perception and natural language reasoning to navigate and interpret in-the-wild spatial sounds using synthesized audio data and a novel spatial audio encoder.

How do Transformers Perform In-Context Autoregressive Learning ?

https://openreview.net/forum?id=kZbTkpnafR

Compressor summary: The paper investigates how Transformers learn to predict the next token in a sequence by training on a simple first-order autoregressive task and shows that they do so through an in-context autoregressive learning procedure, where they learn orthogonal matrices to capture data patterns.

Hyperbolic Optimizer as a Dynamical System

https://openreview.net/forum?id=kZKopcDp2q

Compressor summary: The text presents a novel way to study optimizers in neural networks using hyperbolic geometry and dynamical systems tools, focusing on their long-term behavior and stability.

More Benefits of Being Distributional: Second-Order Bounds for Reinforcement Learning

https://openreview.net/forum?id=kZBCFQe1Ej

Compressor summary: The paper proves that Distributional Reinforcement Learning can achieve better instance-dependent bounds in both online and offline RL than previous methods, and demonstrates its effectiveness in contextual bandits.

Intersecting-Boundary-Sensitive Fingerprinting for Tampering Detection of DNN Models

https://openreview.net/forum?id=kZArjKc64o

Compressor summary: IBSF is a novel fingerprinting method for detecting tampering with DNN models using only top-1 labels, which maximizes the partial Shannon entropy of selected categories near decision boundaries.

Parameter Estimation in DAGs from Incomplete Data via Optimal Transport

https://openreview.net/forum?id=kXde6Qa6Uy

Compressor summary: The text proposes a new framework for estimating graphical model parameters from incomplete data using optimal transport, without making unrealistic assumptions or variational approximations, and shows its effectiveness and robustness in experiments.

R2E: Turning any Github Repository into a Programming Agent Environment

https://openreview.net/forum?id=kXHgEYFyf3

Compressor summary: The paper introduces R2E, a framework that converts GitHub repositories into test environments for evaluating AI coding assistants using program analysis and large language models.

Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms

https://openreview.net/forum?id=kVgpa1rfLO

Compressor summary: The paper introduces a new unsupervised federated learning algorithm (FedGrEM) for mixture models, with a comprehensive finite-sample theory that compares its performance to local single-task learning and other federated EM algorithms.

Slicedit: Zero-Shot Video Editing With Text-to-Image Diffusion Models Using Spatio-Temporal Slices

https://openreview.net/forum?id=kUm9iuvwIQ

Compressor summary: Slicedit is a method for text-based video editing that uses a pre-trained image synthesis model to process spatial and spatiotemporal slices, achieving better results than existing methods.

A Generative Approach for Treatment Effect Estimation under Collider Bias: From an Out-of-Distribution Perspective

https://openreview.net/forum?id=kUj9b2CezT

Compressor summary: The paper proposes a novel method, C$^2$GAM, to generate missing data from different environments and address collider bias in observational studies.

Accelerating Transformer Pre-training with 2:4 Sparsity

https://openreview.net/forum?id=kTaX87Zn6M

Compressor summary: The paper proposes techniques to accelerate and preserve accuracy in pre-training large transformers using 2:4 sparse matrix multiplication on NVIDIA Ampere GPUs.

Fewer Truncations Improve Language Modeling

https://openreview.net/forum?id=kRxCDDFNpp

Compressor summary: Best-fit Packing is a method that optimizes document packing for large language models, reducing truncations and improving coherence and performance.

Variational Schrödinger Diffusion Models

https://openreview.net/forum?id=kRv0WPJd00

Compressor summary: The variational Schr"odinger diffusion model (VSDM) improves scalability and efficiency of transportation plans in diffusion models by linearizing forward score functions with variational inference and optimizing backward scores without simulation-based losses.

Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design

https://openreview.net/forum?id=kQwSbv0BR4

Compressor summary: DFMs are a new flow-based generative model that can handle discrete and continuous data, achieving state-of-the-art results in protein co-design tasks.

Prompt-guided Precise Audio Editing with Diffusion Models

https://openreview.net/forum?id=kQ1dwuheR0

Compressor summary: PPAE is a text-guided, training-free method for precise audio editing using cross-attention maps and a hierarchical pipeline.

MusicFlow: Cascaded Flow Matching for Text Guided Music Generation

https://openreview.net/forum?id=kOczKjmYum

Compressor summary: MusicFlow is a text-to-music model that uses flow matching networks and masked prediction to generate high-quality music from text descriptions efficiently.

Self-Supervised Coarsening of Unstructured Grid with Automatic Differentiation

https://openreview.net/forum?id=kMBvZ40Iu9

Compressor summary: The paper proposes a method to reduce the computational cost of numerical simulations using differentiable physics, $k$-means clustering, and stochastic minimization for coarsening unstructured grids.

Outlier-Efficient Hopfield Layers for Large Transformer-Based Models

https://openreview.net/forum?id=kLiDMGJKx1

Compressor summary: The Outlier-Efficient Modern Hopfield Model improves associative memory retrieval and attention in large transformer-based models, reducing kurtosis and infinity norm of model outputs.

StrWAEs to Invariant Representations

https://openreview.net/forum?id=kLZZWvqlEm

Compressor summary: Key points: - Autoencoders are useful for generative modeling and representation learning - Structural constraints like conditional independence can improve latent variable invariance - Wasserstein autoencodters (WAEs) can easily incorporate such constraints - StrWAEs are a principled way of penalizing autoencoders to impose structural constraints - StrWAEs show promising results on various tasks Summary: The paper introduces StrWAEs, a flexible and principled way of imposing structural constraints on autoencoders using Wasserstein autoencoders, which improve latent variable invariance and perform well on several tasks.

Learning Latent Structures in Network Games via Data-Dependent Gated-Prior Graph Variational Autoencoders

https://openreview.net/forum?id=kKWjZoaRLv

Compressor summary: The GPGVAE model learns the strategic interactions and network structures from observed actions in network games using a spectral GNN encoder, a data-dependent gated prior, and a Transformer mixture of Bernoulli encoder.

ATraDiff: Accelerating Online Reinforcement Learning with Imaginary Trajectories

https://openreview.net/forum?id=kIh7GJmRfD

Compressor summary: ATraDiff is a generative diffusion model that adapts to different trajectory lengths and distribution shifts, improving online RL performance using synthetic trajectories from offline data.

Pluvial Flood Emulation with Hydraulics-informed Message Passing

https://openreview.net/forum?id=kIHIA6Lr0B

Compressor summary: The paper introduces a hydraulics-informed graph neural network for flood simulation that incorporates physics domain knowledge and performs well on complex topography and sparse precipitation data.

RoboDreamer: Learning Compositional World Models for Robot Imagination

https://openreview.net/forum?id=kHjOmAUfVe

Compressor summary: RoboDreamer is an innovative method that learns a compositional world model by factorizing video generation, enabling generalization of language instructions for realistic plan synthesis and multimodal goals in robotic decision-making.

Acquisition Conditioned Oracle for Nongreedy Active Feature Acquisition

https://openreview.net/forum?id=kGXUL6qGso

Compressor summary: The paper presents a new method for sequentially selecting features in health care applications that minimizes costs and improves diagnostic performance using an oracle based approach.

Listenable Maps for Audio Classifiers

https://openreview.net/forum?id=kAfYYg6PX8

Compressor summary: L-MAC is an interpretation method for audio classifiers that generates binary masks to highlight relevant parts of audio signals, which are more faithful and preferred by users than other methods.

Autaptic Synaptic Circuit Enhances Spatio-temporal Predictive Learning of Spiking Neural Networks

https://openreview.net/forum?id=kAIkYOE5pV

Compressor summary: The paper proposes a novel Spatio-Temporal Circuit (STC) model for spiking neural networks to improve their ability to handle complex, dynamic spatio-temporal prediction tasks by incorporating autaptic synapses and two learnable adaptive pathways.

OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shift

https://openreview.net/forum?id=kAFevjEYsz

Compressor summary: Key points: - Existing methods improve adversarial robustness but only on in-distribution (ID) data - OODRobustBench is a benchmark to assess out-of-distribution (OOD) robustness using 23 dataset shifts and 6 threat models - Adversarial robustness suffers from severe OOD generalization issue - ID robustness correlates strongly with OOD robustness in a positive linear way - Existing methods are unlikely to achieve high OOD robustness and novel methods are needed Summary: The paper introduces OODRobustBench, a benchmark to measure OOD adversarial robustness using various shifts and models, and shows that existing methods have poor OOD performance and require new solutions.

Improving SAM Requires Rethinking its Optimization Formulation

https://openreview.net/forum?id=k7G4N1x7f9

Compressor summary: The paper proposes a new version of Sharpness-Aware Minimization (SAM) called BiSAM, which improves the performance of network perturbations using a bilevel optimization approach and a novel lower-bound surrogate loss.

Directly Denoising Diffusion Models

https://openreview.net/forum?id=k5ncz7TIPX

Compressor summary: DDDMs generate realistic images quickly using few-step sampling and Pseudo-LPIPS, outperforming GANs and distillation-based models on benchmark datasets.

Differentially Private Decentralized Learning with Random Walks

https://openreview.net/forum?id=k2dVVIWWho

Compressor summary: This paper studies the privacy of decentralized learning with random walk algorithms using a new differential privacy variant, and shows that it can improve privacy compared to gossip algorithms for nearby nodes.

Self-Driven Entropy Aggregation for Byzantine-Robust Heterogeneous Federated Learning

https://openreview.net/forum?id=k2axqNsVVO

Compressor summary: SDEA is a method for Byzantine-robust aggregation in heterogeneous federated learning that uses a random public dataset and learns aggregation weights to distinguish benign from malicious clients.

Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?

https://openreview.net/forum?id=k1JXxbpIY6

Compressor summary: The authors investigate how large language models perform on arithmetic word problems, comparing their cognitive biases to those of children, and find that LLMs mimic human biases in understanding text and planning solutions, but not in executing arithmetic expressions.

Exploiting Negative Samples: A Catalyst for Cohort Discovery in Healthcare Analytics

https://openreview.net/forum?id=k1J2GbamLi

Compressor summary: The paper proposes a method to find cohorts within negative samples in healthcare analytics using data Shapley values and manifold learning, which can reveal insights on diseases and related conditions.

Learning to Remove Cuts in Integer Linear Programming

https://openreview.net/forum?id=k10805cgak

Compressor summary: The paper proposes a new cutting plane method for solving integer linear programs by removing some existing constraints, which improves the performance compared to adding new ones.

Position: Foundation Agents as the Paradigm Shift for Decision Making

https://openreview.net/forum?id=jzHmElqpPe

Compressor summary: Key points: - Decision making needs perception, memory, and reasoning for optimal policies - Conventional approaches have limitations in sample efficiency and generalization - Foundation models in language and vision can adapt to diverse tasks faster - The authors propose foundation agents as a new learning paradigm inspired by LLMs - They outline the roadmap of foundation agents from data collection/generation, pretraining, adaptation, and alignment with LLMs - They identify critical research questions and trends for foundation agents with real-world use cases Summary: The paper proposes foundation agents, a new learning paradigm for decision making inspired by large language models, that can adapt faster and better to diverse tasks with data collection/generation, pretraining, adaptation, and alignment.

RNAFlow: RNA Structure & Sequence Design via Inverse Folding-Based Flow Matching

https://openreview.net/forum?id=jxvqvZLBuU

Compressor summary: RNAFlow is an AI method for designing RNA structures and sequences by combining a denoising network with an RNA inverse folding model that considers conformational flexibility.

Data-free Distillation of Diffusion Models with Bootstrapping

https://openreview.net/forum?id=jw2f9v59g0

Compressor summary: BOOT is an efficient data-free distillation technique for improving the speed and quality of image generation using diffusion models without requiring additional data or computationally expensive processes.

Minimizing $f$-Divergences by Interpolating Velocity Fields

https://openreview.net/forum?id=jvVWPtJYbc

Compressor summary: The paper proposes a new method to estimate velocity fields for Wasserstein Gradient Flow, which improves the accuracy of particle movement and applies it to domain adaptation and missing data imputation tasks.

Efficient Algorithms for Sum-Of-Minimum Optimization

https://openreview.net/forum?id=jsmaWEdx9g

Compressor summary: Key points: - propose a new optimization model for clustering applications called sum-of-minimum optimization - develop efficient algorithms inspired by k-means and Lloyd's algorithm - prove a tight bound and convergence rate for the algorithms - show numerical results on multiple tasks Summary: The paper introduces sum-of-minimum optimization, a novel clustering model with efficient algorithms based on k-means and Lloyd's algorithm, and demonstrates its effectiveness on various tasks.

Position: Measure Dataset Diversity, Don't Just Claim It

https://openreview.net/forum?id=jsKr6RVDDs

Compressor summary: The text discusses the challenges of defining and measuring diversity in machine learning datasets, and suggests using principles from measurement theory to improve dataset construction.

FedMBridge: Bridgeable Multimodal Federated Learning

https://openreview.net/forum?id=jrHUbftLd6

Compressor summary: FedMBridge is a new method for multimodal federated learning that uses a hypernetwork to handle different client architectures and data types while protecting privacy.

PGODE: Towards High-quality System Dynamics Modeling

https://openreview.net/forum?id=jrE7geZekq

Compressor summary: The paper proposes PGODE, a new method for modeling multi-agent dynamical systems using prototype decomposition and graph ODEs, which improves generalization under system changes and outperforms baselines in various scenarios.

Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration

https://openreview.net/forum?id=jr0W36wOBx

Compressor summary: The paper proposes a new conformal regression method to improve survival analysis models' calibration without compromising their ability to rank subjects accurately.

Efficient Precision and Recall Metrics for Assessing Generative Models using Hubness-aware Sampling

https://openreview.net/forum?id=jnps5YwNlU

Compressor summary: The paper proposes eP&R, a new evaluation metric for deep generative models that uses hubness-aware sampling to reduce computational costs while maintaining accuracy.

A decoder-only foundation model for time-series forecasting

https://openreview.net/forum?id=jn2iTJas6h

Compressor summary: The paper presents a time-series foundation model for forecasting that performs well without any fine-tuning on various public datasets using a large pretrained attention model with input patching.

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought

https://openreview.net/forum?id=jmmji1EU3g

Compressor summary: The In-context Decision Transformer (IDT) improves offline reinforcement learning efficiency by using a hierarchical structure inspired by human decision-making to reconstruct high-level decisions instead of low-level actions, achieving state-of-the-art results in long-horizon tasks and reducing evaluation time significantly.

Seesaw: Compensating for Nonlinear Reduction with Linear Computations for Private Inference

https://openreview.net/forum?id=jklD0TV5Hw

Compressor summary: Seesaw is a novel neural architecture search method that leverages more linear computations and nonlinear result reuse for privacy-preserving machine learning, achieving better accuracy and lower latency than existing methods.

Post-hoc Part-Prototype Networks

https://openreview.net/forum?id=jhWSzTO0Jl

Compressor summary: The paper proposes a method to create post-hoc part-prototype networks that can explain both where and what a model looks for in an image, while maintaining performance and providing more faithful explanations.

Plug-in Performative Optimization

https://openreview.net/forum?id=jh7FDDwDBf

Compressor summary: The paper proposes plug-in performative optimization, a method that uses possibly misspecified models to reduce performative risk and improve prediction in situations where the feedback affects future observations.

BBox-Adapter: Lightweight Adapting for Black-Box Large Language Models

https://openreview.net/forum?id=jdRIaUu3xY

Compressor summary: BBox-Adapter is a novel method for adapting black-box LLMs to specific tasks using ranking-based NCE loss and online data sampling, improving performance and reducing costs.

Online Variational Sequential Monte Carlo

https://openreview.net/forum?id=jbPc3pW6sC

Compressor summary: The paper proposes online VSMC, a method for efficient and accurate parameter estimation and latent state inference in state-space models using stochastic gradient approximation.

Simple Ingredients for Offline Reinforcement Learning

https://openreview.net/forum?id=japBn31gXC

Compressor summary: The text discusses how offline reinforcement learning algorithms struggle with diverse data from different tasks, and suggests using larger policy sizes to improve performance.

Unmasking Vulnerabilities: Cardinality Sketches under Adaptive Inputs

https://openreview.net/forum?id=jaJxpKkBcL

Compressor summary: The paper studies how cardinality sketches perform in adaptive settings and reveals their vulnerabilities, showing an attack that exploits simple non-adaptive queries to generate adversarial inputs.

FiT: Flexible Vision Transformer for Diffusion Model

https://openreview.net/forum?id=jZVen2JguY

Compressor summary: The Flexible Vision Transformer (FiT) is a new transformer architecture that can generate images with unrestricted resolutions and aspect ratios by conceptualizing images as sequences of dynamically-sized tokens, thus promoting resolution generalization and eliminating biases.

Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models

https://openreview.net/forum?id=jZEY5SxbL4

Compressor summary: Key points: - CraFT is a method for fine-tuning black-box VLMs using input prompts and output predictions only - CraFT consists of two modules: prompt generation and prediction refinement, with an auxiliary loss to ensure consistency - CraFT outperforms white-box methods on few-shot classification with less queries, faster training, and less memory Summary: CraFT is a novel approach for fine-tuning black-box VLMs without accessing their parameters. It uses prompt generation and prediction refinement modules to improve few-shot classification performance, efficiency, and memory footprint.

Conditional Language Learning with Context

https://openreview.net/forum?id=jXn1qIcjyG

Compressor summary: The paper introduces conditional finetuning, a method that allows language models to learn useful knowledge from a corpus while avoiding unnecessary biases, leading to improved performance on downstream tasks and lifelong learning.

SyCoCa: Symmetrizing Contrastive Captioners with Attentive Masking for Multimodal Alignment

https://openreview.net/forum?id=jWHU4b7Yk6

Compressor summary: SyCoCa is a method that improves multimodal alignment between language and vision by introducing bidirectional interactions on global and local representations using textual and visual cues.

MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion

https://openreview.net/forum?id=jVXJdGQ4eD

Compressor summary: MagicPose is a diffusion-based model that can generate realistic human images with controlled poses and facial expressions while preserving the identity, using a two-stage training strategy to disentangle appearance and motions.

From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation

https://openreview.net/forum?id=jU6iPouOZ6

Compressor summary: The paper introduces a novel framework called Vision to Audio and Beyond (VAB) that learns representations and generates modalities within latent spaces for various audio-visual tasks.

Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference

https://openreview.net/forum?id=jTn4AIOgpM

Compressor summary: DDVI is a novel method that uses denoising diffusion SDEs and neural networks to infer the posterior distribution of sparse inducing points in deep Gaussian processes, improving efficiency and reducing bias.

No Dimensional Sampling Coresets for Classification

https://openreview.net/forum?id=jS3CMHtYJD

Compressor summary: The paper presents new coresets for classification problems that have smaller size, distributional input support, and various loss function applications.

Position: On the Societal Impact of Open Foundation Models

https://openreview.net/forum?id=jRX6yCxFhx

Compressor summary: Open foundation models have benefits and risks related to innovation, power distribution, and transparency, but their misuse risk is unclear due to insufficient research and varying assumptions in past work.

The Perception-Robustness Tradeoff in Deterministic Image Restoration

https://openreview.net/forum?id=jQA5iutPzd

Compressor summary: The paper studies how well deterministic methods solve inverse problems in imaging and shows that they need higher Lipschitz constants for better perceptual quality and consistency, making them more vulnerable to attacks.

Accurate LoRA-Finetuning Quantization of LLMs via Information Retention

https://openreview.net/forum?id=jQ92egz5Ym

Compressor summary: The paper proposes IR-QLoRA, a method to improve the accuracy of quantized large language models for deployment on resource-constrained devices by retaining original information through statistics-based and finetuning-based technologies.

Learning Divergence Fields for Shift-Robust Graph Representations

https://openreview.net/forum?id=jPaEOH56JB

Compressor summary: The paper proposes a geometric diffusion model with learnable divergence fields and causal inference to improve generalization for interdependent data, and provides three model instantiations based on GCN, GAT, and Transformers.

Training Greedy Policy for Proposal Batch Selection in Expensive Multi-Objective Combinatorial Optimization

https://openreview.net/forum?id=jP8mf34iCW

Compressor summary: The paper proposes a new subset selection algorithm for active learning that optimizes batch acquisition directly on the combinatorial space using sequential greedy sampling and improves efficiency in solving expensive multi-objective combinatorial optimization problems.

Learning the Target Network in Function Space

https://openreview.net/forum?id=jP1zeEqHli

Compressor summary: The paper proposes Lookahead-Replicate, a new reinforcement learning algorithm that maintains function space equivalence between online and target networks, leading to better deep RL performance on Atari games.

Fast Timing-Conditioned Latent Audio Diffusion

https://openreview.net/forum?id=jOlO8t1xdx

Compressor summary: The paper proposes an efficient generative model that creates long-form, variable-length stereo audio from text prompts using latent diffusion and convolutional autoencoders, outperforming existing models in quality and structure.

Partially Stochastic Infinitely Deep Bayesian Neural Networks

https://openreview.net/forum?id=jNab9mXEyj

Compressor summary: The paper introduces a new type of neural network that combines partial stochasticity and infinite depth to improve efficiency, expressivity, and performance on various tasks.

How Transformers Learn Causal Structure with Gradient Descent

https://openreview.net/forum?id=jNM4imlHZv

Compressor summary: The paper shows how gradient descent on a simplified two-layer transformer learns latent causal structure from in-context learning tasks, and proves that the attention matrix encodes mutual information between tokens.

Averaging $n$-step Returns Reduces Variance in Reinforcement Learning

https://openreview.net/forum?id=jM9A3Kz6Ki

Compressor summary: Compound returns, a weighted average of step returns, can reduce variance in reinforcement learning methods, improving their sample efficiency.

BetterV: Controlled Verilog Generation with Discriminative Guidance

https://openreview.net/forum?id=jKnW7r7de1

Compressor summary: BetterV is a Verilog generation framework that uses fine-tuned large language models and generative discriminators to create correct and optimized Verilog code for various electronic design automation tasks.

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

https://openreview.net/forum?id=jKYyFbH8ap

Compressor summary: SAFIM is a new benchmark to evaluate Large Language Models on code Fill-in-the-Middle tasks, focusing on syntax-aware completions and providing a robust framework for accurate and fair comparisons.

ERQ: Error Reduction for Post-Training Quantization of Vision Transformers

https://openreview.net/forum?id=jKUWlgra9b

Compressor summary: ERQ is a two-step method that reduces quantization error in vision transformers by strategically updating weights with full-precision and refining rounding directions of quantized weights.

Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning

https://openreview.net/forum?id=jJmGl01S4l

Compressor summary: The paper explains how spikes in training loss during neural network training with stochastic gradient descent (SGD) are related to improved feature learning and generalization by increasing alignment with the true predictor's average gradient outer product.

Non-clairvoyant Scheduling with Partial Predictions

https://openreview.net/forum?id=jJLcXGB2uA

Compressor summary: The paper studies how to schedule jobs efficiently when only some predictions are available, and proposes an algorithm that is robust, consistent, and smooth under this restriction.

Executable Code Actions Elicit Better LLM Agents

https://openreview.net/forum?id=jJ9BoXAfFa

Compressor summary: This paper proposes using executable Python code as a unified action space for large language models, improving their performance on agent-oriented tasks and enabling them to perform sophisticated actions like model training.

Cross-view Masked Diffusion Transformers for Person Image Synthesis

https://openreview.net/forum?id=jEoIkNkqyc

Compressor summary: X-MDPT is a novel pose-guided human image generation model using masked diffusion transformers, which improves scalability and efficiency over existing approaches.

Tight Partial Identification of Causal Effects with Marginal Distribution of Unmeasured Confounders

https://openreview.net/forum?id=jEWpcEyuUl

Compressor summary: The paper reassesses the role of marginal confounder distribution in partial identification and provides a criterion for determining its impact on causal inference.

Discovering Multiple Solutions from a Single Task in Offline Reinforcement Learning

https://openreview.net/forum?id=j6rG1ETRyu

Compressor summary: The study proposes and tests algorithms for learning multiple distinct solutions from a single task in offline reinforcement learning.

Hybrid Neural Representations for Spherical Data

https://openreview.net/forum?id=j6QZy90B93

Compressor summary: The paper proposes a new method, Hybrid Neural Representations for Spherical data (HNeR-S), to better handle nonlinear spherical signals like weather and CMB data using positional features from feature-grids and a multi-layer perceptron.

Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models

https://openreview.net/forum?id=j5wf1NNhFs

Compressor summary: The U-ext-Hop model improves memory retrieval in Hopfield networks by using a learnable feature map that transforms the energy function into kernel space, enhancing memory capacity and reducing confusion.

Position: Fundamental Limitations of LLM Censorship Necessitate New Approaches

https://openreview.net/forum?id=j5csKrtyAe

Compressor summary: The text discusses the limitations of current methods to prevent misuse of large language models by censoring their outputs and calls for re-evaluation and new approaches to ensure safety.

FADAS: Towards Federated Adaptive Asynchronous Optimization

https://openreview.net/forum?id=j56JAd29uH

Compressor summary: FADAS is a novel method that combines adaptive federated optimization with asynchronous updates, improving efficiency and resilience in privacy-preserving machine learning.

MILP-FBGen: LP/MILP Instance Generation with Feasibility/Boundedness

https://openreview.net/forum?id=j4HtfTqr0f

Compressor summary: The paper introduces MILP-FBGen, a machine learning technique for generating realistic and feasible Linear Programming and Mixed-Integer Linear Programming instances that preserve key properties and improve downstream task performance.

On Multi-Armed Bandit with Impatient Arms

https://openreview.net/forum?id=j35VcooKG8

Compressor summary: The paper studies a hard Multi-Armed Bandit problem where neglected arms leave the game, proposes FC-SE algorithm with regret bounds, and extends it to handle new arms with FC-Entry algorithm.

Distributional Bellman Operators over Mean Embeddings

https://openreview.net/forum?id=j2pLfsBm4J

Compressor summary: A new algorithmic framework for distributional reinforcement learning uses mean embeddings of return distributions and simple linear-algebraic computations to update the sketch Bellman operator, with theoretical and empirical evidence for its effectiveness.

Differentiability and Optimization of Multiparameter Persistent Homology

https://openreview.net/forum?id=ixdfvnO0uy

Compressor summary: The article develops a general framework for optimizing vector-valued functions using multiparameter homological descriptors from persistent homology, showing that it improves performance over one-parameter descriptors.

Visual Transformer with Differentiable Channel Selection: An Information Bottleneck Inspired Approach

https://openreview.net/forum?id=iup9NElHji

Compressor summary: The paper proposes a new transformer block that reduces the computational cost of visual transformers while maintaining or improving their accuracy, and introduces a novel variational upper bound for information bottleneck optimization.

AI Alignment with Changing and Influenceable Reward Functions

https://openreview.net/forum?id=itYGbe0Cs1

Compressor summary: The paper introduces DR-MDPs to model preference changes in AI alignment, shows that static preferences can lead to undesirable AI influence, and explores potential solutions while acknowledging their limitations.

Learning from Integral Losses in Physics Informed Neural Networks

https://openreview.net/forum?id=itDhUBY2xf

Compressor summary: The paper proposes methods to train networks under partial differential equations, which require many evaluations, by addressing the bias caused by naive integral approximations.

EvGGS: A Collaborative Learning Framework for Event-based Generalizable Gaussian Splatting

https://openreview.net/forum?id=isUSVgS7W1

Compressor summary: Event cameras can reconstruct 3D scenes from sparse event data using a generalizable framework that estimates depth, intensity, and Gaussian regression, outperforming existing methods.

Position: What Can Large Language Models Tell Us about Time Series Analysis

https://openreview.net/forum?id=iroZNDxFJZ

Compressor summary: The paper argues that large language models can revolutionize time series analysis, enabling efficient decision-making and new possibilities like modality switching and question answering.

Residual-Conditioned Optimal Transport: Towards Structure-Preserving Unpaired and Paired Image Restoration

https://openreview.net/forum?id=irBHPlknxP

Compressor summary: The paper proposes a new image restoration method using optimal transport and residual information to preserve the original image structure better than existing methods.

Statistical Properties of Robust Satisficing

https://openreview.net/forum?id=iqAyWVLUEO

Compressor summary: This paper analyzes the theoretical properties of the Robust Satisficing (RS) model, a streamlined approach to robust optimization with better statistical guarantees and performance than existing methods.

The Expressive Power of Path-Based Graph Neural Networks

https://openreview.net/forum?id=io1XSRtcO8

Compressor summary: PATH-WL is a powerful new method for graph neural networks that uses paths and shortest path distance information to achieve strong empirical results and solve complex problems like strongly regular graphs.

Mol-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective

https://openreview.net/forum?id=inEuvSg0y1

Compressor summary: Mol-AE is a new auto-encoder model for 3D molecular representation learning with positional encoding and 3D Cloze Test objective that outperforms existing methods.

Causal Customer Churn Analysis with Low-rank Tensor Block Hazard Model

https://openreview.net/forum?id=ihv6pWuILN

Compressor summary: The study proposes a new causal model to analyze customer churn using tensor completion methods and shows its effectiveness in practice.

Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding

https://openreview.net/forum?id=igRjCCAz2a

Compressor summary: EDDPMs are versatile probabilistic models that combine encoding-decoding with diffusion for broad applicability and enhanced performance across text, proteins, and images.

Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimation

https://openreview.net/forum?id=igRAPavrrS

Compressor summary: The paper analyzes standard differentially private gradient descent for linear regression, showing its accuracy and sample complexity match non-private methods with proper hyperparameter choices and adaptive confidence intervals.

On the Unexpected Effectiveness of Reinforcement Learning for Sequential Recommendation

https://openreview.net/forum?id=ie3vXkMvRY

Compressor summary: The text discusses how Reinforcement Learning (RL) improves session-based recommendation by promoting better embeddings of user interactions, and suggests using an auxiliary loss instead of RL to achieve similar performance gains.

Evaluating Model Bias Requires Characterizing its Mistakes

https://openreview.net/forum?id=idyUNsoZ75

Compressor summary: SkewSize is a new metric that measures and characterizes bias in model predictions across subgroups, improving upon existing benchmarks.

Best of Both Worlds Guarantees for Smoothed Online Quadratic Optimization

https://openreview.net/forum?id=icijMMWwdG

Compressor summary: The paper studies smoothed online quadratic optimization with different costs and analyzes optimal algorithms for both adversarial and stochastic settings, proposing a new distribution-agnostic dynamic interpolation algorithm called Lazy Adaptive Interpolation (LAI).

DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency

https://openreview.net/forum?id=ibwxzYCep9

Compressor summary: The paper proposes a new framework for image restoration that balances visual quality and faithfulness to the original image using a stochastic degradation model and early-stopping.

Modeling Caption Diversity in Contrastive Vision-Language Pretraining

https://openreview.net/forum?id=iaV2fU6Dif

Compressor summary: Llip is a new pretraining method for vision-language models that leverages diverse captions to improve image representation and performance on zero-shot tasks.

Gated Linear Attention Transformers with Hardware-Efficient Training

https://openreview.net/forum?id=ia5XvxFUJT

Compressor summary: The paper presents FlashLinearAttention and gated linear attention (GLA), which improve linear attention's efficiency and performance for parallel training and inference in Transformers.

Diffuse, Sample, Project: Plug-And-Play Controllable Graph Generation

https://openreview.net/forum?id=ia0Z8d1DbY

Compressor summary: PRODIGY is a method to generate graphs with precise control using pre-trained diffusion models, handling both soft and hard constraints, achieving high constraint satisfaction for various applications like drug discovery.

Locally Interdependent Multi-Agent MDP: Theoretical Framework for Decentralized Agents with Dynamic Dependencies

https://openreview.net/forum?id=iYYA5zDoCm

Compressor summary: This paper introduces a decentralized model for multi-agent systems with varying dependencies and proposes three near-optimal, scalable policies for it.

Model Alignment as Prospect Theoretic Optimization

https://openreview.net/forum?id=iUwHnoENnl

Compressor summary: The paper proposes KTO, a new human-aware loss function for LLMs that directly maximizes human utility based on Kahneman-Tversky prospect theory, and shows its effectiveness compared to other HALOs and cross-entropy minimization at different scales.

Learning Decision Policies with Instrumental Variables through Double Machine Learning

https://openreview.net/forum?id=iRcmqXZjeK

Compressor summary: DML-IV is a non-linear IV regression method that reduces bias in two-stage IV regressions and effectively learns high-performing policies using a novel learning objective and DML framework.

Beyond the Calibration Point: Mechanism Comparison in Differential Privacy

https://openreview.net/forum?id=iQTElQbAqo

Compressor summary: The text proposes a new method to compare differentially private machine learning mechanisms by measuring their worst-case privacy risks and shows its usefulness through examples.

Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformers

https://openreview.net/forum?id=iPFuWc1TV2

Compressor summary: The Triplet Graph Transformer is a novel model that uses triplet attention and aggregation to capture third-order interactions in graphs, enabling better molecular property prediction and achieving state-of-the-art results on various benchmarks.

Adaptive Hierarchical Certification for Segmentation using Randomized Smoothing

https://openreview.net/forum?id=iOEReiiTit

Compressor summary: The paper proposes a novel certification method for machine learning models that uses a multi-level hierarchy to reduce abstain rates and increase information gain compared to existing methods.

Block Acceleration Without Momentum: On Optimal Stepsizes of Block Gradient Descent for Least-Squares

https://openreview.net/forum?id=iLyUEPZ0fR

Compressor summary: Block coordinate descent with optimal stepsizes can achieve faster convergence than gradient descent and momentum for some problems.

DeepPolar: Inventing Nonlinear Large-Kernel Polar Codes via Deep Learning

https://openreview.net/forum?id=iLfk2CwEHA

Compressor summary: DeepPolar codes use neural networks to generalize and improve Polar codes for error correction at short-to-medium block lengths.

CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding

https://openreview.net/forum?id=iLSgF7jMtI

Compressor summary: CogDPM is a new model that integrates diffusion probabilistic models with predictive coding theory to improve visual world prediction by incorporating precision weighting mechanism.

Debating with More Persuasive LLMs Leads to More Truthful Answers

https://openreview.net/forum?id=iLCZtl7FTa

Compressor summary: Debate can help weaker AI models assess stronger ones without human labels, and optimizing expert debaters for persuasiveness improves their performance.

The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

https://openreview.net/forum?id=iKkFruh4d5

Compressor summary: This paper studies how multi-pass gradient descent with batch re-use improves learning for two-layer neural networks when target functions have multiple indexes.

Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs

https://openreview.net/forum?id=iJlPJsTw2B

Compressor summary: The paper proposes a new optimal format for large language models and explores the tradeoff between accuracy and chip area using different datatypes.

Sampling-based Multi-dimensional Recalibration

https://openreview.net/forum?id=iJWeK2snMH

Compressor summary: The paper proposes a method to calibrate probabilistic forecasts for multi-dimensional outputs using highest density regions, which considers the joint distribution across dimensions.

CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations

https://openreview.net/forum?id=iHSgfGob9j

Compressor summary: CoLoRA is a method that uses low-rank adaptive neural networks to predict the evolution of solution fields for partial differential equations, achieving fast and accurate results even in data-scarce regimes.

Gibbs Sampling of Continuous Potentials on a Quantum Computer

https://openreview.net/forum?id=iGMTxygzcJ

Compressor summary: A quantum algorithm using quantum Fourier transforms can efficiently estimate Gibbs measures for periodic functions and improve sampling precision in high temperature regimes.

Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise

https://openreview.net/forum?id=iE2lMjeXRR

Compressor summary: The paper explores the theoretical advantages of Adam optimizer in optimization problems using an online learning framework.

On the Effectiveness of Supervision in Asymmetric Non-Contrastive Learning

https://openreview.net/forum?id=iC8l9DI1ZX

Compressor summary: Supervised contrastive representation learning methods, SupSiam and SupBYOL, improve on self-supervised learning by using labels to reduce intra-class variance and avoid collapse, leading to better results across various tasks and datasets.

Rapid Learning without Catastrophic Forgetting in the Morris Water Maze

https://openreview.net/forum?id=i9C4Kwm56G

Compressor summary: The study proposes a novel task for machine learning models that mimics animals' ability to adapt quickly and maintain proficiency, using a combination of neural networks and biological inspirations.

Auto-Regressive Next-Token Predictors are Universal Learners

https://openreview.net/forum?id=i56plqPpEa

Compressor summary: The authors present a theory for how simple language models can perform complex tasks by predicting the next token in a sequence and show that linear networks and MLPs have surprising abilities in text generation and arithmetic tasks.

Self-Correcting Self-Consuming Loops for Generative Model Training

https://openreview.net/forum?id=i0nVanexij

Compressor summary: The paper proposes self-correction functions to stabilize generative models trained on synthetic data, especially for tasks like human motion synthesis.

LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery

https://openreview.net/forum?id=hz8cFsdz7P

Compressor summary: The Scientific Generative Agent (SGA) combines the strengths of large language models and simulations to enhance scientific discovery by proposing hypotheses, reasoning about discrete components, and receiving feedback on continuous parts.

Energy-Guided Diffusion Sampling for Offline-to-Online Reinforcement Learning

https://openreview.net/forum?id=hunSEjeCPE

Compressor summary: The paper introduces EDIS, a method that uses a diffusion model to extract prior knowledge from offline data and energy functions to distill it for better online learning, improving efficiency and safety in RL.

On the Tractability of SHAP Explanations under Markovian Distributions

https://openreview.net/forum?id=htq0FbPOsY

Compressor summary: The article studies the computational complexity of the SHAP framework for local explainability of ML models under realistic assumptions and shows that it can be computed in polynomial time for some model families.

RMIB: Representation Matching Information Bottleneck for Matching Text Representations

https://openreview.net/forum?id=hsHIxrnrMx

Compressor summary: The paper proposes a method to match text representations in tasks with different domains by optimizing information bottleneck, which improves the performance of such tasks.

Prompt-tuning Latent Diffusion Models for Inverse Problems

https://openreview.net/forum?id=hrwIndai8e

Compressor summary: The paper introduces P2L, a method that uses text-to-image latent diffusion models with prompt tuning to solve imaging inverse problems better than existing methods.

Truly No-Regret Learning in Constrained MDPs

https://openreview.net/forum?id=hrWte3nlzr

Compressor summary: The paper proposes a new primal-dual algorithm for constrained reinforcement learning that avoids error cancellations and achieves sublinear regret.

Stochastic positional embeddings improve masked image modeling

https://openreview.net/forum?id=hr8OXXMb7a

Compressor summary: This paper introduces StoP, a method that incorporates location uncertainty in MIM by using stochastic positional embeddings, which improves downstream performance on various tasks.

Nearest Neighbour Score Estimators for Diffusion Generative Models

https://openreview.net/forum?id=hqNz4LDuhn

Compressor summary: The text introduces a new score function estimator for diffusion generative models that reduces variance and improves training speed and sample quality.

Behavior Generation with Latent Actions

https://openreview.net/forum?id=hoVwecMqV5

Compressor summary: VQ-BeT is a new model that improves on BeT for generating complex behaviors by tokenizing continuous actions with hierarchical vector quantization, achieving faster inference speed and better performance in various environments.

Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians

https://openreview.net/forum?id=hnqlgwcRxb

Compressor summary: The paper investigates theoretical properties of variational inference for non-Gaussian Mixture of Gaussians, showing how it can be cast as optimizing Dirac positions using gradient descent and studying errors in the process.

ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking

https://openreview.net/forum?id=hlvKd7Vdxm

Compressor summary: The paper proposes ExCP, a framework that compresses large language models' checkpoints significantly while maintaining high accuracy on various tasks.

Privacy Profiles for Private Selection

https://openreview.net/forum?id=hgHQvrvwH9

Compressor summary: The paper proposes an easy-to-use recipe to improve privacy profiles of ReportNoisyMax and PrivateTuning using base algorithms' privacy profiles, leading to better private learning experiments.

Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning

https://openreview.net/forum?id=hg4wXlrQCV

Compressor summary: The paper presents Craftax-Classic, a faster version of Crafter, and Craftax, a more challenging benchmark for RL research that requires deep exploration and planning.

Position: Machine Learning-powered Assessments of the EU Digital Services Act Aid Quantify Policy Impacts on Online Harms

https://openreview.net/forum?id=hdpv6mall8

Compressor summary: The text discusses the potential harmful applications of machine learning and the need for evaluating the European Union's Digital Services Act to curb these harms.

Grokking Group Multiplication with Cosets

https://openreview.net/forum?id=hcQfTsVnBo

Compressor summary: The text describes how researchers reverse engineered a neural network that learned the arithmetic of permutation groups and challenges the interpretability of another paper on the same topic.

Peeking with PEAK: Sequential, Nonparametric Composite Hypothesis Tests for Means of Multiple Data Streams

https://openreview.net/forum?id=hcASxFvmZ5

Compressor summary: PEAK is a novel nonparametric sequential test that uses a betting scheme to control type-I error and power in multiple data streams, reducing sample complexity and computational overhead.

Online Adaptive Anomaly Thresholding with Confidence Sequences

https://openreview.net/forum?id=hbsKxUEreL

Compressor summary: The paper proposes an algorithm for anomaly detection that adapts to distribution shifts, has low false positive and negative rates, and uses offline data effectively.

Hard Tasks First: Multi-Task Reinforcement Learning Through Task Scheduling

https://openreview.net/forum?id=haUOhXo70o

Compressor summary: SMT is a novel multi-task RL algorithm that prioritizes harder tasks, uses a task difficulty metric for efficient resource allocation, and resets network parameters to mitigate simplicity bias.

Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion

https://openreview.net/forum?id=hZ0fWhgVch

Compressor summary: The paper proposes a training-free approach to improve text-to-image alignment by optimizing images directly with the supervision of vision-language models and incorporating score distillation sampling.

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

https://openreview.net/forum?id=hYHsrKDiX7

Compressor summary: GaLore is a training strategy for large language models that reduces memory usage by up to 82.5% without sacrificing performance, enabling pre-training on 7B models with consumer GPUs and 24GB memory.

Near-Optimal Regret in Linear MDPs with Aggregate Bandit Feedback

https://openreview.net/forum?id=hXQOO6VsxH

Compressor summary: The paper proposes two efficient algorithms for RL with Aggregate Bandit Feedback, which allows feedback at episode end instead of individual rewards, and achieves near-optimal regret guarantees using linear function approximation and new randomization techniques.

Mind the Boundary: Coreset Selection via Reconstructing the Decision Boundary

https://openreview.net/forum?id=hWng0GXeE4

Compressor summary: The paper presents a novel geometry-based coreset construction method that efficiently selects training data to reconstruct the decision boundary of a deep neural network, achieving high data pruning rate with minimal accuracy loss and showing strong cross-architecture transferability.

From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions

https://openreview.net/forum?id=hTiNFCNxM1

Compressor summary: The paper proposes DCEM, an algorithm to learn from selective labels with disparate censorship, and shows that it reduces bias without sacrificing performance on synthetic and real clinical data.

Progressive Inference: Explaining Decoder-Only Sequence Classification Models Using Intermediate Predictions

https://openreview.net/forum?id=hRX1o7FBhT

Compressor summary: The paper introduces Progressive Inference, a method to explain decoder-only transformer models' predictions by evaluating intermediate classifications at different input positions, using either Single Pass or Multi Pass approaches.

Chasing Convex Functions with Long-term Constraints

https://openreview.net/forum?id=hRBdOHVn7y

Compressor summary: The paper studies online metric problems with long-term constraints for resource allocation in sustainable energy/computing systems, and proposes optimal algorithms for bounded hitting cost gradients and weighted $\ell_1$ metrics.

Subequivariant Reinforcement Learning in 3D Multi-Entity Physical Environments

https://openreview.net/forum?id=hQpUhySEJi

Compressor summary: The paper introduces Subequiriant Hierarchical Neural Networks (SHNN) for learning policies in multi-entity 3D environments, which use task assignment and subequivariance to reduce complexity and improve performance on a new benchmark called Multi-entity Benchmark (MEBEN).

Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis

https://openreview.net/forum?id=hLuNVjRnY3

Compressor summary: The authors propose a new model merging algorithm called CCA Merge, which uses Canonical Correlation Analysis to maximize the correlations between linear combinations of the models' features and improves performance over past methods in both two-model and multi-model settings.

Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization

https://openreview.net/forum?id=hLGxDYo0eF

Compressor summary: The paper analyzes the performance of a policy optimization reinforcement learning algorithm that learns from human feedback without knowing the reward function, providing bounds on query complexity and novel techniques for inferring reward parameters.

Hyperbolic Active Learning for Semantic Segmentation under Domain Shift

https://openreview.net/forum?id=hKdJPMQvew

Compressor summary: HALO is a hyperbolic neural network that uses epistemic uncertainty to select data points for pixel-level active learning, achieving state-of-the-art results in semantic segmentation under domain shift with minimal supervision.

Multiply Robust Estimation for Local Distribution Shifts with Multiple Domains

https://openreview.net/forum?id=hJaWoU3Emh

Compressor summary: The paper proposes a two-stage method to improve machine learning models' performance and generalization when data distributions vary across multiple segments of the population, using linear combinations and refinement steps for each segment.

Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows

https://openreview.net/forum?id=hG6gddAKnJ

Compressor summary: The paper explores and characterizes conservation laws in non-Euclidean geometries and momentum-based dynamics for neural network training.

Discovering Features with Synergistic Interactions in Multiple Views

https://openreview.net/forum?id=hFEgae0od4

Compressor summary: The paper proposes a deep learning method that selects synergistic feature subsets from multi-view data using interaction information to understand target outcomes better.

SiBBlInGS: Similarity-driven Building-Block Inference using Graphs across States

https://openreview.net/forum?id=h8aTi32tul

Compressor summary: SiBBlInGS is a graph-based method for discovering interpretable units in time series data across different states that captures complex variability and adapts to varying session lengths and sample sizes.

Infinite-Horizon Distributionally Robust Regret-Optimal Control

https://openreview.net/forum?id=h3SGdpI4Ta

Compressor summary: The paper studies how to control linear systems with uncertain disturbances within a Wasserstein-2 ambiguity set, and proposes efficient algorithms to compute near-optimal control policies.

Adaptive Sampling of k-Space in Magnetic Resonance for Rapid Pathology Prediction

https://openreview.net/forum?id=h2uBuQvpp8

Compressor summary: ASMR is a method that learns to selectively sample k-space measurements for faster and accurate disease detection in MR imaging.

Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learning

https://openreview.net/forum?id=gzis9n5r7e

Compressor summary: The paper proposes a method called Transition Discriminator-based Imitation Learning (TDIL) that uses a transition discriminator to compute surrogate rewards from one expert trajectory, addressing reward sparsity and achieving expert-level performance in single-demonstration imitation learning.

Q-Probe: A Lightweight Approach to Reward Maximization for Language Models

https://openreview.net/forum?id=gxOQEMRbRa

Compressor summary: Q-probing adapts pre-trained language models to new tasks using a linear function on embeddings that reweights candidate completions based on task-specific rewards or policy objectives, and can improve performance in data-limited regimes.

Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency

https://openreview.net/forum?id=guFsTBXsov

Compressor summary: Minimal Frame Averaging (MFA) is a new framework for achieving exact equivariance in machine learning systems by constructing minimal frames that encode symmetries efficiently and effectively across various tasks, including physics simulations and complex-valued domains.

Generalized Preference Optimization: A Unified Approach to Offline Alignment

https://openreview.net/forum?id=gu3nacA9AH

Compressor summary: Generalized preference optimization is a unified framework for offline learning that encompasses existing methods and allows tuning of regularization through convex functions.

LAGMA: LAtent Goal-guided Multi-Agent Reinforcement Learning

https://openreview.net/forum?id=gtYdvSGMYV

Compressor summary: LAGMA improves cooperative multi-agent reinforcement learning by generating a goal-reaching trajectory in latent space and providing an incentive for agents to follow it.

Fast, Scalable, Warm-Start Semidefinite Programming with Spectral Bundling and Sketching

https://openreview.net/forum?id=gqA8ZHO0j8

Compressor summary: USBS is a fast and scalable spectral bundle method for solving SDPs that can leverage warm-start initialization, achieving dramatic speedups compared to existing methods.

StackSight: Unveiling WebAssembly through Large Language Models and Neurosymbolic Chain-of-Thought Decompilation

https://openreview.net/forum?id=gn5AsHIIwb

Compressor summary: StackSight is a novel technique that combines LLMs with program analysis to decompile complex WebAssembly code into readable C++ snippets, improving understanding and decompilation.

Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation

https://openreview.net/forum?id=glfcwSsks8

Compressor summary: The authors investigate the geometry of Large Language Models (LLMs) to understand their inner mechanisms and develop novel solutions for tasks like toxicity detection.

VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context

https://openreview.net/forum?id=gjoUXwuZdy

Compressor summary: The authors introduce VisionGraph, a benchmark for testing large multimodal models' ability to solve graph theory problems using a Description-Program-Reasoning chain that improves their accuracy and performance.

SelfIE: Self-Interpretation of Large Language Model Embeddings

https://openreview.net/forum?id=gjgRKbdYR7

Compressor summary: SelfIE is a framework that lets large language models explain their own reasoning in natural language, enabling control over their responses for reliability, transparency, and future development.

Can AI Assistants Know What They Don't Know?

https://openreview.net/forum?id=girxGkdECL

Compressor summary: Key points: - LLMs can make factual errors in open-domain question answering - The paper explores if AI assistants can know what they don't know and express it through natural language - They create an Idk dataset and align the assistant with it using different methods - The aligned assistant is more truthful and declines less often Summary: The paper investigates how to make AI assistants aware of their knowledge gaps and express them in natural language, creating a new dataset and alignment methods that improve their truthfulness.

PolySketchFormer: Fast Transformers via Sketching Polynomial Kernels

https://openreview.net/forum?id=ghYrfdJfjK

Compressor summary: PolySketchFormer is a fast and accurate Transformer-based language model that replaces softmax attention with polynomial attention and uses sketching techniques for linear-time computation.

Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision

https://openreview.net/forum?id=ghNRg2mEgN

Compressor summary: The study examines whether weak model supervision can elicit the full capabilities of stronger models and finds that while there is some success, techniques like reinforcement learning may be needed for superhuman models without further work.

First-Order Manifold Data Augmentation for Regression Learning

https://openreview.net/forum?id=geajNKab7g

Compressor summary: FOMA is a new domain-independent data augmentation method for regression problems that samples from the tangent planes of the train distribution and improves generalization and robustness.

Quality-Weighted Vendi Scores And Their Application To Diverse Experimental Design

https://openreview.net/forum?id=gbD9MAc9p0

Compressor summary: The paper proposes quality-weighted Vendi scores to balance quality and diversity in experimental design, improving data collection and discovery in various applications.

Accelerating Federated Learning with Quick Distributed Mean Estimation

https://openreview.net/forum?id=gWEwIlZrbQ

Compressor summary: The paper proposes a novel DME method for federated learning that improves the NMSE guarantee and computational efficiency by using off-the-shelf solvers and quantization.

Iterated Denoising Energy Matching for Sampling from Boltzmann Densities

https://openreview.net/forum?id=gVjMwLDFoQ

Compressor summary: iDEM is a fast and scalable algorithm that uses only the energy function and its gradient to generate independent samples from unnormalized probability distributions, achieving state-of-the-art performance on various tasks.

Long Range Propagation on Continuous-Time Dynamic Graphs

https://openreview.net/forum?id=gVg8V9isul

Compressor summary: CTAN is a new method for modeling spatio-temporal information in C-TDGs that outperforms existing methods on long-range tasks.

VNN: Verification-Friendly Neural Networks with Hard Robustness Guarantees

https://openreview.net/forum?id=gUFufRkzjV

Compressor summary: The text proposes a framework to generate Verification-Friendly Neural Networks that balance prediction performance and verification-friendliness, enabling more robustness in safety-critical applications.

Subsampling is not Magic: Why Large Batch Sizes Work for Differentially Private Stochastic Optimisation

https://openreview.net/forum?id=gTBjkJvadC

Compressor summary: Large batch sizes reduce gradient variance in differentially private stochastic gradient descent (DP-SGD) by decreasing subsampling-induced variance and are beneficial especially in the asymptotic regime.

Quasi-Monte Carlo Features for Kernel Approximation

https://openreview.net/forum?id=gSMUjrkRRk

Compressor summary: Quasi-Monte Carlo methods can improve kernel approximation error for certain kernels, leading to fewer random features needed and better performance in kernel ridge regression.

Representing Molecules as Random Walks Over Interpretable Grammars

https://openreview.net/forum?id=gS3nc9iUrH

Compressor summary: The paper proposes a data-efficient and interpretable graph grammar model for representing and reasoning over complex molecular structures, enabling better design and property prediction.

Applying language models to algebraic topology: generating simplicial cycles using multi-labeling in Wu's formula

https://openreview.net/forum?id=gQz30hTkRE

Compressor summary: The paper proposes machine learning methods to understand the group structure of homotopy groups of spheres by generating simplicial cycles from algorithmic datasets related to Dyck languages.

Position: A Roadmap to Pluralistic Alignment

https://openreview.net/forum?id=gQpBnRHwxM

Compressor summary: The text discusses the importance of designing AI systems to serve diverse human values and suggests a roadmap for achieving pluralistic alignment using large language models and different types of benchmarks.

Discovering Environments with XRM

https://openreview.net/forum?id=gPStP3FSY9

Compressor summary: The paper introduces Cross-Risk Minimization (XRM), an algorithm that automatically discovers environments within datasets for robust out-of-distribution generalization without relying on human-annotated environment labels.

Stochastic Gradient Flow Dynamics of Test Risk and its Exact Solution for Weak Features

https://openreview.net/forum?id=gPBMkJG7bt

Compressor summary: The paper studies how test risk changes with learning rate in continuous time stochastic gradient flow dynamics and applies the theory to a weak features model, finding the impact of stochasticity on test risk.

New Bounds on the Cohesion of Complete-link and Other Linkage Methods for Agglomerative Clustering

https://openreview.net/forum?id=gL5djEYLx2

Compressor summary: This paper improves existing bounds on the maximum diameter and cohesion of complete-link and average-link hierarchical clustering algorithms for metric spaces, supporting the preference of complete-link over single-link for producing compact clusters.

Causal-IQA: Towards the Generalization of Image Quality Assessment Based on Causal Inference

https://openreview.net/forum?id=gKPkipJ3gm

Compressor summary: This paper introduces Causal-IQA, an end-to-end blind Image Quality Assessment method that uses causality to improve estimation accuracy and generalization by mitigating confounding effects between distortion types, image contents, and human ratings.

Learning to Intervene on Concept Bottlenecks

https://openreview.net/forum?id=gEbl6XNLK6

Compressor summary: CB2Ms are memory-enhanced concept bottleneck models that learn from past interventions and can generalize them to new situations, improving interpretability and performance.

Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matching

https://openreview.net/forum?id=gE7qZurGH3

Compressor summary: The paper proposes a new method for lossless graph condensation by using curriculum learning, expanding window matching, and a customized loss function to train expert trajectories that preserve the original graph's structure and performance.

Small-loss Adaptive Regret for Online Convex Optimization

https://openreview.net/forum?id=gDQuupz8mm

Compressor summary: This paper proposes new algorithms that achieve small-loss adaptive regret bounds for various types of convex functions and handle changing environments.

Can Gaussian Sketching Converge Faster on a Preconditioned Landscape?

https://openreview.net/forum?id=gB3E8IwQZy

Compressor summary: Key points: - The paper introduces GSGD, a novel gradient sketching method for large-scale optimization - GSGD does not require importance sampling but can match the convergence rate of methods with it - GSGD can exploit non-smooth regularization terms for faster convergence - Experimental results show the effectiveness and efficiency of GSGD Summary: The paper presents GSGD, a new gradient sketching method that achieves fast convergence in large-scale optimization without importance sampling or exploiting smooth regularization terms.

SceneCraft: An LLM Agent for Synthesizing 3D Scenes as Blender Code

https://openreview.net/forum?id=gAyzjHw2ml

Compressor summary: SceneCraft is an LLM agent that creates Blender scripts from text descriptions to render complex scenes, using spatial planning, image analysis, and library learning.

Policy-conditioned Environment Models are More Generalizable

https://openreview.net/forum?id=g9mYBdooPA

Compressor summary: Policy-conditioned model (PCM) learning improves reinforcement learning by adapting the dynamics model to different evaluation policies for better prediction accuracy.

Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion

https://openreview.net/forum?id=g8AigOTNXL

Compressor summary: The paper proposes SCG, a new method for symbolic music generation that works with non-differentiable rules and improves quality and controllability over existing methods.

Learning to Predict Mutational Effects of Protein-Protein Interactions by Microenvironment-aware Hierarchical Prompt Learning

https://openreview.net/forum?id=g89jAdrnAF

Compressor summary: Key points: - The paper proposes a method to predict the effects of amino acid mutations on protein-protein binding using hierarchical prompt learning - The method models the joint distribution of each mutation with various microenvironmental features - The method outperforms existing pre-training-based methods and can be applied to optimize antibodies against SARS-CoV-2 Summary: The paper introduces a hierarchical prompt learning framework that predicts how amino acid mutations affect protein-protein binding by modeling the microenvironmental changes. The method is more efficient and effective than previous pre-training methods and can help design better antibodies against COVID-19.

Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environments

https://openreview.net/forum?id=g43yUNWX4V

Compressor summary: The paper proposes two algorithms for Federated Reinforcement Learning (FRL) that can handle large levels of environment heterogeneity and achieve state-of-the-art convergence results.

Latent variable model for high-dimensional point process with structured missingness

https://openreview.net/forum?id=g1Gf0hoPSz

Compressor summary: The paper proposes a latent-variable model that uses Gaussian processes to handle high-dimensional data with missing values and unknown measurement times in various fields.

MOKD: Cross-domain Finetuning for Few-shot Classification via Maximizing Optimized Kernel Dependence

https://openreview.net/forum?id=fz9PaJNViP

Compressor summary: MOKD is a bi-level optimization framework that improves few-shot classification by learning class-specific representations and maximizing kernel dependence between them and labels, while minimizing dependence among all samples.

Feature Importance Disparities for Data Bias Investigations

https://openreview.net/forum?id=fywWm06IGn

Compressor summary: The paper proposes a method to identify features that have different importance in subgroups of a dataset, which can help detect and rectify biases in classifiers trained on such data.

Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarks

https://openreview.net/forum?id=fwxnHViGNj

Compressor summary: The paper compares multi-task benchmarks in machine learning to electoral systems, shows a trade-off between diversity and sensitivity to irrelevant changes, and introduces new quantitative measures and approximation algorithms for these aspects.

SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks

https://openreview.net/forum?id=fuX4hyLPmO

Compressor summary: The paper introduces SLEB, a new pruning method that removes redundant transformer blocks from large language models to speed up inference while maintaining accuracy and perplexity.

On Universally Optimal Algorithms for A/B Testing

https://openreview.net/forum?id=ft5jK9uPgC

Compressor summary: No algorithm beats uniform sampling in A/B testing with fixed budget and Bernoulli rewards.

On Mechanistic Knowledge Localization in Text-to-Image Generative Models

https://openreview.net/forum?id=fsVBsxjRER

Compressor summary: The authors propose LocoGen and LocoEdit, methods that use causal tracing to locate and edit specific visual attributes in text-to-image models, enabling efficient model editing.

Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo

https://openreview.net/forum?id=frA0NNBS1n

Compressor summary: The paper proposes using Sequential Monte Carlo with learned twist functions to improve probabilistic inference for various language model techniques such as safety, capability, and harmlessness training.

Differentially Private Bias-Term Fine-tuning of Foundation Models

https://openreview.net/forum?id=fqeANcjBMT

Compressor summary: DP-BiTFiT is a new method for privacy-preserving fine-tuning of large pre-trained models that achieves high accuracy, efficiency, and parameter efficiency.

Encodings for Prediction-based Neural Architecture Search

https://openreview.net/forum?id=fqPH6ejwGi

Compressor summary: This paper explores different types of neural network architectures encodings, introduces unified encodings, and proposes FLAN, a predictor that significantly reduces the cost of training NAS accuracy predictors.

Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch

https://openreview.net/forum?id=fq0NaiU8Ex

Compressor summary: DARE is a method that sparsifies and merges language model parameters from different tasks without retraining, enabling larger models with diverse capabilities and improved performance.

Using AI Uncertainty Quantification to Improve Human Decision-Making

https://openreview.net/forum?id=fowZNENcVJ

Compressor summary: The text discusses how providing uncertainty information from AI can enhance human decision-making and reports positive results from two experiments testing this idea.

Pursuing Overall Welfare in Federated Learning through Sequential Decision Making

https://openreview.net/forum?id=foPMkomvk1

Compressor summary: The text proposes a method to achieve client-level fairness in federated learning by using an adaptive aggregation scheme that optimizes online convex optimization and improves decision making for cross-device and cross-silo settings.

EDISON: Enhanced Dictionary-Induced Tensorized Incomplete Multi-View Clustering with Gaussian Error Rank Minimization

https://openreview.net/forum?id=fiugPLSXjK

Compressor summary: The paper proposes a robust and efficient incomplete multi-view clustering method called EDISON, which uses an enhanced dictionary representation strategy and Gaussian error rank approximation for tensor data.

SFC: Achieve Accurate Fast Convolution under Low-precision Arithmetic

https://openreview.net/forum?id=fgBWtOw66T

Compressor summary: SFC is a new transform for fast and accurate quantized convolution using symbolic computing and the Discrete Fourier Transform, achieving better efficiency than Winograd and FFT algorithms.

Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often!

https://openreview.net/forum?id=ffS0aYP6mk

Compressor summary: The paper studies how frequently communicating in Federated Learning affects generalization error and provides bounds and experiments for different learning models.

MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models

https://openreview.net/forum?id=ffLblkoCw8

Compressor summary: MAGDi is a method to improve reasoning in smaller models by distilling knowledge from multiple LLMs using graph representations and three objective functions, achieving efficiency and generalization improvements.

Prompting is a Double-Edged Sword: Improving Worst-Group Robustness of Foundation Models

https://openreview.net/forum?id=fdroxYsgzQ

Compressor summary: Summary: The paper proposes Prompting for Robustness (PfR), a method to improve machine learning models' robustness against spurious correlations by using foundation models to predict the spurious attribute and learn a classifier that performs well across different labels.

Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Sampling

https://openreview.net/forum?id=fVg9YrSllr

Compressor summary: The text introduces a benchmark to evaluate sampling methods from intractable distributions using a standardized task suite and various performance criteria, including new metrics for mode collapse.

Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints

https://openreview.net/forum?id=fSnMqHZ8xr

Compressor summary: BE-CBO is a new Bayesian optimization method that learns constraints with neural networks to efficiently explore the boundary between feasible and infeasible designs in black-box optimization problems.

Graph Neural Networks Use Graphs When They Shouldn't

https://openreview.net/forum?id=fSNHK7mu3j

Compressor summary: The paper shows that Graph Neural Networks (GNNs) often use the input graph structure even when it's not needed, leading to suboptimal solutions, and suggests using regular graphs to improve performance and avoid this bias.

Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning

https://openreview.net/forum?id=fRG45xL1WT

Compressor summary: FeatLLM is a novel in-context learning framework that uses LLMs to generate optimal features for tabular predictions, achieving high performance few-shot learning.

Riemannian Accelerated Zeroth-order Algorithm: Improved Robustness and Lower Query Complexity

https://openreview.net/forum?id=fPwWfoyxL1

Compressor summary: Key points: - The paper proposes a zeroth-order algorithm for optimization problems on Riemannian manifolds with improved efficiency and robustness. - The algorithm achieves state-of-the-art function query complexity and almost sure convergence in the asymptotic sense. - The algorithm requires larger smoothing parameters, improving the existing result by a factor of $ ilde{\mathcal{O}}(\epsilon^{7/8}d^{-1/2})$. Summary: The paper presents a Riemannian accelerated zeroth-order algorithm that is more efficient and stable than previous ones for optimization problems on Riemannian manifolds.

Learning Low-dimensional Latent Dynamics from High-dimensional Observations: Non-asymptotics and Lower Bounds

https://openreview.net/forum?id=fOBas5H4Xc

Compressor summary: Key points: - The paper proposes an algorithm to learn low-dimensional models from high-dimensional observations of LTI systems. - The algorithm has an optimal sample complexity up to logarithmic factors and dimension-independent constants. - The paper also considers a meta-learning problem where the observer column space can be learned from multiple LTI systems. Summary: The paper presents an optimal algorithm for learning low-dimensional models of LTI systems from high-dimensional data, and extends it to a meta-learning problem with collective observer column space learning.

Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition

https://openreview.net/forum?id=fO31YAyNbI

Compressor summary: The paper proposes MotionEpic, a novel video Multimodal Large Language Model, and VoT, a Video-of-Thought reasoning framework that use spatial-temporal scene graphs and Chain-of-Thought techniques to improve video understanding and reasoning.

Scale-Free Image Keypoints Using Differentiable Persistent Homology

https://openreview.net/forum?id=fNJbcxhxRj

Compressor summary: The paper presents MorseDet, a novel computer vision method that uses algebraic topology tools to learn keypoints with scale-invariant and flexible performance.

Reshape and Adapt for Output Quantization (RAOQ): Quantization-aware Training for In-memory Computing Systems

https://openreview.net/forum?id=fM9xTkpAdu

Compressor summary: RAOQ is a method to improve in-memory computing by mitigating ADC quantization error and adapting AI models for better performance in computer vision and NLP tasks.

Amend to Alignment: Decoupled Prompt Tuning for Mitigating Spurious Correlation in Vision-Language Models

https://openreview.net/forum?id=f8G2KSCSdp

Compressor summary: The paper introduces CoOPood, a fine-grained prompt tuning method for VLMs that aligns text with invariant features and avoids spurious ones, improving OOD generalization.

How Deep Do We Need: Accelerating Training and Inference of Neural ODEs via Control Perspective

https://openreview.net/forum?id=f6QenZyyeP

Compressor summary: The paper proposes two methods to optimize Neural ODEs, improving their training and inference speed by drawing inspiration from control theory.

Self-Composing Policies for Scalable Continual Reinforcement Learning

https://openreview.net/forum?id=f5gtX2VWSB

Compressor summary: A growable and modular neural network architecture can learn continually from previous tasks without forgetting or interference, and scales well with the number of tasks.

Data Poisoning Attacks against Conformal Prediction

https://openreview.net/forum?id=f49AkFT5jf

Compressor summary: The paper proposes new black-box data poisoning attacks against conformal prediction methods, which can manipulate the uncertainty of specific examples more effectively than traditional attacks.

Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning

https://openreview.net/forum?id=f47ZK6gy3I

Compressor summary: The paper introduces FineSSL, a new semi-supervised learning approach that adapts pre-trained foundation models, improving their performance, reducing training cost, and integrating with other algorithms.

HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

https://openreview.net/forum?id=f3TUipYU3U

Compressor summary: HarmBench is a standardized framework for evaluating automated red teaming methods against large language models, revealing new insights and improving LLM robustness.

Dense Reward for Free in Reinforcement Learning from Human Feedback

https://openreview.net/forum?id=eyxVRMrZ4m

Compressor summary: The authors propose a method to improve reinforcement learning for language models by using attention weights from the reward model to redistribute the reward, making it easier to optimize and potentially leading to better results.

Image Fusion via Vision-Language Model

https://openreview.net/forum?id=eqY64Z1rsT

Compressor summary: FILM is a novel image fusion method that uses textual descriptions generated by ChatGPT to guide the fusion process, enhancing feature extraction and contextual understanding.

Asymptotically Optimal and Computationally Efficient Average Treatment Effect Estimation in A/B testing

https://openreview.net/forum?id=eqIGoEoI10

Compressor summary: The paper proposes adaptive policies for A/B testing that estimate the average treatment effect with a desired confidence interval width and probability, using an optimal sample size lower bound derived from a non-convex optimization problem.

Agnostic Learning of Mixed Linear Regressions with EM and AM Algorithms

https://openreview.net/forum?id=eo88noTbb5

Compressor summary: The paper analyzes how agnostic learning of mixed linear regression can be achieved by EM and AM algorithms without assuming generative models.

AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers

https://openreview.net/forum?id=emtXYlBrNF

Compressor summary: The paper proposes a method to attribute and understand large language models by extending Layer-wise Relevance Propagation to handle attention layers, improving faithfulness and efficiency over existing methods.

NDOT: Neuronal Dynamics-based Online Training for Spiking Neural Networks

https://openreview.net/forum?id=elF0QoBSFV

Compressor summary: NDOT is a new online training method for SNNs that uses neuronal dynamics to compute gradients efficiently and accurately on large-scale datasets.

Contrastive Learning for Clinical Outcome Prediction with Partial Data Sources

https://openreview.net/forum?id=elCOPIm4Xw

Compressor summary: CLOPPS is a machine learning model that predicts clinical outcomes using information from different data sources and performs better than existing models in real-world scenarios.

Conformal Predictions under Markovian Data

https://openreview.net/forum?id=efzkSbpyRw

Compressor summary: The paper studies how split Conformal Prediction performs on Markovian data, showing its coverage gap depends on the mixing time of the chain, and proposes a method called $K$-split CP that adapts to the data's properties.

Interaction-based Retrieval-augmented Diffusion Models for Protein-specific 3D Molecule Generation

https://openreview.net/forum?id=eejhD9FCP3

Compressor summary: The IRDiff model uses a network of protein-molecule interactions to generate ligands that bind well to specific proteins, using references with desired properties as guidance.

One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts

https://openreview.net/forum?id=edHLN40DWu

Compressor summary: Mixture-of-Prompts (MoP) is a method that automates instruction design for large language models by dividing the problem space into sub-regions, each governed by a specialized expert with an instruction and demos, achieving high win rates on benchmarks.

Deletion-Anticipative Data Selection with a Limited Budget

https://openreview.net/forum?id=ecvuJWE1YY

Compressor summary: The text discusses using supervised data subset selection and active learning techniques to reduce data acquisition costs for machine learning models, but considering the impact of future data deletions under GDPR and proposing deletion-anticipative data selection methods to optimize utility.

Speech Self-Supervised Learning Using Diffusion Model Synthetic Data

https://openreview.net/forum?id=ecnpYYHjt9

Compressor summary: DiffS4L uses diffusion models to generate synthetic speech data with different variations, improving self-supervised learning for low-resource languages and under privacy concerns.

MF-CLR: Multi-Frequency Contrastive Learning Representation for Time Series

https://openreview.net/forum?id=ecO7WOIlMD

Compressor summary: MF-CLR is a self-supervised contrastive learning method for representing multi-frequency time series data, achieving excellent results on various financial downstream tasks.

Harnessing Hierarchical Label Distribution Variations in Test Agnostic Long-tail Recognition

https://openreview.net/forum?id=ebt5BfRHcW

Compressor summary: The paper presents a new mixture-of-expert approach, DirMixE, that captures both global and local variations in test label distributions for long-tail recognition tasks.

Leveraging Attractor Dynamics in Spatial Navigation for Better Language Parsing

https://openreview.net/forum?id=eapFRURALQ

Compressor summary: The PHE-trinity model explores how the hippocampus contributes to language comprehension by using a modular continuous attractor network to represent syntactic structure and two separate input streams, and demonstrates its effectiveness in learning from limited data.

Learning Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking

https://openreview.net/forum?id=eaNLvrP8n1

Compressor summary: AVTrack is a framework that adapts the computation of transformer-based visual trackers for real-time UAV tracking by dynamically optimizing ViT architecture and learning view-invariant representations.

Position: Leverage Foundational Models for Black-Box Optimization

https://openreview.net/forum?id=ea2MgKn3sV

Compressor summary: The paper explores how large language models can revolutionize black-box optimization by using their comprehension, flexibility, and performance prediction abilities.

Optimal Hessian/Jacobian-Free Nonconvex-PL Bilevel Optimization

https://openreview.net/forum?id=eZiQWM5U0E

Compressor summary: The paper proposes a new method (HJFBiO) for solving nonconvex-PL bilevel optimization problems without Hessian/Jacobian matrices, with optimal convergence and gradient complexity.

Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds

https://openreview.net/forum?id=eY98MVffrD

Compressor summary: The paper proposes learning-rate-free algorithms for stochastic optimization over Riemannian manifolds that eliminate hand-tuning and provide optimal convergence guarantees.

Towards Theoretical Understanding of Learning Large-scale Dependent Data via Random Features

https://openreview.net/forum?id=eY4jrFe6Qc

Compressor summary: The paper analyzes how kernel ridge regression with random feature mapping performs on large-scale nonparametric regression with different types of data dependence structures, showing optimality under exponential decay but sub-optimality under polynomial decay.

Novel Spectral Algorithms for the Partial Credit Model

https://openreview.net/forum?id=eW0pZmziBH

Compressor summary: The paper introduces a fast and accurate statistical algorithm for inference under the Partial Credit Model used in psychometrics and other fields, with applications to education, recommendation systems, and finance.

StrokeNUWA—Tokenizing Strokes for Vector Graphic Synthesis

https://openreview.net/forum?id=eVlx8DaG9h

Compressor summary: StrokeNUWA is a method that uses vector graphics and semantic "stroke" tokens to enable more natural and efficient visual synthesis with large language models.

Graph-enhanced Large Language Models in Asynchronous Plan Reasoning

https://openreview.net/forum?id=eVGpdivOnQ

Compressor summary: The study investigates if large language models can plan asynchronously, finding that they struggle without illustrations, and proposes a new technique called Plan Like a Graph to improve performance.

Calibration Bottleneck: Over-compressed Representations are Less Calibratable

https://openreview.net/forum?id=eRThYD9BGD

Compressor summary: The paper proposes a new training method (PLP) that improves uncertainty calibration in deep neural networks by avoiding over-compression of top layers and using weak classifier heads.

Fast Decision Boundary based Out-of-Distribution Detector

https://openreview.net/forum?id=eQaOb4r6YC

Compressor summary: The paper proposes a fast and effective out-of-distribution detector that uses feature distances to decision boundaries without auxiliary models, achieving good performance and low latency.

Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddings

https://openreview.net/forum?id=ePDnv4xESI

Compressor summary: MixPro is a data-efficient method for adapting pretrained models to new distributions using mixed embeddings and linear classifiers.

Ensemble Pruning for Out-of-distribution Generalization

https://openreview.net/forum?id=eP3vsbB5wW

Compressor summary: The paper proposes a method to prune deep neural network ensembles under distribution shifts using a topology graph, improving predictive diversity and generalization performance on out-of-distribution data.

Characteristic Guidance: Non-linear Correction for Diffusion Model at Large Guidance Scale

https://openreview.net/forum?id=eOtjMYdGLt

Compressor summary: Characteristic guidance is a non-linear correction method for DDPMs that improves semantic features and image quality by respecting the Fokker-Planck equation without additional training.

Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment

https://openreview.net/forum?id=eN1T7I7OpZ

Compressor summary: The paper presents Shūkai, a DRL agent for fighting games that improves generalizability and sample efficiency with Heterogeneous League Training and specific rewards, and shows its effectiveness in Naruto Mobile.

Towards efficient deep spiking neural networks construction with spiking activity based pruning

https://openreview.net/forum?id=eMQyb1tvvc

Compressor summary: The paper proposes a method to compress spiking neural networks by dynamically pruning and regenerating convolutional kernels based on their activity levels, achieving low-power and high-efficiency performance while maintaining model accuracy.

RoboMP$^2$: A Robotic Multimodal Perception-Planning Framework with Multimodal Large Language Models

https://openreview.net/forum?id=eJFQROkaj0

Compressor summary: The paper introduces a new framework for robotic manipulation that combines multimodal perception and planning using tailored language models and retrieval-based policy learning.

Simplicity Bias of Two-Layer Networks beyond Linearly Separable Data

https://openreview.net/forum?id=eGZH3HCuGm

Compressor summary: This paper investigates how simplicity bias affects general neural networks, especially two-layer ones, and suggests that features learned in the middle stages of training may improve out-of-distribution generalization.

OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning

https://openreview.net/forum?id=eG42XBhV9a

Compressor summary: The paper introduces OLLIE, a method that improves offline-to-online Imitation Learning by learning a better policy initialization and an aligned discriminator initialization, achieving better performance and efficiency in various domains.

Provably Efficient Exploration in Quantum Reinforcement Learning with Logarithmic Worst-Case Regret

https://openreview.net/forum?id=eFvoL7BOny

Compressor summary: The paper proposes a novel quantum reinforcement learning algorithm that achieves provably efficient exploration-exploitation trade-off and breaks the $\Omega(\sqrt{T})$-regret barrier in classical RL.

Generalization Bounds for Heavy-Tailed SDEs through the Fractional Fokker-Planck Equation

https://openreview.net/forum?id=eFSppFiVYG

Compressor summary: The paper proposes new generalization bounds for heavy-tailed stochastic optimization algorithms using fractional Fokker-Planck equation and shows that heavy tails can be beneficial or harmful depending on the problem structure.

Harnessing Neural Unit Dynamics for Effective and Scalable Class-Incremental Learning

https://openreview.net/forum?id=eDtty9ZCvt

Compressor summary: The paper proposes AutoActivator, a connectionist model with adaptive neural unit dynamics for class-incremental learning, which can expand its capacity when needed and reactivate required units at inference time without forgetting old classes.

Break the Sequential Dependency of LLM Inference Using Lookahead Decoding

https://openreview.net/forum?id=eDjvSFOkXw

Compressor summary: Lookahead decoding is a parallel algorithm that accelerates large language model decoding without needing auxiliary models or data stores, achieving up to 4x speedup on multiple GPUs.

Improving Instruction Following in Language Models through Proxy-Based Uncertainty Estimation

https://openreview.net/forum?id=eCCaHZKdl4

Compressor summary: The Uncertainty-aware Reward Model (URM) improves instruction following in language models by estimating response quality and uncertainty using Bayesian approximation, leading to better performance on benchmarks.

Saliency strikes back: How filtering out high frequencies improves white-box explanations

https://openreview.net/forum?id=eC1OOpOGZW

Compressor summary: FORGrad is a new method that improves the performance of white-box attribution methods by filtering out high-frequency artifacts in gradient signals, making them more accurate and computationally efficient for model explanations.

Simple linear attention language models balance the recall-throughput tradeoff

https://openreview.net/forum?id=e93ffDcpH3

Compressor summary: The text explores a new language model architecture called BASED that balances memory efficiency and recall ability by combining linear and sliding window attention, achieving competitive results on perplexity and real-world tasks.

Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?

https://openreview.net/forum?id=e76GrGhIgf

Compressor summary: The paper explores the dense-to-sparse gating MoE, proposes a novel activation gate to improve convergence rates, and validates the results with simulations.

Adaptively Learning to Select-Rank in Online Platforms

https://openreview.net/forum?id=e5tA3Apbmy

Compressor summary: The paper proposes a user response model to adaptively rank items for heterogeneous users, using contextual bandits and an upper confidence bound, achieving low regret and improving user satisfaction.

Position: A Call for Embodied AI

https://openreview.net/forum?id=e5admkWKgV

Compressor summary: Embodied AI (E-AI) is proposed as a key step toward Artificial General Intelligence (AGI), focusing on embodiment, cognitive architectures, and active inference to enhance AI's ability to communicate, collaborate, and coexist with humans.

Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning

https://openreview.net/forum?id=e3geukCBw6

Compressor summary: Momentor is a Video-LLM that can understand and locate specific video segments using a large-scale dataset called Moment-10M.

Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression

https://openreview.net/forum?id=e3Dpq3WdMv

Compressor summary: This study evaluates the trade-off between compression efficiency and trustworthiness in large language models using various techniques and dimensions, finding that quantization within a moderate bit range is more effective than pruning for achieving both goals.

Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient

https://openreview.net/forum?id=e1jPdRJeo7

Compressor summary: The paper introduces a new method called Zeroth-order Proximal Double Variance Reduction (ZPDVR) that reduces sampling and coordinate-wise variances in zeroth-order optimization, with less computational cost and better performance than existing methods.

Positive Concave Deep Equilibrium Models

https://openreview.net/forum?id=e0SKaKEEdr

Compressor summary: pcDEQ models improve DEQ models by ensuring positive weights and concave activation functions, providing theoretical guarantees for their fixed point and convergence, and performing well in language modeling and computer vision tasks.

Discovering Bias in Latent Space: An Unsupervised Debiasing Approach

https://openreview.net/forum?id=dztd61efGy

Compressor summary: SteerFair is a method to reduce biases in foundation models' QA capabilities by steering their internal representations away from spurious associations between input characteristics and correctness likelihood.

Imitation Learning from Purified Demonstrations

https://openreview.net/forum?id=dyfsPNuYCk

Compressor summary: The paper proposes a two-step purification method using diffusion models to remove noises in imperfect expert demonstrations for imitation learning, improving performance in real-world scenarios.

Robust Inverse Graphics via Probabilistic Inference

https://openreview.net/forum?id=dwWef5w2cR

Compressor summary: The paper proposes a Bayesian method called robust inverse graphics to infer 3D scenes from single images with unknown corruptions, using a strong scene prior and an uninformative uniform corruption prior, and shows that it outperforms other methods.

Slot Abstractors: Toward Scalable Abstract Visual Reasoning

https://openreview.net/forum?id=duyl8sy8qV

Compressor summary: Slot Abstractors is a new approach that combines slot-based methods and relational abstraction to enable scalable abstract visual reasoning with many objects and relations.

Repeat After Me: Transformers are Better than State Space Models at Copying

https://openreview.net/forum?id=duRRoGeoQT

Compressor summary: Key points: - Transformers are widely used for sequence modeling but GSSMs have fixed-size latent state that does not depend on sequence length - The paper shows that GSSMs are less efficient and generalize worse than transformers on tasks that require copying from the input context - The paper evaluates pretrained language models and finds transformers significantly outperform GSSMs at copying and retrieving information from context Summary: The paper compares transformers and GSSMs, showing that transformers are superior for tasks that need copying from the input context.

Feedback Efficient Online Fine-Tuning of Diffusion Models

https://openreview.net/forum?id=dtVlc9ybTm

Compressor summary: The paper proposes a new RL method to find high-reward samples in complex distributions by exploring the feasible manifold efficiently, with theory and experiments in images, biological sequences, and molecules.

Position: Automatic Environment Shaping is the Next Frontier in RL

https://openreview.net/forum?id=dslUyy1rN4

Compressor summary: The text argues that to advance robotics with sim-to-real reinforcement learning, researchers should focus on automating environment shaping procedures rather than tuning RL algorithms.

Community-Invariant Graph Contrastive Learning

https://openreview.net/forum?id=dskLpg8WFb

Compressor summary: The paper proposes a graph contrastive learning method that preserves the graph community structure during augmentation, improving robustness and generalization.

Does Label Smoothing Help Deep Partial Label Learning?

https://openreview.net/forum?id=drjjxmi2Ha

Compressor summary: The paper proposes label smoothing to improve deep partial label learning classifiers and provides theoretical and empirical evidence for its effectiveness.

Offline Transition Modeling via Contrastive Energy Learning

https://openreview.net/forum?id=dqpg8jdA2w

Compressor summary: Energy-based transition models capture complex real-world transitions and improve offline reinforcement learning performance.

An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks

https://openreview.net/forum?id=dqdctVbSfs

Compressor summary: The paper develops a new analysis of neural TD learning algorithms that improves the sample complexity and achieves an $ ilde{\mathcal{O}}(\epsilon^{-1})$ error bound under Markovian sampling.

Exploration by Optimization with Hybrid Regularizers: Logarithmic Regret with Adversarial Robustness in Partial Monitoring

https://openreview.net/forum?id=dplgaRn4Ae

Compressor summary: The paper proposes a new exploration by optimization approach with a hybrid regularizer to improve regret bounds in online decision-making problems under limited feedback, achieving nearly optimal performance in both stochastic and adversarial environments.

ACPO: A Policy Optimization Algorithm for Average MDPs with Constraints

https://openreview.net/forum?id=dmfvHU1LNF

Compressor summary: The paper presents a new policy optimization algorithm (ACPO) for average-CMDPs with theoretical guarantees and experimental results showing its effectiveness in various challenging environments.

Causality Based Front-door Defense Against Backdoor Attack on Language Models

https://openreview.net/forum?id=dmHHVcHFdM

Compressor summary: Key points: - The paper proposes FABE, a new framework based on causal inference to protect language models from backdoor attacks. - FABE creates a 'front door' that maps out the actual causal relationships and filters out spurious associations. - FABE achieves state-of-the-art results in defending against various attack methods. Summary: FABE is a new framework that uses causal inference to defend language models from backdoor attacks by creating a 'front door' that separates legitimate and spurious associations, improving the defense effect significantly.

Balanced Resonate-and-Fire Neurons

https://openreview.net/forum?id=dkdilv4XD4

Compressor summary: The balanced resonate-and-fire neuron (BRF) is an improved spiking neural network model that achieves higher performance, lower spike count, fewer parameters, faster convergence, and better stability than previous models.

Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning

https://openreview.net/forum?id=disVlUOH4b

Compressor summary: HOP is a novel multi-agent algorithm that efficiently adapts to co-players in mixed-motive environments by hierarchically modeling opponents' goals and using Monte Carlo Tree Search for planning.

ELTA: An Enhancer against Long-Tail for Aesthetics-oriented Models

https://openreview.net/forum?id=dhrNfAJAH6

Compressor summary: ELTA is a technique that enhances aesthetic image assessment by improving minority feature representation, aligning features and labels, and refining output distribution, especially for long-tailed datasets.

LQER: Low-Rank Quantization Error Reconstruction for LLMs

https://openreview.net/forum?id=dh8k41g775

Compressor summary: LQER is a method to reduce quantization errors in large language models, enabling near-lossless compression and improved downstream task performance with less hardware resources.

Consistent Long-Term Forecasting of Ergodic Dynamical Systems

https://openreview.net/forum?id=dfR6FU53qk

Compressor summary: The text describes a method to improve long-term forecasting of dynamical systems using techniques from operator theory and statistics, with uniform error bounds on infinite time horizons.

Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixtures

https://openreview.net/forum?id=ddjRdm3wUW

Compressor summary: The paper analyzes implicit neural networks' spectral behavior using random matrix theory and proposes a method to design shallow explicit networks that match their kernel matrices.

On Which Nodes Does GCN Fail? Enhancing GCN From the Node Perspective

https://openreview.net/forum?id=dcwUGaK9sQ

Compressor summary: The paper proposes DaGCN, a framework that improves Graph Convolutional Networks by handling nodes that do not fit the label smoothness assumption and are not well-represented by existing GCNs.

Equivariant Graph Neural Operator for Modeling 3D Dynamics

https://openreview.net/forum?id=dccRCYmL5x

Compressor summary: Equivariant Graph Neural Operator (EGNO) is a novel method that learns 3D dynamics as trajectories using equivariant temporal convolutions, outperforming existing methods in multiple domains.

MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark

https://openreview.net/forum?id=dbFEFHAD79

Compressor summary: The paper introduces a benchmark called MLLM-as-a-Judge to evaluate multimodal large language models, finding they perform well in pair comparisons but struggle with scoring evaluation and batch ranking tasks due to various biases and inconsistencies.

Reparameterized Importance Sampling for Robust Variational Bayesian Neural Networks

https://openreview.net/forum?id=da7MMwICjC

Compressor summary: RIS is a novel sampling method that reduces variance and improves convergence, performance, and uncertainty estimation in BNNs.

Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation

https://openreview.net/forum?id=dZsEOFUDew

Compressor summary: The text proposes a perspective to understand how pre-training language models with next-token prediction enables them to reason by aggregating indirect paths seen during pre-training on knowledge and reasoning graphs.

On The Statistical Complexity of Offline Decision-Making

https://openreview.net/forum?id=dYDPcx78tm

Compressor summary: The paper analyzes how well offline data can be used for online decisions, finding near-optimal performance bounds based on function approximation and a new policy concept that covers all existing data-related notions.

Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning

https://openreview.net/forum?id=dWxb80a0TW

Compressor summary: The paper proposes a universal geometric graph representation for 3D molecular complexes and a Generalist Equivariant Transformer (GET) model to capture interactions between various molecule types using one model that preserves fine-grained information.

Denoising Autoregressive Representation Learning

https://openreview.net/forum?id=dW29JZj0G5

Compressor summary: The paper proposes DARL, a simple decoder-only Transformer that learns strong visual representations for image generation by using tailored noise schedules and longer training in larger models.

Stop Regressing: Training Value Functions via Classification for Scalable Deep RL

https://openreview.net/forum?id=dVpFKfqF3R

Compressor summary: The paper proposes using categorical cross-entropy for training value functions in deep reinforcement learning, which improves performance and scalability across various domains.

NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models

https://openreview.net/forum?id=dVhrnjZJad

Compressor summary: The paper proposes factorized diffusion models for text-to-speech generation that disentangle speech into different attributes and generate them individually, leading to improved speech quality and naturalness.

A Unified View of FANOVA: A Comprehensive Bayesian Framework for Component Selection and Estimation

https://openreview.net/forum?id=dV9QGostQk

Compressor summary: The paper introduces a flexible and scalable Bayesian framework for FANOVA models that can handle different sparsity levels and unify various methods, while enabling uncertainty quantification and novel model developments.

Contrasting Multiple Representations with the Multi-Marginal Matching Gap

https://openreview.net/forum?id=dV9B9qFeGi

Compressor summary: The paper proposes a new loss function, M3G, that uses multi-marginal optimal transport theory to learn representations from multiple views, showing improved performance in self-supervised and multimodal tasks.

Human vs. Generative AI in Content Creation Competition: Symbiosis or Conflict?

https://openreview.net/forum?id=dT6ZbSxh33

Compressor summary: The text discusses how generative AI impacts content creation, introduces a competition model to study the balance between humans and AI, and suggests a stable equilibrium is possible.

Adaptive Observation Cost Control for Variational Quantum Eigensolvers

https://openreview.net/forum?id=dSrdnhLS2h

Compressor summary: The paper proposes a method called SubsCoRe that uses Gaussian process surrogate to control the cost and accuracy of SMO in VQE by adjusting the required accuracy and number of measurement shots.

Solving Poisson Equations using Neural Walk-on-Spheres

https://openreview.net/forum?id=dQveBV9lZl

Compressor summary: The paper introduces Neural Walk-on-Spheres, a new neural network method for solving high-dimensional Poisson equations efficiently, with better accuracy, speed, and reduced memory usage compared to existing methods.

Scalable Wasserstein Gradient Flow for Generative Modeling through Unbalanced Optimal Transport

https://openreview.net/forum?id=dMhF96PfQi

Compressor summary: Semi-dual JKO is a scalable Wasserstein gradient flow model with reduced training complexity that achieves competitive results in image generation.

Compositional Text-to-Image Generation with Dense Blob Representations

https://openreview.net/forum?id=dMOhgHNYAf

Compressor summary: BlobGEN is a text-to-image model that uses dense blob representations to capture fine-grained scene details, enabling better controllability and compositionality with large language models.

Position: A Safe Harbor for AI Evaluation and Red Teaming

https://openreview.net/forum?id=dLojMSgSFW

Compressor summary: Independent evaluation of generative AI systems is crucial for safety, but current terms of service and research access programs discourage it; developers should provide legal and technical safe harbor for public interest research without fear of account suspension or legal reprisal.

In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering

https://openreview.net/forum?id=dJTChKgv3a

Compressor summary: The proposed in-context vectors (ICV) approach improves the efficiency and effectiveness of in-context learning for large language models, enabling them to follow demonstration examples better and handle diverse tasks more flexibly.

Deep Functional Factor Models: Forecasting High-Dimensional Functional Time Series via Bayesian Nonparametric Factorization

https://openreview.net/forum?id=dHXKCyaIkp

Compressor summary: The Deep Functional Factor Model (DF2M) is a Bayesian nonparametric model that combines the Indian Buffet Process, multi-task Gaussian Processes, and a deep kernel function to analyze high-dimensional functional time series, offering explainability and better predictive accuracy than conventional deep learning models.

Sign is Not a Remedy: Multiset-to-Multiset Message Passing for Learning on Heterophilic Graphs

https://openreview.net/forum?id=dGDFZM018a

Compressor summary: Key points: - Graph Neural Networks (GNNs) are powerful for homophilic graphs but not for heterophilic graphs with dissimilar node features - Signed Message Passing (SMP) is a widely used method to handle heterophilic graphs but has limitations - The paper proposes Multiset to Multiset GNN (M2M-GNN), a novel message-passing function that overcomes the limitations of SMP and performs better Summary: The paper introduces M2M-GNN, a new method for graph neural networks that handles heterophilic graphs with different node features better than existing methods like SMP.

Self-Supervised Interpretable End-to-End Learning via Latent Functional Modularity

https://openreview.net/forum?id=dFEeI51O5j

Compressor summary: MoNet is a modular network that learns task-specific decision-making processes without supervision, enabling effective and interpretable visual navigation in indoor environments.

A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity

https://openreview.net/forum?id=dBqHGZPGZI

Compressor summary: The study examines how direct preference optimization (DPO) reduces toxicity in pre-trained language models and reveals that it bypasses rather than removes capabilities learned from pre-training.

Position: The Causal Revolution Needs Scientific Pragmatism

https://openreview.net/forum?id=dBMLtuKH01

Compressor summary: The authors propose scientific pragmatism, a balanced approach between scientific perfectionism and system-centric biases, to advance causal models and methods in knowledge generation and application.

Fully-Dynamic Approximate Decision Trees With Worst-Case Update Time Guarantees

https://openreview.net/forum?id=d5tJWH5yCi

Compressor summary: Key points: - The paper proposes a dynamic algorithm for maintaining decision trees under adversarial updates. - The algorithm guarantees good tree quality and fast update time. - The algorithm works for various metrics and types of decision trees, including boosted ones. Summary: The paper presents a fast and high-quality dynamic algorithm for updating different types of decision trees under data changes, with provable worst-case bounds.

KernelSHAP-IQ: Weighted Least Square Optimization for Shapley Interactions

https://openreview.net/forum?id=d5jXW2H4gg

Compressor summary: The paper proposes an approach to interpret higher-order interactions in complex ML models using the Shapley Interaction Index and shows its effectiveness with KernelSHAP-IQ.

InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks

https://openreview.net/forum?id=d5LURMSfTx

Compressor summary: InfiAgent-DABench is a benchmark to evaluate LLM-based agents on data analysis tasks using a format-prompting technique and an agent framework.

From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning

https://openreview.net/forum?id=d2vONO90Rw

Compressor summary: The authors propose a new method called supervised pinpoint tuning (SPT) that selectively fine-tunes specific modules in large language models to reduce sycophancy without compromising their general abilities.

GRATH: Gradual Self-Truthifying for Large Language Models

https://openreview.net/forum?id=d2f2sCXQuI

Compressor summary: The paper proposes GRATH, a method to improve the truthfulness of large language models using out-of-domain question prompts and direct preference optimization.

Multiplicative Weights Update, Area Convexity and Random Coordinate Descent for Densest Subgraph Problems

https://openreview.net/forum?id=d2E2i5rJ4x

Compressor summary: The paper presents new algorithms for finding and decomposing dense subgraphs in graphs, improving on previous work in terms of iteration complexity, convergence rate, and practicality.

Neural Jump-Diffusion Temporal Point Processes

https://openreview.net/forum?id=d1P6GtRzuV

Compressor summary: The paper proposes a new type of temporal point process model, called Neural Jump-Diffusion Temporal Point Process (NJDTPP), that uses neural networks to parameterize its intensity dynamics and achieves better performance than existing models.

Collapse-Aware Triplet Decoupling for Adversarially Robust Image Retrieval

https://openreview.net/forum?id=cy3JBZKCw1

Compressor summary: The paper introduces a method called CA-TRIDE to improve image retrieval robustness against adversarial examples by addressing the limitations of existing deep metric learning approaches, and shows its effectiveness on three datasets.

Weakly-Supervised Residual Evidential Learning for Multi-Instance Uncertainty Estimation

https://openreview.net/forum?id=cxiqxDnrCx

Compressor summary: This paper introduces a new method, MIREL, for uncertainty estimation in weakly-supervised learning scenarios with multiple instances and weak annotations.

Mean Estimation in the Add-Remove Model of Differential Privacy

https://openreview.net/forum?id=cwIhvoTzuK

Compressor summary: The paper proposes an optimal algorithm for one-dimensional mean estimation under the add-remove model of differential privacy, showing that it performs similarly to the swap model and improves upon existing methods.

Dynamic Spectral Clustering with Provable Approximation Guarantee

https://openreview.net/forum?id=coP4kPdhKr

Compressor summary: The paper proposes a fast dynamic spectral clustering algorithm for evolving graphs that can approximate cluster structures well under certain conditions.

Hierarchical Integral Probability Metrics: A distance on random probability measures with low sample complexity

https://openreview.net/forum?id=cmy38XZlJu

Compressor summary: The paper introduces a new distance for comparing probabilities that is faster, easier to estimate, and has more properties than the existing Wasserstein distance.

Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimization

https://openreview.net/forum?id=cmD5E6ami4

Compressor summary: Symmetric replay training (SRT) improves sample efficiency in deep reinforcement learning for combinatorial optimization by using high-reward samples to explore under-explored symmetric regions without additional online interactions.

BOtied: Multi-objective Bayesian optimization with tied multivariate ranks

https://openreview.net/forum?id=cj5HbaX14p

Compressor summary: The paper proposes a new acquisition function called BOtied, which uses the CDF indicator to efficiently optimize multiple competing objectives using copulas.

Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes

https://openreview.net/forum?id=cit0hg4sEz

Compressor summary: FedKSeed is a method for federated full-parameter tuning of large language models that reduces communication cost by using zeroth-order optimization, random seeds, and probability-differentiated seed sampling, improving performance over existing methods.

Taylor Videos for Action Recognition

https://openreview.net/forum?id=chhIZGqlUG

Compressor summary: The authors propose a new video format called Taylor video that extracts dominant motions from videos using Taylor expansion and show its effectiveness for action recognition with different architectures and modalities.

Beyond Individual Input for Deep Anomaly Detection on Tabular Data

https://openreview.net/forum?id=chDpBp2P6b

Compressor summary: The paper proposes a new anomaly detection method using Non-Parametric Transformers to capture feature and sample dependencies and achieves state-of-the-art performance on 31 benchmark datasets.

Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurts

https://openreview.net/forum?id=ccSSKTz9LX

Compressor summary: This paper studies how fine-tuning affects long-tail learning tasks and proposes LIFT, a lightweight fine-tuning algorithm that reduces training time, parameters, and improves performance.

Trained Random Forests Completely Reveal your Dataset

https://openreview.net/forum?id=cc72Vnfvoc

Compressor summary: The authors present an optimization-based method to reconstruct datasets used for training random forests, using common libraries and showing that most methods are vulnerable to this attack.

Understanding the Impact of Introducing Constraints at Inference Time on Generalization Error

https://openreview.net/forum?id=cbZTnjqIib

Compressor summary: This paper examines how constraining outputs of an ML model at inference time affects its generalization error and suggests choosing a proper loss function for this approach.

Graph Generation with Diffusion Mixture

https://openreview.net/forum?id=cZTFxktg23

Compressor summary: The paper proposes a generative framework that models the topology of graphs by explicitly learning the final graph structures of the diffusion process, improving graph and molecule generation tasks.

Continuous Treatment Effects with Surrogate Outcomes

https://openreview.net/forum?id=cZNuYKtoOZ

Compressor summary: The paper proposes a method to estimate continuous treatment effects using surrogate variables and labeled/unlabeled data, improving accuracy and addressing selection bias.

The Max-Min Formulation of Multi-Objective Reinforcement Learning: From Theory to a Model-Free Algorithm

https://openreview.net/forum?id=cY9g0bwiZx

Compressor summary: The paper presents a new approach for multi-objective reinforcement learning that ensures fairness among multiple goals and improves performance over existing methods.

Variational Learning is Effective for Large Deep Networks

https://openreview.net/forum?id=cXBv07GKvk

Compressor summary: The paper argues that a new optimizer called IVON performs well for training large neural networks and has advantages over Adam in terms of predictive uncertainty, finetuning, and sensitivity to data.

Just Cluster It: An Approach for Exploration in High-Dimensions using Clustering and Pre-Trained Representations

https://openreview.net/forum?id=cXBPPfNUZJ

Compressor summary: The paper proposes a density estimation-based exploration method for 3-D environments using clustering on random or pre-trained representations, showing its effectiveness and integration of pre-trained biases in exploration.

FESSNC: Fast Exponentially Stable and Safe Neural Controller

https://openreview.net/forum?id=cVp8blEw2i

Compressor summary: The FESSNC is a fast neural controller for nonlinear systems that ensures stability and safety using heuristic learning, projection operators, and Hutchinson's trace estimator.

A sampling theory perspective on activations for implicit neural representations

https://openreview.net/forum?id=cVkqItmYLQ

Compressor summary: This paper explores implicit neural representations using sampling theory, finding that $\mathrm{sinc}$ activations are optimal for shallow encodings and connecting them to dynamical systems.

SIN: Selective and Interpretable Normalization for Long-Term Time Series Forecasting

https://openreview.net/forum?id=cUMOVfOIve

Compressor summary: The paper introduces SIN, a selective and interpretable method for normalizing time series data to improve forecasting accuracy.

Forget Sharpness: Perturbed Forgetting of Model Biases Within SAM Dynamics

https://openreview.net/forum?id=cU20finY8V

Compressor summary: Sharpness-aware minimization (SAM) improves generalization by discarding undesirable model biases through perturbed forgetting, which is related to information bottleneck principle and outperforms standard SAM on various benchmarks.

Understanding Finetuning for Factual Knowledge Extraction

https://openreview.net/forum?id=cPsn9AcOYh

Compressor summary: The impact of QA fine-tuning data on factuality depends on the familiarity of facts, with lesser-known facts reducing factuality and better-known ones maintaining or improving it.

Accelerating Heterogeneous Federated Learning with Closed-form Classifiers

https://openreview.net/forum?id=cMige5MK1N

Compressor summary: Fed3R is a method for federated learning that works well with non-identical data distributions, is efficient, and can be fine-tuned with other algorithms.

Discovering Mixtures of Structural Causal Models from Time Series Data

https://openreview.net/forum?id=cHJAUdam3i

Compressor summary: The paper proposes MCD, a variational inference framework to discover causal models from time series data with different underlying causal structures, and shows its superior performance on synthetic and real datasets.

Towards Causal Foundation Model: on Duality between Optimal Balancing and Attention

https://openreview.net/forum?id=cFDaYtZR4u

Compressor summary: CInA is a new self-supervised method that uses multiple unlabeled datasets for causal learning and enables zero-shot causal inference on unseen tasks with high accuracy, potentially paving the way for causal foundation models.

Position: Rethinking Post-Hoc Search-Based Neural Approaches for Solving Large-Scale Traveling Salesman Problems

https://openreview.net/forum?id=cEJ9jNJuJP

Compressor summary: The paper questions the effectiveness of using machine learning to generate heatmaps for guiding Monte Carlo tree search in solving large-scale traveling salesman problems, and suggests future research directions.

AST-T5: Structure-Aware Pretraining for Code Generation and Understanding

https://openreview.net/forum?id=cBWVJh5Fvf

Compressor summary: AST-T5 is a novel pretraining method that uses Abstract Syntax Trees to improve code generation, transpilation, and understanding tasks, outperforming similar-sized language models in various scenarios.

Transforming and Combining Rewards for Aligning Large Language Models

https://openreview.net/forum?id=cAWbm9KRZO

Compressor summary: The LSC-transformation method improves language model alignment by emphasizing poorly-performing outputs, preventing underfitting and reward hacking, and allowing principled aggregation of multiple rewards.

Bayesian Adaptation of Network Depth and Width for Continual Learning

https://openreview.net/forum?id=c9HddKGiYk

Compressor summary: The paper proposes a new Bayesian method to adjust network depth and width in dynamic architecture-based continual learning, achieving better or similar results than existing methods and working well for unsupervised learning too.

Scalable Pre-training of Large Autoregressive Image Models

https://openreview.net/forum?id=c92KDfEZTg

Compressor summary: The paper presents AIM, a set of vision models inspired by Large Language Models that scale well with data and model size, and can be pre-trained on a 7 billion parameter model on 2 billion images for high ImageNet-1k performance.

A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models

https://openreview.net/forum?id=c8qWiNiqRY

Compressor summary: DiPmark is a watermarking technique that preserves the original content's distribution while being accessible and robust to token changes.

Hybrid Reinforcement Learning from Offline Observation Alone

https://openreview.net/forum?id=c6rVlTKpb5

Compressor summary: The paper explores the challenges and solutions for hybrid reinforcement learning with observation-only offline data, and proposes an algorithm that performs well even without a reset model of the environment.

Position: Insights from Survey Methodology can Improve Training Data

https://openreview.net/forum?id=c3ls5AVOw7

Compressor summary: The text discusses the importance of high-quality data for AI/ML models, the challenges of collecting such data, and how survey methodology can help improve data quality and reduce biases.

Deep Neural Room Acoustics Primitive

https://openreview.net/forum?id=c2CKmP9l5X

Compressor summary: The paper proposes DeepNeRAP, a method to learn a continuous neural field that encodes sound propagation dynamics in 3D spaces and infers room impulse response without direct ground truth access.

Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-constraint

https://openreview.net/forum?id=c1AKcA6ry1

Compressor summary: The paper explores the theory and algorithms of aligning generative models with RLHF, proposes new methods that outperform existing ones, and shows their effectiveness on a large language model.

A3S: A General Active Clustering Method with Pairwise Constraints

https://openreview.net/forum?id=c18noxRh3X

Compressor summary: A3S is a novel framework that improves active clustering performance by adjusting initial cluster results using Normalized mutual information gain and reducing human queries.

A Language Model’s Guide Through Latent Space

https://openreview.net/forum?id=c0LoolDFw4

Compressor summary: The paper explores how to control language models with different concepts beyond truthfulness and evaluates their effectiveness using a new metric and extensive experiments.

What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement

https://openreview.net/forum?id=bzNwexOPWm

Compressor summary: Key points: - Language models make errors and suffer from catastrophic forgetting when updated with corrected instances - The goal is to predict which upstream examples will be forgotten for better replay control and interpretability - A partially interpretable model based on logit scores performs well on BART but not on T5 - A black-box classifier based on inner products of representations outperforms the interpretable model - Replaying forecasted forgotten examples reduces forgetting and shows practical utility Summary: The paper proposes models to predict which upstream examples will be forgotten by language models after update, using either interpretable or black-box methods, and shows that replaying these examples improves performance and reduces forgetting.

Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling

https://openreview.net/forum?id=byxXa99PtF

Compressor summary: Input clarification ensembling is a framework for decomposing the uncertainty of large language models into aleatoric (data) and epistemic (model) components, improving reliability and interpretability.

Autoformalizing Euclidean Geometry

https://openreview.net/forum?id=bylZbZOsGA

Compressor summary: The paper presents a neuro-symbolic approach to automatically translate informal Euclidean geometry proofs into formal theorems using theorem provers, large language models, and semantic evaluation.

Plug-and-Play image restoration with Stochastic deNOising REgularization

https://openreview.net/forum?id=byAXJTk0LH

Compressor summary: SNORE is a new PnP algorithm that applies the denoiser only on images with appropriate noise levels, improving image restoration results.

An Unsupervised Approach for Periodic Source Detection in Time Series

https://openreview.net/forum?id=bwZlD7mYoa

Compressor summary: The paper proposes a new method to detect periodic patterns in noisy time series data without labels or augmentations and shows significant improvements over existing methods.

Optimal Ridge Regularization for Out-of-Distribution Prediction

https://openreview.net/forum?id=bvPYroQgc3

Compressor summary: The study examines how optimal ridge regularization and risk behave in out-of-distribution prediction scenarios, showing that negative regularization can be optimal and the tuned risk depends on data aspect ratio.

Feature Distribution on Graph Topology Mediates the Effect of Graph Convolution: Homophily Perspective

https://openreview.net/forum?id=buW1Bi6XFw

Compressor summary: Randomly shuffling feature vectors among nodes of the same class improves graph neural network performance by reducing the dependence between graph topology and features.

Precise Accuracy / Robustness Tradeoffs in Regression: Case of General Norms

https://openreview.net/forum?id=btYeH65fI3

Compressor summary: The paper studies how adversarial attacks affect linear regression models and finds the best balance between robustness and accuracy for different scenarios.

Challenges and Considerations in the Evaluation of Bayesian Causal Discovery

https://openreview.net/forum?id=bqgtkBDkNs

Compressor summary: The paper discusses challenges in evaluating Bayesian Causal Discovery methods due to uncertainty in inferred causal graphs and proposes factors to consider for better assessment.

SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models

https://openreview.net/forum?id=bq1JEgioLr

Compressor summary: SciBench is a benchmark suite for testing large language models on complex scientific problems across math, chemistry, and physics domains, revealing their current limitations and areas for improvement.

Batch and match: black-box variational inference with a score-based divergence

https://openreview.net/forum?id=bplNmU2ROC

Compressor summary: BaM is a new black-box variational inference method that uses a score-based divergence and optimizes Gaussian variational families with full covariance matrices, which converges faster and better than ELBO-based methods.

Sequential Kernel Goodness-of-fit Testing

https://openreview.net/forum?id=bmeUeCUMHA

Compressor summary: A possible summary is: The authors propose a novel sequential goodness-of-fit testing method that adapts to data complexity and maintains statistical power by using a betting strategy.

Nonlinear Filtering with Brenier Optimal Transport Maps

https://openreview.net/forum?id=blzDxD6bKt

Compressor summary: The paper proposes a new nonlinear filtering method based on Brenier optimal transport that uses neural networks and stochastic optimization to handle degenerate likelihoods, high-dimensional states, and multi-modal distributions.

Learning Decision Trees and Forests with Algorithmic Recourse

https://openreview.net/forum?id=blGpu9aGs6

Compressor summary: Key points: - New algorithm (AR) for accurate tree-based models with recourse actions - Ensures existence of actions by constraining tree growth - Uses adversarial training and greedy algorithm - Applies to random forest and improves accuracy and efficiency Summary: The paper introduces AR, a new algorithm that learns accurate tree models with guaranteed recourse actions using adversarial training and a greedy approach. It works on random forests and outperforms baselines in accuracy and efficiency.

Fundamental Limits of Distributed Covariance Matrix Estimation Under Communication Constraints

https://openreview.net/forum?id=biE1uHyG0l

Compressor summary: This paper studies how two agents can help a central server estimate a high-dimensional covariance matrix by communicating limited information about disjoint samples.

Unbiased Multi-Label Learning from Crowdsourced Annotations

https://openreview.net/forum?id=bgP8Rxv2eB

Compressor summary: Key points: - CMLL problem: multiple true labels, unreliable labels from annotators - Existing methods: focus on inferring true labels, not predicting - New method: unbiased risk estimator based on transition matrices, decoupled autoencoder to exploit label correlations - Generalization error bound for convergence Summary: The paper proposes a new method for CMLL that uses an unbiased risk estimator and a decoupled autoencoder to handle unreliable labels and improve prediction.

Submodular framework for structured-sparse optimal transport

https://openreview.net/forum?id=bfQCO9Vqhk

Compressor summary: The paper proposes sparsity-constrained optimal transport methods for learning sparse transport plans with efficient algorithms and theory.

High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling

https://openreview.net/forum?id=beXQVQorse

Compressor summary: Teacher-Student Bayesian Optimization (TSBO) is a novel semi-supervised learning approach that uses unlabeled data, teacher and student models to minimize labeled data queries and improve sample-efficiency in global optimization tasks.

Riemannian coordinate descent algorithms on matrix manifolds

https://openreview.net/forum?id=bdKaQmrM81

Compressor summary: The paper presents coordinate descent algorithms for various matrix manifolds that update fewer variables per iteration than Riemannian optimization while maintaining feasibility and providing low cost and efficiency.

State-Constrained Zero-Sum Differential Games with One-Sided Information

https://openreview.net/forum?id=bcN7KSB2YS

Compressor summary: The paper studies games with state constraints and one-sided information, and provides theoretical results for computing behavioral strategies and belief manipulation in such games.

On the Recoverability of Causal Relations from Temporally Aggregated I.I.D. Data

https://openreview.net/forum?id=bZNH0SU37Y

Compressor summary: The effect of temporal aggregation on non-temporal causal discovery depends on the consistency criterion used, the degree of nonlinearity, and the presence of partial linearity or prior information.

Split-and-Denoise: Protect large language model inference with local differential privacy

https://openreview.net/forum?id=bZ4fzw1iz7

Compressor summary: SnD is a private inference framework that splits LLMs and adds noise to embeddings before transmitting them to servers, enhancing privacy while maintaining performance.

Timer: Generative Pre-trained Transformers Are Large Time Series Models

https://openreview.net/forum?id=bYRYb7DMNo

Compressor summary: The paper introduces the Time Series Transformer (Timer), a large language model pre-trained on a curated dataset of heterogeneous time series, which can perform various tasks such as forecasting, imputation, and anomaly detection.

Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews

https://openreview.net/forum?id=bX3J7ho18S

Compressor summary: The paper proposes a method to estimate LLM-generated text in scientific peer reviews and finds that 6.5%-16.9% of the review text could be modified by LLMs, with higher occurrence in low confidence reviews and near deadline submissions.

Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models

https://openreview.net/forum?id=bWZKvF0g7G

Compressor summary: The text discusses a new vision-language safe instruction-following dataset called VLGuard that helps improve the safety of large language models without compromising their helpfulness.

Position: TrustLLM: Trustworthiness in Large Language Models

https://openreview.net/forum?id=bWUU0LwwMp

Compressor summary: The paper introduces TrustLLM, a study on trustworthiness in large language models, evaluating 16 mainstream models across eight dimensions and finding that proprietary models generally outperform open-source ones, but some open-source models come close to them.

Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data

https://openreview.net/forum?id=bWNPx6t0sF

Compressor summary: The paper analyzes various methods for fine-tuning language models with preference data and finds that approaches using on-policy sampling and negative gradient outperform offline and maximum likelihood objectives, which are unified under mode-seeking objectives for categorical distributions.

Controlled Decoding from Language Models

https://openreview.net/forum?id=bVIcZb7Qa0

Compressor summary: The paper proposes controlled decoding (CD), a modular solver that uses a prefix scorer to learn a value function and control the generation of a frozen base model, achieving effective alignment of language models with multiple rewards.

Self-Infilling Code Generation

https://openreview.net/forum?id=bV9yT24t9B

Compressor summary: The paper introduces a new method for generating code called self-infilling, which can create context and content simultaneously, improving output quality and regularizing the generation process.

Differentiable Model Scaling using Differentiable Topk

https://openreview.net/forum?id=bULHOW1RXM

Compressor summary: DMS is a method that efficiently searches for optimal network width and depth, improving performance on various tasks such as image classification, object detection, and language modeling.

Graph-Triggered Rising Bandits

https://openreview.net/forum?id=bPsohGR6gD

Compressor summary: The paper introduces a new model for bandits where arms' rewards depend on each other through a graph, studies its optimal and suboptimal policies, and presents regret minimization algorithms with structured graphs and no-regret properties.

Modular Learning of Deep Causal Generative Models for High-dimensional Causal Inference

https://openreview.net/forum?id=bOhzU7NpTB

Compressor summary: Modular-DCM is an efficient algorithm that uses pre-trained models to sample from identifiable causal queries in high-dimensional data, outperforming baselines and handling latent confounders.

Robust Data-driven Prescriptiveness Optimization

https://openreview.net/forum?id=bNgAdyv7ZP

Compressor summary: The paper proposes a new measure called the coefficient of prescriptiveness to compare optimization techniques that use side information for better decisions and introduces a model with this measure, which is solved by a bisection algorithm.

Watermarks in the Sand: Impossibility of Strong Watermarking for Language Models

https://openreview.net/forum?id=bM2s12t4hR

Compressor summary: This paper proves that strong watermarking schemes for generative models are impossible under realistic assumptions and shows an efficient attack on three existing schemes using a "quality" and "perturbation" oracle.

Genie: Generative Interactive Environments

https://openreview.net/forum?id=bJbSbJskOS

Compressor summary: Genie is a large-scale generative model that can create diverse virtual environments from text or images and can be controlled by user actions without any supervision.

Policy Evaluation for Variance in Average Reward Reinforcement Learning

https://openreview.net/forum?id=bID9PiBFpT

Compressor summary: The paper proposes a new algorithm for reinforcement learning that considers risk as measured by asymptotic variance and shows it converges in finite time.

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization

https://openreview.net/forum?id=bBzlapzeR1

Compressor summary: The paper examines how importance re-weighting improves kernel ridge regression in high dimensions with covariate shifts by analyzing the bias-variance trade-off and providing asymptotic expansions of kernels under covariate shift.

Learning Solution-Aware Transformers for Efficiently Solving Quadratic Assignment Problem

https://openreview.net/forum?id=bBkQ51PmjC

Compressor summary: The paper proposes a novel learning-based approach for solving Quadratic Assignment Problems (QAPs) using a Solution AWare Transformer (SAWT) architecture that encodes facility and location nodes separately, enabling scalability to larger problem sizes.

Analyzing $D^\alpha$ seeding for $k$-means

https://openreview.net/forum?id=b9uHveqszc

Compressor summary: The paper analyzes how the $D^\alpha$ seeding algorithm performs better than standard $k$-means for any $\alpha>2$, and gives theoretical and empirical evidence for this improvement.

Fine-grained Classes and How to Find Them

https://openreview.net/forum?id=b9VfvegTEO

Compressor summary: FALCON is an unsupervised method that uses coarse labels to discover and relate fine-grained classes in image and single-cell classification tasks, improving performance significantly.

Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generators

https://openreview.net/forum?id=b89JtZj9gm

Compressor summary: Modern Text-to-Image generators like Stable Diffusion can improve the robustness and generalization of neural image classifiers by simulating interventions over environmental factors in the training data.

Graph As Point Set

https://openreview.net/forum?id=b6yHkQpSwZ

Compressor summary: The paper proposes a novel way to convert graphs into sets and use set encoders like Transformers to learn from them, improving the expressivity and performance of Graph Neural Networks.

ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL

https://openreview.net/forum?id=b6rA0kAHT1

Compressor summary: Key points: - The paper proposes an algorithmic framework for multi-turn RL with LLMs that balances flexibility and efficiency. - The framework, ArCHer, combines a high-level value function learner and a low-level token-by-token policy learner. - ArCHer outperforms prior methods in sample efficiency and performance on multi-turn LLM tasks. Summary: The paper introduces ArCHer, an algorithmic framework for multi-turn RL with LLMs that uses a high-level value function and a low-level token-by-token policy to optimize long-term objectives efficiently and effectively.

Probabilistic Subgoal Representations for Hierarchical Reinforcement Learning

https://openreview.net/forum?id=b6AwZauZPV

Compressor summary: The paper proposes a probabilistic subgoal representation function for hierarchical reinforcement learning using Gaussian Processes, which improves performance in various tasks and environments with stochastic uncertainties and diverse rewards.

Conformal Prediction for Deep Classifier via Label Ranking

https://openreview.net/forum?id=b3pYoZfcoo

Compressor summary: SAPS is a novel algorithm for conformal prediction that reduces prediction set size by discarding probability values except for the maximum softmax probability, while preserving uncertainty information and improving conditional coverage rates.

IM-Unpack: Training and Inference with Arbitrarily Low Precision Integers

https://openreview.net/forum?id=b2D9PBNNQ2

Compressor summary: The paper proposes a simple algorithm to represent heavy matrix entries with low bit-width integers and achieve efficiency gains in GEMM operations for Transformer-based models.

Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics

https://openreview.net/forum?id=b1iurBHDck

Compressor summary: The authors propose a novel algorithm that uses a multimodal variational autoencoder to reconstruct nonlinear dynamical systems from various types of data, including symbolic data, in a generative framework.

Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective

https://openreview.net/forum?id=b1YQ5WKY3w

Compressor summary: The paper investigates whether in-context learning in large language models is equivalent to Bayesian inference using the martingale property, and finds evidence against this hypothesis.

ILILT: Implicit Learning of Inverse Lithography Technologies

https://openreview.net/forum?id=b0lxGL2n3d

Compressor summary: The paper presents a new machine learning framework that can generate high-quality masks for chip design without using inverse lithography solvers, achieving better performance than existing methods.

Category-Aware Active Domain Adaptation

https://openreview.net/forum?id=axwrD8F1yq

Compressor summary: The text proposes a novel method for active domain adaptation that focuses on improving individual categories without harming others by identifying the most important unlabeled data samples using influence function analysis.

Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text

https://openreview.net/forum?id=axl3FAkpik

Compressor summary: The Binoculars method accurately detects machine-generated text from various large language models using simple calculations with two pre-trained models, achieving state-of-the-art performance without any training data or model-specific modifications.

Language-guided Skill Learning with Temporal Variational Inference

https://openreview.net/forum?id=awo5H10K6v

Compressor summary: The algorithm uses LLMs to segment trajectories and then merges them using variational inference and an auxiliary objective to discover reusable skills for agents in different environments.

Towards Theoretical Understandings of Self-Consuming Generative Models

https://openreview.net/forum?id=aw6L8sB2Ts

Compressor summary: The paper studies how mixing real and synthetic data in generative models affects data distributions, derives bounds on the TV distance between them, and reveals a phase transition point where the distance declines.

Language Models Represent Beliefs of Self and Others

https://openreview.net/forum?id=asJTE8EBjg

Compressor summary: The study reveals how large language models represent and decode beliefs, showing their importance for social reasoning in various tasks.

Position: Opportunities Exist for Machine Learning in Magnetic Fusion Energy

https://openreview.net/forum?id=arwP5FA2dO

Compressor summary: Key research challenges in fusion energy that could benefit from Machine Learning applications are discussed, highlighting six areas where ML can contribute to advancing fusion as a carbon-free energy source.

DPN: Decoupling Partition and Navigation for Neural Solvers of Min-max Vehicle Routing Problems

https://openreview.net/forum?id=ar174skI9u

Compressor summary: The paper proposes a novel attention-based encoder for solving the min-max vehicle routing problem, which improves the efficiency and optimality of sequential planning by decoupling customer partition and navigation tasks and using an agent-permutation-symmetric loss function.

Privacy-Preserving Embedding via Look-up Table Evaluation with Fully Homomorphic Encryption

https://openreview.net/forum?id=apxON2uH4N

Compressor summary: The paper proposes an efficient algorithm for privacy-preserving word embedding using homomorphic encryption and coded inputs.

Toward Adaptive Reasoning in Large Language Models with Thought Rollback

https://openreview.net/forum?id=aoAPOOtN9E

Compressor summary: The paper introduces Thought Rollback, a new reasoning framework for LLMs that allows them to adaptively build thought structures and revise mistaken ones to solve problems better under hallucinations.

EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs

https://openreview.net/forum?id=anM1M5aoM8

Compressor summary: EvoluNet is a novel framework for non-IID transfer learning on dynamic graphs that leverages temporal encoding and domain unification modules to improve generalization performance.

Position: Why Tabular Foundation Models Should Be a Research Priority

https://openreview.net/forum?id=amRSBdZlw9

Compressor summary: The authors propose developing large tabular models (LTMs) that can contextualize multiple datasets, which could have significant impacts on various fields and tasks involving tabular data.

FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees

https://openreview.net/forum?id=akyElNlUVA

Compressor summary: FedLMT and pFedLMT are frameworks for federated learning that address resource heterogeneity among clients using pre-factorized low-rank models, improving model accuracy and reducing costs.

Multi-layer Rehearsal Feature Augmentation for Class-Incremental Learning

https://openreview.net/forum?id=aksdU1KOpT

Compressor summary: The paper proposes Multi-layer Rehearsal Feature Augmentation (MRFA) to improve generalization and reduce catastrophic forgetting in Class-Incremental Learning by optimizing the all-layer margin on rehearsal samples.

Exploiting Human-AI Dependence for Learning to Defer

https://openreview.net/forum?id=aiz79FxjaI

Compressor summary: The paper introduces dependent Bayes optimality and a deferral principle for learning to defer frameworks that exploit the dependence between models and experts, and propose a novel consistent surrogate loss based on these concepts.

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity

https://openreview.net/forum?id=ahEm3l2P6w

Compressor summary: The paper introduces OWL, a novel pruning method for large language models that uses non-uniform layerwise sparsity ratios based on outlier features to improve performance and reduce model size.

Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context

https://openreview.net/forum?id=ah1BlQcLv4

Compressor summary: Transformers can learn gradient descent and non-linear functions by implementing them in function space, enabling efficient learning for various non-linear tasks and activation choices.

Gaussian Processes on Cellular Complexes

https://openreview.net/forum?id=afnyJfQddk

Compressor summary: The paper introduces Gaussian processes on cellular complexes, a generalization of graphs that captures polyadic relations, and proposes two new kernels to model interactions between vertices, edges, and higher-order cells.

Accelerated Policy Gradient: On the Convergence Rates of the Nesterov Momentum for Reinforcement Learning

https://openreview.net/forum?id=aeXRBnLoPP

Compressor summary: The paper introduces Accelerated Policy Gradient (APG), a method to improve convergence rates in reinforcement learning by adapting Nesterov's accelerated gradient to policy optimization, and proves its theoretical convergence properties.

Why Do You Grok? A Theoretical Analysis on Grokking Modular Addition

https://openreview.net/forum?id=ad5I6No9G1

Compressor summary: The paper explains why models can improve on modular addition problem even after overfitting, and suggests that this phenomenon is due to a transition from kernel-like behavior to limiting behavior of gradient descent on deep networks.

Graph Geometry-Preserving Autoencoders

https://openreview.net/forum?id=acTLXagzqd

Compressor summary: This paper introduces a new autoencoder framework that preserves the data's geometric structure using a similarity graph and Riemannian geometry, leading to better latent representation learning.

Socialized Learning: Making Each Other Better Through Multi-Agent Collaboration

https://openreview.net/forum?id=aaeJpJw5Ur

Compressor summary: The text proposes Multi-Agent Socialized Collaboration (MASC), a method for achieving socialized learning in multi-agent systems, which prioritizes original expert classes' accuracy while acquiring new abilities.

Predicting Lagrangian Multipliers for Mixed Integer Linear Programs

https://openreview.net/forum?id=aZnZOqUOHq

Compressor summary: The paper proposes a deep learning method for finding Lagrangian Multipliers that improves bounds on Mixed Integer Linear Programs by using graph neural networks to encode and decode relaxed constraints.

Improving Open-Ended Text Generation via Adaptive Decoding

https://openreview.net/forum?id=aXD94eATtT

Compressor summary: Adaptive decoding is a mechanism that helps language models choose better candidates for tokens during generation by increasing confidence based on an entropy-based metric.

Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains

https://openreview.net/forum?id=aVqqoFAavs

Compressor summary: The paper proposes a method to extend neural operators for learning solutions of PDEs to non-equispaced point distributions by efficiently evaluating spectral transformations, achieving faster training and comparable or improved accuracy.

Self-attention Networks Localize When QK-eigenspectrum Concentrates

https://openreview.net/forum?id=aRZjRj41WQ

Compressor summary: The text discusses how self-attention mechanisms in machine learning can be improved by reducing localization and addressing rank and entropy collapses for better model performance.

UP2ME: Univariate Pre-training to Multivariate Fine-tuning as a General-purpose Framework for Multivariate Time Series Analysis

https://openreview.net/forum?id=aR3uxWlZhX

Compressor summary: The paper proposes UP2ME, a general-purpose framework for multivariate time series tasks that combines univariate pre-training and multivariate fine-tuning to achieve state-of-the-art performance in forecasting, imputation and anomaly detection.

SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization

https://openreview.net/forum?id=aQl4xiwVBc

Compressor summary: The paper proposes PRepBN and SLA modules to improve efficiency of transformers for computer vision and natural language tasks, achieving lower latency and similar performance compared to existing methods.

A Universal Transfer Theorem for Convex Optimization Algorithms Using Inexact First-order Oracles

https://openreview.net/forum?id=aPhwhueqjR

Compressor summary: The paper presents a method to adapt any convex optimization algorithm to work with inexact first-order information, without knowing the algorithm's details, and applies to various types of algorithms and nonconvex problems.

On the Embedding Collapse when Scaling up Recommendation Models

https://openreview.net/forum?id=aPVwOAr1aW

Compressor summary: The paper proposes a multi-embedding design to address embedding collapse, which restricts scalability in large recommendation models, and shows its effectiveness through experiments.

How Smooth Is Attention?

https://openreview.net/forum?id=aP0H8A1ywk

Compressor summary: The paper studies the Lipschitz properties of self-attention and masked self-attention in Transformers, showing how sequence length and layer normalization affect their bounds, and proposes a novel mean-field framework for masked self-attention.

Boosting Offline Optimizers with Surrogate Sensitivity

https://openreview.net/forum?id=aLSA3JH08h

Compressor summary: The text discusses improving offline optimization by developing a sensitivity measurement for surrogate models and using it to create a regularizer that enhances optimization performance.

Copula-Nested Spectral Kernel Network

https://openreview.net/forum?id=aK1FyEP2Sn

Compressor summary: CokeNet is a novel spectral density function for spectral kernel networks that enhances diversity and captures complex data dependencies, leading to better machine learning performance.

BayOTIDE: Bayesian Online Multivariate Time Series Imputation with Functional Decomposition

https://openreview.net/forum?id=aGBpiEcB8z

Compressor summary: BayOTIDE is an online imputation method that models multivariate time series as a combination of low-rank temporal factors with different patterns, using Gaussian Processes and state-space priors.

Learning to Explore for Stochastic Gradient MCMC

https://openreview.net/forum?id=aECamk9izk

Compressor summary: Key points: - BNNs with high-dimensional parameters have multi-modal posterior distributions - SGMCMC with cyclical learning rate scheduling can explore them but is computationally expensive - The paper proposes a meta-learning strategy to build SGMCMC that efficiently explores the posteriors - The algorithm transfers well to various tasks and improves sampling efficiency Summary: The paper introduces a meta-learning method for building SGMCMC that can efficiently explore multi-modal posterior distributions of BNNs, achieving better performance on image classification and other tasks.

Protein Conformation Generation via Force-Guided SE(3) Diffusion Models

https://openreview.net/forum?id=aC1LSa4nXs

Compressor summary: ConfDiff is a force-guided diffusion model for protein conformation generation that incorporates physical prior knowledge to produce diverse and accurate conformations, outperforming existing methods.

Simulation of Graph Algorithms with Looped Transformers

https://openreview.net/forum?id=aA2326y3hf

Compressor summary: This paper studies how looped transformer networks with extra attention heads can simulate various graph algorithms and proves their theoretical properties.

Rolling Diffusion Models

https://openreview.net/forum?id=a9bzTv9SzO

Compressor summary: Rolling Diffusion is a novel method for denoising temporal data that adapts to uncertainty by adding more noise to later frames, outperforming standard diffusion methods in complex scenarios like video prediction and fluid dynamics.

Critical windows: non-asymptotic theory for feature emergence in diffusion models

https://openreview.net/forum?id=a8ZpjLJuKk

Compressor summary: The authors develop theory to understand critical windows, narrow time intervals in image generation when specific features emerge, and use the framework to analyze and diagnose issues in diffusion models.

SurfPro: Functional Protein Design Based on Continuous Surface

https://openreview.net/forum?id=a8QpoEJCRI

Compressor summary: SurfPro is a method that generates functional proteins with desired surfaces and biochemical properties by encoding the geometric shape and biochemical features of a protein surface and decoding an amino acid sequence.

A New Computationally Efficient Algorithm to solve Feature Selection for Functional Data Classification in High-dimensional Spaces

https://openreview.net/forum?id=a7MW5kFFOf

Compressor summary: The paper presents a new method for selecting important features and classifying functional data in scenarios with categorical responses and multivariate longitudinal features, which is faster and more accurate than existing methods.

Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodings

https://openreview.net/forum?id=a6wCNfIj8E

Compressor summary: The paper proposes functional reward encoding (FRE), a method to pre-train a generalist agent from unlabeled data using a transformer-based variational auto-encoder, enabling it to solve new tasks in a zero-shot manner with minimal supervision.

Position: What makes an image realistic?

https://openreview.net/forum?id=a6366lEzbX

Compressor summary: The text discusses the difficulty of quantifying realism in generated data and proposes a novel concept called universal critic that could help solve this problem without adversarial training.

Reward Shaping for Reinforcement Learning with An Assistant Reward Agent

https://openreview.net/forum?id=a3XFF0PGLU

Compressor summary: The paper presents a dual-agent reward shaping method that improves reinforcement learning's sample efficiency and stability in sparse-reward tasks by generating auxiliary reward signals.

Efficient Pareto Manifold Learning with Low-Rank Structure

https://openreview.net/forum?id=a2uFstsHPb

Compressor summary: The authors propose a novel multi-task learning method that uses low-rank matrices and orthogonal regularization to efficiently learn the Pareto front and achieve better performance, especially on large datasets.

PAC-Bayesian Error Bound, via Rényi Divergence, for a Class of Linear Time-Invariant State-Space Models

https://openreview.net/forum?id=a1Olc2QhPv

Compressor summary: The paper develops a PAC-Bayesian error bound for linear time-invariant stochastic state-space models, which are used in control engineering and econometrics, and have applications in recurrent neural networks.

$\mathtt{VITS}$ : Variational Inference Thompson Sampling for contextual bandits

https://openreview.net/forum?id=a1GvTbadqA

Compressor summary: The paper introduces a new Thompson sampling variant called Varational Inference TS (VITS) that uses Gaussian Variational Inference for efficient posterior approximation and achieves sub-linear regret in linear contextual bandits.

Diffusion Models Encode the Intrinsic Dimension of Data Manifolds

https://openreview.net/forum?id=a0XiA6v256

Compressor summary: The authors show that diffusion models can estimate the intrinsic dimension of data manifolds by approximating their normal bundles using the score function, which points towards the manifold when noise is low.

Ditto: Quantization-aware Secure Inference of Transformers upon MPC

https://openreview.net/forum?id=ZzXNCQGzqT

Compressor summary: Ditto is a framework that enables efficient and secure quantization-aware inference for Transformers using multi-party computation techniques, reducing computation and communication overhead.

MC-GTA: Metric-Constrained Model-Based Clustering using Goodness-of-fit Tests with Autocorrelations

https://openreview.net/forum?id=ZzFTrzo0Cp

Compressor summary: MC-GTA is a novel clustering algorithm that uses feature similarity and metric autocorrelation to improve performance, stability, and computational efficiency for various data analysis tasks.

Estimating Canopy Height at Scale

https://openreview.net/forum?id=ZzCY0fRver

Compressor summary: The paper presents a framework that uses satellite data to estimate canopy heights globally with improved accuracy compared to existing methods, enabling better ecological analyses.

Federated Self-Explaining GNNs with Anti-shortcut Augmentations

https://openreview.net/forum?id=ZxDqSBgFSM

Compressor summary: The paper proposes a method called Federated Graph Rationalization (FedGR) that uses anti-shortcut augmentations to generate explanations for predictions made by graph neural networks in federated learning settings.

AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractors

https://openreview.net/forum?id=ZwrfsrCduj

Compressor summary: The paper proposes a method to distinguish between task-irrelevant visual distractors using an Implicit Action Generator (IAG) that learns the behavior of distractors and improves performance on various visual control tasks.

Position: AI-Powered Autonomous Weapons Risk Geopolitical Instability and Threaten AI Research

https://openreview.net/forum?id=ZwUThOE7Zc

Compressor summary: Machine learning in autonomous weapons systems increases the risk of low-intensity conflicts, arms race, and undermines geopolitical stability and AI research transparency.

Minimum-Norm Interpolation Under Covariate Shift

https://openreview.net/forum?id=Zw7TcnTmHj

Compressor summary: The paper investigates transfer learning in high-dimensional linear regression with benign overfitting, showing how overparameterized models behave under different types of covariate shifts.

Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features

https://openreview.net/forum?id=Zw52bJCZXc

Compressor summary: The paper introduces two new stochastic Frank-Wolfe methods for optimization problems with structured constraints that have better convergence guarantees and avoid issues of large batches or full gradients.

Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement

https://openreview.net/forum?id=ZvJ2lQQKjz

Compressor summary: MERL is a multimodal learning framework for ECGs that uses reports and text prompts to classify heart diseases without training data, outperforming eSSL methods.

The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright BreachesWithout Adjusting Finetuning Pipeline

https://openreview.net/forum?id=ZvFLbEPv6x

Compressor summary: The study proposes a stealthy backdoor attack, SilentBadDiffusion, that can induce copyright infringement in text-to-image diffusion models without controlling their training, and shows its effectiveness on various model architectures.

SeMOPO: Learning High-quality Model and Policy from Low-quality Offline Visual Datasets

https://openreview.net/forum?id=ZtOXZCTgBa

Compressor summary: SeMOPO is a new model-based offline RL approach that separates latent states into endogenous and exogenous parts, reducing bias in uncertainty estimation when dealing with complex distractors.

Minimally Modifying a Markov Game to Achieve Any Nash Equilibrium and Value

https://openreview.net/forum?id=ZtMqsSkIHX

Compressor summary: The text discusses modifying a game's rewards to achieve a specific policy goal, studying the conditions for success and proposing an algorithm to solve this problem efficiently.

GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformer

https://openreview.net/forum?id=Zsz9Pdfvtg

Compressor summary: Key points: - Cross-modal transformers integrate different modalities for vision tasks - GeminiFusion is a pixel-wise fusion approach that combines intra-modal and inter-modal attentions - GeminiFusion adapts to the complexity of the input and achieves superior performance on various multimodal tasks Summary: GeminiFusion is a novel cross-modal transformer that fuses different modalities using aligned representations and adaptive attentions, achieving high performance on image-to-image translation, 3D object detection, and semantic segmentation.

Smoothing Proximal Gradient Methods for Nonsmooth Sparsity Constrained Optimization: Optimality Conditions and Global Convergence

https://openreview.net/forum?id=Zs3qW8Njov

Compressor summary: Smoothing Proximal Gradient Methods (SPGM) are explored as solutions to nonsmooth sparsity constrained optimization problems, with two variants (SPGM-IHT and SPGM-BCD) showing improved performance over existing methods in theory and practice.

Bridging Environments and Language with Rendering Functions and Vision-Language Models

https://openreview.net/forum?id=ZrM67ZZ5vj

Compressor summary: The paper proposes a novel method for building language-conditioned agents using vision-language models by first finding an optimal environment configuration and then using a goal-conditioned policy to reach it, achieving better zero-shot generalization than multi-task RL baselines.

Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation

https://openreview.net/forum?id=Zos5wsaB5r

Compressor summary: The paper proposes a method for online Vision-and-Language Navigation that adapts model parameters using fast-slow tests and shows improved performance on four benchmarks.

Discounted Adaptive Online Learning: Towards Better Regularization

https://openreview.net/forum?id=ZoTIdyExx6

Compressor summary: The text studies an adaptive online optimization algorithm for online learning in changing environments and shows how it improves regularization and uncertainty prediction.

Probabilistic Constrained Reinforcement Learning with Formal Interpretability

https://openreview.net/forum?id=Zo9zXdVhW2

Compressor summary: AWaVO is a novel method for reinforcement learning that provides interpretability through convergence guarantee, training transparency, and intrinsic decision-interpretation, achieving good performance in simulation and real-world quadrotor tasks.

Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time

https://openreview.net/forum?id=Zn44XGFGam

Compressor summary: The paper analyzes the performance of two-layer ReLU networks with weight decay regularization and their convex relaxations, showing that random training data can lead to a small optimality gap and improved convergence rates for local gradient methods.

Optimal Acceleration for Minimax and Fixed-Point Problems is Not Unique

https://openreview.net/forum?id=ZeF75iQcAc

Compressor summary: The paper presents new acceleration mechanisms for minimax optimization and fixed-point problems with the same optimality as existing anchor-based methods, but different characteristics.

Generalization Bound and New Algorithm for Clean-Label Backdoor Attack

https://openreview.net/forum?id=ZdqiT0McON

Compressor summary: The paper derives generalization bounds for the clean-label backdoor attack scenario and proposes a new attack method using a combination of adversarial noise and indiscriminate poison.

Score-Based Causal Discovery of Latent Variable Causal Models

https://openreview.net/forum?id=ZdSe1qnuia

Compressor summary: The paper proposes new score-based methods for identifying causal structures involving latent variables, addressing challenges of existing constraint-based methods and providing identifiability guarantees.

OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization

https://openreview.net/forum?id=ZctlF8RlV4

Compressor summary: The paper introduces a novel optimization framework for one-shot structured pruning that improves efficiency and accuracy on vision and language models, including very large ones with tens of billions of parameters.

MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions

https://openreview.net/forum?id=Zc22RDtsvP

Compressor summary: The paper introduces MagicLens, a self-supervised image retrieval model that supports open-ended text instructions to find images with rich relations beyond visual similarity by using implicit relations from web pages.

Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-Mind

https://openreview.net/forum?id=ZZ7UKgK4c1

Compressor summary: The paper introduces a new dataset and prompting approach to study how AI can understand characters' mental states in movies, and shows that existing models perform worse than humans in this task.

CaPS: Collaborative and Private Synthetic Data Generation from Distributed Sources

https://openreview.net/forum?id=ZXsNkm3bxu

Compressor summary: The text proposes a framework for generating synthetic data from distributed sources using secure multi-party computation and differential privacy, allowing data sharing without entrusting raw data to a central entity.

Improving Sample Efficiency of Model-Free Algorithms for Zero-Sum Markov Games

https://openreview.net/forum?id=ZVmMV3AHjC

Compressor summary: The paper proposes a model-free algorithm for two-player Markov games that achieves optimal sample complexity, using variance reduction and reference value functions.

Safe Exploration in Dose Finding Clinical Trials with Heterogeneous Participants

https://openreview.net/forum?id=ZUXvpIrz5l

Compressor summary: SAFE-T is an adaptive dose-finding procedure that uses Bayesian optimization to learn non-parametric models for toxicity and efficacy, while satisfying safety constraints and improving utility for heterogeneous participants in early phase clinical trials.

MorphGrower: A Synchronized Layer-by-layer Growing Approach for Plausible Neuronal Morphology Generation

https://openreview.net/forum?id=ZTN866OsGx

Compressor summary: MorphGrower is a learning-based method that mimics neuron growth to generate realistic and valid neural morphologies, outperforming previous approaches.

Online Learning and Information Exponents: The Importance of Batch size & Time/Complexity Tradeoffs

https://openreview.net/forum?id=ZSQAf5YlvN

Compressor summary: The optimal batch size for training two-layer neural networks with SGD depends on the target's hardness and information exponent, and a new protocol called Correlation loss SGD can improve training speed beyond traditional batch sizes.

On Convergence of Incremental Gradient for Non-convex Smooth Functions

https://openreview.net/forum?id=ZRMQX6aTUS

Compressor summary: The paper studies SGD algorithms with arbitrary data ordering and improves convergence guarantees for incremental gradient and single shuffle SGD.

Cooperative Graph Neural Networks

https://openreview.net/forum?id=ZQcqXCuoxD

Compressor summary: The paper introduces a flexible and dynamic message-passing framework for graph neural networks, where nodes can choose to listen, broadcast, or isolate, enabling more effective exploration of the graph topology.

Sampling is as easy as keeping the consistency: convergence guarantee for Consistency Models

https://openreview.net/forum?id=ZPiEIhQpos

Compressor summary: The paper provides a convergence guarantee for Consistency Models, one-step generative models that can match Diffusion Models in sample quality.

AdsorbDiff: Adsorbate Placement via Conditional Denoising Diffusion

https://openreview.net/forum?id=ZMgpE58PMj

Compressor summary: The authors propose a new method to find the best position and orientation for adsorbates on a slab, which can accelerate the discovery of novel catalysts by using denoising diffusion, machine learning force field, and DFT.

The Fundamental Limits of Least-Privilege Learning

https://openreview.net/forum?id=ZGEICuuUJo

Compressor summary: The text formalizes the least-privilege principle for machine learning and proves its trade-off between utility and leakage, showing it is not possible to learn highly useful representations that prevent inference of unrelated attributes.

Getting the most out of your tokenizer for pre-training and domain adaptation

https://openreview.net/forum?id=ZFYBnLljtT

Compressor summary: This paper investigates how different tokenizer designs affect the performance of language models for code generation tasks, and provides recommendations for optimizing them.

A Unified Adaptive Testing System Enabled by Hierarchical Structure Search

https://openreview.net/forum?id=ZFRrOiZruJ

Compressor summary: The paper presents a new ATS framework that learns from data to create optimal personalized exams, reducing errors and questions while maintaining accuracy.

NExT-Chat: An LMM for Chat, Detection and Segmentation

https://openreview.net/forum?id=ZAW37OZ6ig

Compressor summary: The paper introduces the pix2emb method for object location modeling in LMMs, enabling them to handle various multimodal tasks using different location formats.

Identification and Estimation for Nonignorable Missing Data: A Data Fusion Approach

https://openreview.net/forum?id=Z7MzVDFWDV

Compressor summary: The paper proposes a method to identify and estimate a parameter in missing not at random data by fusing it with auxiliary data that is missing at random.

How Universal Polynomial Bases Enhance Spectral Graph Neural Networks: Heterophily, Over-smoothing, and Over-squashing

https://openreview.net/forum?id=Z2LH6Va7L2

Compressor summary: UniFilter is a novel GNN that adapts polynomial bases to the heterophily degree of a graph, improving convolution and propagation, and facilitating graph explanation.

Learning Reward for Robot Skills Using Large Language Models via Self-Alignment

https://openreview.net/forum?id=Z19JQ6WFtJ

Compressor summary: The paper proposes a method that uses Large Language Models (LLM) and self-alignment to learn reward functions more efficiently for robot skills without human guidance.

Towards Robust Model-Based Reinforcement Learning Against Adversarial Corruption

https://openreview.net/forum?id=Z0S6fUdW68

Compressor summary: The paper proposes two algorithms for adversarial corruption in model-based reinforcement learning, one for online and one for offline settings, and provides theoretical guarantees for their performance.

Neural Tangent Kernels for Axis-Aligned Tree Ensembles

https://openreview.net/forum?id=YxmcEfcgp3

Compressor summary: The paper studies how imposing the axis-aligned constraint on soft trees affects their learning behavior and explores various tree architectures using Neural Tangent Kernel analysis and Multiple Kernel Learning.

Transolver: A Fast Transformer Solver for PDEs on General Geometries

https://openreview.net/forum?id=Ywl6pODXjB

Compressor summary: Transolver uses Physics-Attention to learn intrinsic physical states from discretized geometries and improve solving partial differential equations.

Non-stationary Online Convex Optimization with Arbitrary Delays

https://openreview.net/forum?id=YvPNwLedpQ

Compressor summary: The paper proposes an algorithm for online convex optimization with arbitrary delays in non-stationary environments and shows its performance bounds and improvements over existing methods.

AND: Audio Network Dissection for Interpreting Deep Acoustic Models

https://openreview.net/forum?id=YvAyOYeGlo

Compressor summary: Possible summary: The paper introduces AND, a framework that explains acoustic neurons' behaviors and features using natural language summaries from large language models.

Model-Free Robust $\phi$-Divergence Reinforcement Learning Using Both Offline and Online Data

https://openreview.net/forum?id=Yug1IEkvcb

Compressor summary: The paper proposes two model-free algorithms for learning optimal robust policies in high-dimensional systems using $\phi$-divergences, one that uses only historical data and another that combines historical and online sampling.

CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modelling

https://openreview.net/forum?id=YuNFJSEkTi

Compressor summary: CasCast is a cascaded framework that uses a diffusion transformer to improve precipitation nowcasting and forecasting of extreme events.

Sample Average Approximation for Conditional Stochastic Optimization with Dependent Data

https://openreview.net/forum?id=YuGnRORkJm

Compressor summary: The paper proposes a Sample Average Approximation method for Conditional Stochastic Optimization with dependent data and provides theoretical guarantees of its consistency, sample complexity, and finite sample guarantee under mild conditions.

On the Generalization of Equivariant Graph Neural Networks

https://openreview.net/forum?id=Yqj3DzIC79

Compressor summary: The authors provide the first generalization bound for EGNNs, showing how spectral norms and layer weights affect their performance and propose a new regularizer based on these insights.

Principled Preferential Bayesian Optimization

https://openreview.net/forum?id=YqMOM5W9GF

Compressor summary: We propose an optimistic algorithm for preferential Bayesian optimization using preference feedback and a confidence set, achieving an information-theoretic bound on cumulative regret and guaranteed convergence rate, outperforming existing heuristics without guarantees.

Beyond the Norms: Detecting Prediction Errors in Regression Models

https://openreview.net/forum?id=YqIIhl2ToH

Compressor summary: The paper proposes a method to measure and quantify uncertainty in regression algorithms, helping detect unreliable behavior and improve error detection.

Measures of diversity and space-filling designs for categorical data

https://openreview.net/forum?id=YoUb2vW9WP

Compressor summary: The paper proposes new methods for selecting diverse subsets of categorical data using combinatorial optimization and submodular optimization techniques.

Unsupervised Evaluation of Code LLMs with Round-Trip Correctness

https://openreview.net/forum?id=YnFuUX08CE

Compressor summary: Round-trip correctness (RTC) is a new evaluation method for large language models that allows assessing their performance on various code-related tasks without manual curation, by checking if the model's prediction and synthesis match semantically.

Unveiling the Cycloid Trajectory of EM Iterations in Mixed Linear Regression

https://openreview.net/forum?id=Yn8xnK90mS

Compressor summary: The paper studies how fast the EM algorithm converges for two-component mixed linear regression, providing explicit expressions and characterizing the trajectory of iterations, leading to improved error bounds and convergence exponent estimates.

Enhancing Cross-Modal Fine-Tuning with Gradually Intermediate Modality Generation

https://openreview.net/forum?id=YlcSyCz21c

Compressor summary: The paper proposes a method called PaRe that uses a gating mechanism to transfer knowledge from large-scale pretrained models to various target modalities by creating intermediate data-rich modalities and improving cross-modal fine-tuning stability and performance.

Supervised Matrix Factorization: Local Landscape Analysis and Applications

https://openreview.net/forum?id=YlJy1FcM9E

Compressor summary: The paper analyzes the optimization landscape of supervised matrix factorization (SMF), derives applications, and proposes a block coordinate descent algorithm with convergence guarantees and a GPU-friendly neural implementation.

Stability and Multigroup Fairness in Ranking with Uncertain Predictions

https://openreview.net/forum?id=YiblhkVl2w

Compressor summary: The paper studies ranking functions that account for predictors' uncertainty and shows they can achieve both stability and fairness in classification tasks.

Improving Interpretation Faithfulness for Vision Transformers

https://openreview.net/forum?id=YdwwWRX20q

Compressor summary: FViTs are a rigorous approach to improve ViTs' explanations and robustness by applying Denoised Diffusion Smoothing (DDS) to their self-attention vectors and prediction distributions.

Unified Training of Universal Time Series Forecasting Transformers

https://openreview.net/forum?id=Yd8eHMY1wz

Compressor summary: The text proposes Moirai, a new time series Transformer model that addresses challenges in universal forecasting by using a pre-trained Large Time Series Model on diverse datasets and achieves competitive or superior performance.

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

https://openreview.net/forum?id=YbHCqn4qF4

Compressor summary: The paper proposes a new efficient generic vision backbone, Vim, using bidirectional Mamba blocks that improve performance and memory efficiency over existing Transformer-based models on various computer vision tasks.

Harmonizing Generalization and Personalization in Federated Prompt Learning

https://openreview.net/forum?id=YYwERRXsJW

Compressor summary: FedPGP is a method for federated prompt learning that balances generalization and personalization using CLIP guidance and low-rank adaptation.

To Cool or not to Cool? Temperature Network Meets Large Foundation Models via DRO

https://openreview.net/forum?id=YWuSLBkfOw

Compressor summary: The paper proposes a small network (TempNet) to predict personalized temperature for large foundation models, which enhances their performance and can be applied to new tasks.

OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models

https://openreview.net/forum?id=YT1dtdLvSN

Compressor summary: OptiMUS is an LLM-based agent that can formulate and solve linear programming problems from natural language descriptions, outperforming existing methods on easy and hard datasets.

RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback

https://openreview.net/forum?id=YSoMmNWZZx

Compressor summary: RL-VLM-F automatically generates reward functions for agents to learn new tasks using text and visual inputs, by querying vision language models for preference feedbacks.

Two-Stage Shadow Inclusion Estimation: An IV Approach for Causal Inference under Latent Confounding and Collider Bias

https://openreview.net/forum?id=YRWdiaupCr

Compressor summary: The paper proposes a novel Two-Stage Shadow Inclusion (2SSI) method that addresses latent confounding bias and collider bias in causal inference using the treatment residual as a shadow variable.

Conformal Prediction with Learned Features

https://openreview.net/forum?id=YPbcUBcTAk

Compressor summary: PLCP is a novel framework that learns uncertainty-guided features for improving conditional validity of prediction sets using calibration data, machine learning models, and alternating gradient descent, with theoretical and empirical results showing its superior performance.

Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation

https://openreview.net/forum?id=YNvGFaOG1p

Compressor summary: The paper presents tight convergence bounds for actor-critic and natural actor-critic algorithms in reinforcement learning with compatible function approximation, addressing challenges such as stochastic bias and non-ergodicity.

What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks

https://openreview.net/forum?id=YNbCbcGyXE

Compressor summary: The study investigates how varying depth of transformer architecture affects its performance in different sequence learning tasks, showing that at least two attention layers are needed for reasoning and generalization, while three may be required for contextual generalization.

Interpretability Illusions in the Generalization of Simplified Models

https://openreview.net/forum?id=YJWlUMW6YP

Compressor summary: The study warns that simplifying deep learning models may lead to inaccurate predictions when the models encounter out-of-distribution data, as they might overfit the training set.

Contextualized Policy Recovery: Modeling and Interpreting Medical Decisions with Adaptive Imitation Learning

https://openreview.net/forum?id=YEQM0asWCH

Compressor summary: CPR is a multi-task learning framework that enables interpretability and adaptability in complex decision processes by generating context-specific policies on-demand as observations change.

IBD-PSC: Input-level Backdoor Detection via Parameter-oriented Scaling Consistency

https://openreview.net/forum?id=YCzbfs2few

Compressor summary: IBD-PSC is a simple input-level backdoor detection method for deep neural networks that uses parameter-oriented scaling consistency to filter out malicious images and requires adaptive selection of batch normalization layers.

Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Features Model

https://openreview.net/forum?id=YBetKvUlF7

Compressor summary: The paper studies how deep neural networks behave when trained on imbalanced datasets, showing that their features collapse to a structure of orthogonal vectors with lengths depending on the number of training samples.

Subhomogeneous Deep Equilibrium Models

https://openreview.net/forum?id=YBXwr7wF7i

Compressor summary: The paper analyzes the existence and uniqueness of fixed points for implicit-depth neural networks using subhomogeneous operators and Perron-Frobenius theory, leading to a more flexible framework for well-defined networks.

On Gradient-like Explanation under a Black-box Setting: When Black-box Explanations Become as Good as White-box

https://openreview.net/forum?id=YB1O99gK7b

Compressor summary: GEEX is a new method for explaining deep learning models without accessing their internal workings, which produces gradient-like explanations using only query-level access and has proven theoretical and empirical advantages over existing black-box methods.

Random Latent Exploration for Deep Reinforcement Learning

https://openreview.net/forum?id=Y9qzwNlKVU

Compressor summary: RLE is a new exploration technique for deep RL that adds structured random rewards to original task rewards in random states to improve performance on Atari and IsaacGym benchmarks.

Emergence of In-Context Reinforcement Learning from Noise Distillation

https://openreview.net/forum?id=Y8KsHT1kTV

Compressor summary: AD$^\varepsilon$ is a new method for in-context Reinforcement Learning that uses noise injection to learn from unlabeled data and improve performance without requiring optimal policies.

GeoMFormer: A General Architecture for Geometric Molecular Representation Learning

https://openreview.net/forum?id=Y5Zi59N265

Compressor summary: The text introduces GeoMFormer, a novel Transformer-based molecular model that learns invariant and equivariant features for molecular systems using cross-attention modules.

Nash Learning from Human Feedback

https://openreview.net/forum?id=Y5AmNYiyCQ

Compressor summary: The study introduces NLHF, an alternative pipeline for fine-tuning LLMs using pairwise human feedback that aims to align them better with human preferences.

Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs

https://openreview.net/forum?id=Y50K6DSrWo

Compressor summary: DiverseNO uses multiple neural operator heads to improve uncertainty estimates and out-of-domain performance, while Operator-ProbConserv updates the model with well-calibrated UQ estimates.

Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback

https://openreview.net/forum?id=Y4wxCICbD0

Compressor summary: The paper proposes Linear Alignment, a new algorithm that aligns AI assistants with human preferences in one step, without needing data annotation or model training, improving their performance and efficiency.

CuTS: Customizable Tabular Synthetic Data Generation

https://openreview.net/forum?id=Y4VgJfbjfl

Compressor summary: CuTS is a novel framework for generating customizable synthetic tabular data that supports various requirements and outperforms existing approaches in several tasks.

Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC

https://openreview.net/forum?id=Y2wRKE0Qor

Compressor summary: SINGD is a memory-efficient and numerically robust second-order method for neural net training that performs well even in low precision, addressing limitations of KFAC and outperforming AdamW.

Towards a Self-contained Data-driven Global Weather Forecasting Framework

https://openreview.net/forum?id=Y2WorV5ag6

Compressor summary: The paper proposes a new data-driven weather forecasting framework that combines an AI model with a traditional data assimilation algorithm to generate accurate and efficient global weather forecasts.

Prediction Accuracy of Learning in Games : Follow-the-Regularized-Leader meets Heisenberg

https://openreview.net/forum?id=Y0sH9HGMwq

Compressor summary: The paper studies how covariance information and uncertainty affects the accuracy of prediction in zero-sum games using deterministic learning dynamics and different discretization methods.

Sliced Wasserstein with Random-Path Projecting Directions

https://openreview.net/forum?id=XyxuhLtFA2

Compressor summary: The paper proposes an optimization-free method for slicing distribution selection that speeds up Monte Carlo estimation using random-path projecting direction and two variants of sliced Wasserstein.

Robust Learning-Augmented Dictionaries

https://openreview.net/forum?id=XyhgssAo5b

Compressor summary: RobustSL is a novel data structure that uses predictions of access frequencies to implement optimal and robust dictionaries with logarithmic runtime.

Realistic Unsupervised CLIP Fine-tuning with Universal Entropy Optimization

https://openreview.net/forum?id=XxCfToC9pJ

Compressor summary: The paper proposes Universal Entropy Optimization (UEO), a method that improves CLIP's performance on downstream tasks by adjusting textual prompts and visual transformations for out-of-distribution detection and known class recognition.

Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning

https://openreview.net/forum?id=XwnABAdH5y

Compressor summary: The paper proposes a reinforcement learning algorithm that helps language models become more trustworthy in retrieval augmentation without explicit supervision.

Generalization Analysis of Stochastic Weight Averaging with General Sampling

https://openreview.net/forum?id=XwVkqvyziD

Compressor summary: The paper analyzes stochastic weight averaging method's advantages over stochastic gradient descent in non-convex and realistic scenarios, using mathematical induction to derive stability and generalization bounds.

Probability Distribution of Hypervolume Improvement in Bi-objective Bayesian Optimization

https://openreview.net/forum?id=XvmooikuHE

Compressor summary: The paper derives an exact expression for the probability distribution of hypervolume improvement in bi-objective optimization using Gaussian process models and proposes a new acquisition function, $\varepsilon$-probability of hypervolume improvement, which outperforms other methods in high uncertainty settings.

Autoencoding Conditional Neural Processes for Representation Learning

https://openreview.net/forum?id=XuQPA4D396

Compressor summary: PPS-VAE is a model that learns which pixels to observe in order to improve contextual image completion and extract meaningful information from images.

Transferring Knowledge From Large Foundation Models to Small Downstream Models

https://openreview.net/forum?id=XtDJaSe8jE

Compressor summary: AFT adaptively transfers useful pre-trained features to small, task-specific downstream models, improving performance and allowing combination of multiple pre-trained models.

How Learning by Reconstruction Produces Uninformative Features For Perception

https://openreview.net/forum?id=XsDWw1Mn2p

Compressor summary: The paper discusses how input space reconstruction can lead to misalignment between learning to reconstruct and learning for perception, and suggests that learning by denoising might be a better approach.

Non-confusing Generation of Customized Concepts in Diffusion Models

https://openreview.net/forum?id=XoencoHWy7

Compressor summary: CLIF is a method to improve text-guided diffusion models for generating customized concepts by fine-tuning CLIP using contrastive learning.

Value-Evolutionary-Based Reinforcement Learning

https://openreview.net/forum?id=XobPpcN4yZ

Compressor summary: VEB-RL combines evolutionary algorithms with value-based reinforcement learning, enhancing sample efficiency and performance for RL optimization.

How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization

https://openreview.net/forum?id=XoSF46Pc2e

Compressor summary: The text describes a framework to design differentially private algorithms that efficiently find approximate solutions for various classes of non-convex loss functions.

Pseudo-Calibration: Improving Predictive Uncertainty Estimation in Unsupervised Domain Adaptation

https://openreview.net/forum?id=XnsI1HKAKC

Compressor summary: PseudoCal is a novel post-hoc calibration framework for unsupervised domain adaptation that uses inference-stage mixup to generate labeled pseudo-target data, addressing the challenge of poorly calibrated predictive uncertainty on target data.

Generative Marginalization Models

https://openreview.net/forum?id=XmLNDlQuzO

Compressor summary: Marginalization models are a new type of generative model that can quickly and flexibly approximate arbitrary marginal probabilities for high-dimensional discrete data, using energy-based training to enable any-order generative modeling.

MAGNOLIA: Matching Algorithms via GNNs for Online Value-to-go Approximation

https://openreview.net/forum?id=XlgeQ47Ra9

Compressor summary: The paper proposes a graph neural network to solve online Bayesian bipartite matching problems, which estimate expected weights of matchings and leverage local graph structures.

A Closer Look at the Limitations of Instruction Tuning

https://openreview.net/forum?id=XkHJo8iXGQ

Compressor summary: The paper explores the limitations of Instruction Tuning (IT) for large language models, showing that it fails to enhance knowledge or skills and can even degrade response quality.

Neuro-Visualizer: A Novel Auto-Encoder-Based Loss Landscape Visualization Method With an Application in Knowledge-Guided Machine Learning

https://openreview.net/forum?id=XiemSZpvh0

Compressor summary: The paper introduces Neuro-Visualizer, a new auto-encoder-based method for visualizing neural network loss landscapes that is more flexible and accurate than existing linear methods and provides useful insights for knowledge-guided machine learning applications.

LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned Proportions

https://openreview.net/forum?id=XhH1OKLANY

Compressor summary: LeaPformer is a new approach that uses dynamic proportions instead of fixed positions to improve linearized transformers' performance in various tasks.

Graph Structure Extrapolation for Out-of-Distribution Generalization

https://openreview.net/forum?id=Xgrey8uQhr

Compressor summary: The paper proposes a novel data augmentation method for graph out-of-distribution generalization that extrapolates structure spaces to generate unseen graph data without compromising causal mechanisms.

On the Minimal Degree Bias in Generalization on the Unseen for non-Boolean Functions

https://openreview.net/forum?id=Xeh8171Fce

Compressor summary: The study explores how random feature models and Transformers learn in different domain settings and shows that minimal degree interpolators are only learned in specific cases like the Boolean setting with roots of unities.

FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler

https://openreview.net/forum?id=XecUTmB9yD

Compressor summary: FedCal is a novel approach for local and global calibration in federated learning that improves prediction accuracy and reduces calibration error by leveraging client-specific scalers and weight averaging.

Position: Benchmarking is Limited in Reinforcement Learning Research

https://openreview.net/forum?id=Xe7n2ZqpBP

Compressor summary: The paper discusses the challenges of conducting reliable reinforcement learning experiments and suggests an alternative approach due to high computational costs.

On the Last-Iterate Convergence of Shuffling Gradient Methods

https://openreview.net/forum?id=Xdy9bjwHDu

Compressor summary: Shuffling gradient methods, a popular machine learning technique, have improved theoretical guarantees for their performance in different optimization settings.

Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF

https://openreview.net/forum?id=Xb3IXEBYuw

Compressor summary: The paper proposes a new algorithmic framework for solving bilevel RL problems with dynamic objective functions using penalty formulation and shows its effectiveness through simulations.

CW Complex Hypothesis for Image Data

https://openreview.net/forum?id=XXioxiADDC

Compressor summary: The authors propose a new hypothesis that image data is distributed in "manifolds with skeletons" to explain why diffusion models struggle to generate detailed shapes like human hands, and they support their hypothesis through visualization and testing on natural images.

Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms

https://openreview.net/forum?id=XWkRyIjYDp

Compressor summary: The paper analyzes the stability and generalization of stochastic compositional optimization algorithms using statistical learning theory, introducing a new concept called "compositional uniform stability" and deriving dimension-independent excess risk bounds for two popular algorithms.

Magicoder: Empowering Code Generation with OSS-Instruct

https://openreview.net/forum?id=XUeoOBid3x

Compressor summary: Magicoder is an open-source LLM for code that uses OSS-Instruct to create realistic and controllable synthetic instruction data, outperforming other models on various coding benchmarks.

Towards a Better Theoretical Understanding of Independent Subnetwork Training

https://openreview.net/forum?id=XUc29ydmLX

Compressor summary: The text discusses how Independent Subnetwork Training (IST) addresses communication and memory issues in large-scale machine learning by analyzing its optimization performance on a quadratic model.

Parameter-Efficient Fine-Tuning with Discrete Fourier Transform

https://openreview.net/forum?id=XUOHKSsurt

Compressor summary: FourierFT is a method that compresses trainable parameters in fine-tuning foundation models using the Fourier transform, achieving comparable or better performance than LoRA with fewer parameters.

Harmonic Self-Conditioned Flow Matching for joint Multi-Ligand Docking and Binding Site Design

https://openreview.net/forum?id=XTrMY9sHKF

Compressor summary: HarmonicFlow is a simple and general method to generate 3D protein-ligand binding structures, while FlowSite improves it further by jointly designing protein pockets and ligand structures for better binding site design.

Clifford-Steerable Convolutional Neural Networks

https://openreview.net/forum?id=XTglHJjzQI

Compressor summary: Clifford-Steerable Convolutional Neural Networks (CS-CNNs) are a new type of equivariant CNNs that work with multivector fields and outperform baseline methods in fluid dynamics and relativistic electrodynamics.

Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States

https://openreview.net/forum?id=XT6iF8FDZx

Compressor summary: The paper investigates how policy gradient in reinforcement learning biases controllers towards unseen initial states based on exploration during training, and suggests selecting initial states for training to improve real-world performance.

Fair Classification with Partial Feedback: An Exploration-Based Data Collection Approach

https://openreview.net/forum?id=XSsoggg8pz

Compressor summary: The approach trains classifiers using available data and explores subpopulations to collect outcome data, ensuring fairness and convergence while minimally reducing predictive accuracy.

Network Tight Community Detection

https://openreview.net/forum?id=XQz7ytgETQ

Compressor summary: The text introduces a new community detection method (TCD) that identifies tight communities without including scattered nodes, which improves accuracy and reveals biological implications in networks.

Stability Evaluation through Distributional Perturbation Analysis

https://openreview.net/forum?id=XPP6K57bop

Compressor summary: The paper proposes a stability evaluation method using optimal transport that can handle data corruptions and sub-population shifts in real-world scenarios, and provides convex formulations and computational methods for different loss functions.

From Neurons to Neutrons: A Case Study in Interpretability

https://openreview.net/forum?id=XMlUlY7ONf

Compressor summary: MI allows us to understand how neural networks learn meaningful low-dimensional representations of complex data, which can reveal insights about the underlying problem and help derive new knowledge.

Test-Time Degradation Adaptation for Open-Set Image Restoration

https://openreview.net/forum?id=XLlQb24X2o

Compressor summary: The paper proposes a test-time adaptation framework for open-set image restoration that adapts to unknown degradations using a diffusion model and an adapter.

Bounding the Excess Risk for Linear Models Trained on Marginal-Preserving, Differentially-Private, Synthetic Data

https://openreview.net/forum?id=XKxuTZRCXq

Compressor summary: The paper proposes using differentially-private synthetic data to train ML models without revealing sensitive information from real data, and provides bounds on the risk of linear models trained on synthetic data.

Risk-Sensitive Reward-Free Reinforcement Learning with CVaR

https://openreview.net/forum?id=XGq30hC5MW

Compressor summary: The paper proposes a novel risk-sensitive exploration framework for reinforcement learning based on Conditional Value-at-Risk (CVaR), which works for any reward function and has near-optimal sample complexity.

Enforcing Constraints in RNA Secondary Structure Predictions: A Post-Processing Framework Based on the Assignment Problem

https://openreview.net/forum?id=XGGcnKelda

Compressor summary: The paper proposes a new algorithm for post-processing ML predictions on RNA secondary structures that ensures biological relevance and improves performance.

Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities

https://openreview.net/forum?id=XDz9leJ9iK

Compressor summary: The paper discusses how foundation models could worsen existing disparities against marginalized communities in various aspects of AI and suggests ways to address these issues.

Enhancing Implicit Shape Generators Using Topological Regularizations

https://openreview.net/forum?id=XBNhJQU84y

Compressor summary: The paper proposes a method to fix topological artifacts in 3D shape generative models by using topological regularization losses on an implicit shape generator.

Mixtures of Experts Unlock Parameter Scaling for Deep RL

https://openreview.net/forum?id=X9VMhfFxwn

Compressor summary: The paper shows that using Soft MoE modules in value-based networks improves the scalability and performance of reinforcement learning models.

OT-CLIP: Understanding and Generalizing CLIP via Optimal Transport

https://openreview.net/forum?id=X8uQ1TslUc

Compressor summary: The authors analyze CLIP using Optimal Transport and propose new losses for image and text tasks, as well as a graph-based inference method for zero-shot classification and related subtasks.

Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency

https://openreview.net/forum?id=X8Ha2NiQcy

Compressor summary: Sparse Iso-FOP Transformations (Sparse-IFT) improves accuracy of dense neural network models by efficiently using sparsity to maintain FLOPs.

DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training

https://openreview.net/forum?id=X7UnDevHOM

Compressor summary: Key points: - Pre-training improves efficiency and performance of neural operators for PDE data - New auto-regressive denoising pre-training strategy is stable, efficient, and generalizes well - Flexible and scalable model architecture based on Fourier attention allows large-scale pre-training - Achieve SOTA on 10+ PDE benchmarks and enhance performance on diverse downstream tasks Summary: The paper proposes a new pre-training strategy and a flexible model architecture for neural operators on PDE data, achieving state-of-the-art results and improving generalization to various tasks.

Finite Volume Features, Global Geometry Representations, and Residual Training for Deep Learning-based CFD Simulation

https://openreview.net/forum?id=WzD4a5ufN8

Compressor summary: This paper proposes new geometric representations and features for graph neural network-based computational fluid dynamics simulations, improving accuracy with low-resolution data.

Online Cascade Learning for Efficient Inference over Streams

https://openreview.net/forum?id=Wz4lgc8dsN

Compressor summary: Online cascade learning uses smaller models to imitate large language models, reducing inference costs while maintaining accuracy in data stream processing tasks.

Best Arm Identification for Stochastic Rising Bandits

https://openreview.net/forum?id=WwLtwPHmSM

Compressor summary: The paper studies sequential decision-making problems with improving options and proposes two algorithms to identify the best option within a fixed budget.

MoMo: Momentum Models for Adaptive Learning Rates

https://openreview.net/forum?id=WvvkbWD1vL

Compressor summary: The authors propose MoMo and MoMo-Adam, adaptive learning rates that improve performance and reduce tuning of momentum-based methods for various machine learning tasks.

Leveraging (Biased) Information: Multi-armed Bandits with Offline Data

https://openreview.net/forum?id=WvIHbQhrTq

Compressor summary: The paper proposes MIN-UCB, an online policy that uses offline data effectively when they are informative, and performs better than UCB in stochastic multi-armed bandits.

Exploring the LLM Journey from Cognition to Expression with Linear Representations

https://openreview.net/forum?id=WtvI3QijEF

Compressor summary: The paper analyzes how cognitive and expressive abilities in bilingual language models evolve during different phases of training, finding that cognitive capacity may limit expressive potential.

Position: Do pretrained Transformers Learn In-Context by Gradient Descent?

https://openreview.net/forum?id=WsawczEqO6

Compressor summary: Our study questions the connection between In-Context Learning (ICL) in pre-trained language models and Gradient Descent (GD), revealing inconsistencies in their behavior and output modifications, leaving the equivalence open.

Rethinking Decision Transformer via Hierarchical Reinforcement Learning

https://openreview.net/forum?id=WsM4TVsZpJ

Compressor summary: The authors present a hierarchical RL framework that improves Decision Transformer by enabling seamless stitching of sub-optimal trajectories, leading to better offline RL performance on control and navigation tasks.

Time Weaver: A Conditional Time Series Generation Model

https://openreview.net/forum?id=WpKDeixmFr

Compressor summary: TIME WEAVER is a novel model that uses heterogeneous metadata to improve time series generation and introduces a new evaluation metric for conditional generation approaches.

Watermark Stealing in Large Language Models

https://openreview.net/forum?id=Wp054bnPq9

Compressor summary: The text discusses the vulnerabilities of language model watermarking schemes, which can be bypassed by watermark stealing attacks that enable spoofing and scrubbing of AI-generated content.

Cross-domain Open-world Discovery

https://openreview.net/forum?id=WofwaWjIf7

Compressor summary: CROW is a prototype-based method that discovers and matches novel classes in cross-domain open-world settings using foundation models and a well-structured representation space.

Conditional Common Entropy for Instrumental Variable Testing and Partial Identification

https://openreview.net/forum?id=Wnni3cu39x

Compressor summary: This paper proposes a method to bound causal effects using instrumental variables under weak confounding and a criterion to falsify the IV with extra information on the confounder.

Universal Gradient Methods for Stochastic Convex Optimization

https://openreview.net/forum?id=Wnhp34K5jR

Compressor summary: The paper introduces universal gradient methods for stochastic convex optimization that adapt to noise and smoothness without prior knowledge and achieve state-of-the-art convergence rates.

Variational Inference with Coverage Guarantees in Simulation-Based Inference

https://openreview.net/forum?id=Wn4QwCrDvH

Compressor summary: CANVI is a scalable and easily implemented method for simulating posterior approximations with guaranteed marginal coverage and high predictive efficiency in likelihood-free settings.

Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Control

https://openreview.net/forum?id=WjvEvYTy3w

Compressor summary: HyperDistill combines morphology-conditioned hypernetworks and policy distillation to learn efficient robot policies that perform as well as a universal transformer teacher on various morphologies, reducing model size and computational cost significantly.

FedREDefense: Defending against Model Poisoning Attacks for Federated Learning using Model Update Reconstruction Error

https://openreview.net/forum?id=Wjq2bS7fTK

Compressor summary: FedREDefense is a method to defend against model poisoning attacks in federated learning by identifying and filtering out malicious clients based on discrepancies in their model update reconstruction errors.

From Generalization Analysis to Optimization Designs for State Space Models

https://openreview.net/forum?id=WjNzXeiOSL

Compressor summary: Key points: - SSM is a foundation model in time series analysis, alternative to transformers - Paper studies generalization of SSMs and proposes improvements to training algorithms - Data-dependent generalization bound for SSMs shows interplay between parameters and temporal dependencies - Scaling rule for initialization and new regularization method introduced based on generalization bound - Numerical results validate the proposed methods Summary: The paper analyzes the generalization of SSMs, a model in time series analysis, and proposes new training algorithms based on a data-dependent generalization bound that considers parameters and temporal dependencies.

Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion

https://openreview.net/forum?id=Wj5wm3Os5v

Compressor summary: QUAG probes reveal that VideoQA Transformers may not effectively use multimodal information and CLAVI dataset challenges them further.

Decouple then Classify: A Dynamic Multi-view Labeling Strategy with Shared and Specific Information

https://openreview.net/forum?id=WfJuiIiFzB

Compressor summary: The paper proposes a dynamic labeling strategy for semi-supervised learning that uses shared and specific information to improve sample classification confidence and performance.

Graph Neural PDE Solvers with Conservation and Similarity-Equivariance

https://openreview.net/forum?id=WajJf47TUi

Compressor summary: This study introduces a generalizable machine learning architecture using graph neural networks that incorporates physical laws and symmetries to solve partial differential equations more reliably.

Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities

https://openreview.net/forum?id=WYi3WKZjYe

Compressor summary: Audio Flamingo is a new audio language model that can understand diverse sounds, adapt to tasks quickly, and engage in multi-turn dialogues, achieving state-of-the-art results on various audio understanding tasks.

Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF

https://openreview.net/forum?id=WXg6MJo1FH

Compressor summary: RLHF uses human feedback to teach AI models human values, but its reward model often deteriorates after one epoch; a new algorithm called 'Iterative Data Smoothing' improves it by updating data with soft labels instead of hard ones.

GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model

https://openreview.net/forum?id=WWo9G5zyh0

Compressor summary: The paper presents GeoReasoner, a vision-language model enhanced with human inference knowledge that improves geo-localization performance significantly.

Criterion Collapse and Loss Distribution Control

https://openreview.net/forum?id=WVORGH73Cg

Compressor summary: The paper studies when optimizing one performance metric leads to optimizing another, focusing on error probability minimization under different learning criteria, and shows that non-monotonic criteria can prevent collapse in some cases.

Allocation Requires Prediction Only if Inequality Is Low

https://openreview.net/forum?id=WUicA0hOF9

Compressor summary: The authors propose an evaluation framework for predictive allocation systems in settings with hierarchical units and show that their efficacy is limited by between-unit inequality, intervention budget, and other factors.

One Size Fits All for Semantic Shifts: Adaptive Prompt Tuning for Continual Learning

https://openreview.net/forum?id=WUi1AqhKn5

Compressor summary: AdaPromptCL is a new method for continual learning that adapts to varying degrees of semantic shifts between tasks using assign-and-refine semantic grouping.

Cascade-CLIP: Cascaded Vision-Language Embeddings Alignment for Zero-Shot Semantic Segmentation

https://openreview.net/forum?id=WUdq1WFUPr

Compressor summary: Cascade-CLIP aligns multi-level visual features with text embeddings using independent decoders to improve zero-shot semantic segmentation performance.

Data-Efficient Learning via Clustering-Based Sensitivity Sampling: Foundation Models and Beyond

https://openreview.net/forum?id=WUQ4YzIQt2

Compressor summary: The paper proposes a data selection method using k-means clustering and sensitivity sampling to efficiently train machine learning models with low error.

Gradient-based Visual Explanation for Transformer-based CLIP

https://openreview.net/forum?id=WT4X3QYopC

Compressor summary: Grad-ECLIP is a method to explain how CLIP, a vision-language model, matches image-text pairs by producing heat maps that show the influence of image regions or words on the results.

Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions

https://openreview.net/forum?id=WSpPC1Jm0p

Compressor summary: The paper proposes FreeShap, a more robust and efficient method for instance attribution in foundational models, which can improve explainability and performance in various data-centric tasks.

A Subquadratic Time Algorithm for Robust Sparse Mean Estimation

https://openreview.net/forum?id=WSi4IiMaCx

Compressor summary: The paper proposes an efficient algorithm for robust sparse mean estimation that runs in subquadratic time using few samples.

Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers

https://openreview.net/forum?id=WRIn2HmtBS

Compressor summary: The Hourglass Diffusion Transformer (HDiT) is a high-resolution image-generative model that uses the Transformer architecture and trains efficiently without special techniques.

Memorization Through the Lens of Curvature of Loss Function Around Samples

https://openreview.net/forum?id=WQbDS9RydY

Compressor summary: The paper proposes a curvature-based metric to measure memorization in deep neural networks and shows its effectiveness in detecting mislabeled samples and unique failures.

Active Label Correction for Semantic Segmentation with Foundation Models

https://openreview.net/forum?id=WPt9HRmMrG

Compressor summary: Key points: - Semantic segmentation requires pixel-wise annotations, which are hard to obtain and error-prone - The paper proposes an active label correction (ALC) framework that uses correction queries to fix pseudo labels of pixels - The method is more annotator-friendly than standard ones and leverages foundation models for zero-shot predictions on superpixels - The method improves semantic segmentation and label correction performance on various datasets, including PASCAL Summary: The paper presents an ALC framework that uses annotator-friendly correction queries to fix pixel-wise annotations for semantic segmentation, leveraging foundation models and superpixels.

Position: Exploring the Robustness of Pipeline-Parallelism-Based Decentralized Training

https://openreview.net/forum?id=WPfYVdJHPk

Compressor summary: The paper discusses potential threats to decentralized machine learning training, proposes a poisoning attack, and suggests a robust training framework with detection and efficiency mechanisms.

Why Larger Language Models Do In-context Learning Differently?

https://openreview.net/forum?id=WOa96EG26M

Compressor summary: Larger language models are more sensitive to noise in the test context, while smaller ones are more robust to noise due to different attention mechanisms during in-context learning.

Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models

https://openreview.net/forum?id=WLPhywf1si

Compressor summary: The paper proposes an unsupervised adversarial fine-tuning method to make the CLIP vision encoder more robust against attacks that could spread fake information or defraud users in multi-modal foundation models like LLaVA and OpenFlamingo.

Tackling Non-Stationarity in Reinforcement Learning via Causal-Origin Representation

https://openreview.net/forum?id=WLGWMDtj8L

Compressor summary: COREP is a new algorithm that addresses complex non-stationarity in reinforcement learning by implicitly tracing the causal origin of changes in the environment and learning a stable graph representation for the state, leading to improved policy learning.

Robust Graph Matching when Nodes are Corrupt

https://openreview.net/forum?id=WJn1BAx9aj

Compressor summary: The text introduces two models for matching graphs with corrupt nodes and studies their detection and estimation properties.

Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agents

https://openreview.net/forum?id=WJ5fJhwvCl

Compressor summary: The study introduces S-DQN and S-PPO, novel algorithms that improve clean rewards, empirical robustness, and robustness guarantee in deep reinforcement learning with randomized smoothing.

Linear Explanations for Individual Neurons

https://openreview.net/forum?id=WIbntm28cM

Compressor summary: The paper proposes a new way to understand the function of individual neurons in neural networks by explaining them as linear combinations of concepts and evaluating their explanations using simulations.

An Empirical Study of Realized GNN Expressiveness

https://openreview.net/forum?id=WIaZFk02fI

Compressor summary: The authors propose a new dataset, BREC, to measure the realized expressiveness of Graph Neural Networks (GNNs) that surpass the 1-dimensional Weisfeiler-Lehman test and show the gap between theory and practice in GNN expressiveness.

Language Models as Science Tutors

https://openreview.net/forum?id=WFyolnFZOR

Compressor summary: TutorEval and TutorChat are new NLP datasets for training language models to assist in STEM education using long textbook chapters and dialogues.

Two Heads Are Better Than One: Boosting Graph Sparse Training via Semantic and Topological Awareness

https://openreview.net/forum?id=WDgV1BJEW0

Compressor summary: Graph Sparse Training (GST) is a new method that dynamically adjusts the sparsity of graphs to optimize performance on Graph Neural Networks (GNNs) while preserving topological and semantic information.

Agnostic Interactive Imitation Learning: New Theory and Practical Algorithms

https://openreview.net/forum?id=WCwxFM7n5S

Compressor summary: The paper proposes MFTPL-P, an oracle-efficient algorithm for interactive imitation learning with provable guarantees and wide applicability, and Bootstrap-DAgger, a practical variant without extra samples.

GistScore: Learning Better Representations for In-Context Example Selection with Gist Bottlenecks

https://openreview.net/forum?id=WCVC5wGZyz

Compressor summary: Example Gisting is a novel approach for selecting informative examples that improves In-Context Learning performance in Large Language Models, and can be used for new tasks without additional training.

Learning High-Order Relationships of Brain Regions

https://openreview.net/forum?id=WC14xZIaC2

Compressor summary: HyBRiD is a novel method that extracts maximally informative and minimally redundant high-order relationships from fMRI data to improve phenotypic predictions in neuroscience.

Time Series Diffusion in the Frequency Domain

https://openreview.net/forum?id=W9GaJUVLCT

Compressor summary: The paper explores using Fourier analysis as an inductive bias for score-based diffusion models, showing that frequency diffusion models outperform time diffusion models on real-world datasets.

Spectral Phase Transition and Optimal PCA in Block-Structured Spiked Models

https://openreview.net/forum?id=W97gFmrKe6

Compressor summary: The paper investigates an inhomogeneous Wigner spike model to study structured noise and finds an optimal spectral method for detecting signal-noise separation.

Noise-Adaptive Confidence Sets for Linear Bandits and Application to Bayesian Optimization

https://openreview.net/forum?id=W8hBNk1FhQ

Compressor summary: The paper proposes a new semi-adaptive and variance-adaptive confidence set for linear bandits, which improves exploration efficiency when the noise level is unknown.

Position: Quo Vadis, Unsupervised Time Series Anomaly Detection?

https://openreview.net/forum?id=W7Vqx1Jvc2

Compressor summary: The paper critiques the current machine learning research on Timeseries Anomaly Detection, highlighting issues with evaluation metrics and methods, and advocating for improved benchmarking practices and simpler models.

FlowMM: Generating Materials with Riemannian Flow Matching

https://openreview.net/forum?id=W4pB7VbzZI

Compressor summary: FlowMM is a pair of generative models for predicting and proposing stable crystal structures using Riemannian Flow Matching with extended symmetries and flexibility, achieving state-of-the-art performance and efficiency in comparison to other methods.

Generalization Analysis for Multi-Label Learning

https://openreview.net/forum?id=W4mLp5KuKl

Compressor summary: Key points: - Generalization theory of multi-label learning is still in early stage - Novel vector-contraction inequalities are developed to derive generalization bounds - Bounds depend weakly on the number of labels and capture label correlations - Macro-Averaged AUC bound is derived and analyzed with class-imbalance Summary: The paper develops novel inequalities to analyze and improve the generalization of multi-label learning, considering label correlations and class-imbalance.

In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization

https://openreview.net/forum?id=VyoY3Wh9Wd

Compressor summary: FT-PFN is a fast and accurate surrogate for Freeze-thaw Bayesian optimization, achieving state-of-the-art results in deep learning hyperparameter tuning.

Graph Neural Networks with a Distribution of Parametrized Graphs

https://openreview.net/forum?id=VyfEv6EjKR

Compressor summary: The paper proposes a method to improve graph neural networks by generating multiple graphs from latent variables and estimating their distribution using EM and MCMC techniques, leading to better node classification.

Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts

https://openreview.net/forum?id=VyGo1S5A6d

Compressor summary: P4D is a tool that tests the reliability of safety mechanisms in diffusion models by finding problematic prompts that can bypass them.

Symmetric Matrix Completion with ReLU Sampling

https://openreview.net/forum?id=VxI0gInNlh

Compressor summary: The paper studies symmetric positive semi-definite low-rank matrix completion with entry-dependent sampling and proposes a tailor-designed initialization for gradient descent to achieve global optimality.

Self-Consistency Training for Density-Functional-Theory Hamiltonian Prediction

https://openreview.net/forum?id=Vw4Yar2fmW

Compressor summary: The authors propose self-consistency training, a machine learning method for predicting molecular Hamiltonians in density functional theory without labeled data, which improves generalization and efficiency in data-scarce and out-of-distribution scenarios.

Automating the Selection of Proxy Variables of Unmeasured Confounders

https://openreview.net/forum?id=VuoB86HiCL

Compressor summary: The paper proposes methods to estimate causal effects using proxy variables of unobserved confounding in linear models with multiple unmeasured confounders, without prior knowledge of their validity.

Guidance with Spherical Gaussian Constraint for Conditional Diffusion

https://openreview.net/forum?id=VtqyurB4Af

Compressor summary: Diffusion with Spherical Gaussian constraint (DSG) is a method that improves conditional diffusion models by constraining guidance steps within data manifold, leading to better sample quality and faster sampling process.

BAGEL: Bootstrapping Agents by Guiding Exploration with Language

https://openreview.net/forum?id=VsvfSMI5bs

Compressor summary: BAGEL is a method that helps language model agents learn to follow natural language instructions in digital environments by converting random trajectories into synthetic demonstrations using two noisy components, improving their performance and reducing failures.

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning

https://openreview.net/forum?id=VrwIrAa1Lc

Compressor summary: Random Masking is a simple and effective method to fine-tune language models using fewer parameters by creating a flatter loss landscape and allowing larger learning rates.

Towards Realistic Model Selection for Semi-supervised Learning

https://openreview.net/forum?id=VoMPNYTZud

Compressor summary: SSL models use less labeled data and are hard to evaluate with validation sets, so a new method called SLAM combines spectral complexity and margin distribution to improve their generalization performance without relying on validation data.

Auctionformer: A Unified Deep Learning Algorithm for Solving Equilibrium Strategies in Auction Games

https://openreview.net/forum?id=VnI9200eeL

Compressor summary: Auctionformer is an efficient transformer-based method that solves equilibria of diverse auctions using tokenization and Nash error as a loss term, outperforming existing approaches.

Position: Will we run out of data? Limits of LLM scaling based on human-generated data

https://openreview.net/forum?id=ViZcgDQjyG

Compressor summary: The paper explores the potential limitations of large language models due to the limited availability of public human-generated text data and suggests alternative methods to continue improving them.

Improved Bounds for Pure Private Agnostic Learning: Item-Level and User-Level Privacy

https://openreview.net/forum?id=VfWrXJtLSL

Compressor summary: The paper studies pure private learning in the agnostic model, deriving improved upper bounds for item- and user-level privacy and presenting tighter results for user-level privacy and learning thresholds.

Reinforcement Learning and Regret Bounds for Admission Control

https://openreview.net/forum?id=Vdr87ZUfnl

Compressor summary: The paper analyzes the regret of reinforcement learning algorithms for an admission control problem in queuing systems and proposes a new algorithm with improved bounds.

Towards Neural Architecture Search through Hierarchical Generative Modeling

https://openreview.net/forum?id=VdZfEMuoj2

Compressor summary: The proposed method generates a tailored search space for Neural Architecture Search using a two-level generative model hierarchy, achieving state-of-the-art performance with low computational costs on various tasks.

A Single-Loop Robust Policy Gradient Method for Robust Markov Decision Processes

https://openreview.net/forum?id=VaZVZQSgTP

Compressor summary: The paper proposes a new single-loop robust policy gradient method for solving Markov Decision Processes with global optimality guarantee and better convergence performance than existing methods.

VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Model

https://openreview.net/forum?id=Va7mhTVy5s

Compressor summary: VoroNav is a novel semantic exploration framework for household robots that uses Reduced Voronoi Graph to generate text-based descriptions of paths, enabling commonsense reasoning and outperforming existing methods in ZSON task.

Rényi Pufferfish Privacy: General Additive Noise Mechanisms and Privacy Amplification by Iteration via Shift Reduction Lemmas

https://openreview.net/forum?id=VZsxhPpu9T

Compressor summary: The paper proposes a Rényi divergence-based variant of Pufferfish to improve its applicability, utility, and composition guarantees for privacy preservation in machine learning algorithms.

Codebook Features: Sparse and Discrete Interpretability for Neural Networks

https://openreview.net/forum?id=VZ5A0LPbnc

Compressor summary: The paper proposes codebook features, which quantize continuous neural network features into discrete vector codes, enabling interpretability and control over the model's behavior.

Discrete Latent Perspective Learning for Segmentation and Detection

https://openreview.net/forum?id=VWCpm39peL

Compressor summary: The paper presents a novel framework called Discrete Latent Perspective Learning (DLPL), which enables networks to understand images from different perspectives using single-view images and improves performance on various computer vision tasks.

Non-parametric Online Change Point Detection on Riemannian Manifolds

https://openreview.net/forum?id=VW7Jk8KhNC

Compressor summary: The paper introduces an online algorithm for detecting changes in data from Riemannian manifolds using a generalized Karcher mean computed by stochastic Riemannian optimization, and provides theoretical and empirical performance analysis.

Simplicity Bias via Global Convergence of Sharpness Minimization

https://openreview.net/forum?id=VUTyzH63Xa

Compressor summary: Label noise SGD makes two-layer neural networks simple by always using a single linear feature across all neurons.

Latent Optimal Paths by Gumbel Propagation for Variational Bayesian Dynamic Programming

https://openreview.net/forum?id=VSwrXRqD9o

Compressor summary: The authors propose a method to solve dynamic programming problems with variational Bayesian inference, Gibbs distributions, and message-passing algorithms, and apply it to generate text-to-speech and singing voice.

Equivariant Diffusion for Crystal Structure Prediction

https://openreview.net/forum?id=VRv8KjJNuj

Compressor summary: EquiCSP is a new equivariant diffusion-based model for Crystal Structure Prediction (CSP) that improves accuracy and convergence compared to existing methods.

Positional Knowledge is All You Need: Position-induced Transformer (PiT) for Operator Learning

https://openreview.net/forum?id=VOcsmIBiXE

Compressor summary: PiT is a more efficient and interpretable attention mechanism for Transformer-based operator learning, inspired by numerical methods for PDEs.

Rate-Optimal Policy Optimization for Linear Markov Decision Processes

https://openreview.net/forum?id=VJwsDwuiuH

Compressor summary: The paper presents a fast and efficient policy optimization algorithm that achieves the optimal regret bound in both stochastic and adversarial online Markov decision processes.

Second-Order Uncertainty Quantification: A Distance-Based Approach

https://openreview.net/forum?id=VJjjNrUi8j

Compressor summary: The text discusses challenges in measuring predictive uncertainty using second-order probability distributions and proposes formal criteria and a framework for developing better uncertainty measures, using the Wasserstein distance as an example.

Mobile Attention: Mobile-Friendly Linear-Attention for Vision Transformers

https://openreview.net/forum?id=VHtIDVaOKC

Compressor summary: The paper proposes Mobile-Attention, a novel attention mechanism for ViTs that balances efficiency and capability on mobile devices by using a head-competition mechanism and information flow to prevent overemphasis on less important subspaces while preserving essential ones.

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models

https://openreview.net/forum?id=VHO4nE7v41

Compressor summary: MeZO-SVRG is a memory-efficient optimization method for fine-tuning language models, improving accuracy and reducing computation time compared to previous methods.

Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks

https://openreview.net/forum?id=VF177x7Syw

Compressor summary: The study proposes Hare \& Tortoise, a method for neural networks that combines rapid adaptation and gradual knowledge integration to balance plasticity and generalization.

Stealing part of a production language model

https://openreview.net/forum?id=VE3yWXt3KB

Compressor summary: The text describes a method to extract information from black-box language models by exploiting their API access, revealing hidden dimensions and projection matrices.

On the Convergence of Projected Bures-Wasserstein Gradient Descent under Euclidean Strong Convexity

https://openreview.net/forum?id=VDgfJnOEMV

Compressor summary: The paper proposes a general convergence rate guarantee for BW gradient descent with constraints and provides a fast implementation method for constrained problems, showing improved performance in experiments.

A Neural-Preconditioned Poisson Solver for Mixed Dirichlet and Neumann Boundary Conditions

https://openreview.net/forum?id=VAKkoJjVpn

Compressor summary: The authors propose a novel neural network preconditioner for Poisson equations with mixed boundary conditions that works efficiently even when domain shapes, boundary conditions, or grid sizes change and outperforms existing methods.

An Embodied Generalist Agent in 3D World

https://openreview.net/forum?id=V4qV08Vk6S

Compressor summary: LEO is a multi-modal agent that excels in various 3D tasks, trained with 3D vision-language and -action alignment using large-scale datasets and LLMs.

Provable Contrastive Continual Learning

https://openreview.net/forum?id=V3ya8RlbrW

Compressor summary: The authors provide theoretical insights into the performance of contrastive continual learning, propose a novel algorithm called CILA that uses adaptive distillation coefficients, and achieve state-of-the-art results on standard benchmarks.

Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models

https://openreview.net/forum?id=V3OpGwo68Z

Compressor summary: Flash-Diffusion is a novel method that adapts compute power and inference times to the difficulty of reconstruction tasks in inverse problems using estimated degradation severities.

No Free Prune: Information-Theoretic Barriers to Pruning at Initialization

https://openreview.net/forum?id=Uzb45nolTb

Compressor summary: This text discusses how lottery tickets in deep learning are sparse subnetworks that require heavy data-dependent masks and pruning near initialization is infeasible due to mutual information between sparsity mask and data.

Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentations

https://openreview.net/forum?id=Uz4Qr40Y3C

Compressor summary: Connect Later framework combines pretraining with targeted augmentations to improve out-of-distribution generalization for domain adaptation tasks.

Bring Your Own (Non-Robust) Algorithm to Solve Robust MDPs by Estimating The Worst Kernel

https://openreview.net/forum?id=UqoG0YRfQx

Compressor summary: EWoK is an online approach that learns robust policies for sequential decision-making by simulating the worst transition kernel scenarios while using any off-the-shelf non-robust RL algorithm.

Arrows of Time for Large Language Models

https://openreview.net/forum?id=UpSe7ag34v

Compressor summary: The study investigates time directionality in Autoregressive LLMs and finds a consistent but subtle difference in their predictability, which they explain using sparsity and computational complexity.

Do Transformer World Models Give Better Policy Gradients?

https://openreview.net/forum?id=Uoved2xD81

Compressor summary: Key points: - The paper proposes Action-conditioned World Models (AWMs) for reinforcement learning with transformers - AWMs provide more direct routes for gradient propagation and easier optimization landscapes - AWMs outperform baselines in long-horizon tasks Summary: The paper introduces AWMs, a class of world models that use transformers and condition on actions to improve gradient propagation and policy learning in long-horizon reinforcement learning problems.

Explain Temporal Black-Box Models via Functional Decomposition

https://openreview.net/forum?id=Uo3LNg5SLY

Compressor summary: FDTempExplainer is a novel explanation method for black-box time series models that reveals temporal interactions and outperforms existing approaches.

On dimensionality of feature vectors in MPNNs

https://openreview.net/forum?id=UjDp4Wkq2V

Compressor summary: The paper shows that, for non-polynomial activation functions, MPNNs with constant-dimensional feature vectors can simulate the WL test, by using linear independence over rationals instead of reals.

Outlier-aware Slicing for Post-Training Quantization in Vision Transformer

https://openreview.net/forum?id=Uh5XN9d2J4

Compressor summary: This paper proposes reconstruction granularity as a novel solution to mitigate outliers in quantized transformer models and develops an algorithm for finding the optimal granularity, achieving state-of-the-art performance in post-training quantization.

ELF: Encoding Speaker-Specific Latent Speech Feature for Speech Synthesis

https://openreview.net/forum?id=Ug1m4P4AKf

Compressor summary: The paper presents a new method for modeling multiple speakers, improving their performance on feature learning and representation, enabling subjective similarity evaluation and generation of artificial speakers.

Zero-Sum Positional Differential Games as a Framework for Robust Reinforcement Learning: Deep Q-Learning Approach

https://openreview.net/forum?id=UdXDUDxq11

Compressor summary: The paper introduces a new reinforcement learning approach based on differential game theory that improves robustness to uncertainty and outperforms existing methods in real-world applications.

Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models

https://openreview.net/forum?id=UcOze9EXEc

Compressor summary: DFML can adapt to new tasks without data by using pre-trained models, but it needs to consider model heterogeneity and balance task conflicts and overfitting risk for effective learning.

Initial Guessing Bias: How Untrained Networks Favor Some Classes

https://openreview.net/forum?id=UZstTlLq1E

Compressor summary: The paper analyzes how the structure of a deep neural network can lead to all predictions being assigned to the same class before training, influenced by factors like preprocessing methods, activation functions, and network depth.

Irregular Multivariate Time Series Forecasting: A Transformable Patching Graph Neural Networks Approach

https://openreview.net/forum?id=UZlMXUGI6e

Compressor summary: This study introduces t-PatchGNN, a novel method to model correlations between irregular multivariate time series using transformable patches and adaptive graphs for improved forecasting.

Active Ranking and Matchmaking, with Perfect Matchings

https://openreview.net/forum?id=UZZaWUR0n4

Compressor summary: The text discusses a novel algorithm that efficiently ranks items/players with varying values using optimal matching and minimizing comparison costs.

Rethinking Momentum Knowledge Distillation in Online Continual Learning

https://openreview.net/forum?id=UW5nO9NGjt

Compressor summary: The paper proposes and analyzes Momentum Knowledge Distillation (MKD) for Online Continual Learning (OCL), a challenging problem where neural networks are trained on a continuous data stream with severe constraints, and shows its effectiveness in improving accuracy.

Graph Positional and Structural Encoder

https://openreview.net/forum?id=UTSCK582Yo