Optimization at NeurIPS 2024

Notes on optimization research presented at NeurIPS 2024

Optimization was one of the top five topics at NeurIPS 2024, reflecting a clear trend of increasing interest in this area compared to previous years. While optimization in machine learning often focuses on specific tasks like training neural networks (using algorithms such as gradient descent and its variants), my focus here is on optimization in a broader sense.

This includes research exploring the use of AI methods for classical optimization problems, bridging the gap between traditional optimization techniques and modern AI-driven approaches. Below is a curated list of papers that stood out in this domain.

Non-Linear Programming

  • Dual Lagrangian Learning for Conic Optimization
  • BPQP: A Differentiable Convex Optimization Framework for Efficient End-to-End Learning
  • IPM-LSTM: A Learning-Based Interior Point Method for Solving Nonlinear Programs

Combinatorial Optimization

  • Controlling Continuous Relaxation for Combinatorial Optimization
  • Learning to Handle Complex Constraints for Vehicle Routing Problems
  • ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution

Stochastic and Contextual Optimization

  • Optimal Algorithms for Online Convex Optimization with Adversarial Constraints
  • Improved Algorithms for Contextual Dynamic Pricing
  • Conformal Inverse Optimization
  • Multi-Stage Predict+Optimize for (Mixed Integer) Linear Programs
  • There is No Silver Bullet: Benchmarking Methods in Predictive Combinatorial Optimization
  • Regret Minimization in Stackelberg Games with Side Information

Linear Programming and MILP

  • GLinSAT: The General Linear Satisfiability Neural Network Layer
  • SymILO: A Symmetry-Aware Learning Framework for Integer Linear Optimization
  • Learning Generalized Linear Programming Value Functions
  • Rethinking the Capacity of Graph Neural Networks for Branching Strategy
  • On the Power of Small-Size Graph Neural Networks for Linear Programming

Application-Specific Optimization

  • Randomized Sparse Matrix Compression for Large-Scale Constrained Optimization in Cancer Radiotherapy
  • DistrictNet: Decision-Aware Learning for Geographical Districting
  • Approximately Pareto-Optimal Solutions for Bi-Objective k-Clustering
  • Autoregressive Policy Optimization for Constrained Allocation Tasks

New Perspectives

  • Optimization Algorithm Design via Electric Circuits
  • FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning

Optimization for Machine Learning

While there are many works related to optimization in machine learning, the following papers stood out for their relevance to OR techniques.

  • Safe and Efficient: A Primal-Dual Method for Offline Convex CMDPs under Partial Data Coverage
  • Gradient-Free Methods for Nonconvex Nonsmooth Stochastic Compositional Optimization
  • Can Learned Optimization Make Reinforcement Learning Less Difficult
  • Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization
  • SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization
  • ROIDICE: Offline Return on Investment Maximization for Efficient Decision Making
  • Adaptive Proximal Gradient Method for Convex Optimization
  • Functional Bilevel Optimization for Machine Learning
  • First-Order Minimax Bilevel Optimization
Back to Blog