Welcome to the OR-Exchange, your site for questions and answers in operations research.

3

From what I can gather, there is not much theory behind local search methods like Tabu Search, Simulated Annealing, Large Neighborhood Search, etc. By theory I mean some mathematical argument as to why these methods work in the cases that they do.

Despite that I feel there might exist some good theories about such methods that I would be interested in.

If someone knows of a good set of such references (maybe survey papers for instance), I would be grateful.

flag

4 Answers

3

A list of textbooks and compilations (far from being complete or representative) discussing theoretical aspects -- as well as run-time behaviour and computational results -- of local search techniques ...

Collections:

  • Handbook of Approximation Algorithms and Metaheuristics (Gonzalez. 2007)
  • Search Methodologies - Introductory Tutorials in Optimization and Decision Support Techniques (Burke, Kendall. 2005)
  • Handbook of Metaheuristics (Glover et al. 2003)
  • Local Search in Combinatorial Optimization (Arts, Lenstra. 1997)
  • Modern Heuristic Techniques for Combinatorial Problems (Reeves ...)

Monographs:

  • Tuning Metaheuristics - A Machine Learning Perspective (Birattari. 2009; 2ed)
  • Theoretical Aspects of Local Search (Michiels, Aarts, Korst. 2007)
  • Stochastic Local Search - Foundations and Applications (Hoos, Stützle. 2004)
  • Algorithmics for Hard Problems - Introduction to Combinatorial Optimization, Randomization, Approximation, and Heuristics (Hromkovic. 2003)

"Classics":

  • Tabu Search (Glover ...)
  • Theoretical and Computational Aspects of Simulated Annealing (Laarhoven. 1998)
  • Simulated Annealing and Boltzmann Machines - A Stochastic Approach to Combinatorial Optimization and Neural Computing (Aarts, Korst. 1989)
  • Simulated Annealing - Theory and Applications (Laarhoven, Aarts. 1987)

PhD Thesis:

Basics, Rationale and Bibliographies:

  • Handbook of Combinatorial Optimization, Vol. 1-3 + Supplements A+B (Du, Pardalos ...)
  • Combinatorial Optimization - Algorithms and Complexity (Papadimitriou, Steiglitz. 1998)
  • Annotated Bibliographies in Combinatorial Optimization (Dell'Amico et al. 1997)
  • Combinatorial Optimization - Annotated Bibliographies (Lenstra et al. 1985)
  • Surveys in Combinatorial Optimization (Martello et al. 1987)
link|flag
3

There is a large body of work studying the finite-time convergence properties of Simulated Annealing, based on Markov chain models. You could start with the following paper:

  • Nolte and Schrader (2000) A note on the finite time behaviour of simulated annealing. Mathematics of Operations Research, 25(3) 476-484.

There isn't as much theoretical study made into the convergence of deterministic methods like tabu search, even though they have been proven empirically to be more successful in general.

link|flag
2

If you look at something like "Game of life" there are some interesting theory on that (for example you can prove that in the limit your machine starts to make a highway regardless of initial conditions). Most of these methods fall under probabilistic algorithms so there is basically no argument about convergence or complexity. But you can use some of the results from stochastic systems for example we know that symmetric 3D random walks are transient while 2D random walks are null recurrent that explains why most of the agent based methods work nicely in 2D but fail in 3D. (basically a drunk ant will get back home eventually but a drunk bird gets lost forever)

link|flag
1

I liked Pascal Van Hentenryck.'s book on 'constraint-based local search'.

link|flag

Your Answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.