Why Classical Optimisation Fails for Some Problems
Classical numerical optimisation โ gradient descent, Newton-Raphson, sequential quadratic programming โ works beautifully for smooth, continuous, differentiable objective functions. Given a starting point, these methods follow the gradient downhill to a local minimum. For well-behaved problems (convex functions, continuous variables, smooth constraints), they find the global optimum reliably and efficiently. But engineering problems frequently violate these conditions:
- Discrete variables: Number of bolts must be an integer. Pipe diameter must be a standard size. No gradient exists for discrete decisions.
- Multi-modal landscapes: Many local optima. Gradient methods converge to whichever local optimum is nearest the starting point, potentially missing the global optimum.
- Non-differentiable objectives: Optimising for minimum cost with discrete catalogue sections. Cost function has discontinuities at each step change in section size.
- High dimensionality: Topology optimisation with thousands of design variables, or truss layout optimisation with arbitrary member connectivity.
The Biological Inspiration
John Holland introduced genetic algorithms in 1975, inspired by Darwin's theory of natural selection. In biological evolution, a population of organisms competes for survival; those with traits better suited to the environment reproduce more successfully; their genetic material โ encoded in DNA and mixed through sexual reproduction โ propagates to the next generation. Over many generations, the population improves. Mutation introduces random variation, preventing convergence to a single genotype and maintaining the diversity that allows exploration of new solutions. Holland's key insight was encoding candidate solutions as "chromosomes" โ strings of bits or real numbers โ and applying selection, crossover, and mutation operators to evolve better solutions over successive generations. The mathematics of schema theorem explains why the approach works: short, high-fitness building blocks (schemas) are sampled and combined exponentially more frequently than the actual evaluation rate, giving GAs implicit parallelism in their search.
Crossover, Mutation, and Selection: The Key Operators
Selection determines which solutions reproduce. Tournament selection (randomly choose k solutions, let the best reproduce) is robust and widely used. Roulette wheel selection weights reproduction probability by fitness. Elitism โ always carrying the best few solutions unchanged to the next generation โ prevents losing good solutions through random variation. Crossover combines genetic material from two parents. For binary-encoded problems, single-point crossover swaps all genes after a random cut point; uniform crossover independently swaps each gene with 50% probability. For real-valued problems, blend crossover creates offspring within a range around the parent values. For engineering topology problems, structured crossover preserving valid configurations requires careful design. Mutation introduces random perturbations to maintain diversity and prevent premature convergence. For binary chromosomes, mutation flips individual bits with a small probability (typically 1/chromosome_length). For real-valued chromosomes, Gaussian mutation adds a normally distributed random value to each gene. Mutation rate is a critical parameter: too low, the algorithm stagnates; too high, it degenerates into random search.
GA vs Gradient Methods: Choosing the Right Tool
Genetic algorithms are not a replacement for gradient-based methods โ they are a complement. Gradient methods are faster and more precise for smooth, continuous, unimodal problems. GAs are more robust for discrete, multi-modal, and black-box problems where gradients don't exist or are unreliable. A common engineering workflow combines both: use a GA to find the basin of attraction of the global optimum (approximate answer, large design space), then refine with a gradient method (precise answer, local region). This hybrid approach gets the robustness of GA with the precision of gradient descent. Modern evolutionary computation extends far beyond the basic GA: particle swarm optimisation, differential evolution, CMA-ES (Covariance Matrix Adaptation Evolution Strategy), and NSGA-II for multi-objective problems. Each has strengths for different problem types. The field continues to develop rapidly, with connections to machine learning (neuroevolution, hyperparameter tuning) adding new application domains every year. While EngForge's Monte Carlo tool performs probabilistic sampling rather than evolutionary optimisation, it provides similar insights into how design variables affect outcomes โ sampling the design space to identify which parameters matter most.