During the Zemi today, I caught myself jotting down some thoughts about topographical approaches in evolutionary computation, and open and closed optimization. I'll write down these thoughts below as seeds for myself and others, but be forwarned that there is no clear conclusions or messages in this post, just open ended ideas.
Topographical Approaches
In recent months, I have spent a lot of time thinking about, and encouraging my students to think about, topographical approaches. A topographical approach is when some key part of the algorithm depends on spatial relations. So you have CGA (Cellular GA) where individual solutions can only perform crossover with neighboring solutions in a grid, or some PSO variants where the calculation of the global optima is constrained to a neighborhood graph, or island model genetic algorithms.
I think the key benefit of topographical approaches is that they promote an implicit decentralization of the algorithm. It seems that my thinking is always coming back to this concept of decentralization, as independent, but connected, algorithimic structures. Decentralization can promote diversity, as each component of the system can diverge down a diferent path, smaller costs for each component, and robustness in a "don't keep all your eggs in one basked" sense. It suffers from possible overhead and loss of benefits of scale.
More generally, natural evolution is an inherently decentralized process, and affected heavily by topographical effects (in particular the existence of niches).
Open and Closed Optimization
One problem I've been working on recently is the parameter setting of physical models through optimization. The idea is that an optimization algorithm can find the best parameters of some model by minimizing the error in the output of a model configured by a set o parameters.
One assumption in this task is that there is a single "truth", the reality that the model is trying to represent. There are some caveats for this assumption, but it informs many subsequent tasks and approaches. However, I caught myself thinking today whether evolutionary computation could be truly an appropriate approach for problems with this kind of assumption.
Evolutionary Computation mimics the process of natural evolution to find optimal solutions to a given problem. However, as much as natural evolution is a good optimizer, it seems that its key strength is its ability to find a variety unique solutions to a problem -- or in other words, on the fact that there are many solutions that exploit different facets of the same problem. We see this clearly on the diversity of life in our planet.
The contradiction between heuristic optimization, where you start with a diverse population and narrow down to a few down optimal solutions, and the process of evolution, where a single primordial soup evolved into a diverse ecosystem, is not a new thought to me. But it came back to haunt me as we discussed the idea of using "one truth" as an assumption for certain methods.
Of course, from a mathematical point of view, meta-heuristic search based optimizers such as Evolutionary Computation work by approximating the distribution of the search landscape through iterated sampling, and do not actually depend on ecological niches. This is specially true in methods that make the sampling distribution more explicit, such as the ES family.
Still, worth thinking about the nebulous edge between Evolutionary Computation and Computational Evolution.