Research

Value-based decision making of honeybee swarms

A well-known example of collective decision making in natural systems is the nest site selection of honeybees. In this decision scenario the swarm of bees chooses unanimously a new home among several options advertised by scout bees. Obviously, it is advantageous for the collective of bees to choose the site of highest possible quality. Therefore, the qualities of the potential new nest sites should influence the decision dynamics of the honeybee swarm as follows:

• If there is one superior option of high quality it is optimal for the swarm to reach consensus as quickly as possible and select this option.

• If all options are of equal quality with high quality value then the honeybees may select any of the available options.

• If all options are of equal quality but have low quality values then the honeybees should wait until a better option will be discovered.

Pais et al. (2013) introduced a mathematical model for two options exhibiting these features. We generalise this value-sensitive model to the nest site selection problem where honeybees choose from N options. In addition, we introduce a new parameterisation, which emphasises the importance of individual spontaneous transitions contrasted with interactions between different scout bees.

Our analysis reveals conflicting pressures on the ratio of spontaneous transitions and mutual interactions in symmetric and best-of-N decisions, which could be solved through a time-dependent signalling strategy. This result is summarised in the video below, which shows the bifurcation diagram for different quality values.

Here, x1, x2 and x3 represent populations of bees voting for three different options with qualities v1, v2 and v3. In this example option 1 (quality value v1) is the superior option and options 2 and 3 have equal qualities.

The bifurcation parameter is r – the ratio of frequencies of interactions over spontaneous transitions. The parameter κ gives the ratio between the inferior and the superior options. Blue lines and dots depict stable branches of the bifurcation diagram, and green lines and dots represent unstable equilibria, that are not accessible for the honeybee swarm.

When swiping through the different quality values, the video clearly shows the different states the swarm might be in, including decision deadlock, deadlock breaking and reaching consensus.

Read more about our best-of-N nest site selection study.

Nonlinear neuro-models of value-based decision making

Modelling decision making of individuals is another concern of the DiODe project. There is now ample evidence that value-based decision making takes place in humans and monkeys, for example see Teodorescu et al. (2016) and Pirrone et al. (2016). However, it is important to discriminate between values that represent magnitudes of input signals and values that measure a reward, although value-sensitivity is present in both types of decisions.

Using generic nonlinear stochastic models of decision making that build on linear models as, for instance, analysed in Bogacz et al. (2006), we study the influence of value-sensitivity together with effects of different neuronal inhibition and excitation mechanisms. The different motifs include feed-forward inhibition, mutual inhibition and interneuronal inhibition.

The drift-diffusion model of decision making, where the decision variable describes a random walk subject to a possible bias until a threshold is crossed, has been the standard model model since several decades. Furthermore, it has been shown by Bogacz et al. (2006) that this model is statistically optimal and thus particularly appealing. However, the drift-diffusion model does not account for value-sensitivity as recently observed in experiments by Teodorescu et al. (2016) and Pirrone et al. (2016).

In our modelling, value-sensitivity is inherent in the nonlinear mechanisms that govern the decision making process. Due to the nature of nonlinear systems we identify critical parameters and observe bifurcations that characterise different states of the decision maker, which may explain short and long decision times that deviate from average behaviour. In addition, we are also able to quantitatively compare the nonlinear models with the statistically optimal behaviour as reflected by the drift-diffusion model.

This is illustrated in the figure below, where we show the relation between mean decision time ⟨DT⟩ (normalised with respect to the total delay time Dtot) and error rate p(err), relating to a decision between two options (correct and false).

The solid line was obtained from the drift-diffusion model. Symbols represent results from simulating the nonlinear stochastic differential equations of the interneuronal inhibition motif for different excitation-over-inhibition ratios r. The parameter zopt is the threshold which makes the drift-diffusion model optimal and zsim is the threshold used in the nonlinear model to closely resemble the optimal behaviour.

The figure indicates that tuning model parameters in the nonlinear system can make the decision maker perform close to optimal, which is particularly relevant for adjusting excitation and inhibition in neuronal circuits (here modelled by means of the parameter r).

Value-based decision making in robot swarms

We employ our understanding from the theoretical studies of value-sensitive decision making to implement distributed robotics solutions. Implementation of the theories in this manner is a crucial test of their robustness in physical environment, as well as providing additional data to inform theories.

An additional benefit from implementing these behaviours in physical robots is technological; swarm robotics is frequently expounded as the next major advance in robotics, with autonomous robust robot groups facilitating exploration and other tasks in hazardous environments that are inhospitable or inaccessible to humans or larger robots.

We propose design solutions for swarm robotics systems able to make value-sensitive decisions. We test our solutions on swarms of hundreds of Kilobot robots. Implementing our experiments in the lab with large numbers of simple robots provides a proof-of-concept for these collective behaviours, that should eventually be deployable in real environments to solve real problems, when the next generation of collective robots are developed.

The video below shows a swarm of 150 Kilobots taking a value-sensitive decentralised decision between two options (red and blue). The swarm must select the best quality option if the quality is higher than a given threshold (in this study, greater than 1.5). In this experiment, the options have quality v=5 thus the swarm makes a decision for the option blue.

The overlaying coloured circles show the two options localised in the environment. The options are signalled through two static Kilobot robots acting as beacons that send infrared messages with the option’s ID and quality. The robots light up their LED in a colour that corresponds to their internal commitment state: green for the uncommitted state and red and blue for commitment to the option of the respective colour.

Supplementary video of the paper:

Reina, A, Bose, T, Trianni, V and Marshall, J A R (2016) ‘Effects of spatiality on value-sensitive decisions made by robot swarms’. DARS 2016.