2 Literature review
2.1 Representation and sortition
What does it mean to be representative?
Every sortition selection method shares the same basic aim: a panel that is fair and representative of the population. But what does that actually mean?
Fairness means the selection process is transparent, unbiased and inclusive – everyone should have an equal chance of being chosen, regardless of their background or demography. In practice, this comes down to minimising the gap in selection likelihood across the pool, typically achieved by introducing randomness into the process (Flanigan et al. 2021a); without it, selection bias creeps in implicitly.
Representativeness, meanwhile, is about whether the selected panel actually reflects the demographics and perspectives of the population it was drawn from – diverse, inclusive, and populated by people from the range of backgrounds the process set out to capture. This can be checked directly, by comparing a panel’s demographic makeup against the population’s. In practice, though, population data is typically recorded and reported one characteristic at a time – a count by age bracket, a separate count by tenure status – rather than as the joint combinations of characteristics that actually describe a person; a demographic breakdown that gets each margin right can still obscure whether specific intersections, like older renters, are proportionally represented.
Approaching it procedurally, the sortition process can be thought of in four stages:
Defining the population and traits of interest
Sampling from a random subset of the population (through survey or other outreach)
Panel selection (through algorithmic means or other method)
Panel evaluation, confirmation and engagement
Each step is a chance for bias to creep in and representativeness to suffer, which is why it matters to evaluate representativeness (or potential representativeness) at every stage, not just the end.
If the population and the traits of interest are poorly defined, for instance, the selection process may fail to capture the group’s actual diversity, leaving the panel less representative. This is a real challenge in itself: as discussed above, most population descriptions record demographic characteristics separately rather than capturing how they intersect, which is central to representativeness. Addressing that gap properly would mean wading into broader data collection and storage practices, which is beyond this thesis’s scope – but it is worth keeping in mind that representation goes beyond the margins.
The same applies to survey response: if certain groups are less likely to respond, they end up underrepresented in the pool of potential panelists before a panel is even assembled. If a survey of an area that is 60% women somehow drew no female respondents at all, representativeness would already be violated well before selection begins. Cases like this raise the question of how to account for non-response bias, and whether visualisation can help show the gap between the respondent pool and the population – which is part of what this thesis sets out to do.
Panel selection
Suppose there is a pool of individuals numbered 1 to n, from which a panel of size k is to be chosen. Let K be the set of all feasible panels – any group of k individuals satisfying the required constraints, which generally take the form of quotas.
Quotas specify a desired count, or range of counts, for individuals in a given trait class (e.g. 20-22 males, or 15-17 rural), tied to whichever demographics a project has deemed important to represent. The core goal of sortition, then, is to build a panel that meets those quota constraints equitably, robustly, and in a way that genuinely represents the population it was drawn from.
Existing methods
Non-algorithmic/self-selection
This is the simplest method of panel selection: individuals are simply allowed to self-select, for example by advertising the opportunity and taking sign-ups on a first-come, first-served basis. These methods tend not to produce representative panels, for two reasons. First, they invite self-selection bias – involvement depends on exposure rather than random chance, and the people who put their hand up are more likely to already hold formed opinions, ahead of ever joining the panel. Second, they tend toward lower engagement, since there is no guarantee of reaching a particular level of exposure in the first place.
LEGACY heuristic (Flanigan et al. 2021a; Sortition Foundation 2024)
Already an improvement on non-algorithmic methods, the LEGACY selection heuristic builds a panel by selecting respondents one at a time until it is full. At each step, it calculates a “need” value to find the trait currently furthest from its desired count, then picks a random individual who has that trait. If selecting someone would push a trait past its upper quota, everyone else with that trait is removed from the pool before the next pick. This repeats until the panel is complete, or the process runs out of eligible individuals and restarts. LEGACY satisfies the basic requirement of meeting quotas, but it does not maximise representativeness or fairness: it ignores each individual’s actual selection likelihood (which, in extreme cases, can end up at 0%) and makes no attempt to minimise the maximum deviation from an equal chance of selection. For a time, this was the Sortition Foundation’s own method of panel selection, implemented in their GroupSelect tool, before the algorithmic fairness methods below were developed and adopted.
MaxiMin (Flanigan et al. 2021b) and MiniMax (Baharav and Flanigan 2024)
This shortcoming is addressed by the first two algorithmic sortition methods, MaxiMin and MiniMax, which tackle the problem in opposite ways. MaxiMin generates potential panels and selection probabilities, then sets out to maximise the minimum selection probability across the pool. This reduces the chance of any individual never being picked, improving fairness, but makes the method vulnerable to manipulation – someone who misreports belonging to an under-represented quota group can inflate their own odds. MiniMax instead minimises the maximum selection probability, making selection more robust against manipulation, but at the cost of fairness, since the risk of a 0% selection chance returns.
LexiMin (Flanigan et al. 2021a)
LexiMin – short for Lexicographic MaxiMin – works similarly to MaxiMin, but resolves ties sequentially rather than stopping at the single minimum. It first identifies whoever has the lowest selection chance and maximises that, then moves to the next-lowest, and so on, until every individual’s selection probability has been addressed and made as equal as possible across participants. This gives everyone the highest chance of selection achievable without disadvantaging someone already prioritised – but, like MaxiMin, it inherits a vulnerability to manipulation.
Goldilocks (Baharav and Flanigan 2024)
This brings the discussion to the imminent gold standard: the aptly-named Goldilocks algorithm. Goldilocks simultaneously maximises minimum selection probabilities (for fairness) and minimises maximum selection probabilities (for robustness), aiming for a “best of both worlds” balance between MaxiMin and MiniMax. The motivation is clear: both MaxiMin and LexiMin place no ceiling on how high an individual’s selection probability can climb, so a participant who misreports their trait membership to appear under-represented can disproportionately inflate their own odds. By capping the maximum selection probability, as MiniMax does, Goldilocks limits the payoff from that kind of manipulation while still prioritising the lowest probabilities enough to preserve most of MaxiMin and LexiMin’s fairness guarantees – a selection rule that is simultaneously hard to manipulate and reasonably fair.
Why focus on LexiMin
Goldilocks may be the newer and, in theory, the better algorithm, but this thesis works with LexiMin instead. The reasons are practical. LexiMin is the algorithm actually in use: it runs inside Panelot (Flanigan et al. 2020), the tool practitioners currently use to put real panels together, while no public version of Goldilocks existed at the time of writing. Testing LexiMin means testing the algorithm actually shaping panels today, not one that only exists on paper so far.
It also makes little difference to the specific problem this thesis is looking at. Goldilocks was built to fix manipulation, not to fix how panels handle intersections of traits – and on that front, it works the same way LexiMin does. Both algorithms set selection probabilities purely from marginal quotas, and neither one looks at how traits combine. Capping the highest selection probability, which is all Goldilocks adds on top of LexiMin, has no obvious reason to change how well a panel’s mix of traits reflects its sample. So the gap this chapter identifies below should apply to Goldilocks just as much as it applies to LexiMin. Working with LexiMin here is a matter of what could actually be tested and what practitioners actually use, not a claim that it matters more – the same argument should carry over to Goldilocks once it can be tested directly.
LexiMin approach (Flanigan et al. 2021a)
Rather than committing to a single panel immediately, LexiMin constructs a probability distribution over the full set of feasible panels K, assigning each panel S \in K a non-negative probability p_S that sums to one across the set. An individual’s overall selection chance is then the sum of the probabilities of every panel that includes them. Sorting these chances from smallest to largest, LexiMin proceeds lexicographically: it maximises the smallest probability first, then – holding that value fixed – maximises the second-smallest, and so on until every individual has been addressed.
Formal representation, algorithm and logic
This process can be expressed formally. Let p_S denote the probability assigned to panel S, which must be non-negative and sum to one across every feasible panel in K,
p_S \ge 0, \qquad \sum_{S \in K} p_S = 1. \tag{2.1}
The resulting selection probability for an individual i is the sum of the probabilities of every feasible panel containing them, giving the full vector of individual selection probabilities across the population,
\pi_i(p) = \sum_{S \in K,\ i \in S} p_S, \qquad \pi(p) = (\pi_1(p), \pi_2(p), \ldots, \pi_n(p)). \tag{2.2}
Sorted from smallest to largest, \pi_{(1)}(p) \le \pi_{(2)}(p) \le \ldots \le \pi_{(n)}(p), LexiMin chooses the distribution p that maximises this vector in lexicographic order – the smallest entry first, then the second-smallest subject to the first remaining maximised, and so on down the vector.
In practice, this runs as a sequence of steps: every feasible panel in K is listed alongside every possible probability distribution over those panels, and each candidate distribution’s resulting selection probabilities are computed and sorted as above. Only the distribution (or distributions) that maximise the smallest probability \pi_{(1)}(p) are retained; if several remain tied, this narrows further to those that also maximise the second-smallest, and so on, resolving ties one probability at a time until a single distribution remains, from which a final panel is then selected.
At its core, then, LexiMin is purely a fairness rule: it prioritises whoever has the lowest chance of selection, improves their odds first, then works through the next-lowest probabilities until no further improvement is possible without disadvantaging someone already prioritised – leaving selection probabilities as equal as possible across the population, within the panel’s quota constraints.
2.2 Visualisation methods
Current sortition practice communicates a sample or panel’s composition almost exclusively through pie charts, or bar charts showing the same information in a different shape – one chart per demographic category, each showing that variable’s marginal breakdown in isolation. Both are easy to produce and easy for a non-specialist audience to read at a glance, which likely explains their popularity with practitioners reporting on panel composition. But neither can encode more than a single categorical variable at a time: there is no visual channel left over to show how two variables relate to one another. For example, a practitioner working only from pie or bar charts can confirm that a panel’s age breakdown looks right, and separately that its tenure breakdown looks right, but has no way of seeing whether the right combinations of age and tenure made it onto the panel. Faceting a bar chart by a second variable does not fix this: each facet’s bars are rescaled relative to that facet alone, rather than the whole population, so an association between the two variables – which can only be seen when proportions stay relative to the full population – is exactly what gets obscured by splitting the chart apart in the first place.
Mosaic plots are a longstanding answer to exactly this limitation. A mosaic plot divides a rectangle into tiles whose widths and heights are proportional to the categories of two (or more) variables, so that each tile’s area represents the joint frequency of that specific combination, rather than either variable’s marginal frequency alone (Hofmann 2003). Graphical methods for categorical data are comparatively underdeveloped next to those available for numeric data, and mosaic plots have been established in the statistical graphics literature as one of the more effective options for visualising relationships between multiple categorical variables at once. Because tile area scales with both variables simultaneously, a departure from proportional tiling signals that the two variables are not independent, in a way no pie chart or ordinary bar chart can show. The current R package for building mosaic plots within the ggplot2 grammar of graphics is ggmosaic2 (Friendly et al. 2026), an actively maintained extension of the original ggmosaic implementation (Jeppson et al. 2026).
2.3 Discussion and new investigation
This approach is effective at making selection probabilities as equal as possible, which helps make sortition fairer by reducing situations where specific individuals are disadvantaged. It does not, however, provide any way to impose joint trait requirements on subsets of the population – something that may be necessary for accurate representation of community groups. That gap reflects a broader pattern across this literature: algorithmic effort has gone almost entirely into optimising the fairness and robustness of the selection mechanism, rather than validating the outcome that mechanism produces. A panel satisfying every fairness and robustness guarantee an algorithm offers is not thereby guaranteed to represent the population it was drawn from well – these are related but distinct properties, and the literature so far has focused heavily on the former at the expense of the latter, undermining representativeness as a goal given the motivation for sortitioned panels in the first place: filling a room with individuals who accurately and descriptively reflect a community’s population diversity and characteristics.
One specific way this gap shows up: no algorithmic technique surveyed here stratifies for combinations of demographic traits in its decision-making – each instead affects individual selection chances purely through quotas set on each trait at the margin. This is a real limitation, since it means the selection process may fail to capture the group’s actual diversity, leaving a less representative panel. In some situations – side-panels built for specific subgroups, or communities with known intersectional minorities of interest – ensuring specific intersectional combinations are included may be necessary for genuine representation. This gap is generally patched in practice by bringing in “specialist speakers” during deliberation (Redman and Carson 2022), but that is a different kind of inclusion than being part of the panel and its decision-making directly.
Notably, this gap is rarely argued for directly; it is largely left unaddressed. The implicit assumption running through this literature appears to be that satisfying marginal quotas is enough – that selecting individuals to hit the right counts for each trait independently will, through the randomness built into the selection mechanism, naturally produce a panel whose combinations of traits are also reasonably representative, without this needing to be checked or enforced directly. None of the algorithmic methods reviewed above test whether this actually holds, and the literature more broadly does not appear to examine it either.
This thesis treats that assumption as an open empirical question rather than a safe default. The simulations that follow test how much intersectional inclusion happens naturally under current methods, and establish a way to measure how much intersectionality and association survives the selection process.