Measurement

What the academy measures

Measure changes in people, institutions, livelihoods and communities, not only activities delivered.

Activity is easy to count and easy to mistake for effect. A programme that reports workshops run, people attended and prototypes built has described its own effort, not its results. The academy measures the two separately.

The academy anchors quality to recognised frameworks: the principles of asking why, designing at scale for all, empowering teachers, engaging the ecosystem and being data-driven[54]; and the pillars of safety first, impact on learning, designed for children, accessibility, and inclusivity and equity[53]. Assessment in maker programmes is feasible and precedented, most surveyed makerspaces already do it[21].

The organisation is testing a longer progression: access → learning → capability → innovation → livelihood or enterprise → community impact → system change. That is a hypothesis to test, and it is not compulsory for any participant. Employment, research, further education, community leadership, teaching and venture creation are all valid destinations.

Six levels of result

Activities, attendances, prototypes, innovations, ventures and impacts are not interchangeable. A workshop delivered is not a skill gained. A skill gained is not a job. A prototype is not a solved problem. Reports break this rule constantly, usually by moving a result one line up the table when the numbers on the correct line are disappointing.

Innovation ladder: four things reported as one

Within the innovation story specifically, four different things get reported as one. The academy separates them and measures the drop-off between them, because the drop-off is where the information is.

Eight instruments that make the model testable

The academy builds these before it adds indicators, and adds an indicator only when it supports a defined programme, management or funding decision. Indicators added before these are in place produce numbers without the baseline needed to interpret them.

The five questions the mission is tested against

1. Was the community demonstrably underserved on a defined dimension?
2. Did the community participate meaningfully in decisions?
3. Did participants acquire demonstrated capability?
4. Did they use it after the academy's direct involvement ended?
5. Did they keep solving problems without depending on the academy?

At present the first is answered partly, the third weakly, and the fourth and fifth mainly through individual cases rather than an organisation-wide system. The academy is building the eight instruments to close that gap.

When measurement happens

Bridging: how connections are counted

Bridging is repeatedly named as this organisation’s differentiator and has essentially no external research literature supporting it as a mechanism. The academy's own record is therefore the only evidence that will exist, so these indicators carry the weight.

• Jobs and paid assignments that came through a connection
• Partnerships that produced an observable opportunity
• Local organisations that gained funding or visibility through a connection
• Research placements and university connections
• Partner organisations independently adopting the methodology
• Former participants becoming trainers, managers and directors, and the share of delivery led by local alumni

Two reporting rules apply. A partnership counts as an outcome only when it produced an observable result, delivery, funding, placement, employment, research, infrastructure, adoption or policy change. And one-way bridging does not count: a relationship where value flows only inward is sourcing, and the return leg is measured too.

How the academy reports results

1. Count unique people separately from attendances.
2. Record leavers and reasons at every stage.
3. Report ventures as active, dormant, closed or transformed, separately.
4. Do not infer community impact from a prototype or an award.
5. Do not infer income improvement without an intake baseline and follow-up.
6. Report stopped projects as well as successful ones.
7. Separate self-report, observation, administrative record and independent verification.
8. Measure retention and drop-off, not only immediate post-programme gain.
9. Adjust causal claims for attribution, deadweight, displacement and drop-off where feasible.
10. Collect only data that informs a material decision and can be protected responsibly.

How a result is presented

baseline → target → actual result → difference → explanation → corrective action

The academy sets targets after a reliable baseline exists, not before. Every indicator declares its result level and causal link, its exact numerator and denominator, whether it counts people or attendances, its baseline and target, its data source and instrument, who collects it and how often, the disaggregation required, its consent and privacy rules, its verification method, and its known limitations.

Disaggregation is by gender, disability, geography and socioeconomic condition. Age group is not a field. Age does not determine placement, and an age band entering through an intake form is how a retired band re-enters a model. Age-related information is held only for safeguarding, legal duty of care and supervision.

Income evidence: what the academy asks funders for

The income link is a hypothesis, and a sceptical search found the evidence thin. It turned up no clean result linking a rural makerspace to income, and the most rigorous skills trial fades[55, 56]. Education, creativity and community-impact outcomes are well evidenced[24, 29]. Income is the thing that has to be measured rather than asserted.

That shapes what the academy asks funders for.