The evidence gap
A deliberately sceptical search for evidence that makerspaces cause income, jobs or new businesses in rural areas and in the Global South found none. No claim that makerspaces cause these results held up when it was checked.
The closest studies:
| Study | Finding | Why it does not close the gap |
|---|---|---|
| Youth skills randomised trial, Côte d’Ivoire[55] | A large short-term rise in employment | The effect was gone by 12 to 15 months, and the programme did not involve making. |
| Makerspaces and entrepreneurship[32] | More people started businesses | United States data. Not rural, not the Global South, and income is not measured. |
| Vocational training systematic review[56] | Effects mostly small, and close to zero for paid jobs | A reality check that any claim linking skills training to jobs has to get past first. |
| Youth enterprise trial, Rwanda[41] | Pushing participants toward starting businesses had effects nobody intended | Points the other way: this kind of programme can make things worse for people it does not suit. |
What is well evidenced is the learning side.
- A systematic review of many studies finds that makerspaces help build creativity[24].
- Making builds skills recognised as important for future work, and can give rise to new ways of meeting social needs[29].
- More people take part when participants are given real control over the work[27].
The learning claim is solid. The livelihood claim is not.
Tempting responses that do not work
Say it anyway
Write “creates sustainable livelihoods” in the brochure and hope nobody checks. This works until a funder’s evaluator checks, and then it costs more than the grant was worth.
Borrow evidence from elsewhere
Quote the United States makerspace study and hope nobody notices the different setting. The study is real. The conclusion drawn from it is not, and the difference in setting is the whole question.
Drop the claim
Talk only about creativity and confidence. That is safe, but it gives up on the result the community came for: people come for a livelihood, not for creativity.
What the academy does instead
DreamSpace Academy does four things.
- Split the claim into tiers. “Earn” is three claims of very different strength: small income, a sustained livelihood, and work created for others. The first is near and low-risk. The second is the hypothesis to test. The third appears in the research but has not been measured here. Once they are separated, each can be judged on its own evidence, instead of all three resting on the weakest.
- Treat the middle tier as a theory of change with a baseline. That means a stated idea of how the work should lead to a livelihood, measured from a starting point (the baseline), checked against related research, and never presented as a proven result.
- Design for results that last, because fading is the known way these programmes fail. The strongest trial in this area showed gains gone by 12 to 15 months[55]. That argues for an ongoing relationship, with mentoring and links to local businesses, rather than a course that ends and a certificate that goes in a drawer.
- Produce the evidence the field lacks. An organisation that runs the programme itself is in a position almost nobody else is in. It can keep a register of participants, record each person's livelihood when they join, and follow up at 3, 6, 12, 24 and 36 months, including the people who left.
The eight instruments, the registers and checks that make each result measurable, exist for this. They follow one rule of order: instruments before indicators. An indicator added before there is a register of individual participants and a starting measurement produces numbers with nothing to compare them with.
What this asks of a funder
Fund the intervention and the evidence the field lacks.
Measuring this costs money. Following people up five times over three years, including finding people who stopped coming, takes real staff time and produces no photographs. It is the first line cut from most budgets, and it is the only line that can settle the question.
It also means accepting in advance that DreamSpace Academy will publish results that are not flattering.
- The academy will report stopped projects alongside successful ones.
- It will report each venture as active, dormant, closed or transformed, separately.
- It will publish limitations and gaps in the evidence next to positive findings.
If the answer turns out to be that this work does not improve livelihoods, that finding is worth more to the field than another set of attendance figures.
Commitments the academy can be held to
DreamSpace Academy commits to the following:
- No income claim without a starting measurement when people join, and follow-up afterwards.
- No community-impact claim inferred from a prototype or an award.
- The number of different people always counted separately from the number of attendances.
- People who leave, and their reasons, recorded at every stage and reported.
- Targets set only after a reliable starting measurement exists, never before.
- Every result shown in the same order: starting point, target, actual result, the difference, the explanation, and what will be done about it.
- Two conversion measures published, not one, because in this area a single ratio can go above 100% and mean nothing.
One more: where a number would be more persuasive than the truth, the academy publishes the truth and explains why the number is missing.