Doing more with less: detecting shifts, spotting weak signals - SenseMaker I

I promised a series of posts talking about using SenseMaker. They'll be interspersed over the coming weeks with others - I don't particularly want to be singing a one-note song and sending you all away in search of diversity...

I railed against targets last week, thinking specifically about the aid and development world. Targets, of course, are not restricted to that sector - recent discussions have been in healthcare and education, community cohesion and volunteering, citizen engagement and security. These are all worlds where proxy targets have been set to measure an impact or outcome - usually one that was seen as difficult to measure directly.

So, in schools, we look at records to see that records are filled in on children, that lesson planning is done. In hospitals we measure time to be treated in A&E, etc, etc. And these miss the point - we want children to be taught well and, where possible, inspired. We want people to be treated for injuries properly.

But the targets have produced something else - teachers who can fill in plans, but can't ignite a pupil's mind; ambulances that sit with patients outside the A&E department so that the clock doesn't start ticking yet.

The narrative (and hence SenseMaker) approach is radically different. Instead of measuring how many times a particular behaviour is met, we collect stories in response to our prompting questions. Usually from the person most effected - the patient or the pupil. And we have them signify their stories, which is both quicker and easier than your average survey form once you've done it a couple of times. And we ask for positive and negative stories.

In the past, colleagues have given iPods to patients and visitors as they arrive, asking them to record a note of anything they think important. And we've had children doing it with audio recorders or, for those old enough to type, entering them directly on a website. And from the signifiers we get great indications of where people are - a good baseline measure.

The great thing is that, once we've done that and the system is in place, we get continuous collection of more data - so we can start to see shifts easily. And that means we can test new approaches, get very quick feedback on what effect it's having and then, if necessary, read the narratives that explain why it's having that effect. Making it surprisingly easy to decide what to do next.

More with less

One approach we've been discussing recently is around finding new ways of supporting local policing arrangements. By gathering stories into SenseMaker around fear of crime and other community issues, we can tell what is giving a particular local community greatest concern. And, in these days of having to do more with less, we can start some simple experiments to see what effect we can have. If burglaries are perceived as an issue, then changing the times of uniformed officers walking in the area might have an effect.

Outliers and weak signals

We can also see where there are narratives and events that sit outside the most common events - stories that show up as being more extreme/negative/positive/inspirational. And by seeing these, we can get insight into what we might encourage or discourage. In schools, these could be indicators of intolerance to sexuality or racial differences; in hospitals, these could be examples of patients feeling more cared-for through small actions.

Impact measurement

Given that these patterns can be seen in SenseMaker, it then becomes possible to use them as impact measures - not targets - that give us indication of what is improving, what is deteriorating. The stories that sit beneath the patterns then have two-fold benefits - firstly, they illuminate what is going on (and frequently why) and secondly, they give us content that allows us to illustrate the desired impacts.

One system, impact measurement, situation explanation and content for simpler interventions. It just requires a leap to understand that mass collection and analysis of narrative is now practical.