On Engagement

Toward a More Reliable Measurement of Nebulous Engagement
Customer Experience

While it might be easy to judge that something is two inches thinner, three times as fast or “now has 10 percent more!”, what is there to measure in a claim that one experience is more engaging than its precursor or a competing alternative? Is there a way to meaningfully evaluate engagement in a way that makes comparing experiences possible?

As researchers, we’re convinced that a metric for engagement can be built. There have been a host of previous attempts to define engagement, but few have held up to careful scrutiny. In 2006, the Advertising Research Foundation itself endeavored to end the confusion and establish an industry standard definition: Engagement is turning on a prospect to a brand idea enhanced by the surrounding context.

Nearly a decade has passed and no one seems to know exactly what that means.

So we’ve begun looking at behaviors from our own usual starting point: consumer experience.

A measurable difference

There is a difference between the idea of an “engaging moment” and the longer-term notion of “engagement” (an aspect of a relationship). Characterizing something as an engaging moment is a way of marking an instance in which a person invests time, attention and effort into an interaction; the moment can be clearly demarcated by its beginning and end, and ultimately results in a change of some sort (usually a positive one).

 

The difference between an “engaging moment” and “engagement”

It all begins with an “engaging moment” or an instance where a person invests in an interaction. These moments accumulate over time, with their collective impact on a person’s life representing the longer-term notion of “engagement.”

A moment in contrast to a second or an hour is an experiential unit of time. Its onset is marked by an abrupt shift in one or more of the key indicators of attention and will most likely be followed by an additional amount of focused interaction with the object of the experience.

So, how does it end?

We think the final moment of an experience is something we cannot see until we look at the data retrospectively: Something has to change. If everything returns to the status quo ante, then it is unlikely that it has been an engaging experience.

Approaching metrics empirically

We hypothesize that we’ll be able to more effectively look at engagement as an outcome if we first figure out how to see something measurable in engaging moments. Through those measures, we can look carefully at the conditions and outcomes surrounding those moments.

  • What can we see?
  • What can we count?
  • What can we prove?

Granted, subjectivity plays a huge role in experience, but we are looking to quantify observable aspects of engagement – the exterior markers of an interior state. We suspect that engagement is a persistent relationship created by many interwoven engaging moments over time, and this is something we intend to experiment with more closely.

Routine and change

Individuals move through time in ways that can be meaningfully characterized, despite being (in the infinite combinations of moments, histories and situations) undoubtedly unique. Days have rhythms and routines. Individuals have life cycles, as do families and friendships. Arguments have arcs, as do careers. A “moment” is given meaning by scales that can vary significantly, but that are ultimately limited by our senses at one end and a lifespan at the other.

A deviant stimulus, something outside of the expected, gets an extra dose of attention. This “noticing” marks the “in point” of an engaging moment and can ultimately lead to change.

 

The joy of deviants stimuli

Instrumenting for moments

Understanding whether or not an interaction is working if it is doing what you intended, expected or needed it to do is where “instrumented intelligence” comes in. We believe there are ways of being inside a moment ways that don’t involve self-recall, self-reporting or multimillion dollar functional magnetic resonance imaging machines and are testing out a wide range of inexpensive (mostly wearable) technologies that provide a continuous readout of different behavior indicators.

Gadgets we’ve been tinkering with during this proof-of-concept phase include:

  • Off-the-shelf “activity trackers” (Fitbit Force and Basis Band)
  • Google Glass
  • GPS/location identification apps on personal smartphones
  • Activity tracker apps on smartphones and laptops
  • Passive data sensor package (also on smartphones, in this case)

 

 

We’ve currently settled on a handful of measures that assess or are proxies for time, attention and effort. And we’ve tried a few direct measures of what psychologists call “activation” a heightened level of emotional intensity regardless of valence in order to approximate the “positive change” aspect of the definition. They’re not exactly what one would expect anyone and everyone to have, but they don’t require a trip to a NASA supply hut either.

Testing, observation and inference

Engaging moments aren’t as rare and mysterious as some believe. They are simply a different sort of everyday event, one that can be both measured and understood.

Tom is one of the ur-nerds at SapientNitro (he’s a real guy). We outfitted Tom with a collection of gadgets that report simple, observable phenomena back to us. We put together this collection of wearable sensors as an initial pass at an instrumentation for engagement. The data help us describe behaviors and relationships between activity, time and context. This is not out of a desire to predict or control individual behavior, but rather to begin to build a map of various engagement landscapes from a unique perspective built upon continuous, in-the-moment experiences.

And begin building a map we did. To see and read about the results, including how we converted and aligned the data, download the full article PDF below.

More Insights

Locations

46 Cities.
5 Continents.
1 Vision.