The benefits and risks of objective tracking

When trying to understand customers, studying their behavior is the gold standard. We’ve all experienced situations where customers say one thing but do something completely different.

A study recently came out in the journal “Nature” that looked at mobile phone use in 100,000 people (somewhere in Europe). The researchers were able to see where people were, roughly, based on which cell tower their handset was communicating with.

The aggregated data gave a picture similar to that we can get from telemetry tracking data in our products and websites.

This data gives us a clear picture of what people are really doing, although it does not provide us with information about why they are doing it.

There has been a backlash since the release of the Nature study results. This AP article makes it sound like the researchers were doing something sneaky or unethical. In fact, what they did was NOT LEGAL in the United States because they did not have permission from the people they tracked.

Some products have internal tracking, as do many websites, but we do not always include this capability in everything we build. I think we need to be doing this all the time! There are definite benefits to having this capability:
• We can see what people really do. A usability test can tell us what barriers might exist, but it doesn't show us what people would really do. Ethnographic research can give us insights into what people really do, but the data may not hold up across all of our users (perhaps only a subset of users do it that way... maybe the majority, but not everyone).
• We can see shifts in behavior, that can inform us about the effectiveness of different aspects of a design. For example, many websites do massive numbers of A/B tests on the live site to understand what drove the most traffic to the product.

The risks are:
• Jumping to the wrong conclusion about WHY something is happening (we really need to follow up with other types of research such as surveys, site visit studies, and other methods)
• Coming across as sneaky or sly. We need to ensure that data is aggregated or anonymous, or that people opt-in or are aware of the tracking so we don’t get accused of invading their privacy.