Analytics has a user experience problem

By Wojciech Gryc (on April 8th, 2012)

In brief: most analytics platforms are developed to solve as many data-related problems as possible, and are not catered to specific uses or solutions. This makes them difficult to use by non-statisticians, and hinders successful applications of analytic techniques. We outline three ways to overcome this challenge and make analytics more user friendly.

Analytics is not user friendly. Most statistics packages, machine learning tools, and related platforms were created by scientists, for scientists. There are huge benefits to this as a result: the platforms are programmable, versatile, and often very quick when it comes to solving difficult analytic problems.

Over the last few years, however, the analytics industry has changed. Today, many non-statisticians and non-scientists base their decision-making on mathematical models and statistical analysis. Like most people in the industry, I welcome this change and am excited to see what innovations will come about as a result. The challenge, however, is ensuring these analytic tools are usable by people without complex training or statistical backgrounds.

Ask yourself: when was the last time someone discussed the wonderful user interface of their preferred analytics tool? In my case, never. Instead, I hear numerous complaints, and they focus around three core challenges around user experience and analytics platforms:

1. Outputs with too many (or irrelevant) details. Very few people actually understand or care about analytic aides like P values, standard errors, R² statistics, etc.
2. Inability to use results for specific actions or activities. For example, some tools will give you high-level results around your customers (e.g., a model) but won’t give specific predictions tied to individual customer IDs.
3. Complex and confusing scripting languages required for any complex work.

These challenges are not a result of poor design of existing systems, but rather the unique needs of a new user base. This leads us to a unique approach to developing our own platform, through three key design requirements for new platforms and analytic tools:

1. Cater to the specific needs and activities of individual user groups. Rather than building all-purpose statistical tools, there is an opportunity to focus on specific needs and functions of end users. For example: accountants, actuaries, marketers, and advertisers all use logistic regressions, but do so in different ways. Cater the experience to each unique use case.
2. Understand the level of analytic rigour needed by users. For example: academic researchers worry about P values, standard errors, R² metrics, and so on. Many people do not. Reframe model results to speak to the analytic rigour/experience of your end users.
3. Develop modelling tools that are customized to the specific needs of users. While existing quality metrics (e.g., R², root mean squared error, etc.) are useful for comparing models, they become confusing when discussing applicability to real-world scenarios. If my model’s R² metric is 0.73, how can I expect it to perform in forecasting sales in my stores next week? Reframe or rebuild analytic models to meet the needs of the (non-academic, extremely applied) end user.

These are formidable challenges and much needs to be done in the coming years. It is exciting to see what future analytic tools will look like, but one thing is certain: the analytics landscape will be significantly different in 2 or 3 years.