What is Little Data? And Why Do Project Managers Need It?

October 13, 2017 by Andy SilberAndy Silber



One of the fastest-growing technology trends today is Big Data, probably behind Internet of Things and ahead of Virtual Reality. For instance, a search on the job site Indeed for “Big Data” returned almost 20,000 entries.

But what about Little Data (which only gets four hits on Indeed)?

Big data is mining huge, heterogeneous data sets and pulling out subtle information that can inform all sorts of decisions.

Let’s look at climate change science as an example. Data comes from atmospheric measurements over Hawaii, temperature data across the globe, ice cores from Antarctica and Greenland, underwater measurements from all the world’s seas, and more. Some of the data were taken by satellite last week; others written in notebooks centuries ago.

[Further Reading: How to Be a Project Team of the Future: Preparing for Industry 4.0]

It’s been pored over by scientists from every country in the world. The data and analysis needed to predict how the climate is changing and will change is complicated. To really understand it requires a PhD. The details are so complex that we have been unable to decisively act on this critical issue.

Little data is the opposite. It’s the 2+2=4 kind of things.

Little data is the obvious observations and conclusions that those paying attention will catch and can use to their advantage. It’s looking outside, seeing it’s raining, and deciding to put on a jacket. It’s noticing that the prices and quality of the food is better at one store than another and using that information to decide where to shop. It’s noticing that if you drink coffee after 5 p.m., you have trouble going to sleep, so you stop drinking coffee after 5 p.m.

Little data has three steps:

To some extent, little data and big data are close to the same thing, it’s just a matter of degree. The biggest difference is that the analysis for little data is straightforward. If you’re looking for someone with a PhD in math to help with your analysis, that’s not little data. For little data, you should be able to do the analysis in Excel. The challenge is knowing how to respond to your results.

Here’s a straightforward way to use little data to improve your schedules: record task estimates and actuals. The data should include who did the estimation and the work. Using a pivot table in Excel, you can see which estimators typically underestimate or overestimate. You can also see which of your team members take more or less time than was predicated. There are many ways this data can improve your organization, including:

As is often the case, data acquisition is straightforward, analysis simple, and the response requires further digging.

Key Performance Indicators (KPIs) are a common little data management technique. Leadership decides that certain easily measurable metrics are key to the organization’s success, targets are set, and data acquired. If the performance does not reach the target, then some form of response is taken.

For example, you may be managing a manufacturing line. Your KPI is the number of units manufactured per hour. In creating the manufacturing process, you know you can build 100 units an hour, so you set your target at 80 units an hour to account for the normal hiccups (e.g. you’re training a new team member).

[Further Reading: AI, IoT, and the Future of Manufacturing]

Data collection and analysis are easy. If you’re meeting your target, you can move on to other issues or raise the target. Ideally, if you don’t meet your target, the response is agreed to prior to acquiring the data. Often it just indicates you need to dig deeper, as in this case. As is all small data analysis, the challenge is in the response, not acquiring or analyzing the data.

One of the most controversial examples of little data are standardized school tests. The data is homogenous and straightforward (if time consuming) to collect. The naïve analysis is trivial (average score by grade and school). The response is complex and fraught with challenges.

In 2010, an elementary school near my house was labeled as failing according to the No Child Left Behind law. A majority of the students were from refugee and immigrant families. Many didn’t speak English at home, which certainly posed a challenge for the school.

The metric, test scores, didn’t determine what action was called for, but made clear there was a problem. The district responded by bringing in a new principal and new teachers, and a concerted effort was undertaken to improve performance.  After four years, the performance of the school went from one of the worst sch...

(RSS generated with FetchRss)

Previous Article
Using the 5 Whys Method to Get to the Bottom of Your Problems
Using the 5 Whys Method to Get to the Bottom of Your Problems

When something goes wrong, people always want to know why. Why did this happen? Why did this go wrong? I...

Next Article
October Product Update: Focus on What’s Important with New Alerts
October Product Update: Focus on What’s Important with New Alerts

When you see a column of bright red flames running alongside your project plans, it can be a bit unsettling...