Big data and performance management

Big Data refers to the large collections of data that may be analysed to reveal patterns, trends and associations, especially relating to human behaviour and interactions. Big Data has already been explained in another article, entitled ‘Big Data’ (see 'Related links'). This article will describe some real life examples of the use of Big Data for performance management and measurement purposes.

Performance management involves managing the organisation in order to ensure that it meets its objectives. Broadly, Big Data is relevant to performance management in the following ways:

  • Gaining insights (eg about customers’ preferences) which can then be used to improve marketing and sales, thus increasing profits and shareholders’ wealth.
  • Forecasting better (eg customer’s future spending patterns, when machines will need replacing) so that more appropriate decisions can be made.
  • Automating of high level business processes (eg lawyers scanning documents) which can lead to organisations becoming more efficient.
  • Providing more detailed and up to date performance measurement.

This article demonstrates some practical applications of Big Data.

Walmart’s Polaris search engine

Walmart is an American retailer that operates in 28 countries around the world. It is the world’s largest company based on revenues. Many of Walmart’s customers buy online through the company’s website. Walmart wanted to make sure that customers can find what they are looking for on its website, so it developed its Polaris search engine. If customers are looking for a particular product, they enter the description in a search box, and the website displays products which meet that description.

What is unusual about Polaris is the way it ranks the search results. It attempts to show the products that the customer is most likely to buy towards the top of the list. The algorithm takes into account many factors, including the number of likes that the product has on social media networks and how many favourable reviews it has.

The system also uses artificial intelligence to learn so that it can continually provide better search results. If a phrase has been entered that the engine did not initially understand, for example, the engine can ‘learn’ what that phrase meant based on what the customer actually bought. Thus the system was soon able to figure out that when a user entered ‘House’ into the search box, they were probably looking for merchandise connected with the TV series of that name, not furniture or other items for their house. If someone searches for ‘Flats’, the engine has learned that they probably want to buy shoes, not apartments or flat screen TVs.

The metric that is used to measure the success of the website is customer conversion rate – the number of customers that actually buy a product after a search. It is estimated that the Polaris search engine has increased the conversion rate by between 10% and 15%. That is worth billions of dollars in extra revenue.


Beredynamic is a manufacturer of high quality audio products such as microphones and headphones. The company is based in Germany, but has a wide international sales and distribution network. The company wanted to improve its analysis of sales. Most ad hoc reports required data to be extracted from its legacy systems into a spreadsheet where the reports would then be manually compiled. This was time consuming, leading to delays in producing the reports. The reports themselves were not always accurate either.

The company developed a data warehouse that automatically extracts transactions from its existing ERP and financial accounting systems. The structure of this warehouse was carefully designed so that standard information is stored for each transaction such as product codes, country code, customer and region. This is supplemented by a web based reporting solution that enables managers to create their own reports, both standard and ad hoc, based on the data held in the warehouse.  

The system allows the company to perform detailed analysis of sales, which helps it to identify trends in different products or markets. This leads to two business advantages. The first is that the sales and distribution strategy can be changed when demand changes in certain markets – for example, when sales of gaming headphones began to increase in Japan, the company introduced promotions for all its gaming products in that country, including a large advertising campaign and introduction of product bundles specially for the Japanese market. The second advantage is that production plans can quickly be changed as demand changes. If demand is falling, production is slowed to ensure that the company is not left with excessive inventory. If demand is expanding, production is increased to take advantage of higher sales.

The ability to provide more detailed analysis quickly can also be used for performance measurement and appraisal, for example, comparing actual sales with targets by region, assessing whether a promotion achieved the expected increase in profits. Such reports can be produced quickly based on real time data, meaning that management can respond quickly to any adverse variances.

The success of the new system is measured in terms of the growth in revenues and profits. While this seems simple, it has to be recognised that some growth would have been expected even if the system had not been implemented, so determining how much revenue growth has resulted from the greater analysis can be difficult. Assumptions need to be made.


British supermarket group Tesco has operations in several countries around the world. In Ireland, the company developed a system to analyse the temperature of its in-store refrigerators. Sensors were placed in the fridges that measured the temperature every three seconds and sent the information over the internet to a central data warehouse. Analysis of this data allowed the company to identify units that were operating at incorrect temperatures. The company discovered that a number of fridges were operating at temperatures below the -21◦C to -23◦C recommended. This was clearly costing the company in terms of wasted energy. Having this information allowed the company to correct the temperature of the fridges. Given that the company was spending €10 million per year on fridge cooling costs in Ireland, an expected 20% reduction in these costs was a significant saving.

The system also allowed the engineers to monitor the performance of the fridges remotely. When they identified that a particular unit was malfunctioning, they could analyse the problem then visit the store with the right parts and replace them. Previously the fridges would only be fixed when a problem had been discovered by the store manager, which would usually be when the problem had developed into something more major. The engineers would have to visit the store, identify the problem, and then make a second visit to the store with the required parts.

Morton’s Steak House

A customer jokingly tweeted US chain Morton’s and requested that dinner be sent to the Newark airport where he was due to arrive late. Morton’s saw the tweet, realised he was a regular customer, pulled up information on what he typically ordered, figured out which flight he was on and then sent a waiter to meet him at the airport and serve him dinner.

Clearly this action was a publicity stunt which the restaurant hoped that their customer would publicise in future tweets. What it demonstrates is how easy it was for Morton’s to identify the customer who sent the tweet, and to ascertain what his favourite meal was. It also shows how companies like to influence social media users who have a large following as a means of increasing their own publicity.

It is difficult to measure the impact of interventions into social media. No doubt the happy customer would have communicated this story, and this may have improved the reputation of the restaurant, but it is very difficult to measure the impact of this on sales.


The cases above have shown how detailed analysis of data can be used in a number of different ways to improve the performance of an organisation. Big data can be used to understand customers and trends better, to provide insights into costs, and to make it easier for customers to find what they want on the website. Companies are likely to continue to identify innovative uses of the increasing volumes of data available to them, and analysis of Big Data is likely to grow in importance as an important strategic tool for many businesses. 

Nick Ryan is lead tutor for performance management subjects.