As digital transformation takes hold across all industries, company leaders must dramatically shift their thinking. They are leaving a business world of constraints and entering an Age of Abundance: of digital data and the technology tools to create innovations from that data.
Every large company today is swimming in a rising tide of digital data. That data resides in their information systems. But it also resides in the ecosystem that surrounds them – the data of suppliers, distribution channel partners, and others with which they do business.
The sources and types of data are rising at an exponential pace. Some 90% of the world’s digital data today has been created in the last two years alone. By itself, the U.S. produces an average of more than 2.6 million gigabytes of Internet data every minute, a number that figures to increase as the world of connected devices grows.1 Soon, people will be routinely talking about “zettabytes” – 1 trillion gigabytes of digital data – a concept that would have been unfathomable only a few years ago.
As the world’s population surges, data will not only change companies and whole industries, it will be the key to addressing societal issues like managing traffic, growing food, and feeding countries. Indeed, IT researcher International Data Corp. (IDC) predicts that by the end of this year, half of large enterprises will be selling the digital data they generate in their businesses, as well as metrics, insights, and recommendations on what to do with it.2
For a case study in leveraging data effectively, consider ABB, the $34.3 billion industrial firm that services power plants and automates industrial processes. The company credits its ABB Ability platform of services3 with driving an 11% increase in orders for its software and services in 2017.4 The platform collects real-time data, analyzes and monitors conditions at industrial facilities (such as factories, oil wells, and power plants) to reduce operating costs, and improve safety and maintenance. ABB estimates the platform offers the potential for $20 billion in annual sales in the future.5
While those prospects may be tantalizing, other research indicates most enterprises are neither getting enough from their current data nor preparing themselves to tap into the new data that will soon be available.6
There is plenty of technology and expertise today to help companies capitalize on their digital data. But most of it goes untapped because there’s a shortage of something else: imagination. In short, too many companies lack the capabilities of generating insights from their data — i.e., of creating profound new observations about what customers want, why they buy, what they like and don’t like about a company’s current offerings and the purchase and post-purchase experiences they provide with their offerings, and much more.
When companies are data-rich but insight-poor, innovation suffers. Ideas for new products, services and business processes are episodic and often off-target.
Widespread Weakness: A Lack of Data Maturity
The root of this problem is clear to us. These companies lack strong capabilities in analyzing data.
In a survey of 196 organizations around the world, Gartner found that 91% had not yet reached a “transformational” level of maturity in data and analytics, despite this being the top investment priority for CIOs in recent years.7 Indeed, 60% of those surveyed rated themselves average or below average when it came to data maturity.
What exactly does it mean to lack maturity in data analysis? People throughout an organization don’t share data. Companies also don’t take full advantage of the data that business partners collect.
What’s more, companies with immature analytics capabilities operate in “data fiefdoms” where pockets of insight-generating analytics benefit only one part of the organization.
The experience of Monsanto, the $14.6 billion agricultural and biotech company, illustrates the benefits of sharing data and expertise across functional boundaries. To increase revenue and improve profitability, Monsanto launched a cloud analytics platform it calls “science@scale.” The platform enables the company’s data experts and scientists to collaborate, and it has accelerated the development of analytics-driven decision models, says Monsanto CIO James Swanson.8 And because the platform enables scientific work to be done in hours rather than months, the company estimates it has saved $15 million in operating costs and boosted revenue by $17 million.
Becoming Data and Insight Rich Begins at the Top
The minority of companies today that generate game-changing insights from their digital data operate quite differently than the rest. That difference begins at the top of the organization, with a leadership team that worships digital data. We call this having a data-centric statement of purpose. Leaders believe analytics and data are the foundation of their company’s digital transformation, and that they are the source of superior customer experiences and superior new product/service offerings.
Leaders then must determine exactly how the insights from both internal and external data can create new opportunities. They need to first assess their organization’s data maturity (existing data inventory, what the company should capture and analyze), and then map out how to reach the target state.
This must include determining how valuable insights can be shared across the organization, which often requires eliminating cultural barriers that prevent such sharing.
Three Sources of Data to Harvest
In our work, we found three sources of digital data that many companies don’t fully tap:
Existing, siloed data
Companies often have data that provides tremendous value to one business function such as marketing, but is not shared with other functions that could gain similar value from it. This data must be made available in digital form to be analyzed by anyone within the organization. This is easier than it sounds. Disconnected data in U.S. and U.K. companies create waste, duplications of efforts and missed opportunities, according to a survey of 500 IT decision makers and business application users by market research firm Vanson Bourne.9 Nearly three-quarters (72%) believe their organization is missing out on opportunities because of data silos. More than half (56%) feel those silos are obstacles to meeting business objectives while 47% believe disconnected data hampers innovation.
Missing internal data
Companies capture only a small percentage of the data from their operations—data that is within their control. Companies that are insight-rich typically collect data from their operations that competitors aren’t capturing in their own businesses. Further, many organizations outsource some of their business operations to third party vendors. During such operations, the service providers collect large volumes of data. However, vendors may not have a mechanism to make data effectively available to their partner organizations to take informed business decisions. In addition, these partners may not have advanced technologies to collect all relevant data points that have a bearing on customer experience.
For instance, airline companies generally outsource the wheelchair operations to service providers, who may not use Internet of Things (IoT) or mobility solutions to manage the operations on a real-time basis. As a result, airline companies may potentially fail to collect and analyze valuable data points, which otherwise would have helped them positively address their customer experience initiatives or operational delays.
This principle is applicable across all major industries. Take for example, the oil and gas industry. In a hazardous environment such as an oil exploration site, improving personnel and equipment safety is paramount. Therefore, it is crucial to collect data that helps industry players improve processes, makes more-informed business decisions, and manages safety, maintenance, and costs. Frequency of data collection will also play a crucial role in meeting the business objectives.
New kinds of external data
The data marketplace provides all sorts of information that, properly used, can create tremendous value. The growing data brokerage industry, which collects and markets data on consumers’ habits, is expected to reach the $250 billion mark in 2018.10
For example, ride-share and taxi services in India can identify the restaurants and establishments where people eat. That data could be sold to other restaurants and businesses that want these people to buy their products and services. The offers, based on insights about customers’ dining histories and locations, create opportunities to collaborate in an urban area digital ecosystem, one that could enhance customer experiences and expand sales.
New data can come from unexpected places—like washing machines. Consider WASH Multifamily Laundry Systems, an El Segundo, Calif.-based laundry facilities management service provider with 75,000 locations in the U.S. and Canada. It collects a vast amount of data from its extensive network of hundreds of thousands of interconnected washer, dryers, vending machines, and payment systems. Working with apartmentbuilding owners, WASH uses this data to model and test manager’s questions, such as whether it is cost-effective to switch from cash to payment cards, before committing to changes.11
New types of public-private partnerships are evolving in this age of abundant data. The Dutch capital of Amsterdam uses GPS data from mobile and navigation devices, gathered by a private company, to create models to study traffic issues in the city.12 The possibilities that come from having almost limitless access to information are often just the constraints of imagination and will.
It is crucial to collect data that helps industry players improve processes, makes more-informed business decisions, and manages safety, maintenance, and costs.
Making Data Easier to Use
Digital data is not the only product that is abundant today. So are people who can make sense of the data. This may sound counterintuitive, given an acute shortage of data scientists, statisticians and other professionals sought by companies near and far to analyze “Big Data.”
Companies today have numerous people without PhDs in statistics or data science but who have deep familiarity with customers and business processes. In many enterprises, they are an underutilized resource in generating insights from data.
Today, analytics are largely dependent on having people with advanced analysis capabilities to manage data, create algorithms, and take other steps to turn data into insight. While this approach could work in the past when companies had smaller data sets and the pace of business was slower, it is inadequate at a time when data is abundant.
To begin with, data is growing so fast that hiring and training enough experts to keep up with the onslaught of information is impossible. Depending on a core of analytics experts creates a bottleneck. It prevents an organization from scaling its use of data.
Instead, companies must embrace the democratization of analytics, where data tools are used by people with varying degrees of analytics expertise at all levels of the organization. It requires new technologies that make analytics simpler such as machine learning and new programming languages like Python or quantum computing.
Quantum computing has been the talk of the industry for some time now. Unlike current systems, quantum computers are able to complete complex tasks at a fraction of the time consumed by contemporary processing systems. This opens up new opportunities in terms of real-time data processing.
Consider a wealth management firm. Financial advisors rely on research to design investment plans for their high-net-worth clients. As the process generally involves complex calculations, traditional data mining solutions are used. However, lack of data processing in real-time could potentially affect the revenue and hence the performance of the financial firm. In such scenarios, the firm can use quantum computing to process and analyze both structured and unstructured data to improve the client experience by, for example, providing both personalized service and rapid responses to customer questions about investments. This capability makes the firm more agile, better able to respond to market conditions, and drive business growth.
Farmers Insurance is another example. The insurance firm is using an AI platform called DataRobot to bring machine-learning techniques to a team that analyzes customer behavior and examines the designs of different insurance policies.13
The second improvement is around data interfaces. Just as analytics technologies must be easier to use, the democratization of analytics hinges on making the use of data simpler and more intuitive. For example, organizations could create models where people talk to an AI-based system like Amazon’s Alexa, in which they verbally ask for and receive information they need.
New data presentation tools, such as mixed reality and augmented reality (AR), can also help. They present data in an intuitive, visual manner. This is already happening in many industries. Logistics company DHL tested smart glasses and AR in a warehouse in the Netherlands. Workers guided through the warehouse by graphics displayed on the smart glass were able to speed the picking process and reduce errors, boosting efficiency by 25%.14
Lowes, the large home improvement retail chain in the U.S., is experimenting with VR headsets to help their customers visualize furnishings for a kitchen remodel.15
ThyssenKrupp, an elevator manufacturer, is using Microsoft’s HoloLens to visualize an elevator repair before the technician reaches the site. Onsite, technicians can use augmented reality to view digital overlays of manuals and repair guides while they fix the elevator.16
This type of technology is extending to a vast array of purposes and industries. Ford is using virtual reality (VR) to create virtual models of cars and enable designers to better collaborate on design changes. The technology reduces the need to build physical prototypes and allows the engineers to more easily explore creative designs.17 Cosmetics companies are testing their products by creating “digital twins” of animal skin, so they can see the reaction to test products in a virtual environment without harming animals.
As these examples attests, the pathway to the Age of Abundance depends on subtraction and addition: removing human beings from the data preparation process, while augmenting human beings so they can more effectively use that data.
Overcoming the Barriers to Generating Game-Changing Insights
Every company today needs to determine how it will gain crucial insights from the vast and growing amount of digital data it can tap. But the biggest barriers are not technological ones. In fact, they are primarily cultural.
Liberating data so it can be effectively harnessed is largely a changemanagement exercise that must account for the many ways, and many reasons, that data is not widely shared and used. We have found five cultural barriers and one technology management hurdle to leveraging data in many organizations.
Data Held Captive
Departments can be reluctant to relinquish it for a number of reasons. They are concerned that the information can be misused or tampered with. The data sharing might also reveal mistakes, such as a bank ignoring some regulatory rules to onboard customers faster. The data sharing can also erase competitive advantages one business line may have over others that compete for the same customers.
Too Many Data Choices Companies have the potential to use vast amount of missing and external data, as new sources of this data continue to emerge. But it can be difficult to determine exactly which data will provide the most value.
To fully make use of available data, companies need to ensure they have proper consent, following both privacy regulations and respecting consumer boundaries.
Poor Data Sharing Processes
Some companies make the process of sharing data overly laborious and time-consuming. For example, some banks, even in developed nations, require digital data to be printed out, scanned and re-entered, which makes it difficult for employees to use the data in a timely way.
Lack of Direction from the Top
Accessing abundant data resources depends on an over-arching perspective that can only be shaped through top-down direction. Too often, this direction is missing, which reduces cross-functional collaboration and creates data fiefdoms.
The technology management hurdle relates to cost optimization. While the hyperbolic growth of data represents a great opportunity for companies to gain new insights, the abundance of data also imposes a heavy burden on computation, storage, and communication in data centers. Task assignment, data placement and data movement further influence the operational expenditure for business. It is therefore important for leaders to be aware of this tension—and to identify means for optimizing the benefits while making the best use of resources to minimize costs.
These barriers are not insurmountable. But to overcome them, company leaders must mandate that data be transparent and shared across the organization.
Leaders today need to make the Age of Abundance a clarion call. Those who do will take an important first step to helping their organization liberate data from its silos and unleash their workforce to gain crucial new insights without being overly reliant on experts.
- 1 IFL Science, July 26, 2017, accessed at, http://www.iflscience.com/technology/how-much-data-does-the-world-generate-every-minute/
- 2 IDC FutureScape, Worldwide Analytics and Information Management 2018 Predictions, Nov. 15, 2017
- 3 ABB Ability, accessed at , http://new.abb.com/abb-ability
- 4 ABB quarterly results press release, October 26, 2017, accessed at , http://new.abb.com/docs/default-source/investor-center-docs/annual-report/ annual-report-2017/abb-group-annual-report-2017-english.pdf
- 5 ABB Ability, accessed at: , http://new.abb.com/abb-ability
- 6 Gartner press release, “Gartner Survey Shows 42% of CEOs Have Begun Digital Business Transformation,” April 24, 2017, accessed at , https://www.gartner.com/newsroom/id/3689017
- 7 “Gartner Survey Shows Organizations Are Slow to Advance in Data and Analytics,” press release, February 5, 2018, accessed at, https://www.gartner.com/newsroom/id/3851963
- 8 Monsanto’s CIO Develops Data Science Platform of the Future,” Forbes.com, October 23, 2017, accessed at , https://www.forbes. com/sites/peterhigh/2017/10/23/monsantos-cio-develops-data-science-platform-of-the-future/
- 9 Vanson Bourne, “The high cost of disconnected data,” published by Snaplogic, 2017,
- 10 New York Times, “It’s Time to Tax Companies for Using Our Personal Data,” November 14, 2017, accessed at , https://www.nytimes.com/2017/11/14/business/dealbook/taxing-companies-for-using-our-personal-data.html
- 11 MIT Sloan Management Review, “Data Sharing and Analytics Drive Success with IoT,” September 8, 2016, accessed at, https://sloanreview.mit.edu/projects/data-sharing-and-analytics-drive-success-with-internet-of-things/
- 12 MIT Sloan Management Review, “Data Sharing and Analytics Drive Success with IoT,” September 8, 2016, accessed at, https://sloanreview.mit.edu/projects/data-sharing-and-analytics-drive-success-with-internet-of-things/
- 13 MIT Technology Review, “You Could Become an AI Master Before You Know It. Here’s How,” October 17, 2017, accessed at , https://www.technologyreview.com/s/608921/you-could-become-an-ai-master-before-you-know-it-heres-how/.
- 14 DHL press release, “ DHL successfully tests Augmented Reality application in warehouse,” January 26, 2015, accessed at , http:// www.dhl.com/en/press/releases/releases_2015/logistics/dhl_successfully_tests_augmented_reality_application_in_warehouse.html
- 15 Forbes, “Augmented and Virtual Reality Fuel the Future Workplace,” November 11, 2016, accessed at , https://www.forbes.com/sites/maribellopez/2016/11/11/augmented-and-virtual-reality-fuel-the-future-workplace/
- 16 ThyssenKrupp, “Bringing new vision to elevator maintenance with Microsoft HoloLens,” YouTube video, September 15, 2016, accessed at , https://www.youtube.com/watch?v=8OWhGiyR4Ns
- 17 Fortune, “How Ford goes further with virtual reality,” September 23, 2015, accessed at , http://fortune.com/2015/09/23/fordvirtual-reality/