THE ERA OF "BIG DATA" — the next frontier for innovation, competition, and productivity

The amount of data in our world has been exploding. Analyzing large data sets, so-called big data, will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus. Here’s what you should know.

“Big data” refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze.

Companies are churning out a rapidly increasing volume of transactional data, capturing trillions of bytes of information about their customers, suppliers, and operations. One has only to shop on Amazon and notice the recommendations the web site offers, which are based on users' previous shopping history and searches, for an indication of big data that Amazon collects and analyzes from its millions of site visitors. Millions of networked sensors are being embedded in the physical world in devices such as mobile phones, smart energy meters, automobiles, and industrial machines that sense, create, and communicate data in the age of the Internet of Things.

Indeed, as companies and organizations go about their business and interact with individuals, they are generating a tremendous amount of digital “exhaust data.” Social media sites, smartphones, and other consumer devices including PCs and laptops have allowed billions of individuals around the world to contribute to the amount of big data available. And the growing volume of multimedia content has played a major role in the exponential growth of the amount of big data. Each second of high-definition video, for example, generates more than 2,000 times as many bytes as required to store a single page of text. In a digitized world, consumers going about their day—communicating, browsing, buying, sharing, searching—create their own enormous trails of data.

Fueling a growing torrent…

• $600 to buy a disk drive that can store all of the world’s music
• 5 billion mobile phones in use in 2010
• 30 billion pieces of content shared on Facebook every month
• 40% projected growth in global data generate per year vs. 5% growth in global IT spending
• 235 terabytes of data collected by the US Library of Congress in April 2011
• 15 out of 17 sectors in the US have more data stored per company than the US Library of Congress

Every day, we create 2.5 quintillion bytes of data–so much that 90% of the data in the world today has been created in the last two years alone, according to IBM. This data comes from everywhere: from sensors used to gather climate information, posts to social media sites, digital pictures and videos posted online, transaction records of online purchases, and from cell phone GPS signals to name a few.

In itself, the sheer volume of data is a global phenomenon, but what does it mean? Many people around the world regard this collection of information with deep suspicion, seeing the data flood as nothing more than an intrusion of their privacy. But there is strong evidence that big data can play a significant economic role to the benefit not only of private commerce, but also of national economies and their citizens. Research by McKinsey Global Institute finds that data can create significant value for the world economy, enhancing the productivity and competitiveness of companies and the public sector and creating substantial economic surplus for consumers.

Digital data is now everywhere, in every sector, economy, organization, and user of digital technology. While this topic might have once concerned a few data geeks, big data is now relevant for leaders across every sector, and consumers of products and services stand to benefit from its application. The ability to store, aggregate, and combine data and then use the results to perform deep analyses have become much more accessible as trends. In computing, its equivalent in digital storage and cloud computing continues to lower costs and other technology barriers. Further, the ability to generate, communicate, share, and access data has been revolutionized by the increasing number of people, devices, and sensors that are now connected by digital networks. Radical customization, constant experimentation, and novel business models will be new hallmarks of competition as companies capture and analyze huge volumes of data.

The take-away:

• Data has swept into every industry and business function and are now an important factor of production, alongside labor and capital.

• There are five broad ways in which using big data can create value.
  1. Big data can unlock significant value by making information transparent and usable at much higher frequency.
  2. As organizations create and store more transactional data in digital form, they can collect more accurate and detailed performance information on everything from product inventories to sick days, and therefore expose variability and boost performance. Leading companies are using data collection and analysis to conduct controlled experiments to make better management decisions; others are using data for basic low-frequency forecasting to high-frequency nowcasting to adjust their business levers just in time.
  3. Big data allows ever-narrower segmentation of customers and therefore much more precisely tailored products or services.
  4. Sophisticated analytics can substantially improve decision making.
  5. Big data can be used to improve the development of the next generation of products and services. For instance, manufacturers are using data obtained from sensors embedded in products to create innovative after-sales service offerings such as proactive maintenance.

• The use of big data will become a key basis of competition and growth for individual firms. From the standpoint of competitiveness and the potential capture of value, all companies need to take big data seriously. In most industries, established competitors and new entrants alike will leverage data-driven strategies to innovate, compete, and capture value from deep and up to real time information.

• The use of big data will underpin new waves of productivity growth and consumer surplus. For example, it’s estimated that a retailer using big data to the full has the potential to increase its operating margin by more than 60 percent. Big data offers considerable benefits to consumers as well as to companies and organizations. For instance, services enabled by personal location data can allow consumers to capture $600 billion in economic surplus.

• While the use of big data will matter across sectors, some sectors are set for greater gains. The computer and electronic products and information sectors, as well as finance and insurance, and government, are poised to gain substantially from the use of big data.

• There will be a shortage of talent necessary for organizations to take advantage of big data. According to McKinsey Global Institute, by 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.

• Several issues will have to be addressed to capture the full potential of big data. Policies related to privacy, security, intellectual property, and even liability will need to be addressed in a big data world. Organizations need not only to put the right talent and technology in place but also structure workflows and incentives to optimize the use of big data. Access to data is critical—companies will increasingly need to integrate information from multiple data sources, often from third parties, and the incentives have to be in place to enable this.

 

Contact Software Consortium or call 1-877-850-9393 if you would like to discuss how to leverage our top-level talent to empower your business.

 

Baltimore Office
Local Phone: 410.583.9393
sales@softwareconsortium.com