The Impact of Big Data Analytics Across Industries

Big data has long evolved from being confined to IT sectors to becoming a business imperative. In 2018, the International Data Corporation (IDC) forecasted that global revenue for big data and business analytics solutions would reach $60 billion in 2022 with a compound annual growth rate of 11.9% from 2017 to 2022. However, the IDC’s latest Spending Guide placed that figure at $215.7 billion in 2021. 

As companies continue to find new ways to better leverage the massive amounts of data being collected every moment to enable solutions and retain a competitive edge, we take a look at several case studies of how big data is applied in five different industries.  

 

Human Resources: Driving business performance via people analytics 

 

A McKinsey case study details a major restaurant chain with thousands of outlets around the world looking to improve customer satisfaction and grow revenue. Business leaders believe could be done by solving the company’s high staff turnover problem by better understanding people. 

New and existing data were collected from individuals, shifts, and restaurants across the US market including the financial and operational performance of each outlet. Some points considered include personality traits of employees, day-to-day management practices, as well as staff interactions and behaviors.  

The more than 10,000 data points were used to build a series of models to determine the relationship, if any, between the desired outcomes and drivers. The model was used to test over 100 hypotheses, many of which were posited by senior management based on their own observations and instincts from years of experience. 

Noting that some of the hypotheses were proven while others were disproven, McKinsey reported: “This part of the exercise proved to be especially powerful, confronting senior individuals with evidence that in some cases contradicted deeply held and often conflicting instincts about what drives success.” 

Ultimately, the analysis revealed four insights that have gone on to inform the company’s day-to-day people management in its pilot market.  

Just four months in, the company experienced: 

  • Over 100% increase in customer satisfaction scores 
  • 30 seconds improvement in speed of service  
  • Decrease in attrition for new joiners 
  • 5% increase in sales  
 

Supply Chain: Improving cost and service efficiency 

 

A multi-location manufacturer sought to mine its vast library of inventory, shipping, and freight billing data to find ways to improve spending while maintaining service levels. They also wanted to identify opportunities for better inventory management, trip reductions, and order consolidation.  

Using available data, the solution provider created an integrated data management and analytics platform. This was supplemented by a custom order management algorithm.  

The system helped the company consolidate orders heading out to the same location in order to ship them out in one go, thereby reducing congestion at the shipping dock and reducing freight costs by 25%.   

Predictive analysis applied to the company’s supply chain management also led to: 

  • 10% increase in shipping capacity 
  • Improved service-level metrics 
  • 10% decline in inventory levels  
  • Less shipment backlog during peak seasons 
  • Clarity on freight spend drivers 
 

Healthcare: Effective screening and treatment of diseases 

 

In China, there has been a rise in cerebrovascular diseases such as strokes. In response, the government launched a Healthy China 2020 plan aimed at improving public health. 

Following that, medical professionals investigated how best to treat strokes and related medical conditions by identifying three key areas: accurate screenings, precise treatments, and meticulous rehabilitation.  

They wanted a more effective way to analyze data than just using the traditional manual paperwork system, which was not scalable.  

Partnering with IBM, the Shanghai Changjang Science and Technology Department along with China’s top three hospitals developed an intelligent stroke assessment and management platform. The AI-enabled platform analyzes patient information, applies a screening model, and compares these details with known risk factors.  

Patients that have been identified as high-risk are then channeled to the appropriate physician with treatment recommendations and corresponding probabilities of success.  

This application of big data analysis led to: 

  • 15% improvement in diagnostic accuracy of stroke risks in patients 
  • 80.89% accuracy in predicting treatment outcomes 
  • Scaling risk screenings to cover a larger population and encouraging early treatment 
 

Financial Services: Post-trade analysis 

 

The National Bank of Canada’s Global Equity Derivatives Group (GED) provides trading solutions that manage securities such as stocks, futures, funds, and options. It collects and processes a high volume of stock-market financial data, but faces a challenge when it comes to data analysis.  

The bank sought to find a more effective and scalable way to process and analyze structures and unstructured data, as well as historical data, in order to develop a better analytical solution.  

Using an open-source big data processing framework and moving its processes to the cloud allowed the bank to achieve its goal of scalability. The GED was able to analyze hundreds of terabytes of trade and historical data. This now enables their business analysts to conduct quicker post-trade analysis.  

Big data analysis allowed the bank to: 

  • Reduce post-trade analysis process from a few weeks to a few hours 
  • More robust post-trade analysis 
  • Improved trading operations 
  • Increase revenue 
  • Increased customer satisfaction 
 

Manufacturing: Predicting Equipment Anomalies 

 

A major manufacturing company looked to deploy digital twin technology to make manufacturing more flexible and efficient. The company, which was struggling to meet its production targets due to unscheduled downtime, created an IoT sensor-enabled digital copy of its critical equipment to predict potential anomalies and maintain the flow of its assembly lines. 

Falling short of its production target also meant that the company faced increased operating costs, customer dissatisfaction, and lost market share to its competitors.  

Applying IoT-supported digital twins technology allowed the company to collect real-time data. When analyzed with other data sets – historical and maintenance-related – the company was able to remotely monitor and assess its physical assets.  

The ML-based algorithm sifted through plenty of data to help detect abnormal equipment behavior and proactively suggested corrective actions before failure. This led to: 

  • 100% achievement of production target 
  • 25% reduction in operation costs 
  • 54% increase in profit margins 
  • Timely product delivery 
  • Higher customer satisfaction and increased market share 

No Business without IT

Wanting to implement innovations quickly, companies often develop digital process inside the different departments without the adequate involvement of IT. This leads to isolated solutions within the organization. But solutions can only deliver true added value for the entire company if they can be scaled and integrated with each other.

The digital transformation has reached the company. However, departments often introduce SaaS-based applications on their own, or they develop their own solutions. This leads to the uncontrolled growth of incompatible systems.

 

Examples of isolated solutions in a production operation

  1. The purchasing department has a platform for supplier management that enables digital purchase orders. The specifications and volumes, on the other hand, are e-mailed to the production department and must then be entered manually.
  2. The sales department uses an independently developed web portal that provides customers with 3D models of products, which can then be customized. However, the product information is manually entered into the tool since the interface to the product information management system does not work properly.
  3. Production uses a manufacturing execution system that digitally displays the various production steps. But forecasts about production capacity, the finishing of individual products and defective products must be determined manually by analysis and provided to other departments as Excel reports.
  4. The development department uses a CAD system that sends drawings directly to the various machines. But the department is missing the experience data for cost-effective and reliable materials from production and purchasing.

This means that manual interfaces are required to exchange data, but they also represent a potential source of error. Therefore the systems should be integrated across the departments to warrant complete data integrity and availability.

In the case of a production company, this would allow customers to modify their product during the production process, actually see the progress and track the shipment in the web portal. The supplier management tool automatically receives data on current purchase orders and inventories. Supplier orders are automatically adjusted based on forecasts for product demand. And the development department always has access to current prices and production experience. The result: the cost-effective and efficient series production of individual products.

The role of IT in the digitization process: from service provider and enabler to driver

As a result, digitization requires a holistic approach for companies, their value chains and in particular their IT organizations. But IT must also accept and be allowed to practice this new role. In practice, it often finds itself trying to balance the requirements for rapid, efficient, agile, scalable and innovative digitization in the company with the growing IT independence of the various departments. These often view the IT organization as a hindrance, inflexible or old-fashioned. And so they go ahead and do their own thing – using Cloud services or external developers.

But it is exactly these types of isolated solutions that frequently lead to rising administrative expenses, more complexity and not least increased security risks since the existing governance requirements and guidelines do not cover these cases. The result:

Therefore the IT department (whether the other departments like or not) must strictly control the use of customized solutions and approaches during the digitization process. But by doing so, it cannot act primarily as a hindrance, but rather as the keeper and enabler of new business models. The IT department has several trump cards over its colleagues in the other departments: It focuses on what is good for the entire company and it has the flexibility to pro-actively find the required service providers. In addition, it can organize or manage tenders to negotiate the best terms with external providers.

The new IT organization: DevOps, agility and business partnering

To complete this transformation, IT departments must develop and pave the way for the future particularly with regard to the IT organization, business centricity and technology. They must assume responsibility for the scalability of the new digital solutions. In addition, they must ensure that processes are fully thought through, developed and automated, and that they can be integrated into the overall organization in a flexible and (if needed) agile manner.

This means: The processes in the IT department are increasingly changing in the direction of an agile collaboration with departments. Moreover, the IT team increasingly assumes advisory and managing functions. To this end, it must push for the following:

  1. recruit employees with the right skills, who understand agile methods and carry them into the company
  2. make data-based decisions on the basis of Data Analytics and prevent incorrect decisions due to a lack of skill or information
  3. despite higher levels of security, reduce the complexity in the operation while remaining flexible to reduce the required amount of time and resources
  4. lower IT costs with transparent IT controlling and service management to remain competitive
  5. develop and implement an IT sourcing strategy to speed up the process of finding the right service providers and concentrate on processes that differentiate the company from the competition

In addition, interdisciplinary DevOps Teams in the IT department also help to increase software quality and availability and therefore customer satisfaction. Business Centricity must also be improved (or introduced), and a detailed understanding must be created for the company’s business processes and value creation. To this end, the IT department must develop joint solutions with the other departments.

This can be done with:

1. Professional Partner/Business Centricity: IT developments are often based on the viewpoint of management. Therefore IT must see itself as a partner in the development of joint ideas and solutions to ensure rapid and flexible compliance with business requirements and to guarantee competitiveness.

2. Co-Innovation: Co-Innovation: Innovations are led and promoted jointly by IT and the departments. Particularly in agile environments, this means that IT experts work in teams with staff from other departments so that the steadily changing requirements can be met as much as possible and the investment expenditures are fairly distributed over the participating departments.

3. IT Service Management: The coordination between the service provider and the service recipient is intensified. Here too, it is all about promoting and practicing small and flexible partnerships between IT and its internal clients, so that technological changes and opportunities can be tailored to the requirements of the business. This provides a high degree of automation for standard business processes.  amirite?!

On the whole, it means that the existing employees in the IT department must increase their knowledge and skills for supporting business processes because company-wide innovations require IT departments to take a close look at the new requirements and the associated fundamental technological developments, which promise a lot of added value for all departments today.

They primarily include:

1. Cloud Computing such as IaaS, PaaS or SaaS to address the need for flexibility, efficiency, productivity and scalability, and to facilitate the collaboration of the teams.

2. Platforms for improving and accelerating processes to reduce manual error sources.

3. Analyses of Big Data (Analytics), so that relevant information from customer data, deliveries, orders, transactions, product details or manufacturer information can be extracted in a very short time (e.g. to improve the pricing process).

4. The IT-Security, which must always be scrutinized so that the required security concepts can be developed for the current company-specific structures and requirements, which also provide the requisite protection for critical business data in the digitization age.

It is only by including these issues that IT has the flexibility to meet the requirements of the departments, while also guaranteeing the company’s security and efficiency.

Conclusion

The idea of aligning the IT department to the digital transformation of the entire company sounds pretty simple in theory: It “only” has to make its own organization agile, maintain a strong connection to the business, focus on the main technological trends and actively promote these to management and the various departments.

However, this fundamentally changes the conduct of IT as a service provider and requires significant restructuring of the IT organization and its resources. In practice, the IT department must also become an enabler for employees.

At the same time, it must manage on-going operations and provide sufficient resources for modernization and the introduction of new approaches and technologies.

Therefore, in order to achieve an optimum and successful digitization process, IT departments should be considerably strengthened with regard to their skills and impact – or they should take the initiative in this regard. An IT transformation is not possible without a mature and enabled IT department that is viewed as a partner by the business. And without an IT transformation, there can be no long-term and sustainable company-wide digital transformation.

Mark Gallagher: Driving The Future Towards High-Performance Through Big Data

Mark_Gallagher_Driving_The_Future_Big_Data

The future of data-driven organizations has arrived and spearheading businesses towards operational excellence is the vision that Mark Gallagher, the founder and CEO of Performance Insights and Industry Analyst at Formula One, continues to advocate.

As organizations start to adopt more data-driven strategies, Gallagher shares with us the challenges and solutions in which Big Data presents, the opportunities in which disruptive technologies can provide in tandem with data and analytics, and the future it holds for businesses and beyond.

Mark_Gallagher_Driving_High_Performance_Through_Big_Data


The Challenges and Solutions of Data-Driven High Performance

Big Data and analytics have quickly become the key ingredient that businesses need to integrate to remain as a high-performing and agile organization in today’s modern industry. Nevertheless, there are challenges that businesses have to overcome before being able to transform into a data-driven organization.

One such challenge that Gallagher notes is the need for organizations to understand and find which data is most relevant to unlocking new opportunities and not rely on established systems.

We may wish to gather data from the areas where we have some understanding,” notes Gallagher. “However, the real opportunity comes from questioning established systems and processes and examing data around the unknowns.

While finding and utilizing data effectively is still a major challenge for most organizations, Gallagher believes the solution lies in organizations finding the right partners and using the right emerging technology to help improve performances.

It is vital to work with the right partners to develop systems that can make rapid use of data”, Gallagher points out. “Real-time data encourages and facilitates real-time decision making, and this is where the power of AI kicks in.

In the world of Formula One, Gallagher found that both the quality and speed of decision-making have improved dramatically with the help of partners and AI to help understand and utilize data. This enabled Gallagher to guarantee much higher levels of quality, reliability, risk management and performance, allowing them to “avoid negative outcomes and guarantee more positive ones.

Utilizing The Power of Disruptive Technology

On its own, Big Data has proven to be a disruptive technology. However, Gallagher believes that several emerging technologies can be “game-changers” for the traditional business processes.

The opportunities afforded by AI and Blockchain technology are only just being realized, and far from dehumanizing businesses”. Gallagher continues, “these tools will enable more people and organizations to work together seamlessly to drive improved outcomes for their customers, businesses and supply chains.

The benefits of emerging technologies go beyond organizational efficiency and Gallagher points out how Internet of Things (IoT) and artificial intelligence have helped build a more connected and data-driven environment in Formula One.

We operate a fully connected environment so that we can manage our assets remotely, monitoring performance, quality, gathering diagnostic information and ultimate managing the product life cycle better than ever”, Gallagher remarked on the usage of IoT and artificial intelligence platforms.

Gallagher sees the innovation that IoT and artificial intelligence brings to Formula One, providing information to make better use of their resources and dramatically improve their manufacturing systems. “In creating a digital twin of our product, we have moved to an environment where we can manufacture and manage much more efficiently.

The Big Future of Big Data and Analytics

Focusing on the data that matters should be the priority for organizations, and as vast amounts of data become increasingly available, Gallagher and Formula One needs to work with solutions that cut to the core of the issues and opportunities that are affecting businesses.

Gallagher points out how Big Data and analytics can be utilized in new ways for businesses and society as a whole in the future, noting that in “a data-rich world, we can mine more opportunities to add value.

Beyond the profit margins, Big Data has the opportunity to develop innovative solutions and Gallagher shares this enthusiasm saying that he is ”very optimistic that many of the problems facing the world today will find their solutions in technology that develops as the result of having the data to understand issues properly.

At the end of the day, data is just information and when businesses can access a better quality of information, they can expect to improve outcomes across all areas of operations.