Uncertain Times Call for Real-Time Distributed Analytics

2020 has demonstrated that real-time analytics is business-critical. Enterprises demand even more agility, efficiency, and resiliency of their operations, their employees, and their decision-making in a post-COVID world. Mark Cusack, CTO at Yellowbrick, explains how better analytics makes it possible.

It’s forgivably difficult to look to the future while we’re living within the day-to-day awfulness of this pandemic. But look to the future we must, and it is critically important that we take forward the lessons we’ve learned over the past year and apply them to a post-COVID world. So, what have enterprises learned, and what will they do differently in the future to better cope with massive disruption? 

This year, enterprises have had their ability to do business tested to the extreme. Companies that believe we’ll return to “business as usual” are sorely mistaken. Even companies far along the digital transformation path experienced unprecedented challenges as cracks appeared in their operations. The need for agility from the top to the bottom of the business has never been more apparent, particularly in IT operations. 

Let’s look at three seismic changes that occurred this year:

1. Buying habits changed overnight. During the initial lockdowns in the spring, we experienced a run on consumer products that wasn’t predicted. This placed a huge strain on supply chains and inventory management for many retailers. Existing demand forecasting models had to be thrown out of the window.
2. Working and learning from home and video conferencing became the norm. Offices and schools shut down, placing a latency and bandwidth strain on corporate networks and cloud-based IT services.
3. CIOs rapidly reviewed and modified their plans to transition technology stacks to the public cloud. Business continuity became top-of-mind, and hybrid cloud – a blend of on-premises and public cloud operations – became the advocated approach.
   

Changes such as these have forced a rethink in IT strategy, particularly around data and analytics and the cloud, which will lead to new modes of operation in the future. 

Learn More: 3 Ways To Build an Automation Program That Offers Lasting Value

Predicting Real-Time Customer Behavior Will Become the New Normal 

The inability to forecast fast-changing customer demand signifies that we need to take a fresh approach to understand buyer behavior. Enterprises need to invest more in AI/ML-augmented decision making and real-time anomaly detection, combined with automated processes that drive demand down the supply chain instantly. Companies will need to break down the data silos from across their organizations to feed these new models.  

Data warehouses have been traditionally used for business intelligence reporting and decision-making, and they typically hold the most valuable data in a company’s estate. This industry is being disrupted, and the role of data warehousing is expanding to be a platform not just for descriptive analytics to be consumed by business analysts but also a tool to support the predictive and prescriptive analytics developed by data scientists. 

The new breed of data warehouses can process real-time data and give answers and recommendations in the moment. They are seamlessly elastic in their compute and storage capabilities and are able to support down-stream predictive modeling applications at any scale. 

The most forward-looking enterprises are merging their data lakes and data warehouses, complementing cheap, ubiquitous storage from the former with the low-latency SQL analytics from the latter to break down data silos and facilitate real-time analytics and modeling of all aspects of their business operations. 

Learn More: A Case for Modernizing and Optimizing Legacy Platforms 

Cloud-Edge Applications: The New Normal 

The pressure placed on networks from the distributed nature of working during this crisis is leading leaders to rethink where key services are placed geographically and logically. Enterprises are considering whether they need to distribute or replicate their key applications across different clouds and across different locations to ensure that the bandwidth and latency requirements are met and their employees remain as productive as possible.  

The data and analytics platforms that support these applications are naturally following suit. Increasingly, data warehouses and data lakes are being deployed in the public cloud in regional data centers to be closer to their end users. 

We were already seeing a move to the public cloud, driven by changing data gravity patterns and the perception of lower costs compared to running in your own data center. However, the need to place analytic applications closer to consumers is accelerating the process. 

Learn More: The Benefits of Serverless Computing for B2B Businesses 

Rise of Hybrid Cloud Deployment 

Before COVID-19, many enterprises were formulating plans to move their data and analytics activities to the public cloud as part of a “cloud-first” mandate. However, we’ve seen a revision of this all-in strategy emerge this year, partly due to several high-profile public cloud outages and security breaches. With an eye on business continuity and security, CIOs are thinking about minimizing risks by favoring a hybrid approach, where operations span their own data center as well as the public cloud.   

To meet this revised need, the most progressive data warehouse vendors are providing both on-premises and public cloud deployment Opens a new window options, with the same guaranteed levels of performance and scale everywhere. They also provide geo-replication and disaster recovery capabilities that allow enterprises to quickly failover from on-premises to cloud systems or vice-versa. 

From a security perspective, these vendors offer encryption of data at rest and on the move, whether in the data center or in the public cloud, or within a hybrid mix of both. The need to provide an always-on analytics service to help run the business in uncertain times is becoming critical.  

In summary, enterprises are seeking ways to increase the agility, efficiency, and resiliency of their operations, their employees, and their decision-making in a post-COVID world. We can see an accelerating shift towards distributed computing and cloud as the underlying technology paradigms to enable these objectives. 

Nowhere is this more evident than in data warehousing, where the ability to scale out to provide the performance necessary to power real-time business decisions is critical. The freedom to choose where to deploy a data warehouse within a hybrid context is also key – the need to be close to the data and close to the consumer has never been more apparent. 

Did you enjoy reading this article? Let us know your thoughts on LinkedInOpens a new window , TwitterOpens a new window , or FacebookOpens a new window . We would love to hear from you!