In Part I, Venkat Venkataramani, co-founder and CEO, Rockset, explored the evolution of operational intelligence/business observability, from NASA to the cloud to today’s data-driven enterprises. In this blog, he shares examples of companies using operational intelligence in action to protect their mission-critical data-driven operations from current and future problems.
Operational intelligence is a holistic, real-time view of the health of your data-driven business operations and provides predictive insights that can prevent future problems.Â
Companies are increasingly relying on real-time data to guide tactical decisions, as well as autonomically drive mission-critical internal and external processes such as web customer personalization, logistics management, fraud and other anomaly detection, and more. So does their financial health.Â
Data-driven businesses cannot afford operational bottlenecks, much less any downtime. So, they cannot rely on the stale business intelligence produced by batch-based data warehouses.
They need operational intelligence, also known as business observability.
Detecting and Preventing Consumer Fraud
There is a fast-growing buy now, pay later (BNPL) company with 100+ million customers and hundreds of thousands of merchants worldwide. Its online payments service, being its sole revenue source, is mission-critical. Monitoring for problematic payments and failed transactions is critical. If all the Apple Pay payments for a large retailer in Switzerland are suddenly down, that can wipe out hundreds of thousands in lost sales.Â
The company had a cloud data warehouse that crunched transaction data through anomaly detection models every six hours. As the company grew and the number of transactions increased, these anomaly detection jobs started exceeding six hours to complete. The company wanted to upgrade to a real-time monitoring system that could comb through millions of payments per day and trigger instant alerts without a flood of false positives.Â
The company built a business observability system around a real-time analytics database. The database continuously ingests transaction data and provides up to the second accurate real-time metrics across a wide range of dimensions, including merchant, payment method, region, time of day etc.Â
The data is analyzed against statistical models every minute instead of every six hours. The company’s incident response team is notified immediately of serious issues. That team is equipped to analyze both recent and historical payment data to investigate and resolve the issue quickly.
Operational intelligence is saving this BNPL company tens of millions of revenue losses from downtime every year. Even more importantly, it allows them to immediately alert their customer, in this case, that large online retailer, and protect its business too, thereby increasing customer loyalty and decreasing churn.
See More: How Brands Can Keep Pace With Third-Party Data Changes
Operational Intelligence for Supply Chain ManagementÂ
Unsurprisingly, supply chain management could be improved and elevated with the added insight of operational intelligence. Command Alkon, which provides a SaaS logistics service for construction companies and their suppliers, is a good example.Â
The most popular construction material, concrete, is also the most time-sensitive. By law, concrete must be delivered and poured within 60-90 minutes to avoid premature hardening and weakening.
Millions of concrete shipping deliveries a day are tracked by Command Alkon in real-time, giving its customers granular visibility into the status of their shipments, as well as instant alerts if shipments may be delayed. Customers can also start with high-level trend data around shipments and suppliers and slice-and-dice it instantly to dive deeply into trends and problems.
This is essential real-time operational intelligence that Command Alkon shares with customers.Â
Operational Intelligence for Education
Seesaw, a popular K-12 e-learning platform, also built a real-time business observability system that provides operational intelligence to a variety of roles. They include engineers monitoring the platform’s health, salespeople researching usage by school district employees to ensure the satisfaction of Seesaw’s paying customers, and executives looking for visually informed trend data to plan Seesaw’s product roadmap.Â
Seesaw’s system also provides key insights to external users such as teachers and school principals who need to track in real-time how many students are turning in assignments.Â
In other words, operational intelligence improves Seesaw’s operations and customer service.
Build or Buy?Â
All three of the above companies built their business observability systems around a real-time analytics database, choosing this over a cloud data warehouse, NoSQL database, or a pre-built business observability solution.
That’s no surprise. The operational intelligence/business observability category is still in its infancy. Most so-called business observability solutions still focus on specific domains and teams within an enterprise. Even with that narrow view, they are expensive, inflexible and non-interoperable with other enterprise systems.
For instance, the online payments company I mentioned earlier looked seriously at a vertically integrated alerting system. It eventually decided against that pre-packaged solution because of the price and the inability to customize the misfiring anomaly detection engine. “It was a gigantic black box,†the company’s engineering manager told me.
It also reviewed a number of streaming engines and real-time databases, including Apache Flink, Spark Streaming, Amazon Kinesis, Amazon’s new time-series database, Timestream, as well as Apache Druid. It rejected all of them because of the higher management burden and operational costs.Â
In the end, the online payments company chose a real-time analytics platform for a number of compelling reasons: flexibility, ease of management, ultra-fast SQL queries, a decoupled architecture that allows data storage and queries to scale up independently, and many other features.Â
Uber, Meta, AirBnB and others are using operational intelligence to leapfrog competitors. Their executives are like the NFL coaches analyzing in-game trends on tablet dashboards and calling plays with the help of the data analyst on their headset. The legacy companies being disrupted are like the coaches still thumbing through paper playbooks and analyzing game films on Mondays.Â
Intelligently Operational
Transforming your operations with data and cloud technology is not enough if you can’t protect it. As Seesaw, Command Alkon, and other customers can attest, operational intelligence provides real-time visibility and rich, instant, and predictive insights. Operational intelligence replaces traditional BI and safeguards mission-critical data-driven business operations.Â
While packaged business observability solutions exist, their high cost, immature features, and inflexibility outweigh any turnkey promises. After carefully weighing the costs, risks and benefits, many companies are choosing to build their own operational intelligence systems with a real-time analytics database as the core.Â
The data-driven digital enterprise is the future of business. Protect it with operational intelligence.
Tell us your insights from this series on business observability and operational intelligence on LinkedInOpens a new window , TwitterOpens a new window , or FacebookOpens a new window . We’d love to know!