The Problem With Relying on Your IT Department for Data Analytics

Too often, organizations count on IT to pull data sets for analysis, but by the time the query results are available, the data is likely to be stale. Katie Horvath, CMO, Aunalytics, explains how modern data analytics is the most efficient way to achieve critical business insights.

According to a Gartner Top 10 Data and Analytics Trends for 2021Opens a new window report, organizations are beginning to understand the importance of using data analytics to accelerate their digital business initiatives to remain competitive. Data analytics is shifting from being a secondary focus to a core function. However, business leaders often underestimate the complexities of data and end up missing opportunities. Another problem is that very often, they rely on their IT departments for data analytics, which can compromise the real-time value of data, making it unreliable for genuine business insights. 

Your IT department is primarily concerned with maintaining the security of your systems and keeping them operational. IT owns the business function of minimizing internal and external security risks and vulnerabilities and maintaining core business systems and operations. IT has service goals and focuses on help desk management, including responding to tickets, issue and resolution performance, and tracking. IT is concerned with uptime and minimizing downtime on both internal and customer-facing systems. This is critical and where you want to keep your IT department focused. 

Yet, in most companies, business analysts rely upon the IT department to pull data sets for analysis. Once the data set is received, the analyst can analyze it to answer business questions, look for trends, find growth and cost-cutting opportunities, and create segmented customer lists for targeted marketing. 

The IT department views query requests as less important ad hoc asks compared to keeping your systems live – as it should. The result is that by the time the business analyst gets query results, data is often stale. This means that windows of opportunity are being missed due to delayed analysis. 

Further, data analytics is not the skill set of most IT department team members. Data analytics involves expertise in data management, data modeling, and machine learning. Most IT departments do not include data engineers and data scientists. Accordingly, by asking your IT department to implement data analytics, you are asking them to take the focus off of what they are trained to do and dabble into new areas of technology without having the expertise to do so.

Learn More: Master Multi-Entity Finance: 3 Strategies for Consolidating Companies

At the same time, most IT departments are accustomed to problem-solving, and very few will tell you that they cannot add data analytics implementation to their list of projects. You end up with impatient executives wanting answers to pressing business questions, frustrated business analysts waiting for query results to be able to provide answers to the executives, and a stressed IT department not having time to focus on pulling queries. 

Relying on the IT Department for Analytics – A Case Study

For example, the IT department of a regional healthcare system in Michigan could barely keep up with the hospital acquisitions that required merging or migrating EMRs, appointment scheduling, billing systems, HR functions and more. IT was faced with multiple data mapping projects to attempt to bring data from an acquired hospital, physician practice group, or outpatient facility into the healthcare system data sources. 

It was common for the healthcare system to be running five or more EMRs at a time, in addition to multiple billing and appointment scheduling systems due to strong M&A activity. Attempting to bring data together for whole-system reporting was daunting. It primarily existed in the form of data exported into Excel spreadsheets from multiple disparate sources and then stitched together. The data preparation required to normalize multiple disparate data sets for a common reporting format, with columns matching up like data, having different field names and being presented in different column order from different databases, was time-consuming.

It took hundreds of hours to produce data from which simple graphs could be drawn for a dashboard for hospital leadership. And by the time the IT department had the Excel spreadsheets ready for reporting, the data was stale. The idea of real-time reports, let alone advanced data analytics to inform executive decision-making, began to look as likely as sending a manned rocket of healthcare providers safely to Jupiter.  

The hospital IT department had top-notch network engineers, security technicians, and service desk personnel. But when it came down to it, there was no one with skill sets for data engineering or data science expertise. Building out an in-house data science and data engineering department in addition to its IT department made little sense. The healthcare system could not support this type of spend when government and private insurer reimbursements were tight, and its primary focus was on providing healthcare. 

The system board of directors knew that they needed data analytics to better compete and that it would save operational expense. But the system was so accustomed to relying on its IT department for anything technology-related that it assumed this over-worked group would be able to absorb data analytics. It could not. Senior leadership remained frustrated by not having regular, timely data for reports and not having insights that only data analytics can provide. The healthcare system felt that its competitive edge was slipping.

Learn More: Is It Time To Re-Examine the Role of Devops Engineers?

A Data Solution With on-Demand Access, Self-Service Availability, and Real-Time Data

What is needed is universal, on-demand data access in a centralized analytics platform built for non-technical business users. What is needed is self-service data availability as a ‘single source of truth’ for data combined from multiple data sources and systems. What is needed is an accurate stream of data ready for analytics in real-time so that analytic results are timely and based upon current information. 

This data should be consistent no matter which department grabs a data set. This means that data access needs to originate from a centralized place to pull queries instead of basing analytics on different ad hoc queries involving different systems that may likely create inconsistent results due to data inconsistencies across business applications. Without proper mitigation, these challenges breed data distrust. 

The same is true for dashboards and reporting. If the underlying data feeding the dashboard, report or analytics is not accurate, the results cannot be accurate. If the underlying data is not trusted, the results will not be trusted. 

If a business question is posed, the results should not vary based upon which data source is queried. Data should be the same set of facts across an organization. Accordingly, data from multiple systems in the line of business applications must be 

1. 1. Integrated,
   2. Cleansed for data quality issues, errors and duplicate records, and 
   3. Matched and merged to create holistic data models and views, achieving a ‘golden record’ ready for analytics. 

Modern solutions now include cloud-native data analytics solutions. These solutions have API integrations to data lakes with built-in data management for faster integration, use change data capture technology to better handle data in motion, and efficiently create a real-time analytics-ready stream of accurate information. The analytics must be powered by deep learning models, AI and/or machine learning capable of mining transactional data to realize patterns, trends, opportunities, and other insights based upon transactional behaviors. 

Modern solutions must include no code or low code query capabilities so that non-technical business users may perform the queries, instead of IT, without having to write SQL or other code. At the same time, modern solutions must include built-in data governance and the ability for technical users to access query code to review and edit, if desired, to gain trust in the solution, the data and the analytic results.

Learn More: 4 Reasons Why Data Virtualization Might Not Solve Your Migration Problem

Finally, modern solutions should include visualizations to better understand analytics results and should be capable of integration with preferred and custom dashboards for further visualization options and reporting. Historically, achieving this type of data integration while keeping the data cleansed as new data is created in business operations daily requires lengthy data source mapping projects for integration and large price tags for data warehouses and data management.

Data Solution Critical Requirements: 

• • Built for non-technical business users
  • Self-service data availability
  • Cloud-native
  • Fast API-based data integration
  • Auto-mapping of new data sources
  • Change data capture efficiency for data ingestion
  • Built-in data cleansing to tackle data quality issues 
  • Automated real-time generation of the golden record 
  • of data ready for analytics 
  • Data lake with built-in data management
  • Built-in data governance
  • No code query functionality
  • Machine learning-powered analytics
  • Ability to include transactional data in analytics
  • Visualization options for analytics results.

Data Solution Improvements: 

• • NLP query functionality
  • AI, ML and deep learning-powered analytics 
  • Industry-specific data model design
  • Relational data models
  • Relational golden records 
  • Audit trail of changes made to data 
  • Built-in Data Catalog/ Data Dictionary 
  • Right to Be Forgotten compliance enablement

Let your IT department do what it is best at doing – maintaining security and ensuring systems are operational, minimizing risk, and maximizing uptime while decreasing downtime. A modern data analytics solution is the most efficient way to achieve critical insights, identify trends, growth and cost-cutting opportunities, and perform well-targeted marketing to accelerate business.