DataOps is still an emerging part of DevOps; it is utilized by large organizations with teams of data scientists, developers and other data-focused roles that train machine learning models and deploy them to production. The goal of using a DataOps methodology is to create an agile, self-service workflow that fosters collaboration and boosts creativity while respecting data governance policies.
A DataOps practice supports cross-functional collaboration, and fast time-to- value. It is characterized by processes as well as the use of enabling technologies. As with other areas of innovation, there is always a danger of fragmentation within day-to- day work as different platforms and systems are employed. Getting there, to a single platform for all data across every cloud has been something of a holy grail.
MapR, for example, has established itself as a provider of a single modern data fabric, where volumes of data are ingested once and then become accessible as a single source from on-premise data centers, across clouds and to the remote edge.
The company recently rolled out an upgrade which addresses many of the current concerns and issues that DevOps practitioners face, including automation of platform health and security and a groundbreaking database for next-generation applications.
According to Anoop Dawar, Vice President for Product Management at MapR, the three key areas of focus for the update surround security data and governance, faster time to machine learning and analytics, and automated cluster health and administration.
“DataOps is an important movement, ultimately letting organizations turn their data into value as quickly as possible,†he explains.
Users across business lines should be able to quickly find the data they need or data that could be useful for them in their analysis, but only if they have the appropriate rights to that data. Single click security enhancements, such as enforcement of authentication and more comprehensive encryption on the wire can help to take much of the guesswork out of configuring security.
Data analysis is increasingly being driven by machine learning and artificial intelligence, enabling teams to gain quick, accurate and actionable insights. Yet, data scientists are the driving force behind the DataOps movement.
MapR’s answer to those demands is real-time integration with version 6.0 of its Converged Data Platform. This includes real-time data integration and continuous stream processing. Multiple applications, including machine learning models and deployments can share information and be synchronized in real time.
MapR has recently launched what it calls a Data Science Refinery that can give clients complete, self-service access to all data from within the same cluster.
As developers move towards increasingly sophisticated machine learning models, so the DevOps function is being challenged by some fairly core data management problems. While MapR may be an early mover, other providers of cloud solutions in the market will need to develop similar services. Without the proper management of data for machine learning models, projects can quite literally grind to a halt.
Key Takeaways:
- DataOps is the backbone of any data-driven enterprise but too often lacks the tooling to make it a scalable and repeatable process.
- In a recent survey conducted by Nexla, integration, trouble shooting and building data pipelines took up 47% of respondents’ time.
- Based in California, MapR is currently the only converged data platform that allows its customers to harness the power of Big Data by combining analytics in real-time operational applications that can improve businesses outcomes.
- Cloud provider marketplaces like Microsoft Azure, Amazon Web Services and Oracle Cloud will make version MapR version 6.0 available by the end of the year.