Amazon Web Services subscribers, take note: If your business yields a tsunami of data and you struggle to harness its potential, a new AWS development could help.
In a bid to strengthen its cloud offeringOpens a new window , Amazon’s cloud computing arm has announced the general availability of AWS Lake FormationOpens a new window , a fully-managed service that simplifies the job of building, securing and managing so-called ‘data lakes’ (essentially a centralized store of dataOpens a new window ).
Once formed, customers can employ machine learning techniques and analyticsOpens a new window across their data, no matter their format or location.
Creating a data lake requires a series of manual steps. However, AWS Lake Formation, which was unveiled last November, automates many of these processes, including collecting, cleaning and cataloguing big data.
A centralized dashboard allows administrators to oversee data access policies, governance and auditing, as well as offering a searchable catalogue showing available datasets and how what they can be used.
The new service can be integrated with your choice of AWS analytics and machine learning services, including Amazon Redshift, Amazon Athena, and AWS Glue, with Amazon EMR, Amazon QuickSight and Amazon SageMaker.
Customer Demand
“Our customers tell us that Amazon S3 [Amazon Simple Storage Service] is the ideal place to house their data lakes, which is why AWS hosts more data lakes than anyone else – with tens of thousands and growing every day,†says Raju Gulabani, vice-president for databases, analytics and machine learning at AWS. “They’ve also told us that they want it to be easier and faster to set up and manage their data lakes.
“That’s why we built AWS Lake Formation, so customers can spend more time learning from their data and innovating, rather than wrestling that data into functioning data lakes. AWS Lake Formation is available today and we’re excited to see how customers use it as one of the building blocks for growing and transforming their businesses and customer experiences.â€
For data management, a repository such as a data warehouse or lake can help. While the warehouse stores information in an orderly, siloed fashion, the lake holds data in its purest form — uncategorized, unanalyzed and unprocessed.
The advantages of a data lake include flexibility, meaning the relative ease of changing models and queries tailored to the job necessary at a specific time, the relative simplicity of setting one up and its cheaper cost.
However, lakes are seen as potentially less secure given that all data is pooled in one place and can require the skills of a data scientist to harness to their full potential.
Still, the two options can work in parallel for the specific needs of your business.
AWS Lake Formation has already attracted big-name customers including the professional services outfit Accenture, online retailer Zalando, in-flight entertainment and communications provider Panasonic Avionics Corporation and biotech Amgen.
Growing Market
Zacks Equity ResearchOpens a new window  notes that strengthening customer momentum from the new service should help the company smooth passage into the burgeoning data lake market which, according to a report from Mordor IntelligenceOpens a new window , should see compound annual growth of 27.4% between 2019 and 2024.
Capgemini says that more than 60% of U.S. financial institutions believe big data analytics gives them an important advantage over competitors.
Other big players jostling for room in the segment include Microsoft, offering Azure Data Lake, IBM and Google.
AWS Lake Formation is now available in certain of the company’s zones, including the eastern US (Ohio and northern Virginia), western US (Oregon), Asia Pacific (Tokyo) and Europe (Ireland), with more regions the company says to be added soon.