Cloud outages are becoming more and more common, but as our reliance on cloud storage grows, so too does the need to ensure your data is available when it’s needed, every time it’s needed. Glen Shok, Panzura’s CTO & VP Cloud Architects, looks at how cloud mirroring lets you place the same set of data in two separate object stores, giving you cloud redundancy with high availability if your primary cloud ever goes down.Â
Cloud outages among the world’s largest cloud providers are a wake-up call. Amazon Web Services experienced a string of major failures last year due to a “relatively small addition of capacity,†causing services from organizations including Adobe, Autodesk, Fidelity, and even the Metropolitan Transport Authority in New York City to go down without warning.
Failures with Google Cloud Platform and Microsoft Azure have led to interruptions that have been no less dramatic, affecting thousands of companies. Service interruptions in critical sectors such as finance and healthcare have underscored the fragility of the cloud, where reliance on globe-spanning infrastructure is teetering in the balance.
As current political events on the international stage have made clear, cloud infrastructure also represents the world economy’s pivotal nodes. Failure due to human error and random acts of nature are now overshadowed by the danger of state actors intent on taking down entire systems crucial to modern society and business. The key is adapting to failures before the next catastrophe strikes.
Multi-cloud Storage Redundancy
Mirroring or remote replication is a familiar concept in IT operations. It is typically implemented to achieve two goals. Firstly is to ensure that data is always available. Secondly, and as a broader IT strategy, mirroring provides complete or almost complete redundancy of data. It is a familiar concept for structured data such as that held in a database.
In the event of an outage, the Microsoft SQL Server database mirroring in high-safety mode with automatic failover will quickly bring the standby copy of the database online without data loss. Mirroring a core data store with CloudKit lets iOS app developers store app data in iCloud and keep everything available and up-to-date across devices and the web.
Cloud mirroring extends this paradigm to unstructured data in the cloud, focusing on reliability by design. It places the same data set in two different object stores, achieving continuous availability should an outage occur in a single provider. The most advanced applications of cloud mirroring go beyond sending replicated data from one location to a secondary storage location, eventually creating a mirror image of the original. Instead, they write data to two storage locations simultaneously, resulting in the holy grail of mirroring – a redundant data set that is always and immediately a replica of the primary copy, regardless of where the data has been created.
Redundancy of data and workloads across more than one cloud provider, an organization’s private cloud, or both dramatically reduces the chances that access to data and systems will fail all at once. It also means problems are localized to prevent industry-wide failures at the hand of overreliance on a single cloud storage vendor. Panzura, for example, switches to a secondary cloud object store when a primary cloud provider is offline or unavailable.
Approaches to cloud mirroring now employ real-time split captures of new data in cloud storage instances as new files are created or edits are made. Should a primary cloud store suffer sustained failure, cloud mirroring, in this case, will automatically failover to the secondary store, allowing front-end file services for systems or users to continue operating. When the primary cloud store is available again, cloud mirroring in this scenario then switches back and automatically synchronizes both clouds to a consistent dataset.
Protection Against Data-loss
While redundancy is the first line of defense, the enterprise also needs to set up a system where lost data can be quickly recovered. However, despite the exponential growth of data, IT organizations have not necessarily changed cloud storage strategies. The purpose of regular backups, for instance, is to protect data that is relied on for everyday operations.
The other real advantage of cloud mirroring is securing data such as business-critical files from cyberattacks, deletion or unauthorized encryption as with ransomware. Cloud mirroring may also be deployed as an air-gapped solution between cloud providers, which provides a full backup in a trusted environment beyond single cloud replication, automatic failover and immediate consistency in case of cloud failure.
Again, IT departments are familiar with the concept of mirroring to protect their organizations from data loss. Dell EMC synchronous and asynchronous replication technologies provide users with immediate access to point-in-time copies of data in the event of a cyber attack. NetApp SnapMirror replicates data over LAN or WAN, providing data replication, although not immediate in this case, for applications in both virtual and traditional environments. The goal is to keep data current and available when it is needed.
Not only does cloud mirroring eliminate dependency on a single cloud vendor, but it is also a multi-cloud strategy that protects an organization against disruption and data loss from accidental cloud bucket deletion and other potentially catastrophic data storage events such as cyberthreats against a cloud provider.
Cloud Mirroring In Operation
Modern cloud mirroring can be designed to use a cloud connector to communicate with any compatible object store via that cloud’s RESTful API. This object store may be a public cloud, private (on-premise) cloud or completely “dark†cloud with no external connection.
Any two compatible object stores can ideally be designated as primary and secondary clouds. These may be two cloud regions from the same provider, two data tiers from the same provider, or two distinct object stores from unrelated providers, including on-premises storage. Each store should have its independent cloud connector.
Cloud mirroring can ensure that both cloud stores are entirely consistent, holding the same set of data down to the last byte at all times. Should it become necessary to failover to the secondary store, it is critical that every single file is there and accessible to every authorized user.
Each location should ideally read from the primary cloud in real-time, anticipating and locally caching the most used files. For instance, every 60 seconds, each site in the storage network can be configured to simultaneously write new and changed data to both the primary and secondary clouds, storing it as immutable data. Immutability is a crucial factor, as it ensures data cannot be altered or changed should a direct attack on the cloud store occur.
This write-split ensures a complete, redundant copy of any additional data is captured in the secondary cloud and securely stored in the primary cloud and available to every other location for immediate data consistency.
In a cloud failure, effective cloud mirroring will automatically disable read and write operations to the failed cloud. With cloud mirroring activated, a sustained primary cloud outage should failover to the secondary cloud for read and write operations until the primary cloud is restored.
Because every location can ostensibly be configured to write to both clouds simultaneously, data held in the secondary cloud will be completely consistent with the data in the primary cloud, resulting in a seamless experience for users with no data loss or file inconsistencies, even in the event of a catastrophic outage.
See More: Mitigating the Data-Driven Burden of Unstructured EHR in the Age of Ransomware
Resilience In Chaos
The criticality of data makes it imperative that organizations adopt strategies that make it safe, secure, durable, and available. Cloud outages with AWS, Azure and other pervasive cloud providers have taught us that organizations need to be prepared for the unexpected. Global events continue to bring this into an even sharper focus.
Cloud mirroring allows file operations to continue uninterrupted. Its most advanced applications can also maintain a record of locally-created data that has not been synced to the primary cloud. When the primary cloud is available again, cloud mirroring with Panzura, by way of example, will resume writing to both clouds in parallel. Read operations will switch back to the primary cloud once it is stable.
In the background, under this scenario, the system will draw on data stored locally, incorporating all changes made while it was unavailable, to make the primary cloud consistent.
With cloud mirroring, it is possible to achieve high availability with automatic failover that guards against cloud outages, cyber-attacks made directly against cloud storage providers, cloud bucket deletion and other downtime events.
Putting faith in a single cloud provider, expecting a secure and always-on ecosystem, is tempting the statistical probabilities. Ramping up the push for business continuity while protecting data from the inevitable ransomware incursion or breach requires a multi-cloud strategy that includes cloud mirroring in both concept and practice.
Have you used strategies like cloud mirroring? Share your experience with us on LinkedInOpens a new window , TwitterOpens a new window , or FacebookOpens a new window . We’d love to learn from you!