Scality Showcases Real-World Resilient Cloud Storage Solutions
Scality brought the heat at Cloud Field Day 23, telling their story through the lens of customer success. The company, which has been working in the storage software arena for the last 15 years, discussed the latest with Scality Ring. Their history started with the rise of the cloud, responding to service providers needing a storage foundation extending into enterprise on-prem cloud storage solution delivery.
With this foundation, Scality began pivoting to AI data lakes a few years ago, adding integration of security and data aggregation, feeding the initial phases of data cleaning before being utilized for AI training. They differentiate with extreme scaling, supporting 1 to 1000s of apps, 100 petabytes (PB) per failure domain, 100s of billions of objects, and more.
We walked through a number of use cases, starting with Splunk. Ben Morge walked through a case study of a US-based bank with a 30-40 PB Splunk data store across multiple sites and 1-year data retention. Ben described the configuration featuring SSD-based hot-tier data connected to Ring warm-tier backup. Data is replicated across sites tapping HPE Apollo servers. The complexities of this configuration involved mitigating migration ingest overload and unexpectedly high production GET traffic, addressed by increasing server capacity to maintain a 75 GB/GET throughput per site. The result? Tiering with this approach provided a resilient multi-site storage solution with the required retention policy.
Next up was Nick Sayer, describing a challenge tackled for CNES, a multi-PB satellite imaging storage with data retention of new imagery in hot tier for 6 months along with multi-hundred PBs of cold-tier image data. The customer had a hodgepodge of storage environments and wanted to convert multiple formats to S3 in a cost- and energy-efficient environment.
The solution? A hot/warm tier Ring across three data centers in a single namespace S3 data lake coupled with a cold-tier tape solution. Any user query seeking data on tape is prompted to restore from cold tier on a temporary basis to utilize the data before it’s returned to its tape home. The multi-site tape solution stretched Ring capabilities, requiring support across multiple public clouds and tape providers and requiring a unique API development by the Scality team to deliver the resilience required for tape solutions.
Next up was Aurelien Gelbart providing insights on a French bank deployment integrating multiple storage solutions for internal data with an aim of lower price per TB with a bill-back solution to business groups for usage. The challenge was moving over 2,000 applications’ data to be integrated into one Ring platform, including database backup, financial data, data lake, DMS, artifacts and media. The solution required rapid scale, growing from 1 to 50 PB over seven years. Scality tackled this solution with software improvements that handled this complex environment as well as architectural improvements that allowed server flexibility. The solution today supports 100 PB of storage and over 300 billion objects and 1 billion client operations per day, offering incredible scale and resiliency for the bank…all with a cost reduction of 75%, primarily driven by consolidation of systems across business groups.
The TechArena Take – Resilient and Scalable Storage
With these customers and more relying on Scality’s scale, flexibility, and engineering know-how, it’s no wonder that global brands rely on the software for storage integration. We like the way the team shared how they work with customers to solve unique challenges, speaking to the deep level of collaboration required for these large-scale deployments. We see value in how they tackle unique opportunities like tapping cold-tier tape and mitigating the complexity with collaborations and custom API development. We see no end of demand as more companies seek AI data lakes to fuel broad scale AI deployments. In other words…we expect to see more fantastic customer deployment stories from Scality in the months ahead.
Scality brought the heat at Cloud Field Day 23, telling their story through the lens of customer success. The company, which has been working in the storage software arena for the last 15 years, discussed the latest with Scality Ring. Their history started with the rise of the cloud, responding to service providers needing a storage foundation extending into enterprise on-prem cloud storage solution delivery.
With this foundation, Scality began pivoting to AI data lakes a few years ago, adding integration of security and data aggregation, feeding the initial phases of data cleaning before being utilized for AI training. They differentiate with extreme scaling, supporting 1 to 1000s of apps, 100 petabytes (PB) per failure domain, 100s of billions of objects, and more.
We walked through a number of use cases, starting with Splunk. Ben Morge walked through a case study of a US-based bank with a 30-40 PB Splunk data store across multiple sites and 1-year data retention. Ben described the configuration featuring SSD-based hot-tier data connected to Ring warm-tier backup. Data is replicated across sites tapping HPE Apollo servers. The complexities of this configuration involved mitigating migration ingest overload and unexpectedly high production GET traffic, addressed by increasing server capacity to maintain a 75 GB/GET throughput per site. The result? Tiering with this approach provided a resilient multi-site storage solution with the required retention policy.
Next up was Nick Sayer, describing a challenge tackled for CNES, a multi-PB satellite imaging storage with data retention of new imagery in hot tier for 6 months along with multi-hundred PBs of cold-tier image data. The customer had a hodgepodge of storage environments and wanted to convert multiple formats to S3 in a cost- and energy-efficient environment.
The solution? A hot/warm tier Ring across three data centers in a single namespace S3 data lake coupled with a cold-tier tape solution. Any user query seeking data on tape is prompted to restore from cold tier on a temporary basis to utilize the data before it’s returned to its tape home. The multi-site tape solution stretched Ring capabilities, requiring support across multiple public clouds and tape providers and requiring a unique API development by the Scality team to deliver the resilience required for tape solutions.
Next up was Aurelien Gelbart providing insights on a French bank deployment integrating multiple storage solutions for internal data with an aim of lower price per TB with a bill-back solution to business groups for usage. The challenge was moving over 2,000 applications’ data to be integrated into one Ring platform, including database backup, financial data, data lake, DMS, artifacts and media. The solution required rapid scale, growing from 1 to 50 PB over seven years. Scality tackled this solution with software improvements that handled this complex environment as well as architectural improvements that allowed server flexibility. The solution today supports 100 PB of storage and over 300 billion objects and 1 billion client operations per day, offering incredible scale and resiliency for the bank…all with a cost reduction of 75%, primarily driven by consolidation of systems across business groups.
The TechArena Take – Resilient and Scalable Storage
With these customers and more relying on Scality’s scale, flexibility, and engineering know-how, it’s no wonder that global brands rely on the software for storage integration. We like the way the team shared how they work with customers to solve unique challenges, speaking to the deep level of collaboration required for these large-scale deployments. We see value in how they tackle unique opportunities like tapping cold-tier tape and mitigating the complexity with collaborations and custom API development. We see no end of demand as more companies seek AI data lakes to fuel broad scale AI deployments. In other words…we expect to see more fantastic customer deployment stories from Scality in the months ahead.
.png)
