AI storage: a new requirement for the shift in computing and analytics – Information Age

AI storage is needed as the boundaries of traditional computing and analytics shift to a new era

Data or AI storage is necessary to solve the emerging data challenges caused by the move away from traditional computing and analytics.

The universe of supercomputing has expanded rapidly to incorporate AI, advanced data analytics and cloud computing. The era of serial data is ending, with parallel data management replacing network file systems (NFS).

This shift has corresponded to the rise of AI, with investments in the technology hitting a new record in 2021. As an example, Microsoft invested $1 billion in an artificial intelligence project, co-founded by Elon Musk.

This shift in the boundaries of traditional computing and analytics has caused several data challenges that need to be resolved:

Data talent there is need to source new data science talent, maintain currency and up to date skills sets in a rapidly changing software environment.

Data sources there is a need to ingest high volume data from broad sources through a variety of ingest methods at rates well beyond traditional computing requirements.

Data processing there is a need for a different type of data processing to implement large scale GPU environments to bring the parallelism needed for training and inference in real-time.

Data governance there is a need to label, track and manage that data (forever) and share that data across organisations with the right security policies. Explainable AI on an application level and available data on a platform level.

During The IT Press Tour in San Francisco, James Coomer, Sr. Vice President Products Data at DDN, explained that data is the source code of AI, data is imperative for AI and storage is imperative for AI.

Storage cant be an afterthought, as its key for data ingestion, sourcing, management, labelling and longevity, which is critical for AI.

Data or AI storage is necessary to solve the emerging data challenges caused by the move away from traditional computing and analytics:

>Read here: To find out more about the other companies in the latest edition of The IT Press Tour in San Francisco and Silicon Valley

Traditionally, DDN Storage has focused on traditional data storage for unstructured data and big data in enterprise, government and academic sectors.

Now, it is redefining the imperatives that are driving it as a company, focusing on AI storage, with its solution, AI, which is at the heart of its growth strategy.

In action, over the last two years DDN has acted as the core backend storage system for NVIDIA to increase performance & scale and flexibility to drive innovation.

NVIDIA commands nearly 100% of the market for training AI algorithms and has multiple AI clusters, according to Karl Freund, analyst at Cambrian AI Research.

Following this success, DDN is powering the UKs most powerful supercomputer, Cambridge 1, which went live in 2021 and is focused on transforming AI-based healthcare research.

The AI storage vendor is also working with Recursion, the drug discovery company.

Our at-scale data needs require fast ingest, optimised processing and reduced application run times, said Kris Howard, Systems Engineer at Recursion.

Working with DDN, the drug discovery company achieved up to 20x less costs and raised the possibilities for accelerating the drug discovery pipeline with new levels of AI capability.

It previously ran on the cloud, but now operates more efficiently, with greater value for money on-premise.

DDN pioneered accelerated data-at-scale to tackle what ordinary storage cannot. We make data environments for innovators to create the future. Were the largest AI storage provider in the world, proven in an array of different industries and customers, from financial services to life sciences, added Coomer.

1. Transforming cancer care with managed services: DDN as a service for precision Oncology

Customer:

Data challenge:

DDN solution:

2. Simplifying data management for a global financial services and venture firm

Customer:

Data challenge:

The DDN Solution:

3. Transforming research data storage: from management and maintenance to universal resources

Customer:

Data challenge:

The DDN Solution:

See also: Mining the metadata and more Tips for good AI data storage practices

Here is the original post:

AI storage: a new requirement for the shift in computing and analytics - Information Age