Traditional Data warehouses struggle with 3V’s (Variety, Volume and Velocity)

1. Variety – Different spices of data
2. Volume- Volume of Data
3. Velocity – Speed of arriving Data

Traditional Data warehouse has maintain structured and semi-structure data.

Does it handle current and modern data with volume and velocity?.

Variety

Traditional DW focused only on structured and semi-structured data. Data warehouses weren’t designed to deal with anything but structured data.

Currently SaaS vendors, under pressure to make data available for any form like text, audio and video, so started building application APIs using JSON file format as a popular way to exchange data between systems. It provides lots of flexibility. JSON will provide semi-structured format like other formats such as Avro or Protocol buffers.

Limited to processing data in either the data warehouse’s built-in SQL engine or a warehouse-specific stored procedure language.

Handling of a range of data varieties is limited in a traditional data warehouse.

Volume

In a traditional DW, storage and processing are coupled which significantly limiting scalability and flexibility. For more volume of data, which requires more hardware and infrastructure is needed, which is slow and expensive.

 Velocity

Traditional DW are batch-oriented or near real-time oriented. The speed at which data arrives into your DW and is processed for analytical real-time process.

It’s a just a question of when, NOT IF.

Data Warehouses are optimized for batch, not streaming data.

Does Data Lakes the rescue?

As per What is.com

A storage repository that holds vast amount of raw data in its native format until it’s needed.

In addition by Gartner Research:

Collection of storage instances of various data assets additional to the originating data sources. These assets are stored in a next – exact or event exact copy of source format. As a result, the data lake is an un-integrated, non-subject-oriented collection of data.

Organizations desperately needed a way to deal with increasing no. of data formats and growing volume and velocities of data that traditional DW could not handle. Here, Data lake place where it could bring any data we want, from different sources, structured, unstructured, semi-structured and Binary. The place we could storage and process all our data in a scalable manner.

Conclusion

Traditional DW has limitations of data varieties, volume and velocity.

Combine of Data lakes, cloud and DW will handle the traditional limitations.