Diving into Data Lakes
by Tim York
Big data is essential to understanding consumer trends, knowing what’s working well in a company, and figuring out where to invest resources to maximize ROI. What’s the best way to store and manage all the information you’ve carefully collected about various aspects of your business? A data warehouse? Or a data lake?
Warehouses for Extracted Data
Data warehouses have traditionally been used to store and organize data in a highly structured way. Information drawn from operational systems is added to the warehouse after management decides how that data will ultimately be used. Sources of raw data are carefully scrutinized and the input data is preselected based on anticipated reporting needs.
Lakes for Unsorted Data
Data lakes, on the other hand, contain vast amounts of unsorted data in its raw state. Whether or not there’s a planned use for the information is irrelevant. Not only is the volume of data huge (thanks to ever-cheaper storage capacity and increased data retrieval and processing speeds), the variety of data types you can collect in a data lake is endlessly flexible and inclusive. Just like a real lake that may contain fish, weeds, and power boats, a data lake welcomes text, images, and good old comma separated files. Archival data accumulates like sediment at the bottom of a lake and is available for sampling and comparisons when needed.
Data No Matter What You’re Fishing For
All types of data are available in the lake for all kinds of users - from managers looking at sales trends to data scientists interested in exploring complex relationships between different types of inputs. All users are able to utilize the underlying resource of the original raw data regardless of their reporting or analysis needs. Going back for more is never a problem if reporting needs change or new analyses are required.
Data Lakes and Warehouses Can Coexist
Data lakes can be created right alongside your traditional warehouse structure. As data trickles (or floods) in and begins to fill the lake, users can be provided with strategies and tools to dip into these deep waters and the abundance of information they contain.
At QBIX Analytics we understand that knowledge is power and data is valuable. Whether you use a data lake, a warehouse, or both, let our experts help your company make the most of the information you need to succeed.