Dredging the data lake

October 10, 2018 by David Bradley, Inderscience
Credit: CC0 Public Domain

Data lakes allow information to be added to a system without pre-processing or modelling. Contrast this with a conventional database where data must be delivered in a much more refined and formal manner. Thus a data lake offers much timelier speed of entry. However, as research from Brazil shows, even though a data lake preserves highest granularity level of the data, that useful flexibility can be problematic too. "If not managed, it is easy to lose control of the repository because of the volume it holds and its growth," the team explains.

The researchers explain further that data lakes carry none of the semantics of a conventional database, but while this can be advantageous in avoiding certain types of bias when re-extracting and analyzing days, it does mean that understanding the contents of the data lake can become a rather cumbersome task. This, the team suggests, has perhaps undermined the widespread adoption and use of data lakes within the corporate environment and stymied acceptance of this useful tool because of certain misconceptions regarding how they might be used in data science efforts.

The team has now turned to knowledge management models to help them address the issues associated with data lake use and to enrich the data floating within to enhance information usability. They also add that through the use of a data portal platform and associated metadata they reason that their approach would provide easy access to the maintaining and boosting its usefulness and precluding its denigration into a so-called swamp.

Explore further: Researchers predict invasion risk of starry stonewort in upper Midwest

More information: Jano Moreira De Souza et al. Using knowledge management to create a Data Hub and leverage the usage of a Data Lake, International Journal of Knowledge Management Studies (2018). DOI: 10.1504/IJKMS.2018.10015483

Related Stories

Online photos provide evidence for the value of clean water

February 3, 2015

Think of the last time you planned a visit to a lake. Why did you choose the lake you did? Did you consider the quality of the water? The answers to these questions are critical to understanding how lake users make decisions ...

Recommended for you

Pushing lithium ion batteries to the next performance level

December 13, 2018

Conventional lithium ion batteries, such as those widely used in smartphones and notebooks, have reached performance limits. Materials chemist Freddy Kleitz from the Faculty of Chemistry of the University of Vienna and international ...

Uber filed paperwork for IPO: report

December 8, 2018

Ride-share company Uber quietly filed paperwork this week for its initial public offering, the Wall Street Journal reported late Friday.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.