Dredging the data lake

October 10, 2018 by David Bradley, Inderscience
Credit: CC0 Public Domain

Data lakes allow information to be added to a system without pre-processing or modelling. Contrast this with a conventional database where data must be delivered in a much more refined and formal manner. Thus a data lake offers much timelier speed of entry. However, as research from Brazil shows, even though a data lake preserves highest granularity level of the data, that useful flexibility can be problematic too. "If not managed, it is easy to lose control of the repository because of the volume it holds and its growth," the team explains.

The researchers explain further that data lakes carry none of the semantics of a conventional database, but while this can be advantageous in avoiding certain types of bias when re-extracting and analyzing days, it does mean that understanding the contents of the data lake can become a rather cumbersome task. This, the team suggests, has perhaps undermined the widespread adoption and use of data lakes within the corporate environment and stymied acceptance of this useful tool because of certain misconceptions regarding how they might be used in data science efforts.

The team has now turned to knowledge management models to help them address the issues associated with data lake use and to enrich the data floating within to enhance information usability. They also add that through the use of a data portal platform and associated metadata they reason that their approach would provide easy access to the maintaining and boosting its usefulness and precluding its denigration into a so-called swamp.

Explore further: Researchers predict invasion risk of starry stonewort in upper Midwest

More information: Jano Moreira De Souza et al. Using knowledge management to create a Data Hub and leverage the usage of a Data Lake, International Journal of Knowledge Management Studies (2018). DOI: 10.1504/IJKMS.2018.10015483

Related Stories

Online photos provide evidence for the value of clean water

February 3, 2015

Think of the last time you planned a visit to a lake. Why did you choose the lake you did? Did you consider the quality of the water? The answers to these questions are critical to understanding how lake users make decisions ...

Recommended for you

Archaeologists discover Incan tomb in Peru

February 16, 2019

Peruvian archaeologists discovered an Incan tomb in the north of the country where an elite member of the pre-Columbian empire was buried, one of the investigators announced Friday.

Where is the universe hiding its missing mass?

February 15, 2019

Astronomers have spent decades looking for something that sounds like it would be hard to miss: about a third of the "normal" matter in the Universe. New results from NASA's Chandra X-ray Observatory may have helped them ...

What rising seas mean for local economies

February 15, 2019

Impacts from climate change are not always easy to see. But for many local businesses in coastal communities across the United States, the evidence is right outside their doors—or in their parking lots.

The friendly extortioner takes it all

February 15, 2019

Cooperating with other people makes many things easier. However, competition is also a characteristic aspect of our society. In their struggle for contracts and positions, people have to be more successful than their competitors ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.