powershelldba.de · Uwe Janke

Data Mining, ETL, BI, Data Lake: Same Idea, New Bandwagon Every Few Years

There is a German saying for it: alle paar Jahre wird eine neue Sau durchs Dorf getrieben, every few years a new pig gets driven through the village. The pig doesn't survive translation. The English idiom for the same thing does: every few years, the entire industry jumps on a new bandwagon, cheers for a while, and quietly climbs back off once the wheels come off. This is a short, mostly accurate history of exactly that, on a schedule you could set a calendar reminder to.

The Short List

Data Mining. ETL. Business Intelligence. Data Lake. Every one of those was, in its moment, the thing that would finally fix how organizations use their own data. Every one of those got quietly absorbed into "how things are done" once the next bandwagon arrived. None of them actually disappeared. They just stopped being interesting enough to put on a conference agenda.

The full parade is longer than four entries, and it goes back further than most people in it were working.

The Hall of Fame

Era The New Bandwagon What Was Actually Underneath
1970s–80sManagement Information Systems / Decision Support SystemsReports, printed overnight, on paper with holes down the sides
1990sData Warehousing (Inmon vs. Kimball, a genuine religious war)A second database you copied the first database into, on a schedule
1990sOLAP CubesSUM() and GROUP BY, pre-computed so nobody had to wait for it live
Late 1990sData MiningStatistics, run on a bigger table than your professor ever had, with worse documentation
2000sETL as its own discipline (Informatica, DataStage)Copying data from one place to another, with a mapping document nobody kept current
Mid-2000sBusiness Intelligence (Cognos, BusinessObjects, MicroStrategy)Dashboards showing numbers that Finance disputed anyway
Mid-2000sService-Oriented Architecture / Enterprise Service BusSystems calling each other, now with more XML and a consulting retainer
2000sMaster Data Management, "one single source of truth"A new database that also needed reconciling against the other seven
2008–2013Big Data, Hadoop, NoSQLA bigger, slower way to do what the data warehouse already did, minus the schema
2012+Data Science, "the sexiest job of the 21st century"A statistician with a GitHub profile and a Python notebook
2013+Self-Service BI (Tableau, QlikView, Power BI)Business users building their own reports, and their own version of the truth, in parallel
2015+Data LakeA folder, usually in S3 or ADLS, usually undocumented within eighteen months
2015–2018IoT AnalyticsTime-series data at a scale where the sensors were more reliable than the pipeline
2017–2019Blockchain for Enterprise DataA slower, append-only database nobody could query, quietly retired
2018+Citizen Data Scientist, Data DemocratizationHanding the statistics to whoever asked for the dashboard first
2019+Data MeshGiving every department its own data warehouse and calling the resulting chaos "domain ownership"
2020+Data FabricA vendor diagram promising to connect everything nobody had connected in twenty years
2020+LakehouseThe Data Lake and the Data Warehouse, merged back into one thing, which is what a data warehouse already was
2020+MLOps / DataOpsDevOps, for the pipeline that used to just be called "the ETL job"
2022+Generative AI, "AI-powered insights"A chatbot bolted onto the same dashboard, now with a natural-language box on top
2024–2026Agentic AIThe chatbot now runs the ETL job too, and nobody has fully reviewed what it changed

Nobody sat through every keynote on that list, and nobody was expected to. That's what the analyst reports and the consultants were for.

Five Reasons There's Always a New Bandwagon

1. Vendors Need a New SKU

A product that has been quietly doing its job since 2011 does not renew a licensing contract on its own. A product that has been "reinvented for the AI era" does. Rebranding is cheaper than rebuilding, and it resets the sales conversation to a point where the incumbent has no advantage.

2. Analysts Need a New Quadrant

An industry analyst firm cannot sell the same report two years running. A new category, Data Fabric this year, Agentic Data Platforms next, creates a fresh axis to rank vendors on, a fresh reason to commission the research, and a fresh reason for vendors to buy placement in it.

3. Résumés Need Refreshing

"ETL Developer" and "Data Engineer" describe, in most job postings, the identical set of responsibilities. Only one of them reads as current on a profile. The underlying skill, moving and shaping data correctly, doesn't expire. The label attached to it has a shelf life of about five years before it starts working against you in a job search.

4. Consultancies Need a New Keynote

Nobody buys a three-day workshop titled "Same Data Warehousing Practices as Last Decade, Executed Slightly Better." They buy the one titled "Data Mesh: Rearchitecting Your Organization for the Domain-Driven Future." The billable hours are often nearly identical work. The pitch deck is not.

5. The Real Problem Is Boring, and Fixing It Doesn't Announce Itself

Getting clean, well-documented, trustworthy, timely data into the hands of the people who need it is genuinely hard, and has been the actual problem, unchanged, since the first MIS report ran overnight. Declaring a new paradigm is far easier and far more fundable than doing the unglamorous work of fixing data quality, ownership, and documentation. Every bandwagon promises to finally solve the boring problem. None of them are actually about the boring problem.

Kimball's star schema, written up in the 1990s, is still running quietly underneath most "modern data platforms" that put a cloud logo on the architecture diagram. The bandwagon changes. The fact table usually does not.

What Doesn't Change

Underneath every entry in that table is the same short list of things that were true before the rebrand and stayed true after it: understand your source systems, model the data honestly, document what you built, and don't let the pipeline outlive the person who understood it. None of that is exciting enough for a keynote. All of it is what actually determines whether the next bandwagon, Agentic AI, does anything an organization can trust, or just runs the same undocumented ETL job faster and with less supervision.

The safest bet in this industry has never been picking the right bandwagon. It's building the fact table well enough that it survives the next one.