The Rocky Horror Data Show: Disastrous duplicates…
Data shouldn’t be a wild and untamed thing, but sometimes it is just that — wild… and untamed. And unfortunately for our friend Tim, he’s about to find out just how wild and untamed data can be. As this is ‘The Rocky Data Horror Show’… where the data is not what it seems.
If you missed the previous episodes, you can read them here:
The Rocky Data Horror Show — where the data is not what it seems
The Rocky Horror Data Show: Did you get what you asked for?
The Rocky Horror Data Show: Not everyone’s on board…
The Rocky Horror Data Show: Disastrous data definitions…
Fresh from developing their data glossary, there’s been a worrying development in The Magical Wish Factory. And it has left Tim and Janet quite perplexed. Over the last couple of weeks, it has become apparent that some subscribers of The Magical Wish Factory have been getting more than their monthly quota of wishes.
Tim and Janet have been investigating the issue — which has cost the MWF hundreds of wishes and a lot of profit so far — and have discovered that a data input error has resulted in several customers having duplicate records on the system, and therefore getting double or sometimes three time more than their quota of wishes.
“This is a disaster” wailed Janet, “This has cost us so much money and there’s so many wasted wishes!”
“It’s not great” conceded Tim, “but that’s why you brought me in all these months ago, because the Data Governance here isn’t where it needs to be… and I’m sure we can get is all sorted out!”
“We need to get the data cleaned as soon as possible and these duplicate records deleted!” stated Janet, quite firmly.
“Yes, the data does need cleaned, and the duplicate records need condensed and removed… but if we clean the data without addressing the source of the issue, then we’re going to end up in this mess again in a few months — and that’s quite counterproductive, Janet,” said Tim.
“Okay — so what do you suggest then?” asked Janet sceptically.
Tim explained to Janet that the Magical Wish Factory needs to get out the habit of constantly fighting fires and get to the source of ignition and deal with that. Yes, the data could just be cleaned and fixed at the point of use, but it is fundamentally the wrong place to start.
Tim says: “If we clean and fix the data but don’t address why it is wrong and figure out why it is happening in the first place it will be clean for a day or two and then immediately start to deteriorate again.”
“I found a person in my old job who used to spend three weeks manually fixing role codes in their HR system every six months. It became a regular process to fix it because they never fixed the source of the issue, think about all that time wasted!”
Tim went on to explain further that the Magical Wish Factory needs to continue to press ahead with its Data Governance initiative first as that will ensure that you look at things strategically instead of a mentality of fixing things tactically all the time.
Tim also explained that they needed to establish a master of all wish makers and that Data Governance would help them identify and fix the source of the issue, not just the resulting data at the time you use it!
“Ah, that makes a lot of sense…” said Janet.
“Yes, Data Governance first and foremost always… that’s my wish!” replied Tim.