Data Vault Certification – Day 2
Still really cold here in Montreal…
But the classroom discussions heated up a bit today as we dove deeper into the secrets of the Data Vault. Today we got into the nitty-gritty of things like Point in Time (PIT) tables, Bridge tables, DV loading, DV agility, set logic, and designing for performance. We got into some brand new material that has not been in prior certification class. All great stuff.
Dan sure knows a lot about a lot of things (hardware, software, operating systems, disk blocks, cache, etc.). His broad knowledge and experience definitely contributed to what is now Data Vault. We got to hear several juicy stories about real world issues he encountered over his illustrious career that lead him to architect Data Vault to have all the neat features it has (hint: lots of unnamed 3-letter gov’t agencies were apparently involved). Dan is bound and determined to help as many people as possible avoid the many pitfalls he has seen in data warehouse architectures.
What I learned today:
I FINALLY understand the legitimate use for a Bridge table and why it is a good idea (i.e., better query performance getting data out of a DV). The examples in the class got through to me (this time). It is all spelled out in the book, but now I really do get it.
ETL Golden Rule (for performance improvement):
1 process per target per action (i.e., Insert, Update, Delete)
In other words don’t make your ETL too complicated by trying to account for everything in one routine. It becomes too complex to maintain over time and it reduces the amount of parallelization you can do (hence it is SLOWER).
Think about it – new rows get inserted, right? So why waste the cycles forcing them through update logic? In a Data Vault structure it is very easy to separate these out. The SQL is pretty simple actually (but that is another post by itself).
Dan will be teaching more about this in his upcoming online Implementation training course. Stay tuned for that.
Data Vault Data Load Golden Rule:
100% of the data loaded 100% of the time
That really makes the load process pretty easy and streamlined. It is correlated to the Data Vault concept of loading the facts – straight from the source, no modifications. This is why the Data Vault is 100% audit-able.
So for Day 3 we will get even more into how to load a Data Vault.
And then there is the TEST (insert dramatic music here).
We get four hours. 60 questions (half of which are essay!).
Guess I better get studying!
Look for my Day 3 wrap up tomorrow night (assuming I can still write by then).
See ya.
Kent
The Data Warrior 