The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the category “Data Warehouse”

21 Oct 2012

Standards? We Don’t Need No Stinking Standards!

Well, actually we do need standards.

Especially if we want to have any consistency in the systems we develop, or the models we build.

For years people in the data warehouse arena have literally begged Dan Linstedt, inventor of the Data Vault Model and Methodology, to create books and training materials on Data Vault.

They wanted to know how he got the results he was getting for his clients.

They wanted to understand how to properly build a Data Vault.

They wanted STANDARDS.

Well, ask and ye shall receive.

Recently Dan put this on his blog:

For many many years I have written and maintained Data Vault Standards v1.x.  Well, I’ve released them on Amazon for you.  These are the DV1.0 Standards, and are the same standards document I used to hand out in my certification classes.

Apparently there are folks out there who either don’t know about the standards, or who have had some confusion over the fact that there ever were standards.

I wanted to make it free – but unfortunately I was not able to do that.  So, I’ve made the price to be $0.99 USD on Amazon.  Again, these are Data Vault 1.0 Modeling Standards and Data Vault 1.0 Loading Standards.

via Data Vault Modeling & Methodology – Data Vault 1.0 Modeling and Loading Standards.

So there you have it – the official standards for Data Vault 1.0.

Go get ’em here.

Read ’em.

Use ’em.

Your data warehouse will thank you.

Kent

Oracle Data Warrior

Posted in Data Modeling, Data Vault, Data Warehouse and tagged , , , , , | Leave a comment
04 Oct 2012

Oracle OpenWorld 2012: Day 4

It was another beautiful, sunny day in San Francisco. I started the day, again, with some morning Chi Gung, then I enjoyed the morning keynote watching the big screen in Yerba Buena Gardens. Quite a pleasant way to listen to these talks.

It was a light day session-wise for me, but it did set off a few light bulbs.

The best session of the day (and for me the whole conference), was Gwen Shapira’s  (from Pythian) talk on building an integrated data warehouse with Hadoop. Gwen did a superb job of explaining what Big Data is and what it isn’t.

Her simple, and straightforward, definition:

Big Data Defined

The “cheaply” part seems to be the key. Oracle, and other databases, can handle really HUGE amounts of data. Petabytes in fact. But putting all that data into an RDBMS can cost a lot more money than having it stored in a less sophisticated file system on commodity drives (like HDFS).

So just having lots of data in your warehouse does not mean you have Big Data, you just have a Very Large Data Warehouse (VLDW).

She went on to expand the definition:

Big Data Defined 2

This part shed even more light on Big Data for me. This really helped clarify even more when you might be dealing with Big Data.

The talk was filled with lots of technical details, limitations,  and tools ( Sqoop, Flume,  Fuse-DFS) you can look at for integrating Hadoop into Oracle. Of course there are Oracle’s offerings as well, like Oracle Loader for Hadoop and Oracle Direct Connector for HDFS.

Gwen also gave use several use case examples that illustrated when to use Hadoop. Bottom line – learn to use Hadoop appropriately, not just because it is cool. With tech we can:

Make the impossible, possible. That might no make the possible easy.

If you went to OOW, find and download Gwen’s slides. And follow her on twitter (@gwenshap).

It was Big Data day for me. My other session was  Ian Abramson’s session on Agile & Data. Two of my favorite topics.

Ian discussed the Agile Manifesto and Big Data and how he has been able to use agile techniques to make his projects successful.

To start, here is his simple definition on Big Data:

What is Big Data?

Ian had a nice picture of the overall architecture as well:

Another Big Data Picture

To be successful in applying agile to data projects, Ian has determined the projects must be driven by data value – that is the sprint priorities are set based on the data that can best help the customer achieve their goals. To stay on track and keep velocity, it is important to have daily touch-points with the team members as well. Ian does a daily stand-up for 15 minutes.

Ian shared lots of details and answered a lot of my annoying questions too. He came up with a great tree graphic to illustrate important factors in having a high performance project:

Agile Tree

Again, find and download the slides once Oracle uploads them. In the meantime, follow Ian on twitter (@iabramson). A data-centric agilest is hard to find. For more on agile and data warehousing check out my classic white paper on the subject.

After Ian’s session I got to go to my first Oracle blogger meet up. It was nice to put more faces to names. Thanks to Pythian and OTN for sponsoring it.

Blogger Meetup

Then back to the hotel to pack and then stand inline (for an hour!) to get to the appreciation event and see Pearl Jam live. It was a good concert. Hard to beat live music outdoors!

Huge crowd for Pearl Jam

Pearl Jam Live!

Well that’s it for me on OOW2012. I am back home in Houston now and heading into the office tomorrow. Then I need to write another abstract or two for KScope13 and RMOUG TD2013. Then it will be time to plan for OOW2013 and The America’s Cup finals…

Nap time.

Kent

Posted in Data Modeling, Data Warehouse, Oracle and tagged , , | Leave a comment
29 Jul 2012

Big Blunder means extended discount opportunity for Data Vault fans

Sometimes the best laid plans…they just don’t quite work out. Here’s what I mean:

If you were at my Data Vault session at the recent ODTUG KScope conference in San Antonio, you got a special code to enter at http://www.LearnDataVault.com/kscope12  to get $200 off the new online Data Vault Implementation training class. (For more on the training check out this post).

That deal was supposed to end this week on July 31st.

Well…

I was just informed that there was an error in the code on the BUY button.

Error in the code? Say it isn’t so! (Hey – I did not do the programming)

If you went to the link and put in the code you got some sort of an error message. If you forgot the code and just pushed the button you went to Dan’s offer to try the first module for only $7.00 (USD) – which is a really great deal on its own.

As you may imagine, I was not happy to hear that the Kscope participants were unable to get the discount I promised them and that there are only a few days left for this deal. 😦

So as compensation for this mistake, I was able to negotiate an extension of this offer until August 30th.

So that means you now have until August 30th to take advantage of the $200 discount.

Just go to http://www.LearnDataVault.com/kscope12 and put in the secret code. All the details about the course are there on the page for you to review.

And don’t forget, Dan offers a 7 day, 100% money back guarantee.

So if you have been putting off investing in this course, you just got a 30 day reprieve to get the $200 discount.

Don’t waste it. There are not that many 2nd chances.

Kent

P.S., Since speakers at KScope do not get attendee lists and emails, I have no way to directly contact the folks who attended my session. So do me a favor, help me get the word out. Please re-post and tweet this to your networks so we can be sure no one misses out. Thanks a bunch.

Posted in Data Vault, Data Warehouse, ODTUG, Oracle, User Groups and tagged , , , , , , | Leave a comment
28 Jun 2012
2 Comments

ODTUG KScope: Day 5 – Happy Trails

Well the final day of KScope12 finally arrived and it was another hot one with the final sessions and the Texas heat. Another bright red sunrise greeted us as it has all week.

image

Today I managed to get a picture of the group that showed up for Chi Gung every day at 7 AM. We even had some new people today (officially the last day). They all enjoyed the sessions and learned (hopefully) enough to practice a bit once they return home.

I am grateful to all the participants for showing up early each morning with enthusiasm and a willingness to try something new. It made my job to lead them much easier. (There will be a You Tube video sometime next week for people to review, so stay tuned)

The first order of business for the day (after Chi Gung) was the official KScope closing session. Even though there were still two sessions to go afterward we had the closing at 9:45 AM. We were entertained, yet again, with some photo and video footage taken throughout the week, including one interview with me! We also learned who got the presenter awards for each track and for the entire event.

Then we all got beads to remind us to go to KScope13 in New Orleans.

Next was my final session for the event: Reverse Engineering (and Re-Engineering) an Existing Database with Oracle SQL Developer Data Modeler.

I had a surprising number of people for the last day after the closing session. I think there was about 70 people wanting to learn more about SDDM. Apparently most people are unaware of the features of the tool (which I have written about on several posts).

So, that was nice.

Finally I went to JP Dicjks talk about Big Data and Predicting the Future.

His basic premise is that we should now never throw away any data as it all can be used to extend the depth of analytics. We can react to events in real time and proactively change outcomes of those events.

The diagram above shows the basics of one way that data moves through the world and into the Hadoop file systems. I am oversimplifying but it is a cool diagram.

Part of the challenge is uncovering un-modeled data. I guess that is where the recent Oracle acquisition, Endeca, comes in with their Data Discovery tool (again oversimplifying) .

And that was pretty much it for the show. It was a great week with lots of learning and networking (and tweeting). We all had a good time and learned enough to make our heads explode.

I look forward to meeting folks again next year at KScope13 in New Orleans.

Kent

Posted in Chi Gung, Data Warehouse, ODTUG, Oracle, SQL Developer Data Modeler and tagged , , , , , , , ,
27 Jun 2012
2 Comments

ODTUG KScope12: Day 4 – Another Day in Paradise

Well folks, it is really late/early so, for now I am just putting up some pictures without a lot of detail.

This was the sunrise that greeted me on the way to Morning Chi Gung. Going to be another hot one!

image

This is the view of the bluff and waterfall that we see every day when we practice Chi Gung on the lawn. Very soothing and relaxing. It definitely enhances the experience and feeling of connectedness to the earth.

First session of the day was Maria Colgan (the optimizer queen) talking about Tuning SQL in a Data Warehouse. A huge amount of information to digest. Mostly over my head but very useful for a data warehouse dba.

She did however forget her glasses this morning and could not really see the people in the back row too well. 🙂

Next up was Mr. Kevin McGinley (BI Track Lead) giving us his thoughts about Exalytics and what is meant by “Speed of Thought”.

This is a picture of his four kids before the session. They did Kevin’s introduction today. Quite cute.

Not sure where they got it from (just say’n). 😉

After Kevin’s entertaining talk, I went to see John Jeffries talk about Oracle Golden Gate. John is one of the world experts on Golden Gate having published the Oracle Golden Gate 11g Implementer’s Guide.

John had a nice diagram (below)  of what you can use Golden Gate for. Very useful.

After lunch I went to see Dr. Holger Friedrich who gave us a comparison between ODI 11g (Oracle Data Integrator) and OWB (Oracle Warehouse Builder). OWB is going away in the not too distant future so it is important for OWB shops to get a handle on it and start to lean about ODI. This presentation was a great start.

Holger is a very interesting and intelligent guy. He is from Switzerland and holds a PhD in Robotics and Machine Learning.

Not sure how he ended up doing Oracle data warehousing.

Tonight was our BIG EVENT: Dinner and Rodeo and Dancing at the Knibbe Ranch. It was hoot!

Here I am with my armadillo. I actually “won” an armadillo race.

We had a great BBQ dinner and a great country band to listen to. Hard find a bad band in this neck of the woods.

After dinner was the main event: rodeo. This is the big show: The board of directors for ODTUG got to ride into the arena on horseback for the opening ceremonies of the rodeo. It did appear that Edward Roske (conference chair) really knew how to ride a horse.

That’s it for now. I have to sleep a little before Morning Chi Gung. Then I have my last presentation tomorrow morning.

Good thing I tested everything and got ready before going to the special event.

Check back in a day or so as I will fill in some details on the technical presentations.

Adios for now.

Kent

Posted in Chi Gung, Data Warehouse, ODTUG, Oracle and tagged , , , , , , , ,

Post Navigation

← Older Entries
Newer Entries →