The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the category “Data Warehouse”

Data Vault Informatica Class is Live!

Just a quick note to let you all know that Dan has finally released the class on how to easily implement a Data Vault using Informatica.

I wrote about the class here.

I have gone through a few of the lessons already and can tell you the instruction is very clear and easy to follow (even for me!) and the audio and video is excellent. The audio seems to come on a bit load so just be sure you have your volume turned down a bit when you start the videos.

And there is a money back guarantee if for some reason you decide the class is not for you.

If you did not get on Dan’s early notice list you can still sign up by going directly here: http://learndatavault.com/kentdvi

And, since you are a reader of my blog, if you sign up in the next few weeks and enter the coupon code DATAWARRIOR13, you can get $100 off !

So if you use Informatica and plan to do a Data Vault, you owe it to yourself to take a look at this course.

Take care.

Kent

Data Vault and the Oracle Reference Architecture

Thanks to Mark Rittman and Twitter, I found out just before RMOUG that Oracle had published a new reference architecture.  It used to be called the Data Warehouse Reference Architecture, now it is called the Information Management Reference Architecture.

Oracle Information Management Ref Architecture

Oracle updated the architecture to allow for unstructured and big data to fit into the picture.

In my talks about Data Vault over the last few years I have been referring to the Foundation Layer of the architecture as the place where Data Vault fits. The new version of the architecture actual fits the definition of the Data Vault even better.

Now the Foundation Layer is defined as “Immutable Enterprise Data with Full History”.

If that is not the definition of Data Vault, I don’t know what is!

Immutable – does not change. Data Vault is insert only, no update – ever.

Enterprise Data – well duh! That pretty well fits any real data warehouse architecture. The model covers an enterprise view of the data not just a departmental views (like a data mart).

Full History – tracks data changes over time. That is one of the keys to the data Vault approach. We track all the data change history in Satellites so we can always refer to a view of the data at any point in time  That allows us to build (or re-build) dependent data marts whenever we need or whenever the business changes the rules.

So it is possible to do a Data Vault approach and be compliant with Oracle’s reference architecture.

Guess Dan was just a bit ahead of the game…

Later

Kent

How to Use Informatica to Build a Data Vault

Yes, its true – you will soon be able to get online training on how to build a Data Vault data warehouse using Informatica.

Dan Linstedt has been working hard for several months now to put together some top notch training for all you who use Informatica.

Dan will teach you all his best practices for getting the job done quickly using Informatica for your ETL tool.

If you want in, it’s not too late to get on the VIP early notification (which entitle you some discounts). Get in on the list here.

Here are a few common questions that Dan recently answered:

Q. What are the pre-requisties?
A. You must know Informatica PowerCenter and Data Vault modeling basics. The training works with Informatica PowerCenter v8.x and v9.x, but the mappings will only import to version 9.x or higher.

Q. Do I need access to an Informatica installation?
A. You will, if you want to do any of the hands on portions. We can’t help you with this. They used to provide a limited developer edition with a devnet membership, but that seems to have been discontinued.

Q. Will I learn Informatica PowerCenter?
A. No! This course assumes, you have at least 3 months experience in Informatica and know the difference between mapping, session and workflow objects. If you’ve never worked with Informatica tools, then we recommend that you DO NOT invest in it.

Q. Do I need to know Data Vault Modeling?
A. Yes, and the knowledge in the book “Super Charge your Data Warehouse” is sufficient for the course. It’s better if you have  more hands on experience though.

Q. Would it benefit me if I’ve gone through  the Data Vault Implementation and Best Practices course?
A. Yes.

Want to know more? Check out this video that has more details about the class and what it covers.

That’s it for now.

Later.

Kent

RMOUG Training Days 2013 – Day 2

So on this 2nd and final day of the annual RMOUG Training Days event, I started out by attending an excellent session on Exadata for Oracle DBAs.

Even though I am not a DBA these days I thought it would be good for me to get a better understanding of Oracle’s engineered Exadata machine.

I feel very luck to have attended this session given by Oracle Technologist of the Year, and ACE Director, Arup Nanda. He had some of the best graphics and clearest explanations of the basic anatomy of an Oracle database I have ever seen or heard.

Technologist of the Year, Arup Nanda, Database Machine Administrator

Technologist of the Year, Arup Nanda, Database Machine Administrator

He gave some pretty detailed explanations of what he called the “magic” of Exadata and why it works so well. Arup even coined a new job title, which he claims for himself, DMA – Database Machine Administrator. Because Exadata is an engineered system, it contains database, storage, and networking all in one rack. This requires some skills beyond what most dbas have or are expected to have.

He gave us a nice break down based on his experience using Exadata.

Break down of skills needed to be a successful Exadata DMA

Break down of skills needed to be a successful Exadata DMA

After this talk I can see why he was give the awards. He really knows his stuff and how to communicate it. You can follow him on Twitter @ArupNanda and see for yourself.

Next I went to see my friend,and ACE Director, Galo Balda from Austin, Texas. He gave a very informative talk about Regular Expressions.

ACE Director, Galo Balda, doing his very first presentation

ACE Director, Galo Balda, doing his very first presentation

His presentation was very informative with easy to understand examples of how to write and use regular expression and associated metacharacters to do some pretty neat things with SQL. If you attended the conference be sure to download his slides. They will make a great cheat sheet.

You can follow him on Twitter @GaloBalda or go to his blog.

After a nice vegetarian lunch, I went to see Maria Coglan talk about using (or not using) hints in SQL and how it affects the optimizer. Last year at Kscope12, I attended one of her optimizer sessions and felt like my head would explode becuase of all the information she gave. She assured me this talk would not be as bad.

She was right. It was a very informative talk.

A full house to see Maria Coglan discuss Hints and the Optimizer

A full house to see Maria Coglan discuss Hints and the Optimizer

Her main message was to always use caution when using hints. You really need to understand what you are or doing or you could make your application or reports run worse rather than better.

Maria even explained how to work with applications that already have hints embedded in them.

Approach to ignoring hints in an existing application

Approach to ignoring hints in an existing application

Get her slides and follow her on Twitter @SQLMaria

After Maria’s session I did my final session for the event. I talked about my Top 10 favorite cool tools in SQL Developer Data Modeler. There were 30 or so people in attendance. Most of them even stayed through the whole talk!

Which is pretty good since I ran over my time. There was just so many tips and tricks to show.  I will put it up on SlideShare in the next few days.

The final session for the event that I attended was done by RMOUG President, my long time friend, Tim Gorman.

Tim talked about the various options for data compression in the Oracle stack.

Tim Gorman (in the shadows) giving the last talk

Tim Gorman (in the shadows) giving the last talk

Tim gave some pretty detailed explanations and tried to depict how compression works with some nice graphics. He also told us which ones cost additional license fees.

Data Lifecycle when using Compress for OLTP

Data Lifecycle when using Compress for OLTPut how have

For me, the most useful part was his explanation about how having columns at the end of the table allows a default sort of compression to take place. I had heard this a long time ago. It was the reason so many of us were taught to put all mandatory columns at the beginning of the table – it saves space. In recent years I have been told by various DBAs that the rule no longer applies or made sense.

They were wrong! Tim gave us a real world example of how putting populated columns at the end of a table cost a lot of extra space to be used.

I will be taking that tidbit of information back to the office for sure.

You can follow Tim on Twitter @timothyjgorman.

A side note about RMOUG: At lunch, Tim shared with the attendees an agenda from 1991 for the 2nd RMOUG Training Days. We now realize that we started this event in 1990 and next year will be the 25th anniversary! (I say “we” because I was part of the planning committee back then and one of the early speakers too).

Another interesting notes was that 2/3 of the speakers came form out of town. Many, including me, paid there own way. Several speakers and attendees I know even had to take vacation time from their jobs to attend.

It is that important and that good an event!

So put it on your calendar to attend what is probably the most successful and longest running regional Oracle user conferences in the country. It will be in early February 2014. Watch www.rmoug.org for details.

And of ocurse count on me to post it here too.

Ciao for now! I am off to ski with some RMOUGers tomorrow.

Kent

RMOUG Training Days 2013 – Day 1

Unlike many conferences, today started off not with the keynote but with an actual session (probably some advanced psychology at work here). 🙂

I started off with John King’s session on Oracle 11g features that developers should know about. (He was going to talk about 12c but since it has not been released yet, he could not speak about it)

John King giving Session 1 at RMOUG 2013

John King giving Session 1 at RMOUG 2013

John is a great speaker and gave us some very detailed information.

One very interesting piece to me, as a data modeler and data warehouse designer, was the addition of Virtual Columns. With this you can declare a virtual, calculated/derived column to be part of a table definition. With this you can define a calculation once and have it appear when querying the table without actually physically adding a column to the table. Looks promising.

John told us about lots of new things like Pivot, Unpivot, Results Cache, PL/SQL Results cache and Nth Value functions. Some of them are shown in the following pictures.

SQL PIVOT Example

SQL PIVOT Example

Example of UNPIVOT

Example of UNPIVOT

Another cool SQL Function: Nth Value

Another cool SQL Function: Nth Value

All neat options I did not really know about.

Next up was the keynote speech by Mogens Norgaard from Denmark. Mogens is an ACE Director, CEO of his own consulting firm, and a brew master. Interesting guy.

He showed up in his bathrobe to talk to us all about how the smartphone is taking over  the world and all the cool apps you could build (and some he has built).

Mogens Norgaard in his keynote best.

Mogens Norgaard in his keynote best.

Next was my turn – my first session of the conference – 5 Ways to Make Data Modeling Fun (based on a blog post).

I was pleasantly surprised that I had 40-50 people attend and most stayed for the whole talk. It was a good, interactive session. My good buddy Jon Arnold assisted me in administering some of the activities. It was great fun getting the attendees to actually collaborate on activities during a session.

Great participant collaboration during my talk

Great participant collaboration during my talk

As promised, I did give out prizes for some of the activities (all branded Data Warrior LLC stuff).

Next was the ACE Director networking lunch where they put our names on tables so people could sit with us to ask questions (if they wanted too).

Networking Lunch

Networking Lunch

After lunch we some vendor sessions (which I skipped) and several panel discussions. These included the Women in Technology Panel and an Oracle Career Roundtable.

Women in Technology Panel

Women in Technology Panel

Oracle Careers Roundtable

Oracle Careers Roundtable

Anyone notice that the Women in Tech had one male on the panel but the Oracle Career panel had no women? Just sayin’ folks…

Next I sat in for part of a session on Oralce TimesTem database for real-time BI. It turned out to be the same stuff I heard at Oracle Open World so I did not stay.

Last for my day at RMOUG was my joint session with Stewart Bryson on Data Vault and OBIEE. Unfortunately due to the late slot (5:15 PM) we had a very low turn out. 😦 But is was a good session as I discovered all the things Stewart learned trying to use the data vault model for virtualizing the data mart layer (in OBIEE). It was all very good and reinforced my belief that Data Vault is a great way to model an EDW and that non-data vault people could understand it and apply it to dimensional modeling (or that Stewart is really exceptional).

Adios for now.

Kent

P.S. Forgot to mention again that I will be conducting another morning Chi Gung class at & AM above the registration area. Please join!

Post Navigation