The Data Warrior

Changing the world, one data model at a time. How can I help you?

Adventures in the Cloud

I just got back from a whirlwind tour that included a trip to San Francisco to visit the Snowflake HQ, attend the annual Oracle ACE Directors briefing, meetings and events at Oracle OpenWorld (OOW), speaking at OakTable World (OTW), then off to Chicago to speak at Enterprise Dataversity.

Yes it was packed. And lots of talk about cloud everything!

 

Thanks to the great folks at OTN (Roland and Jennifer) for hosting us again this year at Oracle HQ.

Here are a few highlights:

Morning Chi Gung

Now an annual event is doing Chi Gung by the lake behind the Sofitel with a few ACE Director friends before the *top secret* briefing.

morning-chigung

 

Pre-OOW Athletics

Well, sort of. This year I walked the Golden Gate Bridge (5K) and took a dip in SF Bay in the 5th Annual Swim The Bay. Proud to say I have attended all 5 years of that event.

Nothing like a brisk morning walk to start the day!

Nothing like a brisk morning walk to start the day!

 

Then there is the brisk swim!

Then there is the brisk swim!

And this year, I got a pretty good size group to do Chi Gung on the beach after the swim (to warm ourselves back up!)

Time to get that chi flowing after the swim!

Time to get that chi flowing after the swim!

OTW 2016

I think this is the 3rd time I have presented at the un-conference run in parallel to OOW. But this year was the 1st time I did it as an Oaktable member! Gave my talk on Building a Virtualized ODS using SQL Developer Data Modeler.

No picture from that talk (sorry) but I did a very short Chi Gung set with the attendees at lunch time to get them moving out of their seats.

 

OTN Interview

This was the best part – getting to do an interview with my good friend Laura Ramsey from OTN about data modeling and data warehousing in the cloud. Check out my full interview here.

otninterviewoow16

Live cast interview from the Moscone at OOW 2016

 

The Drake Hotel

For the Dataversity event in Chicago, I got to stay at the very classy Drake Hotel right on the lake shore with a great view.

Awesome View!

Awesome View!

thedrake

Oh, and my talk on Data Warehousing in 2016 and Beyond went great. I learned a lot in the various keynotes and even got an introduction to Data Science.

 

Well that is my little update for now.

Check my speaking page to see where I will be next!

Ciao.

Kent

The Data Warrior

 

 

 

Maintaining disabled FK’s, wisdom or farce?

A while back, I wrote a post about having FKs (foreign keys) in your data warehouse.

Well, a similar question came up recently on an Oracle forum with the above title. It is a fair question and it does surface fairly regularly in a variety of contexts (not just data warehousing).

Of course, as The Data Warrior, I felt is was my duty to respond.

The Question

Is there any reason to maintain a permanently disabled FK in the data model?  I’m not envisioning a reason to do it.  If it is not going to be enabled, then from my perspective, it would not make any sense to have it defined.  If anything, provide the definition of the relationship in the comment of the child column.

My Answer

Yes, by all means keep the FK please!

I see three good reasons for doing so:

  1. It is valuable metadata (& documentation). If somebody reverse engineers the database (say with ERWin or Oracle Data Modeler), the FK shows up in the diagram (way better than having to read a column comment to find out)
    Data Vault 2.0 Example

    A picture is worth a thousand words!

    .

  2. BI Metadata – If you want to use any sort of reporting or BI tool against the database, most tools will import the FK definition with the tables and build the proper join conditions. Way better than having someone guess what the join will be and then manually adding it to the metadata layer in the reporting tool. Examples that can read the Oracle data dictionary include OBIEE, Business Objects, COGNOS, Looker, and many others.(Note here that since the FK is not enforced on the remote databases, you might want to make sure these are treated as outer joins, lest you lose some transaction in the reports).
  3. The Oracle optimizer will use disabled constraints to improve query performance of joins. Again, this is metadata in the data dictionary which the optimizer can read. This is documented in the Oracle Data Warehouse guide and I have validated it on multiple occasions with Oracle product management.

While #3 applies specifically to Oracle, for other databases like MS SQL Server and Snowflake, #1 and #2 still apply.

Even if only one of the above is true for a given database, that, in my opinion, still justifies keeping the disabled constraint around.

Final Answer = Wisdom

What do you think? Feel free to comment below.

And please share on your favorite social media platform!

Model on!

Kent

The Data Warrior

 

One more time: Do we still need Data Modeling?

More specifically do we still need to worry about data modeling in the NoSQL, Hadoop, Big Data, Data Lake, world?

This keeps coming up. Today it was via email after a presentation I gave last week. This time the query was about the place of data modeling tools in this new world order.

Bottom line: YES, YES, YES! We still need to do data modeling and therefore need good data modeling tools and skills.

Snowflake with RI

A picture can say so much!

 

In order to get any business value out of the data, regardless of where or how it is stored, you have to understand the data, right?

That means you have to understand the model of the data. Even if the model (or schema) is not needed upfront to store the data (schema-on-write), you must discern the model in order to use it (schema-on-read).

It is (mostly) impossible to get repeatable, auditable metrics, KPIs, dashboard, or reports that bring value to the business without understanding the semantics of the data – which means you at least need a conceptual or logical model.

And if you want/need to join data from multiple source then you really have to understand each source or there is no way to properly join it all together to get meaningful results.

There are a few data cleansing, discovery,and “virtualization” tools out there that will help you figure out those relationships but they are expensive and mostly rely on standard data profiling techniques to find similar data objects across the sets and propose “relationships”. Some allow for the definition of fairly sophisticated matching rules including customizations. But a human still needs to figures those out, test, and validate the results.

In the end you still have to know your data.

One of the best ways to do that, in my opinion, is to model that data. Otherwise your data lake will likely become a data swamp!

So keep your data modeling tool and keep building your data dictionary with your business folks.

Final Stage Table

A good modeling tool can act as a visual data dictionary too!

If you agree with me, please share on social media!

#LoveYourData

Kent

The Data Warrior

P.S. If you need a good modeling tool, check out Oracle SQL Developer Data Modeler. And check out my books and training offering for SDDM on the blog sidebar.

Where is the Data Warrior Now?

Hi folks. Time to update y’all on some upcoming speaking engagements for this summer and fall.

Here are a few talks I have scheduled:

Houston DAMA 

When: August 9, 2016 1:30 PM

WhereBP Westlake Campus, Helios Plaza, Houston, TX

Topic 1: Harnessing the Elasticity of the Cloud for Analytics with Snowflake and Tableau

Topic 2: Agile Data Warehousing – Building a Virtualized ODS

Register: Houston DAMA Meetup

Agile Denver: BI Meetup #1 

When: August 17, 2016 5:30 PM

Where: Charles Schwab, Lone Tree, CO (Denver Tech Center)

Topic: Agile Data Warehousing – Building a Virtualized ODS

RegisterAgile BI Meetup

Agile Denver: BI Meetup #2

When: August 18, 2016 5:30 PM

Where: WebPT, Westminster, CO

Topic: Agile Data Warehousing – Building a Virtualized ODS

RegisterAgile BI Meetup

Enterprise Dataversity

EDV2016-ImSpeakingWhen: September 21, 2016 8:30 AM

Where: The Drake Hotel, Chicago, Ill

Topic: Data Warehousing in 2016 and Beyond

Register: Enterprise Dataversity

 

Utah Oracle Users Group (UTOUG) Fall Symposium

When: October 26, 2016

WhereSalt Lake Community College Miller Campus, SLC, Utah

Session #1Agile Data Warehousing: Using Oracle Data Modeler (SDDM) to build a Virtualized ODS

Session #2: Agile Methods and Data Warehousing: How to Deliver Faster

Session #3: Data Warehousing in the Real World

Register: UTOUG Registration – Coming Soon!

 

Southern Fried Agile

When: October 28, 2016

Where: Charlotte Convention Center, NC

Topic:  Agile Methods and Data Warehousing: How to Deliver Faster

Register: SFA Registration

 

East Coast Oracle (ECO) Users Conference

When: November 2 & 3, 2016

Where: Hilton North Raleigh/Midtown Hotel, Raleigh, NC

Topic #1Agile Data Warehousing: Using Oracle Data Modeler (SDDM) to build a Virtualized ODS

Topic #2: Data Warehousing in 2016

Register: ECO Registration

 

There are more talks in the planning process so be sure to check back.

Hope to see you at one of these events!

Love your data!

Kent

The Data Warrior

Data Warrior LLC

Quote of the Day: Mark Twain

It is better to deserve honors and not have them

        than to have them and not deserve them.

— Mark Twain

Post Navigation

%d bloggers like this: