The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the tag “#SQLDevModeler”

22 Dec 2016
3 Comments

Early Christmas: The New #SQLDev Data Modeler is Here!

Thanks to the gang at Oracle for an early Christmas present – the newest version of Oracle SQL Developer Data Modeler (SDDM) is ready for download and use.

The best FREE data modeling tool on the planet just got better!

To be clear this is Early Adopter (EA) version 2 of SDDM 4.2. You can get it here right now!

#SQLDev Data Modeler New Features

Of course there are some bug fixes from EA1, but also some new features for you to enjoy:

Import from Oracle Database

  •   performance and filtering enhancements
  •   ability to define Oracle Client for thick connections
  •   view and materialized view driving query and columns now parsed and validated

Versioning

  •   improvements in performance
  •   new models are shown as a single node in pending changes window

Reporting

  • PDF reports allow diagrams to be embedded with links from diagram to details part into report
  • HTML report for tables now include diagrams

 

SQL Developer Data Modeler EA2 adds diagrams to HTML reports

#SQLDev Data Modeler HTML report with diagrams embedded

So go download and unwrap that present!

Cheers!

Kent

The Data Warrior

P.S. If you need training on Oracle Data Modeler, be sure to check out my online video training course along with my tips and tricks ebook. (HINT: Buy them now, and you may be able to deduct the cost from your 2016 taxes as an educational expense.)

Posted in Oracle, SQL Developer Data Modeler and tagged , , , , , , , , , , , , ,
31 Aug 2016
10 Comments

Maintaining disabled FK’s, wisdom or farce?

A while back, I wrote a post about having FKs (foreign keys) in your data warehouse.

Well, a similar question came up recently on an Oracle forum with the above title. It is a fair question and it does surface fairly regularly in a variety of contexts (not just data warehousing).

Of course, as The Data Warrior, I felt is was my duty to respond.

The Question

Is there any reason to maintain a permanently disabled FK in the data model?  I’m not envisioning a reason to do it.  If it is not going to be enabled, then from my perspective, it would not make any sense to have it defined.  If anything, provide the definition of the relationship in the comment of the child column.

My Answer

Yes, by all means keep the FK please!

I see three good reasons for doing so:

  1. It is valuable metadata (& documentation). If somebody reverse engineers the database (say with ERWin or Oracle Data Modeler), the FK shows up in the diagram (way better than having to read a column comment to find out)
    Data Vault 2.0 Example

    A picture is worth a thousand words!

    .

  2. BI Metadata – If you want to use any sort of reporting or BI tool against the database, most tools will import the FK definition with the tables and build the proper join conditions. Way better than having someone guess what the join will be and then manually adding it to the metadata layer in the reporting tool. Examples that can read the Oracle data dictionary include OBIEE, Business Objects, COGNOS, Looker, and many others.(Note here that since the FK is not enforced on the remote databases, you might want to make sure these are treated as outer joins, lest you lose some transaction in the reports).
  3. The Oracle optimizer will use disabled constraints to improve query performance of joins. Again, this is metadata in the data dictionary which the optimizer can read. This is documented in the Oracle Data Warehouse guide and I have validated it on multiple occasions with Oracle product management.

While #3 applies specifically to Oracle, for other databases like MS SQL Server and Snowflake, #1 and #2 still apply.

Even if only one of the above is true for a given database, that, in my opinion, still justifies keeping the disabled constraint around.

Final Answer = Wisdom

What do you think? Feel free to comment below.

And please share on your favorite social media platform!

Model on!

Kent

The Data Warrior

 

Posted in Data Modeling and tagged , , , , , , , , , ,
15 Aug 2016
12 Comments

One more time: Do we still need Data Modeling?

More specifically do we still need to worry about data modeling in the NoSQL, Hadoop, Big Data, Data Lake, world?

This keeps coming up. Today it was via email after a presentation I gave last week. This time the query was about the place of data modeling tools in this new world order.

Bottom line: YES, YES, YES! We still need to do data modeling and therefore need good data modeling tools and skills.

Snowflake with RI

A picture can say so much!

 

In order to get any business value out of the data, regardless of where or how it is stored, you have to understand the data, right?

That means you have to understand the model of the data. Even if the model (or schema) is not needed upfront to store the data (schema-on-write), you must discern the model in order to use it (schema-on-read).

It is (mostly) impossible to get repeatable, auditable metrics, KPIs, dashboard, or reports that bring value to the business without understanding the semantics of the data – which means you at least need a conceptual or logical model.

And if you want/need to join data from multiple source then you really have to understand each source or there is no way to properly join it all together to get meaningful results.

There are a few data cleansing, discovery,and “virtualization” tools out there that will help you figure out those relationships but they are expensive and mostly rely on standard data profiling techniques to find similar data objects across the sets and propose “relationships”. Some allow for the definition of fairly sophisticated matching rules including customizations. But a human still needs to figures those out, test, and validate the results.

In the end you still have to know your data.

One of the best ways to do that, in my opinion, is to model that data. Otherwise your data lake will likely become a data swamp!

So keep your data modeling tool and keep building your data dictionary with your business folks.

Final Stage Table

A good modeling tool can act as a visual data dictionary too!

If you agree with me, please share on social media!

#LoveYourData

Kent

The Data Warrior

P.S. If you need a good modeling tool, check out Oracle SQL Developer Data Modeler. And check out my books and training offering for SDDM on the blog sidebar.

Posted in Big Data, Data Modeling and tagged , , , , , , , ,
30 Apr 2016

Best #DataVault Event Ever!

Hard to believe it is less than 25 days to WWDVC 20016. I’m going. Are you?

If not, why not?

The 3rd Annual World Wide Data Vault Consortium is going to be epic!

What is the WWDVC?

It’s the only conference that focuses on everything happening in the Data Vault world, and this time, the keynote speaker is the father of Data Warehousing himself Bill Inmon. He created the industry in which we earn our daily bread.

The sessions start prior to the conference with a closed room meeting of Data Vault certified brainstorming. I will be there for sure to talk about Data Vault in the Cloud and of course experiences in virtualizing layers of the data vault architecture. I can hardly wait to discuss what I have learned this year with the other data vault brainiacs.

This is followed by a day of hands on workshops:

  • How to build a DV, a Data Mart, and End User Analytics from Scratch in 3 hours

  • Extend the power of the Data Vault to a real-time use case using a Spark based Lambda architecture

  • Generate code using the award winning Mapping Manager tool

And, these are all before the conference even starts.

There’s a lot more during the conference including the famous networking sessions which this event is known for. If you’ve already booked your seat, see you there. If not, what are you waiting for?

For something so valuable, you would expect a hefty price tag like the vendors do it.

Nope, it’s dirt cheap.

Check it out for yourself here:

WWDVC Registration

It’s a conference for Data Warehouse people with a Data Vault focus. Folks like you. It does have many things for the business person as well such as selling the Data Vault concept to the business owner (Peter Aiken), case studies including implementation of an Near Real Time DV 2.0 in the cloud, and a customer centric analytics case study, and more.

Give Your Brain a Treat

Some of the smartest people in Data Warehousing I know, have been in attendance in the past. This is a once a year chance to meet them in person, increase your network, and have a great time

This time it appears to have been kicked up a notch with the likes of John Giles (Universal Data Vault), Peter Aiken (In one instance their DV 2.0 helped save a client a whopping $25M/year), Bill Inmon (The father of the DW), Dale Anderson (Used the DV to build one of the world’s first DaaS).

There are also the repeat attendees like Mary Mink and Sam Bendayan (Used DV 2.0 to implement an NRT DW sourced from a NoSQL platform serving 3000+ customers), Michael Olschimke (Co-author of the new DV 2.0 book), Roelant Vos (Not only did he build his own automation, he likes to experiment … with architecture).

And … vendors of automation tools. Several of the world’s leading Data Warehouse Automation tools will not only be there, they are sponsoring (and buying dinner and drinks!).

And as if that is not enough, you will get to rub elbows with Dan Linstedt, the inventor of Data Vault, Sanjay (co-founder of LearnDataVault.com), and of course me, Kent, The Data Warrior.

So what are you waiting for? Go buy your ticket to the conference, book your room, and figure out how you will get there (hitch hike if you have to).

See you in Vermont!

Kent

The Data Warrior

P.S. Don’t forget Saturday at WWDVC is crazy shirt day.

P.P.S. The company I work for, Snowflake Computing, is a sponsor and is giving me a GoPro! to raffle off, so, don’t miss my talk on Thursday if you want in on the raffle.

 

 

Posted in Big Data, Data Modeling, Data Vault, Data Warehouse, SnowflakeDB and tagged , , , , , , , , , | Leave a comment
02 Feb 2016

The Data Warrior Speaks 2016: Updated

As expected, I have been booked to speak a few more places this year.

Here is my updated speaking schedule as of today:

RMOUG Training Days 2016 – February 9-11 in Denver, CO (I have 2 hour deep dive on Feb 9th). Register here.

TDWI Nashville – March 8th in Nashville (of course). I will be discussing how to apply Agile Methods to Data Warehousing. You can get more details (soon) and sign up here.

Tampa Analytics Professionals – March 22 at the St Pete College Epicenter. Again talking about how to apply Agile Methods to Data Warehousing. You can get details and sign up here.

Agile Alliance Technical Conference 2016 – April 7-9 at the Raleigh Marriott Crab Tree Valley in Raleigh, North Carolina. I will present Agile Data Engineering: Introduction to Data Vault Data Modeling on Thursday April 7th. The Super Early Bird and Early Bird rates are still available. Register here.

Enterprise Data World – April 17-22 at the Sheraton Marina in San Diego, California. Register early for discounts. My talk here will be Agile Data Warehousing: Building a Virtualized ODS.

Data Science Maryland Meetup – May 16th (Tentative). I expect to be talking about how to apply Agile Methods to Data Warehousing.Keep your eyes on the meetup page for details and to sign up.

World Wide Data Vault Consortium (WWDVC) – May 25-28 in Stowe, Vermont. I am now confirmed to speaking at WDVC for the 3rd time! And this year, Snowflake Computing will also be a sponsor. My talk this year will be Agile Data Warehousing: Building a Virtualized ODS with Oracle SDDM. Register here soon as this event has limited space.

ODTUG KScope16 – June 26-30 in Chicago, IL. Register early and be sure to book the hotel! My talk this year will be Data Warehousing in the Real World. I will also be running my annual Morning Chi Gung sessions.

And that is the first half of the year. I have nothing confirmed yet for the 2nd half, but am sure something will pop up.

Stay tuned.

I look forward to see y’all at one of these events.

Kent

The Data Warrior

P.S. I will also be working the Snowflake booth at both the Gartner BI and Analytics Summit this March in Dallas, and the HIMSS event in Las Vegas at the end of February. Stop by and say “hi” if you will be at either of these events.

Posted in Big Data, Data Modeling, Data Vault, Data Warehouse, Oracle, SnowflakeDB, User Groups and tagged , , , , , , , , , , , , , , , , , , | Leave a comment

Post Navigation

← Older Entries
Newer Entries →
%d bloggers like this: