The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the tag “agile”

Agile Amped Video – Live from Southern Fried Agile

A few weeks back I had the pleasure of attending and speaking at the Southern Fried Agile event in Charlotte, North Carolina.

It was quite an event with over 700 people in attendance. This is a primarily local event drawing people mostly from the Charlotte area. To say agile is big in that town is an understatement. There were some very large, name brand, companies present. If you are interested in learning more about taking an agile approach to development, I can highly recommend this event which draws excellent speakers (and vendors) from all over. (And the traditional southern fried lunch was pretty darn good too!)

southernfriedlunch

My talk was on applying agile methods to data warehousing projects. I had a great audience of about 50 who wanted to learn how they might approach their DW projects differently.

In addition to giving my talk I got the opportunity to participate in my 2nd Agile Amped interview. This time my interviewer was Leslie from SolutionsIQ. We talked a lot about the rise of agile in the data warehousing space and how new technology solutions like Snowflake’s Cloud-Native Elastic Data Warehouse are helping organizations achieve a more agile deployment cycle.

Here is the entire interview for your viewing pleasure:

If you want to watch more Agile Amped video podcasts you can find them online here.

Enjoy.

Kent

The Data Warrior

Advertisements

Where is the Data Warrior Now?

Hi folks. Time to update y’all on some upcoming speaking engagements for this summer and fall.

Here are a few talks I have scheduled:

Houston DAMA 

When: August 9, 2016 1:30 PM

WhereBP Westlake Campus, Helios Plaza, Houston, TX

Topic 1: Harnessing the Elasticity of the Cloud for Analytics with Snowflake and Tableau

Topic 2: Agile Data Warehousing – Building a Virtualized ODS

Register: Houston DAMA Meetup

Agile Denver: BI Meetup #1 

When: August 17, 2016 5:30 PM

Where: Charles Schwab, Lone Tree, CO (Denver Tech Center)

Topic: Agile Data Warehousing – Building a Virtualized ODS

RegisterAgile BI Meetup

Agile Denver: BI Meetup #2

When: August 18, 2016 5:30 PM

Where: WebPT, Westminster, CO

Topic: Agile Data Warehousing – Building a Virtualized ODS

RegisterAgile BI Meetup

Enterprise Dataversity

EDV2016-ImSpeakingWhen: September 21, 2016 8:30 AM

Where: The Drake Hotel, Chicago, Ill

Topic: Data Warehousing in 2016 and Beyond

Register: Enterprise Dataversity

 

Utah Oracle Users Group (UTOUG) Fall Symposium

When: October 26, 2016

WhereSalt Lake Community College Miller Campus, SLC, Utah

Session #1Agile Data Warehousing: Using Oracle Data Modeler (SDDM) to build a Virtualized ODS

Session #2: Agile Methods and Data Warehousing: How to Deliver Faster

Session #3: Data Warehousing in the Real World

Register: UTOUG Registration – Coming Soon!

 

Southern Fried Agile

When: October 28, 2016

Where: Charlotte Convention Center, NC

Topic:  Agile Methods and Data Warehousing: How to Deliver Faster

Register: SFA Registration

 

East Coast Oracle (ECO) Users Conference

When: November 2 & 3, 2016

Where: Hilton North Raleigh/Midtown Hotel, Raleigh, NC

Topic #1Agile Data Warehousing: Using Oracle Data Modeler (SDDM) to build a Virtualized ODS

Topic #2: Data Warehousing in 2016

Register: ECO Registration

 

There are more talks in the planning process so be sure to check back.

Hope to see you at one of these events!

Love your data!

Kent

The Data Warrior

Data Warrior LLC

It WAS the #Best #DataVault Event Ever!

Last week I had the pleasure of spending a few days in lovely Stowe, Vermont at the Stoweflake Mountain Resort and Spa attending the 3rd Annual World Wide Data Vault Consortium (#WWDVC). Not only was the location picturesque, the weather was near perfect, the beer was tasty, and the learning and networking were outstanding.

We had 75 attendees coming from all over the world – Germany, Switzerland, Canada, England, Australia, New Zealand, The Netherlands, USA, Finland, and India. Quite a turnout!

20160523_190630.jpg

20160523_190617.jpg

Day 1- Data Vault Brainstorming

This year I arrived early enough to participate in what is arguably the best part of the event – a full day, open forum discussion with certified Data Vault modelers and practitioners, lead by the inventor of Data Vault, Dan Linstedt.

The brain power in the room was stunning. There were about 30 people in all and we all got to introduce ourselves and talk a bit about what we had been doing with Data Vault. It was great to hear the many and varied ways in which Data Vault is being used across multiple industries (including a US intelligence agency – but that is a secret). Everything from traditional data warehousing and BI, to realtime streaming IoT data, to virtual Data Vaults and virtualized information marts, to using Data Vault to help with Master Data Management (MDM). It was eye opening and exciting to hear all these applications and opportunities.

If you are not yet certified, get certified! Then you can attend this session at WWDVC 2017 (spoiler – at Stoweflake again!). And you are in luck as Dan just announced three classes later this year in St Albans, Vermont. Plus there are multiple classes coming up in Europe as well.

Day 2- Hands on Workshops

Another unprecedented day at WWDVC.  The three platinum sponsors, AnalytixDS, Talend, and Varigence, all ran 3-hour hands on workshops. These were a fantastic opportunity to see how these vendors have really stepped up to the plate to support quickly building Data Vault solutions with their tools.

These were great sessions, led but highly qualified folks. They showcased some great solutions and answered a lot of questions.

All three sessions were standing room only – with over 35 attendees. (We had to drag in chairs from other rooms!)

Be sure to make time to attend these next year as I am sure they will be on the agenda again.

Day 3 – The Main Event Begins

Yes, all the way to Day 3 before the official kickoff with keynote and speakers.

Dan of course got us started with welcome, thanks to all the sponsors, and housekeeping. Nicely this event only has one room and one track so no one has to pick between sessions!

20160526_080206.jpg

Keynote

The keynote this year was Swimming in the Data Lake by none other than the Father of Data Warehousing, Bill Inmon. I greatly enjoyed his somewhat irreverent look at our industry and his discussion on Big Data and the Data Lake concepts. It was quite a humorous talk (“I don’t mean to offend anybody, but….”). I would say it is one of the best talks I have ever heard Bill give over my 20+ years of knowing him (so I have heard a few).

WWDVC_BillInmon_DataLake

And being a prolific author, Bill of course has a new book out on Data Lakes (available now on Amazon here).

WWDVC_BillInmon_DataLakeBook

Being good geeks, several of us did manage to get our picture taken with Mr. Inmon as “social evidence” that we know him (well, I actually did co-author a book with him back in the day).

Kent with Bill 2016

Lots of Talks

Yes it was a full day with tons of stuff to fill our heads with ideas: new, useful and occasionally controversial. (stay tuned for videos on all these!)

Dan’s business partner, Sanjay Pande, came all the way from India to talk about Data Vault 2.0 on Hadoop. Roelant Vos came again from Australia to give us a business based view of a data vault project at his company (Allianz) about Customer Centric Analytics. Mary Mink and Sam Bendayan of Ultimate Software came for the 2nd year to talk about how their SaaS company is using Data Vault to provide customer value. This time they talked about their efforts to move to virtual information marts (very cool).

I did my presentation on Building a Virtualized ODS. This was a real life example from my consulting last year on doing an agile data warehouse project based on Data Vault architectural principles. It was a fun talk with lots of interaction. I love challenging the norm, then proving it works!

KentWWDVC16_VODS

Of course I did have to do a little intro promo about my employer, Snowflake Computing. I am happy to say there was quite a bit of interest in our cloud-native, elastic data warehouse offering.

KentWWDVC16_Snowflake

After my talk I did a drawing for a GoPro camera (courtesy of Snowflake). I am happy to say it went Russell Searle from Australia! This man loves Data Vault so much he has paid his own way to Vermont twice now to attend WWDVC. Now that is dedication!

KentWWDVC16_GoPro

Days 4 & 5

Sadly I had other commitments back in Texas and could not stay for these days (but did follow along a bit on twitter). If you want to see everything that happened, search Twitter for #WWDVC.

One fun thing on Day 4 was a few people got to go up in a tethered hot air balloon. Hopefully I can try that next year.

WWDVC_StoweflakeBalloon

Other Fun Stuff

Of course not everything happens in the sessions. Lots of good networking and information exchange happens informally at these events. I did several impromptu demonstrations of Snowflake. The German and Australian contingents were quite interested and can’t wait until Snowflake is available in their regions.

20160525_143828.jpg

Thanks to my friends Paul and Raphael at WhereScape for loaning me their big monitor!

I introduced a very international crowd to the best northern-style, southern BBQ at the Sunset Grille. We had good Data Vault, and non-DV, conversations along with finger licking ribs, brisket, and pulled pork (and beer of course).

20160524_193334.jpg

Take Aways

Every year, as he closes out the event, Dan tries to summarize key learnings for everyone to take home. Here they are for WWDVC 2016:

WWDVC_conclusions

Well that is it for this time around. With such a great event it is impossible to adequately cover everything but I hope this is enough to get you to put WWDVC 2017 on your event calendar. Ask for the time off now!

Safe travels to all the attendees. See you again soon.

Kent

The Data Warrior

A Snow Storm of Snowflake Webinars

Good Monday Morning!

Been itching to learn more about the Snowflake Elastic Data Warehouse? Well, now is your chance.

Over the next two weeks we have a bunch of great webinars coming up so I figured I should just give you a an easy list to review with links to sign up. Here it is:

WEBINAR #1

Wednesday, 04/27/2016 10am PT 

CapSpecialty: Leveraging data to deliver faster business results linked to Key Performance Indicators 

Abstract:

CapSpecialty is upping its game to become the preferred provider of specialty insurance products using MicroStrategy Analytics and Snowflake Cloud Data Warehousing.

Featured partner: MicroStrategy

Hosted by: MicroStrategy

Featured Customer: CapSpecialty

Register here! 

WEBINAR #2

Wednesday, 04/27/2016 11am PT

4 Big Data Strategies You Can’t Go Without 

Abstract:

You’ve got questions about big data, our panel has answers.

When it comes to customer relationships, big data can usher in big opportunities or big problems. That’s why it’s vital for organizations to take a strategic approach to big data. 

They must clean their data, integrate it, maximize data value, comply with security and governance requirements, and make sure the right people have the right access to the data at the right times.

Media partner: CRM Magazine

Hosted by: CRM Magazine 

Featured partner: Informatica + Looker

Featured use case: Pitney Bowes

Featured presenter: Kent Graziano, The Data Warrior

Register here! 

WEBINAR #3

Thursday, 04/28/2016 10am PT

Using the Cloud For Speed-of-Thought Analytics on All Your Data

1.5 TB of data per day? No problem! Learn how Ask.com turned to Snowflake’s cloud-native data warehouse combined with Tableau’s data visualization solution to address their challenges.

Featured partner: Tableau

Hosted by: Snowflake

Featured Customer: Ask.com

Featured presenter: Jon Bock, VP Product and Marketing, Snowflake

Register here!

WEBINAR #4

Thursday, 05/05/2016  10am PT 

The Right Choice: Why Spark + a Cloud Data Warehouse = Success 

The first rule of data analytics for fast-growing companies? Measure all things. When putting in place a robust data analytics strategy to go from measurement to insight, you’ve got lots of options for tools — from databases and data warehouse options to new “big data” tools such as Hadoop, Spark, and their related components. But tools are nothing if you don’t know how to put them to use. 

Media partner: VentureBeat

Hosted by: VentureBeat

Featured Customer: Celtra

Featured presenter: Jon Bock, VP Product and Marketing, Snowflake

Register here!

The Data Warrior Live in Chicago!

Later this week on Thursday April 28th, I will be speaking about Data Vault and Agile Data Engineering at a special Snowflake half-day workshop in downtown Chicago. You can sign up for that here.

So, no excuse for not learning more about Snowflake in the coming weeks. Sogn up for one or more of these events today.

Have a good week!

Kent

The Data Warrior

4 Keys to Succeeding with Agile Data Warehousing in 2016

I have been out giving talks again on using agile methods for data warehouse and business intelligence projects, so I thought it was time for me to share my thoughts about the 4 key elements you need to be successful with an Agile DW project in 2016.

Adopt an Agile Methodology

By this I am talking about SCRUM, Kanban, ScrumBan, or DAD (Disciplined Agile Development), among others.

Go read the blogs, read the books, study these methods. Attend a conference (like Agile Tech in April). Figure out what will work for your organization’s culture and leverage the skills of your staff. One size does not fit all.

In past engagements I have used approaches primarily based on SCRUM and Kanban. Both have been very effective once we got our processes down.

If you need/want help, find a good agile coach.

Use an Agile Data Engineering Approach

If you want to develop your data warehouse in an agile, iterative manner, then you need a way to design your EDW repository that lends itself to this approach without causing huge re-engineering pains (known as refactoring) in future iterations.

The best way I have found is using the Data Vault modeling approach. It was designed specifically for building data warehouses in this manner. I have written much about this approach and give many talks showing examples of successful agile projects using Data Vault. And there is plenty of material available to help you learn how to do it (see the books on the sidebar of this blog).

Also keep an eye on Dan Linstedt’s twitter feed and blog for his training classes.

Use Data Warehouse Automation Software

No better way to get agile and deliver results fast, than to automate as much of your development work as possible. If you use repeatable patterns (like Data Vault) in your design methodology, then it is even easier to automate and greatly reduce your time to market.

There are two vendors in the market that I like a lot and have had some experience with. They are WhereScape and AnalytixDS. And both support not only “traditional” approaches to data warehousing (like automating the ETL for a Type 2 Slowly Changing Dimension) but they both also support Data Vault (and both will be at WWDVC 2016).

Which of these tools you might use depends on your approach, your current tools, and your skills.

If you are coming from a more traditional DW paradigm and use ETL tools like Informatica, Talend, or DataStage, then I would recommend you look at AnalytixDS Mapping Manager which allows you to generate your ETL code from source to target mappings.

If you are just getting started or are committed to more of a database-centric approach and want your ETL or ELT code to run in the database, then look at WhereScape’s products.

Both are great companies with knowledgable people and happy customers.

Your third option is to write your own automation routines. There are many shops doing that as well. Just be sure you have the appropriate skills in house and can allocate the upfront time to get going (a month or so at least).

Deploy on an Agile Data Warehouse Platform

So now that I have learned about Elastic Data Warehousing in the cloud, I can’t imagine trying to do an agile DW project any other way.

Of course I am referring to Snowflake Computing’s DWaaS (data warehouse as a service) offering. Yes, I might be a bit biased since I do work for them now, but…this tech is really good!

From a features perspective, what I am talking about is having a high powered, easily scalable database that supports BI and analytic workloads and does not require a ton of time to configure and tweak.

Why do I think that is a success criteria? Because I have spent way too many months on way too many “agile” projects waiting to get access to the hardware! Or I get access and we either run out of space (e.g., “we had no idea you need THAT much storage”) or we can’t properly test production level loads and queries because the development box does not have enough horsepower.

Taking advantage of the elasticity of the cloud solves both of these problems and the folks at Snowflake have successfully built an RDBMS in the cloud that specifically harnesses these features and leverages them for data warehouse and analytic workloads by providing the ability to scale up and scale down both storage and compute resources on demand.

That and its many other features, give me the infrastructure I need to get an agile data warehouse project off the ground almost instantly. And I can do a Data Vault on Snowflake too.

Very cool.

So what do you think? Are you ready to accelerate your team’s performance and adopt an agile approach to data warehousing?

I hope this post gives you a few ideas on how to make that happen.

Model on!

Kent

The Data Warrior

 

Post Navigation

%d bloggers like this: