The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the tag “#AgileDW”

27 Apr 2022

Why Automation is No Longer a Choice for Your Data Architecture

Back in the saddle again for The Data Warrior! Here is a piece I just did for the folks at Wherescape about one of my favorite topics – Automation!

The world of data has changed for sure. Especially over the past several years. In fact, the pandemic accelerated some changes, like the migration to cloud-based data platforms. When everyone needed to be remote, it just made sense to move to the cloud and use a service for your data platform.

Along with that came more data, more data types, and an actual business needs to move faster. Companies had to adapt very quickly during the pandemic if they wanted to survive. Many did and thrived while others, well, not so much.

As the demand for data continues to grow at unprecedented rates, and as it becomes a non-negotiable asset for organizational success, the requirement to rapidly deliver value from that data (i.e., turn it into information for data-driven decision making) has become an imperative.

So how do we deliver value faster with our data warehouses, data meshes, and enterprise data hubs? Automate, automate, automate.

Check out the full post here – Why Automation is No Longer a Choice for Your Data Architecture

Enjoy!

Kent

The Data Warrior

Posted in Data Modeling, Data Warehouse and tagged , , , , , | Leave a comment
23 Feb 2022

WWDVC 2022 is almost here – Are you ready to “up” your Data Vault game?

Vermont is beautiful in May!

As the world is coming out of its pandemic induced isolation, its time we all got back to meetin’ and greetin’ our colleagues and peers in person. One of the things the last two years has taught us it that humans are social animals. We like being around other humans and some variety is good for our mental and emotional health (not that we don’t all love the families that we have been locked up with during this time). Not only that, but we learn and communicate better face to face. So much of our communication is nonverbal and just can’t be seen behind a mask, or a Zoom square.

Are you ready to get back out there, network with your peers, meet some Data Vault Masters, and work on your professional development around Data Vault? If you are then it is time to secure your seat at the 2022 World Wide Data Vault Consortium (WWDVC). As in past years, it will be held near the end of May at the lovely, and family friendly, Stoweflake Lodge in Stowe, Vermont.

Check out all the details, including HOLs, keynotes, and session topics, about WWDVC 2022 in this post by Cindi at DataRebels – WWDVC 2022 is almost here

If you are already planning to go, you can just register here.

Keep on modeling!

Kent

The Data Warrior

Posted in Data Modeling, Data Vault and tagged , , , , , | Leave a comment
11 Dec 2020

Data Vault 2.0 Automation with erwin and Snowflake

I am seeing a HUGE uptick in interest in Data Vault around the globe. Part of the interest is the need for agility in building a modern data platform. One of the benefits of the Data Vault 2.0 method is the repeatable patterns which lend themselves to automation.  I am please to pass on this great new post with details on how to automate building your Data Vault 2.0 architecture on Snowflake using erwin! Thanks to my buddy John Carter at erwin for taking this project on.

The Data Vault methodology can be applied to almost any data store and populated by almost any ETL or ELT data integration tool. As Snowflake Chief Technical Evangelist Kent Graziano mentions in one of his many blog posts, “DV (Data Vault) was developed specifically to address agility, flexibility, and scalability issues found in the other mainstream data modeling approaches used in the data warehousing space.” In other words, it enables you to build a scalable data warehouse that can incorporate disparate data sources over time. Traditional data warehousing typically requires refactoring to integrate new sources, but when implemented correctly, Data Vault 2.0 requires no refactoring.

Successfully implementing a Data Vault solution requires skilled resources and traditionally entails a lot of manual effort to define the Data Vault pipeline and create ETL (or ELT) code from scratch. The entire process can take months or even years, and it is often riddled with errors, slowing down the data pipeline. Automating design changes and the code to process data movement ensures organizations can accelerate development and deployment in a timely and cost-effective manner, speeding the time to value of the data.

Snowflake’s Data Cloud contains all the necessary components for building, populating, and managing Data Vault 2.0 solutions. erwin’s toolset models, maps, and automates the creation, population, and maintenance of Data Vault solutions on Snowflake. The combination of Snowflake and erwin provides an end-to-end solution for a governed Data Vault with powerful performance.

Get the rest of the details here: Data Vault Automation with erwin and Snowflake

Vault away my friends!

Kent

The Data Warrior

Posted in Data Vault, SnowflakeDB and tagged , , , , , , , , , , , , , | Leave a comment
15 Sep 2019

Better Data Modeling: Agile Data Engineering

You asked for it, you got it!

Ever since I wrote my Kindle book on Agile Data Engineering and Data Vault 2.0, many, many people have asked me to provide it in a hardcopy format. Well, I finally managed to find time to convert that ebook into a paperback book (I even corrected a few errors in the process).

If you forgot what the book was about, here is the description:

This book will give you a short introduction to Agile Data Engineering for Data Warehousing and Data Vault 2.0. I will explain why you should be trying to become Agile, some of the history and rationale for Data Vault 2.0, and then show you the basics for how to build a data warehouse model using the Data Vault 2.0 standards.In addition, I will cover some details about the Business Data Vault (what it is) and then how to build a virtual Information Mart off your Data Vault and Business Vault using the Data Vault 2.0 architecture.So if you want to start learning about Agile Data Engineering with Data Vault 2.0, this book is for you.

So here it is – Introduction to Agile Data Engineering – now available to purchase on Amazon.

Get your copy now. Next time you see me at an event, I will be happy to sign it for you. 🙂

Enjoy!

Kent

The Data Warrior

Posted in Data Modeling, Data Vault, Data Warehouse and tagged , , , , , , , , , , , , | Leave a comment
11 Sep 2019

Adapting Agile Principles for Data Warehousing

Despite the clear benefits of data warehousing programs, business units often question what they perceive to be long delivery timelines and a lack of data accessibility. Nearly half (48%) of the respondents in a TDWI survey reported that personnel in their organizations spend at least 61% of their time finding and preparing data. Only 28% said that their business users and analysts can access and analyze new data, including external data, without close IT support.

Check out how to solve these type of issues, and more, by applying the principles of Agile to data warehousing in my newest post and ebook:

Adapting the Agile Manifesto Principles to Data Warehousing

Cheers.

Kent, The Data Warrior &

Chief Technical Evangelist, Snowflake

Posted in Data Warehouse, SnowflakeDB and tagged , , , , | Leave a comment

Post Navigation

← Older Entries
%d bloggers like this: