The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the category “Big Data”

The Data Warrior Way – DataOps: Past, Present, and Future

Heads up gang – I will be doing a webinar on #TrueDataOps on Wednesday, Nov 30th!

4 pm UTC – 10 am CDT, 11 am ET, 4 pm London, 5 pm CET

During this 60-minute live webinar, I will delve deeper into this topic to share my thoughts on DataOps. What is it? How is it different from DevOps? Where is it headed?

Join me to learn about the seven key pillars of #TrueDataOps:

  • ELT and the spirit of ELT
  • Continuous Integration / Continuous Deployment (CI/CD)
  • Code design and maintainability
  • Environment management
  • Governance and change control
  • Automated testing and monitoring
  • Collaboration and self-service

Come learn what DataOps REALLY IS, and how it’s guiding philosophy can help you deliver data projects faster and better.​

Sign up today to reserve you space! (Yes – it will be recorded!)

The Data Warrior Way – DataOps: Past, Present, and Future | WhereScape

See you there!

Kent

The Data Warrior

The Hype:  Data Mesh Will Become Obsolete – Really?

Data Mesh is indeed a hot topic these days. Is it the answer to all our big data problems? Probably not, but it will help some organizations. Is it obsolete? See what I think in this recent blog post…

In the summer of 2022, Gartner® published its Hype Cycle on Data Management, and this was not without some controversy.

In particular, it asserted that Data Mesh will be obsolete before it reaches what Gartner calls the Plateau of Productivity. It is hard to fathom how they came to that conclusion when there is such a vibrant Data Mesh Community complete with a very active Slack channel, and when DataOps.live customers like Roche and OneWeb have successfully built a Data Mesh using DataOps.live and Snowflake. But we’ll come back to that in a moment.

Data Mesh is in essence a conceptual approach and architectural framework, not a specific off-the-shelf tool or technology. One of the main precepts for its creation is angled towards improving productivity and time to value by eliminating the perceived bottlenecks that have been experienced by organizations building out large-scale enterprise data warehouses and data lakes.  Zhamak Deghani, the creator of the Data Mesh concept, has referred to it as a decentralized sociotechnical approach – meaning it involves people, processes and technology. So, it seems likely that no two implementations will be the same by their nature. How one organization defines productivity based on its specific needs and outputs on its own application of Data Mesh will differ from another. Yes, Data Mesh is (kind of) new and yes, it is maturing, but that does not mean we should ignore or discount it this early in the game.

It is hard these days to separate the hype from the reality so check out the entire post over at the DataOps.live blog.

Cheers.

Kent

The Data Warrior

It Depends (or Does It?) – Episode 13

A few weeks back, right after attending Snowflake Summit, I had the pleasure of chatting with Sanjeev Mohan on his podcast – It Depends. We spent an hour or so chatting about trends in modern data management, data mesh, data vault, my now semi-retired life as an advisor to several Snowflake partners, and my love of martial arts.

So sit back, relax (I did), and enjoy!

Aloha.

Kent

The Data Warrior

Snowday! Sunny, with a 100% Chance of Innovation

What a great event! So many announcements and great demos, plus and awesome live Q&A with our Snowflake leaders.

At Snowday 2021, Snowflake announced exciting new product capabilities that expand what is possible in the Data Cloud. In addition to announcing Python support in Snowpark (currently in private preview), these latest innovations make it easier for organizations to maintain business continuity across clouds and regions; help data engineers and data scientists build pipelines, ML workflows, and data applications faster; and remove the complexity of getting the right data into the hands of customers.

The Snowflake Data Cloud is a global network connecting organizations through data, creating new opportunities for collaboration to improve business outcomes, and fundamentally changing what is possible across industries. For Kraft Heinz, its data science teams are able to build and test models dramatically faster in Snowflake compared with its prior data lake. For NBCUniversal, it’s building brand-new advertising targeting and measurement products, in a secure and privacy-compliant way using Snowflake’s governance and data sharing capabilities. And for 84.51°, it’s built a Collaborative Cloud that takes complexity off the table and unlocks new possibilities for grocers and CPGs sharing and collaborating on data.

Snowflake continues to expand the scope and possibilities of the Data Cloud, delivering unique innovations that enable customers to:

  • Operate globally
  • Eliminate silos
  • Build faster
  • Create new businesses

Catch up on all the details on the  Snowflake blog.

Check it out!

Kent

The Data Warrior

P.S. Registration for Snowflake Summit 2022 is now open!

 

Snowflake Launches Unstructured Data Support in Public Preview

This is great news that many of us have been waiting for! Now we can have all our data in one place.

From Day 1, Snowflake has supported structured and semi-structured data. Snowflake has provided exceptional performance for those data types and has been a pioneer in processing them. Today, Snowflake is adding support for unstructured data to allow customers to deliver more use cases with a single platform. The support for unstructured data management includes built-in capabilities to store, access, process, manage, govern, and share unstructured data in Snowflake. Now you can get all the benefits of the Snowflake Data Cloud with performance, concurrency, and scale for unstructured data.

Read to entire blog for all the details on how you can use this feature.  Unstructured Data Support in Public Preview

This really opens up a lot of new uses cases and makes doing analytics on unstructured data much easier.

Enjoy y’all!

Kent

The Data Warrior

Post Navigation

%d bloggers like this: