The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the tag “software”

KScope13 Day Four: Agile, Big Data, and a Very Special Event

Mid-week. Hump day. The day of the BIG event for KScope13.

Lots of anticipation for the annual Special Event… (which I will write about in a minute or so)

Morning Chi Gung as usual, but with 24 people showing up. Biggest group this week. We even have a few locals joining us now. Everyone seems to be enjoying these sessions.

KScope attendees starting the day with Morning Chi Gung on the plaza in front of Harahs casino.

KScope attendees starting the day with Morning Chi Gung on the plaza in front of Harrahs casino.

In fact, the Chi Gung class at KScope may be the original cross over session! Attendees are from across the spectrum from DBAs, to developers, to Hyperion/EPM folks to spouses of attendees.

There is something for everyone in Morning Chi Gung.

Kanban and Scrum

Everyone wants to be “agile” these days. Stew Stryker of Dartmouth University came to KScope to share with us his experience in applying first Kanban then Scrum to the software development life cycle in his IT department.

Stew Stryker, from Dartmouth College, discusses how his team has use Kanban, and now SCRUM, to improve their software development process.

Stew Stryker, from Dartmouth College, discusses how his team has used Kanban, and now SCRUM, to improve their software development process.

One of Stew’s insights was that to effectively implement a change in methodology like this and get adoption it is first necessary for the powers-that-be to recognize the current approach (usually water fall) is failing.

If you do not know you have a problem, there is no motivation to fix it, right?

A key recommendation he had was to get a consultant that knows and has implemented Kanban for database projects to come in and work with you. Don’t try to do it by just reading articles and books or going to training. There are too many nuances and organizational dynamics to account for.

A simple comparison of aspects of a traditional waterfall methodology compared to the Kanban approach.

A simple comparison of aspects of a traditional waterfall methodology compared to the Kanban approach.

Another key to succes was to prevent context switching – that is keep everyone focused on the task at hand for the duration of the interval (or sprint). He did a great little exercise with us that really showed how task switching costs a lot of time. In some case up to 10 times longer.

It was great to hear real world experiences that we could all take back to our offices and implement and discuss. His team has experienced some great success but with lots of lessons learned, which he shared.

They have now switch to SCRUM with even more success.

Hands On Lab #2

I attended my second lab of the conference to learn from Maria Colgan (@SQLMaria) on how to prevent sub-optimal plans on SQL Statements.

Oracle Senior Product Manager Maria Colgan walks us through how to analyze and and tune some queries.

Oracle Senior Product Manager Maria Colgan walks us through how to analyze and tune some queries.

It was a great session using the Oracle Demo Days virtual box image again (from OTN). Maria walked us through several queries with Explain Plans that did not seem quite right and showed us how to diagnose and fix the potential problems.

It was a little tough for those of us who have not used Linux/Unix or command line in a few years but I did learn a lot and should be able to apply that knowledge when we have poor performing queries at my clients. Worse case, I can always start up the vm again and run through the lab.

Inside the Oracle 12c Opimizer

Another killer session from Maria showing us enhancements and new features to the query optimizer in the recently released Oracle 12c.

Overview of how adaptive query optimization works on Oracle 12c

Overview of how adaptive query optimization works on Oracle 12c

How the new Adaptive Execution Plans work in Oracle 12c

How the new Adaptive Execution Plans work in Oracle 12c

The key phrase for 12c “self-healing” and “adaptive”. Remember when there were just 17 rules for the optimizer that we could control with the syntax of the query?

Long ago.

I guess this is better, but there are still rules to know to make the optimizer work well.

And Maria definitely knows them!

Big Data

These days every tech event has to talk about big data. KScope13 is no different.

Alex Shlepakov, from Accenture’s Oracle BI practice, gave a nice talk about integrating Hadoop with OBIEE using ODI.

He did a really nice job explaining all the concepts and moving parts and how Oracle addressed these things.

Alex presented about doing big data analysis using Oracle BI tools.

Alex presented about doing big data analysis using Oracle BI tools.

All the Oracle products that support the analysis of data in a Hadoop environment

All the Oracle products that support the analysis of data in a Hadoop environment

Pretty sure these products cost lots of money too! But if you want to get value out of your big data, you may have to spend big money for the tools to help (unless you have a lot of programers with really big brains).

My main take away from this session is that the tools to support Hadoop and big data analysis are evolving to make it easier for most programmers to get to the data without having to be Map Reduce programmers.

But it will still be pretty hard, so you better have a good business case for digging into it.

Special Event (aka the big party)

As in past years, ODTUG really did it up right. This was truly a special event to remember – we went to Mardi Gras World!

The annual Special Event was held at Mardi Gras World where we got to see some of the big floats from the famous parade.

The annual Special Event was held at Mardi Gras World where we got to see some of the big floats from the famous parade.

What a treat to see some of the big floats used in the famous parade. I even found a full scale replica of the Bat Boat tucked away in the back. (There was a huge Batman statue as well)

The Oracle Data Warrior finds Batman's boat!

The Oracle Data Warrior finds Batman’s boat!

The tour of the Mardi Gras warehouse included plenty of bead throwing from the floats by the board of directors and the various KScope vendors. This was followed by a nice evening of drinks and a buffet dinner with lots of great food (even some gluten free and vegetarian options).   There was plenty of dancing to great cover band called The Mixed Nuts.

We finished the evening with a spectacular fire works display (which seems to becoming a standard at this event).

We had a spectacular fireworks display (shot off a barge) at the annual KScope Big Event

We had a spectacular fireworks display (shot off a barge) at the annual KScope Big Event

Over too soon, it was last call, last dance, then back to the buses and a short ride to the hotel.

And then there were the after parties….

Stay tuned for my notes on our final day in New Orleans.

Ciao!

Kent

Oracle Designer Lives!

Amazing as it seems, I picked this article up on Twitter today.

An up to date, current, and NEW article about automating builds of applications from the Oracle Designer repository.

How very agile…

Thanks to all the gang over at AMIS (http://technology.amis.nl/) for keeping the technology alive and for being innovative enough to adapt it for the modern agile development world.

Running Oracle Designer Generation from Ant and Hudson

Introduction

Oracle Designer is a windows client-server development tool that is meant to be manually operated by a developer. Anyone trying to integrate Designer with an automatic build environment will find that it does not provide an API or a commandline version to kick-off any generation automatically.

There is however a hook that can be exploited by generating so-called GBU files directly from the Designer Repository. These GBU files are then fed to an executable called dwzrun61.exe that executes the actual generation of DDL scripts and forms.

This article describes how this can be done using examples from a real world situation. It shows how to generate the GBU files, the different strategies that can be followed and some of the pitfalls you might run into trying to pull this off yourself.

The code of the program we wrote can be found on here and is free to be adjusted to fit any other situation than ours.

via Running Oracle Designer Generation from Ant and Hudson.

If you want to meet some of the guys from AMIS and pick their brains, be sure to sign up for KScope13 and come meet them live in person.

See you there.

Kent

RMOUG Training Days 2013 – Day 2

So on this 2nd and final day of the annual RMOUG Training Days event, I started out by attending an excellent session on Exadata for Oracle DBAs.

Even though I am not a DBA these days I thought it would be good for me to get a better understanding of Oracle’s engineered Exadata machine.

I feel very luck to have attended this session given by Oracle Technologist of the Year, and ACE Director, Arup Nanda. He had some of the best graphics and clearest explanations of the basic anatomy of an Oracle database I have ever seen or heard.

Technologist of the Year, Arup Nanda, Database Machine Administrator

Technologist of the Year, Arup Nanda, Database Machine Administrator

He gave some pretty detailed explanations of what he called the “magic” of Exadata and why it works so well. Arup even coined a new job title, which he claims for himself, DMA – Database Machine Administrator. Because Exadata is an engineered system, it contains database, storage, and networking all in one rack. This requires some skills beyond what most dbas have or are expected to have.

He gave us a nice break down based on his experience using Exadata.

Break down of skills needed to be a successful Exadata DMA

Break down of skills needed to be a successful Exadata DMA

After this talk I can see why he was give the awards. He really knows his stuff and how to communicate it. You can follow him on Twitter @ArupNanda and see for yourself.

Next I went to see my friend,and ACE Director, Galo Balda from Austin, Texas. He gave a very informative talk about Regular Expressions.

ACE Director, Galo Balda, doing his very first presentation

ACE Director, Galo Balda, doing his very first presentation

His presentation was very informative with easy to understand examples of how to write and use regular expression and associated metacharacters to do some pretty neat things with SQL. If you attended the conference be sure to download his slides. They will make a great cheat sheet.

You can follow him on Twitter @GaloBalda or go to his blog.

After a nice vegetarian lunch, I went to see Maria Coglan talk about using (or not using) hints in SQL and how it affects the optimizer. Last year at Kscope12, I attended one of her optimizer sessions and felt like my head would explode becuase of all the information she gave. She assured me this talk would not be as bad.

She was right. It was a very informative talk.

A full house to see Maria Coglan discuss Hints and the Optimizer

A full house to see Maria Coglan discuss Hints and the Optimizer

Her main message was to always use caution when using hints. You really need to understand what you are or doing or you could make your application or reports run worse rather than better.

Maria even explained how to work with applications that already have hints embedded in them.

Approach to ignoring hints in an existing application

Approach to ignoring hints in an existing application

Get her slides and follow her on Twitter @SQLMaria

After Maria’s session I did my final session for the event. I talked about my Top 10 favorite cool tools in SQL Developer Data Modeler. There were 30 or so people in attendance. Most of them even stayed through the whole talk!

Which is pretty good since I ran over my time. There was just so many tips and tricks to show.  I will put it up on SlideShare in the next few days.

The final session for the event that I attended was done by RMOUG President, my long time friend, Tim Gorman.

Tim talked about the various options for data compression in the Oracle stack.

Tim Gorman (in the shadows) giving the last talk

Tim Gorman (in the shadows) giving the last talk

Tim gave some pretty detailed explanations and tried to depict how compression works with some nice graphics. He also told us which ones cost additional license fees.

Data Lifecycle when using Compress for OLTP

Data Lifecycle when using Compress for OLTPut how have

For me, the most useful part was his explanation about how having columns at the end of the table allows a default sort of compression to take place. I had heard this a long time ago. It was the reason so many of us were taught to put all mandatory columns at the beginning of the table – it saves space. In recent years I have been told by various DBAs that the rule no longer applies or made sense.

They were wrong! Tim gave us a real world example of how putting populated columns at the end of a table cost a lot of extra space to be used.

I will be taking that tidbit of information back to the office for sure.

You can follow Tim on Twitter @timothyjgorman.

A side note about RMOUG: At lunch, Tim shared with the attendees an agenda from 1991 for the 2nd RMOUG Training Days. We now realize that we started this event in 1990 and next year will be the 25th anniversary! (I say “we” because I was part of the planning committee back then and one of the early speakers too).

Another interesting notes was that 2/3 of the speakers came form out of town. Many, including me, paid there own way. Several speakers and attendees I know even had to take vacation time from their jobs to attend.

It is that important and that good an event!

So put it on your calendar to attend what is probably the most successful and longest running regional Oracle user conferences in the country. It will be in early February 2014. Watch www.rmoug.org for details.

And of ocurse count on me to post it here too.

Ciao for now! I am off to ski with some RMOUGers tomorrow.

Kent

Tech Tip: Connect to SQL Server Using Oracle SQL Developer (updated)

I spend a lot of time reverse engineering client databases to see what kind of design they are working with or to simply create a data model diagram for them (so they know what they have).

Along the way I often need to actually look at the data as well to do some analysis and profiling.

Often this means looking at data and models in SQL Server as well as Oracle.

What’s an Oracle Data Warrior to do?

Hook up my FREE handy dandy Oracle SQL Developer to the SQL Server database.

How do you do that?

First you need to get the right driver. You can find it here: http://sourceforge.net/projects/jtds/files/jtds/1.2.5/jtds-1.2.5-dist.zip/download

NOTE: For SQL Developer 4.0EA3 and SQL Developer Data Modeler 4.0 (production) you now need jtds-1.3.1. Get it here: http://sourceforge.net/projects/jtds/files/

Then follow these steps:

  1. Download and unzip the file into the main SQL Developer directory (or the directory of your choice).
  2. In SQL Developer go to Tools -> Preferences -> Database -> Third party JDBC Drivers
  3. Click the “add entry” button
  4. Navigate to the jtds-1.2.5.jar file. (or the 1.3.1 file for 4.x installs)
  5. Save and exit preferences.
  6. Close and restart SQL Developer
  7. Open “Add Connection” – there should now be a SQL Server tab.
SQL Developer Preferences

SQL Developer Preferences

With this in place, you can now connect to SQL Server without having to load any other software.

Pretty useful.

Happy Querying!

Kent

P.S. You can connect to other non-Oracle dbs as well. Check out this post by Jeff Smith for even more details.

Additional Notes on SSO errors:

Lots of folks, including me, have had issues getting the native Windows SSO connection to SQL Server to work. Based on answers on the OTN Forum and this post (http://www.oracle-base.com/blog/2013/10/01/sql-developer-4-ea2-connecting-to-sql-server/) I finally got my new 4.x versions to work.

For SQL Developer 4.0EA3, I did as suggested in the article: http://www.oracle-base.com/blog/2013/10/01/sql-developer-4-ea2-connecting-to-sql-server/. I put the ntlmauth.dll where my JDK 1.7 was installed: C:\Program Files\Java\jdk1.7.0_40\jre\bin

For Data Modeler 4.0.13 (production), based on a suggestion from Jeff Smith, I put the dll file here: C:\SQLDeveloper\SQLDeveloper4.0.13\sqldeveloper\sqldeveloper\bin

If I was better at setting windows paths, I am sure there is a better way to do this.

Post Navigation

%d bloggers like this: