Showing posts with label ODI debugger. Show all posts
Showing posts with label ODI debugger. Show all posts

Wednesday, December 4, 2013

ODI debugger - Part 2

Descendant Sessions debugging

In ODI one Session can launch a child Session in different ways.
  1. user can have an OdiStartScen tool invocation either in Package or Procedure
  2. user can have an OdiInvokeWebService invocation either via Package or Procedure
  3. user can have an OSCommand or OdiOSCommand tool invocation calling startscen.sh/cmd or restartsess.sh/cmd either in Package or Procedure
ODI debugger will notify the client about debuggable child Sessions start only in case (1). In this case you need to be aware of one fact. Child Sessions inherit debug flags set on the parent Session unless they are explicitly overridden by debugger properties set on OdiStartScen. So if parent Session has the flags DEBUGGABLE=true and DEBUG_DESCENDANT_SESSIONS=true set, then any child Session would be automatically debuggable and *its* children would be debuggable and would start any of their child Sessions also in debuggable mode. This gives the user a very flexible debug configuration. User may want to launch the root Session in non-debuggable mode, but have a specific child Session debuggable. Or they might want it the other way, some children to be debuggable while others are not.

There is another subtle gotcha. When ODI Studio notifies you that a child Session is starting, if user chooses not to attach Studio to that debuggable Session, then user *will not receive any notifications for any debuggable child Sessions started by this child Session*

Starting Session in debug mode


All the ways for starting a Session have debugger flags that user may set. This includes startscen.sh/cmd, restartsess.sh/cmd, ODI incoming web service (a.k.a public web service), OdiStartScen tool, SDK APIs and Studio.

Thursday, October 17, 2013

ODI 12c debugger

Debugging ODI is hard

Why is it so hard? Well, the following are a few reasons. ODI has tools such as OdiStartScen that will let an ODI Session launch another ODI Session on a wholly different agent. ODI interfaces (mappings in 12c) can use DB-native utilities for pumping data. This means that ODI does not have a view into the data being transferred. In ODI 12c we have introduced multi-threading in ODI Tasks. This means that like ODI Loadplans, ODI Tasks can be run in serial or parallel mode.

What have we done then?

While designing and implementing the debugger was a blast, it was disheartening to succumb to some limitations. However there are not that many of them. Just to bring it up in front, ODI debugger will let you suspend the execution flow before/after an ODI Step or before/after an ODI Task. Breakpoints can be set on design-time artifacts like Package or Procedure or on runtime artifact - ODI Blueprint.

Know your Sessions

The problem of distributed debugging is solved for the client by ODI detecting a new Session launch that is associated with a parent Session and making the client aware of the same. A caveat here is that the client must be actively debugging the parent Session. Otherwise there is no place for us to send the notification. So the good thing is that the client gets a transparent debugging experience. Don't worry, client has the option to not connect to the child Session. There are options so that even upfront you can say that the child Session is not candidate for debugging. In this case child Session is allowed run on and client cannot debug it.

See my data before or after

Data debugging is  only allowed at the beginning and ending of an ODI Task. The fact that was mentioned earlier viz. ODI may use native utilities to pump data in either direction prevents ODI from providing a transfer-by-transfer data debugging facility. However, for most usecases, the current data debugging facility will be more than sufficient, since at the end of the ODI Task we will be letting the client see uncommitted data. Note that this is a view-only tool. No modification of data is allowed. You may think of this as a limitation, but there is a way around it. ODI Debugger allows JIT modification of code. For example, if you suspend execution before an ODI Task and then modify the code of the Task, you can push that code to the repository. So when ODI gets to the point of actually executing that Task it is your modified code that will get executed. How can this help you? Well, sprinkle some dummy ODI Tasks in your ODI flow. Then when and if you see problems with your data and the next Task is a dummy Task, pause at the beginning of the Task and insert code to modify your data.

ODI threads not Java threads

ODI debugger also transparently takes care of multi-threaded tasks for the client. It provides you with a view of the threads of execution that are tied to the current ODI Session. You can see the execution stack of each of these threads and also open the Session viewer to see the precise point at which the execution has been paused. Right now, if there are multiple threads of execution in a Session and more than one of threads has been paused it is a bit difficult to make out from the Session editor's Blueprint tab as to which suspend point belongs to which thread, but it is a minor point.

See your variables' state

You will also be glad to see a variables view. Historically ODI variables have had a lot of bugs associated with it - be it actual bugs or user perception problem. The debugger can help here to a large extent in that it lets you see the ODI Variables and updates the displayed values as and when they change. Not only that the debugger will even let you dynamically modify the value of the Variable. That should be a pretty powerful tool.

So there you have it - a feature-rich debugger for ODI that should help customers and DW developers immensely.