ODI JMS driver IKMs in 12c

IKM...LKM...knowledge!

ODI has the concept of knowledge modules that encapsulate and re-use the heavy lifting logic required for data movement. There are several KMs authored targeting different data sources. A big shift occurred in the world of ODI data flow control by the introduction of flow-based mapping in 12c. This component obviated many of the limitations imposed by ODI Interfaces in pre-12c days. 12c also introduced the concept of component KMs and global KMs. When desinging a mapping, most of the time an appropriate component KM gets chosen by default - whether it be for loading or integration.

JMS....JMS..wherefore art thou JMS

A pretty significant gotcha crept in for the particular case of JMS or JMS XML as target. If you design a mapping with a datastore from these technology types as target then you know that for the data to be actually sent to the JMS Queue/Topic you need to choose the JMS-specific IKM. Here lies the catch. Although you may have imported the IKM into your project when you go to the target datastore configuration to set the IKM you may not find it there. If this happens to you check the LKM that is being used. The JMS-specific IKMs are designated as multi-connect KMs and in ODI 12c if you want to use a multi-connect IKM then your LKM must also be multi-connect.
So the solution is to switch your LKM to one of the multi-connect component KMs or template KMs.

If you see similar symptoms with other KM combinations remember this : check your KM to see if they are multi-connect.

ODI JMS (XML) data server peculiarities

ODI JMS data servers

ODI has JMS Queue, JMS Topic, JMS XML Queue and JMS XML Topic drivers. The Queue/Topic drivers are slightly different from XML Queue/Topic drivers. Of course the obvious difference is that the former is for Delimited/Fixed width message types and the latter is for XML message types. In addition to this there is one more subtle difference.

Go on and create a new JMS Queue data server. Give it some name and then switch to the JNDI tab. You will see drop-down for authentication type, text fields for user name and password, drop-down for JNDI protocol, text fields for JNDI Driver (the fully qualified name of the InitialContextFactory class for your JNDI server), JNDI URL and JNDI Resource.

Gotcha 101

 Many people mistakenly think that JNDI Resource is where you put the JNDI name of your Queue/Topic. This is wrong. This field is for entering the JNDI name of your JMS connection factory. It can be a custom connection factory created on the JMS server by you or an in-built one for the JMS server. If you are using Weblogic you will have the following connection factories available without any configuration : javax.jms.QueueConnectionFactory, javax.jms.TopicConnectionFactory, weblogic.jms.ConnectionFactor and weblogic.jms.XAConnectionFactory. This concludes all the configuration you can do the data server.

Gotcha 102

Did you notice anything funny? Even on the Physical or Logical schema you have no place to provide the name of JMS Queue/Topic. For this you actually have to jump through another hoop. It will be explained below. But first think about the impact of this limited configuration. You have provided JNDI URL to your JNDI server, credentials, authentication mechanism and connection factory, but no Queue/Topic information. This means that when you press that nice helpful 'Test connection' button all it is testing is creation of the InitialContext object. It does not test actual connectivity to the Queue/Topic.

Now let us say you want to connect to your Queue. Create a Model baased on the Logical Schema of your JMS Queue data server. Again, you cannot reverse engineer a JMS Queue/Topic. So you have two options :

1. create a Datastore under this model and define the column structure by hand or
2. make a copy of a Delimited/Fixed width File Datastore

 After the Datastore has been created there is one more thing to do : there is a text field on the Definition tab of the Datastore named 'Resource name'. For a File Datastore this would point to the actual file. This is where you enter the JNDI name of your JMS Queue/Topic.

Phew! That was a tortuous setup.

What about JMS XML Queue/Topic? Now that has some differences. For example the JNDI URL takes a key-value pair where the key is JMS_DESTINATION and the value is the JNDI name of your Queue/Topic. So all the JMS configuration is in one place. Also, a JMS XML Model can be reverse engineered.
You may now be saying to yourself 'All right!!!!!! Let me make all my JMS Queue/Topics XML-based messages'. Hold your horses, because there is a down side : you cannot right-click a JMS XML Queue/Topic Datastore and do 'Data' or 'View data'. It is not supported.

So there you are. A little the worse for wear owing to this trip over ODI JMS. Wait for the next few blog posts that will reveal to you some more hidden pitfalls and capabilities.

What is new for ODI XML JDBC driver in Oracle Fusion Middleware 11g R1 (11.1.1.7.0)

ODI XML driver issues

ODI XML driver had several issues such as

1. would not support elements with same name but different types. For example a local element with same namespace URI and local name as a global variable would not be handled.
2. does not support list/union simpleTypes, any, anyType, mixed type and so on.

In ODI 11.1.1.7.0 a new compatibility mode parameter has been added. This is an XML driver property called 'compat_mode'. Its default value is 'v2'. But if you specify the value 'v3' for this parameter, you can avoid issue (1). As you might imagine, if you use 'v3' for the value of this parameter, you will run across some changes in the behaviour of the driver.

Differences between 'v2' and 'v3' modes

1. Handling of elements with same local name and resolved type. In current XML driver, data from such elements end up stored in the same table, but the rows have different FK references. With default mode, we will use different tables to store the elements. This has implication for elements that are 'ref's of a global element. Each of the reference (and descendants) gets its own table.
2. Determining the root element. The XML JDBC driver allows users to specify the local name of the root element, and not a qualified name. One plain assumption here is that the root element has a unique local name. In 'v2' mode, if a root element name is not specified then the first element encountered by the parser is set as root element. This is the first element from the XSD hierarchy where the XSD does not have an 'include' or 'import'. In 'v3' mode this is indeterminate. The XSD should either have only one global element or user must specify the name of the element to be used as root. Failing this, one of the global elements from one of the target namespaces is used as root.
3. As a corollary to (2), only a global element is allowed to be used as root element. Old parser would allow use of even local elements. This is carry-over from DTD times where all elements are global elements. If user specifies a local element name (which also is not the same a global element name) then an exception is raised. Note that for the old parser, if the user had provided an XSD, then the restriction applies that root element must be a global element.
4. When using a DTD/specifying XML driver URL parameter to generate DTD, user must provide root element name. The new XML driver converts the DTD to XSD in-memory and uses this XSD to construct the data model. This conversion requires that a root element be specified. However, DTDs that are for XML structures that need to use namespaces must continue to use 'v2' mode.

Note that since JMS XML Queue/Topic drivers and Complex File driver uses XML driver internally, these drivers will also accept 'compat_mode' parameter. 11g JMS XML driver though, will not accept this parameter. It only supports a subset of XML driver parameters. Starting from 12c this is not the case