SOA / Web Services / Java

A Technology Blog

Archive for the ‘IBM Websphere Message Broker’ Category

WMB – Create and View Trace Logs

Posted by Vivek on February 2, 2015

Trace logs are important when one has to find out root cause of any problem occurred during message flow execution. This post helps you understand the commands that are required:

You can create user trace by running the following command:

Start user trace:

mqsichangetrace <brokername> -u -e <egroup> -f “<Message Flow Name>” -l debug -r -c 50000

You can stop user trace with the following command:

Stop trace:

mqsichangetrace <brokername> -u -e <egroup> -f “<Message Flow Name>” -l none

Retrieve the trace log for the specified component:

mqsireadlog <brokername> -u -e <egroup> -f -o flowtrace.xml

Format XML trace file for viewing purpose:

mqsiformatlog -i flowtrace.xml -o userflowtrace.txt

Similarly, you can create service trace by executing a series of commands:

Service Level Trace:

Start trace.

mqsichangetrace <brokername> -t -e <egroup> -l debug -r -c 100000

Stop trace.

mqsichangetrace <brokername> -t -e <egroup> -l none

Retrieve the trace log for the specified component.

mqsireadlog <brokername> -t -e <egroup> -f -o flowtrace.xml

Format the XML trace file.

mqsiformatlog -i flowtrace.xml -o serviceflowtrace.txt

The serviceflowtrace.txt will be in the current working directory

Posted in IBM Websphere Message Broker | Tagged: | Leave a Comment »

Using SOAPUi to put/get message in/from IBM MQ

Posted by Vivek on July 8, 2014

We can use HermesJMS which provides out of the box feature to communicate with any of the messaging system. It comes with both soapUI open source and pro version. However, the drawback is message will contain JMS header and the responsibility of converting JMS to native header format lies with the developer/ tester. I have used IBM MQ as messaging system and in order to avoid any transformation from JMS to MQMD, I have used groovy to put and read messages.

Add a groovy test step to get hold of the XML message to be sent

def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )

def holder = groovyUtils.getXmlHolder(“AddDiscussionBoardItem#Request”)

log.info holder.getXml()

def xml = holder.getXml()

//Connect to the queue manager

// Queue used for putting message

def propIQueue = testRunner.testCase.getProperty(“inputqueue”)

//Queue used for reading message

def propOQueue = testRunner.testCase.getProperty(“outputqueue”)

 //setting hostname

def propHostname = testRunner.testCase.getProperty(“hostname”) 

MQEnvironment.@hostname = propHostname.getValue()

 //setting port

def propPort = testRunner.testCase.getProperty(“port”)

MQEnvironment.@port = Integer.parseInt(propPort.getValue())                       

//setting Channel

def propChannel = testRunner.testCase.getProperty(“channel”)

MQEnvironment.@channel = propChannel.getValue()        

//setting Queue Manager

def propQM = testRunner.testCase.getProperty(“queuemanager”)

def queueManager = new MQQueueManager(propQM.getValue())

//Put message in queue

 def putMsg = new MQMessage();

int putOpenOpts = MQC.MQOO_OUTPUT | MQC.MQOO_FAIL_IF_QUIESCING;

def putQ = queueManager1.accessQueue(propIQueue.getValue(), putOpenOpts);

putMsg.writeString(xml);

def pmo = new MQPutMessageOptions();

putQ.put(putMsg, pmo);

putQ.close()

//Read message from queue

def getMsg = new MQMessage();

int getOpenOpts = MQC.MQGMO_WAIT | MQC.MQGMO_BROWSE_FIRST;

MQGetMessageOptions gmo=new MQGetMessageOptions();

def getQ = queueManager1.accessQueue(propOQueue.getValue(), getOpenOpts);

getQ.get(getMsg, gmo);

def response = getMsg.readString(getMsg.getMessageLength())

getQ.close()

Posted in Groovy, IBM Websphere Message Broker, IBM Websphere MQ | Tagged: , , | 1 Comment »

Deploying BAR (Broker Archive) using ANT

Posted by Vivek on July 7, 2014

This is not very straight-forward. WMB provide mqsideploy script to deploy the BAR file but before invoking the script, you need to source the mqsiprofile batch file. mqsiprofile is not a perl script and cannot be invoked directly. On windows platform, it is present in MQSI_Home/bin folder as a batch command file (.bat). This batch file has to run before mqsideploy.exe within the same script. Create a new batch command file (deploywmb.cmd) and add the following lines:

 

CALL “C:\Program Files\IBM\MQSI\7.0\bin\mqsiprofile.cmd”

CALL C:\Progra~1\IBM\MQSI\7.0\bin\mqsideploy.exe -i %1 -p %2 -q %3 -b %4 -e%5 -a %6

 

Where,

%1 corresponds to ipaddress, %2 corresponds to port and so on and are passed from Ant script.

The batch file is called and arguments are passed from Ant script as follows:

<target name=”deploy-bar”>

<property file=”${basedir}/build-properties/config/${Env}/DEMO.properties”/>

<property name=”bar-location” value=”${basedir}//bars//${Env}_${bar.file}.bar”/>

<echo message=”MQSI Deploy “></echo>

<exec executable=”C:\utilities\wmbbuild\deploywmb.cmd”

spawn=”false”

logError=”true”

vmlauncher=”false”

failonerror=”true”

append=”true”>

<arg value=”${ipaddress}” />

<arg value=”${port}” />

<arg value=”${queuemgr}” />

<arg value=”${broker}” />

<arg value=”${exegroup}” />

<arg path=”${bar-location}” />

</exec>

<echo message=”Deploying Broker Archive file – ${bar.name} ” />

</target>

Where,

${Env} is the environment for which BAR is created. Example – Dev, test, UAT

DEMO.properties file is the file for storing environment specific properties. The following properties are read from DEMO.properties file:

${ipaddress} is the name of the host of the Queue Manager

${port} is the port number of the listener

${queuemgr} is the Queue Manager name of the broker

${broker} is the broker name whose execution group is to be used for deployment

${exegroup} is the name of the execution group

${bar-location} is the location of BAR file which needs to get deployed

Posted in IBM Websphere Message Broker, IBM Websphere MQ, integration | Tagged: , , , , | Leave a Comment »

MQ JARs – Build and Deployment

Posted by Vivek on July 7, 2014

We miss most of the jars in the classpath that give classnotfoundexception during operations related to WMB. Here is the list of jars required in MQ library to enable build and deployment of message flows on execution group:

com.ibm.mq.commonservices.jar

brokerutil.jar

com.ibm.mq.headers.jar

com.ibm.mq.jar

com.ibm.mq.jmqi.jar

com.ibm.mq.pcf.jar

ConfigManagerProxy.jar

connector.jar

Posted in IBM Websphere Message Broker, IBM Websphere MQ | Leave a Comment »

Improving performance – IBM Websphere Message Broker/ Message Queue

Posted by Vivek on December 29, 2012

For those building SOA using IBM’s Websphere Message Broker, this article will provide collection of some of the important tips and tricks that will enhance the performance of their solution:

1. Use RouteToLabel node to reduce the cost of processing where multiple paths are desired.

2. Choose your parser carefully. Use compact parsers for discarding white spaces and comments. For example, use XMLNSC for XML parsing but note that MRM XML can provide validation while XMLNSC does not.

3. Parsing only as far as is needed. Use Opaque parser if you do not want to reference the sub tree in message flow processing. However, validation is disabled if you choose Opaque parsing.

4. You can selectively parse headers like MQMD, MQRFH2 or JMS Properties and it might save parsing the whole body where only routing is important.

5. Use XPath carefully. By default an XPath expression will search for all instances of an element in the message. This happens at the expense of a full parse.

6. Use reference variables/pointers in ESQL and Java.

7. Keep the number of tree copy occurance to a minimum. Use minimum number of compute and Java Compute Nodes.

8. Copy data to Environment and refer data from environment, whereever applicable. It reduces the number of times a message tree is copied. If using message body, copy data to be used at the highest level i.e., keep navigation to as high as possible.

9. Use non-persistent messages where reliability is high.

10. Subflows sometimes adds additional processing overhead. Avoid usage of subflows where reusability is not desired.

11. Reduce the number of commits performed by the message flow by choosing Batchsize and Batchint sensibly. Default batchsize is 50.

12. Each additional execution group will need a minimum of 150MB. So ensure enough memory is allocated.

13. Use shared variables to access common data if multiple message flows are running in a single execution group.

14. Review log buffer settings if using persistent messages. Choose circular and linear logging based on your requirement. Linear logging guarantees recovery but at the expense of additional disk space. Automate archiving/ deleting of linear logs. Remember Every MQPUT, MQGET, MQCMIT of a persistent message causes a log record to be created for recovery purposes. Keep the log files on fast disks.

15. Where possible keep the applications and broker as close as possible in order to reduce the overall overhead.

16. MQCONN is expensive. Use carefully. MQOPEN is also expensive as compared to MQPUT/MQGET.

17. Use MQSIRELOAD to bounce (stop and start) execution groups at times.

18. Try to cache queue handles if more than one messages are to be processed. Use MQPUT1 if processing only 1 message.

19. Use Fastpath binding to remove IPC (Inter-Process Communication) component. Fastpath binding means that application is trusted. To define a broker as a trusted application, use the -t flag on the mqsicreatebroker command

20. Try to keep message size below 100 MB when using shared queues.

21. Disable trace. To disable the product trace, use the mqsichangetrace command with a level of none.

22. Use multiple channels to increase performance. For example, there can be a separate channel for large messages or messages that of high priority.

23. Using a separate ReplyToQ for each client will affect performance. Try to use a shared ReplyToQ and segregate message using MsgId or correlationId, if applicable.

24. Use Compute node (ESQL) over JCN.

25. When an exception occurs in MQOutput node with the fail terminal not wired, memory allocated to the message and LocalEnvironment is never reclaimed.

26. When you SET or CREATE a FIELD in the Environment/LocalEnvironment remember to DELETE it, and not only DETACH it, before the end of the flow.

Posted in IBM Websphere Message Broker, IBM Websphere MQ | 1 Comment »

Why IBM is not a best choice for SOA?

Posted by Vivek on May 5, 2012

SOA offers great benefits by simplifying the architecture by virtue of its priniciples and helps in making an enterprise more agile. But implementing SOA is not easy and it takes a well-defined strategy. It is a long-term investment and if the first step taken is in wrong direction, then integration costs and complexity can increase exponentially.

For example, consider an organization which is in Retail domain and wants to integrate its 2 channels – Store and E-Commerce. This can be done by choosing any of ESB available in the market which supports the protocols, message formats etc of the requesting and providing applications. However, integration might not stop here. There can be another legacy system which the organization has acquired and wants to integrate. Also, management wants to increase agility by defining worflows and dynamically changing rules according to business needs in a faster changing market. It needs to be understood that SOA is a journey and it is business to decide which mode of transport they take to embark on this journey.

IBM is one of the leaders in middlware but does its products make a better choice when implenting SOA?

1. IBM has a wide array of products. Deciding which product suits which requirement is a not an easy task. If there is an ESB requirement then IBM offers the following products: Websphere ESB, Websphere Message Broker and Datapower. Not all products offer the same benefits and each one has its own strength and pitfalls.
2. These ESBs alone may not suffice. If complex transformations are required, then you might also need a Websphere Transformation Extender. If a legacy application needs to be integrated and a third-party or customized adapter is required, then providing a seamless integration will be a challenge.
3. As SOA evolves and business grows, it almost becomes a necessity to have a BPM tool to facilitate agility. IBM provides Websphere Process Server as a BPM tool for process based integration.
4. Websphere Process Server is mounted on top of Websphere Application Server. It also comes with a Business Rule engine to enrich process workflows. However, there is another powerful product, websphere ILOG, which also offers business rule management.
5. ILog JRules is installed separately and has its own development and runtime environments. While you already have a Rational Application Developer, an eclipse based IDE, you will be using another IDE for rule authoring. Not only this, you are also using an IDE for designing and developing ESB flows. For example, if you are using websphere message broker, then you will be using a message broker toolkit as IDE.

Posted in IBM Websphere Message Broker, IBM Websphere MQ, integration | Tagged: , | Leave a Comment »

Execution Group properties of a local/remote WMB

Posted by Vivek on September 10, 2011

Those using IBM WMB sometimes face issues when they have to connect to a remote broker and deploy their code to an execution group.  The utility (mqsireportproperties) provided by IBM Websphere Message Broker does not allow to read the properties of a remote broker and it is sometimes not possible to log on to a remote broker machine to execute the mqsireportproperties command. Common problems faced by users in such situation are –

1. User requires MB toolkit/ MQ Explorer 7 to connect to broker and see EG details. It is difficult for a user who does not have these tools on his/her machine.

2. User cannot see/ set the properties like debug port, HTTP port on an execution group.

Most user do not know that they can leverage the ConfiguarionManagerProxy APIs as a workaround mainly because WMB users are not generally java savvy. Here is a utility (Simple Java Program) which will help to set and report the properties on a local or remote broker. It is easy to alter and run from command line.

Prerequisites:

1. Java 6

2. Following jars are added in  classpath – ConfigManagerProxy.jar (located at ../MQSI/7.0/classes), com.ibm.mq.jmqi.jar (located at ../Websphere MQ/Java/lib) and com.ibm.mq.jar (located at …/Websphere MQ/Java/lib)

import java.util.Enumeration;

import com.ibm.broker.config.proxy.BrokerConnectionParameters;

import com.ibm.broker.config.proxy.BrokerProxy;

import com.ibm.broker.config.proxy.ConfigManagerProxyLoggedException;

import com.ibm.broker.config.proxy.ConfigManagerProxyPropertyNotInitializedException;

import com.ibm.broker.config.proxy.ExecutionGroupProxy;

import com.ibm.broker.config.proxy.MQBrokerConnectionParameters;

import com.ibm.broker.config.proxy.MessageFlowProxy;

/**

 * This utility connect to a broker and reports the properties on an execution

 * group.

 *

 * @author vivek

 *

 */

public class EGStatusReport {

      public static void main(String args[]) {

            try {

                  BrokerConnectionParameters bcp = new MQBrokerConnectionParameters(

                              “localhost”, 1425, “MYQMGR”);

                  BrokerProxy b = BrokerProxy.getInstance(bcp);

                  Enumeration<ExecutionGroupProxy> listEG = b

                              .getExecutionGroups(null);

                  while (listEG.hasMoreElements()) {

                        ExecutionGroupProxy egIter = listEG.nextElement();

                        // Get Execution Group Name

                        System.out

                                    .println(“Execution Group Name – ” + egIter.getName());

                        //GetDebugPort

                        System.out.println(“Debug Port of EG – ”

                                    + egIter.getDebugPort());

                        //GetHTTPPort

                        System.out.println(“HTTP port enabled on EG – ”

                                    + egIter.getRuntimeProperty(“HTTPConnector/port”));

                        // Get List of Message Flows and their details

                        Enumeration<MessageFlowProxy> listMsgFlow = egIter

                                    .getMessageFlows(null);

while (listMsgFlow.hasMoreElements()) {

                              MessageFlowProxy msgFlow = listMsgFlow.nextElement();

                              // Get name of message flow

                              System.out.println(“Messsage Flow Name – ”

                                          + msgFlow.getName());

                              // Get BAR File Name

                              System.out.println(“BAR File Name – ”

                                          + msgFlow.getBARFileName());

                              // Get time of deployment

                              System.out.println(“Deployed at – ”

                                          + msgFlow.getDeployTime());

                              // Get run status of Message Flow

                              System.out.println(“Is run enabled? – ”

                                          + msgFlow.isRunEnabled());

                        }

                        // Get number of deployed objects

                        System.out.println(“No of deployed objects – ”

                                    + egIter.getDeployedObjectsCount(null));

                  }

            } catch (ConfigManagerProxyLoggedException e) {

                  e.printStackTrace();

            } catch (ConfigManagerProxyPropertyNotInitializedException e) {

                  e.printStackTrace();

            }

      }

}

Refer to the following URL for complete list of methods: 

http://publib.boulder.ibm.com/infocenter/wmbhelp/v7r0m0/index.jsp?topic=%2Fcom.ibm.etools.mft.cmp.doc%2Fcom%2Fibm%2Fbroker%2Fconfig%2Fproxy%2Fpackage-overview.html

Posted in IBM Websphere Message Broker, integration, Java/J2EE | 1 Comment »