BizTalk SOAP Adapter - Consuming Remote Web Services Bottleneck

This morning I was told by my colleague that he found hundreds of instances running in the BizTalk Server and from the look of it, most of them are the send port SOAP adapter instances.

We have an orchestration which consumes a web services, so the first thing that came to my mind is that it's gotta be the web services which can not handle the load.

I was thinking that there's gotta be hundreds of concurrent calls to the web services, then i tried to open up the Performance Counter -> ASP.NET Apps v1.1.4322 -> Requests Executing for the particular web services, i was surprised to see that there are only 2 requests executing concurrently :|

Then my other colleage told me that there's a settings in the machine.config file for this, to limit the numbers of call to particular connection.

<system.net>
<connectionManagement>
<add address="*" maxconnection="2">
<add address = "http://www.contoso.com" maxconnection = "5" />
<add address = "http://www.northwind.com" maxconnection = "2" />
<add address = "200.200.200.1" maxconnection="12">
</connectionManagement>
</system.net>

After changing this settings to a higher value, the instances were getting completed gradually and not building up anymore :)

You can find more about this at :
http://channel9.msdn.com/wiki/default.aspx/PerformanceWiki.HowToTuneASPNET
http://support.microsoft.com/kb/821268
http://msdn.microsoft.com/en-us/library/aa560822.aspx / BizTalk 2006 Server Documentation -> SOAP Adapter Configuration and Tuning Parameters
http://msdn.microsoft.com/en-us/library/1tkaca2y.aspx

However, in the BizTalk 2006 Server Documentation, it states that the default is 20 where .Net Framework Documentation states only 2.

Hope this helps anyone with the same issue ;)

Application (Message) Based Routing in BizTalk Orchestration

This may not be a common scenario, but a few weeks ago, I was given this requirement :

If a message comes from Application 1, please send it to Web Service A in Server X, if it comes from Application 2, please send it to Web Service A in Server Y, so on and so forth.


Notes : Web Services A is a same webservices deployed on both Server X and Server Y

What are the purposes of having this requirement?

1. Development :
   
• We have numbers of projects which may be developed in different development servers but for the BizTalk parts, they are all deployed in a single BizTalk Development box. When we're calling the a particular web services, it is possible that Project A will use Development Server A and Project B will use Development Server B.
  
2. Production :
• For Load management purpose, we can point Application 1 calls to Web Server A and other Application calls to Web Server B.

In BizTalk, we can do it this way below, however since Web Service A in Server X and Server Y are actually a same web services, BizTalk will not be able to process the messages because they will have same namespace and same schema information. Unless you want to use different pipelines for each send port, imagine if you have 10 or more applications, do you really want to create additional custom pipelines for each of them :P

To solve the namespace and schema issue, we can create a common orchestration which can be shared / called by the applications. But wait, there's only 1 send port can be defined in this solution, we still want Application 1 to call Web Services A in Server X and Application 2 to call Web Services A in Server Y.


Now, this is where Role Link comes to play. If you see the diagram below, between the Orchestration and the send ports, there is a Role Link Provider. This is just like an additional routing layer which we can use instead of correlating the orchestration and the send port directly.


How to use Role Link is quite similar to regular Send port, you may notice when clicking on the port surface, there is New Role Link option ;) and you will need to choose between Provider & Consumer which is basically to define whether you will be sending message through this port or receiving message through this port.


The next question will be how to route the message?

1. We will need to have a value in the message to differentiate where this message comes from and should be routed into
2. We define the destination party before we send the message, e.g :
   varDestinationPartyName = HelperClass.GetDestinationPartyName(partyName);
   send_port1(Microsoft.XLANGs.BaseTypes.DestinationParty) = new Microsoft.XLANGs.BaseTypes.Party(varDestinationPartyName,"OrganizationName");
3. We create new Party with the Organization name that we define, e.g : Application 1, Application 2, etc.
4. After deployed the orchestration, we should be able to find the provider we defined in the orchestration in the Role Links folder under Orchestrations folder (BizTalk Administration Console)
5. Click on the provider to open the the window. In this window, we can bind the party and the send ports. For illustration :
• Party : Application 1, Logical Port : send_port1, Physical Port : sp_WSA_ServerX
• Party : Application 2, Logical Port : send_port1, Physical Port : sp_WSA_ServerY
• Party : Default, Logical Port : send_port1, Physical Port : sp_WSA_ServerY
  

Notes : I provide a default party as well, so in case if i have a new application 3 and i haven't added it to the party list, it will use the default party :)

How to check whether the party exists?

public static string GetDestinationPartyName(string partyName)
{
    string result = Constant.DefaultDestinationPartyName;
    BtsCatalogExplorer catalog = null;

    try
    {   
    // Create the root object and set the connection string
    catalog = new BtsCatalogExplorer();
    catalog.ConnectionString = ConfigurationHelper.BizTalkMgmtDBConnectionString;
    
    // Get the requested party from the collection
    Microsoft.BizTalk.ExplorerOM.Party party = catalog.Parties[applicationUserID];
    
    if (party == null)
    {
        //Log as warning that the party could not be found
    }
    else
    {
        result = party.Name;
    }
    }
    catch (Exception ex)
    {
    //Log the exception
    }
    finally
    {
    if (catalog != null)
        catalog.Dispose();
    }

    return result;
}

Notes : Party List is not available at WMI

If you're getting this exception "Microsoft.BizTalk.ExplorerOM.BtsException: The database or the database version is incompatible with the installed version of this product.", you can just use sql objects to retrieve the party name from database instead of using the BizTalkCatalogExplorer :

cmd = new SqlCommand("select nvcName from bts_party where nvcName = @partyname", cn);
SqlParameter prmPartyName = new SqlParameter("@partyname", applicationUserID);
cmd.Parameters.Add(prmPartyName);

Hopefully, this interests you as it is for me ;)

BizTalk Consume WebServices with SOAP Header

When working with Web Services, the common scenario is to pass in the credential in the SOAP Header.

In this post, I will provide the steps how to do this in BizTalk Orchestration.
This will be a simple flow like below screen shot :

Flow Detail :

1. Web Application will call an exposed web services by Biztalk Orchestration
2. The request object consists of 3 elements : UserName, Password, Message
   
3. BizTalk Orchestration will construct the SOAP Header Message with UserName and Password
4. BizTalk Orchestration will assign the Message as the body and assign the SOAP Header
5. BizTalk Orchestration calls the Web Services (WSSOAPHeader)
6. Web Services will read the SOAP Header message and return back some message
7. BizTalk will also return back the message to the web application

BizTalk Orchestration layout

One of the most crucial thing in here is the SOAP Header message.
To create the SOAP Header message schema :

1. Create a property schema file
2. In the schema properties, change the Target Namespace to http://schemas.microsoft.com/BizTalk/2003/SOAPHeader Set the Node Name property to the SOAP Header Name and the Property Schema Base to MessageContextPropertyBase

After creating this property, we can assign the SOAP Header into the message like this below

varXMLSOAPHeader = msgSOAPHeader;
msgWSRequest(MyBizTalkApp.AuthenticationSOAPHeader) = varXMLSOAPHeader.OuterXml;

Note :

• When assigning the SOAP Header, it expects string value, that's why I assign the msgSOAPHeader into an XML document variable and get the string from OuterXml property.
• If the SOAP Header is not received by the web services even though you have assigned it in the code, make sure that you have followed each of the steps to create the SOAP Header message schema above, especially the namespace part. It caused me 2 days to notice this one in the end :P

You can find and download the whole VS 2005 solution at here :


Here is the link to the MSDN Site : Consuming Web Services with SOAP Headers

Hope this helps