Creating a Service, WSDL, Schema for WSDL and operations with TIBCO BW 6.0

Hi,

Recently our BizTalk team started to work on a huge mission - migration from BizTalk Server to TIBCO ActiveMatrix BusinessWorks 6.

TIBCO ActiveMatrix BusinessWorks 6 is one of the most powerful enterprise integration platforms on the market.

As the project leader, one of our goals is to write clear documents which match the development strategy.

So here is the first piece. 

In this document, we would create a "Human" service with two operations: Walk & Sleep.

TibSrv_Human

Create Process

1. Choose Processes --> New --> BusinessWorks Process

2. Set "Package" to "Starters"

Set "Process Name" to "TibSrv_Human"

Press Finish.

Create Service & Operation

3. Now we would create the service inside the process.

Navigate to the green arrow on the right and click it.

4. In the "New Service" pop-up, choose "Create a new WSDL".

Set "File Name" to "TibSrv_Human.wsdl"

Set "Target Namespace" to "http://thisismytest.com/WSDL/TibSrv_Human/TibSrv_Human.wsdl"

Set "Interface Name" to "TibSrv_Human"

Set "Operation Name" to "operation_Walk"

Mark "Output" and "Throw a Fault" with V sign.

Press Next.

5. In the next screen you need to assign a schema to the new wsdl.

Set "File Name" to "TibSrv_Human.xsd"

Set "Target Namespace" to "http://thisismytest.com/Schemas/TibSrv_Human/TibSrv_Human.xsd"

       Press Finish.

After pressing Finish, your workspace should look like this:

6. Press right click on "operation_Walk" --> Operation --> Implement Constructor Operation

After clicking, you should get the following screen:

7. Choose from Project Explorer -->  Module Descriptors --> Components

Under "ComponentTibSrv_Human", choose "TibSrv_Human" and press Properties view.

8. Choose "Bindings" tab, and press "Add Binding" button on the right:

9. Choose "SOAP Binding"

    Press Finish.

10. Set "Name"

a. For HTTP binding - to "TibSrv_Human_HTTP" 

b. For JMS binding - to "TibSrv_Human_JMS" 

Also choose the relevant "Transport Type", "HTTP Connector Name" and "Endpoint URI"

Create Schemas

11. Now we will create Request\Response schemas.

From project explorer choose Schemas --> New --> XML Schema File

12. Set "Name" to "Walk_Request.xsd".

Press Finish.

By pressing double-click on "Walk_Request.xsd" schema, you should see the schema GUI:

13. Set "Target namespace" to "http://thisismytest.com/Schemas/TibSrv_Human/Walk_Request.xsd".

14. Press right click on "Elements", and then choose "Add Element".

15. Set "Name" to "Walk_Request", and from "Type" drop-down-list choose "New"

16. You should see the "NewType" pop-up.

Set to "Complex Type", and mark "Create as local anonymous type" option.

Press OK.

17. Now that you have a new root element, please press double click on it.

You should be moved to this screen:

18. Right click on "(Walk_RequestType)" and click on "Add Element".

19. Set "Name" to "Kilometers".

20. For the response schema (Walk_Response.xsd) repeat 11-19 steps.

Configuring the schemas inside the abstract WSDL

21. On project explorer, open "Service Descriptors" and choose "TibSrv_Human.wsdl".

22. Press the arrow near "operation_WalkRequest".

23. Right click on "in" parameter from "(operation_WalkRequestType)".

Press Delete.

24. Right click on "(operation_WalkRequestType)".

Choose "Add Element Ref".

25. From the "Reference" drop-down list, choose "Browse…"

26. From the "Set element reference" pop-up, Set "Search scope" to "Workspace", and then choose "Walk_Request".

Press OK.

27. Repeat 22-26 steps in order to configure the response schema.

Create another operation

28.       Right Click on the service name à Refactor à Create Operation

29.       From the "Create Operation" pop-up:

Set "Operation Name" to "operation_Sleep"

Choose "Request-Response".

Tick "Throw a Fault" option.

Press OK.

30.       From "Basic Activities", choose "Constructor" shape, and drag it to the process GUI.

31.       Choose the green starter (Marked with "1" in the following image).

Click "Select a Partner Link" arrow (Marked with "2" in the following image), choose "TibSrv_Human".

Click "Select an operation" arrow (Marked with "3" in the following image), choose "operation_Sleep".

32.       From "Basic Activities", choose "Reply" shape, and drag it to the "operation_Sleep" GUI.

33.  Click the "Reply" shape, and Set "Name" to "operation_SleepOut".

Click "Select a Partner Link" arrow and choose "TibSrv_Human".

Click "Select an operation" arrow and choose "operation_Sleep".

34.       Repeat steps 11-26, in order to create the operation schemas and relate them to the service wsdl.

Creating a Sub-Process with TIBCO BW 6.0

A sub-process is actually a process without a component which exposes it. A sub-process will not have a SOAP binding (JMS/HTTP), but would have a wsdl.

1.       On project explorer, choose "Processes" à "New" à BusinessWorks Process

2.   Set "Package" to "Walk"

Set "Process Name" to "WalkDispatcher"

Press Finish.

3.       Now we would create the service inside the process.

Navigate to the green arrow on the right and click it.

4.   In the "New Service" pop-up, choose "Create a new WSDL".

                        Set "File Name" to "WalkDispatcher.wsdl"

Set "Target Namespace" to "http://thisismytest.com/WSDL/TibSrv_Human/WalkDispatcher.wsdl"

Set "Interface Name" to " WalkDispatcher"

Set "Operation Name" to "operation_Walk"

Mark "Output" and "Throw a Fault" with V sign.

Press Next.

5.       We would use WSDL inline schema in order to represent the service.

Choose "Create WSDL inline schema".

Press Finish.

6.       In Project explorer, choose Components.

Delete "ComponentWalkDispatcher".

7.       Now you can go to "WalkDispatcher.wsdl", and define appropriate Request & Response schemas (In each operation defined in the inline wsdl schema. See also 21-26 steps).

Summary

That's it! 
Now you have a Process which contains a Service (with Service WSDL named "TibSrv_Human.wsdl" and "TibSrv_Human.xsd" attached to it). 

"TibSrv_Human.xsd" contains the service operations elements, and inside them you can configure elements or a reference to other schema.

You can now add sub-process invokation to the main process, which is the entry point of all operations: 

On "TibSrv_Human" process, add an invoke shape which calls "WalkDispatcher" sub-process from "operation_Walk".

See you next time with some more interesting stuff!

Using Couchbase NoSQL Database as a caching layer for our BizTalk Processes - Part 2

Previous Parts:

Part 1

Before I'll start, I would like to recommend on "Couchbase Essentials" book by John Lablocki.

That book goes through many important subjects of Couchbase Server, and also includes explanations about using it on .NET.
You can purchase the book on: http://bit.ly/1wtEbwl

"Failover" a node in Couchbase

Couchbase NoSQL concept is to have as many servers as possible in one Couchbase cluster.

Couchbase engine spread cluster data across the cluster nodes.

If one node (= server) goes down, part of the data would be unavailable until:

1. Failed node is up again
2. "Failover" is performed on that failed node.

Faling over a node would eliminate it from cluster, and other nodes would serve its data in memory (by promote replica documents of the failed node to "active" status. Replica documents are placed on disk of each one of the cluster nodes).
Replica documents gets updated when data changes, and used as a backup for in-memory data of each node.

Thumb rule: You can only "Failover" as many nodes as you have replicas.
For instance: If a cluster has 10 servers, and 1 server is down - 10% of the data would be unavailable until failing over that node (manually or automatically) or failed node is up again.

You can read more about Couchbase architechture and replication over here:
http://www.couchbase.com/couchbase-server/architecture

http://www.couchbase.com/docs/couchbase-manual-1.8/couchbase-introduction-architecture-replication.html

Scenario Architecture

Our BizTalk environment contains only 2 servers.
The decision was to keep them in separate clusters for two reasons:
1. If one of those nodes is down, 50% of the cache is unavailable. That is too much.
2. Automatic "Failover" takes 30 seconds (which is too much for us), and can only be configured on clusters which contains 3 nodes or more.

Each of those servers contains its own Couchbase Cluster (= cache), and each cluster backs up the other.

For instance:

If Server A (= cluster A) goes down, Server B is independent and all requests coming from Server B process infrastructure would go to Server B cache.
More than that, if Server A Couchbase windows service is down for some reason (= NoSQL database is down), all requests from process infrastructure on Server A would be routed to Server B (= cluster B). After a configured period of time, requests should hand over again to Server A.

In order to implement backup functionality for clusters, it was needed to write a cache access layer which is built on top of Couchbase Client Library.
In order to keep the cache data identical on both clusters, i've configured an uni-directional XDCR (Cross Datacenter Replication) from Server A to Server B (That means that only Server A loads data from SQL).

To Summarize:

Implement Cache Access with Cluster Backup functionality

First, I've download Couchbase .NET Client Library from http://www.couchbase.com/communities/net/getting-started

I've extracted 3 Dll's from the downloaded zip:
1. Couchbase.dll - Couchbase client dll in order to access Couchbase NoSQL database
2. Enyim.Caching.dll - Memcached dll. Couchbase NoSQL is built on top of it.
3. Newtonsoft.Json.dll - Json.NET is a popular high-performance JSON framework for .NET. I use it to convert JSON documents (from cache) to .NET Dictionary object.

I've added those 3 Dll's to my project, and wrote "CouchbaseClientManager" class:

public class CouchbaseClientManager

    {

        private const string DEAFULT_CLIENT = "couchbase";

        private const string BACKUP_CLIENT = "backup";

        private const string TEST_KEY = "test";

        private Timer timer { get; set; }  // Timestamp for backup to replce the default  

        private string currentClient; // Current client type to create instance

       

        public CouchbaseClientManager()

        {

            this.currentClient = DEAFULT_CLIENT;

            this.timer = new System.Timers.Timer(TimeSpan.Parse(ConfigurationManager.AppSettings["backupTime"]).TotalMilliseconds);

            this.timer.AutoReset = true;           

        }

        public T Get<T>(string key)

        {

            T row;

            if (this.currentClient == DEAFULT_CLIENT)

            {

                if (this.CheckConnectivity(this.currentClient, TEST_KEY))

                {

                    row = this.Get<T>(this.currentClient, key);

                }

                else

                {

                    this.SetCurrentClient(BACKUP_CLIENT); // Setting backup client as the current client

                    this.timer.Elapsed += (sender, arguments) => this.SetCurrentClient(DEAFULT_CLIENT); // When the timer is up, set the current client to default client

                    this.timer.Start();

                    row = this.Get<T>(BACKUP_CLIENT, key);

                }

            }

            else // if currentClient set to BACKUP_CLIENT

            {

                row = this.Get<T>(BACKUP_CLIENT, key);

            }

            return row;

        }

        private T Get<T>(string clientType, string key)

        {

            T row;

            using (var client = new CouchbaseClient(clientType))

            {

                row = client.Get<T>(key);

            }

            return row;

        }

        private void SetCurrentClient(string clientType)

        {

            this.currentClient = clientType;

        }

        private bool CheckConnectivity(string clientType,string key)

        {

            if (System.String.IsNullOrEmpty(this.Get<string>(clientType,key)))

                return false;

            return true;

        }

    }

Few things about the code above:

1. Code reads configurations from config file (Default and Backup cluster configurations and backup time period).

2. Getting a cache row in Json is done by: client.Get<T>(key);

3. Connectivity Check is done by using a "test" key I've places in each cluster.

Here is a flow chart of the code:

Let's take a look on Operations class, which provides a layer for getting data from cache.

public class Operations

    {

        private static readonly Operations instance = new Operations();

        private string urlCacheConfig;

        private List<SyncEntity> syncEntities;

        private CacheConfig.Operations cacheConfigOperations;

        private CouchbaseClientManager couchbaseClientManager;

        static Operations()

        {

        }

        private Operations()

        {

            this.urlCacheConfig = ConfigurationManager.AppSettings["cacheConfigFile"];     

            cacheConfigOperations = new CacheConfig.Operations(this.urlCacheConfig);

            LoadCacheEntities();

            cacheConfigOperations.fileConfigChanged += new System.IO.FileSystemEventHandler(fileConfigChanged);

            this.couchbaseClientManager = new CouchbaseClientManager();

        }

        public static Operations Instance

        {

            get

            {

                return instance;

            }

        }

        private void LoadCacheEntities()

        {            

            syncEntities = cacheConfigOperations.GetCacheItems();           

        }

        private void fileConfigChanged(object sender, System.IO.FileSystemEventArgs e)

        {

            LoadCacheEntities();

        }

        public IDictionary GetDictionary(string table, string key)

        {

            SyncEntity tableSyncEntity = syncEntities.Where(se => se.tableName.Equals(table)).FirstOrDefault();

            if (tableSyncEntity == null)

                return null;

            string column = tableSyncEntity.keys[0];

            string fullKey = table + "_" + column + "_" + key;

            Dictionary<string, string> dicRow;

            string row = this.couchbaseClientManager.Get<string>(fullKey);

            System.Web.Script.Serialization.JavaScriptSerializer js = new System.Web.Script.Serialization.JavaScriptSerializer();

            dicRow = js.Deserialize<Dictionary<string, string>>(row);

            return dicRow;

        }

    }

Operations class keeps entities (List<SyncEntity>) from config in memory.
If config file is changed, Operations loads it again. BizTalk process infrastructure would use "GetDictionary" method in order to get a row from cache.
Each table has a constant key column, and we needed to get data by that column value.

For instance:

If we have defined "Name" as key column, then running GetDictionary("Products","Phone"); would return a key-value dictionary which contains:

table - Cached table name.

key - Value of column.

Client configuration file look like this one, with minor changes (like server and database names etc.):

<?xml version="1.0"?>

<configuration>

  <configSections>   

    <section name="couchbase" type="Couchbase.Configuration.CouchbaseClientSection, Couchbase, Version=1.2.6.0, Culture=neutral, PublicKeyToken=12b9c6b5a9ec94c3"/>

    <section name="backup" type="Couchbase.Configuration.CouchbaseClientSection, Couchbase, Version=1.2.6.0, Culture=neutral, PublicKeyToken=12b9c6b5a9ec94c3"/>

  </configSections>

  <connectionStrings>

    <add name="connectionString" connectionString="Persist Security Info=False;Integrated Security=True;Initial Catalog=DBName;server=SQLServerAddress" />

  </connectionStrings>

  <appSettings>

    <add key="cacheConfigFile" value="C:\CouchbaseCache\BTSCachingTasksConfiguration.xml"/>

    <add key="backupTime" value="00:01:00"/>

  </appSettings>

  <couchbase>

    <servers bucket="default" bucketPassword="private">

      <add uri="http://currentServer:8091/pools/default"/>

    </servers>

  </couchbase>

  <backup>

    <servers bucket="default" bucketPassword="private">

      <add uri="http://otherServer:8091/pools/default"/>

    </servers>

  </backup>

  <startup>

    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>

  </startup>

</configuration>

Here are some remarks regarding config file:

1. 

BTSCachingTasksConfiguration.xml - Contains table names and columns keys to cache.

For instance:

<?xml version="1.0" encoding="utf-8"?>

<ArrayOfCachingTask>

  <entity>

   <connectionKey>NotRelevant</connectionKey>  

   <tableName>Products</tableName>

   <keys>Name</keys>

   <timer>12:00:00</timer> 

  </entity>

</ArrayOfCachingTask>

As you can see, cached table name is "Products", and column key is: "Name".

"timer" is an interval value to load that table again (in order to refresh it on cache).
That load isn't done by that client library. It's done by a windows service I wrote that runs on background.

2. Connection String to SQL database is used when BTSCachingTasksConfiguration.xml is changed, and data gets load again from SQL to Couchbase NoSQL (cache). The load process is done by CacheConfig.Operations class.

3. BizTalk is working only with signed dll's placed on GAC, therefore I put all relevant assemblies there.

What's next?

Part 3 would speak about a windows service I wrote to initialize cache, and refresh it on a configured time interval. I would also go into detail of Coucbase Console Administration and Map-Reduce functions.