TIBCO Products - JMS vs. Rendezvous, Queue vs. Topic, Route vs. Bridge

Hi again,

When dealing with TIBCO products you can easily get confused by many related buzz words or methodologies.
I've decided to write a brief comprasion.

Rendezvous	JMS
Radio broadcaster	Telephone	Real-life example
Yes	No	Sending messages to all clients
No	Yes	Requires server
No	Yes	Persistence
No	Yes	Reliable messaging
Yes	No	High-Speed messaging
-	Yes	Clustering & Failover

  

 

 

Topic	Queue
Publish/Subscribe model	Point-to-Point model	Architecture
Multiple clients subscribe to the message	Only one consumer gets the message	Number of clients
No	Yes	Messages sent by order
No	Yes	Messages are processes only once
No	Yes	Destination is known
Yes	No	Consumer needs to be active
No	Yes	Consumer Acknowledgement

 

 

 

Bridge	Route
On the same server	On different server	Route messages to a queue
Yes	No	Possible to have different type of source and destination (For instance: queue to topic)
No	Yes (Topic only)	Can be transitive   (a->b->c means a->c)
Yes	Yes (Topic only)	Filters via selectors
Yes	No	Restart JMS server when config changes
-	Single-Hop	Hops allowed
bridges.conf	routes.conf	Config file

Tips for a good ESB integration system

1. Build SOA Environment –

Share common logics around processes with exposed services.

Benefits:

Changes will be easy – If a common logic gets changed, then you would only need to deploy a service again and each process would work with the new logic.

Services logic would be shared around ALL applications – 

These days, almost every develop tool or language is able to consume a web-service. That means you can share logic easily around an organization. Use it in order to minimize duplicate code and maintenance.

2. Less-Code and Out Of The Box solutions –

More code = More complexity & maintenance. 

Use products for minimum coding, reliable solutions and support from professional vendors.
 

 

3. End-To-End Tracing –

A good BPM would always have tracing and information regarding messages being transferred in the wire.
 
Suppose message doing this route -
Application A --> Integration System --> Application B, Application C, Application D.
Make sure there's a field to identify the message in each step and each application writes to the same trace log.

Application A writes that message 12203 was created in a folder.
Integration System writes that message 12203 was taken and process started.
Integration System writes that message 12203 is now sent to Application B, Application C, Application D.
Application B writes that message 12203 was received.
etc.

Write important business and technical process data.

Many organizations use SQL Server for tracing but the innovative solution is to use ElasticSearch which includes free-text search on a huge mass of data.

Building reports for customers (based on those traces) is always a good thing because they can help with maintenance.

 

4. Cache layer for lookups –

Reading and writing to and from memory is faster than any I/O.

If many processes uses configurations or lookup values stored in SQL Server then consider an appropriate cache layer which uses memory to store data.

Redis & Memcached are classic for caching, but you can also use NoSQL solutions which stored data in memory first before being persisted (e.g couchbase).

5. XSLTs for Mapping -

DON'T write C#, Java or whatever language code in order to transform XML's.

Develop XML maps ONLY with XSLTs which can be reused in any integration tool which deals with XML transformation.

 

6. Use Stored Procedures for queries -

It's better for your application to execute stored procedure for queries, rather than sending those as text to the DB.

It's easier for a DBA to control & manage organization logic, and to avoid "BAD" queries.

 

7. Dynamic configuration –

Think deeply on every process configurations you should be able to modify on runtime without changing any code.

Don't exaggerate with dynamic configurations because it has a maintenance price. You do not want an interface with a huge number of configurations, if most of them are values which will be remain unchanged. 

 

8. Well-defined schemas –

It's important that schema nodes, elements and attributes would have real limitations (Min/Max occurs, Value types etc.).

It's possible to restrict schema field value types in many ways (int, string, regular expressions, enums etc.).

Benefits:

Data integrity – Well, that's the obvious reason. Data is more reliable on each point of entire process.

Less complexity in process – Suppose a process need to get an e-mail address from a request message.

If schema would enforce a valid e-mail regex in that field, then there is no need
to perform this validation on process.

 

9. ASync web-services is better than Sync web-services –

If there is a possibility of consuming asynchronous web-service – ALWAYS prefer it over a synchronous web-service.

Synchronic method (Two-Way) would open a session, send a request, wait for a response and only then closes the session.

Asynchrony method (One-Way) would open a session, send a request and closes the session immediately without waiting for a response.

More sessions = More CPU and more memory usage on server.

Take this tip in mind especially with offline interfaces. In those kind of processes the execution doesn't has be immediate so entire interface can last longer. 

 

10. Avoid direct-connection between Production & Test environments –

Some organizations tend to send messages from production to test\development environments.

That's really bad because if a resource isn't available in test\development then production would be affected.

 

 

11. Identical backup environment (or clustered in an active-passive mode) –

Maintain a backup (not-activated) environment for production (an exact copy of it).

Make sure backup environment is always up to date with the latest changes of the WORKING production environment.

 

12. Retry mechanism when using resources –

Consider using retry mechanism on a process which uses resources (SQL, Web-Services etc.).

Take notice you can implement retry only on OFFLINE processes/ASync services (i.e. when no one waits with an open session for an answer from that service).

 

13. Centralized production environments -

With the major communication progress during the last few years, geographic distance is not an issue as it used to be.

Less environments = Less maintenance.

A good centralized environment contains distributed servers which share data processing.
Some integration tools like "BizTalk" includes load balancing mechanism out of the box, but if your tool doesn't - a load balancing software would help.

 

14. Health and performance monitor system –

Monitor & alert crucial resources which is used by processes to early identify issues. "SCOM" and "Nagios" are popular products. Use "Watcher" for elasticsearch logs.

For instance: 

If a server CPU stays on 100% for the last 30 seconds – send an e-mail. 

If files are piling up in a directory which should be empty – send an SMS.

 

14. Archive source messages\requests –

Gives power for an integration system to rerun messages in case process went wrong and to be self-dependent from end-to-end.

 

15. Persistence –

Some integration tools (like BizTalk, which is DB-based) have persistence functionality built-in as part of their engine.
Products that put an emphasis on performance doesn't have it (e.g TIBCO) but it's possible to implement persistence logic inside a process.
Analyze each process before implementing and think when persistence is necessary if everything breaks down.

 

16. Parallel processing –

Use parallel processing if process logic allows it but beware from over-complex.
Keep in-mind that code should be thread-safe.

Authenticate with Windows Authentication (using JDBC Resource) in TIBCO BW 6

Hi,

In this post we will learn how to use Windows Authentication in TIBCO BW 6.

Follow those steps:

1. Download Microsoft JDBC drivers for sql server from:

https://www.microsoft.com/en-us/download/details.aspx?id=11774

2. Extract the contents of the download driver and copy “sqljdbc_auth.dll” (inside <installation directory>\sqljdbc_<version>\<language>\auth\) to \Windows\System32 folder.

3. In your JDBC Connection item: choose the appropriate sql driver, and use the following connection string (with integrationSecurity=true):
jdbc:sqlserver://MyServerName;databaseName=MyDBName;integratedSecurity=true;

You still need to provide a dummy user name and password. BW6 does not allow an empty user name and password.

SOA environment with TIBCO BW 6

Hi,

As I've mentioned in my previous posts, my team at Clalit Health Services is in charge of all interfaces in Clalit's hospitals and community clinics.

These days we work on a migrate process from BizTalk 2010 to TIBCO BW 6.

That actually means developing all organization interfaces from scratch.

PROBLEM BACKGROUND

TIBCO BW 6 is a brand new product, so we have spent a lot of hours figuring how exactly we should work with it. Of course we ran into some product bugs (TIBCO were really helpful and even fixed some in later versions), but the big question was how to build a SOA environment (services has to be loosely-coupled).

For instance, let's say "Service A" needs to use "Service B".

"Service A" should have the option to invoke "Service B" by consuming its wsdl (using "Binding Reference"), but do NOT invoke "Service B" process directly (using "Process Binding").

"Binding Reference" advantages:

1.       "Service A" and "Service B" are loosely-coupled, and there is more control over the messages being passed (because the use of SOAP - via JMS or HTTP). All of our TIBCO services are exposed with JMS, and SOA allows full control on the messages going in and out from queues.

2.       An easy deploy after a change in service functionality. A change in service functionality will only affect the deployment package of the service itself. After deploying a changed service again, all other applications invoking it with "Binding Reference" would use the new service functionality.

"Process Binding" (tightly-coupled services) disadvantages: 

1. There is no control over messages being sent from one to another (because they are sent using TIBCO internal engine and not with SOAP).
2. When a service process gets changed – you would have to redeploy all applications that refer to it with "Process Binding" (because the service functionality is included in each application deployment package).

TIBCO Project types

TIBCO 6 project types:

Shared Module

A unit of shared resources.

In a short brief, whenever you want to share some resources across application modules – use should put them inside a "shared module" project.

(TIBCO doc: https://docs.tibco.com/pub/activematrix_businessworks/6.2.0/doc/html/GUID-FF60A1AA-4859-4E5F-A353-71DB4704FB5A.html)

Application Module

Application module is actually a shared module, but without the "sharing" part.

In contrast to shared modules, application modules cannot export its functionality.

Each application module can use (include) several shared modules.

(TIBCO doc: https://docs.tibco.com/pub/activematrix_businessworks/6.2.0/doc/html/GUID-AC119968-4F28-4ECA-878B-4EB251B70444.html)

Application

Contains the deployment package.

(TIBCO doc: https://docs.tibco.com/pub/activematrix_businessworks/6.2.0/doc/html/GUID-B09ABEE9-C6AD-4053-AB00-D6677F7DC2C9.html)

WSDL Types

Abstract wsdl 

A contract of each TIBCO process (service). Create an abstract wsdl when you develop a TIBCO process (service). It contains input, output and fault operations (each defined by an external or inline schema) for the process.

Concrete wsdl

Used for consume of a TIBCO process (service), because it contains a definition of "PortType". Create a concrete wsdl after you finish developing a TIBCO process that needs to be exposed (service).

You can set several types of concrete wsdl:

1.       Concrete wsdl which embed its abstract wsdl

A concrete wsdl has to know its abstract wsdl. You can actually embed a service abstract wsdl inside its concrete wsdl.

Why not? Whenever the service abstract wsdl would change during development, then you will have to regenerate (or change) the concrete wsdl as well.

Instead, hold inside the service concrete wsdl a reference to the service abstract wsdl using "wsdl:import" tag.

For instance:

<wsdl:import namespace="http://myns.com/WSDL/TibSrv_Cache.wsdl" location="TibSrv_Cache.wsdl"/>

("TibSrv_Cache.wsdl" is the abstract wsdl)

2.       Concrete wsdl which embed schemas

A concrete wsdl can contain all required schemas definitions in it.

Why not? If schema is defined more than once (For instance: inside a .xsd file and inside a service wsdl) in the same project, it will cause namespace collisions.

Instead, hold inside the service abstract wsdl a reference to the required schemas by adding "wsdl:types" and "xsd:import" tags.

For instance:

<wsdl:types>

                <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">

                                <xsd:import namespace="http://myns.com/Schemas/Fault"                     schemaLocation="../Infrastructure/tibco.infrastructure.shared/Schemas/Fault.xsd">

                                </xsd:import>

</xsd:schema>

</wsdl:types>

3.       A light-weight service concrete wsdl which holds a reference to its abstract wsdl.

As you probably realized from 1 & 2 paragraphs – use option 3.

NOTES:

1. Writing reference links (locations) in an abstract or a concrete wsdl isn't enough.

All resources required for a concrete wsdl (abstract wsdl resource) and an abstract wsdl (schemas resources), should be placed in the same project.

2. Generate a concrete wsdl with embedded abstract wsdl and schemas for other clients (like .NET, JAVA etc.) that needs to invoke a TIBCO BW service.

Implementation

Use shared modules to create a small "expose" unit for each service.

For instance: Creating an "expose" shared module for Service A.

"Service A shared module" would contain a concrete wsdl, an abstract wsdl and schemas.

As mentioned before, the concrete wsdl reference to the abstract wsdl, which reference to the schemas.

If "Service B" wants to invoke "Service A" - it would add "Service A" shared module to its service appliaction module.

Environment architecture diagram:

Application C, Application D – represents typical TIBCO applications with processes that doesn't exposed as a web-service.

ESB Service A, ESB Service B – represents typical TIBCO applications with processes that exposed as a web-service.

ESB Service A shared module – Contains shared resources of ESB Service A (its abstract wsdl, concrete wsdl and schemas). Basically, it's all resources that Service A concrete wsdl needs.

ESB Service B shared module – Contains shared resources of ESB Service B (its abstract wsdl, concrete wsdl and schemas). Basically, it's all resources that Service B concrete wsdl needs.

Infrastructure shared module – Contains shared resources across ALL modules (for instance: fault schemas, trace schemas, infrastructure sub-processes etc.).

Diagram explanation:

1 - "ESB Service A" includes "ESB Service A shared module" –

The service process inside "ESB Service A" application module needs its abstract wsdl and schemas resources from "ESB Service A shared module".

3 - "ESB Service A" includes "ESB Service B shared module" –

"ESB Service A" wants to invoke "ESB Service B", so it needs "ESB Service B" concrete wsdl (placed in "ESB Service B shared module").

2 - "ESB Service A" includes "Infrastructure shared module" –

"ESB Service A" needs to use global resources.

For instance: process which uses a shared trace schema.

4 - "ESB Service A shared module" includes "Infrastructure shared module" –

"ESB Service A shared module" needs to use global resources.

For instance: if the abstract wsdl points to a shared fault schema.

Application "Includes" example:

How to consume service in the process ?

After including a service shared module project, just use "Invoke" shape in the process. 

Choose "Binding Reference", then the service abstract wsdl and at last add a SOAP binding. 

Under "WSDL Port:" choose the service port type (which is defined in the concrete wsdl).

Creating a Service, WSDL, Schema for WSDL and operations with TIBCO BW 6.0

Hi,

Recently our BizTalk team started to work on a huge mission - migration from BizTalk Server to TIBCO ActiveMatrix BusinessWorks 6.

TIBCO ActiveMatrix BusinessWorks 6 is one of the most powerful enterprise integration platforms on the market.

As the project leader, one of our goals is to write clear documents which match the development strategy.

So here is the first piece. 

In this document, we would create a "Human" service with two operations: Walk & Sleep.

TibSrv_Human

Create Process

1. Choose Processes --> New --> BusinessWorks Process

2. Set "Package" to "Starters"

Set "Process Name" to "TibSrv_Human"

Press Finish.

Create Service & Operation

3. Now we would create the service inside the process.

Navigate to the green arrow on the right and click it.

4. In the "New Service" pop-up, choose "Create a new WSDL".

Set "File Name" to "TibSrv_Human.wsdl"

Set "Target Namespace" to "http://thisismytest.com/WSDL/TibSrv_Human/TibSrv_Human.wsdl"

Set "Interface Name" to "TibSrv_Human"

Set "Operation Name" to "operation_Walk"

Mark "Output" and "Throw a Fault" with V sign.

Press Next.

5. In the next screen you need to assign a schema to the new wsdl.

Set "File Name" to "TibSrv_Human.xsd"

Set "Target Namespace" to "http://thisismytest.com/Schemas/TibSrv_Human/TibSrv_Human.xsd"

       Press Finish.

After pressing Finish, your workspace should look like this:

6. Press right click on "operation_Walk" --> Operation --> Implement Constructor Operation

After clicking, you should get the following screen:

7. Choose from Project Explorer -->  Module Descriptors --> Components

Under "ComponentTibSrv_Human", choose "TibSrv_Human" and press Properties view.

8. Choose "Bindings" tab, and press "Add Binding" button on the right:

9. Choose "SOAP Binding"

    Press Finish.

10. Set "Name"

a. For HTTP binding - to "TibSrv_Human_HTTP" 

b. For JMS binding - to "TibSrv_Human_JMS" 

Also choose the relevant "Transport Type", "HTTP Connector Name" and "Endpoint URI"

Create Schemas

11. Now we will create Request\Response schemas.

From project explorer choose Schemas --> New --> XML Schema File

12. Set "Name" to "Walk_Request.xsd".

Press Finish.

By pressing double-click on "Walk_Request.xsd" schema, you should see the schema GUI:

13. Set "Target namespace" to "http://thisismytest.com/Schemas/TibSrv_Human/Walk_Request.xsd".

14. Press right click on "Elements", and then choose "Add Element".

15. Set "Name" to "Walk_Request", and from "Type" drop-down-list choose "New"

16. You should see the "NewType" pop-up.

Set to "Complex Type", and mark "Create as local anonymous type" option.

Press OK.

17. Now that you have a new root element, please press double click on it.

You should be moved to this screen:

18. Right click on "(Walk_RequestType)" and click on "Add Element".

19. Set "Name" to "Kilometers".

20. For the response schema (Walk_Response.xsd) repeat 11-19 steps.

Configuring the schemas inside the abstract WSDL

21. On project explorer, open "Service Descriptors" and choose "TibSrv_Human.wsdl".

22. Press the arrow near "operation_WalkRequest".

23. Right click on "in" parameter from "(operation_WalkRequestType)".

Press Delete.

24. Right click on "(operation_WalkRequestType)".

Choose "Add Element Ref".

25. From the "Reference" drop-down list, choose "Browse…"

26. From the "Set element reference" pop-up, Set "Search scope" to "Workspace", and then choose "Walk_Request".

Press OK.

27. Repeat 22-26 steps in order to configure the response schema.

Create another operation

28.       Right Click on the service name à Refactor à Create Operation

29.       From the "Create Operation" pop-up:

Set "Operation Name" to "operation_Sleep"

Choose "Request-Response".

Tick "Throw a Fault" option.

Press OK.

30.       From "Basic Activities", choose "Constructor" shape, and drag it to the process GUI.

31.       Choose the green starter (Marked with "1" in the following image).

Click "Select a Partner Link" arrow (Marked with "2" in the following image), choose "TibSrv_Human".

Click "Select an operation" arrow (Marked with "3" in the following image), choose "operation_Sleep".

32.       From "Basic Activities", choose "Reply" shape, and drag it to the "operation_Sleep" GUI.

33.  Click the "Reply" shape, and Set "Name" to "operation_SleepOut".

Click "Select a Partner Link" arrow and choose "TibSrv_Human".

Click "Select an operation" arrow and choose "operation_Sleep".

34.       Repeat steps 11-26, in order to create the operation schemas and relate them to the service wsdl.

Creating a Sub-Process with TIBCO BW 6.0

A sub-process is actually a process without a component which exposes it. A sub-process will not have a SOAP binding (JMS/HTTP), but would have a wsdl.

1.       On project explorer, choose "Processes" à "New" à BusinessWorks Process

2.   Set "Package" to "Walk"

Set "Process Name" to "WalkDispatcher"

Press Finish.

3.       Now we would create the service inside the process.

Navigate to the green arrow on the right and click it.

4.   In the "New Service" pop-up, choose "Create a new WSDL".

                        Set "File Name" to "WalkDispatcher.wsdl"

Set "Target Namespace" to "http://thisismytest.com/WSDL/TibSrv_Human/WalkDispatcher.wsdl"

Set "Interface Name" to " WalkDispatcher"

Set "Operation Name" to "operation_Walk"

Mark "Output" and "Throw a Fault" with V sign.

Press Next.

5.       We would use WSDL inline schema in order to represent the service.

Choose "Create WSDL inline schema".

Press Finish.

6.       In Project explorer, choose Components.

Delete "ComponentWalkDispatcher".

7.       Now you can go to "WalkDispatcher.wsdl", and define appropriate Request & Response schemas (In each operation defined in the inline wsdl schema. See also 21-26 steps).

Summary

That's it! 
Now you have a Process which contains a Service (with Service WSDL named "TibSrv_Human.wsdl" and "TibSrv_Human.xsd" attached to it). 

"TibSrv_Human.xsd" contains the service operations elements, and inside them you can configure elements or a reference to other schema.

You can now add sub-process invokation to the main process, which is the entry point of all operations: 

On "TibSrv_Human" process, add an invoke shape which calls "WalkDispatcher" sub-process from "operation_Walk".

See you next time with some more interesting stuff!