Developing BizTalk Save File pipeline component

Hi all,

Case scenario for a BizTalk developer:
You need to receive a flat file through a receive location, and one of the customer requirements is
to save the file. This flat file should go through a custom pipeline with ‘Flat File Disassembler’ component in order to create flat file schema.

Reminder: when using BizTalk 'FILE' adapter, it deletes the file from the file system or network share.

So how can we solve this issue ?

- The idea to create a 'Send port' pointing to target location and using filters - would work only if the file data passed the pipeline stage successfully (include parsing and flat file disassembler).
In addition, we should use send pipeline with 'Flat file assembler' (Transforming
flat file schema back to the original flat file data). This will require another send pipeline for each application.

- We can use 'PassThruReceive' pipeline on a Receive Port, which will transform the flat file to XML and send it to the MessageBox. In addition, we will create a Send Port (‘PassThruTransmit’ Pipeline) with a filter on the Receive Port, which will send XML file to an archive folder we’ll choose. After this step, because we need to use ‘Flat File Disassembler’ on that file, we will create another send port which will send it to another location (The process will start to run from that folder). So that way is possible, but it requires two more Send Ports for the application. Here is the diagram for that solution:

--> -->

-->In conclusion: Getting an out of the box solution is not so elegant in that case.

Let's go for developing a simple component for our custom pipeline:
'Save File Assembler Component'. The only way to add custom functionality to a ReceivePort is through an adapter or a pipeline component. Writing a custom FILE adapter would be complicated, so using a custom pipeline component is the better solution.

1. Add a 'Class Library' project named 'SaveFilePipelineComponent'.
2. Implement those interfaces: IBaseComponent - in order to define the name, version and description of the component. IComponentUI - in order to use the pipeline component within the Pipeline Designer environment.
Microsoft.BizTalk.Component.Interop.IComponent - in the 'Execute' method, we will process
the input message. IPersistPropertyBag - in order to configure our pipeline at Design Time.
save the file to a new location.

-->

[ComponentCategory(CategoryTypes.CATID_PipelineComponent)]

[ComponentCategory(CategoryTypes.CATID_Decoder)]

[ComponentCategory(CategoryTypes.CATID_Any)]

[ComponentCategory(CategoryTypes.CATID_Validate)]

[ComponentCategory(CategoryTypes.CATID_Streamer)]

[System.Runtime.InteropServices.Guid("9FA15923-E1F5-4ea1-A58E-CC09BEE58D11")]

public class SaveFileComponent : IBaseComponent, IComponentUI, Microsoft.BizTalk.Component.Interop.IComponent, IPersistPropertyBag

-->3. Creating pipeline properties is done with implementing the IPersistPropertyBag interface.

-->'Save' method, saves the properties to propertyBag object at Design Time :

-->

void IPersistPropertyBag.Save(IPropertyBag propertyBag, bool clearDirty, bool saveAllProperties)

{

object propertyObject = (object)ArchiveFolderPath;

propertyBag.Write("ArchiveFolderPath", ref propertyObject);

propertyObject = (object)UserName;

propertyBag.Write("UserName", ref propertyObject);

propertyObject = (object)Password;

propertyBag.Write("Password", ref propertyObject);

}

-->'Load' method, reading the properties from properyBag object at Design Time and Run Time:

-->

void IPersistPropertyBag.Load(IPropertyBag propertyBag, int errorLog)

{

object propertyObject = null;

try

{

propertyBag.Read("ArchiveFolderPath", out propertyObject, 0);

ArchiveFolderPath = propertyObject != null ? propertyObject as string : String.Empty;

propertyBag.Read("UserName", out propertyObject, 0);

UserName = propertyObject != null ? propertyObject as string : String.Empty;

propertyBag.Read("Password", out propertyObject, 0);

Password = propertyObject != null ? propertyObject as string : String.Empty;

}

catch (ArgumentException argEx) // IMPORTANT! Handles the first initialize case (When adding the component at Design Time) - all the properties are null and the exception is catched but not thrown!

{

}

catch (Exception ex)

{

throw new ApplicationException("Error reading propertybag: " + ex.Message);

}

}

-->4. Let's focus on the 'Execute' method:

-->

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)

{

try

{

IBaseMessageContext context = inmsg.Context;

string bodyReceived = this.ReadMsgBody(inmsg);

string targetPath = new AdapterFilePath(inmsg, ArchiveFolderPath, UserName, Password).GetTargetPath();

this.WriteStringToPath(bodyReceived, targetPath);

}

catch (Exception e)

{

const string source = "BizTalk Custom Pipeline: SaveFileComponent";

const string logType = "Application";

if (!EventLog.SourceExists(source))

EventLog.CreateEventSource(source, logType);

EventLog.WriteEntry(source, e.Message, EventLogEntryType.Warning);

}

return inmsg;

}

-->First step: Getting the message context to IBaseMessageContext object.

Second step: A call to 'ReadMsgBody' and reading the message body with a streamer:

-->

private string ReadMsgBody(IBaseMessage pInMsg)

{

string bodyReceived = String.Empty;

Stream originalDataStream = pInMsg.BodyPart.GetOriginalDataStream();

StreamReader readMsgStream = new StreamReader(originalDataStream);

bodyReceived = readMsgStream.ReadToEnd();

do

{

readMsgStream.BaseStream.Seek(0, SeekOrigin.Begin);

}

while (readMsgStream.BaseStream.Position != 0);

return bodyReceived;

}

-->Third step: Getting the target path by using 'AdapterFilePath' class. This class performs quite simple, so I won't get in to it. In general, the class gathers the information it needs (i.e: file name, adapter type) from the message context and builds the target path:

-->

///

/// System namespace

///

private const string systemNamespace = "http://schemas.microsoft.com/BizTalk/2003/system-properties";

///

/// Getting the adapter type from a message received

///

public static string GetAdapterType(IBaseMessage inMsg)

{

return inMsg.Context.Read("InboundTransportType", systemNamespace) as string;

}

///

/// Getting the adapter type namespace from a message received

///

public static string GetAdapterTypeNamespace(IBaseMessage inMsg)

{

string adapterType = GetAdapterType(inMsg).ToLower();

return "http://schemas.microsoft.com/BizTalk/2003/" + adapterType + "-properties";

}

///

/// Getting the file name path from a message received

///

public static string GetFileNamePath(IBaseMessage inMsg)

{

string adapterTargetNamespace = GetAdapterTypeNamespace(inMsg);

return inMsg.Context.Read("ReceivedFileName", adapterTargetNamespace) as string;

}

-->

Last step: Writing the message body to the target path by using WriteStringToPath method.

-->

///

/// Writing the string to the path given

///

private void WriteStringToPath(string xmlReceived, string targetPath)

{

using (TextWriter writeMsgTextWriter = new StreamWriter(targetPath))

{

writeMsgTextWriter.Write(xmlReceived);

writeMsgTextWriter.Flush();

}

}

-->

5. Build 'SaveFilePipelineComponent' project and place the dll at:

C:\Program Files\\Pipeline Components

6. Restart Visual Studio.

7.
- Right click on the toolbox --> 'Choose Items'.
- Add the component from 'BizTalk Pipeline Components'. - Drag the component to the 'Decode' stage:

-->The component is built for the 'Decode' stage for two reasons:
1. It's the earliest stage, and we want to keep the original file.
2. We can run several components on that stage. That means the BizTalk engine
will run several 'Execute' methods on different components placed on that stage ('Execution mode' is set to 'All').

That's all for now,

See you next time.