On MSIs and Default Bindings

Some observations (I’ll you to draw your own conclusions, for a change) -

• When you import an MSI, any default binding in it will always be imported, then any binding matching the environment selected in the import MSI wizard (if selected)
• If the second binding contains any settings that were already created as part of the first binding, they will be overwritten
• When you export an MSI, a binding file is created with the existing configuration as the default binding, even if you did not add one as a resource explicitly. This binding will be imported first, before any other binding.
• In the case of receive locations – if two bindings in the MSI contain receive locations for the same port each marked as primary, the last one imported will be the primary receive location.

Why am I writing this?

We had an odd case (which shouldn’t really happen) where we created an application with a particular receive location.
We have then added a binding file for another environment in which we have misspelled the receive location name (the URI was the same)

When we tried to import the MSI to the second environment, selecting the environment name in the import MSI wizard we received an error that looked something like this -

(for the search engines: “Cannot update receive location”, “Address should be unique for Receive Location”)

The reason was that the wizard imported the MSI and initially applied the default binding creating the receive location with the first name. It then went on to import the second, environment specific, binding file, now trying to add a new receive location with the different name but failing as the URIs are the same. had the name been the same the receive location would have been overwritten, had the URI been different a second one would have been successfully added, but given this weird combination the error was produced.

Oslo based solution for deploying BizTalk applications – published

Unfortunately life is a bit hectic for us at the moment, and it’s been a while since I’ve posted anything, or was able to do anything other than family or work.

As part of this general “neglect” I was unable to spend the time required to complete, and publish, my Oslo based solution for deployment BizTalk applications (given the somewhat ugly, but suitably short, name – BTSDF), but as many waited (ok – more than 5), and some kept asking (2), I thought I’d do my best to get something out of the door, so I have.

 

in Codeplex  you can now find an initial version published, which includes – the language definition, the core runtime I have as well as two “executors” in various stages of [non] completeness – my MsBuild generator is already quite useful for simple apps, it generates a set of MSbuild files you can use to deploy the application on any machine as well as the required SDC tasks dlls and targets file; the API executor deploys the app on the current machine, and is quite basic, but a good sample (I think) and a reasonable starting point.

I’m happy to entertain requests for changes/additions, and even more happy to add any one who’s willing and able to put the time as a contributor; Oslo knowledge is optional! :-)

 

You can download the source, build and run it locally – but you will have to remember to copy the two executors dlls from their folders to the main bin\debug folder as there is no compiled reference to them.

 

I’ve also uploaded an “Alpha release” which includes the compiled assemblies and the supporting files required.

There’s still more work to do, but it’s getting shape now, and I’m using the MSbuild executor today; it certainly needs a bit more documentation, which I hope to get around to…

 

I hope this works for you, let me know what you think (the good and the bad), and – if you have some time and will – drop me a line and I’ll add you to the team!

 

Yossi

MSBuild support in BizTalk 2009

In my notes on BizTalk 2009 post I have highlighted the fact that now, as part of the BizTalk installation, you can select the “Project Build Components” which allow you to build BizTalk projects without Visual Studio or BizTalk installed on the build machine.

This is very cool feature of the product, but I have to admit that I have missed something quite important; luckily - Mikael Hakansson hadn’t – the provided tasks do not support deployment of BizTalk projects, but only build (see Mikael’s article here).

What this means is that, while you can use the tasks deployed through the installation to verify that your BizTalk projects build correctly (as part of continuous integration setup, for example), you have no way of verifying that they can be deployed or, more importantly, run any automated tests;

The SDC tasks, with their support for BizTalk deployment, can come in very handy here, and you can certainly use them to augment the built in build support with deployment capabilities, but they add more dependencies to the plate, which has to be planned for.

Of course, attending the MVP summit in Redmond last week we had a great chance of raising this (and various other) subjects with the product group, and we certainly have not let that opportunity go missed, now we just need to wait and see if they “pick up the glove”.

Oslo based solution for deploying BizTalk applications

Since PDC I’ve been working on and off on an “Oslo” based solution for deploying a BizTalk application; unfortunately I couldn’t get a good chunk of time to play with this, so it’s been dragging a bit, but I’m getting close, so here are some details -

I’m a big advocate of automated builds; it’s a topic that probably deserves a post of its own, so I won’t get started on this here, but the idea is that one must have a way to be confident that, when its time to [re-]deploy the app, it will get deployed successfully time after time, without a hitch;

From my experience, deploying a BizTalk application of a decent size is often not trivial, and often it does not go smoothly if done manually (things are being missed, done out of order, wrong versions being picked up, etc), which does not help to boost the confidence in the solution (or BizTalk as a whole) in the organisation; automating the process can save a lot of time (can be done unattended, while in a meeting, out for lunch or overnight), save a lot of head scratching, boost the confidence in the solution and set the ground for proper automated builds, auto testing etc.

To that extent, I have previously built an MSBuild based framework for deploying BizTalk applications.

While I’m sure it does not suit all purposes as it was developed to support those scenarios I had, I think it’s quite comprehensive and had served me well over the past couple of years.

It allows one to provide an MSBuild-formatted file with some parameters and lists and, using the pre-created framework, would do things like remove the existing application (after terminating current instances and un-enlisting services), build the new solution(s), deploy assemblies to BizTalk, add/import binding files, start ports etc.

This is working great for us, and we’ve been using it extensively for quite a while now, but there’s one major downside – it requires one to maintain those MSBuild scripts.

Now -  MSBuild is cool, I do like it very much, but as it’s generic – it does not speak in BizTalk terms, and as it’s XML – it’s quite verbose, and I wanted an easier format for all of us to work with.

So – inspired after seeing Oslo and even more driven after visiting PDC - I’ve decided to come up with an MGrammar based language for describing a BizTalk application to be deployed.

So far I’ve come up with “version 1.0” of my grammar that allows to describe the major artifacts in a BizTalk application and a basic runtime to process source files written in this language, the language allows you to describe things like -

• References to other applications
• Solutions and projects to build
• BizTalk assemblies to deploy
• Orchestrations contained in those (these can then be terminated before attempting to remove the application prior to deployment)
• .net assemblies to deploy
• Binding file to import
• Binding files to add (and the name of the environment to attach to those)

As Oslo hasn’t RTM-ed yet, I can’t quite rely on it yet, and so I cannot use it in production in any shape or form, or can I?
I found a good middle ground for us which allows us to gain from the benefits I’m hoping to get by using my language, while not exposing ourselves too much to the risks of using early technology –

At the moment the runtime I’ve built for this is used to generate the MSBuild scripts we’re already using out of the source files; in this way - if Oslo disappears tomorrow, or significantly changes (not that I think that’s going to be the case), we’re safe – we still have the MSBuild scripts as “checked-in artifacts” and so we have lost nothing.

So – how does an instance of my language looks like? here’s an example:

Application MyApp
{
    //add references to other applications
    add reference SomeOtherApp;
    add reference AndAnother,andAThird;
    
    //build the solution
    build "SomeFolder\SomeOtherFolder\SomeSolution.sln";
    build "SomeFolder\SomeOtherFolder\SomeProject.btproj";
    
    //add required biztalk assemblies
    add biztalk assembly "SomeFolder\SomeOtherFolder\bin\deployment\Schemas.dll"
        version=1.0.0.0 
        culture=neutral 
        publicKeyToken=314bd7037656ea65;

    add biztalk assembly "SomeFolder\SomeOtherFolder\bin\deployment\Maps.dll"
        version=1.0.0.0 
        culture=neutral 
        publicKeyToken=314bd7037656ea65;

    add biztalk assembly "SomeFolder\SomeOtherFolder\bin\deployment\Orchestrations.dll" 
    with orchestrations
        {
            MyApp.SomeOrchestration,
            MyApp.AnottherOrchestration,
            MyApp.AndAnotherOrchestration
        }        
        version=1.0.0.0 
        culture=neutral 
        publicKeyToken=314bd7037656ea65;
    
    //and .net helpers assembly
    add assembly "SomeFolder\SomeOtherFolder\bin\release\HRG.FareWatch.Hotel.Helpers.dll" 
        version=1.0.0.0 
        culture=neutral 
        publicKeyToken=314bd7037656ea65;
    //import dev bindings
    import binding "SomeFolder\SomeOtherFolder\Bindings.xml";

    //add various environment's bindings.
    add binding "SomeFolder\SomeOtherFolder\Bindings.Dev.xml" "development";
    add binding "SomeFolder\SomeOtherFolder\Bindings.Stage.xml" "stage";
    add binding "SomeFolder\SomeOtherFolder\Bindings.Prod.xml" "production";
}

as I’ve said – I already have a runtime that generates the MSBuild scripts required to deploy these - the runtime outputs several files to the temp folder - 

• The “framework” MSBuild script that contains all the targets I’m using



• The Microsoft.Sdc.Tasks.BizTalk.dll and the Microsoft.Sdc.Tasks.dll which contains many useful custom tasks.



• The relevant import file for the SDC tasks



• The generated MSBuild file that contains all the various artifacts based on the source code

That latter file, which is the equivalent of my source code, and what we had to maintain so far, looks like -

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="DeployAll">
  <Import Project="Deploy.Framework" />
  <ItemGroup>
    <MyApp Include="SomeOtherApp" />
    <MyApp Include="AndAnother" />
    <MyApp Include="andAThird" />
  </ItemGroup>
  <ItemGroup>
    <Bindings Include="General">
      <path>SomeFolder\SomeOtherFolder\Bindings.xml</path>
    </Bindings>
  </ItemGroup>
  <ItemGroup>
    <EnvironmentBindings Include="General">
      <path>"SomeFolder\SomeOtherFolder\Bindings.Dev.xml"</path>
      <environment>development</environment>
    </EnvironmentBindings>
    <EnvironmentBindings Include="General">
      <path>"SomeFolder\SomeOtherFolder\Bindings.Stage.xml"</path>
      <environment>stage</environment>
    </EnvironmentBindings>
    <EnvironmentBindings Include="General">
      <path>"SomeFolder\SomeOtherFolder\Bindings.Prod.xml"</path>
      <environment>production</environment>
    </EnvironmentBindings>
  </ItemGroup>
  <ItemGroup>
    <ExternNetAssembly Include="HRG.FareWatch.Hotel.Helpers">
      <Version>1.0.0.0</Version>
      <Culture>neutral</Culture>
      <PublicKeyToken>314bd7037656ea65</PublicKeyToken>
      <ProcessorArchitecture>MSIL</ProcessorArchitecture>
      <path>SomeFolder\SomeOtherFolder\bin\release</path>
    </ExternNetAssembly>
  </ItemGroup>
  <ItemGroup>
    <Solution Include="SomeSolution">
      <path>SomeFolder\SomeOtherFolder</path>
    </Solution>
    <Solution Include="SomeProject">
      <path>SomeFolder\SomeOtherFolder</path>
    </Solution>
  </ItemGroup>
  <ItemGroup>
    <BTSProject Include="Schemas">
      <Version>1.0.0.0</Version>
      <Culture>neutral</Culture>
      <PublicKeyToken>314bd7037656ea65</PublicKeyToken>
      <ProcessorArchitecture>MSIL</ProcessorArchitecture>
      <path>SomeFolder\SomeOtherFolder\bin\deployment</path>
    </BTSProject>
    <BTSProject Include="Maps">
      <Version>1.0.0.0</Version>
      <Culture>neutral</Culture>
      <PublicKeyToken>314bd7037656ea65</PublicKeyToken>
      <ProcessorArchitecture>MSIL</ProcessorArchitecture>
      <path>SomeFolder\SomeOtherFolder\bin\deployment</path>
    </BTSProject>
    <BTSProject Include="Orchestrations">
      <Version>1.0.0.0</Version>
      <Culture>neutral</Culture>
      <PublicKeyToken>314bd7037656ea65</PublicKeyToken>
      <ProcessorArchitecture>MSIL</ProcessorArchitecture>
      <path>SomeFolder\SomeOtherFolder\bin\deployment</path>
    </BTSProject>
  </ItemGroup>
  <PropertyGroup>
    <BTApplicationName>MyApp</BTApplicationName>
    <!-- Set for a remote deployment -->
    <!-- Deploying BizTalk Server name - leave blank if local-->
    <!-- not currently supported by runtime-->
    <BTServerName>
    </BTServerName>
    <!-- Deploying BizTalk Server database - leave blank if BizTalkMsgBoxDb-->
    <BTServerDatabase>
    </BTServerDatabase>
    <!-- Deploying BizTalk Server SQL user name - leave blank if local-->
    <BTServerUserName>
    </BTServerUserName>
    <!-- Deploying BizTalk Server SQL password - leave blank if local-->
    <BTServerPassword>
    </BTServerPassword>
  </PropertyGroup>
</Project>

Would you agree that the former source code is easier to maintain?

 

note: one thing you would notice is that none of the paths contains a root; I assume that this would be used by different developers/IT pros which may have the code in a different path; however, as the assumption is that the code will come from source control, my framework script expects you to provide the root path to your source code and assumes the specified paths start there.

So – what’s next?

From a coding perspective - I plan to add support in my runtime to perform the actual deployment; this is I would like it to do once Oslo is in production so I’ll add it as an option through a command line switch.

This option would tell the runtime to deploy the artifacts to a specified BizTalk server using BTSTask or the Object model instead of generated the MSBuild script.

I also want to make some modification to my language definition to make the MGraph produced cleaner (and better geared towards using the repository and Quadrant at a later stage) as well as, obviously, add support for more features, after which I plan to publish both my language and the runtime somewhere (here, codeplex, DevCente..we’ll see)

I’ll post some more notes on all of these here in the near future hopefully…