Archive

Posts Tagged ‘Automation’

Key to successful BizTalk Deployment

January 25, 2015 Leave a comment

It’s been a while since my last blog, but IMHO this is one of important blog that I’ve ever made that can be useful pointer for every aspiring developer / companies that will/already implement application integration. Below contains a list / summary of best practices I’ve acquired in my more or less 10 years working with BizTalk, SSIS, WCF and .NET integration (primarily Microsoft Technology) but this I think is also applicable to other technology.

Design / Development:

#1 K.I.S.S (Keep it simple, stupid). Don’t get me wrong, I like complex integrations, from  multiple message correlations, requirement to aggregate certain line items with certain condition in the message to having a direct binding with delivery notification set to transmitted (multiple send port / send port groups, this will result in some zombie messages). Anyone from novice to expert BizTalk Developer can build a solution, it can be done & solve  in different ways but the key here is providing a simple and effective solution for a complex integration.

#2 Isolation with maximum reusability. Isolation for me means that the application can be deployed without having too much dependency to other BizTalk artifacts that are shared. During the design it’s always a struggle on how to seperate artifacts, on which solution / projects it should be placed to avoid having to uninstall all referencing application before you can refresh the dependency (GACing works in some situation but not for all, you’ll be surprise how BizTalk deal with it internally). Once identifying or isolating a certain integration flow whenever there’s a new requirement, we should always check whether it fits on already existing solution (extend) without having to build a new integration flow. in this case we will have maximum reuse of existing solution.

#3 Consistency. Every solution that will be build should look as if it’s build / coded by one person. Having a naming convention on BizTalk artifacts, namespace, solutions, project & folder structures should be define first. Based on experience providing a proper name is always a challenge since if you define it incorrectly, it will be a costly mistake later on.

#4 Nuget – all common code / components / libraries should be referenced from a local nuget repository. Having a local nuget repository not only simplifies referencing dependencies but also saves you once you start using TFS Build Server (autorestore feature). This means that shared assemblies is not required to be checked in together with the source codes.

#5 .NET over XLST in complex mapping   Whenever there’s a requirement for complex mapping like grouping, aggregation, unique numbering over a certain combinations or constructing a hashtable in the mapping, I always go for .NET component. Using .NET component it is easier to maintain and tested (unit testing + mockup) vs doing everything in XSLT (in which can be forgotten in time).

Build / Deployment:

#1 Release Pipeline – It’s important to define strategy on how you would build and be able to do ‘Build once’ and deploy to multiple environments. In our case, part of our strategy is to minimize branching and always build from MAIN. In TFS, we’ve implemented a Gated Checkin which means that every checkin will trigger a release pipeline. During the checkin, a build will be triggerred (MSBuild and BTDF build), it will run all unit test and copies the binaries to drop folder (only If all steps are successful). From this point the Ops can just enter a command (application name) and environment and then the automated deployment will take over..

#2 Configuration over Code  for normal deployment most of time you would see that a solution contains a powershell script for deployment, this is not bad but i think this only works on sample codes or tutorials but a problem in Enterprise deployment. For this you need a good deployment framework (compilation of scripts) to do the deployment and during the deployment preparation just replace the values in the templates like servernames, shared folders and etc.  I’ve posted some blogs earlier how to do this:

Centralize Powershell Repository:

https://randypaulo.wordpress.com/2012/02/27/powershell-centralize-powershell-script-modules-repository/

[Powershell] Set user permission in SSRS Item (SQL Reporting Server) using powershell

February 22, 2012 3 comments

Combining both (article1 and article2), i was able to come up with a powershell script to set user permission in SQL Reporting Services (SSRS)

<#
.SYNOPSIS
	Set user permissions in SQL Reporting Services using Web Service

.DESCRIPTION
	Set user permissions in SQL Reporting Services using Web Service

.EXAMPLE
	Add-SSRSItemSecurity -webServiceUrl "http://[ServerName]/ReportServer/ReportService2005.asmx?WSDL" -itemPath "MyReportFolder" -groupUserName RPAULO\User1 -role Browser

.EXAMPLE
	Add-SSRSItemSecurity -url "http://[ServerName]/ReportServer/ReportService2005.asmx?WSDL" -itemPath "MyReportFolder" -u RPAULO\User1 -r "Content Manager"

#>
function Add-SSRSItemSecurity
(
	[Parameter(Position=0,Mandatory=$true)]
	[Alias("url")]
	[string]$webServiceUrl,

	[Parameter(Position=1,Mandatory=$true)]
	[Alias("path")]
	[string]$itemPath,
	
	[Parameter(Position=2,Mandatory=$true)]
	[Alias("u")]
	[string]$groupUserName,
	
	[Parameter(Position=3,Mandatory=$true)]
	[Alias("r")]
	[string]$role,
	
	[Parameter(Position=2)]
	[bool]$inherit=$true
)

{
	
	#Fix item path if not starting with /
	if(!$itemPath.StartsWith("/")) { $itemPath = "/" + $itemPath}
	
	#Create Proxy
	Write-Host "[Add-SSRSItemSecurity()] Creating Proxy, connecting to : $webServiceUrl"
	$ssrsProxy = New-WebServiceProxy -Uri $webServiceUrl -UseDefaultCredential
	
	$type = $ssrsProxy.GetType().Namespace;
	$policyType = "{0}.Policy" -f $type;
	$roleType = "{0}.Role" -f $type;
	
	Write-Host "[Add-SSRSItemSecurity()] Retrieving all existing policies."
	$policies = $ssrsProxy.GetPolicies($itemPath, [ref]$inherit);
	
	$a = 1;
	foreach($policy in $policies)
	{

		foreach($r in $policy.Roles)
		{
			$msg = "[Add-SSRSItemSecurity()]  Existing Policy # {0} Group Name: {1}, Role: {2}" -f $a, $policy.GroupUserName, $r.Name
			Write-Host $msg
		}
		$a+=1;
	}

	$msg = "[Add-SSRSItemSecurity()] Total Existing Policies: " + $policies.Length;
	Write-Host $msg
	
	$Policy = $policies | 
    Where-Object { $_.GroupUserName -eq $groupUserName } | 
    Select-Object -First 1
	
	if (-not $Policy) {
	    $Policy = New-Object ($policyType)
	    $Policy.GroupUserName = $GroupUserName
	    $Policy.Roles = @()
	    $Policies += $Policy
		$msg = "[Add-SSRSItemSecurity()] Adding new policy: '{0}'" -f $GroupUserName
		Write-Host $msg
	}

	$r = $Policy.Roles |
	    Where-Object { $_.Name -eq $role } |
	    Select-Object -First 1
	if (-not $r) {
	    $r = New-Object ($roleType)
	    $r.Name = $role
	    $Policy.Roles += $r
		$msg = "[Add-SSRSItemSecurity()] Adding new role: '{0}'" -f $role
		Write-Host $msg
	}
	
	#Set the policies
	$ssrsProxy.SetPolicies($itemPath,$policies);

}


[Powershell] How to Install/deploy SSRS (rdl files) using Powershell

February 21, 2012 19 comments

Since Powershell v2.0 introduce the new cmdlet New-WebServiceProxy, it’s no longer needed to generate a class library to deploy/undeploy SSRS file (RDL), below are two powershell scripts Install-SSRSRDL and Uninstall-SSRSRDL that can be useful when doing an automated deployment or silent installation of rdl files to Reporting Services.

Install SSRS RDL Script:


<#
.SYNOPSIS
	Installs an RDL file to SQL Reporting Server using Web Service

.DESCRIPTION
	Installs an RDL file to SQL Reporting Server using Web Service

.NOTES
	File Name: Install-SSRSRDL.ps1
	Author: Randy Aldrich Paulo
	Prerequisite: SSRS 2008, Powershell 2.0

.PARAMETER reportName
	Name of report wherein the rdl file will be save as in Report Server.
	If this is not specified it will get the name from the file (rdl) exluding the file extension.

.PARAMETER force
	If force is specified it will create the report folder if not existing
	and overwrites the report if existing.

.EXAMPLE
	Install-SSRSRDL -webServiceUrl "http://[ServerName]/ReportServer/ReportService2005.asmx?WSDL" -rdlFile "C:\Report.rdl" -force

.EXAMPLE
	Install-SSRSRDL "http://[ServerName]/ReportServer/ReportService2005.asmx?WSDL" "C:\Report.rdl" -force

.EXAMPLE
	Install-SSRSRDL "http://[ServerName]/ReportServer/ReportService2005.asmx?WSDL" "C:\Report.rdl" -force -reportName "MyReport"

.EXAMPLE
	Install-SSRSRDL "http://[ServerName]/ReportServer/ReportService2005.asmx?WSDL" "C:\Report.rdl" -force -reportFolder "Reports" -reportName "MyReport"

#>
function Install-SSRSRDL
(
	[Parameter(Position=0,Mandatory=$true)]
	[Alias("url")]
	[string]$webServiceUrl,

	[ValidateScript({Test-Path $_})]
	[Parameter(Position=1,Mandatory=$true)]
	[Alias("rdl")]
	[string]$rdlFile,

	[Parameter(Position=2)]
	[Alias("folder")]
	[string]$reportFolder="",

	[Parameter(Position=3)]
	[Alias("name")]
	[string]$reportName="",

	[switch]$force
)
{
	$ErrorActionPreference="Stop"

	#Create Proxy
	Write-Host "[Install-SSRSRDL()] Creating Proxy, connecting to : $webServiceUrl"
	$ssrsProxy = New-WebServiceProxy -Uri $webServiceUrl -UseDefaultCredential
	$reportPath = "/"

	if($force)
	{
		#Check if folder is existing, create if not found
		try
		{
			$ssrsProxy.CreateFolder($reportFolder, $reportPath, $null)
			Write-Host "[Install-SSRSRDL()] Created new folder: $reportFolder"
		}
		catch [System.Web.Services.Protocols.SoapException]
		{
			if ($_.Exception.Detail.InnerText -match "[^rsItemAlreadyExists400]")
			{
				Write-Host "[Install-SSRSRDL()] Folder: $reportFolder already exists."
			}
			else
			{
				$msg = "[Install-SSRSRDL()] Error creating folder: $reportFolder. Msg: '{0}'" -f $_.Exception.Detail.InnerText
				Write-Error $msg
			}
		}

	}

	#Set reportname if blank, default will be the filename without extension
	if($reportName -eq "") { $reportName = [System.IO.Path]::GetFileNameWithoutExtension($rdlFile);}
	Write-Host "[Install-SSRSRDL()] Report name set to: $reportName"

	try
	{
		#Get Report content in bytes
		Write-Host "[Install-SSRSRDL()] Getting file content (byte) of : $rdlFile"
		$byteArray = gc $rdlFile -encoding byte
		$msg = "[Install-SSRSRDL()] Total length: {0}" -f $byteArray.Length
		Write-Host $msg

		$reportFolder = $reportPath + $reportFolder
		Write-Host "[Install-SSRSRDL()] Uploading to: $reportFolder"

		#Call Proxy to upload report
		$warnings = $ssrsProxy.CreateReport($reportName,$reportFolder,$force,$byteArray,$null)
		if($warnings.Length -eq $null) { Write-Host "[Install-SSRSRDL()] Upload Success." }
		else { $warnings | % { Write-Warning "[Install-SSRSRDL()] Warning: $_" }}
	}
	catch [System.IO.IOException]
	{
		$msg = "[Install-SSRSRDL()] Error while reading rdl file : '{0}', Message: '{1}'" -f $rdlFile, $_.Exception.Message
		Write-Error msg
	}
	catch [System.Web.Services.Protocols.SoapException]
	{
		$msg = "[Install-SSRSRDL()] Error while uploading rdl file : '{0}', Message: '{1}'" -f $rdlFile, $_.Exception.Detail.InnerText
		Write-Error $msg
	}

}

Uninstall SSRS RDL Script:


<#
.SYNOPSIS
	Uninstalls an RDL file from SQL Reporting Server using Web Service

.DESCRIPTION
	Uninstalls an RDL file from SQL Reporting Server using Web Service

.NOTES
	File Name: Uninstall-SSRSRDL.ps1
	Author: Randy Aldrich Paulo
	Prerequisite: SSRS 2008, Powershell 2.0

.EXAMPLE
	Uninstall-SSRSRDL -webServiceUrl "http://[ServerName]/ReportServer/ReportService2005.asmx?WSDL" -path "MyReport"

.EXAMPLE
	Uninstall-SSRSRDL -webServiceUrl "http://[ServerName]/ReportServer/ReportService2005.asmx?WSDL" -path "Reports/Report1"

#>
function Uninstall-SSRSRDL
(
	[Parameter(Position=0,Mandatory=$true)]
	[Alias("url")]
	[string]$webServiceUrl,

	[Parameter(Position=1,Mandatory=$true)]
	[Alias("path")]
	[string]$reportPath
)

{
	#Create Proxy
	Write-Host "[Uninstall-SSRSRDL()] Creating Proxy, connecting to : $webServiceUrl"
	$ssrsProxy = New-WebServiceProxy -Uri $webServiceUrl -UseDefaultCredential

	#Set Report Folder
	if(!$reportPath.StartsWith("/")) { $reportPath = "/" + $reportPath }

	try
	{

		Write-Host "[Uninstall-SSRSRDL()] Deleting: $reportPath"
		#Call Proxy to upload report
		$ssrsProxy.DeleteItem($reportPath)
		Write-Host "[Uninstall-SSRSRDL()] Delete Success."
	}
	catch [System.Web.Services.Protocols.SoapException]
	{
		$msg = "[Uninstall-SSRSRDL()] Error while deleting report : '{0}', Message: '{1}'" -f $reportPath, $_.Exception.Detail.InnerText
		Write-Error $msg
	}

}

Install-SSRSRDL Powershell Script
Uninstall-SSRSRDL Powershell Script

For permission check this blog: https://randypaulo.wordpress.com/2012/02/22/powershell-set-user-permission-in-ssrs-item-sql-reporting-server-using-powershell/

Automating\Silent Installation of BizTalk Deployment Framework (BTDF) using Powershell

January 31, 2012 37 comments

To do a silent install/automate the installation of BizTalk Application using BizTalk Deployment Framework you can use the Powershell script below:


<#
.SYNOPSIS
 Automates BizTalk Application deployment using BTDF 5.0

.DESCRIPTION
 Automates BizTalk Application deployment using BTDF 5.0
  Steps:
   1. It installs the MSI on the specified application path
   2. Calls EnvironmentSettingsExporter to generate the settings xml
   3. Updates Environment Variables
   4. Executes the MSBuild with parameters

.NOTES
 File Name: Install-BizTalkApplication.ps1
 Author: Randy Aldrich Paulo
 Prerequisite: Powershell 2.0, BizTalk Deployment Framework 5.0, BizTalk Server 2010

.PARAMETER MsiFile
 MSI File generated using BizTalk Deployment Framework 5.0

.PARAMETER ApplicationInstallPath
 Location wherein the resource files will be copied, it will be use by the BTDF during the deployment

.PARAMETER Environment
 Name of environment (Local,Dev,Test,Prod) to be used, this value will be passed to
 EnvironmentSettingsExporter and willbe used to construct the environment variable: ENV_SETTINGS

.EXAMPLE
 Install-BizTalkApplication -MsiFile "E:\Installer\Application 1\Application1.msi"
 -ApplicationInstallPath "E:\Program Files\Application 1"
 -Environment DEV

.EXAMPLE
 Install-BizTalkApplication -msi "E:\Installer\Application 1\Application1.msi"
 -path "E:\Program Files\Application 1"
 -env TEST

.EXAMPLE
 Install-BizTalkApplication "E:\Installer\Application 1\Application1.msi"
 "E:\Program Files\Application 1" TEST

.EXAMPLE
 Install-BizTalkApplication "E:\Installer\Application 1\Application1.msi"
 "E:\Program Files\Application 1" TEST -SkipUndeploy $false

#>
function Install-BizTalkApplication
{
 param(
  [Parameter(Position=0,Mandatory=$true,HelpMessage="Msi file should be existing")]
  [ValidateScript({Test-Path $_})]
  [Alias("msi")]
  [string]$MsiFile,
  
  [Parameter(Position=1,HelpMessage="Path wherein the resource file will be installed")]
  [Alias("path")]
  [string]$ApplicationInstallPath,
  
  [Parameter(Position=2,Mandatory=$true,HelpMessage="Only valid parameters are Local,Dev,Test and Prod")]
  [Alias("env")]
  [ValidateSet("Local","Dev","Prod","Test")]
  [string]$Environment,

  [bool]$BTDeployMgmtDB=$true,
  [bool]$SkipUndeploy=$true
  )

 $ErrorActionPreference="Stop"

 #Step 1 : Run MSI 
  $script =
  {
      $args = "-i $MsiFile INSTALLDIR=`"$ApplicationInstallPath`" /qn /norestart"
   Write-Host " Installing MSI File.." -ForegroundColor Cyan
   Write-Host "  MSI File: $MsiFile" -ForegroundColor DarkGray
   Write-Host "      Args: $args" -ForegroundColor DarkGray
   
   $exitCode = (Start-Process -FilePath "msiexec.exe" -ArgumentList $args -Wait -Passthru).ExitCode
   Write-Host " Exit Code: $exitCode"
   
   if($exitCode -ne 0)
   {
    Write-Error "Installing $MsiFile failed!, Exit Code: $exitCode"
   }
   Write-Host " Installed MSI success.." -ForegroundColor Green
   Write-Host ""
  }
  Invoke-Command -scriptblock $script
 
 #Step 2 : Run MSBuild & Deploy
 
  $script=
  {
   <# Start Step 2.2 Run EnvironmentSettingsExporter, this one generates the xml file
   (Exported_DevSettings.xml, Exported_LocalSettings.xml etc..)
   #>
   $args = "`"" + (Join-Path $ApplicationInstallPath "Deployment\EnvironmentSettings\SettingsFileGenerator.xml") + "`"" + " Deployment\EnvironmentSettings"
   $exePath = ("`"" + (Join-Path $ApplicationInstallPath "\Deployment\Framework\DeployTools\EnvironmentSettingsExporter.exe") + "`"")
   Write-Host " Generating Environment Settings File.."  -ForegroundColor Cyan
   Write-Host " Location: $exePath" -ForegroundColor DarkGray
   Write-Host "  Args: $args" -ForegroundColor DarkGray

   $exitCode = (Start-Process -FilePath $exePath -ArgumentList $args -Wait -PassThru).ExitCode
   Write-Host " Exit Code: $exitCode"
   
   if($exitCode -ne 0)
   {
    Write-Error " Generating Environment Settings File failed!, Exit Code: $exitCode"
   }
   Write-Host " Generated Environment Settings File. " -ForegroundColor Green
   Write-Host ""
   <# End Step 2.2 Run EnvironmentSettingsExporter, this one generates the xml file
   (Exported_DevSettings.xml, Exported_LocalSettings.xml etc..)#>
   <# Start Step 2.3 Set the Environment Variables ENV_SETTINGS and BT_DEPLOY_MGMT_DB #>
   $settingsFile = "Deployment\EnvironmentSettings\Exported_{0}Settings.xml" -f $Environment
   $EnvSettings =Join-Path $ApplicationInstallPath $settingsFile

   Write-Host " Setting Environment Variables"  -ForegroundColor Cyan
   
   Write-Host "      ENV_SETTINGS = $EnvSettings" -ForegroundColor DarkGray;
   Set-Item Env:\ENV_SETTINGS -Value $EnvSettings
   
   Write-Host  " BT_DEPLOY_MGMT_DB = $BTDeployMgmtDB"  -ForegroundColor DarkGray;
   Set-Item Env:\BT_DEPLOY_MGMT_DB -Value $BTDeployMgmtDB
   
   Write-Host " Setted Environment Variables"  -ForegroundColor Green
   Write-Host ""
   <# End Step 2.3 Set the Environment Variables ENV_SETTINGS and BT_DEPLOY_MGMT_DB #>
   
   <# Start Step 2.4 Execute MS Build with parameters #>
   
   #Get .NET Version
   $dotNetVersion = gci 'HKLM:\SOFTWARE\Microsoft\NET Framework Setup\NDP' | sort pschildname -des | select -fi 1 -exp pschildname
   if($dotNetVersion = "v4.0") { $dotNetVersion = "v4.0.30319" } #Include other info if .NET 4.0
   
   if (Test-Path ( Join-Path $env:windir "Microsoft.NET\Framework\$dotNetVersion\MSBuild.exe" ))
   {
    $BTDFMSBuildPath = Join-Path $env:windir "Microsoft.NET\Framework\$dotNetVersion\MSBuild.exe"
    Write-Host " Using MSBuild $dotNetVersion" -ForegroundColor DarkGray
   }
   else
   {
    Write-Error " MSBuild not found."
   }
   
   #Assign MS Build Params
   $parms="DeployBizTalkMgmtDB=$BTDeployMgmtDB;Configuration=Server;SkipUndeploy=$SkipUndeploy"
   $logger="FileLogger,Microsoft.Build.Engine;logfile=`"" + ( Join-Path $ApplicationInstallPath "DeployResults\DeployResults.txt" ) + "`""
   $btdfFile="`"" +  (Join-Path $ApplicationInstallPath "Deployment\Deployment.btdfproj") + "`""
   $args = "/p:{1} /l:{2} {0}" -f $btdfFile,$parms,$logger
   
   Write-Host " Executing MSBuild from: $BTDFMSBuildPath"  -ForegroundColor Cyan
   Write-Host " ArgList: $args" -ForegroundColor DarkGray
   
   #Check MSBuild Return Code
   $exitCode = (Start-Process -FilePath $BTDFMSBuildPath -ArgumentList $args -Wait -Passthru).ExitCode
   Write-Host " Exit Code: $exitCode"
   Write-Host ""
   if($exitCode -ne 0)
   {
    Write-Error " Error while calling MSBuild, Exit Code: $exitCode"
   }
  
   #Copy Log File
   Write-Host " Copying  Log file."
   $args =  "Deployment\Framework\CopyDeployResults.msbuild /nologo"
   Start-Process -FilePath $BTDFMSBuildPath -ArgumentList $args
   
   <# End Step 2.4 Execute MS Build with parameters #>
  }
 
  Write-Host " Running MS Build and deploying.." -ForegroundColor Cyan
  Invoke-Command -scriptblock $script
  Write-Host " Deployed application" -ForegroundColor Green

}

Download Script (MSWord): Install-BizTalkApplication – Powershell Script 

 

%d bloggers like this: