Compression Experiments In the Build and Deployment Process

In overhauling our build and deployment process for this release I wanted to use compression for a few tasks:

• Combining staged app files to deploy into a compressed archive
  • Preferably a self-extracting executable archive (SFX)
• Extracting the deployment archive when running the deployment
• Creating backup zip files of the existing install before overwriting it

There are many ways to handle compression and decompression in my context which was:

• App - Windows .NET ClickOnce
• Build Script - MSBuild. Hopefully replaced with psake soon
• Deployment Script - PowerShell

For our production deployments I really wanted to hand over a compressed exe that the person running it could just run without having to open/copy/extract a zip file, find a place to extract to, and navigate to the item to launch and run it. Likewise for continuous deployments the EXE could be used as well.

I did not have any loyalty to a specific tool or technique, I just wanted something that worked reliably without much hassle. Achieving this wasn't overly difficult but it did end up being a bit more hassle than I was expecting.

Experiment #1 - shell.application

From this StackExchange post I started with the below technique for backing up our existing installation when deploying. My only draw to trying it was not having an external dependency for the compression.

$shell_app=new-object -com shell.application 
$filename = "test.zip" 
$zip_file = $shell_app.namespace((Get-Location).Path + "\$filename") 
$destination = $shell_app.namespace((Get-Location).Path) 
$destination.Copyhere($zip_file.items())

That approach only lasted a few minutes as it didn't work reliably across network shares, popped up errors, and resulted in corrupted zip files.

Experiment #2 - Compression with the MSBuild Extension Pack

Meanwhile I shifted my focus to compressing the deployment files that the build process produced. Since I was already using the MSBuild Extension Pack, giving this a whirl made sense.

<PropertyGroup>
	<BuildArtifacts>..\build\artifacts\</BuildArtifacts>
    <DeployPackageFolder>..\build\artifacts\deployPackage\</DeployPackageFolder>
</PropertyGroup>

<ItemGroup>
	<DeployPackageFolderItem Include="$(DeployPackageFolder)"/>
	<FilesToZip Include="$(DeployPackageFolderItem)**\*" Exclude="..\**\*.build; ..\**\*.licx"/>
</ItemGroup>

<Target Name="PackageDeployment" 
DependsOnTargets="BuildClickOncePublishFiles;CreateClickOnceWebPage">

<MSBuild.ExtensionPack.Compression.Zip TaskAction="Create" 
	CompressPath="@(DeployPackageFolderItem)"
    ZipFileName="$(BuildArtifacts)MyApp.deploy.zip" 
	RemoveRoot="@(DeployPackageFolderItem)"/>
</Target>

The Compression task in the Extension Pack worked great. The major limitation that led me away from it was that it didn't support creating self-extracting archives. It may have been feasible to create the SFX from the zip file it produced with another process later but that did not sound ideal even if it would have worked.

Experiment #3 - 7-zip

I have been a user and fan of 7-zip for a while but I have not had the need to automate it before. Since the app works well and supports creating self-extracting archives, I turned my attention to it.

Setup

I began by downloading both the 7-zip command line version and Windows GUI (MSI) from their download page, then installing the MSI. The GUI app and MSI I only installed in order to work with .7z files myself; the automated process would not need it. Next an extra is required to actually create the SFX; I downloaded 7z922_extra.7z from their SourceForge page and opened it using %ProgramFiles%\7-Zip\7zFM.exe. I extracted that to our application's \build\ folder and copied the console app 7za.exe there as well.

Creating the archive

In working with the 7zip command line program and MSBuild, I found it a bit easier initially to create the zip and zipped exe using a batch file instead of PowerShell. I intended to later convert this to PowerShell to be invoked from MSBuild but as time would have it I never got around to it. I know, I know. First I replaced the usage of the MSBuild Extension Pack compression task with a call to this batch file:

<Exec Command="$(BuildFolder)Zip-Install-Create.bat "%(BuildArtifactsItem.FullPath)"" />

Then a Zip-Install-Create.bat script:

@echo off

if "%1" == "" goto usage

REM Expecting trailing \ on folders passed in
set artifactsFolder=%1
set folderToBackup=%artifactsFolder%deployPackage\
set zipLocation=%artifactsFolder%
set buildFolder=%~dp0

set zipFilename=%zipLocation%DeployMyApp.7z
set zipExe=%zipLocation%DeployMyApp.exe

echo Artifacts folder is %artifactsFolder%

REM Create the zip file
7za.exe a %zipFilename% %folderToBackup% 
REM combine SFX (Self Extracting Archive), install config text file, & zip file into EXE
copy /b 7zS.sfx + Zip-Install-Config.txt + %zipFilename% %zipExe%

REM delete zip file now that we have self extracting archive exe
del /q /f %zipFilename%

REM if a second parameter treat as a directory to copy exe to
IF NOT "%2" == "" xcopy /y /r %zipExe% %2

goto exit

:usage
echo Expected an artifacts folder with trailing backslash as the 1st parameter. 
echo 2nd parameter is an optional folder to copy zip exe to.

:exit

This batch file expects to be passed the folder (w/trailing \) where the artifacts the build produces are stored. It will store the zip file there and backup the deployPackage subdirectory contained there. The 7zip console is invoked on line 17 and passed the zip filename and deploy package folder to be compressed. Next on line 19 the script merges the .sfx extra that enables self-extraction with a configuration file that tells it what to run after extraction and the generated .7z zip file itself.

The configuration file (Zip-Install-Config.txt) containing the post-extraction execution info looks like the below. Important Note: I believe this file needs to have UNIX style line endings - only LF not CRLF. In an editor such as Notepad++ you can click the Paragraph icon in the toolbar to show all characters and use the Edit->EOL Conversion menu to help ensure the line endings are correct.

;!@Install@!UTF-8!
Title="Install MyApp"
BeginPrompt=""
Progress="yes"
ExecuteFile="deployPackage\DeploySfxBootstrap.bat"
ExecuteParameters=""
;!@InstallEnd@!

Gasp! Another batch file you say, even though the deployment script is in PowerShell. This one was a bit more required as .ps1 files are not directly executable in the same fashion as batch files. Invoking PowerShell and giving it a file or command did not work either for various reasons, some of which I will get to shortly.

Executing the Deployment Script from the SFX

Once the files are extracted I wanted the deployment PowerShell script run. As indicated I could not do that directly so I created DeploySfxBootstrap.bat which can be run from the SFX and it would invoke the PowerShell script.

REM This is intended to be run only via the self-extracting zip exe created by build.
REM It copies extracted files to a known location in the event of an error, re-deploy, logging output etc.
REM It then hands off the real work to PowerShell for the deployment
@echo off

mode con cols=120

echo You are here: %~dp0
echo Copying extracted files to a known location

set targetDir=%userprofile%\Desktop\MyApp.Install
if exist %targetDir% rmdir /s /q %targetDir%
xcopy /s /i /r *.* %targetDir%

set deployScript="%targetDir%\deployPackage\Deploy.ps1"
echo Starting powershell to run the script %deployScript%
start powershell -file "%deployScript%" -WindowStyle Maximized

REM Below works but it results in powershell running in cmd.exe window, not powershell window host
REM powershell -file "%deployScript%"
REM pause

The bootstrap script does a couple of other tasks, mainly to work around some issues I had with 7-zip self-extracting archives:

1. There was no apparent way to tell 7-zip where you wanted the extracted files to go (temp dir is created).
2. There was no easy way to tell where 7-zip extracted the SFX contents to. (i.e. from config context)
3. Once the post-extract launched process terminates, 7-zip deletes the temp directory where it extracted the files to. That might seem like a good cleanup practice but it makes troubleshooting very difficult and my deployment process creates some output files (like a PowerShell transcript) it writes to the folder it removes. Particularly if the install fails I did not want 7-zip removing the extracted folder.

It is a bit redundant but the script first copies the already extracted files to a known location on the desktop and then the deployment script is invoked from there. 7-zip will take care of removing those same files in the temp directory.

Experiment #4 - DotNetZip

At this point the self-extracting archive was working well and I was back to wanting to backup the existing deployment when starting the deployment process. I thought about using 7zip for this but then I'd have to bundle that into the archive and I remembered using DotNetZip in our app to create an error report package for unexpected exceptions. I felt a little better about bundling its dll and it would be friendly to work with from PowerShell.

First a driver function to load the zip dll and backup the desired directories.

function Backup-ExistingInstall
{
    $sw = [Diagnostics.Stopwatch]::StartNew()
    Set-Activity "Backup Existing Install"
    
    Write-Log "Loading zip library"
    $zipLibrary = (join-path ($_scriptPath) "Ionic.Zip.dll")
    [System.Reflection.Assembly]::LoadFrom($zipLibrary)
    "`n"

    $excludeDirNames = @('dotnetfx', 'NET35SP1', 'Documents', 'Exceptions', '_Backup')
    Backup-Dir -dir $_targetClickOnceDir -archiveName "MyApp.ClickOnce.backup.zip" -excludeDirNames $excludeDirNames
    Backup-Dir -dir $_targetSatelliteDir -archiveName "MyApp.Satellite.backup.zip" -excludeDirNames $excludeDirNames
    Backup-Dir -dir $_targetServicesDir  -archiveName "MyApp.Services.backup.zip" -excludeDirNames $excludeDirNames
    $sw.Stop()
    Write-Log ("Complete backup process finished in {0:##.0} minutes" -f ($sw.Elapsed.TotalMinutes))
}

Next a function to backup a given directory; in my case the desired behavior was creating a _Backup subfolder in each top level directory being backed up and saving the zip file there.

function Backup-Dir([string]$dir, [string]$archiveName, [string[]]$excludeDirNames)
{
    if (!(Test-Path $dir))
    {
        Write-Log "Directory '$dir' does not exist; nothing to backup"
        return;
    }
    
    $sw = [Diagnostics.Stopwatch]::StartNew()
    Write-Log ("Backing up directory '{0}' to '{1}'" -f $dir, $archiveName)
    
    $backupDir = join-path $dir "_Backup"

    if (!(Test-Path $backupDir))
    {
        Write-Log "Creating $backupDir"
        New-Item $backupDir -type directory | out-null
    }
        
    $backupZipFile = join-path $backupDir $archiveName
    if (Test-Path $backupZipFile)
    {
        Remove-Item $backupZipFile -force
    }    

    Write-Log ("Zip: Initialize '{0}'" -f $backupZipFile)
    $_zipFileObject = new-object Ionic.Zip.ZipFile
    Compress-Files -dir $dir -excludeDirNames $excludeDirNames
    
    Write-Log ("Zip: Saving '{0}'. This could take a while..." -f $backupZipFile)
    $_zipFileObject.Save($backupZipFile)
    
    Write-Log "Zip: Cleanup"
    $_zipFileObject.Dispose() # how else to clear all files and reset?
        
    $sw.Stop()
    Write-Log ("Backup finished in {0:###.0} seconds" -f ($sw.Elapsed.TotalSeconds))
}

Finally a recursive function to add the files in the given directory and its subdirectories.

function Compress-Files ([string]$dir, [string[]]$excludeDirNames)
{    
    Write-Log ("Zip: processing directory '{0}'" -f $dir)
    
    if ($null -eq $excludeDirNames) {$excludeDirNames = @()}
    $children = get-childitem -path $dir | where-object { $excludeDirNames -notcontains $_.Name }
    
    foreach ($o in $children)
    {
        if ($o.PSIsContainer)
        {
            Compress-Files ( $o.FullName )
        }
        else 
        {
            if (!($o.FullName.EndsWith(".zip")))
            {
                Write-Log ("Zip: Add {0}" -f $o.FullName)
                $pathInArchive = $o.Directory.FullName.Replace("\\$_targetServerRootDir", "")
                $e= $_zipFileObject.AddFile($o.FullName, $pathInArchive)
            }
        }
    }
}

That worked so well it got me questioning my use of 7-zip...

Experiment #5 - DotNetZip SFX

Once I realized that DotNetZip could create self-extracting archives, I decided it would be better to replace 7-zip with DotNetZip so I was only dealing with a single, friendly compression tool.

Unfortunately, when compressing the contents of my build with it, DotNetZip would get stuck and never finish. I created a small throw-away .net app and wired into various events and did all sorts of troubleshooting. Through a painstaking process of elimination, I narrowed the problem down to a specific combination of 3 files (assemblies and PDBs). I logged an issue about this but never heard anything back. Meanwhile I found other similar issues posted so I gave up on it. Strangely it worked fine backing up a lot of the same files over the network.

In Conclusion

I ended up settling on 7-zip for building the SFX from the build and DotNetZip for backing up the existing install when deploying. Both the implementations could use improvements and I am not fond of using 2 tools for compression tasks. For now this is working well and it can always be refactored another day when time allows.

As always there are other alternatives, be it the PowerShell Community Extensions (PSCX), WinZip command line, pkzip command line, using the .net GZipStream class, Xceed ($$$) etc. I wish there was something better built into Windows / PowerShell itself and I am sure there are a number of solutions I'm not familiar with. I'd love to hear your thoughts on what works well for you.

Windows Phone Login Navigation

A common desire for a mobile application is showing a welcome / login type screen initially when an account is required and the user first runs the app. My past two Windows Phone apps have not needed this but my current one does. I thought it would be very simple and straightforward; just check some user settings in App.xaml.cs and conditionally navigate to a different page depending on whether validated credentials are present.

Some Problems

It turns out it was not quite that straightforward. First of all the process of navigating to MainPage.xaml (or whatever startup page you use) is already underway in App.xaml.cs. Navigating to another page required either cancelling that navigation or using UriMapper to setup the initial page. This post by Peter Torr provided a great starting point. I chose the UriMapper approach and it worked in terms of conditionally changing the initial page.

However, after initially navigating to the welcome / logon page when credentials were missing, I tried redirecting back to MainPage.xaml after credentials were validated but nothing happened. The welcome page remained on screen and it was not obvious why the navigation to MainPage.xaml was not working.

Solutions

In turning to a programmer's best friend (Stack Overflow), I found this WP7 sign-in redirect post. That post suggested that the only real option was making the main page conditionally show a login control or pop-up. I was not fond of that approach; for one it did not seem like a good separation of responsibilities. For another it meant dynamically hiding/showing or covering/uncovering content and it felt like the experience would be worse for both the end user and myself.

Late last night I decided to experiment a bit more with the separate welcome screen approach and did manage to get it working, though there may be a better approach.

The main class looks like this:

public class StartupNavigator : IStartupNavigator
{
	private readonly UserSettings _userSettings;
	private PhoneApplicationFrame _rootFrame;

	private const string MainPage = "/MainPage.xaml";

	public StartupNavigator(UserSettings userSettings)
	{
		_userSettings = userSettings;
	}

	public void Initialize()
	{
		_rootFrame = ((App) Application.Current).RootFrame;
		SetupUriMapper();
	}

	private void SetupUriMapper()
	{
		var map = new UriMapper();
		map.UriMappings.Add(new UriMapping {Uri = new Uri(MainPage, UriKind.Relative)});
		_rootFrame.UriMapper = map;

		var navPath = _userSettings.HasValidatedCredentials ? MainPage 
			: "/Pages/WelcomePage.xaml";
		map.UriMappings[0].MappedUri = new Uri(navPath, UriKind.Relative);
	}

	public void GoHome()
	{
		_rootFrame.Dispatcher.BeginInvoke(() =>
		{
			// recalculate URI mapping - should have a different answer now 
			// on validated credentials. this is needed
			SetupUriMapper();
			// we need to vary the query string, otherwise OS doesn't think we 
			// are navigating to different page and won't do anything. Despite 
			// changing the initial Uri mapping from Main Page to login/welcome, 
			// _rootFrame.CurrentSource remained as /MainPage.xaml the whole time.
			_rootFrame.Navigate(new Uri(MainPage + "?fromLogin=true", UriKind.Relative));
		});
	}
}

Keys to the Solution

The SetupUriMapper method takes care of setting the correct initial page to be navigated to according to whether validated credentials exist in settings. The less obvious part is the GoHome method which will later get invoked from the welcome / login page's ViewModel once the user's credentials have been validated. As the comments indicate, the root frame's UriMapper needed to be set again now that MainPage.xaml is now the starting point instead of WelcomePage.xaml.

Next, when navigating to MainPage.xaml, I had to add a query string parameter just so the Uri changed a bit. Despite changing the initial Uri mapping before, RootFrame.CurrrentSource remained as /MainPage.xaml during the process (set in WMAppManifest.xml). Windows Phone appeared to be ignoring the request to navigate to MainPage because it detected the Uri was not changing.

Perhaps my understanding of this is flawed and there may be a better approach; in either event, I would love to hear about it. This was the main gotcha I ran into. For subsequent page navigation I went with the standard navigation approach I have used in the past - see this bitbucket code for an example.

Hitting the back button after a successful login will take the user back to the welcome page and hitting it again will exit the app. This provides the expected behavior for certification purposes, though you might be able to get away with calling NavigationService.RemoveBackEntry() later in MainPage.xaml if the previous page was WelcomePage.xaml and you did not want to navigate back to the welcome page. Subsequent runs of the app with valid credentials results in a direct navigation to MainPage.xaml where pressing the back button will exit the app.

The Rest of the Story

Hopefully what follows is more typical / obvious...

The user settings class that StartupNavigator uses is straightforward:

public class UserSettings
{
	private readonly ISettingsHelper _settingsHelper;

	public UserSettings(ISettingsHelper settingsHelper)
	{
		_settingsHelper = settingsHelper;
	}

	public string Username
	{
		get { return _settingsHelper.GetString("Username"); }
		set
		{
			_settingsHelper.SetSetting("Username", value);
		}
	}

	public string Password
	{
		get { return _settingsHelper.GetPrivateString("Password"); }
		set
		{
			_settingsHelper.SetPrivateString("Password", value);
		}
	}

	public bool? CredentialsValidated
	{
		get { return _settingsHelper.GetBool("CredentialsValidated"); }
		set
		{
			_settingsHelper.SetSetting("CredentialsValidated", value);
		}
	}

	public bool HasValidatedCredentials
	{
		get
		{
			return !string.IsNullOrEmpty(this.Username) 
				&& !string.IsNullOrEmpty(this.Password)
				&& true == this.CredentialsValidated;
		}
	}

	public void Save()
	{
		_settingsHelper.TrySave();
	}
}

In App.xaml.cs at the end of the constructor the startup navigator is initialized which sets up the UriMapper:

public App()
{
	// ... default App ctor initialization here	
	InitializeIoC();
    IoC.Get<IStartupNavigator>().Initialize();
}

private static void InitializeIoC()
{
	IoC.LoadModules(new MainModule());
}

When validated credentials were not found in settings, WelcomePage.xaml is navigated to which kicks off the WelcomeViewModel:

public class WelcomeViewModel : AppViewModelBase
{
	private readonly UserSettings _userSettings;
	private readonly IUserValidator _userValidator;
	private readonly IStartupNavigator _startupNavigator;

	public WelcomeViewModel(UserSettings userSettings, IUserValidator userValidator, 
		IStartupNavigator startupNavigator)
	{
		_userSettings = userSettings;
		_userValidator = userValidator;
		_startupNavigator = startupNavigator;
	}

	private string _username;
	public string Username
	{
		get { return _username; }
		set
		{
			if (_username != value)
			{
				_username = value;
				RaisePropertyChanged(()=> Username);
			}
		}
	}

	private string _password;
	public string Password
	{
		get { return _password; }
		set
		{
			if (_password != value)
			{
				_password = value;
				RaisePropertyChanged(()=> Password);
			}
		}
	}

	private void GetStarted()
	{
		// go ahead and save settings now. if incorrect and user comes back in 
		// another session, we want those values to stick
		_userSettings.Username = this.Username;
		_userSettings.Password = this.Password;
		_userSettings.Save();

		_userValidator.Validate(this.Username, this.Password, OnLoginSuccess);
	}

	public void Load()
	{
		this.Username = _userSettings.Username;
		this.Password = _userSettings.Password;
	}

	private void OnLoginSuccess(User user)
	{
		_userSettings.CredentialsValidated = true;
		_userSettings.Save();

		// TODO: do something with user result; persist somewhere so we don't have 
		// to fetch data again. Then on MainPage, elsewhere we can access later.
		Debug.WriteLine("Welcome " + user.Fullname);
		
		_startupNavigator.GoHome();
	}

	public ICommand GetStartedCommand
	{
		get {return new RelayCommand(GetStarted); }
	}
}

The user validation class simply creates a service client class that invokes a service method to validate the user credentials entered.

public class UserValidator : IUserValidator
{
	private readonly AppSettings _appSettings;
	private readonly IMessageBoxService _messageBoxService;

	public UserValidator(AppSettings appSettings, IMessageBoxService messageBoxService)
	{
		_appSettings = appSettings;
		_messageBoxService = messageBoxService;
	}

	public void Validate(string username, string password, Action<User> onSuccess)
	{
		if (string.IsNullOrEmpty(username) || string.IsNullOrEmpty(password))
		{
			_messageBoxService.ShowOKDispatch("Username and Password are required.", 
				"Required data");
			return;
		}

		var client = new DataClient(_appSettings.ApiKey, username, password);
		client.UserInfoComplete += (sender, args) =>
		{
			if (string.IsNullOrEmpty(args.Result.ErrorMessage))
				onSuccess(args.Result.Result);
			else
				_messageBoxService.ShowOKDispatch(args.Result.ErrorMessage, 
					"Login Failed");
		};

		client.FetchUserInfo();
	}
}

Sample Project

Download WinPhoneLoginRedirect.zip for a working sample project.

Windows Phone: App Config Settings - Thinking Outside the Box

File this post under the OCD, late night, crazy category. I just started a new Windows Phone application and have been working on a client for a service my app depends on. I wanted to test out the service interactions but with unit tests of my WP7 class library, not via building out a user interface.

Quickly there were a couple minor snags that I allowed myself to get bogged down in:

• State of testing support - The last time I attempted unit testing with WP7 I was not real happy with the testing support. Is it better now? Is there another better approach? More on this in an upcoming post; I changed my mind and decided to follow the single responsibility principle for this post :).
• Where to put service credentials, other test settings - The service API being tested requires a private API key, username and password. I thought the client and tests might go public later and I did not have a separate test account, only my own credentials. Where and how should these values be stored and passed to the tests so they are somewhat protected and configurable?

Config Settings Goals

In this context I am referring to settings as the rough equivalent of standard <appSettings>, not runtime settings for the app and user. My context is also currently one of a unit testing app project but some of this could be applicable to other projects as well.

There are a few options but I wanted one that addressed the below, some of which conflict. Goals:

1. Private settings - Namely keeping the credentials out of source control and any public distribution. I need to keep this data out of the tip as well as all revision history. If I have to remember to remove personal settings prior to a commit or publish, it is only a matter of time before I forget.
2. Configurable - Ideally these settings could be edited in a file w/o being manually typed into app code
3. Isolation - Some way to keep the more private settings separated from everything else, maybe outside the solution / IDE
4. Default settings - Exclusions and isolation implies the container for these settings might not exist (new source control pull etc.). However everything still needs to build and there should be a way to create empty/default settings when not found. It also needs to be obvious where/how people supply their own values for these settings.
5. Runtime performance - Preferably there would not be any real runtime I/O for these settings so test performance does not take a hit. These "app settings" are not changing during the course of an app / test execution run but they may change between runs.

Hard-coding settings

Manually hard-coding the credentials to test with is the easiest option and what I started with but it does not address any of goals above with the exception of #5 Runtime performance.

App.config

There is not really a concept of app.config for windows phone projects. Someone created a modified WP7 version of the mobile configuration block that could be used but it felt a bit forced to me and has some overhead.

Isolated storage

Isolated storage is more suited for runtime user settings / options / preferences to me. IsolatedStorageSettings.ApplicationSettings is an option but I cannot easily create, view and edit app config type settings without doing something programmatically or using some kind of isolated storage browser. The data will not be there to start with and the initial values have to come from somewhere. Additionally changes to the settings will be lost anytime the emulator app is restarted. Even all that aside, isolated storage doesn't quite meet the runtime performance goal and there are various caveats and serialization issues I have run into in the past. If you are interested in runtime user settings however, the code in my Codestock post has an example (just search for settings).

App.xaml

A custom settings class can be created and instantiated in App.xaml resources. First a simple TestSettings class:

public class TestSettings
    {
        public string ApiKey { get; set; }
        public string Username { get; set; }
        public string Password { get; set; }

        private static TestSettings _default;

        public static TestSettings Default
        {
            get { return _default; }
            set
            {
                if (null == _default)
                    _default = value;
            }
        }
    }

Next TestSettings.xaml to instantiate and configure the class.

<ResourceDictionary 
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="clr-namespace:DataBoundApp1"
    >
    <local:TestSettings x:Key="Settings" 
        ApiKey=""
        Username=""
        Password=""
    />      
</ResourceDictionary>

Then in App.xaml the resource is merged in:

<Application 
    x:Class="DataBoundApp1.App"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"       
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:shell="clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone">

    <Application.Resources>
        <ResourceDictionary Source="TestSettings.xaml" />
    </Application.Resources>

	<!-- ... -->
</Application>

Now we can set default settings in App.xaml.cs, or refer to the App class elsewhere:

private void Application_Launching(object sender, LaunchingEventArgs e)
{
	TestSettings.Default = this.Settings;
}

public TestSettings Settings
{
	get { return (TestSettings)this.Resources["Settings"]; }
} 

/* or elsewhere:
//    var settings = ((App)Application.Current).Settings
*/

All of that is fine and good except it does not meet the goal on private settings. If I plug my private settings in TestSettings.xaml, I have to remember to later empty those out. I cannot just exclude the file from source control as it is required to compile the app.

One solution is to use pre and post build events to temporarily swap settings in and out automatically. We can create 2 new copies of TestSettings.xaml outside of Visual Studio to facilitate this. TestSettingsPrivate.xaml will contain the personal / private settings values that we do not want in the project or source control. TestSettingsDefault.xaml will contain the empty (or default) settings that we want to include in the project and source control for others as a starting point. TestSettings.xaml will be our "Main staging area" that will be replaced before and after; sort of like Towers of Hanoi or a shell game, except not. :)

In the pre-build event of the test project (all on one line):

if exist "$(ProjectDir)TestSettingsPrivate.xaml" xcopy /y "$(ProjectDir)TestSettingsPrivate.xaml" "$(ProjectDir)TestSettings.xaml"

and then in the post-build event (all on one line):

if exist "$(ProjectDir)TestSettingsDefault.xaml" xcopy /y "$(ProjectDir)TestSettingsDefault.xaml" "$(ProjectDir)TestSettings.xaml"

If the default and private settings do not exist, as in the case of a new source control pull, then the default/empty contents of TestSettings.xaml get used. In that case you might wonder why bother with TestSettingsDefault.xaml when TestSettings.xaml could be "backed up" and restored automatically. The main reason has to do with the way Visual Studio locks files during a build; trying to make a copy either did not work or generated an "exited with code 2" failure.

Just before the build the test settings file will get replaced by the private settings file and that will get compiled into the assembly. A moment later after the build the test settings file will get reset back to the defaults just as if nothing happened. I thought Visual Studio might pop-up a "this file has been modified outside the environment" type dialog but that was not the case.

If we peek into the assembly contents we can see the private settings were sucked into the DLL while the empty/default TestSettings.xaml remains without the private settings post-build.



Granted when distributing the project I need to be sure not to include the compiled DLL and the private settings file. This is not a problem if I exclude them from source control, only if I zip up the directory. If using Mercurial for source control my .hgignore file might look like the below:

	syntax: glob

	*.csproj.user
	obj/
	bin/
	*.ncb
	*.suo
	_ReSharper.*
	*.resharper.user
	MyApp.Tests/TestSettingsPrivate.xaml

Download DataBoundApp1.zip for an example project of this approach.

T4 and JSON Settings

Another option is T4. First I added a new text file TestSettings.txt to the project. Then I renamed .txt to .t4 (.tt works too) and set the file's Custom Tool to TextTemplatingFileGenerator. In the T4 file I first setup the generation of what will become TestSettings.generated.cs:



Next the T4 file looks for a TestSettings.json file (not included in the project) and if it does not exist it creates it by instantiating a new instance of TestSettings and serializing that to JSON (JSON Helper method shown later). One thing to note is the use of file I/O methods like File.CreateText; these are marked [SecurityCritical] and we cannot call them at runtime in a windows phone app but at "design time" in the IDE it works fine. Another thing to note is that Host.ResolvePath throws an exception if the file does not exist; the reference to App.xaml is just to get a fixed handle on some valid file so we can resolve the directory.



Why JSON for the settings? Well if you have just string properties you can get away with simple string.split, name/value pairs etc. With other data types JSON makes more sense and it is pretty readable and hand-editable (maybe with the exception of dates). Why create a custom format or go through more overhead with XML when JSON works fine? The next block of code reads the json file and builds a C# code block that will instantiate TestSettings in the generated code class.



Finally the T4 has a JSON helper method to serialize the settings object. I tried using JSONSerializer initially but it was throwing "could not find file or assembly ..." exceptions.

TestSettings.json will get created initially when it does not exist with empty values. It can then be hand-edited outside the IDE.

{
  "ApiKey": "SecretKey",
  "Username": "SecretUsername",
  "Password": "SecretPass!",
  "IntProperty": 0,
  "BoolProperty": false,
  "DateProperty": "\/Date(-62135578800000-0500)\/"
}

This T4 will produce a TestSettings.generated.cs that looks like below. Note that for this example I took the existing TestSettings.cs and made it a partial class and added some properties of other data types just for testing.

namespace MyApp.Tests
{
    // see also TestSettings.json outside ide in this folder for default values
	public partial class TestSettings //.generated.cs
	{
		private static readonly TestSettings _defaultInstance;

		public static TestSettings Default
		{
			get {return _defaultInstance;}
		}	

		static TestSettings()
		{
			_defaultInstance = new TestSettings
			{
				ApiKey = "SecretKey",
				Username = "SecretUsername",
				Password = "SecretPass!",
				IntProperty = 0,
				BoolProperty = false,
				DateProperty = System.DateTime.MinValue       
			};
		}
	}
}

At this point you may be thinking, "so what? you've just hard-coded the settings again but in a more elaborate way." That is true to an extent but this generated code class as well as the settings json file are both discardable now. As long as the t4 file is there it will take care of the rest. These can be excluded from source control just like the private XAML settings file. This approach has the advantage of only needing a single settings file and it is the least hit to performance at runtime as the work is already done and compiled by then just as if I hard-coded the settings by hand.

By default with this approach if I update the settings file I have to manually re-run the T4 transformation. However I can use something like Chirpy to automatically run the T4 on every build if desired.

Download: TestSettings.t4

Conclusion

This certainly isn't an exhaustive list of possibilities and some of this may be a bit crazy but at any rate it is food for thought.