Search
Recent Tweets

Entries in config (2)

Wednesday
Jan252012

Windows Phone: App Config Settings - Thinking Outside the Box

File this post under the OCD, late night, crazy category. I just started a new Windows Phone application and have been working on a client for a service my app depends on. I wanted to test out the service interactions but with unit tests of my WP7 class library, not via building out a user interface.

Quickly there were a couple minor snags that I allowed myself to get bogged down in:
  • State of testing support - The last time I attempted unit testing with WP7 I was not real happy with the testing support. Is it better now? Is there another better approach? More on this in an upcoming post; I changed my mind and decided to follow the single responsibility principle for this post :).
  • Where to put service credentials, other test settings - The service API being tested requires a private API key, username and password. I thought the client and tests might go public later and I did not have a separate test account, only my own credentials. Where and how should these values be stored and passed to the tests so they are somewhat protected and configurable?

Config Settings Goals

In this context I am referring to settings as the rough equivalent of standard <appSettings>, not runtime settings for the app and user. My context is also currently one of a unit testing app project but some of this could be applicable to other projects as well.

There are a few options but I wanted one that addressed the below, some of which conflict. Goals:
  1. Private settings - Namely keeping the credentials out of source control and any public distribution. I need to keep this data out of the tip as well as all revision history. If I have to remember to remove personal settings prior to a commit or publish, it is only a matter of time before I forget.
  2. Configurable - Ideally these settings could be edited in a file w/o being manually typed into app code
  3. Isolation - Some way to keep the more private settings separated from everything else, maybe outside the solution / IDE
  4. Default settings - Exclusions and isolation implies the container for these settings might not exist (new source control pull etc.). However everything still needs to build and there should be a way to create empty/default settings when not found. It also needs to be obvious where/how people supply their own values for these settings.
  5. Runtime performance - Preferably there would not be any real runtime I/O for these settings so test performance does not take a hit. These "app settings" are not changing during the course of an app / test execution run but they may change between runs.

Hard-coding settings

Manually hard-coding the credentials to test with is the easiest option and what I started with but it does not address any of goals above with the exception of #5 Runtime performance.

App.config

There is not really a concept of app.config for windows phone projects. Someone created a modified WP7 version of the mobile configuration block that could be used but it felt a bit forced to me and has some overhead.

Isolated storage

Isolated storage is more suited for runtime user settings / options / preferences to me. IsolatedStorageSettings.ApplicationSettings is an option but I cannot easily create, view and edit app config type settings without doing something programmatically or using some kind of isolated storage browser. The data will not be there to start with and the initial values have to come from somewhere. Additionally changes to the settings will be lost anytime the emulator app is restarted. Even all that aside, isolated storage doesn't quite meet the runtime performance goal and there are various caveats and serialization issues I have run into in the past. If you are interested in runtime user settings however, the code in my Codestock post has an example (just search for settings).

App.xaml

A custom settings class can be created and instantiated in App.xaml resources. First a simple TestSettings class:
public class TestSettings
    {
        public string ApiKey { get; set; }
        public string Username { get; set; }
        public string Password { get; set; }

        private static TestSettings _default;

        public static TestSettings Default
        {
            get { return _default; }
            set
            {
                if (null == _default)
                    _default = value;
            }
        }
    }
Next TestSettings.xaml to instantiate and configure the class.
<ResourceDictionary 
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="clr-namespace:DataBoundApp1"
    >
    <local:TestSettings x:Key="Settings" 
        ApiKey=""
        Username=""
        Password=""
    />      
</ResourceDictionary>
Then in App.xaml the resource is merged in:
<Application 
    x:Class="DataBoundApp1.App"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"       
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:shell="clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone">

    <Application.Resources>
        <ResourceDictionary Source="TestSettings.xaml" />
    </Application.Resources>

	<!-- ... -->
</Application>
Now we can set default settings in App.xaml.cs, or refer to the App class elsewhere:
private void Application_Launching(object sender, LaunchingEventArgs e)
{
	TestSettings.Default = this.Settings;
}

public TestSettings Settings
{
	get { return (TestSettings)this.Resources["Settings"]; }
} 

/* or elsewhere:
//    var settings = ((App)Application.Current).Settings
*/
All of that is fine and good except it does not meet the goal on private settings. If I plug my private settings in TestSettings.xaml, I have to remember to later empty those out. I cannot just exclude the file from source control as it is required to compile the app.

One solution is to use pre and post build events to temporarily swap settings in and out automatically. We can create 2 new copies of TestSettings.xaml outside of Visual Studio to facilitate this. TestSettingsPrivate.xaml will contain the personal / private settings values that we do not want in the project or source control. TestSettingsDefault.xaml will contain the empty (or default) settings that we want to include in the project and source control for others as a starting point. TestSettings.xaml will be our "Main staging area" that will be replaced before and after; sort of like Towers of Hanoi or a shell game, except not. :)

In the pre-build event of the test project (all on one line):
if exist "$(ProjectDir)TestSettingsPrivate.xaml" xcopy /y "$(ProjectDir)TestSettingsPrivate.xaml" "$(ProjectDir)TestSettings.xaml"

and then in the post-build event (all on one line):
if exist "$(ProjectDir)TestSettingsDefault.xaml" xcopy /y "$(ProjectDir)TestSettingsDefault.xaml" "$(ProjectDir)TestSettings.xaml"

If the default and private settings do not exist, as in the case of a new source control pull, then the default/empty contents of TestSettings.xaml get used. In that case you might wonder why bother with TestSettingsDefault.xaml when TestSettings.xaml could be "backed up" and restored automatically. The main reason has to do with the way Visual Studio locks files during a build; trying to make a copy either did not work or generated an "exited with code 2" failure.

Just before the build the test settings file will get replaced by the private settings file and that will get compiled into the assembly. A moment later after the build the test settings file will get reset back to the defaults just as if nothing happened. I thought Visual Studio might pop-up a "this file has been modified outside the environment" type dialog but that was not the case.

If we peek into the assembly contents we can see the private settings were sucked into the DLL while the empty/default TestSettings.xaml remains without the private settings post-build.



Granted when distributing the project I need to be sure not to include the compiled DLL and the private settings file. This is not a problem if I exclude them from source control, only if I zip up the directory. If using Mercurial for source control my .hgignore file might look like the below:
	syntax: glob

	*.csproj.user
	obj/
	bin/
	*.ncb
	*.suo
	_ReSharper.*
	*.resharper.user
	MyApp.Tests/TestSettingsPrivate.xaml
Download DataBoundApp1.zip for an example project of this approach.

T4 and JSON Settings

Another option is T4. First I added a new text file TestSettings.txt to the project. Then I renamed .txt to .t4 (.tt works too) and set the file's Custom Tool to TextTemplatingFileGenerator. In the T4 file I first setup the generation of what will become TestSettings.generated.cs:



Next the T4 file looks for a TestSettings.json file (not included in the project) and if it does not exist it creates it by instantiating a new instance of TestSettings and serializing that to JSON (JSON Helper method shown later). One thing to note is the use of file I/O methods like File.CreateText; these are marked [SecurityCritical] and we cannot call them at runtime in a windows phone app but at "design time" in the IDE it works fine. Another thing to note is that Host.ResolvePath throws an exception if the file does not exist; the reference to App.xaml is just to get a fixed handle on some valid file so we can resolve the directory.



Why JSON for the settings? Well if you have just string properties you can get away with simple string.split, name/value pairs etc. With other data types JSON makes more sense and it is pretty readable and hand-editable (maybe with the exception of dates). Why create a custom format or go through more overhead with XML when JSON works fine? The next block of code reads the json file and builds a C# code block that will instantiate TestSettings in the generated code class.



Finally the T4 has a JSON helper method to serialize the settings object. I tried using JSONSerializer initially but it was throwing "could not find file or assembly ..." exceptions.

TestSettings.json will get created initially when it does not exist with empty values. It can then be hand-edited outside the IDE.
{
  "ApiKey": "SecretKey",
  "Username": "SecretUsername",
  "Password": "SecretPass!",
  "IntProperty": 0,
  "BoolProperty": false,
  "DateProperty": "\/Date(-62135578800000-0500)\/"
}
This T4 will produce a TestSettings.generated.cs that looks like below. Note that for this example I took the existing TestSettings.cs and made it a partial class and added some properties of other data types just for testing.
namespace MyApp.Tests
{
    // see also TestSettings.json outside ide in this folder for default values
	public partial class TestSettings //.generated.cs
	{
		private static readonly TestSettings _defaultInstance;

		public static TestSettings Default
		{
			get {return _defaultInstance;}
		}	

		static TestSettings()
		{
			_defaultInstance = new TestSettings
			{
				ApiKey = "SecretKey",
				Username = "SecretUsername",
				Password = "SecretPass!",
				IntProperty = 0,
				BoolProperty = false,
				DateProperty = System.DateTime.MinValue       
			};
		}
	}
}
At this point you may be thinking, "so what? you've just hard-coded the settings again but in a more elaborate way." That is true to an extent but this generated code class as well as the settings json file are both discardable now. As long as the t4 file is there it will take care of the rest. These can be excluded from source control just like the private XAML settings file. This approach has the advantage of only needing a single settings file and it is the least hit to performance at runtime as the work is already done and compiled by then just as if I hard-coded the settings by hand.

By default with this approach if I update the settings file I have to manually re-run the T4 transformation. However I can use something like Chirpy to automatically run the T4 on every build if desired.

Download: TestSettings.t4

Conclusion

This certainly isn't an exhaustive list of possibilities and some of this may be a bit crazy but at any rate it is food for thought.
Tuesday
Oct192010

Web.config automation with T4 and a macro

Recently I blogged about my pain with getting Silverlight, WCF, IIS, Windows Authentication and Oracle to play nicely together. One of the pain points from that was needing one set of config settings for running the app and a different set anytime I needed to update the service reference. Specifically, for updating the service reference, serviceMetadata.httpGetEnabled must be true, IIS needs to allow anonymous access, and the Mex endpoint needs to be defined. When running the app the exact opposite is true.

With at least 3 developers working on the app and more WCF changes, updating the service reference was becoming a pain. Ideally our end state might be getting rid of the service reference and instead using ChannelFactory<T> along with some T4 to do "WCF the manual way, the right way" but without all the manual pain of creating begin/end async methods and all the other hoopla.

Until then we decided to come up with a solution for managing the config pain in the short term.

Some solutions

Web.config transformations This seemed ideal as I could just define what to replace, add or remove that varied per build configuration. It generated the correct config file just fine. The first problem I had with it is only actually uses the custom config when you go to publish the web app. As I'm not deploying to another server this didn't help much. I suppose I could have deployed to my own box but that is not ideal either. I was hoping simply changing the build configuration might determine which web.config to use. The other issue is that a separate build configuration is a hassle with 13 projects, half of which must target x86 and the other Any CPU.
MSBuild, build events and scripts The next thing I tried was directly invoking MSBuild with something like msbuild "%1" /T:TransformWebConfig /P:Configuration=%2. While that generated the specific config directly, the output was buried in subfolders and getting a "reference" to that location and dealing with copying the file and source control was trouble.

I went down the path of custom build events and started both batch files and powershell scripts but each route had some annoying roadblock / limitation of sorts.
T4 T4 seemed ideal in ways in that different web.config files could be created without having to have separate build configurations. I decided to go down this path and would later add a Visual Studio macro to take it one step further.

A starting point

With some quick research I came across Using T4 to auto-generate web.config files. That seemed to be a good start so I installed T4 Toolbox and Tangible T4 Editor (free edition) and began the T4 work. In referencing that blog post though there were a few things that were not obvious to me and some issues I had.

These issues included:
  • Visual Studio hangs - The Tangible T4 editor was hanging Visual Studio on opening any .tt file. After reading through the extension page comments, I realized the initial open built a cache and I had to wait 5+ minutes for it to complete. Later it still could take 10+ seconds opening a .tt file so I emailed the developer who indicated it was a side by side issue with Resharper that was being worked on. Patience is a virtue...
  • Compile transform errors - Next I ran into errors such as "Compiling transformation: The type or namespace name 'Generator' could not be found" and likewise for 'Template'. First I did not realize I needed to include T4Toolbox.tt, located at %PROGRAMFILES%\T4 Toolbox\. Reading over some T4 Toolbox blog entries I next realized the generator and template files needed to have the custom tool file property cleared as they are not intended for direct use but to be called from another template.
  • Excel / CSV file - The post indicated an Excel file for storing values that differed per config but not what format. Turns out it was a tab-delimited CSV file. Editing that in Excel though meant losing the formatting unless saving as xlsx or xls, and saving in those formats meant saving a different copy to be read by the T4 templates. Excel also caused some formatting issues with certain data such as additional quotes. Editing the tab-delimited CSV file in a text editor wasn't much better. I decided to change the "config values" file storage to XML and rewrote the generator to account for that.
  • Generator creation - I could not see where the filename was being passed to the generator in the post. I ended up creating a "calling / driver" T4 template to create and invoke the generator.

WebConfig.xml

This file stores the values that differ per web.config version. The config name can be anything but it will get stamped into the generated filename when run. As this solution was only for local development I only needed two config modes, one for updating the service reference and one for running the app. The individual items can be named as desired but I found it helpful to mimic the name/path in the web.config. Values can be simple values or tags.
<?xml version="1.0" encoding="utf-8" ?>
<configs>
	<config name="UpdateServiceReference">
		<item name="serviceMetadata.httpGetEnabled">true</item>
		<item name="system.web.authorization.deny"/>
		<item name="MexEndpoint">
			<endpoint name="MexEndpoint" address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
		</item>
	</config>
	<config name="Runtime">
		<item name="serviceMetadata.httpGetEnabled">false</item>
		<item name="system.web.authorization.deny">?</item>
		<item name="MexEndpoint"/>
	</config>
</configs>

WebConfigGenerator.tt

The generator takes the path to the "config values" xml file (WebConfig.xml here), reads in the data and creates a dictionary, and generates one config file per configuration using WebConfigTemplate.tt as the template. Make sure to clear the Custom Tool value for this file in file properties.
<#@ include file="WebConfigTemplate.tt" #>
<#@ assembly name="System.Xml.Linq" #>
<#@ import namespace="System" #>
<#@ import namespace="System.IO" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ import namespace="System.Xml.Linq" #>

<#+
public class WebConfigGenerator : Generator
{
	private readonly string webConfigSettingsLocation;
	
	public WebConfigGenerator(string webConfigSettingsLocation)
	{
		this.webConfigSettingsLocation = webConfigSettingsLocation;
	}

	protected override void RunCore()
	{
		XDocument xDoc = XDocument.Load(this.webConfigSettingsLocation);		
		
		foreach (var xConfig in xDoc.Element("configs").Elements("config"))
		{
			var configName = xConfig.Attribute("name").Value.ToString();
			var values = new Dictionary<string, string>();
			
			foreach (XElement xItem in xConfig.Elements("item"))
			{
				if (!xItem.HasElements)
					values.Add(xItem.Attribute("name").Value.ToString(), xItem.Value.ToString());
				else 
				{
					var result = string.Concat(xItem.Nodes());
					values.Add(xItem.Attribute("name").Value.ToString(), result);
				}
			}
			
			var webConfigTemplate = new WebConfigTemplate(values);
			//webConfigTemplate.Output.File = "../Web.config." + configName;
			webConfigTemplate.Output.File = "WebConfig." + configName + ".config";

			if (configName == "local")
				webConfigTemplate.Output.BuildAction = BuildAction.None;
			else
				webConfigTemplate.Output.BuildAction = BuildAction.Content;
			webConfigTemplate.Render();		
		}		
	}
}
#>

WebConfigTemplate.tt

This contains the entire web.config but has variable placeholders for each area that needs to be dynamic. Unlike with a web.config transformation, this is not describing only what needs to change. Therefore with this approach it is best not to directly modify web.config but instead this template, then take the desired T4 config output and update the root web.config with that. This is a disadvantage of this technique though in practice I do not see it as an issue for my needs. Make sure to clear the Custom Tool value for this file in file properties.

<#@ import namespace="System" #>
<#@ import namespace="System.Collections.Generic" #>

<#+
public class WebConfigTemplate : Template
{
	private readonly Dictionary<string, string> values;
	
	public WebConfigTemplate(Dictionary<string, string> values)
	{
		this.values = values;
	}

	public override string TransformText()
	{ // make sure there is no whitespace at all before the xml declaration!!!
	#><?xml version="1.0" encoding="UTF-8"?>
<configuration>
	<!-- PUT YOUR WEB.CONFIG HERE and use the <#= values["MyKeyHere"] #> -->
</configuration>
	
    <#+    return this.GenerationEnvironment.ToString();
	}
}
#>

WebConfig.tt

This is the "top-level", calling template that invokes the generator and passes it the filename where the config settings are stored.
<#@ template  debug="true" hostSpecific="true" #>
<#@ output extension=".config" #>
<#@ Assembly Name="System.Core.dll" #>
<#@ import namespace="System" #>

<#@ include file="T4Toolbox.tt" #>
<#@ include file="WebConfigGenerator.tt" #>


<#
    var file = Host.ResolvePath(@"WebConfig.xml");
	var configGen = new WebConfigGenerator(file);
    configGen.Run();
#>

Configuration generated, now what?



Invoking "Transform all templates" or changing WebConfig.tt will result in adding/updating the different config files and Visual Studio will prompt to checkout from Source Control. However the "real" web.config file still needs to be checked out and its contents replaced with the config version to be switched to...

Enter the macro

The Visual Studio macro takes care of selecting the Web.config file in Solution Explorer, issuing a TFS checkout if not already checked out, and replacing Web.config with the desired target config. When ApplySelectedConfig() is called, it will take whatever WebConfig.configName.config is selected in Solution Explorer (displaying an error if no correct selection) and overwrite web.config with that. Other public methods exist to switch to a known config filename without requiring any selection in Solution Explorer.
Imports System
Imports EnvDTE
Imports EnvDTE80
Imports EnvDTE90
Imports EnvDTE90a
Imports EnvDTE100
Imports System.Diagnostics
Imports System.IO

Public Module WebConfigModule

    Public Sub UpdateServiceReference()
        MsgBox("For now please use Update Service Reference manually.", MsgBoxStyle.Information, "Not Ready")
    End Sub


    Public Sub ApplySelectedConfig()
        Dim solExplore = GetSolutionExplorer()

        If (solExplore Is Nothing OrElse solExplore.SelectedItems Is Nothing) Then
            MsgBox("No selected item detected in solution explorer", MsgBoxStyle.Exclamation, "No Selected Document")
        End If

        Dim selItem As ProjectItem = solExplore.SelectedItems(0).Object

        If IsNothing(selItem) Then
            MsgBox("You must select a file (specific WebConfig.*.config)", MsgBoxStyle.Exclamation, "No Active Document")
            Return
        End If

        If Not selItem.Name.StartsWith("WebConfig.") OrElse Not selItem.Name.EndsWith(".config") Then
            MsgBox("You must select an environment web.config.* file", MsgBoxStyle.Exclamation, "No Env Config Selected")
            Return
        End If

        Dim configFilenameToApply = selItem.FileNames(0)
        ApplyConfigFilename(configFilenameToApply)
    End Sub

    Public Sub ApplyUpdateServiceConfig()
        ApplyConfigName("WebConfig.UpdateServiceReference.config")
    End Sub


    Public Sub ApplyRuntimeConfig()
        ApplyConfigName("WebConfig.Runtime.config")
    End Sub


    '--------------------------------------------------------------------------
    ' Private methods... helpers etc.
    '--------------------------------------------------------------------------

    Private Sub ApplyConfigName(ByVal configName As String)
        Dim solution As Solution2 = DTE.Solution
        If Not (solution.IsOpen) Then
            MsgBox("You must have the solution open", MsgBoxStyle.Exclamation, "No solution")
            Return
        End If

        Dim configProjItem = solution.FindProjectItem(configName)

        If configProjItem Is Nothing Then
            MsgBox("Could not find " + configName, MsgBoxStyle.Exclamation, "Config not found")
            Return
        End If

        configProjItem.ExpandView()
        Dim configFilenameToApply = configProjItem.FileNames(0)
        ApplyConfigFilename(configFilenameToApply)
    End Sub

    Private Sub ApplyConfigFilename(ByVal configFilenameToApply)
        Dim solution As Solution2 = DTE.Solution
        If Not (solution.IsOpen) Then Return

        Dim webConfigProjItem = solution.FindProjectItem("Web.config")
        webConfigProjItem.ExpandView()

        Dim webConfigFilename = webConfigProjItem.FileNames(0)

        Dim SolutionExplorerPath As String
        Dim items As EnvDTE.UIHierarchyItems = DTE.ToolWindows.SolutionExplorer.UIHierarchyItems
        Dim item As Object = FindItem(items, webConfigFilename, SolutionExplorerPath)

        If item Is Nothing Then
            MsgBox("Couldn't find web.config in Solution Explorer.")
            Return
        End If

        ' select web.config in solution explorer
        DTE.Windows.Item(Constants.vsWindowKindSolutionExplorer).Activate()
        DTE.ActiveWindow.Object.GetItem(SolutionExplorerPath).Select(vsUISelectionType.vsUISelectionTypeSelect)

        ' and issue a checkout if needed
        Dim isCheckedOut = DTE.SourceControl.IsItemCheckedOut(webConfigFilename)
        If Not isCheckedOut Then
            DTE.ExecuteCommand("File.TfsCheckOut")
        End If

        'overwrite web.config 
        File.Copy(configFilenameToApply, webConfigFilename, True)
    End Sub

    Private Function FindItem(ByVal Children As UIHierarchyItems, ByVal FileName As String, ByRef SolutionExplorerPath As String) As Object
        For Each CurrentItem As UIHierarchyItem In Children
            Dim TypeName As String = Microsoft.VisualBasic.Information.TypeName(CurrentItem.Object)
            If TypeName = "ProjectItem" Then
                Dim projectitem As EnvDTE.ProjectItem = CType(CurrentItem.Object, EnvDTE.ProjectItem)
                Dim i As Integer = 1
                While i <= projectitem.FileCount
                    Debug.WriteLine(projectitem.FileNames(i))

                    If projectitem.FileNames(i) = FileName Then
                        SolutionExplorerPath = CurrentItem.Name
                        Return CurrentItem
                    End If
                    i = i + 1
                End While
            End If

            Dim ChildItem As UIHierarchyItem = FindItem(CurrentItem.UIHierarchyItems, FileName, SolutionExplorerPath)
            If Not ChildItem Is Nothing Then
                SolutionExplorerPath = CurrentItem.Name + "\" + SolutionExplorerPath
                Return ChildItem
            End If
        Next
    End Function

    Private Function GetSolutionExplorer() As UIHierarchy
        Dim solExplore As UIHierarchy
        solExplore = DTE.Windows.Item(Constants.vsext_wk_SProjectWindow).Object()
        Return solExplore
    End Function

End Module

Running the Macro

There are multiple ways to run a Visual Studio macro but I prefer the immediate/command window:


Afterwards a macro method can be called to revert the config changes though usually I end up undoing pending changes on Web.config when the work is done and I need to switch back to the original configuration.

Conclusion

This is one of a few ways to handle this situation. It may seem a little crazy in ways but it works for now and each technique has its own pros and cons.

One current problem with the T4 generation is that sometimes visual studio prompts to checkout T4Toolbox.tt. I am not sure why this happens yet and in comparing changes, nothing is different. T4Toolbox.tt could be excluded from source control but that could break initial get latest version operations, at least when performed from the solution and not Source Control Explorer.

I started to automate updating the service reference itself but will have to come back to that another day. While SvcUtil could be called directly, source control would likely be a problem. However I'm assuming that inside the macro it would be possible to update the service reference programmatically similar to this code.

Subscribe to this feed