Deploying .NET applications with the Nix package manager

Deploying .NET applications with the Nix packagemanager

Sander van der Burg

Delft University of Technology, EEMCS,Department of Software Technology

Philips Healthcare, Philips Informatics Infrastructure (PII),Best

June 15, 2010

Sander van der Burg Deploying .NET applications with the Nix package manager

Nix deployment system

Nix is a package manager, with some distinct features:

Stores components in isolation from each other

Allows using multiple variants/versions safely next to eachother

Builds components from declarative specifications

Reproducibility

Supports atomic upgrades and rollbacks

Garbage collection

Safely removes obsolete components


Nix deployment system

About 2500 open-source/free software packages are supported(Nixpkgs)

Used as a basis for the NixOS GNU/Linux distribution


Hydra

Continuous build & integration server

Built upon the Nix package manager

Used at TU Delft for building, integration and testing ofseveral projects on multiple platforms


Disnix

Distributed deployment extension for Nix

Manages inter-dependencies

Builds, distributes, activates services in a network of machines


The Nix store

All components are stored in isolation in a Nix store

In Nix every component name consists of a hash code andcomponent name:

/nix/store/30v58jznlaxnr4bca3hiz64wlb43mgpx-hello-2.5

Hash is generated from all the input arguments of the buildfunction: compilers, libraries, build scripts etc.

Multiple variants/versions can be kept safely next to eachother

Can be used for any type of component


The Nix store

/nix/store

44ef921582e840dae...-glibc-2.9

lib

libc.so

a1v58jz2la6nr7bcr...-glibc-2.11.1

lib

libc.so

30v58jznlaxnr4bca...-hello-2.5

bin

hello

0a4d829298227c98c...-hello-2.5

bin

hello


Global Assembly Cache (GAC)

.NET Global Assembly Cache (GAC) only supports strong-namedlibrary assemblies (DLLs):

Other types of components are not supported (executables,native DLLs, compilers)

Strong name is created from name, version, culture andsigning with a public-private key pair

Creating strong-names is sometimes painful

Possible to produce a different assembly with samestrong-name


Nix expressions

{stdenv, fetchurl}:

stdenv.mkDerivation {

name = "hello-2.5";

src = fetchurl {

url = ftp://ftp.gnu.org/gnu/hello/hello-2.5.tar.gz;

sha256 = "0in467phypnis2ify1gkmvc5l2fxyz3s4xss7g74gwk279ylm4r2";

};

meta = {

description = "A program that produces a familiar, friendly greeting";

homepage = http://www.gnu.org/software/hello/manual/;

license = "GPLv3+";

};

}

Specifies how the hello component should be built.


Nix expressions

rec {

stdenv = ...

curl = ...

openssl = ...

fetchurl = import ../build-support/fetchurl {

inherit curl openssl;

};

hello = import ../applications/misc/hello {

inherit stdenv fetchurl;

};

}

The hello build function is called with its arguments


Nix expressions

To build the Hello package:

$ nix-build all-packages.nix -A hello

Will recursively build hello and its dependencies and produces helloin:

/nix/store/30v58jznlaxnr4bca3hiz64wlb43mgpx-hello-2.5


Runtime dependencies

Nix detects runtime dependencies of a build

Nix scans all files of a component for hash-codes of adependency, e.g. RPATH defined in a ELF header

Uses a notion of closures to guarantee completeness:

If a specific component needs to be deployed, all its runtimedependencies are deployed first


Runtime dependencies

/nix/store

44ef921582e840dae...-glibc-2.9

lib

libc.so

a1v58jz2la6nr7bcr...-glibc-2.11.1

lib

libc.so

30v58jznlaxnr4bca...-hello-2.5

bin

hello

0a4d829298227c98c...-hello-2.5

bin

helloSander van der Burg Deploying .NET applications with the Nix package manager

Supporting .NET applications with Nix

Nix is designed for use on UNIX like systems, e.g. GNU/Linux,FreeBSD, Mac OS X


Requirements

Buildtime support:

Compile Visual Studio solutions and produce assemblies

Invoke MSBuild.exe with right parameters

The build process must find its buildtime dependencies

Runtime support:

Assemblies must find its runtime dependencies

Integrate .NET CLR dependency resolver with Nix


Windows support

Nix is supported on Windows through Cygwin:

Offers UNIX implementation on top of Windows

Allows combining UNIX and Windows utilities


Windows support

Several Cygwin utilities map Windows concepts to UNIX conceptsand vica versa:

Can be used to glue UNIX tooling to Windows processes

cygpath, UNIX path names <=> Windows path names


Implementing build support

A .NET Nix build function dotnetenv.buildSolution isimplemented:

Takes source code of a Visual Studio C# solution

Produces output of the solution into a unique store path,based on the input arguments of this function


Build function implementation

{stdenv, dotnetfx}:

{ name, src, slnFile, targets ? "ReBuild"

, options ? "/p:Configuration=Debug;Platform=Win32"

, assemblyInputs ? []

}:

stdenv.mkDerivation {

inherit name src;

buildInputs = [ dotnetfx ];

installPhase = ’’

for i in ${toString assemblyInputs}; do

windowsPath=$(cygpath --windows $i)

AssemblySearchPaths="$AssemblySearchPaths;$windowsPath"

done

export AssemblySearchPaths

ensureDir $out

outPath=$(cygpath --windows $out)\\

MSBuild.exe ${slnFile} /nologo /t:${targets} \

/p:OutputPath=$outPath ${options} ...

’’;

}


Example usage

{dotnetenv, MyAssembly1, MyAssembly2}:

dotnetenv.buildSolution {

name = "My.Test.Assembly";

src = /path/to/source/code;

slnFile = "Assembly.sln";

assemblyInputs = [

dotnetenv.assembly20Path

MyAssembly1

MyAssembly2

];

}

Specifies how a Visual Studio solution should be built.


Example usage

rec {

dotnetfx = ...

stdenv = ...

dotnetenv = import ../dotnetenv {

inherit stdenv dotnetfx;

};

MyAssembly1 = import ../MyAssembly1 {

inherit dotnetenv;

};

MyAssembly2 = import ../MyAssembly1 {

inherit dotnetenv;

};

MyTestAssembly = import ../MyTestAssembly {

inherit dotnetenv MyAssembly1 MyAssembly2;

};

}

The MyTestAssembly function is called with its arguments


Example usage

To build the My.Test.Assembly component:

$ nix-build all-packages.nix -A MyTestAssembly

Recursively builds My.Test.Assembly and its dependencies, i.e.MyAssembly1, MyAssembly2, MyTestAssembly and producesoutput in:

/nix/store/ri0zzm2hmwg01w2wi0g4a3rnp0z24r8p-My.Test.Assembly


Implementing runtime support

.NET runtime locaties assemblies:

Determines the correct version of the assembly (strong-namedassemblies only)

Checks whether the assembly has been bound before(strong-named assemblies only)

If so, it uses this versionShares same assemblies in memory between applications

Checks the Global Assembly Cache (GAC) (strong-namedassemblies only)

Probes the assembly

Checking the <codebase> element in the application .config

Probing heuristics


Assembly probing

A <codebase> element in the application .config file can specifyits own assembly references:

Private assemblies can only reside in the directory or asubdirectory of the executable

Strong-named assemblies can be invoked from an arbitrarylocation, including remote locations

Referenced assembly is cached, therefore a strong-name isrequired


Implementing runtime support in Nix

Not very trivial, 3 options:

Copy dependent assemblies into the same Nix store path asthe executable

Generate an application .config file with <codebase>

section

Symlink dependent assemblies into the same Nix store path asthe executable


Copy dependent assemblies

Copy dependent assemblies into the same Nix store path as theexecutable:

Essentially static linking

Expensive in terms of disk space

Expensive in terms of memory usage

Works with private and strong-named assemblies

Works on Windows 2000, XP


Copy dependent assemblies

/nix/store

99bed1970e5e459bc...-My.Assembly1

My.Assembly1.dll

f4e5d4c96f95c5887...-My.Assembly2

My.Assembly2.dll

ri0zzm2hmwg01w2wi...-My.Test.Assembly

My.Assembly1.dll

My.Assembly2.dll

My.Test.Assembly.exe


Generate an application config file

Generate an application config file with <codebase> section:

References assemblies in other Nix store paths

Nix scans and detects runtime dependencies due to uniquestore paths

Cheap in terms of disk space

Allows efficient upgrading of a system (i.e. only replacingupdated parts)

Only works with strong-named assemblies

Cheap in terms of memory usage (sharing in memory)


Generate an application config file

/nix/store


My.Assembly1.dll


My.Assembly2.dll



My.Test.Assembly.exe.config


Symlink dependent assemblies

Symlink dependent assemblies into the same Nix store path as theexecutable:

References assemblies in other Nix store paths

Nix scans and detects runtime dependencies due to uniquestore paths

Cheap in terms of disk space

Works with private and strong-named assemblies

Allows efficient upgrading of a system (i.e. only replacingupdated parts)

Expensive in terms of memory usage (no sharing)

NTFS symlinks are only supported in Windows Vista+,Windows Server 2008+


Symlink dependent assemblies

/nix/store


My.Assembly1.dll


My.Assembly2.dll


My.Assembly1.dll

My.Assembly2.dll



Result

With this implementation we can build and run .NET softwarewith Nix, some caveats:

.NET framework and Visual Studio tools are not managed byNix

.NET framework has dependencies which must be in the GACand registry

Registry is not managed

Do not use of registry for configuration settings

Some UNIX properties are not enforced when invoking nativeWindows processes:

e.g. chrootMakes it slightly harder to guarantee purity


Possible applications

Use Nix for deployment of .NET applications

Offers the benefits of Nix (e.g. atomic upgrading) for .NETapplicationsAllows the deployment of complete systemsNo need to create your own installers

Hydra, http://nixos.org/hydra

Continuous integration & testing of .NET applicationsBuild and test components for multiple architectures

Disnix, http://nixos.org/disnix

Distributed deployment of .NET services into a network ofmachines


http://nixos.org/hydra

http://nixos.org/disnix

References

Nix, Nixpkgs, Hydra, NixOS, Disnix, http://nixos.org

Cygwin, http://www.cygwin.com

Demystifying the .NET Global Assembly Cache, http://www.codeproject.com/kb/dotnet/demystifygac.aspx

Common MSBuild Project Properties, http://msdn.microsoft.com/en-us/library/bb629394.aspx


http://nixos.org

http://www.cygwin.com

http://www.codeproject.com/kb/dotnet/demystifygac.aspx

http://www.codeproject.com/kb/dotnet/demystifygac.aspx

http://msdn.microsoft.com/en-us/library/bb629394.aspx

http://msdn.microsoft.com/en-us/library/bb629394.aspx

References

How the Runtime Locates Assemblies,http://msdn.microsoft.com/en-us/library/yx7xezcf%

28VS.71%29.aspx

Locating the Assembly through Codebases or Probing,http://msdn.microsoft.com/en-us/library/15hyw9x3%

28VS.71%29.aspx

Assembly Searching Sequence, http://msdn.microsoft.com/en-us/library/aa374224%28VS.85%29.aspx


http://msdn.microsoft.com/en-us/library/yx7xezcf%28VS.71%29.aspx

http://msdn.microsoft.com/en-us/library/yx7xezcf%28VS.71%29.aspx

http://msdn.microsoft.com/en-us/library/15hyw9x3%28VS.71%29.aspx

http://msdn.microsoft.com/en-us/library/15hyw9x3%28VS.71%29.aspx

http://msdn.microsoft.com/en-us/library/aa374224%28VS.85%29.aspx

http://msdn.microsoft.com/en-us/library/aa374224%28VS.85%29.aspx

Questions


Deploying .NET applications with the Nix package manager

Science

Transcript of Deploying .NET applications with the Nix package manager