Depending on your problem .NET remoting may be an awesome life saver. If you have hundreds of drivers, many configurations and independent client applications it can be the module which holds all the pieces together if your design is more or less correct.

Because it worked so well, somebody came up with the idea to store user controls in those drivers. First it seems like a very cool idea “a driver can provide a piece of GUI abstracted behind an interface which can be shown to the advanced user so he can change options”.

If you think more about it however, it causes more problem than it actually solves (localization + customization, anybody?) and you may get burned by this constant application highjacking where you include anonymous user controls all the time. Anyway, that somebody was from “management” and so I had to implement a prototype.

Well, after a full day of prototyping I can tell you that it won’t work (well it does somehow, but shhh )

The problem we have is that all those user control classes extend indirectly from the type MarshalByRefObject (some kind of remotable proxy) like maybe half of the framework does. Now when you invoke such a control directly from your service two things may happen

• The user control gets invoked in the server’s app domain and fails to set up a message queue when the server is wrapped in a Windows service.
• The user control is able to set up a message queue but cannot run because its distributed logic needs to access private methods which does not work as that part is in the server’s app domain.

The ugly solution:

“Give me the type of the user control and invoke it in my app domain.”

which translates into following code (assuming a constructor with one parameter)

Object o = this.localInstanceOfUIType.GetConstructor(new Type[] { typeof( String) } ).Invoke( new Object[] { "Form1" } );
if( o is Control )
  this.Controls.Add( (Control)o);

Of ):

When you write an application for the .net framework it’s very common to separate the various parts into visual studio projects which will be represented in separate assemblies (dll or exes) after compiling them.

This is usually done like that so other applications can link those assemblies without needing to include a whole stack of drivers for a little piece of logic (you know, reusability, reliability, blablabla…) So you end up with two or more applications/dlls depending the same dll.

Nothing special until here.

Deployment

So what happens when the user, like in my case, puts the executable out of directory with the shared dlls?

Exactly, the program cannot be started and fails with a TypeLoadException because it could not load all its dependencies.

The .net way to solve this is by putting the shared dlls into the global assembly cache (GAC). However, this requires the dlls to be signed with a key and will cause a lot of work if you have sub-dependencies because every signed component needs all its references to be signed as well.

And if you sign all the dlls you change the signature of them and break any existing client/dll. That will cause a big redeployment just because somebody wasn’t happy with a simple link and wanted to put the whole exe to his desktop (and we have lots of dlls, we explicitly decided not to spam the GAC and had some other reasons as well)

But of course there will be a solution for this I thought.

Assembly specific configuration is normally defined by a .config file which has the same name like the assembly itself. E.g.

MyApp.exe
MyApp.exe.config

Loads the xml configuration and applies it to the dll/exe while it gets loaded. With “probing” you can specify further directories where the class loader should look for the required types if they could not be found in the expected location. Anyway, you cannot know where the shared directory is located when you write the config file but I let that for now.

<configuration>
<runtime>
<assemblybinding xmlns="urn:schemas-microsoft-com:asm.v1">
	<probing privatepath="dirA;dirB"></probing>
</assemblybinding>
</runtime>
</configuration>

So I referred to my shared dll directory but guess what, it didn’t work! msdn tells me that those probing paths must not be above the application’s root path. Ha! how nice..

Fortunately, I have a remoting server running to which my client apps need to connect anyway so I could implement a way cooler solution.

TypeLoadException

When the client application could not resolve the dependency it calls some handlers which let the developer handle such cases before quitting.
You can register yourself for such cases with

AppDomain.CurrentDomain.AssemblyResolve +=
	new ResolveEventHandler( DoResolve );

and handle it in a method with this signature

System.Reflection.Assembly DoResolve(
	object sender, ResolveEventArgs args)

My plan

When my event handler method gets called, I call my remoting server and ask
“Hey mate, I need an assembly called ‘args.Name’. You have such a thing?”
“Sure. It’s located in MyCompany\Shared\MyCompany.foo.bar.dll”

When I have the the path information I can do something like

System.Reflection.Assembly.LoadFile( path );

and my client will load that assembly and finds the needed type information in it.

Thus in the end it looks like

private Assembly DoResolve(
	object sender, ResolveEventArgs args )
{
  String path = myServer.GetAssemblyLocation( args.Name );
 
  if( path == null )
	return null;
  else
	return System.Reflection.Assembly.LoadFile( path );
}

In the end the only dll left in the GAC is a dll where the resolving is implemented. Thus the only thing the client must do is calling helperDllInGac.AddAutoResolving() in the first line of its main() and a handler for type loading errors in the current AppDomain will be registered and all dlls will be resolved dynamically.

With this mechanism you can move all executables to any place and they will find their dependencies as long as the service is running. This may not be necessary in your case, but if you have lots of little tools it gets very annoying when you need to sign and register almost every dll because you cannot be sure that executable will be in the same directory.

Instead of returning a path you could also provide a byte array representing the assembly in the servers resolving method and gain even more flexibility. However, depending on what you are doing this it not a real solution and may make things even more complicated and slow.

Notes

Stack

AddAutoResolving() must be called *before* any external types appear while the stack gets built.
E.g.

void main( String[] args )
{
	new Helper().AddAutoResolving();
	MyExternalType t;
	new MyGUIApp();
}

will fail because while the CLR builds the method stack it has no idea about the unknown type yet, calls the handlers but there aren’t any registered yet and the app fails anyway.

LoadFile

LoadFile is just for people who know what they are doing. If you need those anti-dll-hell-features like versioning and policy don’t use it.

About a month ago, I was asked by a company to fix a bug in their huge C# application. From time to time their application just crashed. No exceptions, no hints in the logfiles, nothing.
Of course the bug was not reproducable (”it tends to occur under heavy load”), which made things even more interesting.

So what you do when you have no idea and should have fixed it yesterday?

Get A General Idea

Of course I have no idea about the whole project. To get an overview it’s generally a good idea to search for subprojects/solutions. They often have meaningful names and are stored in directories which may indicate some kind of hierarchy. You can get an idea about the structuring in a short time.

After doing that, heh, I know there is plenty code around, but at least I somehow have an idea what is going on and more important, what the application actually does.

Debugging

I add a listener for unhandled AppDomain exceptions so their details get flushed out into a file. Then I start the App.

“Tataaa”, after half an hour I have a crash. Great! My exception handler did not produce a file though.
So I can assume the problem is not in the application’s C# code itself. Thus cluttering the code with silly debug statements won’t help here (That’s what the guy before was doing).
What I need is a better debugger! After some research I even found something useful.

I run the App with my new debugger and after a while the program fails again. Catting trough the output files and there was my problem: 0xc0000005 (STATUS_ACCESS_VIOLATION)

Ou ou…

Well, I don’t think that the App runs into a .NET bug. Access violation… Pointers and stuff? C comes into my mind. In C# there are no pointers though. If they have been used anyway, my exception handler would have gotten a proper exception and would have flushed out some details.

Maybe there are some COM calls somewhere which are causing problems?

It is possible to directly call unmanaged code from a managed C# App. The only thing you have to do is declaring the API methods you want to use with the “DllImport” attribute:

[DllImport("wininet.dll")]
 static extern IntPtr FindFirstUrlCacheGroup(
 	int dwFlags,
 	int dwFilter,
 	IntPtr lpSearchCondition,
 	int dwSearchCondition,
 	ref int lpGroupId,
 	IntPtr lpReserved);

A simple grep and I located the COM stuff. Some further research and this line turns up:

public int Write( byte[] buffer )
{
 
//...
 
if( WriteFile(
	this.fileHandle,
	buffer,
	buffer.Length - 1,
	out this.bytesWritten,
	ref this.writeNativeOverlapped) )
		return bytesWritten;
//...
 
return 0;
}

Whats wrong here?

Maybe you think it’s okay. Well, this is why this post got written.

Garbage Collection

The debugged program is written for the .NET framework. This means we have garbage collection support. This again makes everybody building and using objects and except the IDisposable guys nobody really cares when and how the objects get deleted because it just works.

Nothing new for you I guess.

The garbage collector does also other cool things.
For example it moves the memory used by the objects in the RAM so the objects stay together and thus prevent memory fragmentation.
We therefore also benefit from the locality effect and our fancy L1, L2 and Ln caches finally do something useful. That means the program gets faster, the user is happier, we have less performance issues and hence more time for doing other things.

Everything fantastic until here.

COM

The debugged program does not live just in that world though. By passing our empty buffer we do enter the dark and evil COM world.
Thats where the reason of my program crash lies.

What happens with the COM invoke?
The .NET framework copies some values (value types) and passes a pointer to the system for the others. It needs pointers so e.g. the operating system can fill the buffer. Of course I can also pass a copy of my buffer and the system can fill also that one, but well, I wont be able to get it back into my application anymore. By leaving a reference I can get the result when the system finished writing data into my buffer. Thats why we have to specify the size of the buffer. On the COM side the system just receives a pointer to a byte array and has no idea how big it is.

Anyway, guess what happens when the garbage collector in its infinite wisdom decides to move our buffer into another memory area?
On the .NET side everything will still be okay. Calls on the buffer will be mapped to the right place. The operating system however, still has its pointer from the last call and that pointer points to a memory location where there is no buffer anymore. The OS writes it’s result to some location in the heap. Maybe another data structure is using it or that location doesn’t even belong to us anymore.
However, it writes to that address and because of memory protection the call gets blocked, and somebody kill us. ha!

Holding memory

So I must prevent that the buffer gets moved from the garbage collector while a reference to it is held by the system. The .NET way preventing a buffer to be moved by the Garbage Collector is the fixed keyword.

byte[] buffer = new byte[512];
long size = buffer.Length;
unsafe
{
	long* ptrSize = &size;
	fixed( byte* ptrbuffer = buffer )
	{
		GetComputerName( ptrbuffer, ptrSize );
	}
}

That doesn’t work for me though. I have two threads. One for asynchronous writing and one for asynchronous reading. I cant use that fixed stuff because I would need to wait until I can free the buffer, that would make me synchronous again.

Fortunately there is another solution: Object-Pinning. From msdn:

… This prevents the garbage collector from moving the object and hence undermines the efficiency of the garbage collector. …

ye! By pinning an object you tell the garbage collector it should let its dirty fingers from your shiny objects. That fixed my problem:

• Creating a wrapper class for the buffer. Which holds the size and the buffer array itself.
• Pin those members in the constructor

   bufferHandle = GCHandle.Alloc(
  	localBuffer,
  	GCHandleType.Pinned)

  and unpin them when they get disposed.

   bufferHandle.Free();

• Save them locally before writing and reading so they don’t get disposed by the GC while an IO action is pending

Classical example of framework magic striking back

With the .Net 2.0 release in 2005 it was finally possible to use generics in c#. That dramatically reduced the code complexity and made programming in that language a little bit nicer again. The classes stayed flexible and elegant and despite the generics support was not even close to the magic it has in other languages it was better than nothing. I started using it immediately.

“wow… generics are so cool!”

A few weeks ago I wanted to provide my generic class, lets call it HandlePool<>, in a remoting application. The first problem I run into was that there is no way to access generics when the inner type is unknown.

HandlePool{} h = remotedObject.GetHandlePool();

wont work. Also

HandlePool{Object} h = remotedObject.GetHandlePool();

will fail on runtime while casting the object because of the underlying and fundamental problem that e.g. List<String> may not be casted into a List<Object>. Generics used in a context where you don’t know the types explicitly will cause huge problems because you cannot refer to a “generic base type”. The only possible way is accessing it over an interface (which must not be generic, otherwise the problem starts again).

“Well, generics has it’s drawbacks”

Because the generic class does just provide a handful operations I decided to wrap it by copying its interface to a normal object so I could access it with:

remotedObject.HandlePool.GetHandleByName();

However, when accessing the remoted object like that, the client receives a TypeLoadException.
Soon I noticed that the exception occurred while deserializing the remoted object. After stumbling around a bit, I discovered that during the object’s deserialization the client’s app domain found the type HandlePool<LoggingHandle> in the deserialization context but could not resolve the type because it was not defined in the app domain yet of course. How could it be?

By writing HandlePool<LoggingHandle> in my servers object I defined the new type HandlePool<LoggingHandle> on the fly in my server’s AppDomain. That was unknown to the clients because until then nobody defined that type there. However, I can’t do it in my client anyway because I don’t know yet which types I will receive from the server in future.

“Argh!”

As always in those “Damn! I’m fucked and have to rewrite everything” situations, I glanced at the reflection API. Until that day it always provided a solution for last-minute-design hacks.
So I hoped to be able to invoke the type somehow by generics and avoid the TypeLoadException

The old reflection API provided this way:

Type t; 
 
//this implicitly requires an empty constructor 
 
Object o = t.GetConstructor(
       new Type[]{} ).Invoke( new Object[]{} );

“Nice”

I tried but it still didn’t work. TypeLoadException… mhh…

Haide, it’s .Net, hacks are allowed and everywhere. So I continued trying…

After that I came up with the idea: building the type HandlePool<LoggingHandle> during runtime by reflection. Checking the reflection API.. and..
uh.. No way to specify a generic type. This was the first time I realized that maybe generics is not that nice if used across several modules and its unknown with which specific type you have to deal with.
But after a few minutes I found the MakeGenericType command. From msdn:

Substitutes the elements of an array of types for the type parameters of the current generic type definition and returns a Type object representing the resulting constructed type.

“Wohooo!”

So I need the type information of the type “inside” the generic and I will be able to add the correct type definition into my AppDomain which allows me to deserialize the remoted generic object.

“So how to get that type now?”

If I do something like remotedObject.HandlePool.GetGenericType(), I wont come very far because HandlePool has to be resolved first and that will result in an exception.

Thus I need a helper object with which I somehow can get the type by do something like

helper.GetGenericType()

And then I have the “inside” type in my app domain. The assembly where this type is defined is has to be around as well, but I ignore that now.

So when I have the generic type in my app domain, I can do something like

Type innerType = helper.GetGenericType(); 
 
//building generic type
Type newConcreteGenericType = Type.MakeGenericType(
       new Type[]{ innerType } ); 
 
//now deserialization works
remotedObject.GetHandlePool();

And it worked!

“Uhuu, reflection saved me another time”

Of course compared to static generic definitions this is incredible slow, but I used it just once in an init method and thus for me it was okay. (Note: I skipped the exception handling)

Wasn’t that easy. I will pay attention when using generics the next time. Sometimes they cause more problems than they actually solve.