Yesterday Anton Venter from New Zealand contacted me, because he read about my posts about optimization and stuff like that. And he was looking at different sorting and searching strategies he asked me how to measure execution time and memory comsumption of functions in a black box kind of way.

Timing

Well, the execution time part was answered in my previous post on HighPerformanceCounter and a more general article about measuring performance in C#.

To quickly sum those up those two posts, to time a function use something like:

PLAIN TEXT
C#:
  1. // run a sort first to get on the fly code gen out of the way
  2. sort.run(myarray);
  3. //now time it
  4. timer.start();
  5. sort.run(mysecondarray);
  6. timer.stop();
  7. console.writeline(timer.elapsedmilliseconds);

Memory Consumption

After browsing the documentation I found out that it is not too hard to measure the memory usage caused by functions and/or instantiation of and object using System.GC.GetTotalMemory. BUT that's of course "just" the amount allocated in managed memory. (So it does NOT give you an idea about how much JITing those methods would cause. Just a side-comment.)

PLAIN TEXT
C#:
  1. long m_memoryStart = 0;
  2. long m_memoryMakeList = 0;
  3.  
  4. Thread.MemoryBarrier();
  5. m_memoryStart = System.GC.GetTotalMemory(true);
  6.  
  7. List<int> testList1 = new List<int>();
  8. Thread.MemoryBarrier();
  9. m_memoryMakeList = System.GC.GetTotalMemory(true);
  10.  
  11. int count = testList1.Count;

For a more complete example on how to use that see my sample solution.

I inserted the Thread.MemoryBarrier() in order to absolutely prevent the compiler from reordering statements for optimizations. And the int count = testList1.Count; is there to keep testList1 alive until AFTER the second GetTotalMemory, which is abolutely necessary! Otherwise the difference between the two memory sizes will indeed be zero. :-)

At this point in encourage you to look at my sample. Especially notice how the first execution of Array.Sort does increase memory usage, the second does not.

Should I do timing or memory tests first?

There is no right order in which you should do these. It absolutely depends on what exactly you want to time or measure. But remember. It is best to separate the testing for memory consumption and the timing. Don't try to do both with one execution of a specific function.

Important: To get accurate readings especially for timing always compile your solution to "Release" and run the .exe outside of Visual Studio!

Thanks Anton for contacting me. It was fun thinking about your question. :-)

Memory Consumption Sample Solution

kick it on DotNetKicks.com