James Michael Hare

...hare-brained ideas from the realm of software development...
posts - 137 , comments - 1100 , trackbacks - 0

My Links

News

Welcome to my blog! I'm a Sr. Software Development Engineer in Seattle, WA. I've been doing C++/C#/Java development for over 18 years, but have definitely learned that there is always more to learn!

All thoughts and opinions expressed in my blog and my comments are my own and do not represent the thoughts of my employer.

Blogs I Read

MCC Logo MVP Logo

Follow BlkRabbitCoder on Twitter

Tag Cloud

Archives

Post Categories

C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

The C#/.NET Fundamentals series is geared towards examining fundamental concepts in using C# (and .NET in general) to produce effective solutions.

I wanted to attempt a brief post before the holidays, so I decided to quickly revisit part a post I wrote a few weeks back on The Generic Func Delegates, and in particular, the sidebar on using Func as a generator for unit testing.

At the time, I did not give that short sidebar the attention I really wanted, including showing the setup of the unit tests and discussing the performance impact (if any) of such a practice.  So this week I hope to rectify this by revisiting and enhancing the discussion.

Note: Of course, there are many frameworks that also can be used to accomplish these sort of tasks, the main point is to understand the underlying concepts and show more of the power of the Func delegate.

Dependency Injection Beyond Traditional Data Sources

Many people, when they think of Dependency Injection (DI) think of injecting an interface into a constructor or property of a class, so that the interface can be used to mock a traditional data source such as a database or web service.  While this is one of the most typical uses of DI, this is not the only type of data we can mock.

For instance, often times we write a class or method and don’t realize that we are tying it to a piece of data that is “moving”.  That is, a dependency on logic that varies depending on when or where the test is run.

For example, if we had a piece of code that used DateTime.Now to check for an amount of time to elapse, this would be problematic to unit test because the results of the unit tests could very well vary depending on when the tests are run.

To be sure, this is still data of some form, but many times people write code depending on things like DateTime.Now and then omit unit testing because they think it not easily possible, or instead attempt to write a test harness and vary their system clock, etc. in order to test the logic.

But, there is an easier way!  Instead of using these “moving” dependencies directly, we can have a generator create them for us, and be able to inject a generator that produces “non-moving” values for unit testing.

This really is no different than injecting mock data from a data source.  In essence the system’s clock is the data source in this example, so it makes sense we could use similar DI principles to test logic depending on it.

An Example Dependency on DateTime.Now

Let’s say you were building your own cache and you wanted each item in the cache to have a DateTime so you could keep track of whether the item has gotten stale (expired) or not.  This would be easy enough to write like:

   1: // responsible for maintaining an item of type T in the cache
   2: public sealed class CacheItem<T>
   3: {
   4:     // time the item was cached
   5:     public DateTime CachedTime { get; private set; }
   6:  
   7:     // the item cached
   8:     public T Value { get; private set; }
   9:  
  10:     // item is expired if older than 30 seconds
  11:     public bool IsExpired
  12:     {
  13:         get { return DateTime.Now - CachedTime > TimeSpan.FromSeconds(30.0); }
  14:     }
  15:  
  16:     // creates the new cached item, setting cached time to "current" time
  17:     public CacheItem(T value)
  18:     {
  19:         Value = value;
  20:         CachedTime = DateTime.Now;
  21:     }
  22: }

This is just a simplistic example of logic that is strongly dependent on a “moving” target, which makes writing unit tests for this code much more difficult.  You could, of course, have unit tests which use something like Thread.Sleep() to wait a given number of seconds and see if the item expires, but this really doesn’t give you much granularity or accuracy at all.

Defining a DateTime Generator

Instead, let’s look at how we can replace DateTime.Now with mock data.  So what, in essence is DateTime.Now?  It’s a property that returns a DateTime, right?  It just happens to be the current DateTime.

Thus we could easily set up a generator to return DateTime.Now and store it in a delegate.  Such a method would take no arguments, and return a DateTime thus it’s type can be represented by Func<DateTime>:

   1: // define as a lambda that takes nothign and returns DateTime.Now
   2: Func<DateTime> nowGenerator = () => DateTime.Now;
   3:  
   4: // The generator gives us current DateTime
   5: var now = nowGenerator();

Okay!  Now we have a delegate that returns us the current time.  If you’re scratching your head wondering why we just performed that cartwheel just to return DateTime.Now, consider this: we can now mock the results of this generator with a specific DateTime:

   1: // same delegate, but now mocked to return a particular DateTime
   2: nowGenerator = () => new DateTime(2011, 12, 31, 12, 0, 0);
   3:  
   4: // now is now 12/31/2011 12:00:00
   5: var now = nowGenerator();

So now with this concept we can unit test our logic with very precise time measurements and see if we get the expected results!

We could either build this generator into the CachedItem<T> class, or we could create a separate utility class for it.  For brevity we’ll just build it into our class:

   1: // responsible for maintaining an item of type T in the cache
   2: public sealed class CacheItem<T>
   3: {
   4:     // generator that returns the current time
   5:     private static Func<DateTime> _timeGenerator = () => DateTime.Now;
   6:  
   7:     // allows internal access to the time generator
   8:     internal static Func<DateTime> TimeGenerator
   9:     {
  10:         set { _timeGenerator = value; }
  11:     }
  12:  
  13:     // time the item was cached
  14:     public DateTime CachedTime { get; private set; }
  15:  
  16:     // the item cached
  17:     public T Value { get; private set; }
  18:  
  19:     // item is expired if older than 30 seconds
  20:     public bool IsExpired
  21:     {
  22:         get { return _timeGenerator() - CachedTime > TimeSpan.FromSeconds(30.0); }
  23:     }
  24:  
  25:     // creates the new cached item, setting cached time to "current" time
  26:     public CacheItem(T value)
  27:     {
  28:         Value = value;
  29:         CachedTime = _timeGenerator();
  30:     }
  31: }

I gave the generator internal access so that my unit test can change the generator, but to restrict it so it’s not completely open to public.  This is just one of many ways to do this, but you get the picture. 

If you do make your generator internal and want to also make it visible to your unit test assembly, you’ll need to make your unit test assembly a friend so it has access to internal members.  This is done with an assembly level attribute:

   1: // make the internals here visible to the friend test assemblies.
   2: [assembly: System.Runtime.CompilerServices.InternalsVisibleTo("YourTestAssemblyName")]

Now, we can write unit tests to exercise it!

Unit Testing with the Generator

So, we can start out with a simple test class to verify that the CachedTime property on construction is the “current” DateTime:

   1: [TestClass]
   2: public class CacheItemTest
   3: {
   4:     // verify on construction it has "current" date and time
   5:     [TestMethod]
   6:     public void HasCurrentTimeOnCreation()
   7:     {
   8:         var expected = new DateTime(2011, 12, 31, 12, 0, 0);
   9:         CacheItem<int>.TimeGenerator = () => expected;
  10:  
  11:         var target = new CacheItem<int>(13);
  12:  
  13:         Assert.AreEqual(expected, target.CachedTime);
  14:     }
  15:  
  16:     // ...
  17: }

That passes, so now let’s verify that times under 30 seconds are not expired, so let’s add a few more test methods.  First we’ll verify not expired at time of creation (remember generator will return same DateTime on IsExpired check as construction:

   1: // verify on construction it's not expired (no time elapsed)
   2: [TestMethod]
   3: public void NotExpiredOnCreation()
   4: {
   5:     var nowish = new DateTime(2011, 12, 31, 12, 0, 0);
   6:     CacheItem<int>.TimeGenerator = () => nowish;
   7:  
   8:     var target = new CacheItem<int>(13);
   9:  
  10:     Assert.IsFalse(target.IsExpired);
  11: }

Very good, that passes, so now let’s check at exactly 30 seconds, this should still not be expired since our logic is to expire when greater than 30 seconds:

   1: // verify on 30 seconds even it's not expired
   2: [TestMethod]
   3: public void NotExpiredAtExactlyThirtySeconds()
   4: {
   5:     var expected = new DateTime(2011, 12, 31, 12, 0, 0);
   6:     CacheItem<int>.TimeGenerator = () => expected;
   7:  
   8:     var target = new CacheItem<int>(13);
   9:  
  10:     // reset generator to give us the creation time + 30 seconds
  11:     CacheItem<int>.TimeGenerator = () => expected.AddSeconds(30.0);
  12:  
  13:     Assert.IsFalse(target.IsExpired);
  14: }

Great!  That passes too, now let’s see what happens if we’re even one millisecond over thirty seconds…

   1: // verify on 30 seconds even it's not expired
   2: [TestMethod]
   3: public void ExpiredAboveThirtySeconds()
   4: {
   5:     var expected = new DateTime(2011, 12, 31, 12, 0, 0);
   6:     CacheItem<int>.TimeGenerator = () => expected;
   7:  
   8:     var target = new CacheItem<int>(13);
   9:  
  10:     // reset generator to give us the creation time + 30.001 seconds
  11:     CacheItem<int>.TimeGenerator = () => expected
  12:         .AddSeconds(30.0).AddMilliseconds(1.0);
  13:  
  14:     Assert.IsTrue(target.IsExpired);
  15: }

And that passes too!  So now we have a solid unit test suite that will exercise this CacheItem<T> and give us the results we are expecting regardless of what time we run the test or how slow our system is!

Performance Implications of Generator Delegate

So, instead of calling DateTime.Now directly in the modified CachedItem<T> code, we are now calling a delegate which returns us DateTime.Now.  So how much overhead does that add? 

It turns out not much at all, there are cases where the optimizer may be able to inline certain delegate calls, of course, which helps the performance.  In particular, when I ran tests of both calling DateTime.Now and calling a delegate that returns DateTime.Now (storing the delegate as coded above, not re-creating the lambda every time) the times over 100 million iterations were nearly identical:

   1: Calling DateTime.Now Directly : 75,545 ms (0.000756 ms)
   2: Calling DateTime Generator    : 75,794 ms (0.000758 ms)

Note that there’s only a difference of 0.000002 ms per call using the delegate.  When I repeat this test, I get very similar results every time.  The time difference of the generator in all my tests was never higher than 0.000002 ms per call, and during some test runs was actually faster than the direct call to DateTime.Now.

So, even at the highest measurement of 0.000002 ms, this is a very negligible time difference and well worth the tradeoff to achieve better unit test coverage.  So unless your application is calling DateTime.Now continuously, you’d most likely never see any overall application time difference at all.

Summary

Among other things, the Func<TResult> can be used to create generators that make it easier to unit tests “moving” data dependencies.  Of course, there are other tools you can use (mocking frameworks, Moles, etc) but generators are a very easy and performant solution, and help demonstrate the power of the generic Func delegate family.

Technorati Tags: , , , , , , ,

Print | posted on Thursday, December 22, 2011 5:33 PM | Filed Under [ My Blog C# Software .NET Fundamentals ]

Feedback

Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

for thinks like DateTime.Now, have you considered 'Moles' (http://research.microsoft.com/en-us/projects/moles/)? That's the sort of thing that it's meant for, detouring .Net BCL methods, etc.
12/23/2011 4:54 AM | Yan Cui
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

@Yan:

Yes, I mentioned them very briefly in the summary. They are definitely one way to do it, though I think it important for people to know manual means as well so they aren't tied to the tools as much as the concepts.

But yes, moles and mocking frameworks are great aids for unit testing.

-jim
12/23/2011 7:06 AM | james michael hare
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

have you seen Greg Young's Simple.Testing ? https://github.com/gregoryyoung/Simple.Testing
12/23/2011 7:30 AM | James Tryand
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

@James: I had not, I'll check it out when I have a chance.
12/23/2011 7:47 AM | James Michael Hare
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

0.000002 ms is 4 clock cycles at 2GHz. I think you can call the two methods statistically equivalent :p
12/23/2011 11:55 AM | Andy
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

@Andy: Hah, thanks :-)
12/23/2011 1:08 PM | James Michael Hare
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

James, this is a good approach. I just want to point out that a delegate _is_ an object, the fact that C# makes us think otherwise is a deception. Remember:

"Object is a behavior in a context."

It is largely irrelevant (at OO level) weather object is implemented as a delegate (Func<blah>) or an instance of a class.

At the end of the date "get current time" is a behavior, and an object providing this behavior is a service. And the fact that this service only has one operation doesn't make it less of a service. But by virtue of being a single-operation-service it makes it a perfect candidate to be implemented not as a class (in C# terms) but as a delegate (in C# terms).

I agree with everything you've said. But wanted to note a different perspective on the same subject. For many (including myself) "delegate as a service" in C# context is a fresh idea, but if presented it in original OO terms then it makes perfect sense.
12/27/2011 9:00 AM | Roman Gavrilov
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

Nice Post! Have enjoyed the whole series.
12/27/2011 7:29 PM | Mike Mueller
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

Thanks a lot for this post. I like it!
12/28/2011 9:35 AM | essay
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

Nice article!

My one quibble is with using property injection, especially static property injection. I normally do constructor injection instead.
12/28/2011 12:54 PM | Bill Sorensen
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

@Bill: I often prefer constructor injection as well, though there's one or two places where I've used property injection also. But, in truth most of the time I'm happy to see people doing DI in general. It really makes writing well tested, maintainable code so much easier and safer, yet it still seems you'll come across people writing huge, monolithic, untestable classes.
12/29/2011 9:27 PM | James Michael Hare
Gravatar

# re: C#/.NET Fundamentals: Unit Testing with Func<TResult> Generators

Why you didn't use an internal DateTime property instead of a Func? Thanks.
7/16/2014 7:41 AM | JAck
Post A Comment
Title:
Name:
Email:
Comment:
Verification:
 

Powered by: