Unit testing data

Speaking with a colleague the other day and he was talking about how he was constantly having to check the data feeds he was getting from another consultancy working on the same project.

Obviously schema validation and the like run transparently but what he was talking about was how correct the data was once it had actually been transformed into the right format for loading into his database.

We started talking about "known and expected values" and being able to write unit tests against them to prove that the upstream transformations are putting the right values into the right fields.

I think there is some mileage in this approach - obviously your type of project or data might not applicable to this (too volatile?), however if you receive regular data updates it's worth writing some simple sanity check unit tests to cut off at the legs any time wasted in tracking a bug down under the assumption that it can't be the data.

Comments

Configuration in .Net 2.0

11-Dec-2007 Update I've updated this post to fix the broken images and replaced them with inline text for the example xml and accompanying C# code. This post has been by far the most hit on this blog and along with the comments about the missing images I thought it was time to update it! Whilst recreating the examples below I zipped up the working source code and xml file and loaded this onto my Project Distributor site - please download it to get a full working custom configuration to play with! Just click on the CustomConfigExampleSource link on the right hand side, then the "Source" link to get the zip. We are in the process of converting our codebase to .Net 2.0. We've used Enterprise Library to great effect so decided that we should continue with this in the form of the Jan 2006 release which targets 2.0 and I've got the job of porting our Logging, Data Access etc wrappers to EntLib 2.0. ...And so far so good - the EntLib docs aren't bad and the migrati...

Walk-Thru: Using Wolfpack to automatically deploy and smoke test your system

First, some history... The advent of NuGet has revolutionised many many aspects of the .Net ecosystem; MyGet, Chocolatey & OctopusDeploy to name a few solutions building upon its success bring even more features to the table. I also spotted that NuGet could solve a problem I was having with my OSS System Monitoring software Wolfpack ; essentially this is a core application framework that uses plugins for extension ( Wolfpack Contrib ) but how to unify, standardise and streamline how these plugins are made available? NuGet to the rescue again - I wrapped the NuGet infrastructure (I deem NuGet to be so ubiquitous and stable that is has transcended into the software "infrastrucuture" hall of fame) with a new OSS project called Sidewinder . Sidewinder allows me to wrap all my little extension and plugins in NuGet packages and deploy them directly from the Wolfpack application - it even allows me to issue a new version of Wolfpack and have Wolfpack update itself, sweet huh...

Announcing FluentGeoApi - a C# wrapper to GeoAPI.com

I'm pleased to make public the fruits of my late nights.... FluentGeoApi ! I previously mentioned that I am working on a private/personal project - well it's got an element of geolocation to it and after a bit of internet research I found GeoAPI.com . In order to interact with GeoAPI I decided to write a fluent style wrapper over the top of it and I've just released v1.0.0.0, a .Net 3.5 C# library to take the pain out of making REST calls and dealing with the GeoJson wire format used by GeoAPI. It's not 100% coverage of the API but I plan on getting there ASAP...however I've implemented Create/Modify/Delete a user entity, Simple and Keyword Search which is enough to release it. If you are working with geolocation data/features in your .Net application I would check out GeoAPI.com - I've been really impressed with what if offers (and if you hit the api < 20,000 times a day it won't cost you a penny!) - hopefully you'll also consider using FluentGe...

James Simmonds

Search This Blog