xUnit Style Database Unit Testing ACCU London – 20th January 2011 Chris Oldwood gort@cix.co.uk
Presentation Outline • Database Development Process • The xUnit Testing Model • Test First Development • Continuous Integration/Toolchain • Pub
Legacy Database Development • Shared development environment • Only integration/system/stress tests • No automated testing • Only real data not test data • Referential Integrity – all or nothing • No automated build & deployment
Ideal Development Process • Isolation • Scaffolding • Automation
Example Testable Behaviours • Default constraint • Trigger to cascade a delete • Refactoring to a surrogate key
NUnit Test Model [TestFixture] public class ThingTests { [Test] public void Thing_DoesStuff_WhenAskedTo() { var input = ...; var expected = ...; var result = ...; Assert.That(result, Is.EqualTo(expected)); } }
NUnit Test Runner • Tests packaged into assemblies • Uses reflection to locate tests • In-memory to minimise residual effects • Output to UI/console
SQL Test Model create procedure test.Thing_DoesStuff_WhenAskedTo as declare @input varchar(100) set @input = ... declare @expected varchar(100) set @expected = ... declare @result varchar(100) select @result = ... exec test.AssertEqualString @expected, @result go
SQL Test Runner • Tests packaged into scripts (batches) • Uses system tables to locate tests • Uses transactions to minimise residual effects • Output to UI/console
SQL Asserts • Value comparisons (string, datetime, …) • Table/result set row count • Table/result set contents • Error handling (constraint violations)
Setup & Teardown • Per-Fixture (static data) • Per-Test (specific data) • Use helper procedures
Default Constraint Test create procedure test.AddingTask_SetsSubmitTime as declare @taskid int declare @submitTime datetime set @taskid = 1 insert into Task values(@taskid, ...) select @submitTime = t.SubmitTime from Task t where t.TaskId = @taskid exec test.AssertDateTimeNotNull @submitTime go
Trigger Test create procedure DeletingUser_DeletesUserSettings as ... set @userid = 1 insert into AppUser values(@userid, ...) insert into AppUserSettings values(@userid, ...) delete from AppUser where UserId = @userid select @rows = count(*) from AppUserSettings where UserId = @userid exec test.AssertRowCountEqual @rows, 0 go
Unique Key Test create procedure AddingDuplicateCustomer_RaisesError as ... insert into Customer values(‘duplicate’, ...) begin try insert into Customer values(‘duplicate’, ...) end try begin catch set @threw = 1 end catch exec test.ErrorRaised @threw go
Automation • Enables easy regression testing • Enables Continuous Integration • Performance can be variable
Test First Development • Start with a requirement • Write a failing test • Write production code • Test via the public interface
The Public Interface • Stored procedures • Views • Tables?
Implementation Details • Primary keys • Foreign keys • Indexes • Triggers • Check constraints • Default constraints
Deployment Testing Build version N+1 then run unit tests Build version N then patch to N+1 then run unit tests ==
Buy or Build? • Batch file, SQL scripts & SQLCMD • TSQLUnit & PL/Unit • Visual Studio • SQL Server/Oracle Express
“The Oldwood Thing” http://chrisoldwood.blogspot.com Chris Oldwood gort@cix.co.uk

xUnit Style Database Testing

  • 1.
    xUnit Style DatabaseUnit Testing ACCU London – 20th January 2011 Chris Oldwood gort@cix.co.uk
  • 2.
    Presentation Outline • DatabaseDevelopment Process • The xUnit Testing Model • Test First Development • Continuous Integration/Toolchain • Pub
  • 3.
    Legacy Database Development •Shared development environment • Only integration/system/stress tests • No automated testing • Only real data not test data • Referential Integrity – all or nothing • No automated build & deployment
  • 4.
    Ideal Development Process •Isolation • Scaffolding • Automation
  • 5.
    Example Testable Behaviours •Default constraint • Trigger to cascade a delete • Refactoring to a surrogate key
  • 6.
    NUnit Test Model [TestFixture] publicclass ThingTests { [Test] public void Thing_DoesStuff_WhenAskedTo() { var input = ...; var expected = ...; var result = ...; Assert.That(result, Is.EqualTo(expected)); } }
  • 7.
    NUnit Test Runner •Tests packaged into assemblies • Uses reflection to locate tests • In-memory to minimise residual effects • Output to UI/console
  • 8.
    SQL Test Model createprocedure test.Thing_DoesStuff_WhenAskedTo as declare @input varchar(100) set @input = ... declare @expected varchar(100) set @expected = ... declare @result varchar(100) select @result = ... exec test.AssertEqualString @expected, @result go
  • 9.
    SQL Test Runner •Tests packaged into scripts (batches) • Uses system tables to locate tests • Uses transactions to minimise residual effects • Output to UI/console
  • 10.
    SQL Asserts • Valuecomparisons (string, datetime, …) • Table/result set row count • Table/result set contents • Error handling (constraint violations)
  • 11.
    Setup & Teardown •Per-Fixture (static data) • Per-Test (specific data) • Use helper procedures
  • 12.
    Default Constraint Test createprocedure test.AddingTask_SetsSubmitTime as declare @taskid int declare @submitTime datetime set @taskid = 1 insert into Task values(@taskid, ...) select @submitTime = t.SubmitTime from Task t where t.TaskId = @taskid exec test.AssertDateTimeNotNull @submitTime go
  • 13.
    Trigger Test create procedureDeletingUser_DeletesUserSettings as ... set @userid = 1 insert into AppUser values(@userid, ...) insert into AppUserSettings values(@userid, ...) delete from AppUser where UserId = @userid select @rows = count(*) from AppUserSettings where UserId = @userid exec test.AssertRowCountEqual @rows, 0 go
  • 14.
    Unique Key Test createprocedure AddingDuplicateCustomer_RaisesError as ... insert into Customer values(‘duplicate’, ...) begin try insert into Customer values(‘duplicate’, ...) end try begin catch set @threw = 1 end catch exec test.ErrorRaised @threw go
  • 15.
    Automation • Enables easyregression testing • Enables Continuous Integration • Performance can be variable
  • 16.
    Test First Development •Start with a requirement • Write a failing test • Write production code • Test via the public interface
  • 17.
    The Public Interface •Stored procedures • Views • Tables?
  • 18.
    Implementation Details • Primarykeys • Foreign keys • Indexes • Triggers • Check constraints • Default constraints
  • 19.
    Deployment Testing Build versionN+1 then run unit tests Build version N then patch to N+1 then run unit tests ==
  • 20.
    Buy or Build? •Batch file, SQL scripts & SQLCMD • TSQLUnit & PL/Unit • Visual Studio • SQL Server/Oracle Express
  • 21.

Editor's Notes

  • #2 Who am I (1st talk, ACCU member, not DBA – app dev, predominately SQL server) Audience background (any DBAs, any SQL based, any familiarity with xUnit)
  • #3 Middle section is about xUnit, sandwiched between details of the infrastructure and side-effects What makes unit testing successful is as much the other stuff Questions at the end
  • #4 Michael Feathers’ definition of ‘legacy’ Shared environment (data volume growth – slower feedback cycle, conflicting changes causes downtime, organisation policy - no personal db for unit testing) Only integration/system/stress testing Testing done with real data (not requirements driven data – e.g. handling nulls) No RI means testing more important (static vs dynamic analogy) No automated deployment
  • #5 Isolation during implementation (sandbox, atomic commit of changes) Control over test data to verify behaviour Automated regression testing at unit level
  • #6 Default constraint to set the date (simple) Trigger to cascade a delete (simple) Refactoring natural key to surrogate (harder – unique constraint on old key) No real data required – test data is enough Note: these are implementation options – will come back to encapsulation later
  • #7 Fixture setup/teardown Test setup/teardown Test (illustrate the 3 A’s) Tests packaged together in assemblies
  • #9 Fixture setup/teardown Test setup/teardown Test (with Assert) Use Reflection to discover test cases Rollback to avoid per-test residual effects (not-perfect without nested transactions) Separate schema for tests
  • #16 Regression testing Multiple branches (db naming – physical dependencies) Personal database (isolated development) Performance – time to build from scratch and run test suite?
  • #17 Start with a requirement (‘date submitted’ set automatically) Writing failing test proc Consider implementation options (proc, constraint, trigger) Mocking how? Test only publicly detectable behaviour…
  • #18 Don’t test implementation (e.g. default => constraint, proc, trigger, etc.) Do you test accessors (e.g. selects, views) Use schemas & grant permissions
  • #20 From Scratch == Old + Patch (use Unit Tests) Data migration tests are separate tests
  • #22 Scott Ambler’s book Questions