Brian On Software

This is the second part of a series dedicated to Continuous Integration with .NET.  In the first part we managed to install and configure a source control repository with Subversion.  Now we’re going to learn how to install and configure TeamCity as our CI build server.  At the moment of this writing, version 3.1 is the latest release version of TeamCity.  I’ll make sure to update the present content to reflect new changes in installing and configuring future releases of the product if necessary.

I recommend that TeamCity be installed in a separate (physical or virtual) machine than the one where Subversion is installed because if you need to update TeamCity, to apply a service pack to the operating system or to upgrade the .NET Framework, the machine will have to be restarted and this could cause some major productivity downtime to the team because the repository will be unavailable until the machine is back online.

A last note to consider before proceeding is that I’m also going to show you how to setup MySQL to act as the storage engine for TeamCity.  You’ll be happy to know that configuring MySQL as the storage engine for TeamCity is much easier and simple to do in TeamCity v3.1 than in its previous versions.

If you’re already using a previous version of TeamCity and you want to upgrade to the current version, I strongly recommend you to read JetBrains’ article on “Migrating to an External Database” to know how to properly backup your current data before upgrading to a newer version.

=> Step1. Make sure that you have installed the Microsoft .NET SDK with the latest service packs in the machine where TeamCity will be installed.  If necessary, install and configure Internet Information Services (IIS) too.

=> Step 2. Download TeamCity from JetBrains’ website.  You can read this post for more information between both the Professional Edition and the Enterprise Edition.

=> Step 3. Install TeamCity as you would install any typical Windows application (NEXT->NEXT->…->FINISH).  Here are some tips that might assist you during the installation:

• Make sure you install both the Build Agent Windows Service and the TeamCity Server Windows Service (as shown in figure 1).  The build agent is responsible to take a build script and running the necessary steps to perform a build.  It’ll send the results back to the server.  The server is going to notify you of the build results.  As described in this series’ preface, you can install multiple agents for one TeamCity server, but depending on the license you have, you might be limited to a number of agents a server can be attached to.  Refer to TeamCity’s documentation for more information.

Figure 1. Install both the Build Agent Windows Service and the Server Windows Service

• Take note where the build server configuration has been installed (by default it is located under C:\Documents and Settings\Administrator\.BuildServer),  I’ll refer to this location as the TeamCity data directory.
• Make sure you specify a server port that isn’t in conflict with another port.  For instance, IIS uses port 80 by default, so you might not be able to use the same one unless you shut down IIS or change its default port.  I tend to use port 8080 for TeamCity server (as shown in Figure 2).

Figure 2. Specify an available port for TeamCity server

• Feel free to take a snapshot of the next window (”Configure Build Agent Properties”) and save it somewhere for information purposes.  After clicking on the ‘Save’ button, the following window (”TeamCity Build Agent Properties”) will popup.  Take note of the location where the build agent’s properties file will be saved at for future reference.

Figure 3. Build Agent’s Configuration window

• Make sure you tell the installer to start the build agent and build server Windows services:

Figure 4. Start both the Build Agent service and the TeamCity Server service

=> Step 4. Create a new system environment variable.  The name of the variable should be TEAMCITY_HOME_DIR and its value should be the location where you have installed TeamCity (i.e., c:\TeamCity).  Restart the machine at this point.

=> Step 5. Fire up a browser and connect to TeamCity (i.e., http://localhost:8080).  Accept the EULA at the bottom of the screen and create a new administrator account.  If you click on the Administration tab (upper right side of the screen), you’ll notice the following message:

=> Step 6. In this step and the next one, we’re actually going to install and configure MySQL as a storage engine for TeamCity.  The configuration settings should be appropriate for most situations.  Feel free to configure MySQL as needed for your own business/technical purposes.

=> Step 6.1.  Download MySQL Community Edition from the following URL: http://dev.mysql.com/downloads/mysql/?rz=gdl#downloads (you don’t have to sign up in order to download MySQL.  There’s actually a link underneath the registration page that says “No thanks, just take me to the downloads!”…They’re so polite.)  In this article, I’m assuming you’re downloading the MySQL Essentials 5.0.51a Win32 installer.  The following steps describe how I’ve configured MySQL to act as a storage engine for TeamCity.

=> Step 6.2. Click ‘Next’ on the ‘Welcome’ window

=> Step 6.3. Make sure you selected ‘Custom‘ as the setup type, then click on ‘Next’.

=> Step 6.4. Verify that both the MySQL Server and Client Programs will be installed.  Click on ‘Next’ to continue.

=> Step 6.5. Click on ‘Install’ to continue with the installation.  Feel free to surf the web (or my blog *hint*) while it’s installing…

=> Step 6.6. After finishing installing, you’ll see a bluish window with the MySQL/dolphin logo.  Just click on ‘Next’ until you see the “Configure the MySQL Server now” window.  Click on ‘Finish’, then on ‘Next’ to begin the configuration.

=> Step 6.7. Select the ‘Detailed Configuration‘ as the configuration type, then click on ‘Next’.

=> Step 6.8. Select the ‘Server Machine‘ as the server type, then click on ‘Next’.

=> Step 6.9. Select the ‘Transactional Database Only‘ as the database usage, then click on ‘Next’.

=> Step 6.10. Select a drive and directory where the InnoDB tablespace should be placed, then click on ‘Next’.  In my case, I simply decided to install it under ‘c:\MySQL InnoDB Datafiles\‘.

=> Step 6.11. Select the ‘Decision Support (DSS)/OLAP‘ as the concurrent connections setting, then click on ‘Next’.

=> Step 6.12. Select the ‘Enable TCP/IP Networking‘ option and specify the port number.  I kept the default 3306 port.  Make sure to tick the “Add firewall exception for this port“ checkbox and also select the ‘Enable Strict Mode‘ as the SQL mode.  Then click on ‘Next’.

=> Step 6.13. Select the ‘Best Support For Multilingualism‘ as the default character set (UTF8), then click on ‘Next’.

=> Step 6.14. Select ‘Install As Windows Service‘, provide a service name (I left the one provided by default), select ‘Launch the MySQL Server automatically‘ and also select ‘Include Bin Directory in Windows PATH‘.  Click on ‘Next’ to continue.

=> Step 6.15. Type a password for the root account.  Don’t allow root access from a remote machine on this account.  You don’t need to create an anonymous account.  Remember your password as you will need to enter it later.  Click on ‘Next’ to continue.

=> Step 6.16. Click on ‘Execute’ to start the configuration.  Once the configuration has completed, click on ‘Finish’.

=> Step 6.17. Take a break…seriously…surf the Web, drink some coffee, go to the bathroom…We’re almost done.  The next steps are about configuring MySQL to act as the storage engine for TeamCity.

=> Step 7. Ok, now we’re done with installing and configuring MySQL.  It’s time to configure the MySQL as the storage engine for TeamCity.

=> Step 7.1. Shut down (stop) the TeamCity server Windows service.  You can do this via the Service Controller command or the Windows Services window.  The name of the service should be TeamCity.

=>  Step 7.2. Download the MySQL JDBC driver (ZIP) from the following URL: http://dev.mysql.com/downloads/connector/j/ and extract the JAR file (mysql-connector-java-5.1.6-bin.jar) to <TeamCity home>\webapps\ROOT\WEB-INF\lib.  In my case, this would be C:\TeamCity\webapps\ROOT\WEB-INF\lib.

=> Step 7.3. Now we’re going to create a database that will be used by TeamCity for storing various data (feel free to refer to the following URL to set up an external database with MySQL: http://www.jetbrains.net/confluence/display/TCD3/Setting+up+an+External+Database#SettingupanExternalDatabase-settingUpMySql)

=> Step 7.4. Fire up a MySQL Command Line Client (Start menu -> All Programs -> MySQL -> MySQL Server 5.0), type in the root password from Step 6.15 and hit Enter to continue.

=> Step 7.5. Type the following command to create the database:

CREATE DATABASE TeamCityDB DEFAULT CHARACTER SET utf8;

and then hit Enter to execute the command.  You have now created a database named TeamCityDB which will be used by TeamCity. DO NOT CREATE ANY TABLES AT THIS POINT!

=> Step 7.6. If you don’t want the root account to control the TeamCityDB database, feel free to create a user and grant that user the necessary permissions to modify the database.  See MySQL’s documentation for more information on how to create database users.

=> Step 7.7. In the TeamCity data directory (i.e., C:\Documents and Settings\Brian\.BuildServer\config) rename database.mysql.properties file to database.properties and specify the required settings in this file:

connectionUrl=jdbc:mysql://<host>/<database name>
connectionProperties.user=<user>
connectionProperties.password=<password>

In our case, the <host> is the name of the machine where MySQL is installed, i.e. localhost.  Since MySQL will only be used by TeamCity, I didn’t bother creating a dedicated user.  Therefore, I’m using the root’s account as the database user for TeamCityDB.  This might not be appropriate for your situation.  Figure 5 shows you what I’ve entered for that section:

Figure 5. Fill in the necessary information for the MySQL database (database.properties)

=> Step 7.8. Edit the <TeamCity home>\bin\dbMigration.properties file, entering correct values for the source and target database. Ensure all the properties are supplied and only the necessary lines are uncommented (single-line comments are designated by # character).  The following figure shows you what needs to be modified in the red boxes (the second red box should have the same values as entered in Step 7.7):

Figure 6. Fill in the necessary data in the dbMigration.properties file

=> Step 7.9. Cool! Now we’re ready to run the migration tool.  FYI, configuring MySQL to act as the storage engine for TeamCity is so much easier to do in this version of TeamCity (v3.1) than the previous versions.  All you have to do is execute the following command in a DOS command prompt (make sure the current directory is <TeamCity home>\bin) and the tool will take care of the rest…I love it!

c:\TeamCity\bin\migrateDB.bat migrate

You’ll have to specify the TeamCity data directory.  If you hit Enter, the tool will use the shown directory as its value. The migration tool really simplifies our lives for configuring MySQL with TeamCity.  Excellent job, JetBrains!  The following figure shows you the output of the operation.

Figure 7. This is the output generated after running the migration tool

=> Step 7.10. You’re done!  Start the TeamCity server Windows service and log in your TeamCity account.

In this article, I have presented the necessary steps to install TeamCity as a CI build server and install/configure MySQL to act as the storage engine for TeamCity.  In the next, we’ll actually get our hands dirty and integrate a .NET application with a few build configurations.  Stay tuned!

By the way, if your curious to know, the output generated in Figure 7 shows you ALL the tables created in the TeamCityDB database.  For instance, if you do a SELECT * FROM users; on the TeamCityDB, you will get all the TeamCity user accounts with their passwords in clear text (as shown in Figure 8)!  This is a major security flaw since passwords should be encrypted in some way…else what’s the use?  I hope that JetBrains will fix this issue someday.

Figure 8. You can clearly see the Administrator’s password ‘admin’ in clear text in the ‘users’ table

Wow! Continuous Integration sure seems to be the flavour of the month this year!  I just finished reading a very insightful article on setting up a CI environment by Jay Flowers, creator of CI Factory, on the March 2008 edition of MSDN Magazine.  He took good care to discuss the basics practices and key principles behind CI independently of the build server.  Without going too much into the details of the article, Jay discusses how to setup a CI environment with MSBuild, mbUnit, WatiN and CC.NET, as well as his own creation: CI Factory. 

I’ll be using a different approach for my series on Continuous Integration, but needless to say that the same principles Jay wrote about in the article will be reiterated in the following weeks.  No doubt that Jay’s article will give you more than a few key ideas to consider for your actual or next CI project.

<UPDATE>2008-03-20: As noted by Stefan, we don’t need to check the “Use ‘_svn’ instead of ‘.svn’ directories” checkbox of TortoiseSVN’s settings if you’re using Visual Studio 2005 or Visual Studio 2008.</UPDATE>

In this section, I’m going to describe how to properly install and configure Subversion in Windows.  At the moment of writing this post, the latest version of Subversion is 1.4.6.  I’ll do my best to update this post to reflect the latest version of the tool in the future.

My strategy for Subversion is to install it in a dedicated machine.  It’s recommended for Subversion to be independent of the build integration server, so that if you must update and restart the integration server, the developers will still be able to access the repository.

=> Step 1. Make sure that the host operating system where you’re going to install Subversion is running at a minimum Windows XP SP2, Windows Server 2003 or Windows Vista.

=> Step 2. Download the Subversion installation file svn-1.4.x-setup.exe (Win32 binaries) or svn-1.4.x-setup.zip file from the following URL: http://subversion.tigris.org/servlets/ProjectDocumentList?folderID=91. The Win32 binaries executable file is your typical Windows software installation process (Next->Next->Finish).  If you’re downloading the ZIP file, just extract its content somewhere in your drive, such as in c:\Program Files\Subversion.  At this point, you’re just installing Subversion.  You’ll create a source repository at step 5.

=> Step 3. Verify that the path to the Subversion bin directory is present in the %PATH% environment.  If that’s not the case or if you have extracted the executable files from the ZIP file, make sure to add it manually before continuing.  For instance, if you have installed Subversion under c:\Program File\Subversion, then make sure to append the c:\Program Files\Subversion\bin path to the %PATH% environment.  Do not reboot the system yet, because there’s still another environment variable to add in the next step.

=> Step 4. Add the SVN_EDITOR environment variable.  This is required so that Subversion can launch an external text editor whenever necessary in order to edit some data associated to a repository.  The value for this environment variable must be the full path to your favorite text editor.  For example, if Notepad is your primary text editor (as Seinfeld would say, not that there’s anything wrong with that), the SVN_EDITOR environment variable should have its value set to c:\windows\notepad.exe.  You may now reboot the machine if necessary.

=> Step 5. Create a source repository for your development projects.  I recommend starting with one repository for all your projects and then scale or change your strategy as you get more comfortable with the tool.  The following steps will guide you in creating a Subversion repository.

=> Step 5.1. Launch a DOS command prompt window.

=> Step 5.2.  Type the following command which will create a new repository using FSFS as the default file system of the repository (you can read more about that file system in Subversion’s documentation, which is by far the best documentation for an open source project):

svnadmin create "<repository path>" 

where <repository path> is the location where you would like to create the repository.  For instance, if I want to create the repository in d:\repository, then I will type the following command:

svnadmin create "d:\repository"

The svnadmin command tool will create the directory if it doesn’t exist. The quotes are necessary if the pathname contains spaces.

=> Step 5.3.  Go to the repository’s path and edit the svnserve.conf file under the conf directory.

=> Step 5.4.  Uncomment the following lines under the [general] section of the file to setup the default security permissions (you can also read more about Subversion’s security permissions and mechanisms in its documentation)

[general]
anon-access=read
auth-access=write
password-db=passwd

=> Step 5.5.  Save and close the serve.conf file when done editing the file.

=> Step 5.6.  Under the same conf directory, edit the passwd file and modify it to include the user accounts and passwords for each team member who will have access to the repository.  The format is that of a key-value pair.  The key corresponds to the username, and the value corresponds to the user’s password.  The password is in clear text which pretty much sucks, but if you really need password discretion, I suggest you look into a tool like this one.

[users]
alice=alice-passwd
bob=bob-passwd
charlie=charlie-passwd

IMPORTANT: Also, add a TeamCity account so that it’ll have access to the repository.

=> Step 5.7.  Verify the repository’s integrity by typing the following command:

svnadmin verify "<repository path>".  In our case, we'll type svnadmin verify "d:\repository"

=> Step 6. Create a Windows service to automatically start the Subversion server (svnserve.exe) whenever the host operating system starts.  Type the following command to create the Windows service:

sc create svnserve binpath= "\"c:\Program Files\Subversion\bin\svnserve.exe\" --service -r <repository path>" displayname= "Subversion Server" depend= Tcpip start= auto

In our case, <repository path> will be the repository path created in step 5.2 (d:\repository).  Be sure not to forget the quotes if your repository pathname has some spaces.  It is also important to note that there is a space between the SC command property and its value!

=> Step 7. Start the Windows service by executing the following command:

net start svnserve

In the same manner, you may stop the service by executing the following command:

net stop svnserve

Before we continue with installing TeamCity, it is important to start planning your backup strategy for the repository.  I’ll leave that strategy to you.  The following command can be executed as a CRON job or as a Windows scheduled task to backup your repository:

svnadmin hotcopy "<repository path>" "<target path>" --clean-logs

Then backup the <target path> somewhere place in case of an emergency.

Finally, each developer should install a Subversion client in order to easily access the repository within Windows Explorer and Visual Studio 2005/2008.  I recommend using TortoiseSVN because it integrates quite nicely with Windows Explorer and its documentation is very excellent.  As for Visual Studio, I recommend using either AnkhSVN (open source) or VisualSVN (commercial).  Concerning TortoiseSVN, make sure to check the checkbox “Use ‘_svn’ instead of ‘.svn’ directories” if you’re using Visual Studio .NET 2002/2003.  You’ll need to restart the machine to apply the changes.  The reason we need to check that option is because Visual Studio .NET 2002/2003 seems to have some issues with folders that start with a period.  This issue doesn’t apply if you’re using Visual Studio 2005/2008.

In the course of the next few weeks, I’ll be writing a series of articles on how to set up a Continuous Integration (CI) environment for .NET development using TeamCity as the build integration server (with MySQL as the backend database), Subversion as the version-control server, MSBuild as the software build engine, NUnit as the unit testing framework, NCover as the tests coverage tool, NDepend as the static code analysis tool, and other tools that might give us important feedback on a build.  I hope that this series will be helpful in better managing and communicating your open source projects, academic projects and commercial projects.  Even though that I’m targeting development with .NET, the principles can faithfully be applied in a CI environment which uses different tools than the ones mentioned above.

You might be wondering why I won’t be using CruiseControl.NET as the build integration server.  The answer is pretty simple.  First of all, there are many articles that already describe very well how to install, configure and properly use CruiseControl.NET as a build integration server.  Second of all, I believe that CruiseControl.NET is dead in terms of ongoing development.  I’m not saying that CC.NET isn’t good at all, on the contrary, but the fact that it wasn’t updated in almost a year tells me that it’s probably stalling.  At the present moment, TeamCity is by far the best build integration server out there in terms of features, extensibility, performance, portability and scalability.  Thanks to its open discussion forums, newsgroups and accessible JIRA tracking dashboard, JetBrains is in a good position to redefine the principles behind CI and, most importantly, to further innovate in the CI space.  Another thing you might be wondering is how much setting up this kind of CI environment is going to cost you.  That answer is also very simple.  It’ll cost you a little learning curve to know how to properly use the tools and also some time in properly configuring the tools.  In other words, setting this kind of CI environment will cost you absolutely nothing in terms of financial expenses, except maybe buying a machine to act as a remote server to run your builds.  In fact, since the release of TeamCity 3.1, the Professional Edition is free to use (refer to this post for more information).  I’ll go more into the details of each tool in the later posts in order to separate the concerns and keep this series as loosely coupled as possible so that you can refer to each specific part individually.  I guess the principles of object-oriented programming follow us in everything we do in life…

I will not cover the whole concept, principles and best practices of Continuous Integration because other people have already done an excellent job in doing that.  I highly recommend you to look at Martin Fowler’s article on Continuous Integration, as well as to get a hold of Paul Duvall’s excellent book on Continuous Integration (from the Martin Fowler Signature Series of Addison-Wesley) to further understand some key patterns, best practices and strategies to successfully deploy and use a CI environment.  I read the entire book before setting up a CI environment at a client side, and I’m glad I did because the material that Paul covers in his book are real-world scenarios that most people face when setting up this kind of an ecosystem inside an organization.

On a last note, it is important to know that I’m setting up this kind of infrastructure in a Windows environment.  Written in Java, TeamCity can be installed and executed on any platform that supports the JVM, such as Mac OS X, Linux, Windows, etc.  Subversion can also be installed and executed under Mac OS X, Linux and Windows.  As for the supporting build tools like NUnit and NCover, these only work under Windows.  I really hope that other people will write about setting up this kind of environment in other platforms than Windows to help out those that use different systems.  I’ll also update the information related to this series to reflect changes to the way we use the tools, new strategies to consider and the feedback I’ll receive from these articles.  Another point to consider is that you might want to try out this kind of CI setup in a virtual environment with various virtual machines.  The following approach might give you a general idea how you could implement a solid CI environment:

Here’s an early overview of the activities that will be covered in this series:

• Part I: Beginning the CI Journey with Subversion
• Part II: Setting Up Our Build Server With TeamCity

As I was reading a couple of ObjectMentor’s blog posts, one of them caught my attention.  It was Tim Ottinger’s post on “TDD on Three Index Cards“.  He recalls how someone missed part of a class he was teaching on TDD and told that individual the fundamental essence of TDD on three index cards in less than 15 minutes.  After reading that, I got myself thinking “Hhhmm…not a bad idea! I wonder if it’s physically possible?“. 

Ten minutes later, I’m at Staples buying some index cards and a couple of markers (for pictures purposes…of course…).  Here’s the final product which every credit goes to Tim for this very simple, practical and useful idea.  Kudos to my wife for taking the pictures.

Here’s a recap of the main ideas behind TDD (full credits goes to Tim):

Card #1. Uncle Bob’s Three Laws

• Write no production code except to pass a failing test.
• Write only enough of a test to demonstrate a failure.
• Write only enough production code to pass the test.

Â

Card #2: FIRST Principles

• Fast: Mind-numbingly fast, as in hundreds or thousands per second.
• Isolated: The test isolates a fault clearly.
• Repeatable: I can run it repeatedly and it will pass or fail the same way each time.
• Self-verifying: The Test is unambiguously pass-fail.
• Timely: Produced in lockstep with tiny code changes.

Card #3: Core of TDD

• Red: test fails
• Green: test passes
• Refactor: clean code and tests

Last March, Mike Cohn gave a tech talk at Google about Agile estimation, Scrum and the Planning Poker. In this 90 minutes presentation, Mike shares his insights and experience on software estimation using an Agile perspective and by providing many concrete examples to better understand the concepts. He also provides useful information about how an organization can improve its quality and estimation by embracing a couple of Scrum activities such as the Daily Scrum and the Planning Poker. Furthermore, he informs the audience that there is actually a website dedicated to provide a free and useful tool to help organizations engage with the Planning Poker. The tool is convenient for distributed teams as well and suited for organizations who lack a formal system to track and manage their estimation. I never heard of this tool prior to watching this tutorial. I’m sure some managers at work will find it pretty useful too. I hope.

The idea behind planning poker is simple. Individual stories are presented for estimation. After a period of discussion, each participant chooses from his own deck the numbered card that represents his estimate of how much work is involved in the story under discussion. All estimates are kept private until each participant has chosen a card. At that time, all estimates are revealed and discussion can begin again. Planning poker is the best way we’ve found for agile teams to estimate. It’s primary downside has been that all participants had to be sitting in the same room with a physical deck of cards in their hands.

The whole tutorial is available on YouTube as two separate videos (Part 1 and Part 2).

The PowerPoint slides to the videos can be found here.

I highly recommend to any developer, lead, manager and executive to watch this very educating and entertaining video which I’m sure will help you to better estimate your next work activities and products. My suggestion is to watch it with some co-workers during lunch…with pizzas.

Since the past year, I’ve been practicing the art of writing unit tests to help me drive a design before writing any code to support it. It came to my surprise that this practice, whether known as Test-Driven Development(TDD) or Behavior-Driven Development(BDD), has been used and preached by many leaders of our field for quite some time. In fact, Martin Fowler, a thought leader in software development for the enterprise and in his excellent book about refactoring, "Refactoring: Improving the Design of Existing Code", which was published in 1999 (almost ten years already), mentions the practice of testing your design with unit tests before writing your business or logic code. The fourth chapter of that book, entitled "Building Tests", is dedicated to promote this activity in your software development process. And guess what? It doesn’t even mention the "test-driven" word! Is this an easy route to follow or even to diverge from our comfortable and traditional way of writing code before even fully understanding a requirement?

"Of course, it is not so easy to persuade others to follow this route. Writing the tests is a lot of extra code to write. Unless you have actually experienced the way it speeds programming, self-testing does not seem to make sense. This is not helped by the fact that many people have never learned to write tests or even to think about tests. […] In fact, one of the most useful times to write tests is before you start programming. When you need to add a feature, begin by writing the test. This isn’t as backward as it sounds. By writing the test you are asking yourself what needs to be done to add the function." – Martin Fowler, Chapter 4 "Building Tests". An excerpt of "Refactoring: Improving the Design of Existing Code".

In my opinion and experience, the reason why TDD works is because it helps us to build correctly something that we don’t fully comprehend. This is especially true if the project domain is new to us, if we don’t have all the requirements set before us, if our experience in designing and thinking about software isn’t as broad as other experts, etc. Of course, we might have a general, high-level idea of what the application must do or should do, yet we don’t fully grasp the mechanics behind it. By mechanics, I also throw in the vision the software is trying to realize. Knowing what it must do or should do is a good starting point for laying a foundation on which the rest of the application can be built. Writing or thinking about unit testing helps to understand what the foundation should look like, and feel like, from the user’s point of view of our code. I tend to think it as paint artist drawing a painting. Most of the time, he doesn’t know immediately how to draw something very specific (a shape, a concept, an angle, a shade, etc.) and which exact color will be used to paint it. But, he sure knows what he wants to draw or what needs to be painted. It’s the same thing in music, in martial arts, in cooking and yes, even in software development. It’s the constant battle that goes on in front of a whiteboard, a sheet of paper or a CASE tool when designing and coding. Have you guessed it? The answer is pretty simple. It’s the battle between the what and the how. "What should this functionality accomplish versus how it should be implemented?" "What should this method expect versus how it should handle the expectation?" As a side note, I once heard that Linus Torvalds, the author of the Linux kernel, is an exemption to this reality. Apparently, he sees the code that needs to be written in his head and writes it; and if you tried to reverse engineer his code, it’ll produce a very solid design. Because there aren’t many Linus Torvalds within our organizations, we can look to useful, simple and pragmatic practices. Writing unit tests upfront helps me to drive a design that’s not entirely understood, but at a minimum, partially understood. It also helps me to bring forth changes to my design. In software development, these kinds of changes are known as refactoring. I highly recommend Martin Fowler’s book on refactoring to learn more about the patterns for effective refactoring. So what did the test-first approach to software development taught me so far?

• It taught me that refactoring my work can be done in a much easier and safer way
• It taught me that I would understand the design, therefore the application, much better if I see it through the eyes of someone who will use my code
• It taught me that there is an ongoing battle in my head about knowing WHAT my class should do and HOW the code should do it. This sort of "mental war" is a good one to have, since it forces me to carefully think about various OO principles (encapsulation, relationships, inheritance, etc.) when designing and coding
• It taught me that the old, traditional way of developing software (Big Design And Requirements Upfront) doesn’t work for every projects nowadays
• The concept of testing is not the same as "verification", but rather points to a way of thinking about a solution (how can you verify something that doesn’t still exist?). And that solution is not absolute. It might be the seed to another and better solution. From my experience, this pattern is recursive.

In my next post, I’ll write about some tools and techniques which I use when writing unit tests up front, and which I hope will help you to better use TDD for your advantage. Until then, feel free to share your thoughts on the topic!

Recommended articles for this topic:

• Dave Astels wrote a nice article about Behavior-Driven Design and how it compares to TDD. I think the principles behind his article have more weight than arguing about the naming of such practices. Check out his article about A New Look At Test-Driven Development.
• Thought leader, Scott Ambler, has written a ton of articles about all things related to agile development and TDD. Two of his papers on TDD, Introduction to Test Driven Design and Quality in an Agile World, are highly recommended to learn more about this practice.
• The Way of Testivus is a cool piece of writing from Alberto Savoia in which he shares some funny and smart remarks about unit testing in general.
• TestDriven is a useful site about test-driven development in general (.NET, Java, etc.). Lots of news and awareness on this topic.
• And of course, Google is your friend for more writings on this topic.