Testing: Test Setup

Setting up for testing isn’t too hard, it just requires a bit of preparation ahead of time. We’re going to be working with Visual Studio’s testing suite, but a lot of the principles are the same in other testing suites. This guide assumes we’re working with Visual Studio 2017 and we’ll be making a brand new project.

Creating the Projects

We’re going to need three projects:

A testing project
A project with our testable code
An executable project that is the main application

Tests in visual studio are their own libraries that get injected into Visual Studio’s testing executable and then run. Thus, we need to make our code a static library that we feed into our testing project. This may also be different from how your team has set up their project in the past (static libraries and executables), so we’ll be making an executable project as well so our code is available in both.

Static Library

First, let’s create the project that has the code we want to utilize and test. First, open visual studio and go to New -> Project:

You should get a pop up that looks like this. Select Visual C++ on the top left and then select Empty Project. We’re going to be making a class that detects if something is a palindrome, so name this project Palindrome.

We also don’t want this to be saved in the middle of no where and have to use Window’s horrible search function, so change the location to your desktop. It will create a new folder named Palindrome there.

Click OK and you should see this on the right side of the screen:

The top level item is the solution(Solution ‘Palindrome’), which can contain multiple projects (Palindrome). It then has filters, which don’t actually map to anything in your real folder structure. They are simply convenient ways to sort out your projects files.

Don’t separate headers from source. It sounds like such a great idea, but in general, if you want the header, you probably want the source. If you start to get a lot of files, you’re going to have to mirror the filter structures you create. It’s a nightmare and not worth it. Filter by the type of code in the file (Graphics, Physics, Core, Components), rather than its file type (.cpp, .h, .py).

In order to test this, we need to change a few values in the project. Right click the project and click on properties:

You should see a pop up that looks like this:

At the top is Configuration: Active(Debug) and Platform (Win32). If you already have used Visual Studio, these may be set to another default. A configuration lets you define settings for how this project will be built. The platform is what kind of application you’re compiling down to work on. We want to have settings work for all configurations and platforms if possible, so set Configuration to “All Configurations” and Platform to “All Platforms”:

We need to make sure the library can be found by the test project we’ll create later, so set the Output Directory to $(SolutionDir)$(Platform)\$(Configuration)\

These will expand to the path to the solution directory (where the .sln file is), create a folder named either x64 or win32 (the platform), and create another subfolder there with the configuration’s name.

$(SolutionDir)$(Platform)\$(Configuration)\ is correct. It is not $(SolutionDir)\$(Platform)\$(Configuration)\ (with the extra slash). For some reason, SolutionDir contains the final slash but Platform and Configuration do not. Windows, am I right?

If you want to check where that will be, click the little down arrow on the right side of Output Directory and click edit.

This will display a pop up that tells you the exact expanded path for the configuration you’re on, or for Debug and Win32 if you’re on all configurations:

Next, we need to set this up as a static library:

A static library is one that contains code that will be injected into an executable or another library at compile time. They can be fairly large and the code needs to be copied into the other library/executable, so it can hike up compile times a bit if you make a ton of them. That said, they’re very modular as you can put them into multiple programs. Executable files simply can’t do that!

This next one is just a good practice. Click on VC++ Directories and $(SolutionDir) to Include Directories:

This adds the solution directory to be a starting point for includes in your project. This is good because there’s a lot of ways to reach a file you’d like to include. For some files, this might require you to go back in directories to find it (“../../Palindrome/Palindrome.h”). It might not require as long of a path as other files (“Palindrome.h” for things in Palindrome project, but a fuller path for something else). Let’s say you need to change the location of a file for a restructuring of your folders. How do you find all of them? How do you fix them? What crazy paths will this require now (“../../../Palinedrome/Palindrome.h”)?

With this, you can simply always start at the solution dir and work your way down. This makes it much easier to find the file in question as well because the include tells you exactly where it is.

This project should be all set settings wise, but let’s add some code to test. Click OK and Right click Source Files -> Add -> New Item…:

You should see this:

Make sure you name is Palindrome.cpp. Repeat this process for Palindrome.h. You should see this:

Double click on the file names to open them up. Put this code in each:

Palindrome.cpp:

#include "Palindrome.h"

bool Palindrome::IsPalindrome(const std::string & aString)
{
    for (int i = 0; i < (int)aString.size() / 2; i++)
        if (aString[i] != aString[aString.size() - 1 - i])
            return false;
    return true;
}

Palindrome.h:

#pragma once

#include <string>

class Palindrome {
public:
    bool IsPalindrome(const std::string& aString);
};

This is an answer to a common interview problem of determining if a given string is a palindrome. To build this library, right click on it and click Build:

At the bottom of the screen, you should see it build without warnings:

Testing Project

Next up is the project to test this code. Right click on the solution -> Add -> New project…:

It will pop up a window. On the top left, click on Test. Then click Native Unit Test Project and set the name to Test. In your real application, you’re going to want to be more specific about what you’re testing.

Click OK. You’re solution explorer should look like this now:

unittest1.cpp is the file we want to look at, but first, let’s set the project up correctly. Right click the Tests project -> Properties.

Add $(SolutionDir) to VC++ Directories’ Include Directories:

Next, open Linker and click on General. Add $(SolutionDir)$(Platform)\$(Configuration)\; to Aditional Libary Directories. Remember before when we changed the output directory for our Palindrome project to that value? Well, now we’re telling this project that they can find libraries there.

Next, click on Input and add Palindrome.lib to Additional Dependencies. This explicitly tells it to include that library.

Click OK. Next we need to ensure the build dependencies for this are correct. Right click on the Solution and open up properties.

Click on Project Dependencies and make sure Tests depends on Palindrome. If you build Tests and Palindrome hasn’t been updated, it will build it automatically.

Let’s write the tests now. Double click on unittest1.cpp in the Tests project in the solution explorer. You should see this code:

Let’s include Palindrome.h:

Note the use of absolute directory paths! Next, we can add some testing code. If you type Assert:: into visual studio, intellisense should pop up and bunch of suggestions for things to assert. Let’s add these functions for a toy example:

TEST_METHOD(WillFail)
{
      Assert::IsTrue(false);
}

TEST_METHOD(WillPass)
{
      Assert::IsTrue(true);
}

Right click on the Tests project and click Build. Now click on the tab at the top called Test -> Windows -> Test Explorer.

Assuming you built the Tests project ahead of time, you should see this:

If the Tests doesn’t show up, you need to build Tests first. Click Run All in Test Explorer. You should see this:

Expand it and you’ll see that one test failed:

Notice that it takes a lot longer to fail than it does to pass. If you look at the bottom of the test explorer, you should see this:

“Message: Assert failed” is an incredibly useless message. If you use other Assert methods such as Assert::AreEqual, you’ll actually see the expected and actual values you tried to test. If we want this to be useful, we’ll need to use the second parameter of Assert::IsTrue. Unfortunately, it requires us to use a wide string, so we’ll need a magical convert function:

std::wstring ToWide(const std::string& aString) {
     return std::wstring(aString.begin(), aString.end());
}
		
TEST_METHOD(WillFail)
{
     Assert::IsTrue(false, ToWide("False").c_str());
}

TEST_METHOD(WillPass)
{
     Assert::IsTrue(true, ToWide("True").c_str());
}

You can’t just return a const wchar_t* from the function. Don’t try it. std::wstring will return a copy and do all the proper deletions. const wchar_t* will be deleted because the local will be deleted.

Rebuild Tests, click Run All and now the output will read like this:

Perfect. Now, let’s create some real testing functions:

void TestPalindrome(const std::string& aString, bool aIsPalindrome) {
    if (aIsPalindrome)
        Assert::IsTrue(
            Palindrome().IsPalindrome(aString), 
            ToWide(aString + " was Palindrome but returned false").c_str());
    else
        Assert::IsFalse(
            Palindrome().IsPalindrome(aString), 
            ToWide(aString + " was Palindrome but returned false").c_str());
}

TEST_METHOD(Palindrome_SingleLetter)
{
    TestPalindrome("a", true);
}

We can now create a bunch of these tests to catch any errors in our code.

TEST_METHOD(Palindrome_CanIdentifyAPalindrome)
{

    TestPalindrome("", true);
    TestPalindrome("a", true);
    TestPalindrome("aa", true);
    TestPalindrome("aaa", true);
    TestPalindrome("aaaa", true);
    TestPalindrome("aba", true);
    TestPalindrome("abba", true);
    TestPalindrome("abcba", true);
}

TEST_METHOD(Palindrome_CanIdentifyIfNotAPalindrome)
{
    TestPalindrome("ab", false);
    TestPalindrome("ba", false);
    TestPalindrome("abc", false);
    TestPalindrome("abbc", false);
    TestPalindrome("abca", false);
}

Let’s look at another test we could put in. It’s worth making a test to see if we can handle a long palindrome:

TEST_METHOD(Palindrome_CanIdentifyLongPalindromes)
{
    std::string testString;
    std::string letters[5] = { "a","b","c","d","e" };
    for (int i = 0; i < 1000; i++)
    {
        testString.push_back(letters[i % 5][0]);
        testString.insert(0, letters[i % 5]);
    }
    TestPalindrome(testString, true);
}

What’s wrong with this test? The problem is the fact that it takes a good second of staring to even understand what it’s doing. Tests should be simple to understand and clear what they’re testing. This would be a much better test:

TEST_METHOD(Palindrome_CanIdentifyLongPalindromes)
{
    std::string forwardString = "abcde";
    std::string reversedString = "edcba";
    std::string palindrome;
    
    for (int i = 0; i < 1000; i++)
        palindrome += forwardString;
    for (int i = 0; i < 1000; i++)
        palindrome += reversedString;  

    TestPalindrome(palindrome, true);
}

No weird incantations. No strange math. Just a simple way to produce a long palindrome.

Executable Project

The last thing we need to do is ensure we can use our library in our actual game. This will seem very familiar if you followed the steps above. First, right click the solution ->

Just like before, create a project named Executable:

In Executable’s properties, make sure we do the same standard steps:

In General, change Output Directory to $(SolutionDir)$(Platform)\$(Configuration)\
In VC++ Directories -> Include Directories, add $(SolutionDir);
In Linker – > General -> Additional Library Directories, add $(SolutionDir)$(Platform)\$(Configuration)\;
In Linker -> Input -> Additional Dependencies, add Palindrome.lib;

Next, add a file named main.cpp with the following code to Executable project:

#include <iostream>
#include "Palindrome/Palindrome.h"

int main() {
    
    if (Palindrome().IsPalindrome("aba"))
        std::cout << "It's a palindrome" << std::endl;
    else 
        std::cout << "It's not a palindrome" << std::endl;
    while (1);
    return 0;
}

Right click on the Solution and open its properties. Change the startup project to Executable.