Unit-testing (embedded) C applications with Ceedling

Just like a lot of other embedded software engineers, I used to ship my embedded applications to production without testing them properly. Only some manual tests were done. I was under the impression that there's no real way to test them: you know, embedded applications run in a custom hardware and interact with this specific hardware heavily, which makes them not so easy to test automatically.

But the more I worked in this field, the more I thought that there should be some way to make my applications more reliable. Of course, I strive to develop right application design and write good implementation, but mistakes (sometimes very silly ones) happen. And one day, I've come across the awesome book: Test Driven Development for Embedded C by James W. Grenning. It is a very good book, and it explains the topic very thoroughly; highly recommended.

Although this article is based heavily on what I've learned from this book, it is not just a very short version of it: while trying to put my knowledge to practice, I found some new tools that simplify the process even more. I hope that the article will help you to start quickly.

Embedded development differs from other software engineering fields in that embedded application has to interact with the hardware, and the hardware might be very different from application to application.

Unfortunately, I've seen a great deal of code which performs some application logic while interacting with the hardware directly. It is sometimes done in the name of efficiency (on the very low-end chips, even function calls might be considered as expensive), but eventually it might lead to one's habit, which, of course, is to be avoided.

The main idea (which is very good in itself, not only for unit testing) is to separate the hardware interaction and the application logic as much as possible. Then, we end up with a bunch of separate modules, which can be tested outside of the hardware.

And when we write our application with the testability in mind, we literally have to separate things. So, one “side effect” of writing a testable code is the modularity, which is a good thing to have. Testable code is all good!

So, in this article, we will not test the hardware layer. It has to be as thin as possible, and I still test it manually. What we will test is everything else: the modules that use hardware layer, and the modules that use other modules.

While there are techniques to run tests on the target hardware (and the book by James W. Grenning touches upon them, among other things), this article focuses on testing on the host machine instead.

And it's probably worth mentioning that unit tests are not the silver bullet that will magically turn your projects in completely bug-free ones. This is not true even for desktop programming, where we use the same compiler for tests and for production; and in the embedded world, it's even worse, since:

• We can't test for compiler bugs (and, unfortunately, embedded compilers tend to be… not very good);
• There may be subtle issues in software-hardware interaction;
• etc

Still, from my personal experience, the final outcome is a way better on carefully tested projects than on untested ones. At least, well-written tests will save you from your own silly mistakes. And if you're like me, you definitely want that.

The aforementioned book takes advantage of several awesome tools:

• Unity.

Unity is curiously powerful Unit Testing in C for C. It aims to support most embedded compilers, from 8-bit tiny processors to 64-bit behemoths. Unity is designed to be small, yet still provide you rich expressive assertion set.

• CMock.

CMock is a utility for automagical generation of stubs and mocks for Unity Tests. It's a collection of scripts in Ruby which investigate your C headers and create reusable fake versions for your tests.

These tools are surely life-changing for me, but there is something that is still missing: the test build system. Probably it didn't yet exist when the book was written, so it isn't mentioned in the book at all. Here it is:

• Ceedling

Ceedling is a build system for C projects that is something of an extension around Ruby’s Rake (make-ish) build system. Ceedling is primarily targeted at Test-Driven Development in C and is designed to pull together CMock, Unity, and CException – three other awesome open-source projects you can’t live without if you're creating awesomeness in the C language.

Armed with these great tools, let's move on.

No surprise, we'll be learning how to write unit tests by testing example project. Let it be rather simple device, which I have actually developed firmware for: the indicator for battery charger that runs on 8-bit PIC16 MCU with 16 kB flash and 1 kB RAM. As you see, resources are quite limited.

The repository of this example project can be found here: https://github.com/dimonomid/test_ceedling_example.

In order to bring the project to its initial state, checkout to the tag v0.02:

  $ git checkout v0.02

This device should merely measure some voltages (which are converted to an integer via ADC), and display them properly. It should also be available to communicate with the checker device, which will calibrate the electric circuits.

Initial project tree looks as follows:

.
└── src
    ├── appl
    │   ├── appl_adc_channels.h
    │   └── appl.c
    ├── bsp
    │   ├── bsp_adc.c
    │   └── bsp_adc.h
    └── util
        ├── adc_handler.c
        ├── adc_handler.h
        ├── itoae.c
        ├── itoae.h
        └── util_macros.h

There are three directories:

• appl: main application code;
• bsp: board- and MCU-dependent code;
• util: various utilities.

Since the resources are very limited, we don't use a RTOS: just a super-loop. We also can't use any printf-like functions, as they are too expensive. And we can't use floats: we use integers only.

So, the main() looks like this:

int main(void)
{
    //TODO: MCU-specific init
 
    //-- init MCU-specific ADC stuff
    bsp_adc__init();
 
    //-- enter endless loop
    for (;;){
        bsp_adc__proceed();
    }
}

This is a quite common scheme for non-RTOS solutions: in the super-loop, we just call each module's proceed function, which might do some job. Currently, we have just one module there: bsp_adc.

The bsp_adc module is an MCU-dependent routines to retrieve raw ADC values. Its API looks like this:

/**
 * Type that is used for ADC raw counts.
 */
typedef uint16_t T_BspAdcCounts;
 
 
 
/**
 * Perform module initialization, including the hardware initialization
 */
void bsp_adc__init(void);
 
/**
 * Returns raw ADC counts for the specified channel
 */
T_BspAdcCounts bsp_adc__value__get(enum E_ApplAdcChannel channel_num);
 
/**
 * To be repeatedly called from the application's super loop: stores current
 * measurement when it's ready and switches between different channels as
 * appropriate
 */
void bsp_adc__proceed(void);

As you see, it's very easy. When we call bsp_adc__value__get(), it returns the raw ADC value for specified channel, which is not particularly useful. We'd like to convert it to something more human-readable, right? That is, to the value in Volts.

There is an adc_handler module for that. Basically, it just applies a summand and a multiplier to the raw value returned by bsp_adc__value__get(), and we get an integer value in Volts, multiplied by some scale factor (in this example, the factor 100 is used; so, the value 1250 means 12.5 Volts.

For each instance of ADC handler (struct S_ADCHandler), we call its constructor function:

T_ADCHandler_Res adc_handler__ctor(
        T_ADCHandler *me, const T_ADCHandler_CtorParams *p_params
        );

Passing a pointer to parameters:

/**
 * Constructor params for ADC handler
 */
typedef struct S_ADCHandler_CtorParams {
 
    /**
     * Maximum value that ADC could return. Say, for 10-bit ADC it should be
     * 0x3ff.
     */
    T_ADCHandler_CountsValue   max_counts;
 
    /**
     * The board-dependent maximum voltage that could be measured, it
     * corresponds to the max_counts.
     *
     * It is only needed for calculation of nominal multiplier.
     */
    T_ADCHandler_Voltage      bsp_max_voltage;
 
    /**
     * Calibration data: summand and multiplier.
     *
     * Set just all zeros to use nominal. Nominal mul will be calculated by
     * max_counts and bsp_max_voltage. add will be copied from
     * nominal_add (see below)
     */
    T_ADCHandler_Clb  clb;
 
    /**
     * Nominal summand, in volts
     */
    T_ADCHandler_Voltage nominal_add_volts;
 
} T_ADCHandler_CtorParams;

And then, when we want to convert some raw ADC value my_raw_adc_value to Volts, it is as easy as:

T_ADCHandler_Voltage my_voltage = adc_handler__voltage__get_by_counts_value(
        &my_instance, my_raw_adc_value
        );

As you see, the ADC handler module is completely self-contained: it doesn't have any dependencies. Modules like this are the easiest ones to test, so, let's start our test journey from ADC handler.

Before we can go any further, we need to install the aforementioned Ceedling and all accompanying tools. Thanks to guys from ThrowTheSwitch.org, installation process is very simple. You need ruby for this. Once you have ruby installed, install Ceedling by typing in your terminal:

$ gem install ceedling

And that's it! Now you have all tools ready to work, and among other things, there is a ceedling binary, which we're going to take advantage of.

The ceedling binary allows us to create new project tree by executing the command ceedling new my_project, including even main source directory. Since I usually create my projects in some other way, I need to move things around after ceedling has created new project.

Let's move on: cd to the project's directory (where you have the src directory), and execute the following:

$ ceedling new test_ceedling

This will create the new directory test_ceedling with the following contents:

• test: directory for our test C files;
• build: directory in which our built test will be located;
• vendor: ceedling-provided binaries and all accompanying stuff. It includes CMock, Unity, and other tools. Since we're just users of Ceedling, we have little interest of actual contents of this directory, except the documentation:
• vendor/ceedling/docs: documentation of Ceedling and components. Very useful;
• src: the directory in which Ceedling assumed to have source files, but we're going to store files not here, but in our own src directory;
• rakefile.rb: it is needed to run tests; you don't need to understand it;
• project.yml: the actual project file, which we need to adjust for our needs.

The heart of the test build system is the project file: project.yml. Among other things, it contains paths to your application source files. Since our source files are contained not where ceedling assumed by default, we need to change the project file a bit: open the file project.yml and find the paths section:

:paths:
  :test:
    - +:test/**
    - -:test/support
  :source:
    - src/**
  :support:
    - test/support

As you might have already guessed, we need to change src/** to the path to our actual source files, relative to the location of the project file. Ok, that's easy: let's add all source paths that currently exist. We end up with three paths there:

  :source:
    - ../src/appl
    - ../src/bsp
    - ../src/util

Now, in order to avoid confusion, let's delete the auto-created test_ceedling/src directory, since we're not going to use it. We also want to add some .gitkeep files in our empty directories, so that git will keep them in repository. From the root of the repository, type:

$ touch test_ceedling/build/.gitkeep
$ touch test_ceedling/test/support/.gitkeep

Add files to the repository and commit:

$ git add .
$ git commit

Note: you can get everything done by using the prepared repository. Type there: git checkout v0.03.

Actually, our test build system, though empty, is ready to run! Try it: make sure you're in the test_ceedling directory, and type in the terminal:

$ rake test:all

You should see the following output:

--------------------
OVERALL TEST SUMMARY
--------------------

No tests executed.

It works, and it predictably reports that we have no tests. So, let's add some meat to the bones!

As mentioned above, we'll start by writing tests for our ADC handler, since it is one of the easiest things to test: it has no application-specific dependencies. The job of ADC handler is to convert from raw ADC counts to voltage, and vice versa. We'll test this functionality.

First of all, let's create blank test file. I have a template for this:

/*******************************************************************************
 *    INCLUDED FILES
 ******************************************************************************/
 
//-- unity: unit test framework
#include "unity.h"
 
//-- module being tested
//   TODO
 
 
/*******************************************************************************
 *    DEFINITIONS
 ******************************************************************************/
 
/*******************************************************************************
 *    PRIVATE TYPES
 ******************************************************************************/
 
/*******************************************************************************
 *    PRIVATE DATA
 ******************************************************************************/
 
 
/*******************************************************************************
 *    PRIVATE FUNCTIONS
 ******************************************************************************/
 
 
/*******************************************************************************
 *    SETUP, TEARDOWN
 ******************************************************************************/
 
void setUp(void)
{
}
 
void tearDown(void)
{
}
 
/*******************************************************************************
 *    TESTS
 ******************************************************************************/
 
void test_first(void)
{
    //TODO
}

We can use this template whenever we need to write new test. Our tests are just functions with names that start with test_. In the example above, we have just empty test_first() function.

We also have two special functions: setUp() and tearDown(). The setUp() is called before running each test, and tearDown() is called after running each test. We'll take advantage of them soon. Now, let's add ADC handler test here.

We begin by adding the header of the module being tested:

//-- module being tested: ADC handler
#include "adc_handler.h"

Then, add an instance that we'll run tests against, together with the result code returned from constructor:

/*******************************************************************************
 *    PRIVATE DATA
 ******************************************************************************/
 
static T_ADCHandler     _adc_handler;
static T_ADCHandler_Res _ctor_result;

And then, construct/destruct it in setUp() / tearDown(), respectively.

void setUp(void)
{
    T_ADCHandler_CtorParams params = {};
 
    //-- 10-bit ADC
    params.max_counts = 0x3ff;
 
    //-- board-dependent maximum measured voltage: 10 Volts
    params.bsp_max_voltage = 10/*V*/ * ADC_HANDLER__SCALE_FACTOR__U;
 
    //-- the offset is 0 Volts
    params.nominal_add_volts = 0/*V*/;
 
 
    //-- construct the ADC handler, saving the result to _ctor_result
    _ctor_result = adc_handler__ctor(&_adc_handler, &params);
}
 
void tearDown(void)
{
    adc_handler__dtor(&_adc_handler);
}

Now, the easiest test we can come up with is to check that constructor has returned successful status, i.e. ADC_HANDLER_RES__OK. So, rename our dummy test_first to test_ctor_ok, and add our first assert:

void test_ctor_ok(void)
{
    //-- check that constructor returned OK
    TEST_ASSERT_EQUAL_INT(ADC_HANDLER_RES__OK, _ctor_result);
}

We're ready to run our first test!

$ rake test:all

Test 'test_adc_handler.c'
-------------------------
Generating runner for test_adc_handler.c...
Compiling test_adc_handler_runner.c...
Compiling test_adc_handler.c...
Compiling unity.c...
Compiling adc_handler.c...
Compiling cmock.c...
Linking test_adc_handler.out...
Running test_adc_handler.out...

-----------
TEST OUTPUT
-----------
[test_adc_handler.c]
  - ""

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  1
PASSED:  1
FAILED:  0
IGNORED: 0

It works, and our test has passed. Good!

Among other things, you see that Ceedling has figured out that it needs to build adc_handler.c. How does it work? Ceedling examines the headers we include in our test file, and looks for the appropriate source file. Since we have adc_handler.h included, Ceedling looks for adc_handler.c file in all source paths that we've specified in our project.yml, and compiles it. Pretty easy.

If you used to apply TDD practices, you know that it's good to make sure our tests fail if code behaves in wrong way. We don't do TDD here, since we already have some code before we write tests, but we still can make sure that our tests can fail. If we change adc_handler__ctor() so that it returns, for example, ADC_HANDLER_RES__CLB_ERR__WRONG_PARAMS, our test will fail as follows:

-----------
TEST OUTPUT
-----------
[test_adc_handler.c]
  - ""

-------------------
FAILED TEST SUMMARY
-------------------
[test_adc_handler.c]
  Test: test_ctor_ok
  At line (72): "Expected 1 Was 6"

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  1
PASSED:  0
FAILED:  1
IGNORED: 0

---------------------
BUILD FAILURE SUMMARY
---------------------
Unit test failures.

That's it. Let's change adc_handler back to correct state, and commit.

We definitely don't want to include build output in the repository, so, add the test_ceedling/build directory to ignore list. In the root of the repo, create .gitignore file with the following contents:

test_ceedling/build

And then, commit changes:

$ git add .
$ git commit -m"added test_adc_handler"

Note: you can get everything done by using the prepared repository. Type there: git checkout v0.04.

Now let's check that ADC handler is actually able to convert from ADC counts to voltage. We'll test the function adc_handler__voltage__get_by_counts_value(). With the parameters we've given in setUp() function, we need to make sure that 0 counts are converted to 0.0 Volts, 0x3ff counts are converted to 10.0 Volts, and, for example, 0x3ff / 3 counts are converted to 3.33 Volts.

Here we go:

void test_counts_to_voltage(void)
{
    T_ADCHandler_Voltage voltage;
 
 
    //------------------------------------------------------------------
    voltage = adc_handler__voltage__get_by_counts_value(
            &_adc_handler, 0
            );
    TEST_ASSERT_EQUAL_INT(0/*V*/ * ADC_HANDLER__SCALE_FACTOR__U, voltage);
 
 
    //------------------------------------------------------------------
    voltage = adc_handler__voltage__get_by_counts_value(
            &_adc_handler, 0x3ff
            );
    TEST_ASSERT_EQUAL_INT(10/*V*/ * ADC_HANDLER__SCALE_FACTOR__U, voltage);
 
 
    //------------------------------------------------------------------
    voltage = adc_handler__voltage__get_by_counts_value(
            &_adc_handler, 0x3ff / 3
            );
    TEST_ASSERT_EQUAL_INT((3.33/*V*/ * ADC_HANDLER__SCALE_FACTOR__U), voltage);
}

If we run this test, it should pass:

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  2
PASSED:  2
FAILED:  0
IGNORED: 0

So, we can be sure now that ADC handler performs its basic job.

Note: you can get everything done by using the prepared repository. Type there: git checkout v0.05.

As was mentioned before, we can't use floats since they are too expensive for this cheap MCU, so, we store voltage as integer in Volts, multiplied by the factor 100. Of course, we need to convert the voltage integer value like 1250 to the string like “12.5”. That's what the awfully named module itoae is for. The name “itoae” stands for “integer-to-array-extended”.

Its API looks like this:

/**
 * Itoa a bit extended: allows to set minimal length of the string
 * (effectively allowing us to align text by right edge),
 * and allows to put decimal point at some fixed position.
 *
 * @param p_buf
 *      where to save string data
 * @param value
 *      value to convert to string
 * @param dpp
 *      decimal point position. If 0, then no decimal point is put.
 *      if 1, then it is put one digit from the right side, etc.
 * @param min_len
 *      minimum length of the string. If actual string is shorter than
 *      specified length, then leftmost characters are filled with
 *      fill_char.
 * @param fill_char
 *      character to fill "extra" space
 */
void itoae(uint8_t *p_buf, int value, int dpp, int min_len, uint8_t fill_char);

You can find the source in the file src/util/itoae.c. The implementation is rather dumb: first, we call “regular” itoa that doesn't know anything about decimal points, it just converts integer to string. Then, if we need for decimal point, then move some characters to the right, and insert the ..

As you see, this function is very straightforward to test as well. Let's create new file test_ceedling/test/test_itoae.c from the template above, and include the header of module being tested:

//-- module being tested
#include "itoae.h"

We're going to have a buffer for generated string:

#define _BUF_LEN    20
 
/**
 * Buffer to store generated string data
 */
static uint8_t _buf[ _BUF_LEN ];

As well as the function that wills the buffer with 0xff:

void _fill_with_0xff(void)
{
    int i;
    for (i = 0; i < sizeof(_buf); i++){
        _buf[i] = 0xff;
    }
}

We will call this function before each assert, so that the buffer is reinitialized every time. And a couple of simple tests:

void test_basic( void )
{
    _fill_with_0xff();
    itoae(_buf, 123, 0, 0, '0');
    TEST_ASSERT_EQUAL_STRING("123", _buf);
 
    _fill_with_0xff();
    itoae(_buf, -123, 0, 0, '0');
    TEST_ASSERT_EQUAL_STRING("-123", _buf);
}

void test_dpp( void )
{
    _fill_with_0xff();
    itoae(_buf, 123, 1, 0, '0');
    TEST_ASSERT_EQUAL_STRING("12.3", _buf);
 
    _fill_with_0xff();
    itoae(_buf, 123, 2, 0, '0');
    TEST_ASSERT_EQUAL_STRING("1.23", _buf);
 
    _fill_with_0xff();
    itoae(_buf, 123, 3, 0, '0');
    TEST_ASSERT_EQUAL_STRING("0.123", _buf);
 
    _fill_with_0xff();
    itoae(_buf, 123, 4, 0, '0');
    TEST_ASSERT_EQUAL_STRING("0.0123", _buf);
}

Run the tests:

-------------------
FAILED TEST SUMMARY
-------------------
[test_itoae.c]
  Test: test_dpp
  At line (127): "Expected '12.3' Was '12.3\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF'"

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  4
PASSED:  3
FAILED:  1
IGNORED: 0

Oops! Something went wrong. It seems that although the dot was inserted to the string correctly, the terminating 0x00 wasn't moved to the right by 1 char.

After examining the source, I see that here's the offending piece of code:

        int i;
        for (i = 0; i < (dpp + 1/*null-terminate*/); i++){
            p_buf[len - i] = p_buf[len - i - 1];
        }
        p_buf[len - dpp] = '.';

Although I provided the handling of terminating null char, there is a traditional off-by-one error here. The correct code looks as follows:

        int i;
        for (i = 0; i < (dpp + 1/*null-terminate*/); i++){
            p_buf[len - i + 1] = p_buf[len - i];
        }
        p_buf[len - dpp] = '.';

Save, switch to the terminal, run the tests again:

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  4
PASSED:  4
FAILED:  0
IGNORED: 0

Cool! These very simple tests already saved me from the very silly mistake. Believe it or not, this mistake with itoae has actually happened to me, and tests helped to reveal it. This episode quickly encouraged me to invest my time in tests even more.

Note: not all tests code is included in the article, since it is very repetitive and straightforward. You can get everything done by using the prepared repository. Type there: git checkout v0.06.

Our application needs for some resourceful way to get current voltage on some particular channel, in Volts. It would be unwise to use our bsp_adc + adc_handler modules across the whole application.

Now, let's add the module appl_adc, which will have at least the function to get current voltage on some channel, in Volts.

The header src/appl/appl_adc.h should contain at least the following:

//-- for T_ADCHandler_Voltage
#include "adc_handler.h"
 
//-- for enum E_ApplAdcChannel
#include "appl_adc_channels.h"
 
/**
 * Initialize module
 */
void appl_adc__init(void);
 
/**
 * Get current voltage of the given channel.
 */
T_ADCHandler_Voltage appl_adc__voltage__get(enum E_ApplAdcChannel channel_num);

This function should call bsp_adc__value__get() for the given channel_num, then feed returned value to the appropriate adc_handler, and return resulting value in Volts.

For now, let's just fake the implementation (src/appl/appl_adc.c):

#include "appl_adc.h"
 
void appl_adc__init(void)
{
    //TODO
}
 
T_ADCHandler_Voltage appl_adc__voltage__get(enum E_ApplAdcChannel channel_num)
{
    //TODO
    return 0;
}

And get to tests: create new file test_appl_adc.c, and put the tests template in it. As usual, include the header of the module being tested:

//-- module being tested
#include "appl_adc.h"

Our setUp() and tearDown() are very simple:

void setUp(void)
{
    //-- before each test, re-initialize appl_adc module
    appl_adc__init();
}
 
void tearDown(void)
{
    //-- nothing to do here
}

And get to the test for appl_adc__voltage__get():

void test_voltage_get(void)
{
    // .....
}

As was mentioned before, appl_adc__voltage__get() first of all should call bsp_adc__value__get() for the appropriate channel. How do we test it?

The answer is - with CMock. And Ceedling will help us here as well: if we need to “mock” some module, all we need to do is to include the header of the module to mock, with the mock_ prefix (well, actually, the prefix is customizable in project.yml, and by default it is mock_).

Go on then: add the following include directive:

//-- mocked modules
#include "mock_bsp_adc.h"

Now, the module being tested will use mocked versions of all functions from bsp_adc.h, and we are provided with the “expect” functions. Let's see them in action:

void test_voltage_get(void)
{
    //-- We expect bsp_adc__value__get() to be called:
    bsp_adc__value__get_ExpectAndReturn(
            //-- the argument that is expected to be given to
            //   bsp_adc__value__get()
            APPL_ADC_CH__I_SETT,
 
            //-- and the value that bsp_adc__value__get() should return
            (0x3ff / 2)
            );
 
    //-- actually call the function being tested, that should perform
    //   all pending expected calls
    T_ADCHandler_Voltage voltage = appl_adc__voltage__get(
            APPL_ADC_CH__I_SETT
            );
 
    //-- check the voltage returned (we assume that adc_handler is initialized
    //   with the same params, where 0x3ff is the maximum ADC value, and
    //   it corresponds to the value (10 * ADC_HANDLER__SCALE_FACTOR__U))
    TEST_ASSERT_EQUAL_INT((5 * ADC_HANDLER__SCALE_FACTOR__U), voltage);
}

If we run the tests, we get the following result:

-------------------
FAILED TEST SUMMARY
-------------------
[test_appl_adc.c]
  Test: test_voltage_get
  At line (77): "Expected 500 Was 0"

So it complains that returned value was wrong. Okay, let's fake our dummy appl_adc__voltage__get() even more: make it return 500 instead of 0:

T_ADCHandler_Voltage appl_adc__voltage__get(enum E_ApplAdcChannel channel_num)
{
    //TODO
    return 500;
}

And run tests again:

-------------------
FAILED TEST SUMMARY
-------------------
[test_appl_adc.c]
  Test: test_voltage_get
  At line (56): "Function 'bsp_adc__value__get' called less times than expected."

Oh, cool! It reports that the function bsp_adc__value__get() was called less times than expected (effectively, it wasn't called at all). CMock in action!

Well, it's time to implement appl_adc__voltage__get() more or less completely: we're going to use both bsp_adc and adc_handler there.

appl_adc.c

/*******************************************************************************
 *    INCLUDED FILES
 ******************************************************************************/
 
#include "appl_adc.h"
#include "appl_adc_channels.h"
#include "bsp_adc.h"
 
 
/*******************************************************************************
 *    PRIVATE DATA
 ******************************************************************************/
 
static T_ADCHandler _adc_handlers[ APPL_ADC_CH_CNT ];
 
 
/*******************************************************************************
 *    PUBLIC FUNCTIONS
 ******************************************************************************/
 
void appl_adc__init(void)
{
    enum E_ApplAdcChannel channel;
 
    //-- init all ADC channels
    for (channel = 0; channel < APPL_ADC_CH_CNT; channel++){
        T_ADCHandler_CtorParams params = {};
 
        //-- here, we initialize all channels with the same params,
        //   but in real life different ADC channels may, of course,
        //   have different parameters.
        params.max_counts = 0x3ff;
        params.bsp_max_voltage = 10/*V*/ * ADC_HANDLER__SCALE_FACTOR__U;
        params.nominal_add_volts = 0/*V*/;
 
        //-- construct the ADC handler, saving the result to _ctor_result
        adc_handler__ctor(&_adc_handlers[ channel ], &params);
    }
}
 
T_ADCHandler_Voltage appl_adc__voltage__get(enum E_ApplAdcChannel channel_num)
{
    return adc_handler__voltage__get_by_counts_value(
            &_adc_handlers[ channel_num ],
            bsp_adc__value__get(channel_num)
            );
}

Ok, it seems, everything is right. Try to run tests:

Linking test_appl_adc.out...
build/test/out/appl_adc.o: In function 
0097ppl_adc__init':
/home/dimon/projects/indicator_git/test_ceedling/../src/appl/appl_adc.c:70: undefined reference to 
0097dc_handler__ctor'
build/test/out/appl_adc.o: In function 
0097ppl_adc__voltage__get':
/home/dimon/projects/indicator_git/test_ceedling/../src/appl/appl_adc.c:76: undefined reference to 
0097dc_handler__voltage__get_by_counts_value'
collect2: error: ld returned 1 exit status

....

NOTICE: If the linker reports missing symbols, the following may be to blame:
  1. Test lacks #include statements corresponding to needed source files.
  2. Project search paths do not contain source files corresponding to #include statements in the test.
  3. Test does not #include needed mocks.

Oh, dear. Linker complains about undefined reference to ADC handler functions. And Ceedling is being very kind here by providing us with useful notice: as it suggests, one of the possible reasons is that test lacks #include statements corresponding to needed source files. Do you remember that Ceedling examines the headers we include in our test file, and looks for the appropriate source file? So, to make it work, we should include the header adc_handler.h to our test_appl_adc.c:

//-- other modules that need to be compiled
#include "adc_handler.h"

Now, the tests should finally work:

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  7
PASSED:  7
FAILED:  0
IGNORED: 0

Note: you can get everything done by using the prepared repository. Type there: git checkout v0.07.

As you might have noticed, in the previous section we in fact ended up testing two modules at once: appl_adc.c and adc_handler.c. It doesn't seem right: at least, we already have dedicated tests for adc_handler.c, right? And, after all, even the term “unit test” suggests that we should test one unit at a time. Although sometimes I allow myself to break this rule, let's try to isolate tests for appl_adc.c as much as possible.

You might have already guessed that we're going to mock ADC handler as well, instead of using real code. So, first of all, let's remove #include "adc_handler.h", and include mocked version instead:

//-- mocked modules
#include "mock_bsp_adc.h"
#include "mock_adc_handler.h"

And now, we in fact have several options.

The easiest option is to ignore arguments given to adc_handler__voltage__get_by_counts_value() whatsoever. That's what …_IgnoreAndReturn() functions are for (generated by CMock):

void test_voltage_get(void)
{
    //-- We expect bsp_adc__value__get() to be called:
    bsp_adc__value__get_ExpectAndReturn(
            //-- the argument that is expected to be given to
            //   bsp_adc__value__get()
            APPL_ADC_CH__I_SETT,
 
            //-- and the value that bsp_adc__value__get() should return
            123
            );
 
    //-- Expect call to adc_handler__voltage__get_by_counts_value(),
    //   ignoring arguments. The mocked version just returns 456.
    adc_handler__voltage__get_by_counts_value_IgnoreAndReturn(456);
 
    //-- actually call the function being tested, that should perform
    //   all pending expected calls
    T_ADCHandler_Voltage voltage = appl_adc__voltage__get(
            APPL_ADC_CH__I_SETT
            );
 
    //-- check the voltage returned (it should be 456 from the mock above)
    TEST_ASSERT_EQUAL_INT(456, voltage);
}

Run tests:

-------------------
FAILED TEST SUMMARY
-------------------
[test_appl_adc.c]
  Test: test_voltage_get
  At line (57): "Function 'adc_handler__ctor' called more times than expected."

Oh yes, we forgot that now we have to mock not only adc_handler__voltage__get_by_counts_value(), but the constructor as well, which is called from setUp(). We're going to ignore its arguments too, so, the modified setUp() looks as follows:

void setUp(void)
{
    //-- ADC handler constructor is going to be called for each channel.
    //   Mocked constructors all return ADC_HANDLER_RES__OK.
    enum E_ApplAdcChannel channel;
    for (channel = 0; channel < APPL_ADC_CH_CNT; channel++){
        adc_handler__ctor_IgnoreAndReturn(ADC_HANDLER_RES__OK);
    }
 
    //-- before each test, re-initialize appl_adc module
    appl_adc__init();
}

Now tests pass.

Note: you can get everything done by using the prepared repository. Type there: git checkout v0.08.

The arguments given to adc_handler__voltage__get_by_counts_value() are:

• Pointer to an instance of T_ADCHandler;
• Counts value to convert to Volts.

So, if we want to check arguments, we need to access T_ADCHandler for each particular channel, which is private to appl_adc.c. For this to be done, we have to create test-accompanying “protected” function in appl_adc that will return appropriate pointer:

/**
 * For usage in tests only!
 */
T_ADCHandler *_appl_adc__adc_handler__get(enum E_ApplAdcChannel channel)
{
    return &_adc_handlers[channel];
}

We won't include this function prototype in the header file appl_adc.h; instead, we will include it as the external function prototype to test_appl_adc.c:

/*******************************************************************************
 *    EXTERNAL FUNCTION PROTOTYPES
 ******************************************************************************/
 
extern T_ADCHandler *_appl_adc__adc_handler__get(enum E_ApplAdcChannel channel);

And now, instead of ignoring arguments given to adc_handler__voltage__get_by_counts_value(), we can use …_ExpectAndReturn() mock function:

    //-- Expect call to adc_handler__voltage__get_by_counts_value(),
    //   ignoring arguments. The mocked version just returns 456.
    adc_handler__voltage__get_by_counts_value_ExpectAndReturn(
            //-- pointer to the appropriate ADC handler instance
            _appl_adc__adc_handler__get(APPL_ADC_CH__I_SETT),
            //-- value returned from bsp_adc__value__get()
            123,
            //-- returned value in Volts
            456
            );

Notice that the value returned from mocked bsp_adc__value__get() (i.e. 123) matches the value given to adc_handler__voltage__get_by_counts_value(). And tests pass. If the value mismatches, tests will fail.

Note: you can get everything done by using the prepared repository. Type there: git checkout v0.09.

Apart from easy-to-use helpers, CMock provides us with the very flexible callback helper. The callback should have the same signature as the mocked function, but it takes one additional argument: num_calls. When the function called first time, num_calls is 0, and it will be incremented by 1 with each subsequent call.

In the callback, we might check whatever we want, and if something goes wrong, we can call Unity macro TEST_FAIL_MESSAGE().

Let's implement such a callback:

/*******************************************************************************
 *    PRIVATE FUNCTIONS
 ******************************************************************************/
 
static T_ADCHandler_Voltage _get_by_counts_value_Callback(
        T_ADCHandler *me,
        T_ADCHandler_CountsValue counts_value,
        int num_calls
        )
{
    T_ADCHandler_Voltage ret = 0;
 
    switch (num_calls){
        case 0:
            if (counts_value != 123){
 
                //-- We can check whatever we want here. For example, we may
                //   check the data pointed to by "me", but NOTE that currently
                //   it is just zeros, since we have mocked adc_handler__ctor()
                //   as well, so the original constructor isn't called, and
                //   instances are left unitialized.
 
                TEST_FAIL_MESSAGE(
                        "adc_handler__voltage__get_by_counts_value() was called "
                        "with wrong counts_value"
                        );
            }
 
            ret = 456;
            break;
 
        default:
            TEST_FAIL_MESSAGE(
                    "adc_handler__voltage__get_by_counts_value() was called "
                    "too many times"
                    );
            break;
    }
 
    return ret;
}

And in our test_voltage_get(), we use it as follows:

    //-- Expect call to adc_handler__voltage__get_by_counts_value()
    adc_handler__voltage__get_by_counts_value_StubWithCallback(
            _get_by_counts_value_Callback
            );

Although callbacks like this don't look quite elegant, and for this particular example it is an unnecessary overkill, they are extremely flexible. So, keep it in your toolbox, and use when appropriate.

Note: you can get everything done by using the prepared repository. Type there: git checkout v0.10.

Compilers often have some useful non-standard built-in things. For example, the XC8 Microchip compiler has the function __builtin_software_breakpoint(), which, how its name suggests, puts software breakpoint. If the MCU runs in it with debugger attached, debugger halts execution. This function becomes available if we include the "xc.h" header.

I often use it for some conditions that should never happen. For example, our appl_adc__voltage__get() should never be called with wrong channel_num. Let's add a check for this:

#include "xc.h"
 
// .....
 
T_ADCHandler_Voltage appl_adc__voltage__get(enum E_ApplAdcChannel channel_num)
{
    T_ADCHandler_Voltage ret = 0;
 
    if (channel_num < APPL_ADC_CH_CNT){
        ret = adc_handler__voltage__get_by_counts_value(
                &_adc_handlers[ channel_num ],
                bsp_adc__value__get(channel_num)
                );
    } else {
        //-- illegal channel_num given: should never be here
        __builtin_software_breakpoint();
    }
 
    return ret;
}

Checks like this are a must have in any application, but if we try to run tests, we'll end up with the following error:

Compiling appl_adc.c...
../src/appl/appl_adc.c:15:16: fatal error: xc.h: No such file or directory
 #include "xc.h"

Obviously, GCC (which is used for tests by default) have neither such a built-in function, nor the “xc.h” header file.

We can address this problem by using the Ceedling “support” directory, which is located by default at test/support. Let's create the “xc.h” file in it, and put the following contents there:

xc.h

#ifndef _MY_XC_DUMMY
#define _MY_XC_DUMMY
 
void __builtin_software_breakpoint(void);
 
#endif // _MY_XC_DUMMY

If we run tests now, we'll have different error:

Linking test_appl_adc.out...
build/test/out/appl_adc.o: In function 
0097ppl_adc__voltage__get':
/home/dimon/projects/indicator_git/test_ceedling/../src/appl/appl_adc.c:101: undefined reference to 
0095_builtin_software_breakpoint'
collect2: error: ld returned 1 exit status

Nice: at least, our “xc.h” file is clearly used by appl_adc.c file, and now we need to provide actual implementation of __builtin_software_breakpoint(). The easiest way to do that is to mock it. So, add the following line to our test_appl_adc.c file:

#include "mock_xc.h"

Now, run tests, and they pass!

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  7
PASSED:  7
FAILED:  0
IGNORED: 0

And we can write one more test: let's check that __builtin_software_breakpoint() is called if we pass illegal channel number:

void test_voltage_get_wrong_channel_number(void)
{
    //-- we expect __builtin_software_breakpoint() to be called ...
    __builtin_software_breakpoint_Expect();
 
    //-- ... when we call appl_adc__voltage__get() with illegal
    //   channel number.
    appl_adc__voltage__get(
            APPL_ADC_CH_CNT
            );
}

And tests pass again:

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  8
PASSED:  8
FAILED:  0
IGNORED: 0

You're encouraged to verify that if we remove a call to __builtin_software_breakpoint() in case of illegal channel number, tests will fail.

Note: you can get everything done by using the prepared repository. Type there: git checkout v0.12.

Testing on host machine is quite convenient: running tests is just a matter of a few keystrokes, tests run fast, and we get results almost immediately. But we should take some care, since the architectures are different.

As already discussed above, different compilers have some built-in functions. Apart from this, the memory alignment is often different: at 8-bit MCUs, the alignment is 1 byte, but on your host machine it's usually 4 or 8 bytes (depending on your architecture). So, we have to multi-target our applications.

I often find myself creating a file like my_crosscompiler.h, in which I declare some things depending on the compiler being used. For example, if I need some structure to be packed, I have to use compiler-specific attribute. So it may look like this:

#if defined(__XC8__)
//-- no need for "packed" attr on 8-bit MCU
#   define __MY_CROSS_ATTR_PACKED
#elif defined(__GNUC__)
#   define __MY_CROSS_ATTR_PACKED   __attribute__((packed))
#endif

And in the application code, I use the macro __MY_CROSS_ATTR_PACKED.

This way, we can write code that works both on target MCU as well as on the host machine. Of course, it takes additional effort and time, but so do tests in general. I spend a lot of time writing tests these days. It pays off very well.

Writing tests code is often considered as a tedious process, and I can't entirely disagree. However, I always encourage myself to find some new ways to test things, instead of repeatedly test this and that.

As an example, consider the EEPROM module (Electrically Erasable Programmable Read-Only Memory). We will most likely end up with MCU- or board-specific module bsp_eeprom which can just read plain data to and from specified addresses. As it is heavily hardware-dependent, we can't test it on the host machine.

In addition to bsp_eeprom, it's convenient to have application-dependent module like appl_eeprom, which should have functions to write or read some application entities to and from EEPROM. Of course, these application-dependent functions call bsp_eeprom__... functions underneath. For example, we might have the following functions to read/write the multiplier of each particular ADC channel:

int16_t appl_eeprom__adc_clb_mul__get(enum E_ApplAdcChannel channel);
void appl_eeprom__adc_clb_mul__set(enum E_ApplAdcChannel channel, int16_t mul);

If the application is rather large, there may be tons of functions like that. It is very tedious to test them all separately.

Instead, we may think about the most easy mistake to make. For things like the appl_eeprom module, it is the copy-paste mistake: when we have lots of similar functions (in fact, they all call the same bsp_eeprom functions, but for different addresses), it is easy to copy and paste from one function to another, and it is equally easy to forget to adjust the pasted code properly.

So, I often use the technique like this: define stub callbacks for bsp_eeprom functions, which just check if the given area is “clean”, and if it is, then fill this area with some predefined data (make it “dirty”). If the area is already “dirty”, then error is generated.

Then, perform “write” test: I call every “write” function from appl_eeprom module with all allowed arguments, and after that, the whole working region of EEPROM should be “dirty”, without holes. And, as I said before, each callback also checks whether the region it is going to write is clean. This way, we can easily eliminate these “copy-paste” problems: if some function writes to wrong area, we will end up with overwritten data and “holes”, which will be caught by our tests.

And, of course, exactly the same test should be done for “read” functions.

It is much more fun (and fast) than test each and every function separately, and in the end we'll have tests that are reliable enough.

The tools by guys from ThrowTheSwitch.org allow us to test our C code almost painlessly. Thank you, guys!

I hope this article helps you to get somewhat big picture about Ceedling and its companions, and I encourage you to examine the documentation, which is quite concise.

The easiest way to get documentation of all components in one place is to create new ceedling project by executing:

$ ceedling new my_project

And navigate to my_project/vendor/ceedling/docs directory, which contains several pdf files.

And again, if you feel serious about investing a great deal of time into testing your embedded designs (which is probably a good idea), consider reading the book Test Driven Development for Embedded C by James W. Grenning, which explains various testing methods, approaches and tools very thoroughly.

Let's write C code that doesn't suck!

You might as well be interested in:

• How I ended up writing a new real-time kernel
• Object-oriented techniques in C

Discuss on reddit: