> ## Content Index > Fetch the complete content index at: https://www.smallstepsystems.com/llms.txt > Use this file to discover other available public pages before exploring further. # Building a CI Pipeline with CTest and CDash - URL: https://www.smallstepsystems.com/building-a-ci-pipeline-with-ctest-and-cdash/ - Published: 2022-05-02T19:08:45.000Z - Updated: 2026-01-03T21:15:58.000Z - Author: Burkhard Stubert - Tags: Continuous Delivery We can use CMake, CTest and CDash to build a basic Continuous Integration (CI) pipeline. The pipeline builds the applications, runs the tests, collects coverage information and uploads all the results to a dashboard on a web server. All this is controlled by one CMake script executed by CTest. This pipeline could be triggered by a CI system like Github Actions or Jenkins, when developers try to integrate their code changes into main (e.g., by push or by merge request). Although the CTest pipeline could update the git repository (see the [CTest Update Step](https://cmake.org/cmake/help/latest/manual/ctest.1.html?ref=smallstepsystems.com#ctest-update-step)), we let the CI system do this. If all tests pass, the CI system automatically integrates the changes into the main branch. The CI system could run more checks before integration. For example, it could check that all code changes are covered by tests or that no cyclic dependencies between libraries exist. Integrating the CTest pipeline into a CI system will be the topic of a later post. ## Installing the CDash Server CDash is web server. When we run the pipeline, CTest uploads the build, test and coverage results to CDash. We can access the dashboard of the results from our browser. We can [install CDash](https://github.com/Kitware/CDash/blob/master/docs/docker.md?ref=smallstepsystems.com) on our own server by spinning up a Docker container. ``` $ git clone https://github.com/Kitware/CDash $ cd CDash ``` We clone the CDash repository, change to the CDash directory and edit the `environment` section of the file `docker-compose.yml`. Of course, we adapt the secrets and email addresses to our needs. ``` environment: CDASH_ROOT_ADMIN_PASS: secret CDASH_STATIC_USERS: | USER jdoe@acme.com jdoe_secret ADMIN admin@example.org admin_secret ``` We start the Docker container with the CDash server in the background. The first run takes a couple of minutes, because Docker builds the CDash server. Subsequent runs start the CDash server right away without building. ``` $ docker-compose up -d ``` After starting CDash for the first time, we configure it with the values from `docker-compose.yml`. ``` $ docker-compose run --rm cdash install configure ``` We don't need this command for subsequent starts of CDash. We stop the CDash server with the command ``` $ docker-compose down ``` ## Creating a Project on the CDash Server We enter `localhost:8080` in the address field of our browser to access the CDash dashboard. We click on *Login* in the upper left corner of the page. We enter the email address and password of the USER from `docker-compose.yml` above to log in. We click on *My CDash* in the upper left corner of the page to reach our dashboard. The dashboard shows sections *My Projects*, *Authentication Tokens* and *Administration*. We choose the action *Administration > Create new project*. As a minimum we fill out the project *Name* and the *Home URL*. For example: ``` Name: TerminalApps Home URL: http://localhost:8080 ``` We click on the blue right arrow multiple times to reach the *Miscellaneous* page. There we click the *Create Project* button. The CDash client, CTest, needs the name and the home URL to upload the results of a pipeline run to CDash. ## Running a CI Pipeline with CTest We use a CTest script, `TerminalAppsPipeline.cmake`, to describe a CI pipeline. I use the example from *Chapter 25.9.3 CTest Configuration* of Craig Scott's book [Professional CMake: A Practical Guide (11th edition)](https://crascit.com/professional-cmake/?ref=smallstepsystems.com) as the starting point for my script. The script is located in the top-level directory of the `terminal-apps` repository. ``` /path/to/ # working directory terminal-apps/ # source directory TerminalAppsPipeline.cmake CTestConfig.cmake build-test/ # binary directory ``` In the working directory, we run the pipeline with the command ``` $ ctest -S terminal-apps/TerminalAppsPipeline.cmake ``` The command reads the source files from directory `terminal-apps` and writes all the build artifacts to `build-test`. It deletes the contents of build-test from the last run, configures the build with CMake, builds the applications and the unit tests with Ninja, runs the unit tests, generates the coverage information and uploads the results to CDash. It runs the complete pipeline as specified in `TerminalAppsPipeline.cmake`. ``` # Test configuration include(${CTEST_SCRIPT_DIRECTORY}/CTestConfig.cmake) # General settings site_name(CTEST_SITE) set(CTEST_BUILD_NAME "linux-x86_64-gcc") set(CTEST_SOURCE_DIRECTORY "${CTEST_SCRIPT_DIRECTORY}") set(CTEST_BINARY_DIRECTORY "${CTEST_SCRIPT_DIRECTORY}/../build-test") # Build settings set(CTEST_CMAKE_GENERATOR Ninja) set(CTEST_CONFIGURATION_TYPE RelWithDebInfo) set(configureOpts "-DCMAKE_CXX_FLAGS_INIT=--coverage" "-DCMAKE_PREFIX_PATH=/public/Qt/host-qt6.2.4" ) # Coverage settings set(CTEST_COVERAGE_COMMAND gcov) set(CTEST_COVERAGE_EXTRA_FLAGS "--demangled-names") set(CTEST_CUSTOM_COVERAGE_EXCLUDE "_autogen;machine/tests") # Step 1 - Clean previous pipeline run ctest_empty_binary_directory(${CTEST_BINARY_DIRECTORY}) # Step 2 - Configure ctest_start(Experimental) ctest_configure(OPTIONS "${configureOpts}") ctest_submit(PARTS Start Configure) # Step 3 - Build ctest_build(PARALLEL_LEVEL 16) ctest_submit(PARTS Build) # Step 4 - Run unit tests ctest_test(PARALLEL_LEVEL 16) ctest_submit(PARTS Test) # Step 5 - Collect coverage information ctest_coverage() ctest_submit(PARTS Coverage Done) ``` `CTEST_SCRIPT_DIRECTORY` points to the working directory, which contains the pipeline script. The *Test Configuration* tells the pipeline script (the CDash client) how to communicate with the CDash server. The file [CTestConfig.cmake](https://cmake.org/cmake/help/latest/guide/tutorial/Adding%20Support%20for%20a%20Testing%20Dashboard.html?highlight=ctestconfig%20cmake&ref=smallstepsystems.com) in the working directory contains the test configuration. ``` set(CTEST_PROJECT_NAME "TerminalApps") set(CTEST_SUBMIT_URL "http://localhost:8080/submit.php?project=${CTEST_PROJECT_NAME}") set(CTEST_USE_LAUNCHERS YES) ``` We must set `CTEST_PROJECT_NAME` to the same name that we used when we created the project on the CDash server. It's `TerminalApps` in our case. `CTEST_SUBMIT_URL` is the URL to which CTest uploads the results of the pipeline run. The flag `CTEST_USE_LAUNCHERS` writes the command for each pipeline step into the CDash logs, which is useful for debugging. Let us go back to the pipeline script. The *General Settings* describe the computer, on which the pipeline runs, by the site name (the host name by default) and the build name `linux-x86_64-gcc`. It also sets `CTEST_SOURCE_DIRECTORY` and `CTEST_BINARY_DIRECTORY` to `/path/to/terminal-apps` and `/path/to/build-test`, as described at the beginning of this section. The source and binary directories are used by the pipeline steps. The *Build Settings* determine `Ninja` as the CMake generator. Setting `CTEST_CONFIGURATION_TYPE` to `RelWithDebInfo` and `CMAKE_CXX_FLAGS_INIT` to `--coverage` ensures that the compiler instruments the binaries for coverage collection in Step 5\. We tell the Configure step where to find Qt by setting `CMAKE_PREFIX_PATH` to the Qt installation directory (e.g., `/public/Qt/host-qt6.2.4`). In the *Coverage Settings*, we define that `gcov` is our tool for collecting coverage information and that we want human-readable symbol names (option `--demangle-names`). The option CTEST\_CUSTOM\_COVERAGE\_EXCLUDE takes a semicolon-separated list of regular expressions excluding file and directory paths from the coverage result. The regular expression `_autogen` excludes files generated by moc and rcc. The build places these files in directories containing the substring \_autogen. The regular expression `machine/tests` excludes the directories containing the test cases. This leaves us with the coverage information for the source files in `machine/src`. The remaining steps specify the steps of the CI pipeline. - *Step 1* deletes all the files and directories in the binary directory from the previous pipeline run. - *Step 2* tells the CDash server that a new `Experimental` pipeline run starts (`ctest_start`), configures the build by running CMake (`ctest_configure`), and sends the results of the `Start` and `Configure` steps to CDash (`ctest_submit`). As we define a custom pipeline, we pass `Experimental` as the model to `ctest_start` (see [Dashboard Client Modes](https://cmake.org/cmake/help/latest/manual/ctest.1.html?highlight=nightly&ref=smallstepsystems.com#dashboard-client-modes) for an explanation of the `Experimental`, `Continuous` and `Nightly` models). `Experimental` doesn't update the source code. - *Step 3* builds the CMake project including the applications, libraries and test executables (`ctest_build`). It performs the build on 16 cores in parallel. It submits the build results to CDash. - *Step 4* runs all the tests created with `add_test` on 16 cores in parallel (`ctest_test`) and submits the results to CDash. - *Step 5* collects the coverage information (`ctest_coverage`) and sends the results to CDash. Now, we are ready to run the pipeline from the working directory. ``` $ ctest -S terminal-apps/TerminalAppsPipeline.cmake Each . represents 1024 bytes of output .. Size of output: 1K Each symbol represents 1024 bytes of output. ......... Size of output: 9K ``` By default, CTest only prints the above progress information. We get more telling information with the option `-V` and a lot more information with `-VV`. CTest writes the results for each step, the coverage info for each header and source file, and the tool logs into subdirectories below `build-test/Testing`. ``` /path/to/ # working directory build-test/Testing/ 20220501-1536/ # -