run-cmake and run-vcpkg: Azure DevOps C++ build tasks for CMake and vcpkg
Also available as GitHub actions:
User Manual
- Introduction
- Quickstart
- The run-vpkg task
- The run-cmake task
- Tasks reference: all input parameters
- Samples
- Real world project samples
- Q&As
Developer Manual
Introduction
Build C++ software with vcpkg and CMake (either with CMakeLists.txt or CMakeSettings.json). Samples provided use both self-hosted or Microsoft hosted agent, using Docker and Pipeline Caching as well.
Quickstart
It is highly recommended to use vcpkg as a submodule. Here below the sample where vcpkg is a Git submodule:
# Sample when vcpkg is a submodule
# Cache/Restore the vcpkg's build artifacts.
- task: Cache@2
displayName: 'Cache vcpkg's artifacts'
inputs:
# As 'key' use the content of the response file, vcpkg's submodule fetched commit id and the platform name.
# The key must be one liner, each segment separated by pipe char, non-path segments enclosed by
# double quotes.
key: $(Build.SourcesDirectory)/vcpkg_x64-linux.txt | "$(Build.SourcesDirectory)/.git/modules/vcpkg/HEAD" | "$(Agent.OS)"
path: '$(Build.SourcesDirectory)/vcpkg'
- task: run-vcpkg@0
displayName: 'Run vcpkg'
inputs:
# Response file stored in source control, it provides the list of ports and triplet(s).
vcpkgArguments: @$(Build.SourcesDirectory)/vcpkg_x64-linux.txt
# Location of the vcpkg as submodule of the repository.
vcpkgDirectory: $(Build.SourcesDirectory)/vcpkg
- task: run-cmake@0
displayName: 'Run CMake with CMakeSettings.json'
inputs:
cmakeListsOrSettingsJson: 'CMakeSettingsJson'
# Use the vcpkg's toolchain file for CMake.
useVcpkgToolchainFile: true
# Build all configurations whose name starts with "Linux".
configurationRegexFilter: 'Linux.*'
Another sample when vcpkg is NOT a submodule (not recommended):
# Sample when vcpkg is NOT a submodule. The exact commit id to be fetched needs
# to be explicitly provided then (i.e. the value of vcpkgGitRef in this sample).
variables:
# Exact vcpkg's version to fetch, commig id or tag name.
vcpkgGitRef: 5a3b46e9e2d1aa753917246c2801e50aaabbbccc
# Cache/restore the vcpkg's build artifacts.
- task: Cache@2
displayName: 'Cache vcpkg's artifacts'
inputs:
key: $(Build.SourcesDirectory)/vcpkg_x64-linux.txt | "$(vcpkgGitRef)" | "$(Agent.OS)"
path: '$(Build.BinariesDirectory)/vcpkg'
- task: run-vcpkg@0
displayName: 'Run vcpkg'
inputs:
vcpkgArguments: @$(Build.SourcesDirectory)/vcpkg_x64-linux.txt
# Specify the exact commit id to fetch.
vcpkgGitCommitId: $(vcpkgGitRef)
# URL to fetch vcpkg from (default is the official one).
vcpkgGitURL: http://my.vcpkg.fork.git/
- task: run-cmake@0
displayName: 'Run CMake with CMakeSettings.json'
inputs:
cmakeListsOrSettingsJson: 'CMakeSettingsJson'
useVcpkgToolchainFile: true
# Build all configurations whose name starts with "Linux".
configurationRegexFilter: 'Linux.*'
The run-vcpkg task
The task completes the following steps:
- checks whether vcpkg is already located at the path specified by
vcpkgDirectory(e.g. vcpkg could be a submodule of the Git repository checked out along with the parent repository); - if vcpkg is not there, it uses Git to fetch vcpkg into that directory;
- if needed vcpkg is built;
- eventually vcpkg is launched to build and install the specified ports.
The task sets RUNVCPKG_VCPKG_ROOT task variable, which is automatically used by subsequent 'run-cmake' to consume the vcpkg's toolchain file.
Note: VCPKGRUN_VCPKG_ROOT is set by the task by the first of these conditions:
- the
vcpkgArgumentsinput task contains the--tripletargument; - the vcpkg's reponse file contains the
--tripletargument; vcpkgTripletinput task is set to a value.
In these cases, the first hit determines the content of the RUNVCPKG_VCPKG_ROOT variable.
In all other cases the variable is NOT set, and any run-cmake task instance is not able to reuse vcpkg's toolchain path nor the triplet.
Use vcpkg as a submodule of your Git repository
When using this vcpkg, be aware of how it works, specifically:
- a specific version of vcpkg must be used either locally and on build servers;
- a specific version of vcpkg is identified by the commit id or tag (e.g. release tag
2019.07) of the used vcpkg repository; - it not possible to choose which version of the port to install, instead it is the used version of vcpkg that establishes which version of the port is available;
To sum up, to keep a consistent development experience between local and remote build environments, it is highly suggested to use vcpkg as submodule of your Git repository; this way the version of vcpkg used is implied by the commit id specified by the submodule for vcpkg.
Use vcpkg's response file as parameter
vcpkg accepts a response file that contains its arguments. It is useful to store under source control a text file containing the list of ports to be installed. This helps to run the vcpkg the same exact way locally and on the build servers. For example if you want to run:
vcpkg install boost zlib:x64 libmodbus --triplet x64
it is instead possible to run
vcpkg install @response_file.txt
where response_file.txt contains (with no trailing whitespaces allowed):
boost
zlib:x64
libmodbus
--triplet
x64Caching vcpkg's artifacts
To minimize build time, the Cache task should be used to cache the entire vcpkg's directory (which is specified by vcpkgDirectory of 'run-vcpkg' task).
Note: since only the installed subdirectory and the vcpkg executable file should be cached, everything but those are excluded by the .artifactignore file generated by the vcpkg-run task.
The key provided to the Cache task must uniquely identify the following:
- the vcpkg version, which identifies also all the provided ports and their specific version;
- the used triplet and the list of ports to be built and installed;
- the platform and the toolset used when building ports;
The following sample key should grant most of the above:
key: $(vcpkgCommitId)| @$(Build.SourcesDirectory)/response_file_with_ports_and_triplet.txt | "$(Agent.OS)"where:
-
the
$(vcpkgCommitId)identifies a specific version of vcpkg by means of a Git commit id or tag. It is suggested to use vcpkg as a submodule of your repository, at the root for example, in such case the commit id is stored in file$(Build.SourcesDirectory)/.git/modules/vcpkg/HEAD. Note that a branch name should not be used, as it does not identify uniquely the version of vcpkg; -
the response file contains the list of ports along with the triplet, e.g.
sqlite3 --triplet x64-linuxor another identical examplesqlite3:x64-windows.The
--tripletargument specifies the default value when a port has not the triplet specified. -
$(Agent.OS)captures only the build agent platform. If needed, it might be useful to add further values in the key to uniquely identifies the toolset used when building.
Note: the key must be a one liner, it could be divided in segments with the pipe character, each non-file-path segment must be enclosed with double quotes. As documented, for file-path segments a hash value is computed, while the non-file-path segment values are used as is.
The 'run-vcpkg' flowchart
The run-cmake task
The 'run-cmake' task works with CMakeLists.txt and CMakeSettings.json.
It can leverage the previous execution of the 'run-vcpkg' task by using the RUNVCPKG_VCPKG_ROOT task variable to:
- set the vcpkg's toolchain file if requested, located at
$RUNVCPKG_VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake; - set the environment for the provided triplet when building with msvc on Windows (i.e. building in the environment created by launching
$RUNVCPKG_VCPKG_ROOT/vcpkg env);
The 'run-cmake' flowchart
The flowchart has two entry points as it could be used with a CMakeLists.txt or with a CMakeSettings.json file.
Note: The task does not use th e CMakeSettings.json's attribute called inheritEnvironments. Instead the triplet set by 'run-vcpkg' is used by the task to set up the environment variables where the commands are run. This is only useful when using MSVC, where the environment needs to set up for setting the right target (e.g. x86 or x64 or arm).
Note: The task ignores the buildRoot value specified in the CMakeSettings.json . It is possible to specify the CMake binary directory using the buildDirectory input parameter, which by default it is $(Build.ArtifactStagingDirectory)/<configuration name> .
Tasks reference: all input parameters
A complete reference of all the inputs of the tasks is provided.
Samples
Real world project samples
| project: Microsoft MSVC STL | |
|---|---|
| MSVC STL with cache, vcpkg submodule |
| project: Microsoft MSVC STL (fork) | |
|---|---|
| MSVC STL with cache, vcpkg submodule, self hosted agent Docker based | |
| MSVC STL with cache, vcpkg submodule, MS hosted agent Docker based |
| project: Microsoft cpprestsdk | |
|---|---|
| macOS | |
| Linux/Ubuntu | |
| vs2017 | |
| vs2019 |
| project: evoke | |
|---|---|
| macOS/Linux/Windows |
| project: sysmakeshift | |
|---|---|
| macOS/Linux/Windows |
| project: helgoboss-midi | |
|---|---|
| macOS/Linux/Windows |
| project: vct | |
|---|---|
| macOS/Linux/Windows |
| project: CppOpenGLWebAssemblyCMake | |
|---|---|
| webassembly(with Docker)/Linux/macOS/Windows |
| project: DisCPP | |
|---|---|
| Linux/Windows |
Questions and Answers
Why not one single task?
Because you could use vcpkg without CMake. Or you could use CMake without vcpkg.
Would creating an ad-hoc bash/powershell script be easier?
Absolutely! Anyone can use this task as an inspiration for writing their own scripts to suite specific needs. The purpose of the tasks is to streamline and to simplify the usage of vcpkg and CMake on build servers.
Why the 'run-vcpkg' task runs vcpkg in any case, even when cached port files are restored?
Indeed it is not needed to run vcpkg when the cache is being restored, and you could use Cache task's cacheHitVar parameter to control the execution of the 'run-vcpkg' task. Currently 'run-vcpkg' task defensively runs vcpkg as a sanity check: it should output in the build log that all the requested ports are installed already, spending a neglibile amount of time.
Developers information
Contributing
The software is provided as is, there is no warranty of any kind. All users are encouraged to get the source code and improve the tasks with fixes and new features.
Please read CONTRIBUTING.md for detailed development instructions.
License
All the content in this repository, of the extension and of the 'run-cmake' and 'run-vcpkg' tasks are licensed under the MIT License.
Copyright (c) 2019-2020 Luca Cappa