ExternalProject_Add The ExternalProject_Add() function creates a custom target to drive download, update/patch, configure, build, install and test steps of an external project:
ExternalProject_Add(<name> [<option>...])
The individual steps within the process can be driven independently if required (e.g. for CDash submission) and extra custom steps can be defined, along with the ability to control the step dependencies. The directory structure used for the management of the external project can also be customized. The function supports a large number of options which can be used to tailor the external project behavior.
Most of the time, the default directory layout is sufficient. It is largely an implementation detail that the main project usually doesn’t need to change. In some circumstances, however, control over the directory layout can be useful or necessary. The directory options are potentially more useful from the point of view that the main build can use the ExternalProject_Get_Property() command to retrieve their values, thereby allowing the main project to refer to build artifacts of the external project.
PREFIX <dir> TMP_DIR <dir> STAMP_DIR <dir> DOWNLOAD_DIR <dir> SOURCE_DIR directly instead.SOURCE_DIR <dir> Source directory into which downloaded contents will be unpacked, or for non-URL download methods, the directory in which the repository should be checked out, cloned, etc. If no download method is specified, this must point to an existing directory where the external project has already been unpacked or cloned/checked out.
Note
If a download method is specified, any existing contents of the source directory may be deleted. Only the URL download method checks whether this directory is either missing or empty before initiating the download, stopping with an error if it is not empty. All other download methods silently discard any previous contents of the source directory.
BINARY_DIR <dir> BUILD_IN_SOURCE is enabled.INSTALL_DIR <dir> <INSTALL_DIR> placeholder. This does not actually configure the external project to install to the given prefix. That must be done by passing appropriate arguments to the external project configuration step, e.g. using <INSTALL_DIR>.If any of the above ..._DIR options are not specified, their defaults are computed as follows. If the PREFIX option is given or the EP_PREFIX directory property is set, then an external project is built and installed under the specified prefix:
TMP_DIR = <prefix>/tmp STAMP_DIR = <prefix>/src/<name>-stamp DOWNLOAD_DIR = <prefix>/src SOURCE_DIR = <prefix>/src/<name> BINARY_DIR = <prefix>/src/<name>-build INSTALL_DIR = <prefix>
Otherwise, if the EP_BASE directory property is set then components of an external project are stored under the specified base:
TMP_DIR = <base>/tmp/<name> STAMP_DIR = <base>/Stamp/<name> DOWNLOAD_DIR = <base>/Download/<name> SOURCE_DIR = <base>/Source/<name> BINARY_DIR = <base>/Build/<name> INSTALL_DIR = <base>/Install/<name>
If no PREFIX, EP_PREFIX, or EP_BASE is specified, then the default is to set PREFIX to <name>-prefix. Relative paths are interpreted with respect to CMAKE_CURRENT_BINARY_DIR at the point where ExternalProject_Add() is called.
A download method can be omitted if the SOURCE_DIR option is used to point to an existing non-empty directory. Otherwise, one of the download methods below must be specified (multiple download methods should not be given) or a custom DOWNLOAD_COMMAND provided.
DOWNLOAD_COMMAND <cmd>... generator expressions are supported). If this option is specified, all other download options will be ignored. Providing an empty string for <cmd> effectively disables the download step.URL <url1> [<url2>...] file(DOWNLOAD) command. A local filesystem path may refer to either an existing directory or to an archive file, whereas a URL is expected to point to a file which can be treated as an archive. When an archive is used, it will be unpacked automatically unless the DOWNLOAD_NO_EXTRACT option is set to prevent it. The archive type is determined by inspecting the actual content rather than using logic based on the file extension.URL_HASH <algo>=<hashValue> <algo>=<hashValue> where algo can be any of the hashing algorithms supported by the file() command. Specifying this option is strongly recommended for URL downloads, as it ensures the integrity of the downloaded content. It is also used as a check for a previously downloaded file, allowing connection to the remote location to be avoided altogether if the local directory already has a file from an earlier download that matches the specified hash.URL_MD5 <md5> URL_HASH MD5=<md5>.DOWNLOAD_NAME <fname> ExternalProject module.DOWNLOAD_NO_EXTRACT <bool> <DOWNLOADED_FILE> in subsequent steps or as the property DOWNLOADED_FILE with the ExternalProject_Get_Property() command.DOWNLOAD_NO_PROGRESS <bool> TIMEOUT <seconds> HTTP_USERNAME <username> HTTP_PASSWORD <password> HTTP_HEADER <header1> [<header2>...] TLS_VERIFY <bool> CMAKE_TLS_VERIFY variable (see file(DOWNLOAD)). If that is also not set, certificate verification will not be performed. In situations where URL_HASH cannot be provided, this option can be an alternative verification measure.TLS_CAINFO <file> TLS_VERIFY is enabled. If this option is not specified, the value of the CMAKE_TLS_CAINFO variable will be used instead (see file(DOWNLOAD))NETRC <level> Specify whether the .netrc file is to be used for operation. If this option is not specified, the value of the CMAKE_NETRC variable will be used instead (see file(DOWNLOAD)) Valid levels are:
IGNORED OPTIONAL REQUIRED NETRC_FILE <file> NETRC level is OPTIONAL or REQUIRED. If this option is not specified, the value of the CMAKE_NETRC_FILE variable will be used instead (see file(DOWNLOAD))NOTE: A git version of 1.6.5 or later is required if this download method is used.
GIT_REPOSITORY <url> git command may be used.GIT_TAG <tag> Git branch name, tag or commit hash. Note that branch names and tags should generally be specified as remote names (i.e. origin/myBranch rather than simply myBranch). This ensures that if the remote end has its tag moved or branch rebased or history rewritten, the local clone will still be updated correctly. In general, however, specifying a commit hash should be preferred for a number of reasons:
git fetch needs to be performed to check for changes each time CMake is re-run. This can result in a significant speed up if many external projects are being used.GIT_REMOTE_NAME <name> origin.GIT_SUBMODULES <module>... GIT_SHALLOW <bool> git clone operation will be given the --depth 1 option. This performs a shallow clone, which avoids downloading the whole history and instead retrieves just the commit denoted by the GIT_TAG option.GIT_PROGRESS <bool> git clone operation to report its progress by passing it the --progress option. Without this option, the clone step for large projects may appear to make the build stall, since nothing will be logged until the clone operation finishes. While this option can be used to provide progress to prevent the appearance of the build having stalled, it may also make the build overly noisy if lots of external projects are used.GIT_CONFIG <option1> [<option2>...] git clone. Each option listed will be transformed into its own --config <option> on the git clone command line, with each option required to be in the form key=value.SVN_REPOSITORY <url> SVN_REVISION -r<rev> SVN_USERNAME <username> SVN_PASSWORD <password> SVN_TRUST_CERT <bool> --trust-server-cert option is passed to the svn checkout and update commands.HG_REPOSITORY <url> HG_TAG <tag> CVS_REPOSITORY <cvsroot> CVS_MODULE <mod> CVS_TAG <tag> Whenever CMake is re-run, by default the external project’s sources will be updated if the download method supports updates (e.g. a git repository would be checked if the GIT_TAG does not refer to a specific commit).
UPDATE_COMMAND <cmd>... generator expressions.UPDATE_DISCONNECTED <bool> When enabled, this option causes the update step to be skipped. It does not, however, prevent the download step. The update step can still be added as a step target (see ExternalProject_Add_StepTargets()) and called manually. This is useful if you want to allow developers to build the project when disconnected from the network (the network may still be needed for the download step though).
When this option is present, it is generally advisable to make the value a cache variable under the developer’s control rather than hard-coding it. If this option is not present, the default value is taken from the EP_UPDATE_DISCONNECTED directory property. If that is also not defined, updates are performed as normal. The EP_UPDATE_DISCONNECTED directory property is intended as a convenience for controlling the UPDATE_DISCONNECTED behavior for an entire section of a project’s directory hierarchy and may be a more convenient method of giving developers control over whether or not to perform updates (assuming the project also provides a cache variable or some other convenient method for setting the directory property).
PATCH_COMMAND <cmd>... GIT_TAG will not discard changes from a previous patch, but the patch command will be called again after updating to the new tag.The configure step is run after the download and update steps. By default, the external project is assumed to be a CMake project, but this can be overridden if required.
CONFIGURE_COMMAND <cmd>... CONFIGURE_COMMAND option must be used to override this behavior (generator expressions are supported). For projects that require no configure step, specify this option with an empty string as the command to execute.CMAKE_COMMAND /.../cmake CONFIGURE_COMMAND.CMAKE_GENERATOR <gen> CONFIGURE_COMMAND option.CMAKE_GENERATOR_PLATFORM <platform> CMAKE_GENERATOR_PLATFORM). It is an error to provide this option without the CMAKE_GENERATOR option.CMAKE_GENERATOR_TOOLSET <toolset> CMAKE_GENERATOR_TOOLSET). It is an error to provide this option without the CMAKE_GENERATOR option.CMAKE_GENERATOR_INSTANCE <instance> CMAKE_GENERATOR_INSTANCE). It is an error to provide this option without the CMAKE_GENERATOR option.CMAKE_ARGS <arg>... cmake command line. They can be any argument the cmake command understands, not just cache values defined by -D... arguments (see also CMake Options). In addition, arguments may use generator expressions.CMAKE_CACHE_ARGS <arg>... -Dvar:STRING=value, which are then transformed into CMake set() commands with the FORCE option used. These set() commands are written to a pre-load script which is then applied using the cmake -C command line option. Arguments may use generator expressions.CMAKE_CACHE_DEFAULT_ARGS <arg>... CMAKE_CACHE_ARGS option except the set() commands do not include the FORCE keyword. This means the values act as initial defaults only and will not override any variables already set from a previous run. Use this option with care, as it can lead to different behavior depending on whether the build starts from a fresh build directory or re-uses previous build contents.SOURCE_SUBDIR <dir> CONFIGURE_COMMAND option is specified, the configure step assumes the external project has a CMakeLists.txt file at the top of its source tree (i.e. in SOURCE_DIR). The SOURCE_SUBDIR option can be used to point to an alternative directory within the source tree to use as the top of the CMake source tree instead. This must be a relative path and it will be interpreted as being relative to SOURCE_DIR.If the configure step assumed the external project uses CMake as its build system, the build step will also. Otherwise, the build step will assume a Makefile-based build and simply run make with no arguments as the default build step. This can be overridden with custom build commands if required.
BUILD_COMMAND <cmd>... generator expressions are supported). If this option is not given, the default build command will be chosen to integrate with the main build in the most appropriate way (e.g. using recursive make for Makefile generators or cmake --build if the project uses a CMake build). This option can be specified with an empty string as the command to make the build step do nothing.BUILD_IN_SOURCE <bool> BINARY_DIR option should not be specified if building in-source.BUILD_ALWAYS <bool> SOURCE_DIR is used without a download method and developers might modify the sources in SOURCE_DIR).BUILD_BYPRODUCTS <file>... BYPRODUCTS to the build step’s own underlying call to add_custom_command().If the configure step assumed the external project uses CMake as its build system, the install step will also. Otherwise, the install step will assume a Makefile-based build and simply run make install as the default build step. This can be overridden with custom install commands if required.
INSTALL_COMMAND <cmd>... TEST_BEFORE_INSTALL option below). The external project’s install rules are not part of the main project’s install rules, so if anything from the external project should be installed as part of the main build, these need to be specified in the main build as additional install() commands. The default install step builds the install target of the external project, but this can be overridden with a custom command using this option (generator expressions are supported). Passing an empty string as the <cmd> makes the install step do nothing.The test step is only defined if at least one of the following TEST_... options are provided.
TEST_COMMAND <cmd>... generator expressions are supported). If this option is not given, the default behavior of the test step is to build the external project’s own test target. This option can be specified with <cmd> as an empty string, which allows the test step to still be defined, but it will do nothing. Do not specify any of the other TEST_... options if providing an empty string as the test command, but prefer to omit all TEST_... options altogether if the test step target is not needed.TEST_BEFORE_INSTALL <bool> TEST_AFTER_INSTALL <bool> TEST_BEFORE_INSTALL and TEST_AFTER_INSTALL are enabled, the latter is silently ignored.TEST_EXCLUDE_FROM_MAIN <bool> Each of the following LOG_... options can be used to wrap the relevant step in a script to capture its output to files. The log files will be created in the STAMP_DIR directory with step-specific file names.
LOG_DOWNLOAD <bool> LOG_UPDATE <bool> LOG_CONFIGURE <bool> LOG_BUILD <bool> LOG_INSTALL <bool> LOG_TEST <bool> Steps can be given direct access to the terminal in some cases. Giving a step access to the terminal may allow it to receive terminal input if required, such as for authentication details not provided by other options. With the Ninja generator, these options place the steps in the console job pool. Each step can be given access to the terminal individually via the following options:
USES_TERMINAL_DOWNLOAD <bool> USES_TERMINAL_UPDATE <bool> USES_TERMINAL_CONFIGURE <bool> USES_TERMINAL_BUILD <bool> USES_TERMINAL_INSTALL <bool> USES_TERMINAL_TEST <bool> DEPENDS <targets>... DEPENDS option is the most convenient way to ensure all of those steps depend on the other targets. Simply doing add_dependencies(<name> <targets>) will not make any of the steps dependent on <targets>.EXCLUDE_FROM_ALL <bool> STEP_TARGETS <step-target>... EP_STEP_TARGETS directory property. See ExternalProject_Add_Step() below for further discussion of the effects of this option.INDEPENDENT_STEP_TARGETS <step-target>... EP_INDEPENDENT_STEP_TARGETS directory property. This option is mostly useful for allowing individual steps to be driven independently, such as for a CDash setup where each step should be initiated and reported individually rather than as one whole build. See ExternalProject_Add_Step() below for further discussion of the effects of this option.LIST_SEPARATOR <sep> ..._COMMAND options, replace ; with <sep> in the specified command lines. This can be useful where list variables may be given in commands where they should end up as space-separated arguments (<sep> would be a single space character string in this case).COMMAND <cmd>... Any of the other ..._COMMAND options can have additional commands appended to them by following them with as many COMMAND ... options as needed (generator expressions are supported). For example:
ExternalProject_Add(example
... # Download options, etc.
BUILD_COMMAND ${CMAKE_COMMAND} -E echo "Starting $<CONFIG> build"
COMMAND ${CMAKE_COMMAND} --build <BINARY_DIR> --config $<CONFIG>
COMMAND ${CMAKE_COMMAND} -E echo "$<CONFIG> build complete"
)
It should also be noted that each build step is created via a call to ExternalProject_Add_Step(). See that command’s documentation for the automatic substitutions that are supported for some options.
ExternalProject_Get_Property The ExternalProject_Get_Property() function retrieves external project target properties:
ExternalProject_Get_Property(<name> <prop1> [<prop2>...])
The function stores property values in variables of the same name. Property names correspond to the keyword argument names of ExternalProject_Add(). For example, the source directory might be retrieved like so:
ExternalProject_Get_property(myExtProj SOURCE_DIR)
message("Source dir of myExtProj = ${SOURCE_DIR}")
The ExternalProject_Add() function on its own is often sufficient for incorporating an external project into the main build. Certain scenarios require additional work to implement desired behavior, such as adding in a custom step or making steps available as manually triggerable targets. The ExternalProject_Add_Step(), ExternalProject_Add_StepTargets() and ExternalProject_Add_StepDependencies functions provide the lower level control needed to implement such step-level capabilities.
ExternalProject_Add_Step The ExternalProject_Add_Step() function specifies an additional custom step for an external project defined by an earlier call to ExternalProject_Add():
ExternalProject_Add_Step(<name> <step> [<option>...])
<name> is the same as the name passed to the original call to ExternalProject_Add(). The specified <step> must not be one of the pre-defined steps (mkdir, download, update, skip-update, patch, configure, build, install or test). The supported options are:
COMMAND <cmd>... generator expressions are supported). This option can be repeated multiple times to specify multiple commands to be executed in order.COMMENT "<text>..." DEPENDEES <step>... DEPENDERS <step>... DEPENDS <file>... BYPRODUCTS <file>... BYPRODUCTS option to the add_custom_command() used to implement the custom step internally.ALWAYS <bool> EXCLUDE_FROM_MAIN <bool> WORKING_DIRECTORY <dir> CMAKE_CURRENT_BINARY_DIR at the point where ExternalProject_Add_Step() was called.LOG <bool> STAMP_DIR.USES_TERMINAL <bool> The command line, comment, working directory and byproducts of every standard and custom step are processed to replace the tokens <SOURCE_DIR>, <SOURCE_SUBDIR>, <BINARY_DIR>, <INSTALL_DIR> <TMP_DIR>, <DOWNLOAD_DIR> and <DOWNLOADED_FILE> with their corresponding property values defined in the original call to ExternalProject_Add().
ExternalProject_Add_StepTargets The ExternalProject_Add_StepTargets() function generates targets for the steps listed. The name of each created target will be of the form <name>-<step>:
ExternalProject_Add_StepTargets(<name> [NO_DEPENDS] <step1> [<step2>...])
Creating a target for a step allows it to be used as a dependency of another target or to be triggered manually. Having targets for specific steps also allows them to be driven independently of each other by specifying targets on build command lines. For example, you may be submitting to a sub-project based dashboard where you want to drive the configure portion of the build, then submit to the dashboard, followed by the build portion, followed by tests. If you invoke a custom target that depends on a step halfway through the step dependency chain, then all the previous steps will also run to ensure everything is up to date.
If the NO_DEPENDS option is specified, the step target will not depend on the dependencies of the external project (i.e. on any dependencies of the <name> custom target created by ExternalProject_Add()). This is usually safe for the download, update and patch steps, since they do not typically require that the dependencies are updated and built. Using NO_DEPENDS for any of the other pre-defined steps, however, may break parallel builds. Only use NO_DEPENDS where it is certain that the named steps genuinely do not have dependencies. For custom steps, consider whether or not the custom commands require the dependencies to be configured, built and installed.
Internally, ExternalProject_Add() calls ExternalProject_Add_Step() to create each step. If any STEP_TARGETS or INDEPENDENT_STEP_TARGETS were specified, then ExternalProject_Add_StepTargets() will also be called after ExternalProject_Add_Step(). INDEPENDENT_STEP_TARGETS have the NO_DEPENDS option set, whereas STEP_TARGETS do not. Other than that, the two options result in ExternalProject_Add_StepTargets() being called in the same way. Even if a step is not mentioned in either of those two options, ExternalProject_Add_StepTargets() can still be called later to manually define a target for the step.
The STEP_TARGETS and INDEPENDENT_STEP_TARGETS options for ExternalProject_Add() are generally the easiest way to ensure targets are created for specific steps of interest. For custom steps, ExternalProject_Add_StepTargets() must be called explicitly if a target should also be created for that custom step. An alternative to these two options is to populate the EP_STEP_TARGETS and EP_INDEPENDENT_STEP_TARGETS directory properties. These act as defaults for the step target options and can save having to repeatedly specify the same set of step targets when multiple external projects are being defined.
ExternalProject_Add_StepDependencies The ExternalProject_Add_StepDependencies() function can be used to add dependencies to a step. The dependencies added must be targets CMake already knows about (these can be ordinary executable or library targets, custom targets or even step targets of another external project):
ExternalProject_Add_StepDependencies(<name> <step> <target1> [<target2>...])
This function takes care to set both target and file level dependencies and will ensure that parallel builds will not break. It should be used instead of add_dependencies() whenever adding a dependency for some of the step targets generated by the ExternalProject module.
The following example shows how to download and build a hypothetical project called FooBar from github:
include(ExternalProject) ExternalProject_Add(foobar GIT_REPOSITORY [email protected]:FooCo/FooBar.git GIT_TAG origin/release/1.2.3 )
For the sake of the example, also define a second hypothetical external project called SecretSauce, which is downloaded from a web server. Two URLs are given to take advantage of a faster internal network if available, with a fallback to a slower external server. The project is a typical Makefile project with no configure step, so some of the default commands are overridden. The build is only required to build the sauce target:
find_program(MAKE_EXE NAMES gmake nmake make)
ExternalProject_Add(secretsauce
URL http://intranet.somecompany.com/artifacts/sauce-2.7.tgz
https://www.somecompany.com/downloads/sauce-2.7.zip
URL_HASH MD5=d41d8cd98f00b204e9800998ecf8427e
CONFIGURE_COMMAND ""
BUILD_COMMAND ${MAKE_EXE} sauce
)
Suppose the build step of secretsauce requires that foobar must already be built. This could be enforced like so:
ExternalProject_Add_StepDependencies(secretsauce build foobar)
Another alternative would be to create a custom target for foobar’s build step and make secretsauce depend on that rather than the whole foobar project. This would mean foobar only needs to be built, it doesn’t need to run its install or test steps before secretsauce can be built. The dependency can also be defined along with the secretsauce project:
ExternalProject_Add_StepTargets(foobar build)
ExternalProject_Add(secretsauce
URL http://intranet.somecompany.com/artifacts/sauce-2.7.tgz
https://www.somecompany.com/downloads/sauce-2.7.zip
URL_HASH MD5=d41d8cd98f00b204e9800998ecf8427e
CONFIGURE_COMMAND ""
BUILD_COMMAND ${MAKE_EXE} sauce
DEPENDS foobar-build
)
Instead of calling ExternalProject_Add_StepTargets(), the target could be defined along with the foobar project itself:
ExternalProject_Add(foobar GIT_REPOSITORY [email protected]:FooCo/FooBar.git GIT_TAG origin/release/1.2.3 STEP_TARGETS build )
If many external projects should have the same set of step targets, setting a directory property may be more convenient. The build step target could be created automatically by setting the EP_STEP_TARGETS directory property before creating the external projects with ExternalProject_Add():
set_property(DIRECTORY PROPERTY EP_STEP_TARGETS build)
Lastly, suppose that secretsauce provides a script called makedoc which can be used to generate its own documentation. Further suppose that the script expects the output directory to be provided as the only parameter and that it should be run from the secretsauce source directory. A custom step and a custom target to trigger the script can be defined like so:
ExternalProject_Add_Step(secretsauce docs COMMAND <SOURCE_DIR>/makedoc <BINARY_DIR> WORKING_DIRECTORY <SOURCE_DIR> COMMENT "Building secretsauce docs" ALWAYS TRUE EXCLUDE_FROM_MAIN TRUE ) ExternalProject_Add_StepTargets(secretsauce docs)
The custom step could then be triggered from the main build like so:
cmake --build . --target secretsauce-docs
© 2000–2018 Kitware, Inc. and Contributors
Licensed under the BSD 3-clause License.
https://cmake.org/cmake/help/v3.12/module/ExternalProject.html