Testing Foreman Ansible Modules¶
Foreman Ansible Modules are tested in different ways:
We currently only have unit tests for the
argument_spec translation helpers (see
All other code is tested by the integration tests.
Every module is tested using a playbook (
see tests/test_playbooks) against all supported Ansible versions plus the current
devel branch of Ansible.
Additionally, the modules which support check mode are tested with the latest Ansible release in check mode.
All tests require you to have run
make test-setup. This will install all test dependencies and prepare a configuration file (
The configuration file will need updates if you want to run tests against a live Foreman instance (see below).
To run the unit tests, execute:
$ make test-other
To run all integration tests, execute:
$ make test
To run a specific test or a set of tests, execute:
$ make test_global_parameter # single test $ make test TEST="-k 'organzation or global_parameter'" # select tests by expression (see `pytest -h`)
By default, tests are run with prerecorded server responses using VCRpy. When tests or modules are changed, those responses might not match anymore and you’ll have to re-record the interaction.
To be able to re-record, you need a running instance of Foreman, probably with Katello (use forklift if unsure).
You also need to update
tests/test_playbooks/vars/server.yml with the URL and credentials of said instance.
To re-record, execute:
$ make record_global_parameter
Callback tests are part of integration tests, to run just callback tests, execute:
$ make test TEST="tests/test_callback.py"
To re-record just callback JSON fixtures, execute:
$ make test TEST="tests/test_callback.py --vcrmode record"
To run the Ansible Sanity tests, execute:
$ make sanity
The tests in this repository run playbooks that can be found in
The playbooks should be self contained, target the features of a specific module and be able to run against an actual foreman instance. The structure would usually be something like setup -> run tests -> clean up.
tests/test_playbooks/tasks, preconfigured tasks for individual entities / actions can be found.
They can be reused, to create common dependent resources, as well as to manipulate the entities in the actual test.
If the boolean variable
expected_change is set, such a task fails if the result does not meet this expectation.
The ansible inventory contains two hosts:
localhost: This host runs locally without modification it is meant to setup (and teardown) dependent resources, only when (re-)recording. It can also be useful to run ad hoc commands or small playbooks during development.
tests: This host should run the actual tests. It is used to record the vcr-yaml-files, or to run isolated against those files.
In order to run these tests, the API responses of a running Foreman or Katello server must be recorded.
For this last step,
tests/test_playbooks/vars/server.yml must be configured to point to a running Foreman or Katello server.
make record_<playbook name> must be called, and the resulting vcr files (
test_playbook/fixtures/<playbook_name>-*.yml) must be checked into git.
Recording/storing apidoc.json for tests¶
The tests depend on a valid
apidoc.json being available during execution.
The easiest way to do so is to provide a
<module>.json in the
Most modules can just use a symlink to either
katello.json, depending on whether they need a plain Foreman or Foreman+Katello to function properly.
If you need a setup with different plugins enabled, just get
https://foreman.example.com/apidoc/v2.json from your install and place it in