Unit Testing
Unit testing Cement apps is as straight forward as any other Python library. Cement uses, and recommends the following utilities, but there are no limitations to what tools you choose to use:
Cement has a strict policy that all framework code has 100% test coverage, and 100% style-guide clearance before any releases are published. This is not required for applications built on Cement, but it is highly recommended for quality.
Cement apps sub-class from
cement.core.foundation.App
, however for testing there are some things we want to override to make things more portable/reliable. For example, in testing we do not want to parse ~/.myapp.yml
(example local user configuration file), as this may skew tests from one developer machine to the next.Cement includes an additional
cement.core.foundation.TestApp
that can also be sub-classed to easily override App.Meta for testing.This looks something like:
class TestApp(App):
class Meta:
label = "app-%s" % misc.rando()[:12]
argv = []
core_system_config_files = []
core_user_config_files = []
config_files = []
core_system_config_dirs = []
core_user_config_dirs = []
config_dirs = []
core_system_template_dirs = []
core_user_template_dirs = []
core_system_plugin_dirs = []
core_user_plugin_dirs = []
plugin_dirs = []
exit_on_close = False
To use this class, it is best to create a separate
MyAppTest
class that sub-classes both from TestApp
and your MyApp
:class MyAppTest(TestApp,MyApp):
"""A sub-class of MyApp that is better suited for testing."""
class Meta:
label = 'myapp'
Note the order of sub-classing is important so that
Meta
is overridden properly.As most apps built on Cement are command-line based, passing arguments and options to
MyApp
is important. This is easily handled by passing the argv
list:myargs = ['some-command', '--some-option', '--etc']
with MyAppTest(argv=myargs) as app:
app.run()
The above is equivalent to running the following at the command line:
myapp some-command --some-option --etc
Working with files and directories is common for apps built on Cement, which can be easily done using the
fs.Tmp()
utility in combination with Pytest fixtures. These fixtures can be automatically loaded from tests/conftest.py
:myapp/tests/conftest.py:
import pytest
from cement import fs
@pytest.fixture(scope="function")
def tmp(request):
"""
Create a `tmp` object that generates a unique temporary directory, and file
for each test function that requires it.
"""
t = fs.Tmp()
yield t
t.remove()
This creates a
tmp
fixture that we can use anywhere in our tests, and most importantly when the test is complete the temp object (file and directory) are automatically removed.myapp/tests/test_myapp.py:
def test_myapp(tmp):
with MyAppTest() as app:
# do something with tmp.file/tmp.dir
app.run()
Note the
tmp
fixture is passed to each test function, creating a unique instance of it for each test.In Pytest, tests are loaded from files matching
test_
, and functions matching test_
. Some developers prefer to keep tests alongside the file it is testing (for example: main.py
and test_main.py
). Cement developers prefer putting all tests in a separate tests/
directory:├── CHANGELOG.md
├── README.md
├── myapp
│ ├── __init__.py
│ ├── controllers
│ │ ├── __init__.py
│ │ └── base.py
│ ├── main.py
├── requirements.txt
├── setup.py
└── tests
├── conftest.py
├── controllers
│ └── test_base.py
└── test_main.py
A typical test is just a simple function that executes some logic, and asserts some conditions:
def test_myapp_debug():
# ensure debug is false by default
with MyTestApp() as app:
assert app.debug is False
# ensure debug option toggles app.debug
with MyTestApp(argv=['--debug']) as app:
assert app.debug is True
In the above example, we tested two conditions (whether
--debug
was passed or not), and asserted those conditions. If either assertion failed, the test would fail.Running tests with Pytest can be done by running
pytest
in the root of your project directory. That said, to add support for coverage.py you likely want to create a helper script (Makefile, Fabric, etc) to execute all the things at once.Cement
generate project
includes support for coverage.py
out of the box, and also includes a Makefile
with helpers:myapp/Makefile:
test:
python -m pytest
-v
--cov=myapp
--cov-report=term
--cov-report=html:coverage-report
tests/
And this is run by:
make test
The above example generates an HTML coverage report in
./coverage-report/index.html
. Open that file in your local browser to review code that is not getting hit by tests, and write tests to touch that code for better coverage.Note that
--cov=myapp
is telling coverate to only report on your code, rather than including all other libraries your code has imported/executed.Last modified 1yr ago