Tests in Fjord allow us to make changes and be reasonably sure that the system continues to work. Further, they make it easier to verify correctness for behavioral details.
Note
We use the py.test
script in the root directory rather than the
one installed with py.test in PATH. We need it to set up the path
because we have many of the libraries in vendor/
. Once we get
rid of vendor/
we can use the regular py.test
script.
Before you run the tests, you have to run the collectstatic
command. This
compiles LESS files to CSS files and creates the bundles that some of
the tests require to run. If you don’t do this, then a few of the
tests will fail.
To run collectstatic, do:
./manage.py collectstatic
You don’t have to do this often. I’d do it the first time and then any time you run the tests with a fresh db.
FIXME: This is annoying and it’d be nice to get it fixed.
To run the test suite, do:
./py.test
Default options for running the test are in pytest.ini
. This is a
good set of defaults.
If you ever need to change the defaults, you can do so at the command line.
Helpful command-line arguments:
--pdb
:--create-db
:--showlocals
:--exitfirst
:See ./py.test --help
for more arguments.
The test suite will create a new database named test_%s
where
%s
is whatever value you have for
settings.DATABASES['default']['NAME']
.
There are a bunch of ways to specify a subset of tests to run:
only tests marked with the ‘es_tests’ marker:
./py.test -m es_tests
all the tests but those marked with the ‘es_tests’ marker:
./py.test -m "not es_tests"
all the tests but the suggests ones:
./py.test --ignore fjord/suggests
all the tests that have “foobar” in their names:
./py.test -k foobar
all the tests that don’t have “foobar” in their names:
./py.test -k "not foobar"
tests in a certain directory:
./py.test fjord/suggest/
specific test:
./py.test fjord/suggest/tests/test_dummy.py::DummySuggesterTestCase::test_get_suggestions
See http://pytest.org/latest/usage.html for more examples.
Code should be written so it can be tested, and then there should be tests for it.
When adding code to an app, tests should be added in that app that
cover the new functionality. All apps have a tests
module where
tests should go. They will be discovered automatically by the test
runner as long as the look like a test.
reverse
to return locales in the URL, use
LocalizingClient
instead of the default client for the
TestCase
class.fjord.feedback.tests.ResponseFactory
generates
fjord.feedback.models.Response
instances.README.rst
file in the smoketests/
directory.JavaScript tests are not run in our normal unit test suite. Instead we have a different test system.
We test JavaScript utility functions using QUnit.
These tests are located in fjord/base/static/tests/
.
Launch the server with:
./manage.py runserver
Then go to:
http://127.0.0.1:8000/static/tests/index.html
(You might have to use a different protocol, host and port depending on how you have Fjord set up.)
Note
These are also run as part of the smoketest suite.
To add a new test suite, add a couple of script
lines to index.html
in
the relevant place and then create a new test_FILENAMEHERE.js
file
with your QUnit tests.
We have a smoketest suite. For more details, see that README:
Sometimes, we need to make substantive changes to the site that touch a lot of parts. This test plan covers all the things you should do (at a minimum) to make sure those parts are still working.
This is a good thing to do after doing a Django upgrade.
Note
These tests aren’t run frequently and they’re probably out of date.
Run through the test plan with your code before you make your changes and update parts that have changed. Make sure to add sections for new functionality.
Note
This is labeled “comprehensive”, but probably leave some stuff out. We should improve it as we use it.
Note
This test plan is very terse. You’ll need to know your way around Fjord and Input for this to make a lot of sense. Sorry about that.
./py.test
to run the unit tests with a clean database./manage.py runserver
in one terminal and launch the smoketests
in another terminalNote
This also runs the JavaScript tests.
static/
./manage.py collectstatic
and verify no errors and .js
and .css
files got built./manage.py makemigrations
– it shouldn’t create any new
migrations./manage.py migrate
with a db dump./manage.py check
and verify no errorsrun ./manage.py esstatus
run ./manage.py esreindex --percent=5
to create a new index and
index some stuff
run ./manage.py esstatus
to make sure the new index is there
run ./manage.py esdelete <index>
to delete that index
run ./manage.py esstatus
to make sure the index was deleted
run ./manage.py esreindex --percent=5
to recreate the index
verify feedback is indexed
./manage.py runserver
to launch the serververify reindexing works from admin:
make sure CELERY_ALWAYS_EAGER = False
in
fjord/settings/local.py
run ./manage.py celeryd
to launch celery server
in another terminal, run ./manage.py runserver
open up a browser
log in to the server
go to admin
go to Elasticssearch maintenance
launch a reindexing
Make sure it’s reindexing things. Once you know it’s reindexing things, then you can cut it short. Otherwise it takes forever.
locale/
directory is up to date./manage.py extract
and make sure that it produced or updated
a locale/templates/LC_MESSAGES/django.pot
file and that msgids
did not change./manage.py merge
and make sure it did the right thing./bin/compile-linted-mo.sh
and make sure dennis linted the
.po
files and that the script compiled .mo
files./manage.py runserver
to launch the serverVerify analyzer views load:
./manage.py runserver
to launch the serverVerify the admin views load:
./manage.py runserver
to launch the serverbin/crontab/crontab.tpl
, make sure it worksVerify that the documentatino builds:
cd docs/
make html
and make sure there are no build errorsWith an existing vagrant environment:
vagrant up
./peep.sh install -r requirements/requirements.txt
./peep.sh install -r requirements/dev.txt
vagrant ssh
cd fjord
./py.test
Now we’re going to create a new vagrant environment:
vagrant halt
vagrant destroy --force
vagrant up
vagrant ssh
cd fjord
./py.test
If that works, then the Vagrant development environment probably works fine, too.