GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → omarkohl → pytest-datafiles

omarkohl / pytest-datafiles

Licence: MIT license
pytest plugin to create a tmpdir containing a preconfigured set of files and/or directories.

Programming Languages

python
139335 projects - #7 most used programming language
Makefile
30231 projects

Labels

pytestpytest-plugin

Projects that are alternatives of or similar to pytest-datafiles

pytest-mock-server
Mock server plugin for pytest
Stars: ✭ 19 (-74.67%)
Mutual labels:  pytest, pytest-plugin
pytest-snapshot
A plugin for snapshot testing with pytest.
Stars: ✭ 68 (-9.33%)
Mutual labels:  pytest, pytest-plugin
pytest-eth
PyTest plugin for testing smart contracts for Ethereum blockchain.
Stars: ✭ 23 (-69.33%)
Mutual labels:  pytest, pytest-plugin
pytest-test-groups
A Pytest plugin that gives you a way to split your tests into groups of a specific size
Stars: ✭ 37 (-50.67%)
Mutual labels:  pytest, pytest-plugin
pytest-reportlog
Replacement for the --resultlog option, focused in simplicity and extensibility
Stars: ✭ 36 (-52%)
Mutual labels:  pytest, pytest-plugin
pytest-pipeline
Pytest plugin for functional testing of data analysis pipelines
Stars: ✭ 19 (-74.67%)
Mutual labels:  pytest, pytest-plugin
pytest-localstack
Pytest plugin for local AWS integration tests
Stars: ✭ 66 (-12%)
Mutual labels:  pytest, pytest-plugin
pytest-subprocess
Pytest plugin to fake subprocess.
Stars: ✭ 83 (+10.67%)
Mutual labels:  pytest, pytest-plugin
overhave
Web-framework for BDD: scalable, configurable, easy to use, based on Flask Admin and Pydantic.
Stars: ✭ 61 (-18.67%)
Mutual labels:  pytest, pytest-plugin
pytest-djangoapp
Nice pytest plugin to help you with Django pluggable application testing.
Stars: ✭ 35 (-53.33%)
Mutual labels:  pytest, pytest-plugin
pytest-elk-reporter
A plugin to send pytest test results to ELK stack
Stars: ✭ 17 (-77.33%)
Mutual labels:  pytest, pytest-plugin
Seleniumbase
A Python framework that inspires developers to become better test automation engineers. 🧠💡
Stars: ✭ 2,520 (+3260%)
Mutual labels:  pytest, pytest-plugin
pytest-datafixtures
Data fixtures for pytest made simple
Stars: ✭ 24 (-68%)
Mutual labels:  pytest, pytest-plugin
pytest-snail
Plugin for adding a marker to slow running tests. 🐌
Stars: ✭ 15 (-80%)
Mutual labels:  pytest, pytest-plugin
pytest-csv
CSV reporter for pytest.
Stars: ✭ 16 (-78.67%)
Mutual labels:  pytest, pytest-plugin
pytest-notebook
A pytest plugin for regression testing and regenerating Jupyter Notebooks
Stars: ✭ 35 (-53.33%)
Mutual labels:  pytest, pytest-plugin
pytest-pycodestyle
pytest plugin to run pycodestyle
Stars: ✭ 15 (-80%)
Mutual labels:  pytest, pytest-plugin
pytest-docker-tools
Opionated helpers for creating py.test fixtures for Docker integration and smoke testing environments
Stars: ✭ 61 (-18.67%)
Mutual labels:  pytest, pytest-plugin
pytest-arraydiff
pytest plugin to facilitate comparison of results to a pre-defined reference
Stars: ✭ 12 (-84%)
Mutual labels:  pytest, pytest-plugin
Pudb
Full-screen console debugger for Python
Stars: ✭ 2,267 (+2922.67%)
Mutual labels:  pytest, pytest-plugin
View All Similar Projects ➔

pytest-datafiles

https://coveralls.io/repos/omarkohl/pytest-datafiles/badge.svg?branch=master&service=github Code Climate

pytest plugin to create a tmpdir containing a preconfigured set of files and/or directories.

Note about maintenance: This project is maintained and bug reports or pull requests will be addressed. There is little activity because it simply works and no changes are required.

Features

This plugin allows you to specify one or several files/directories that are copied to a temporary directory (tmpdir) before the execution of the test. This means the original files are not modified and every test runs on its own version of the same files.

Files/directories can be specified either as strings or as py.path objects.

To take advantage of the datafiles fixture in a test function, add datafiles as one of the test function parameters (per usual with pytest fixtures) and decorate the test function with @pytest.mark.datafiles(file1, file2, dir1, dir2, ...). See the examples below.

The datafiles variable in your test function is a py.path object (tmpdir) where the copied files are located. Under Linux systems this will most likely be some subdirectory of /tmp/.

Options

The following options can be specified as keyword arguments (kwargs) to the @pytest.mark.datafiles decorator function:

  • keep_top_dir: For all parameters that represent directories, keep that directory instead of only (recursively) copying its content. Possible values are True or False. False is the default value.
  • on_duplicate: Specify the action to take when duplicate files/directories are found. Possible values are: exception, ignore and replace. The default value is exception.
    • exception: An exception is raised instead of copying the duplicate file/directory.
    • ignore: The second (or subsequent) files/directories with the same name as the first one are simply ignored (i.e., the first file/directory with the duplicate name is kept).
    • replace: The second (or subsequent) files/directories with the same name replace the previous ones (i.e., the last file/directory with the duplicate name is kept).

See below for some examples.

Installation

pip install pytest-datafiles

Usage

Example 1

One possible use case is when you are running tests on very big files that are not included or packaged with your tests. For example, your test files are large video files stored under /opt/big_files/ . You don't want your tests modifying the original files, but the files are required by the tests. You can reference these data files in your test method as follows:

import os
import pytest

@pytest.mark.datafiles('/opt/big_files/film1.mp4')
def test_fast_forward(datafiles):
    path = str(datafiles)  # Convert from py.path object to path (str)
    assert len(os.listdir(path)) == 1
    assert os.path.isfile(os.path.join(path, 'film1.mp4'))
    #assert some_operation(os.path.join(path, 'film1.mp4')) == expected_result

    # Using py.path syntax
    assert len(datafiles.listdir()) == 1
    assert (datafiles / 'film1.mp4').check(file=1)

Example 2

Now for another use case: let's say in the directory where your tests are located, you place a directory named test_files. Here you have a lot of images you want to run tests on. By using this plugin, you make sure the original files under test_files are not modified by every test.

import os
import pytest

FIXTURE_DIR = os.path.join(
    os.path.dirname(os.path.realpath(__file__)),
    'test_files',
    )

@pytest.mark.datafiles(
    os.path.join(FIXTURE_DIR, 'img1.jpg'),
    os.path.join(FIXTURE_DIR, 'img2.jpg'),
    os.path.join(FIXTURE_DIR, 'img3.jpg'),
    )
def test_find_borders(datafiles):
    for img in datafiles.listdir():
        print(img)
        #assert process(img) == some_expected_value

@pytest.mark.datafiles(
    os.path.join(FIXTURE_DIR, 'img4.jpg'),
    os.path.join(FIXTURE_DIR, 'img5.jpg'),
    )
def test_brightness(datafiles):
    for img in datafiles.listdir():
        print(img)
        #assert process(img) == some_expected_value

Example 3

If all (or many) of your tests rely on the same files it can be easier to define one decorator beforehand and apply it to every test like this example:

import os
import pytest

FIXTURE_DIR = os.path.join(
    os.path.dirname(os.path.realpath(__file__)),
    'test_files',
    )

ALL_IMGS = pytest.mark.datafiles(
    os.path.join(FIXTURE_DIR, 'img1.jpg'),
    os.path.join(FIXTURE_DIR, 'img2.jpg'),
    os.path.join(FIXTURE_DIR, 'img3.jpg'),
    os.path.join(FIXTURE_DIR, 'img4.jpg'),
    os.path.join(FIXTURE_DIR, 'img5.jpg'),
    os.path.join(FIXTURE_DIR, 'img6.jpg'),
    os.path.join(FIXTURE_DIR, 'img7.jpg'),
    os.path.join(FIXTURE_DIR, 'img8.jpg'),
    )

@ALL_IMGS
def test_something1(datafiles):
    for img in datafiles.listdir():
        print(img)
        #assert process(img) == some_expected_value

@ALL_IMGS
def test_something2(datafiles):
    for img in datafiles.listdir():
        print(img)
        #assert process(img) == some_expected_value

Example 4

Imagine you have 3 directories (dir1, dir2, dir3) each containing the files (fileA and fileB).

This example clarifies the options on_duplicate and keep_top_dir.

import os
import pytest

FIXTURE_DIR = os.path.join(
    os.path.dirname(os.path.realpath(__file__)),
    '_fixture_files',
    )

@pytest.mark.datafiles(
    os.path.join(FIXTURE_DIR, 'dir1'),
    os.path.join(FIXTURE_DIR, 'dir2'),
    os.path.join(FIXTURE_DIR, 'dir3'),
    on_duplicate='ignore',
    )
def test_dir_ignore(datafiles):
    # datafiles.listdir() will list fileA and fileB originally from dir1
    pass

@pytest.mark.datafiles(
    os.path.join(FIXTURE_DIR, 'dir1'),
    os.path.join(FIXTURE_DIR, 'dir2'),
    os.path.join(FIXTURE_DIR, 'dir3'),
    on_duplicate='replace',
    )
def test_dir_replace(datafiles):
    # datafiles.listdir() will list fileA and fileB originally from dir3
    pass

@pytest.mark.datafiles(
    os.path.join(FIXTURE_DIR, 'dir1'),
    os.path.join(FIXTURE_DIR, 'dir2'),
    os.path.join(FIXTURE_DIR, 'dir3'),
    # on_duplicate='exception' is the default and does not need to be
    # specified
    )
def test_dir_exception(datafiles):
    # An exception will be raised because of duplicate filename fileA
    pass

@pytest.mark.datafiles(
    os.path.join(FIXTURE_DIR, 'dir1'),
    os.path.join(FIXTURE_DIR, 'dir2'),
    os.path.join(FIXTURE_DIR, 'dir3'),
    keep_top_dir=True,
    )
def test_dir_keep_top_dir(datafiles):
    # datafiles.listdir() will list dir1, dir2 and dir3 (each containing
    # fileA and fileB)
    pass

Example 5

You can also use a py.path object instead of str paths.

import os
import py
import pytest

_dir = os.path.dirname(os.path.realpath(__file__))
FIXTURE_DIR = py.path.local(_dir) / 'test_files'

@pytest.mark.datafiles(
    FIXTURE_DIR / 'img1.jpg',
    FIXTURE_DIR / 'img2.jpg',
    FIXTURE_DIR / 'img3.jpg',
    )
def test_fast_forward(datafiles):
    assert len(datafiles.listdir()) == 3

Contributing

Contributions are very welcome. Tests can be run with tox. Please ensure the coverage stays at least the same before you submit a pull request.

To create and upload a new package first update the version number and then:

pip3 install --user -U twine
make clean
make dist
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
# Verify the package is usable
virtualenv -p python3 test-venv
test-venv/bin/pip install pytest
test-venv/bin/pip install --index-url https://test.pypi.org/simple/ pytest-datafiles
# Create some test_example.py (e.g. with one of the examples above)
test-venv/bin/pytest test_example.py
# Set the git tag for final release
git tag -a 2.0
git push --tags
# Upload the package for final release
twine upload dist/*

Finally create a release on GitHub and add the packages from dist/* to it.

Of course this will only work if you have the necessary PyPI credentials for this package.

License

Distributed under the terms of the MIT license, "pytest-datafiles" is free and open source software.

Issues

If you encounter any problems, please file an issue along with a detailed description.

Acknowledgements

Thanks to @flub for the idea to use pytest marks to solve the problem this plugin is trying to solve.

Some ideas to improve this project were taken from the Cookiecutter templates cookiecutter-pypackage and cookiecutter-pytest-plugin.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].