All Projects → MishaKav → pytest-coverage-comment

MishaKav / pytest-coverage-comment

Licence: MIT license
Comments a pull request with the pytest code coverage badge and full report

Programming Languages

javascript
184084 projects - #8 most used programming language

Projects that are alternatives of or similar to pytest-coverage-comment

ghaction-cmake
cmake swiss army knife github docker action
Stars: ✭ 19 (-40.62%)
Mutual labels:  coverage, github-actions
jest-it-up
🌐📈 Automatically bump up global Jest thresholds whenever coverage goes above them
Stars: ✭ 37 (+15.63%)
Mutual labels:  coverage, test
angular-cli-skeleton
angular-cli skeleton to quickly start a new project with advanced features and best practices. All features are described in README.md.
Stars: ✭ 32 (+0%)
Mutual labels:  coverage, test
cover.run
Code coverage
Stars: ✭ 34 (+6.25%)
Mutual labels:  coverage, test
goverreport
Command line tool for coverage reporting and validation
Stars: ✭ 44 (+37.5%)
Mutual labels:  coverage, test
jacoco-badge-generator
Coverage badges, and pull request coverage checks, from JaCoCo reports in GitHub Actions
Stars: ✭ 53 (+65.63%)
Mutual labels:  coverage, github-actions
jest-github-action
Jest action adding checks with annotations to your pull requests and coverage table as comments
Stars: ✭ 134 (+318.75%)
Mutual labels:  test, github-actions
Kahlan
✔️ PHP Test Framework for Freedom, Truth, and Justice
Stars: ✭ 1,065 (+3228.13%)
Mutual labels:  coverage, test
kotlin-plugin-generated
A Kotlin compiler plugin that annotates Kotlin-generated methods for improved coverage reports
Stars: ✭ 33 (+3.13%)
Mutual labels:  coverage, test
xray-action
... a GitHub action to import test results into "Xray" - A complete Test Management tool for Jira.
Stars: ✭ 16 (-50%)
Mutual labels:  test, github-actions
code-coverage-action
GitHub Action that generates code coverage reports
Stars: ✭ 28 (-12.5%)
Mutual labels:  coverage, github-actions
jacoco-report
Github action that publishes the JaCoCo report as a comment in the Pull Request
Stars: ✭ 31 (-3.12%)
Mutual labels:  coverage, github-actions
Alsatian
TypeScript testing framework with test cases
Stars: ✭ 244 (+662.5%)
Mutual labels:  coverage, test
codeclimate-action
GitHub Action to send your code coverage to CodeClimate
Stars: ✭ 145 (+353.13%)
Mutual labels:  coverage, github-actions
Devel Cover
Code coverage metrics for Perl
Stars: ✭ 91 (+184.38%)
Mutual labels:  coverage, test
mylib
Шаблон кросплатформенного CMake-проекта для языка C++ 🇬🇧 Modern CMake crossplatform project template for C++
Stars: ✭ 49 (+53.13%)
Mutual labels:  coverage, github-actions
Shellspec
A full-featured BDD unit testing framework for bash, ksh, zsh, dash and all POSIX shells
Stars: ✭ 375 (+1071.88%)
Mutual labels:  coverage, test
Gogradle
A Gradle Plugin Providing Full Support for Go
Stars: ✭ 712 (+2125%)
Mutual labels:  coverage, test
tarpaulin
📈 GitHub Action for code coverage reporting with tarpaulin
Stars: ✭ 69 (+115.63%)
Mutual labels:  coverage, github-actions
action-junit-report
Reports junit test results as GitHub Pull Request Check
Stars: ✭ 103 (+221.88%)
Mutual labels:  test, github-actions

Pytest Coverage Comment

licience version wakatime

This action comments a pull request or commit with a HTML test coverage report. The report is based on the coverage report generated by your test runner. Note that this action does not run any tests, but expects the tests to have been run by another action already (support pytest only).

Similar Action for Jest


After I see that action become popular, I made similar action (even better) for javascript/typescript that runs jest jest-coverage-comment


You can add this action to your GitHub workflow for Ubuntu runners (e.g. runs-on: ubuntu-latest) as follows:

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml

Inputs

Name Required Default Description
github-token ${{github.token}} An alternative GitHub token, other than the default provided by GitHub Actions runner
pytest-coverage-path ./pytest-coverage.txt The location of the txt output of pytest-coverage. Used to generate the comment
coverage-path-prefix '' Prefix for path when link to files in comment
title Coverage Report Title for the coverage report. Useful for monorepo projects
badge-title Coverage Title for the badge icon
hide-badge false Hide badge with percentage
hide-report false Hide coverage report
report-only-changed-files false Show in report only changed files for this commit, and not all files
junitxml-path '' The location of the junitxml path
junitxml-title '' Title for summary for junitxml
create-new-comment false When false, will update the same comment, otherwise will publish new comment on each run.
hide-comment false Hide the whole comment (use when you need only the output). Useful for auto-update bagdes in readme. See the workflow for example
default-branch main This branch name is usefull when generate "coverageHtml", it points direct links to files on this branch (instead of commit).
Usually "main" or "master".
multiple-files '' You can pass array of titles and files to generate single comment with table of results.
Single line should look like Title, ./path/to/pytest-coverage.txt, ./path/to/pytest.xml
example:
My Title 1, ./data/pytest-coverage_3.txt, ./data/pytest_1.xml
Note: In that mode the output for coverage and color will be for the first file only.

Output Variables

Name Example Description
coverage 30% Percentage of the coverage, get from pytest-cov
color red Color of the percentage. You can see the whole list of badge colors
coverageHtml ... Html with links to files of missing lines. See the output-example
summaryReport ... Markdown with summaryof: Tests/Skipped/Failures/Errors/Time
warnings 2441 Number of warnings, get from pytest-cov
tests 109 Total number of tests, get from junitxml
skipped 2 Total number of skipped tests, get from junitxml
failures 1 Total number of tests with failures, get from junitxml
errors 0 Total number of tests with errors, get from junitxml
time 0.583 Seconds the took to run all the tests, get from junitxml
notSuccessTestInfo example Info from testcase that has failures/errors/skipped, get from junitxml

notSuccessTestInfo

the format will be JSON.stringify in current structure

{
  "failures": [{ "classname": "...", "name": "..." }],
  "errors": [{ "classname": "...", "name": "..." }],
  "skipped": [{ "classname": "...", "name": "..." }],
  "count": 3
}

Output example

Coverage

Coverage Report
FileStmtsMissCoverMissing
functions/example_completed
   example_completed.py641970%33, 39–45, 48–51, 55–58, 65–70, 91–92
functions/example_manager
   example_manager.py441175%31–33, 49–55, 67–69
   example_static.py40295%60–61
functions/my_exampels
   example.py20200%1–31
functions/resources
   resources.py26260%1–37
TOTAL105573930% 

Tests Skipped Failures Errors Time
109 2 💤 1 0 🔥 0.583s ⏱️

Example usage

The following is an example GitHub Action workflow that uses the Pytest Coverage Comment to extract the coverage report to comment at pull request:

# This workflow will install dependencies, create coverage tests and run Pytest Coverage Comment
# For more information see: https://github.com/MishaKav/pytest-coverage-comment/
name: pytest-coverage-comment
on:
  pull_request:
    branches:
      - '*'
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Set up Python 3.8
        uses: actions/setup-python@v2
        with:
          python-version: 3.8

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install flake8 pytest pytest-cov
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

      - name: Build coverage file
        run: |
          pytest --junitxml=pytest.xml --cov-report=term-missing:skip-covered --cov=app tests/ | tee pytest-coverage.txt

      - name: Pytest coverage comment
        uses: MishaKav/pytest-coverage-comment@main
        with:
          pytest-coverage-path: ./pytest-coverage.txt
          junitxml-path: ./pytest.xml

Example GitHub Action workflow that uses coverage percentage as output (see the live workflow)

- name: Pytest coverage comment
  id: coverageComment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml

- name: Check the output coverage
  run: |
    echo "Coverage Percantage - ${{ steps.coverageComment.outputs.coverage }}"
    echo "Coverage Color - ${{ steps.coverageComment.outputs.color }}"
    echo "Coverage Html - ${{ steps.coverageComment.outputs.coverageHtml }}"
    echo "Summary Report - ${{ steps.coverageComment.outputs.summaryReport }}"

    echo "Coverage Warnings - ${{ steps.coverageComment.outputs.warnings }}"

    echo "Coverage Errors - ${{ steps.coverageComment.outputs.errors }}"
    echo "Coverage Failures - ${{ steps.coverageComment.outputs.failures }}"
    echo "Coverage Skipped - ${{ steps.coverageComment.outputs.skipped }}"
    echo "Coverage Tests - ${{ steps.coverageComment.outputs.tests }}"
    echo "Coverage Time - ${{ steps.coverageComment.outputs.time }}"
    echo "Not Success Test Info - ${{ steps.coverageComment.outputs.notSuccessTestInfo }}"

Example GitHub Action workflow that passes all params to Pytest Coverage Comment

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./path-to-file/pytest-coverage.txt
    title: My Coverage Report Title
    badge-title: My Badge Coverage Title
    hide-badge: false
    hide-report: false
    create-new-comment: false
    hide-comment: false
    report-only-changed-files: false
    junitxml-path: ./path-to-file/pytest.xml
    junitxml-title: My JUnit Xml Summary Title

image

Example GitHub Action workflow that runs pytest inside docker It will generate pytest-coverage.txt and pytest.xml in /tmp directory inside docker and share /tmp directory with GitHub workspace.

- name: Run unit tests (pytest)
  run: |
    docker run -v /tmp:/tmp $IMAGE_TAG python3 -m pytest --cov-report=term-missing:skip-covered --junitxml=/tmp/pytest.xml --cov=src tests/ | tee /tmp/pytest-coverage.txt

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: /tmp/pytest-coverage.txt
    junitxml-path: /tmp/pytest.xml

Example GitHub Action workflow that uses multiple files mode (see the live workflow)

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    multiple-files: |
      My Title 1, ./data/pytest-coverage_3.txt, ./data/pytest_1.xml
      My Title 2, ./data/pytest-coverage_4.txt, ./data/pytest_2.xml

Example GitHub Action workflow that will update your README.md with coverage report, only on merge to main branch (see the update-coverage-on-readme workflow) All you need is to add in your README.md the following lines wherever you want. If your coverage html report will not change, it wouldn't push any changes to readme file.

<!-- Pytest Coverage Comment:Begin -->
<!-- Pytest Coverage Comment:End -->
name: Update Coverage on Readme
on:
  push:
jobs:
  update-coverage-on-readme:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
          fetch-depth: 0 # otherwise, you will failed to push refs to dest repo

      - name: Extract branch name
        run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
        id: extract_branch

      - name: Pytest coverage comment
        if: ${{ github.ref == 'refs/heads/main' }}
        id: coverageComment
        uses: MishaKav/pytest-coverage-comment@main
        with:
          hide-comment: true
          pytest-coverage-path: ./data/pytest-coverage_4.txt

      - name: Update Readme with Coverage Html
        if: ${{ github.ref == 'refs/heads/main' }}
        run: |
          sed -i '/<!-- Pytest Coverage Comment:Begin -->/,/<!-- Pytest Coverage Comment:End -->/c\<!-- Pytest Coverage Comment:Begin -->\n\${{ steps.coverageComment.outputs.coverageHtml }}\n${{ steps.coverageComment.outputs.summaryReport }}\n<!-- Pytest Coverage Comment:End -->' ./README.md

      - name: Commit & Push changes to Readme
        if: ${{ github.ref == 'refs/heads/main' }}
        uses: actions-js/push@master
        with:
          message: Update coverage on Readme
          github_token: ${{ secrets.GITHUB_TOKEN }}

Result example

Collapsed comment Result Collapse Example

Expanded comment Result Expand Example

Multiple Files Mode (can be useful on mono-repo projects) Result Multiple Files Mode Example

Badges colors

Coverage 0-40 [0, 40]

Coverage 40-60 [40, 60]

Coverage 60-80 [60, 80]

Coverage 80-90 [80, 90]

Coverage 90-100 [90, 100]

Auto updating badge on README

If you want auto-update the coverage badge on your Readme, you can see the workflow Auto Updating Bagde

🤝 Contributing PRs Welcome

We welcome all contributions. You can submit any ideas as pull requests or as GitHub issues and have a good time! :)

Our Contibutors

Contibutors
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].