vim-UT

Introduction

UT is another Unit Testing Framework for Vim, which main particularity is to fill the quickfix window with the assertion failures.

Features

Main features

Assertion failures are reported in the quickfix window
Meant to be simple to use
- Simple assertion syntax (:Assert expression, :AssertEquals(ref, expr)...)
- Supports :Comments
- Automatic registration of test functions
  - All the s:Test* functions of a suite are executed (almost) independently (i.e., a critical :Assert! failure will stop the Test of the function, and lh#UT will proceed to the next s:Test function
- A suite == a file
- Several s:TestXxx() per suite
Callstack is decoded and expanded in the quickfix window on uncaught exceptions.

Other features

Lightweight and simple to use: there is only one command defined, all the other definitions are kept in an autoload plugin.
Fixtures:
- optional s:Setup(), s:Teardown() executed before and after each test
- optional s:BeforeAll(), s:AfterAll() executed once before and after all tests from a suite
Supports banged :Assert! to stop processing a given test on failed assertions
s:LocalFunctions(), s:variables, and l:variables are supported
Buffer content can be set -- with :SetBufferContent
Buffer content can be tested -- with :AssertBufferMatches
Count successful tests and failed assertions
Command to exclude, or specify the tests to play => :UTPlay, UTIgnore
Add :debug before an assertion to debug its evaluation
Short-cuts to run the Unit Tests associated to a given vim script; Relies on: Let-Modeline/local_vimrc/Project to set g:UTfiles (space separated list of glob-able paths), and on lh-vim-lib#path
Helper scripts are provided to help integration with vimrunner+rspec. See examples of use in lh-vim-lib and lh-brackets.
Takes advantage of BuildToolsWrapper's :COpen command if installed

Demo

Here is a little screencast to see how things are displayed when errors are detected.

Notes:

the diff-mode is entered with D -- sorry, I wasn't able to display the keys typed in this screencast.
the signs and balloons are automatically added by vim-compil-hints

Usage

Start-up

Create a new vim script, it will be a Unit Testing Suite.

One of the first lines must contain

UTSuite Some intelligible name for the suite

Then you are free to directly assert anything you wish as long as it is a valid vim expression, e.g.

Assert 1 > 2
Assert 1 > 0
Assert s:foo > s:Bar(g:var + 28) / strlen("foobar")

debug AssertTxt (s:foo > s:Bar(g:var+28)
      \, s:foo." isn't bigger than s:Bar(".g:var."+28)")
AssertEquals!('a', 'a')
AssertDiffers('a', 'a')
let dict = {}
AssertIs(dict, dict)
AssertIsNot(dict, dict)
AssertMatch('abc', 'a')
AssertRelation(1, '<', 2)
AssertThrows 0 + [0]

tests/lh/README.vim|| SUITE <[lh#UT] Demonstrate assertions in README>
tests/lh/README.vim|27 error| assertion failed: 1 > 2
tests/lh/README.vim|31 error| assertion failed: s:foo > s:Bar(g:var + 28) / strlen("foobar")
tests/lh/README.vim|33 error| assertion failed: -1 isn't bigger than s:Bar(5+28)
tests/lh/README.vim|37 error| assertion failed: 'a' is not different from 'a'
tests/lh/README.vim|40 error| assertion failed: {} is not identical to {}
...

or to define as many independent tests as you wish. A test is a function with a name starting with s:Test. Even if a test critically fails, the next test will be executed, e.g.

function s:Test1()
  let var = SomeFunction()
  Assert! type(var) == type(0)
  Assert var < 42
  Assert! var > 0

  " Some other code that won't be executed if the previous assertion failed
  " /*the wiki does not recognizes vim comments*/
  let i = var / 42.0
  Comment "This comment may never be displayed if {var} is negative or not a number"
endfunction

function s:Test2()
  Assert s:what != Ever()
endfunction

Now run :UTRun on your test script (filename), and ... debug your failed assertions.

Fixtures

Code can be executed before and after each test function with the optional special functions:

s:Setup(): set-up function executed before each test
s:Teardown(): clean-up function executed after each test
s:BeforeAll(): set-up function execute once before all tests from a suite
s:AfterAll(): clean-up function execute once after all tests from a suite

Test on buffers

Most :AssertXxx commands are dedicated to unit testing vim functions. A function returns a result and we test whether its value equals, differs, matches, is...

Since V2, it's now possible set the content of a buffer before transforming it, and to test whether the new buffer state is as expected.

Within the quickfix-window, we can hit D on the message associated to a buffer matching failure in order to display, in a new tabpage, the expected content alongside the expected content in diff-mode.

The diff-mode can be exited by hitting q from any scratch buffer involded.

silent! call lh#window#create_window_with('new') " work around possible E36
try
    SetBufferContent a/file/name.txt
    %s/.*/\U&/
    AssertBufferMatch a/file/NAME.txt
finally
    bw
endtry

Or, with :let=<< syntax

silent! call lh#window#create_window_with('new') " work around possible E36
try
    SetBufferContent << trim EOF
    1
    3
    2
    EOF

    %sort

    AssertBufferMatch << trim EOF
    1
    4
    3
    EOF
finally
    silent bw!
endtry

which results into

tests/lh/README.vim|78 error| assertion failed: Observed buffer does not match Expected reference:
|| ---
|| +++
|| @@ -1,3 +1,3 @@
||  1
|| -4
|| +2
||  3

Note: :SetBufferContent and :AssertBufferMatch with << [trim] EOF syntax can only be used within s:TestXxx functions.

Examples

See:

tests/lh/UT.vim for a classical test,
tests/lh/UT-fixtures.vim for a test with fixtures.
tests/lh/UT-buf.vim for tests on buffer content.

To Do

Check UT works fine under windows (where paths have spaces, etc.), and on UTF-8 files
Simplify s:errors functions
Support Embedded comments like for instance:
```
Assert 1 == 1 " 1 must value 1
```
Ways to test buffers produced
Find a way to prevent the potential script scope pollution
Add a summary at the end of the execution

Design Choices

The assertions supported by this plugin are expected to be made in a Unit Testing file, they are not to be used in regular VimL scripts as a Design by Contract tool. Check my DbC framework in lh-vim-lib, or even Thomas Link's plugin, it is much more suited for that kind of assertions.
In order to be able to produce the quickfix entries, the plugin first parses the Unit Test file to complete all :Assert occurrences with extra information about the line number where the assertion is made. Incidentally, this permits to reduce plugin footprint: none of :AssertXXX commands are actual commands.

Installation

Requirements: Vim 7.+, lh-vim-lib v5.1.0
With vim-addon-manager, install vim-UT (this is the preferred method because of the dependencies)
```
ActivateAddons UT
```
or with vim-flavor, which also handles dependencies
```
flavor 'LucHermitte/vim-UT'
```

or you can clone the git repositories

git clone [email protected]:LucHermitte/lh-vim-lib.git
git clone [email protected]:LucHermitte/vim-UT.git

or with Vundle/NeoBundle:

Bundle 'LucHermitte/lh-vim-lib'
Bundle 'LucHermitte/vim-UT'

Other Tests related plugins for Vim

Tom Link's tAssert plugin, and spec_vim plugin,
Staale Flock's vimUnit plugin,
Meikel Brandmeyer's vimTAP plugin,
Ben Fritz's vim-2html-test plugin,
Ingo Karkat's runVimTests plugin,
See also Paul Mucur article's: Testing Vim Plugins on Travis CI with RSpec and Vimrunner,
Andrew Radev's vimrunner,
Kana's vim-spec
Dhruva Sagar's vim-testify

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

LucHermitte / vim-UT

Programming Languages

Labels

Projects that are alternatives of or similar to vim-UT