🤖 Repeat tests. Repeat tests. Repeat tests.

Repeats tests using different inputs (Data-Driven Testing):

• test runner independent: works with your current setup
• generates test titles that are descriptive, unique, for any JavaScript type (not just JSON)
• loops over every possible combination of inputs (cartesian product)
• can use random functions (fuzz testing)
• snapshot testing friendly

Example

// The examples use Ava but any test runner works (Jest, Mocha, Jasmine, etc.)
const test = require('ava')
const { each } = require('test-each')

// The code we are testing
const multiply = require('./multiply.js')

// Repeat test using different inputs and expected outputs
each(
  [
    { first: 2, second: 2, output: 4 },
    { first: 3, second: 3, output: 9 },
  ],
  ({ title }, { first, second, output }) => {
    // Test titles will be:
    //    should multiply | {"first": 2, "second": 2, "output": 4}
    //    should multiply | {"first": 3, "second": 3, "output": 9}
    test(`should multiply | ${title}`, t => {
      t.is(multiply(first, second), output)
    })
  },
)

// Snapshot testing. The `output` is automatically set on the first run,
// then re-used in the next runs.
each(
  [
    { first: 2, second: 2 },
    { first: 3, second: 3 },
  ],
  ({ title }, { first, second }) => {
    test(`should multiply outputs | ${title}`, t => {
      t.snapshot(multiply(first, second))
    })
  },
)

// Cartesian product.
// Run this test 4 times using every possible combination of inputs
each([0.5, 10], [2.5, 5], ({ title }, first, second) => {
  test(`should mix integers and floats | ${title}`, t => {
    t.is(typeof multiply(first, second), 'number')
  })
})

// Fuzz testing. Run this test 1000 times using different numbers.
each(1000, Math.random, ({ title }, index, randomNumber) => {
  test(`should correctly multiply floats | ${title}`, t => {
    t.is(multiply(randomNumber, 1), randomNumber)
  })
})

Demo

You can try this library:

• either directly in your browser.
• or by executing the examples files in a terminal.

Install

npm install -D test-each

Usage

const { each } = require('test-each')

const inputs = [
  ['red', 'blue'],
  [0, 5, 50],
]
each(...inputs, function callback(info, color, number) {})

Fires callback once for each possible combination of inputs.

Each input can be an array, a function or an integer.

A common use case for callback is to define tests (using any test runner).

info is an object whose properties can be used to generate test titles.

Test titles

Each combination of parameters is stringified as a title available in the callback's first argument.

Titles should be included in test titles to make them descriptive and unique.

Long titles are truncated. An incrementing counter is appended to duplicates.

Any JavaScript type is stringified, not just JSON.

You can customize titles either by:

• defining title properties in inputs that are plain objects
• using the info argument

each([{ color: 'red' }, { color: 'blue' }], ({ title }, param) => {
  // Test titles will be:
  //    should test color | {"color": "red"}
  //    should test color | {"color": "blue"}
  test(`should test color | ${title}`, () => {})
})

// Plain objects can override this using a `title` property
each(
  [
    { color: 'red', title: 'Red' },
    { color: 'blue', title: 'Blue' },
  ],
  ({ title }, param) => {
    // Test titles will be:
    //    should test color | Red
    //    should test color | Blue
    test(`should test color | ${title}`, () => {})
  },
)

// The `info` argument can be used for dynamic titles
each([{ color: 'red' }, { color: 'blue' }], (info, param) => {
  // Test titles will be:
  //    should test color | 0 red
  //    should test color | 1 blue
  test(`should test color | ${info.index} ${param.color}`, () => {})
})

Cartesian product

If several inputs are specified, their cartesian product is used.

// Run callback five times: a -> b -> c -> d -> e
each(['a', 'b', 'c', 'd', 'e'], (info, param) => {})

// Run callback six times: a c -> a d -> a e -> b c -> b d -> b e
each(['a', 'b'], ['c', 'd', 'e'], (info, param, otherParam) => {})

// Nested arrays are not iterated.
// Run callback only twice: ['a', 'b'] -> ['c', 'd', 'e']
each(
  [
    ['a', 'b'],
    ['c', 'd', 'e'],
  ],
  (info, param) => {},
)

Input functions

If a function is used instead of an array, each iteration fires it and uses its return value instead. The function is called with the same arguments as the callback.

The generated values are included in test titles.

// Run callback with a different random number each time
each(['red', 'green', 'blue'], Math.random, (info, color, randomNumber) => {})

// Input functions are called with the same arguments as the callback
each(
  ['02', '15', '30'],
  ['January', 'February', 'March'],
  ['1980', '1981'],
  (info, day, month, year) => `${day}/${month}/${year}`,
  (info, day, month, year, date) => {},
)

Fuzz testing

Integers can be used instead of arrays to multiply the number of iterations.

This enables fuzz testing when combined with input functions and libraries like faker.js, chance.js or json-schema-faker.

const faker = require('faker')

// Run callback 1000 times with a random UUID and color each time
each(
  1000,
  faker.random.uuid,
  faker.random.arrayElement(['green', 'red', 'blue']),
  (info, randomUuid, randomColor) => {},
)

// `info.index` can be used as a seed for reproducible randomness.
// The following series of 1000 UUIDs will remain the same across executions.
each(
  1000,
  ({ index }) => faker.seed(index) && faker.random.uuid(),
  (info, randomUuid) => {},
)

Snapshot testing

This library works well with snapshot testing.

Any library can be used (snap-shot-it, Ava snapshots, Jest snapshots, Node TAP snapshots, etc.).

// The `output` is automatically set on the first run,
// then re-used in the next runs.
each(
  [
    { first: 2, second: 2 },
    { first: 3, second: 3 },
  ],
  ({ title }, { first, second }) => {
    test(`should multiply outputs | ${title}`, t => {
      t.snapshot(multiply(first, second))
    })
  },
)

Side effects

If callback's parameters are directly modified, they should be copied to prevent side effects for the next iterations.

each(
  ['green', 'red', 'blue'],
  [{ active: true }, { active: false }],
  (info, color, param) => {
    // This should not be done, as the objects are re-used in several iterations
    param.active = false

    // But this is safe since it's a copy
    const newParam = { ...param }
    newParam.active = false
  },
)

Iterables

iterable() can be used to iterate over each combination instead of providing a callback.

const { iterable } = require('test-each')

const combinations = iterable(
  ['green', 'red', 'blue'],
  [{ active: true }, { active: false }],
)

for (const [{ title }, color, param] of combinations) {
  test(`should test color | ${title}`, () => {})
}

The return value is an Iterable. This can be converted to an array with the spread operator.

const array = [...combinations]

array.forEach(([{ title }, color, param]) => {
  test(`should test color | ${title}`, () => {})
})

API

each(...inputs, callback)

inputs: Array | function | integer (one or several)
callback: function(info, ...params)

Fires callback with each combination of params.

iterable(...inputs)

inputs: Array | function | integer (one or several)
Return value: Iterable<[info, ...params]>

Returns an Iterable looping through each combination of params.

info

Type: object

info.title

Type: string

Like params but stringified. Should be used in test titles.

info.titles

Type: string[]

Like info.title but for each param.

info.index

Type: integer

Incremented on each iteration. Starts at 0.

info.indexes

Type: integer[]

Index of each params inside each initial input.

params

Type: any (one or several)

Combination of inputs for the current iteration.

Support

If you found a bug or would like a new feature, don't hesitate to submit an issue on GitHub.

For other questions, feel free to chat with us on Gitter.

Everyone is welcome regardless of personal background. We enforce a Code of conduct in order to promote a positive and inclusive environment.

Contributing

This project was made with ❤️. The simplest way to give back is by starring and sharing it online.

If the documentation is unclear or has a typo, please click on the page's Edit button (pencil icon) and suggest a correction.

If you would like to help us fix a bug or add a new feature, please check our guidelines. Pull requests are welcome!

ehmicky 💻 🎨 🤔 📖