Allure Kotlin Integrations

The repository contains Allure2 adaptors for JVM-based test frameworks targeting Kotlin and Java with 1.6 source compatibility.

The core of this library was ported from allure-java. Thanks to that allure-kotlin has the same API, features, test coverage and solutions as allure-java. On top of the core library support for Kotlin and Android test frameworks were added.

Check out the Allure Documentation.

Supported frameworks

• JUnit4
• Android Robolectric (via AndroidX Test)
• Android Instrumentation (via AndroidX Test)

Getting started

JUnit4

Setting up the dependency

repositories {
    mavenCentral()
}

dependencies {
    testImplementation "io.qameta.allure:allure-kotlin-model:$LATEST_VERSION"
    testImplementation "io.qameta.allure:allure-kotlin-commons:$LATEST_VERSION"
    testImplementation "io.qameta.allure:allure-kotlin-junit4:$LATEST_VERSION"
}

Attaching listener

Attach the AllureJunit4 run listener using one of available methods:

• Attach the listener in build tool
  • Maven - use Maven Surfire Plugin
  • Gradle - use JUnit Foundation (example)
• Use predefined test runner

@RunWith(AllureRunner::class)
class MyTest {
    ...
}

@RunWith(AllureParametrizedRunner::class)
class MyParameterizedTest {
    ...
}

Android tests

Setting up the dependency

repositories {
    mavenCentral()
}

dependencies {
    androidTestImplementation "io.qameta.allure:allure-kotlin-model:$LATEST_VERSION"
    androidTestImplementation "io.qameta.allure:allure-kotlin-commons:$LATEST_VERSION"
    androidTestImplementation "io.qameta.allure:allure-kotlin-junit4:$LATEST_VERSION"
    androidTestImplementation "io.qameta.allure:allure-kotlin-android:$LATEST_VERSION"
}

Attaching listener

AndroidX Test introduced a new AndroidJUnit4 class runner that can be used for both Robolectric and on-device instrumentation tests. The same pattern is used for AllureAndroidJUnit4 class runner. It attaches the allure listener to current class runner, but under the hood it uses AndroidJUnit4. All you need to do is to add @RunWith(AllureAndroidJUnit4::class) annotation to your test.

@RunWith(AllureAndroidJUnit4::class)
class MyInstrumentationTest {
    ...
}

Using AllureAndroidJUnit4 over class - works for both robolectric and on-device tests.

Robolectric tests

Robolectric tests are simple unit tests, hence the API is the same. The report data will be placed in the same place as for unit tests.

On-device instrumentation tests

You can also use testInstrumentationRunner for setting up runner.

android {
    defaultConfig {
        testInstrumentationRunner "io.qameta.allure.android.runners.AllureAndroidJUnitRunner"
    }
}

Integration

As on-device instrumentation test run on an actual device, the results have to be saved there as well. You don't need to add any permissions to manifest: results are saved in an app files directory, e.g. /data/data/com.example/files/allure-results.

After the tests are finished you can move the results to the external storage and pull the files using an adb like this one:

# Assuming your package is com.example
adb exec-out run-as com.example sh -c 'cd /data/data/com.example/files && tar cf - allure-results' > allure-results.tar

# Or using pull
$ adb shell
$ run-as com.example sh -c 'cd /data/data/com.example/files && tar cf - allure-results' | tar xvf - -C /data/local/tmp
# Ignore the permission errors
$ exit
$ adb pull /data/local/tmp/allure-results

Finally, you can generate the report via Allure CLI (see the Allure Documentation) or generate report with allure-gradle plugin.

Orchestrator TestStorage

When tests clears app data between each tests then saving test results in app storage will not work because old test results will be cleared when app data is cleared. To save test results directly on sdcard new TestStorage from androidx.test.services can be used.

Enabling test storage for automation tests:

• add allure.results.useTestStorage=true to allure.properties in androidTest resources
• add androidTestUtil("androidx.test:orchestrator:VERSION} to your app dependencies (if you do not have it already)

After that allure will use TestStorage to save test results. Test results will be saved by default into /sdcard/googletest/test_outputfiles/allure-results To get those files from device you can use e.g adb exec-out sh -c 'cd /sdcard/googletest/test_outputfiles && tar cf - allure-results' | tar xvf - -C /output/dir

NOTE: allure-results folder name can be changed using allure.results.directory property.

Features

The Allure Android API includes couple of features to make your reports a bit better.

Screenshot attachment

Screenshot can be taken and appended as an attachment to step or test in which they were executed:

@Test
fun screenshotExample() {
    step("Step screenshot") {
        allureScreenshot(name = "ss_step", quality = 90, scale = 1.0f)
    }
    allureScreenshot(name = "ss_test", quality = 50, scale = 1.0f)
}

Screenshot rule

Test rule to make the screenshot after each test and attach it to the test report. It includes a mode parameter which decides for which tests to make a screenshot:

• SUCCESS - only successful tests
• FAILURE - only failed tests
• END - all tests

@get:Rule
val logcatRule = ScreenshotRule(mode = ScreenshotRule.Mode.END, screenshotName = "ss_end")

Logcat rule

Test rule that clears the logcat before each test and appends the log dump as an attachment in case of failure.

@get:Rule
val logcatRule = LogcatRule()

Window hierarchy rule

You can use WindowHierarchyRule to capture a window hierarchy via uiautomator in case of Throwable during test.

@get:Rule
val windowHierarchyRule = WindowHierarchyRule()

Samples

Different examples of allure-kotlin usage are presented in samples directory. This includes:

• junit4-android - complete Android sample with unit tests, robolectric tests and on device instrumentation tests

Connection with allure-java

Following modules have been migrated:

• allure-model -> allure-kotlin-model
• allure-java-commons -> allure-kotlin-commons
• allure-java-commons-test -> allure-kotlin-commons-test

Following changes have to be made in order to keep the compatibility with Java 1.6:

• java.util.Optional (Java 1.8+) -> Kotlin null type & safe call operators
• java.util.stream.* (Java 1.8+) -> Kotlin collection operators
• java.nio.file.* (Java 1.7+) -> migrating form Path to File
• repeatable annotations (Java 1.8+) -> no alternatives, feature not supported by JVM 1.6

The only part that was not migrated is aspects support.

Contributing

Thanks to all people who contributed. Especially @kamildziadek who started allure-kotlin. Contribute.

License

The Allure Framework is released under version 2.0 of the Apache License.