Soteria

Information about who was Soteria.

If you have any question submit an issue.

Table of Contents

• Presentations
• Setup steps
• soteria.json
  • Root level
  • Modules
    • How to make sure library A is always version x.y and Provided ?
    • How to remove a dependency D completely from a library A ?
    • Is there an easy way to build the dependenciesToRemove tree ?
• Features
  • Scala Style
  • Coveralls
  • For fat-jar assembly build
  • !!!Dangerous!!! Allow compilation even with vulnerability
    • Vulnerability by-pass
    • Compilation warning by pass
  • Debug
• Publishing
• Authors
  • Leo Benkel

Created by gh-md-toc

Presentations

Feel free to watch the talk given at Scala in the city (2020-05-28) with the slide deck.

Setup steps

1. Make sure you are using SBT 1.2.x.
2. Add to ./project/soteria.sbt in your project:

   addSbtPlugin("com.leobenkel" % "soteria" % soteriaVersion)
   

   The latest release is
3. Make sure to have a config file. Take a look at soteria.json for examples
4. If you need a fat-jar:
   1. Add to your build.sbt the following lines:

      assemblyOption in assembly := soteriaAssemblySettings.value
      enablePlugins(DockerPlugin)
      

5. Clean up your build.sbt by removing all pre-set settings:
   1. dependencyOverrides
   2. scalacOptions
   3. All settings related to sbt-assembly and sbt-docker
   4. All test options in Test:
      1. javaOptions in Test
      2. testOptions in Test
      3. parallelExecution in Test
      4. fork in Test
6. Remove all plugins that are already included for you:
   1. sbt-scoverage
   2. sbt-assembly
   3. sbt-docker
   4. sbt-dependency-graph
   5. sbt-scalafix
   6. sbt-scalafmt
   7. scalastyle-sbt-plugin
7. You can run your project the same as before. There can be compilation issues due to vulnerable dependencies.
8. Dangerous! If you are not able to fix compiler issues, add: soteriaSoftOnCompilerWarning := true
9. Dangerous! If you are not able to fix dependencies issues, add: soteriaSoft := true
10. You can now then fix scala style issues:
    1. Run sbt soteriaCheckScalaStyle
    2. Run sbt soteriaCheckScalaFix
    3. Run sbt soteriaCheckScalaFmt
       1. If you are ready to rewrite the broken files:
          1. Create a clean branch
          2. Run sbt soteriaCheckScalaFmtRun

soteria.json

Root level

To override where the config file is read from, update the setting soteriaConfPath in your build.sbt. This setting can be a URL starting by http:// or https:// or a local file path. By default it will search for ./soteria.json.

Root level:

• dockerImage: To set in which image the fat-jar will be built.
• sbtVersion: SBTVersion to enforce. If a project is trying to compile with a different version, it will break.
• scalaVersions: Is an array of authorized scala Version. This is an array and not a value to allow Spark/Play project on different version.
• scalaCFlags: The list of compile flag to add to the build process. If soteriaSoftOnCompilerWarning is not true, then -Xfatal-warnings will be added as well to trigger a compilation failure.
• modules: This is where the bulk of the settings are living.
  • The structure is groupId|com.organization -> artifactName -> description of the constraints

Modules

The modules are the constraint enforced by the plugin related to each dependencies.

The path to each module is groupId|com.organization -> artifactName -> description of the constraints.

A module can accept those keys:

• version: Either None or a version number. If the library is added with a different version number, the compilation will fail
• exactName: Default is True if absent. If false, the artifactName can just be a start. It is used for instance where you want to enforce a version for a library and related ones. circe or spark- can be good examples.
• excludeName: Default empty. It is used to exclude libraries that would be catch by the name + exactName:false. It is used for instance in Play project where you would enforce something for all libraries starting by play- except a few that are behind on version numbers.
• needDoublePercent: By default is false. If true, the conversion to sbt.ModuleID will be with %% instead of %. The same way it would be in the build.sbt.
• shouldDownload: Is true by default and is only used for sbt soteriaGetAllDependencies.
• overrideIsEnough: Default is true. This is related to dependenciesToRemove.
  • If overrideIsEnough is true, the library will be added to dependencyOverrides.
  • If overrideIsEnough is false, the library will be converted to an exclusion rule.
• forbidden: Default is null. If this is set, and the library is added, the message will be displayed as a build failure. For instance you can use it to forbid one MySQL library and advise to use a different one.
• shouldBeProvided: Default is false. If true, the compilation will fail if the library is not set to Provided in build.sbt. It is used for Spark.
• dependenciesToRemove: This is a list of groupID | artifactName libraries to remove from this library. This is when overrideIsEnough come into play.
• scalaVersionsFilter: This field is a list of scala versions, following this format: [+-][MajorVersion].[MinorVersion]<.[SmallVersion]>.

How to make sure library A is always version x.y and Provided ?

"modules": {
    "groupID.A": {
        "artifactName-A": {
            "version": "x.y",
            "shouldBeProvided": true
        }
    }
}

How to remove a dependency D completely from a library A ?

"modules": {
    "groupID.D": {
        "artifactName-D": {
            "version": "None",
            "overrideIsEnough": false
        }
    },
    "groupID.A": {
        "artifactName-A": {
            "dependenciesToRemove": [ 
                "groupID.D | artifactName-D"
            ],
            "version": "vA.A"
        }
    }
}

The dependenciesToRemove in A, will search for D. Since overrideIsEnough is false in D, the plugin will remove D from A using an ExclusionRule. Then, the plugin gather all the library which have been removed, and add them back with the appropriate version. In this case the version of D is None, so it will not be added back.

Is there an easy way to build the dependenciesToRemove tree ?

Yes there is !

First assemble your json:

"modules": {
   "groupID.D": {
       "artifactName-D": {
           "version": "None",
           "overrideIsEnough": false
       }
   },
   "groupID.A": {
       "artifactName-A": {
           "version": "vA.A"
       }
   },
   "groupID.B": {
       "artifactName-B": {
           "version": "vB.B"
       }
   }
}

Then run sbt soteriaDebugAllModules.

This will:

1. Remove all the dependencies from your build.sbt
2. List all the known libraries from your config file
3. Add one library at a time, compile and get the fetched dependencies
4. Compare the fetch dependencies with the known dependencies from your config file
5. When all the libraries have been reviewed, the plugin will display a new json payload that you can just copy paste with all the dependenciesToRemove set to the knowledge you have in your json.

Features

Scala Style

The sbt plugin includes ScalaFix, ScalaStyle and ScalaFmt.

1. Check that you have .scalafix.conf, .scalafmt.conf and scalastyle-config.xml in your project
   • Feel free to copy the one present in this repo to follow the same style guides
2. Run sbt soteriaCheckScalaCheckAll to check that everything is correct.
   • You can run each system independently with:
     • sbt soteriaCheckScalaStyle
     • sbt soteriaCheckScalaFix
     • sbt soteriaCheckScalaFmt
3. To apply the fix for ScalaFmt, you can run sbt soteriaCheckScalaFmtRun

Coveralls

Locally

You can run

sbt soteriaRunTestCoverage

to generate local reporting.

Just open:

./target/scala-2.xx/sbt-1.0/scoverage-report/index.html

Submit to coveralls

To submit to coveralls

sbt soteriaRunSubmitCoverage

This will generate the reports and submit it to coveralls using your COVERALLS token, set in the env var: COVERALLS_REPO_TOKEN.

For fat-jar assembly build

You need to add

assemblyOption in assembly := soteriaAssemblySettings.value
enablePlugins(DockerPlugin)

to your build.sbt file.

You can now call sbt docker to create the fat-jar. It will be located at ./target/docker/0/*.jar.

To change in which docker image the build is ran, you can change dockerImage in soteria.json.

!!!Dangerous!!! Allow compilation even with vulnerability

Vulnerability by-pass

By default, you won't be able to compile if you have errors in your build.

If you need time to fix several issues, you can add:

soteriaSoft := true

to your build.sbt file while you are fixing them.

Compilation warning by pass

If you also want to allow compiler warning, you will need to add:

soteriaSoftOnCompilerWarning := true

to your build.sbt file while you are fixing them

Debug

To print more or less logs for this plugin, you can set soteriaLogLevel.

For Debug:

soteriaLogLevel := Level.Debug

For Error only:

soteriaLogLevel := Level.Error

Publishing

• Update version number in VERSION file.
• Deploy the updated plugin locally: make publishLocal
• Update version number in ./project/soteria.sbt.
• Run the plugin on itself with make publishLocal
• Publish: make publish

Authors

Leo Benkel