jonathanlermitage / manon

1. Project
2. Author
3. Compilation and test
4. Tips
5. License

Project

Some experimentation with Spring Boot 2, JDK11+, JUnit5, TestNG, SQL (HSQLDB, MariaDB, PostgreSQL), NoSQL (Redis, MongoDB), Docker, ELK stack, etc. It demonstrates usage of:

• Maven (latest version) and Gradle build tools
  • migration from Maven to Gradle (but I'll stick to Maven). Stable and functional, but still needs some improvements. See commit cf79b9c (spring5-mvn-to-gradle branch)
• Spring Boot 2 (Spring Framework 5)
  • you can compare spring4 and spring5 branches to study migration from Spring Boot 1 (Spring Framework 4) to Spring Boot 2
• replace Tomcat by Undertow. See last commit of spring5-light-undertow branch. Also, it seems to fix some annoying ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected issues when sending a large number (2000+) of HTTP requests with message body
• Spring Web with a REST API
• Spring Security, to authenticate users via auth_basic and fetch authentication data from SQL database
• Spring Data to serve data from a database
  • simple test to show when a transaction rollback occurs (tip: Error and RuntimeException trigger rollback, Exception doesn't!). See last commit of spring5-light-trx-rollback branch. See also AbstractManagedException javadoc for details.
• use MapStruct to map entity to DTO (this is a short introduction only, mapping can be done in both directions and extended to all entities). See last commit of spring5-light-mapstruct-mapper branch
• Querydsl support for dynamic SQL querying and pagination with Spring Data JPA and Spring Web. See last commit of spring5-light-querydsl-jpa branch
• Spring Batch to schedule and manage some tasks. See commit c0e3422 (spring5-light-batch branch)
  • use an additional datasource to handle Spring Batch tables (via Spring Boot 2.2 new @BatchDataSource annotation), and initialize two datasources via Flyway. See last commit of spring5-light-extra-datasource-for-springbatch branch
• Spring Cache via Redis for application, and prefer an embedded cache provider during tests. See commits a911f6a, 7e26822 and ae6e0e6 (spring5-redis branch)
• Spring Retry. See last commit of spring5-light-retry branch
• JWT (JSON Web Token) authentication (migration from Auth Basic). See last commit of spring5-light-jwt-auth branch
  • manage JWT in database to invalidate living tokens when needed (e.g. a user can disconnect his multiple sessions). See last commit of spring5-light-jwt-managed-in-db branch
  • add some cache (Redis server or in-memory map). See last commit of spring5-light-redis-cache-on-jwt-refs branch
• Flyway support (database migration). See last commit of spring5-light-flyway branch
• GraphQL support (queries and mutations). See last commit of spring5-light-graphql branch. Please note that you may see instrumentation warnings during JaCoCo test coverage, and some frameworks don't work: cant use Mockito's SpyBean (issue) on a Spring service used by GraphQL; can't load Flyway. That's why it's not merged into main branch yet
• send emails to a GreenMail sand boxed email server and check retrieval. See last commit of spring5-light-mailing branch
• Asynchronous Methods support. See last commit of spring5-light-async branch
• Integration Tests and (some) Unit Tests via JUnit5 and REST Assured. I prefer TestNG for its keywords, dataproviders and maturity, but JUnit is the most used testing framework. REST Assured helped me to test API without Spring's magic
  • migration from TestNG to JUnit5 (with Spring Boot, dataproviders, expected exceptions, beforeClass and afterClass annotations). See last commit of spring5-light-testng-to-junit5 branch
  • some test parallelization with JUnit5. See last commit of spring5-light-junit5-parallel-execution branch
  • Java architecture tests via ArchUnit. See last commit of spring5-light-archunit branch
  • JSON tests via JsonUnit Hamcrests matchers. See last commit of jsonunit branch
  • use dockerized MariaDB on Travis CI instead of embedded HSQLDB. See last commit of spring5-light-travis-with-mariadb branch
  • use dockerized PostgreSQL on Travis CI. See last commit of postgres branch
  • split Unit and Integration Tests: run Unit Tests first and, if they don't fail, run Integration Tests. See last commit of spring5-light-separate-integ-unit branch. Due to JDK11+JUnit5+Failsafe issues (that may be fixed in Failsafe 3.0.0), I run both Integration and Unit tests via Surefire Maven plugin I now use Surefire + Failsafe, and it works well (see commit fafac63)
  • use java-faker to generate random test data. See last commit of javafaker branch
  • log web requests (client info, headers, query string, payload) during tests. See last commit of log-web-requests branch
• Pitest mutation testing. See last commit of pitest branch
• Gatling benchmark. See last commit of spring5-light-gatling branch and dedicated README.md file
• migration from MongoDB (used to store regular and authentication data) to MariaDB. See last commit of spring5-light-mongo-to-sql branch
  • tests work with an embedded MongoDB instance (for data) and HSQLDB (for Spring Batch internals only), that means you don't have to install any database to test project
    • use embedded MongoDB during tests. See commits 37e1be5 and 161d321 (spring5-embedmongo branch)
    • make embedded MongoDB work with version from 3.6 to 4.0: see commit a75a917 (spring5-light-mongo4.0.x branch)
• replace HSQLDB by H2. See commit ae4701e (spring5-light-hsqldb-to-h2 branch)
• migration to Java 8 new Date and Time API (JSR-310). See last commit of spring5-light-java8-datetime-api branch
• integration with some free (for open-source) services like AppVeyor (Windows CI), Travis (Linux and MacOS CI), Cirrus (CI), MS Azure DevOps Pipelines (CI), CodeCov (code coverage), SonarCloud (code quality), LGTM (code quality), Better Code Hub (code quality)
• Maven and Gradle wrappers, and a do Bash script that helps you to launch some usefull commands
• code coverage thanks to JaCoCo Maven and Gradle plugin
• some AOP to capture performance of API endpoints
• Spring Actuator web endpoints configured
  • show Git information (branch, commit...) into Actuator's info endpoint. See last commit of actuator-git-info branch
• Swagger UI to provide documentation about REST API endpoints
  • run application and check http://localhost:8080/swagger-ui.html. See commit 834852c (spring5-swagger branch)
  • migration to OpenAPI 3 with Springdoc. Check Swagger UI http://localhost:8080/swagger-ui.html, JSON OpenAPI docs http://localhost:8080/v3/api-docs, YAML OpenAPI docs http://localhost:8080/v3/api-docs.yaml. See last commit of springdoc-openapi-v3 branch
• Docker builds without Docker daemon thanks to Jib. Docker Compose support
  • See DEPLOY.md to package and run application. See commit de7335b (spring5-light-docker-jib branch)
• ELK stack (ElasticSearch Logstash Kibana) via Docker plus Docker Compose to parse application and Nginx logs, and Cerebro to monitor ElasticSearch node
  • see DEPLOY.md to package and run application. Also, see commits 6b76e37 and 966969f (spring5-light-elk branch)
• Prometheus to monitor metrics. They are fetched from Spring Boot Actuator's Prometheus endpoint. See last commit of actuator-metrics-to-prometheus branch
  • Grafana to fetch and display metrics from Prometheus. See last commit of prometheus-to-grafana branch
• integration with SpotBugs (FindBugs)
• integration with Maven enforcer plugin. See last commit of spring5-light-maven-enforcer-plugin branch
• integration with OWASP dependency check plugin. See last commit of spring5-light-owasp-dependency-check-plugin branch
• Spring Boot dependency management without Spring Boot parent POM. See last commit of spring5-light-spring-boot-dependencies branch
• use Glowroot monitoring tool (via regular Docker build, not Jib). Run ./do docker up then check http://localhost:4000. See last commit of spring5-light-glowroot-docker branch
• Spring Boot Admin (server with UI, and client). Run ./do docker up then check http://localhost:28080. See last commit of spring-boot-admin branch (not merged into main branch yet)

For fun and to show some skills 🐱

Author

Jonathan Lermitage ([email protected])
Linkedin profile: jonathan-lermitage-092711142

Compilation and test

First, install JDK8+ and Maven3+.

You can now use the ./do Linux Bash script:

do help         show this help message
do fixgit       set executable flag on git index for required files
do fixexec      set executable permission for required files (chmod +x on bash scripts)
do normgit      call git add --update --renormalize
do conv         generate a Dependency Convergence report in target/site/dependency-convergence.html
do oga          check for deprecated groupId and artifactId couples
do owasp        generate a OWASP dependencies vulnerabilities report in target/dependency-check-report.html
do t            test using embedded HSQLDB
do td           test using dockerized MariaDB and Redis (container is started and stopped by script)
do td-postgres  test using dockerized PostgreSQL and Redis (container is started and stopped by script)
do ut           run unit tests only, no integration tests
do tc           run unit + integration tests and generate coverage data
do itc          run integration tests only and generate coverage data
do gatling      benchmark application via a Gatling container (run './do up' first to start application)
do sc           compute and upload Sonar analysis to SonarCloud
do tsc          similar to "do tc" then "do sc"
do sb           scan with SpotBugs then show GUI
do b            build without testing
do c            clean
do p            package application to manon.jar
do rd           package and run application with dev-mariadb profile 
do w 3.5.2      set or upgrade Maven wrapper to 3.5.2
do cv           check plugins and dependencies versions
do uv           update plugins and dependencies versions
do dt           show dependencies tree
do rmi          stop Docker application, then remove its containers and images
do cdi          clean up dangling Docker images
do dockerreset  stop and remove all containers, remove all images and prune volumes
do docker       build Docker image with Dockerfile to a Docker daemon
do dockerpull   pull 3rd party Docker containers
do jib          build Docker image with Jib to a Docker daemon
do jibtar       build and save Docker image with Jib to a tarball
do up           create and start containers via docker-compose
do stop         stop containers via docker-compose
do upelk        create and start ELK containers via docker-compose
do stopelk      stop ELK containers via docker-compose
do upcerebro    create and start Cerebro container via docker-compose
do stopcerebro  stop Cerebro container via docker-compose
do maria        connect to dockerized MariaDB business database by calling MySQL Client provided by container
do maria-batch  connect to dockerized MariaDB Spring Batch database by calling MySQL Client provided by container
do mariah       connect to dockerized MariaDB business database by calling host MySQL Client (mysql-client package must be installed)
do mariah-batch connect to dockerized MariaDB Spring Batch database by calling host MySQL Client (mysql-client package must be installed)
do e2e          run some end-to-end (e2e) tests with Docker. Application image is built from a Dockerfile
do e2ejib       run some end-to-end (e2e) tests with Docker. Application image is built with Jib

Nota: the Linux Bash script can chain parameters, e.g.: ./do cdi rmi w 3.6.0 c tc docker up.
Nota: a Windows do.cmd script exists, but it's limited to some basic features. Run do.cmd help for details.

Tips

Proxy

You may experience connection issues with many do commands that use Maven Wrapper if you are behind a proxy. Maven Wrapper doesn't pick proxy configuration from Maven's settings, so you should give some proxy parameters to JVM. See this excellent Stack Overflow answer.

Git

Some usefull git alias to put into your .gitconfig file:

[alias]

# Show a pretty commit log
ls = log --pretty=format:"%C(green)%h\\ %C(yellow)[%ad]%Cred%d\\ %Creset%s%C(cyan)\\ [%cn]" --decorate --date=relative

# Checkout a merge request, example: git mr upstream 6
mr = !sh -c 'git fetch $1 merge-requests/$2/head:mr-$1-$2 && git checkout mr-$1-$2' -

# Credit an author on the latest commit
credit = "!f() { git commit --amend --author \"$1 <$2>\" -C HEAD; }; f"

# Interactive rebase with the given number of latest commits
reb = "!r() { git rebase -i HEAD~$1; }; r"

# Show the diff between the latest commit and the current state
d = !"git diff-index --quiet HEAD -- || clear; git diff --patch-with-stat"

License

MIT License. In other words, you can do what you want: this project is entirely OpenSource, Free and Gratis.