Run MongoDB Atlas locally for testing
Prerequisite
-
Java 8+
-
Docker
-
Chart shows local and remote docker support for replicaSetNumber
replicaSetNumber local docker host local docker host running tests from inside a container with mapping the Docker socket remote docker daemon availability of an arbiter node 1 + + + - from 2 to 7 (including) only if adding either host.docker.internal(your Docker version should support it) ordockerhostto the OS host file. See Supported features for details+ + +
Tip: A single node replica set is the fastest among others. That is the default mode for MongoDbReplicaSet. However, to use only it, consider the Testcontainers MongoDB module on GitHub
Getting it
- Gradle:
dependencies {
testCompile("com.github.silaev:mongodb-replica-set:${LATEST_RELEASE}")
}- Maven:
<dependencies>
<dependency>
<groupId>com.github.silaev</groupId>
<artifactId>mongodb-replica-set</artifactId>
<version>${LATEST_RELEASE}</version>
<scope>test</scope>
</dependency>
</dependencies>Replace ${LATEST_RELEASE} with the Latest Version Number If you cannot find a release on Maven, please, use Jitpack
Run on Apple silicon
Use digests for linux/arm64/v8 and Docker Desktop for Apple silicon supporting host.docker.internal (should be in the OS host file). Examples: 1.
MongoDbReplicaSet.builder()
.mongoDockerImageName("mongo@sha256:8a823923d80e819e21ee6c179eabf42460b6b7d8ac3dd5f35b59419ae5413640")
.useHostDockerInternal(true)
.build()`
2
./gradlew clean build -DmongoReplicaSetProperties.mongoDockerImageName=mongo@sha256:8a823923d80e819e21ee6c179eabf42460b6b7d8ac3dd5f35b59419ae5413640 -DmongoReplicaSetProperties.useHostDockerInternal=true
MongoDB versions that MongoDbReplicaSet is constantly tested against
| version | transaction support |
|---|---|
| 3.6.14 | - |
| 4.0.12 | + |
| 4.2.8 | + |
| 4.4.4 | + |
| 5.0.5 | + |
Examples
Click to see a single node example
class ITTest {
@Test
void testDefaultSingleNode() {
try (
//create a single node mongoDbReplicaSet and auto-close it afterwards
final MongoDbReplicaSet mongoDbReplicaSet = MongoDbReplicaSet.builder()
.mongoDockerImageName("mongo:4.4.4")
.build()
) {
//start it
mongoDbReplicaSet.start();
assertThat(
mongoDbReplicaSet.nodeStates(mongoDbReplicaSet.getMongoRsStatus().getMembers()),
hasItem(ReplicaSetMemberState.PRIMARY)
);
assertNotNull(mongoDbReplicaSet.getReplicaSetUrl());
}
}
}Click to see a fault tolerance example
class ITTest {
@Test
void testFaultTolerance() {
try (
//create a PSA mongoDbReplicaSet and auto-close it afterwards
final MongoDbReplicaSet mongoDbReplicaSet = MongoDbReplicaSet.builder()
//with the latest mongo:4.4.4 docker image
.mongoDockerImageName("mongo:4.4.4")
//If true then use host.docker.internal of Docker,
//otherwise take dockerhost of Qoomon docker-host.
//Make sure that your OS host file includes one of them.
//All new Docker versions support the first variant.
.useHostDockerInternal(true)
//with 2 working nodes
.replicaSetNumber(2)
//with an arbiter node
.addArbiter(true)
//create a proxy for each node to simulate network partitioning
.addToxiproxy(true)
.build()
) {
//start it
mongoDbReplicaSet.start();
assertNotNull(mongoDbReplicaSet.getReplicaSetUrl());
//get a primary node
final MongoNode masterNode = mongoDbReplicaSet.getMasterMongoNode(
mongoDbReplicaSet.getMongoRsStatus().getMembers()
);
//cut off the primary node from network
mongoDbReplicaSet.disconnectNodeFromNetwork(masterNode);
//wait until a new primary is elected that is different from the masterNode
mongoDbReplicaSet.waitForMasterReelection(masterNode);
assertThat(
mongoDbReplicaSet.nodeStates(mongoDbReplicaSet.getMongoRsStatus().getMembers()),
hasItems(
ReplicaSetMemberState.PRIMARY,
ReplicaSetMemberState.ARBITER
)
);
//bring back the disconnected masterNode
mongoDbReplicaSet.connectNodeToNetwork(masterNode);
//wait until all nodes are up and running
mongoDbReplicaSet.waitForAllMongoNodesUp();
assertThat(
mongoDbReplicaSet.nodeStates(mongoDbReplicaSet.getMongoRsStatus().getMembers()),
hasItems(
ReplicaSetMemberState.PRIMARY,
ReplicaSetMemberState.ARBITER,
ReplicaSetMemberState.SECONDARY
)
);
}
}
}- See more examples in the test sources mongodb-replica-set on github
- See a full Spring Boot + Spring Data example wms on github
Motivation
- Cross-platform solution that doesn't depend on fixed ports;
- Testing MongoDB transactions to run against an environment close to a production one;
- Testing production issues by recreating a real MongoDB replica set (currently without shards);
- Education to newcomers to the MongoDB world (learning the behaviour of a distributed NoSQL database while dealing with network partitioning, analyze the election process and so on).
General info
Click to see how to create a 3 node replica set on fixed ports via Docker manually
MongoDB starting from version 4 supports multi-document transactions only on a replica set. For example, to initialize a 3 node replica set on fixed ports via Docker, one has to do the following:
- Add
127.0.0.1 mongo1 mongo2 mongo3to the host file of an operation system; - Run in terminal:
docker network create mongo-clusterdocker run --name mongo1 -d --net mongo-cluster -p 50001:50001 mongo:4.0.10 mongod --replSet docker-rs --port 50001docker run --name mongo2 -d --net mongo-cluster -p 50002:50002 mongo:4.0.10 mongod --replSet docker-rs --port 50002docker run --name mongo3 -d --net mongo-cluster -p 50003:50003 mongo:4.0.10 mongod --replSet docker-rs --port 50003
- Prepare the following unix end of lines script (optionally put it folder scripts or use rs.add on each node):
rs.initiate({ "_id": "docker-rs", "members": [ {"_id": 0, "host": "mongo1:50001"}, {"_id": 1, "host": "mongo2:50002"}, {"_id": 2, "host": "mongo3:50003"} ] });
- Run in terminal:
docker cp scripts/ mongo1:/scripts/docker exec -it mongo1 /bin/sh -c "mongo --port 50001 < /scripts/init.js"
As we can see, there is a lot of operations to execute and we even didn't touch a non-fixed port approach. That's where the MongoDbReplicaSet might come in handy.
Supported features
| Feature | Description | default value | how to set |
|---|---|---|---|
| replicaSetNumber | The number of voting nodes in a replica set including a master one | 1 | MongoDbReplicaSet.builder() |
| awaitNodeInitAttempts | The number of approximate seconds to wait for a master or an arbiter node(if addArbiter=true) | 29 starting from 0 | MongoDBReplicaSet.builder() |
| propertyFileName | yml file located on the classpath | none | MongoDbReplicaSet.builder() |
| mongoDockerImageName | a MongoDB docker file name | mongo:4.0.10 | finds first set: 1) MongoDbReplicaSet.builder() 2) the system property mongoReplicaSetProperties.mongoDockerImageName 3) propertyFile 4) default value |
| addArbiter | whether or not to add an arbiter node to a cluster | false | MongoDbReplicaSet.builder() |
| slaveDelayTimeout | whether or not to create one master and the others as delayed members | false | MongoDbReplicaSet.builder() |
| useHostDockerInternal | If true then use host.docker.internal of Docker, otherwise take dockerhost of Qoomon docker-host |
false | finds first set: 1) MongoDbReplicaSet.builder() 2) the system property mongoReplicaSetProperties.useHostDockerInternal 3) default value |
| addToxiproxy | whether or not to create a proxy for each MongoDB node via Toxiproxy | false | MongoDbReplicaSet.builder() |
| enabled | whether or not MongoReplicaSet is enabled even if instantiated in a test | true | finds first set: 1) the system property mongoReplicaSetProperties.enabled 2) propertyFile 3) default value |
| commandLineOptions | command line options, example:Arrays.asList("--oplogSize", "50") |
emptyList | MongoDbReplicaSet.builder() |
a propertyFile.yml example:
mongoReplicaSetProperties:
enabled: false
mongoDockerImageName: mongo:4.1.13License
Additional links
Copyright
Copyright (c) 2022 Konstantin Silaev [email protected]