Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Coherence is a scalable, fault-tolerant, cloud-ready, distributed platform for building grid-based applications and reliably storing data. The product is used at scale, for both compute and raw storage, in a vast array of industries such as critical financial trading systems, high performance telecommunication products and eCommerce applications.

Typically these deployments do not tolerate any downtime and Coherence is chosen due to its novel features in death detection, application data evolvability, and the robust, battle-hardened core of the product that enables it to be seamlessly deployed and adapted within any ecosystem.

At a high level, Coherence provides an implementation of the familiar Map<K,V> interface but rather than storing the associated data in the local process it is partitioned (or sharded) across a number of designated remote nodes. This partitioning enables applications to not only distribute (and therefore scale) their storage across multiple processes, machines, racks, and data centers but also to perform grid-based processing to truly harness the CPU resources of the machines.

The Coherence interface NamedMap<K,V> (an extension of Map<K,V>) provides methods to query, aggregate (map/reduce style) and compute (send functions to storage nodes for locally executed mutations) the data set. These capabilities, in addition to numerous other features, enable Coherence to be used as a framework for writing robust, distributed applications.

Downloading Coherence Community Edition

As Coherence is generally embedded into an application by using Coherence APIs, the natural place to consume this dependency is from Maven:

<dependencies>
    <dependency>
        <groupId>com.oracle.coherence.ce</groupId>
        <artifactId>coherence</artifactId>
        <version>20.12</version>
    </dependency>
</dependencies>

You can also get Coherence from the official Docker site. For other language clients, use C++ and .NET and for the non-community edition, see Oracle Technology Network.

Overview

First and foremost, Coherence provides a fundamental service that is responsible for all facets of clustering and is a common denominator / building block for all other Coherence services. This service, referred to as 'service 0' internally, ensures that the mesh of members is maintained and responsive, taking action to collaboratively evict, shun, or in some cases, voluntarily depart the cluster when deemed necessary. As members join and leave the cluster, other Coherence services are notified, thus enabling those services to react accordingly.

Note: This part of the Coherence product has been in production for more that 10 years, being the subject of some extensive and imaginative testing. While this feature has been discussed here, it certainly is not something that customers, generally, interact with directly, but is important to be aware of.

Coherence services build on top of the cluster service. The key implementations to be aware of are PartitionedService, InvocationService, and ProxyService.

In the majority of cases, customers deal with maps. A map is represented by an implementation of NamedMap<K,V>. A NamedMap is hosted by a service, generally the PartitionedService, and is the entry point to store, retrieve, aggregate, query, and stream data.

Coherence Maps provide a number of features:

Fundamental key-based access: get/put getAll/putAll.
Client-side and storage-side events:
- MapListeners to asynchronously notify clients of changes to data.
- EventInterceptors (either sync or async) to be notified storage level events, including mutations, partition transfer, failover, and so on.
NearCaches - Locally cached data based on previous requests with local content invalidated upon changes in the storage tier.
ViewCaches - Locally stored view of remote data that can be a subset based on a predicate and is kept in sync, real time.
Queries - Distributed, parallel query evaluation to return matching key, values, or entries with potential to optimize performance with indices.
Aggregations - A map/reduce style aggregation where data is aggregated in parallel on all storage nodes, and results streamed back to the client for aggregation of those results to produce a final result.
Data local processing - Ability to send a function to the relevant storage node to execute processing logic for the appropriate entries with exclusive access.
Partition local transactions - Ability to perform scalable transactions by associating data (thus being on the same partition) and manipulating other entries on the same partition, potentially across different maps.
Non-blocking / async NamedMap API
C++ and .NET clients - Access the same NamedMap API from either C++ or .NET.
Portable Object Format - Optimized serialization format, with the ability to navigate the serialized form for optimized queries, aggregations, or data processing.
Integration with Databases - Database and third party data integration with CacheStores, including both synchronous or asynchronous writes.
CohQL - Ansi-style query language with a console for adhoc queries.
Topics - Distributed topics implementation that offers pub/sub messaging with the storage capacity, the cluster, and parallelizable subscribers.

Coherence also provides a number of non-functional features:

Rock solid clustering - Highly tuned and robust clustering stack that enables Coherence to scale to thousands of members in a cluster with thousands of partitions and terabytes of data being accessed, mutated, queried, and aggregated concurrently.
Safety first - Resilient data management that ensures backup copies are on distinct machines, racks, or sites, and the ability to maintain multiple backups.
24/7 Availability - Zero downtime with rolling redeployment of cluster members to upgrade application or product versions.
- Backward and forward compatibility of product upgrades, including major versions.
Persistent Maps - Ability to use local file system persistence (thus avoid extra network hops) and leverage Coherence consensus protocols to perform distributed disk recovery when appropriate.
Distributed State Snapshot - Ability to perform distributed point-in-time snapshot of cluster state, and recover snapshot in this or a different cluster (leverages persistence feature).
Lossy redundancy - Ability to reduce the redundancy guarantee by making backups and/or persistence asynchronous from a client perspective.
Single Mangement View - Provides insight into the cluster with a single JMX server that provides a view of all members of the cluster.
Management over REST - All JMX data and operations can be performed over REST, including cluster wide thread dumps and heapdumps.
Non-cluster Access - Provides access to the cluster from the outside via proxies, for distant (high latency) clients and for non-java languages such as C++ and .NET.
Kubernetes friendly - Enables seamless and safe deployment of applications to k8s with our own operator.

Hello Coherence

Prerequisites

Java - JDK 8 or higher
Maven - 3.6.3 or higher

Note: When building Coherence, JDK 11 is required.

CLI Hello Coherence

The following example illustrated starting a storage enabled Coherence Server, followed by a storage disabled Coherence Console. Using the console, data is inserted, retrieved, and then the console is terminated. The console is restarted and data is once again retrieved to illustrate the permanence of the data.

Note: This example uses the OOTB cache configuration and therefore, explicitly specifying the console is storage disabled is unnecessary.

Note: Coherence cluster members discover each other via one of two mechanisms; multicast (default) or Well Known Addressing (deterministic broadcast). If your system does not support mutlicast, enable WKA by specifying -Dcoherence.wka=localhost for both processes started in the following console examples.

CohQL Console

$> mvn -DgroupId=com.oracle.coherence.ce -DartifactId=coherence -Dversion=20.12 dependency:get

$> export COH_JAR=~/.m2/repository/com/oracle/coherence/ce/coherence/20.12/coherence-20.12.jar

$> java -jar $COH_JAR &

$> java -cp $COH_JAR com.tangosol.coherence.dslquery.QueryPlus

CohQL> select * from welcomes

CohQL> insert into welcomes key 'english' value 'Hello'

CohQL> insert into welcomes key 'spanish' value 'Hola'

CohQL> insert into welcomes key 'french' value 'Bonjour'

CohQL> select key(), value() from welcomes
Results
["french", "Bonjour"]
["english", "Hello"]
["spanish", "Hola"]

CohQL> bye

$> java -cp $COH_JAR com.tangosol.coherence.dslquery.QueryPlus

CohQL> select key(), value() from welcomes
Results
["french", "Bonjour"]
["english", "Hello"]
["spanish", "Hola"]

CohQL> bye

$> kill %1

Coherence Console

$> mvn -DgroupId=com.oracle.coherence.ce -DartifactId=coherence -Dversion=20.12 dependency:get

$> export COH_JAR=~/.m2/repository/com/oracle/coherence/ce/coherence/20.12/coherence-20.12.jar

$> java -jar $COH_JAR &

$> java -cp $COH_JAR com.tangosol.net.CacheFactory

Map (?): cache welcomes

Map (welcomes): get english
null

Map (welcomes): put english Hello
null

Map (welcomes): put spanish Hola
null

Map (welcomes): put french Bonjour
null

Map (welcomes): get english
Hello

Map (welcomes): list
french = Bonjour
spanish = Hola
english = Hello

Map (welcomes): bye

$> java -cp $COH_JAR com.tangosol.net.CacheFactory

Map (?): cache welcomes

Map (welcomes): list
french = Bonjour
spanish = Hola
english = Hello

Map (welcomes): bye

$> kill %1

Programmatic Hello Coherence Example

The following example illustrates starting a storage enabled Coherence server, followed by running the HelloCoherence application. The HelloCoherence application inserts and retrieves data from the Coherence server.

Build `HelloCoherence`

Create a maven project either manually or by using an archetype such as maven-archetype-quickstart
Add a dependency to the pom file:

    <dependency>
        <groupId>com.oracle.coherence.ce</groupId>
        <artifactId>coherence</artifactId>
        <version>20.12</version>
    </dependency>

Copy and paste the following source to a file named src/main/java/HelloCoherence.java:

    import com.tangosol.net.CacheFactory;
    import com.tangosol.net.NamedMap;

    public class HelloCoherence
        {
        // ----- static methods -------------------------------------------------

        public static void main(String[] asArgs)
            {
            NamedMap<String, String> map = CacheFactory.getCache("welcomes");

            System.out.printf("Accessing map \"%s\" containing %d entries\n",
                    map.getName(),
                    map.size());

            map.put("english", "Hello");
            map.put("spanish", "Hola");
            map.put("french" , "Bonjour");

            // list
            map.entrySet().forEach(System.out::println);
            }
        }

Compile the maven project:

    mvn package

Start a Storage Server

    mvn exec:java -Dexec.mainClass="com.tangosol.net.DefaultCacheServer" &

Run HelloCoherence

    mvn exec:java -Dexec.mainClass="HelloCoherence"

Confirm you see output including the following:

    Accessing map "welcomes" containing 3 entries
    ConverterEntry{Key="french", Value="Bonjour"}
    ConverterEntry{Key="spanish", Value="Hola"}
    ConverterEntry{Key="english", Value="Hello"}

Kill the storage server started previously:

    kill %1

Building

In order to build Coherence CE, please ensure that you have JDK 11 installed.

Note: When using Coherence merely as a dependency in a project, without intending to build the project from source code, then an installed JDK 8 version is sufficient.

$> git clone [email protected]:oracle/coherence.git
$> cd coherence/prj

# build all modules
$> mvn clean install

# build all modules skipping tests
$> mvn clean install -DskipTests

# build a specific module, including all dependent modules and run tests
$> mvn -am -pl test/functional/persistence clean verify

# only build coherence.jar without running tests
$> mvn -am -pl coherence clean install -DskipTests

# only build coherence.jar and skip compilation of CDBs and tests
$> mvn -am -pl coherence clean install -DskipTests -Dtde.compile.not.required

Integrations

Documentation