All Projects → IBM → Janusgraph Utils

IBM / Janusgraph Utils

Licence: apache-2.0
Develop a graph database app using JanusGraph

Programming Languages

java
68154 projects - #9 most used programming language
groovy
2714 projects

Projects that are alternatives of or similar to Janusgraph Utils

Framework
Strongly-typed JavaScript object with support for validation and error handling.
Stars: ✭ 136 (-29.17%)
Mutual labels:  schema
Structured Data Json Ld
Collection of structured data snippets in Google preferred JSON-LD format.
Stars: ✭ 157 (-18.23%)
Mutual labels:  schema
Graphql Toolkit
A set of utils for faster development of GraphQL tools
Stars: ✭ 169 (-11.98%)
Mutual labels:  schema
Schemaspy
SchemaSpy code home
Stars: ✭ 2,027 (+955.73%)
Mutual labels:  schema
Troy
Type-safe and Schema-safe Scala wrapper for Cassandra driver
Stars: ✭ 154 (-19.79%)
Mutual labels:  schema
Fielder
A field-first form library for React and React Native
Stars: ✭ 160 (-16.67%)
Mutual labels:  schema
Pandasschema
A validation library for Pandas data frames using user-friendly schemas
Stars: ✭ 135 (-29.69%)
Mutual labels:  schema
Scalable Cassandra Deployment On Kubernetes
In this code we provide a full roadmap the deployment of a multi-node scalable Cassandra cluster on Kubernetes. Cassandra understands that it is running within a cluster manager, and uses this cluster management infrastructure to help implement the application. Kubernetes concepts like Replication Controller, StatefulSets etc. are leveraged to deploy either non-persistent or persistent Cassandra clusters on Kubernetes cluster.
Stars: ✭ 184 (-4.17%)
Mutual labels:  ibmcode
Countries
🌎 Public GraphQL API for information about countries
Stars: ✭ 156 (-18.75%)
Mutual labels:  schema
Newtonsoft.json.schema
Json.NET Schema is a powerful, complete and easy to use JSON Schema framework for .NET
Stars: ✭ 167 (-13.02%)
Mutual labels:  schema
Nope Validator
A small, simple and fast JS validator. Like, wow thats fast. 🚀
Stars: ✭ 142 (-26.04%)
Mutual labels:  schema
Schema Typed
Schema for data modeling & validation
Stars: ✭ 150 (-21.87%)
Mutual labels:  schema
Flama
🔥 Fire up your API with this flamethrower
Stars: ✭ 161 (-16.15%)
Mutual labels:  schema
Node Convict
Featureful configuration management library for Node.js
Stars: ✭ 1,855 (+866.15%)
Mutual labels:  schema
Scalable Wordpress Deployment On Kubernetes
This code showcases the full power of Kubernetes clusters and shows how can we deploy the world's most popular website framework on top of world's most popular container orchestration platform.
Stars: ✭ 173 (-9.9%)
Mutual labels:  ibmcode
Graphql Cli
📟 Command line tool for common GraphQL development workflows
Stars: ✭ 1,814 (+844.79%)
Mutual labels:  schema
Json2react
Use JSON to create React Components.
Stars: ✭ 158 (-17.71%)
Mutual labels:  schema
Protobuf Dynamic
Protocol Buffers Dynamic Schema - create protobuf schemas programmatically
Stars: ✭ 186 (-3.12%)
Mutual labels:  schema
Joiful
TypeScript Declarative Validation for Joi
Stars: ✭ 177 (-7.81%)
Mutual labels:  schema
Schema Utils
Options Validation
Stars: ✭ 162 (-15.62%)
Mutual labels:  schema

Build Status

Develop a graph database app using JanusGraph

This Code Pattern contains sample data and code for running a Twitter-like application in JanusGraph. The utility code illustrates how to use OLTP APIs to define schema, ingest data, and query graph. Developers can use or modify the code to build and operate their custom graph applications, or create similar java and groovy files to interact with JanusGraph.

When the reader has completed this Code Pattern, they will understand how to:

  • Generate a synthetic graph dataset
  • Load a graph schema from json
  • Import graph data in csv files into JanusGraph database
  • Query and update graph data using Console and REST API
  • Setup and configure a distributed JanusGraph system

Flow

Prerequisites: Install and configure JanusGraph, Cassandra, ElasticSearch, janusgraph-utils

  1. The user generates Twitter sample schema and data using JanusGraph utilities
  2. The user loads schema and imports data in backend servers using JanusGraph utilities
  3. The user makes search and update requests in a REST/custom client
  4. The client app sends the REST requests to JanusGraph server
  5. The JanusGraph server interacts with backend to process and return graph data

Included components

  • Apache Cassandra: An open source, scalable, high availability database.
  • JanusGraph: A highly scalable graph database optimized for storing and querying large graphs. JanusGraph v0.1.1 was used for this code pattern development and test.

Featured technologies

  • Databases: Repository for storing and managing collections of data.
  • Java: A secure, object-oriented programming language for creating applications.

Watch the Video

Steps

Run locally

  1. Install prerequisites
  2. Clone the repo
  3. Generate the graph sample
  4. Load schema and import data
  5. Run interactive remote queries

1. Install prerequisites

NOTE: These prerequisites can be installed on one server. The instructions are written for Cassandra 3.10 and ElasticSearch 5.3.0 on Linux. Newer versions should work, but might not have been tested. The folder structures on Mac can be different. Check Cassandra and ElasticSearch official documentations for details.

Install Cassandra 3.10 on the storage server. Make the following changes in /etc/cassandra/cassandra.yaml and restart Cassandra.

start_rpc: true
rpc_address: 0.0.0.0
rpc_port: 9160
broadcast_rpc_address: x.x.x.x (your storage server ip)

Install ElasticSearch 5.3.0 on the index server. Make the following changes in /etc/elasticsearch/elasticsearch.yml and restart ElasticSearch.

network.host: x.x.x.x (your index server ip)

Install JanusGraph on the graph server:

  • Install java (1.8), maven (3.3.9, newer should work), git (2.7.5, newer should work)
  • Run git clone https://github.com/JanusGraph/janusgraph.git
  • Run the following commands in the janusgraph folder:
git checkout 4609b6731a01116e96e554140b37ad589f0ae0ca
mvn clean install -DskipTests=true
cp conf/janusgraph-cassandra-es.properties conf/janusgraph-cql-es.properties
  • Make the following changes in conf/janusgraph-cql-es.properties:
storage.backend=cql
storage.hostname=x.x.x.x (your storage server ip)
index.search.hostname=x.x.x.x (your index server ip)

Install a REST client, such as RESTClient add-on for Firefox, on the client machine.

2. Clone the repo

Clone the janusgraph-utils on the graph server and run mvn package.

git clone https://github.com/IBM/janusgraph-utils.git
cd janusgraph-utils/
mvn package

3. Generate the graph sample

Run the command in janusgraph-utils folder to generate data into /tmp folder.

./run.sh gencsv csv-conf/twitter-like-w-date.json /tmp

Modify the generated user file under /tmp so the sample queries will return with data.

sed -i -e '2s/.*/1,Indiana Jones/' /tmp/User.csv

4. Load schema and import data

A graph schema can be loaded from either the Gremlin console or a java utility. You can check the doc doc/users_guide.md for details. Alternatively, just run one command in janusgraph-utils folder to load schema and import data.

export JANUSGRAPH_HOME=~/janusgraph
./run.sh import ~/janusgraph/conf/janusgraph-cql-es.properties /tmp /tmp/schema.json /tmp/datamapper.json

5. Run interactive remote queries

Configure JanusGraph server by running these commands:

cd ~/janusgraph/conf/gremlin-server
cp ~/janusgraph-utils/samples/date-helper.groovy ../../scripts
cp ../janusgraph-cql-es.properties janusgraph-cql-es-server.properties
cp gremlin-server.yaml rest-gremlin-server.yaml

Add this line to janusgraph-cql-es-server.properties:

gremlin.graph=org.janusgraph.core.JanusGraphFactory

Change the following four lines in rest-gremlin-server.yaml:

host: x.x.x.x (your server ip)
channelizer: org.apache.tinkerpop.gremlin.server.channel.HttpChannelizer
graph: conf/gremlin-server/janusgraph-cql-es-server.properties}
scripts: [scripts/empty-sample.groovy,scripts/date-helper.groovy]}}

Start JanusGraph server:

cd ~/janusgraph; ./bin/gremlin-server.sh ./conf/gremlin-server/rest-gremlin-server.yaml

Now you can query and update graph data using REST. For example, send REST requests using RESTClient in browser with following:

Method: POST
URL: http://x.x.x.x:8182
Body: {"gremlin":“query_to_run"}

You can find sample search and insert queries in samples/twitter-like-queries.txt.

Sample output

Sample output for "Find Indiana Jones' tweets that his followers retweeted"

Links

Learn more

License

This code pattern is licensed under the Apache Software License, Version 2. Separate third party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 (DCO) and the Apache Software License, Version 2.

Apache Software License (ASL) FAQ

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].