sparkling-graph / Sparkling Graph
Licence: bsd-2-clause
SparklingGraph provides easy to use set of features that will give you ability to proces large scala graphs using Spark and GraphX.
Stars: ✭ 139
Projects that are alternatives of or similar to Sparkling Graph
Gaffer
A large-scale entity and relation database supporting aggregation of properties
Stars: ✭ 1,642 (+1081.29%)
Mutual labels: graph, spark, big-data
Verse
Reference implementation of the paper VERSE: Versatile Graph Embeddings from Similarity Measures
Stars: ✭ 98 (-29.5%)
Mutual labels: graph, graph-algorithms
Logisland
Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.
Stars: ✭ 97 (-30.22%)
Mutual labels: spark, big-data
Deepwalk C
DeepWalk implementation in C++
Stars: ✭ 88 (-36.69%)
Mutual labels: graph, graph-algorithms
Graph sampling
Graph Sampling is a python package containing various approaches which samples the original graph according to different sample sizes.
Stars: ✭ 99 (-28.78%)
Mutual labels: big-data, network-analysis
Big Data Engineering Coursera Yandex
Big Data for Data Engineers Coursera Specialization from Yandex
Stars: ✭ 71 (-48.92%)
Mutual labels: spark, big-data
Ogre
Clojure library for querying Apache TinkerPop graphs
Stars: ✭ 118 (-15.11%)
Mutual labels: graph, graph-algorithms
Depthmapx
depthmapX is a multi-platform Spatial Network Analysis Software
Stars: ✭ 120 (-13.67%)
Mutual labels: graph-algorithms, network-analysis
Setl
A simple Spark-powered ETL framework that just works 🍺
Stars: ✭ 79 (-43.17%)
Mutual labels: spark, big-data
Spark Py Notebooks
Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
Stars: ✭ 1,338 (+862.59%)
Mutual labels: spark, big-data
Rsparkling
RSparkling: Use H2O Sparkling Water from R (Spark + R + Machine Learning)
Stars: ✭ 65 (-53.24%)
Mutual labels: spark, big-data
Potiron
Potiron - Normalize, Index and Visualize Network Capture
Stars: ✭ 66 (-52.52%)
Mutual labels: graph, network-analysis
Bigdataclass
Two-day workshop that covers how to use R to interact databases and Spark
Stars: ✭ 110 (-20.86%)
Mutual labels: spark, big-data
Data Structures
Common data structures and algorithms implemented in JavaScript
Stars: ✭ 139 (+0%)
Mutual labels: graph, graph-algorithms
sparkling-graph
SparklingGraph provides easy to use set of features that will give you ability to proces large scala graphs using Spark and GraphX.
Requirements
- Scala 2.11 or 2.12
- Spark 2.4.0 (or compatible)
Versioning
Since commit 3246714 project is using git versioning (for example 0.0.7+140-32467140 or 0.0.7+140-32467140+20190402-2057-SNAPSHOT). All artifacts from now one will be published to snapshot without version overriding. New approach will also add abbility to reproduce each version. Release versions will use normal tag based approach.
Dependencies
Since commit 3246714 you can get artifacts for any master branch commits using git describe command.
Snapshot
resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"
// one or all from:
libraryDependencies += "ml.sparkling" %% "sparkling-graph-examples" % "0.0.8-SNAPSHOT"
libraryDependencies += "ml.sparkling" %% "sparkling-graph-loaders" % "0.0.8-SNAPSHOT"
libraryDependencies += "ml.sparkling" %% "sparkling-graph-operators" % "0.0.8-SNAPSHOT"
Release
// one or all from:
libraryDependencies += "ml.sparkling" %% "sparkling-graph-examples" % "0.0.7"
libraryDependencies += "ml.sparkling" %% "sparkling-graph-loaders" % "0.0.7"
libraryDependencies += "ml.sparkling" %% "sparkling-graph-operators" % "0.0.7"
Current features
- Loading
- Formats:
- CSV
- GraphML
- DSL
- Formats:
- Measures - measures can be configured to treat graphs as directed and undirected
- Measures DSL - easy to use domain specific language that boost productivity of library
- Graph
- Modularity
- Freeman's network centrality
- Vertex
- Closeness
- Local clustering
- Eigenvector
- Hits
- Neighbor connectivity
- Vertex embeddedness
- Betweenness
- Edmonds
- Flow
- Hua
- Edges
- Adamic/Adar
- Common neighbours
- Comunity detection methods
- PSCAN (SCAN)
- Graph coarsening
- Label Propagation based
- Link prediction
- Similarity measure based
- Generators
- Ring
- Watts And Strogatz
- Experiments
- Describe graph using all measures to CSV files
Planned features
- Loading
- GML
- Measures
- Katz
- Comunity detection methods
- Modularity maximization
- Infomap
- More Generators
- API
- Random walk
- BFS
- ML
- Vertex classification
Used by
Supported by:
provides us awesome IDE
How to
Please check API, examples or docs
Citation
If you use SparklingGraph in your research and publish it, please consider citing us, it will help us get funding for making the library better. Currently manuscript is in preparation, so please us following references:
Bartusiak et al. (2017). SparklingGraph: large scale, distributed graph processing made easy. Manuscript in preparation.
@unpublished{sparkling-graph
title={SparklingGraph: large scale, distributed graph processing made easy},
author={Bartusiak R., Kajdanowicz T.},
note = {Manuscript in preparation},
year = {2017}
}
License
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].