GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → intel → Graphbuilder

intel / Graphbuilder

Licence: apache-2.0
The GraphBuilder library provides functions to construct large scale graphs. It is implemented on Apache Hadoop.

Programming Languages

java
68154 projects - #9 most used programming language

Data relationships play a vital role in various data analytics, structured machine learning, and data mining applications. These relationships can be expressed as graphs. GraphBuilder is a Java library for preparing graphs from large volumes of unstructured and semi-structured data.

Processing large graphs in a distributed computing environment presents multiple system-level challenges, from parallelizing the graph construction algorithms to achieving balanced system utilization. In addition, the resultant graph must be carefully partitioned to ensure that subsequent application processing is properly balanced. GraphBuilder addresses these challenges with a carefully-designed multi-stage pipeline that leverages the Apache Hadoop framework to achieve scalability.

How to build the GraphBuilder?

The GraphBuilder is using Maven for building the package, ensure maven is installed on the system. To build package type: $> mvn package

How to use the GraphBuilder?

The GraphBuilder library can be used in multiple ways. Library groups multiple MapReduce jobs in four major groups: a) GraphCreation b) ComputeNetworkInfo c) GraphNormalization d) GraphPartitioning. These group can be chained to form graph construction pipeline depending on an application requirements (e.g. Please refer construction pipeline for link and doc-word graph in demoapps dir).

Library can be used as tool to construct link and doc-word graph for Wikipedia using following command:

  • Link graph construction - $> hadoop jar target/graphbuilder-0.0.1-SNAPSHOT-hadoop-job.jar com.intel.hadoop.graphbuilder.demoapps.wikipedia.linkgraph.LinkGraphEnd2End <# of partition>
  • Doc-word graph construction - $> hadoop jar target/graphbuilder-0.0.1-SNAPSHOT-hadoop-job.jar com.intel.hadoop.graphbuilder.demoapps.wikipedia.docwordgraph.TFIDFGraphEnd2End <# of partition>

Sample application code

Early stage in graph building pipeline is data dependent, hence we abstracted out the data dependent code from the library and provided interface to plug-in data-specific code. Sample code for those interface can be found in demoapps directory. It contains code for generating link-graphs and doc-word graph out of Wikipedia XML dump. Interfaces are documented, please refer a Javadoc in the doc directory.

Contact

More information visit www.01.org/graphbuilder

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].