All Projects → uwplse → Casper

uwplse / Casper

Licence: BSD-3-Clause License
A compiler for automatically re-targeting sequential Java code to Apache Spark.

Programming Languages

java
68154 projects - #9 most used programming language
Lex
420 projects

Projects that are alternatives of or similar to Casper

spark-sql-flow-plugin
Visualize column-level data lineage in Spark SQL
Stars: ✭ 20 (-55.56%)
Mutual labels:  spark
Spark-PMoF
Spark Shuffle Optimization with RDMA+AEP
Stars: ✭ 28 (-37.78%)
Mutual labels:  spark
spark-demos
Collection of different demo applications using Apache Spark
Stars: ✭ 15 (-66.67%)
Mutual labels:  spark
Python Master Courses
人生苦短 我用Python
Stars: ✭ 61 (+35.56%)
Mutual labels:  spark
leaflet heatmap
简单的可视化湖州通话数据 假设数据量很大,没法用浏览器直接绘制热力图,把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后,再使用Apache Spark绘制热力图,然后用leafletjs加载OpenStreetMap图层和热力图图层,以达到良好的交互效果。现在使用Apache Spark实现绘制,可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法,并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .
Stars: ✭ 13 (-71.11%)
Mutual labels:  spark
incubator-linkis
Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.
Stars: ✭ 2,459 (+5364.44%)
Mutual labels:  spark
spark-word2vec
A parallel implementation of word2vec based on Spark
Stars: ✭ 24 (-46.67%)
Mutual labels:  spark
docker
Scripts to build and use docker images including GHDL
Stars: ✭ 27 (-40%)
Mutual labels:  synthesis
BigData-News
基于Spark2.2新闻网大数据实时系统项目
Stars: ✭ 36 (-20%)
Mutual labels:  spark
aut
The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
Stars: ✭ 111 (+146.67%)
Mutual labels:  spark
sentry-spark
Apache Spark Sentry Integration
Stars: ✭ 14 (-68.89%)
Mutual labels:  spark
kafka-compose
🎼 Docker compose files for various kafka stacks
Stars: ✭ 32 (-28.89%)
Mutual labels:  spark
denver.lua
a simple library to help you play custom waveforms with LÖVE
Stars: ✭ 66 (+46.67%)
Mutual labels:  synthesis
spark-acid
ACID Data Source for Apache Spark based on Hive ACID
Stars: ✭ 91 (+102.22%)
Mutual labels:  spark
visions
Type System for Data Analysis in Python
Stars: ✭ 136 (+202.22%)
Mutual labels:  spark
grilops
a GRId LOgic Puzzle Solver library
Stars: ✭ 29 (-35.56%)
Mutual labels:  z3
frovedis
Framework of vectorized and distributed data analytics
Stars: ✭ 59 (+31.11%)
Mutual labels:  spark
GRNLR
granular synthesis plugin part of my bachelors thesis
Stars: ✭ 44 (-2.22%)
Mutual labels:  synthesis
smolder
HL7 Apache Spark Datasource
Stars: ✭ 33 (-26.67%)
Mutual labels:  spark
tpch-spark
TPC-H queries in Apache Spark SQL using native DataFrames API
Stars: ✭ 63 (+40%)
Mutual labels:  spark

Casper

Casper is a compiler for automatically re-targeting sequential Java code fragments to Apache Spark. Casper works by synthesizing high level MapReduce program specifications from raw un-annotated sequential Java source code and using the synthesized specificiations to generate Apache Spark code. The most recent release version for Casper is v0.1.1.

To learn more about Casper, visit our homepage, read our paper or email us, and if you like our tool, please star our repo!

Casper has been implemented as an extension of Polyglot 2.6.1.

Online Demo

The quickest way to try out Casper is by using our online demo. To download and install Casper on your own machine, follow the instructions below.

Getting Started

The easiest way to install Casper is by using our Docker image. It comes with all the dependencies installed and Casper already configured to run. Alternatively, use these instructions to configure and build Casper on your own.

Running in a Docker Container

You can run Casper inside a docker container using our pre-configured image of Ubuntu 16.04 LTS. Once you have Docker set up on your computer (instructions here), pull our ubuntu image using the following command:

$ docker pull maazsaf/casper-ubuntu

Once the image is downloaded, use it to run a Docker container:

$ docker run -t -i maazsaf/casper-ubuntu /bin/bash

Once inside the Docker container, you will find a clone of this repository in the root directory. You can optionally pull the latest changes from git to bring the repository up to date by running git pull. After that, build casper using ant as follows:

$ cd Casper/
$ ant

Optionally, you may want to reset to one of the stable release commits before you build:

$ git reset --hard <commit hash or release tag>

Build Casper from source code

Note: You only need to do this if you are building Casper from scratch. The Docker image already includes all software prerequisites.

Dependencies:

You must set environment variables for Sketch and Dafny for Casper to run successfully. To allow Casper to use your sketch installation, run the following commands under your sketch-frontend directory:

$ export PATH="$PATH:`pwd`"
$ export SKETCH_HOME="`pwd`/runtime"

Similarly, to allow Casper to use your Dafny installation, run the following command:

$ export PATH=$PATH:/path/to/dafny

To check whether you have all the necessary dependencies installed and properly configured, you can run:

$ ./bin/check.sh

You can compile the project by running 'ant' in the project's root directory. Once compiled, you run the tool as follows:

$ ./bin/run.sh [input-file] [output-file]

Benchmarks

To help you get started, we have added some example programs under /bin/benchmarks. In some cases the synthesizer may run for a very long time or require a significant amount of memory. Casper is currently under development and we are rolling out new features and bug fixes frequently. If you experience any difficulties, contact us through our mailing-list.

Contact

Casper is written by Maaz Ahmad at the University of Washington.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].