All Projects → joergen7 → Cuneiform

joergen7 / Cuneiform

Licence: apache-2.0
Cuneiform distributed programming language

Programming Languages

1774 projects

Projects that are alternatives of or similar to Cuneiform

Workflow engine for exploration of simulation models using high throughput computing
Stars: ✭ 120 (-31.43%)
Mutual labels:  workflow-engine, distributed-computing, workflow
An open source platform for managing and analyzing biomedical big data
Stars: ✭ 274 (+56.57%)
Mutual labels:  workflow-engine, bioinformatics, workflow
Data intensive science for everyone.
Stars: ✭ 812 (+364%)
Mutual labels:  workflow-engine, bioinformatics, workflow
common runtime environment for distributed programming languages
Stars: ✭ 20 (-88.57%)
Mutual labels:  otp, distributed-computing, parallelization
Robust, flexible and resource-efficient pipelines using Go and the commandline
Stars: ✭ 826 (+372%)
Mutual labels:  workflow-engine, bioinformatics, workflow
Reusable workflow library for Django
Stars: ✭ 2,136 (+1120.57%)
Mutual labels:  workflow-engine, workflow
A lightweight parallel task engine
Stars: ✭ 108 (-38.29%)
Mutual labels:  workflow-engine, workflow
UGENE is free open-source cross-platform bioinformatics software
Stars: ✭ 112 (-36%)
Mutual labels:  bioinformatics, workflow
A simple finite state machine (FSM) with workflow character where you define your workflows in code.
Stars: ✭ 122 (-30.29%)
Mutual labels:  workflow-engine, workflow
Detect germline or somatic variants from normal or tumour/normal whole-genome or targeted sequencing
Stars: ✭ 124 (-29.14%)
Mutual labels:  bioinformatics, workflow
Lightweight workflow engine
Stars: ✭ 129 (-26.29%)
Mutual labels:  workflow-engine, workflow
🚥 Petrinet framework for PHP
Stars: ✭ 107 (-38.86%)
Mutual labels:  workflow-engine, workflow
A purely functional library to build distributed and event-driven systems
Stars: ✭ 106 (-39.43%)
Mutual labels:  distributed-computing, functional-programming
Pegasus Workflow Management System - Automate, recover, and debug scientific computations.
Stars: ✭ 110 (-37.14%)
Mutual labels:  bioinformatics, workflow
A collection of scripts and notes related to genomics and bioinformatics
Stars: ✭ 101 (-42.29%)
Mutual labels:  bioinformatics, workflow
nFlow is a battle-proven solution for orchestrating business processes. It can be used as microservices orchestrator (Saga-pattern), as business process engine or as persistent finite-state machine.
Stars: ✭ 96 (-45.14%)
Mutual labels:  workflow-engine, workflow
Etl unicorn
数据可视化, 数据挖掘, 数据处理 ETL
Stars: ✭ 156 (-10.86%)
Mutual labels:  workflow-engine, workflow
Django Lb Workflow
Reusable workflow library for Django
Stars: ✭ 153 (-12.57%)
Mutual labels:  workflow-engine, workflow
BatchFlow helps you conveniently work with random or sequential batches of your data and define data processing and machine learning workflows even for datasets that do not fit into memory.
Stars: ✭ 156 (-10.86%)
Mutual labels:  workflow-engine, workflow
Workflow core
[Deprecated, use flor_core instead] A Rails engine which providing essential infrastructure of workflow. It's based on Workflow Nets.
Stars: ✭ 171 (-2.29%)
Mutual labels:  workflow-engine, workflow

Cuneiform: Data analysis open and general Build Status

Cuneiform is a large-scale data analysis functional programming language. It is open because it easily integrates foreign tools and libraries, e.g., Python libraries or command line tools. It is general because it has the expressive power of a functional programming language while using the independence of sub-expressions to automatically parallelize programs. Cuneiform uses distributed Erlang to scalably run in cluster and cloud environments.



Having rebar3 available on your system, compile the project by entering

rebar3 escriptize

Starting Cuneiform

Compiling the Cuneiform client using escriptize creates an Erlang script file cf_worker which allows starting the Cuneiform client via the command line.

To display a help text enter

./cuneiform --help

This will show the command line synopsis, which looks like the following:

Usage: cuneiform [-v] [-h] [-n <n_wrk>] [-w <wrk_dir>] [-r <repo_dir>]
                 [-d <data_dir>]

  -v, --version   Show cf_worker version.
  -h, --help      Show command line options.
  -n, --n_wrk     Number of worker processes to start. 0 means auto-detect 
                  available processors.
  -w, --wrk_dir   Working directory in which workers store temporary files.
  -r, --repo_dir  Repository directory for intermediate and output data.
  -d, --data_dir  Data directory where input data is located.

Starting an Interactive Shell

You can start a shell and program Cuneiform interactively by starting it without any command line parameters like so:


This will open a shell giving the following initial output, along with a number of status messages:

           @@WB      Cuneiform
     [email protected]@@@@WWWWWWL  Type help for usage info
   [email protected]              quit to exit shell
  @@P    [email protected]
  @N____ [email protected]
  "[email protected]@@[email protected]         Jorgen Brandt


Running a Cuneiform Script

Alternatively, Cuneiform can be started by giving it a source file which will only output the final result of the computation. If your Cuneiform script is stored in my_script.cfl start it by entering

./cuneiform my_script.cfl


A collection of self-contained Cuneiform examples is available under joergen7/cuneiform-examples.

Variable assignment

You can assign a value to a variable and retrieve a variable's content like so:

let x : Str =


In the first line we assign the value "foo" to a variable named x declaring its type to be Str. In the last line we query the variable x.

Booleans and Conditions

We can branch execution based on conditions using conditional statements. Conditionals are expressions.

let x : Str =
  if true


The above command the conditional binding the string "bla" to the variable x. Then, we query the variable.


We can construct list literals by enumerating their elements in square brackets and declaring the type of the list elements.

let xs : [Bool] =
  [true, false, true, true : Bool];


Here, we define the list xs whose elements are of type Bool giving four Boolean values of which only the second is false.

Records and Pattern Matching

A record is a collection of fields that can be accessed via their labels. Literal records can be constructed like so:

let r : <a : Str, b : Bool> =
  <a = "blub", b = false>;

( r|a );

We define a record r with two fields a and b, of types Str and Bool respectively. The field associated with a gets the value "blub" while the field associated with b gets the value false. In the last line we access the a field of the record r.

Alternatively, we can access record fields via pattern matching:

let <a = z : Str> = r;

In the first line we associate the variable z with the field a of record r. In the second line we query the content of z.

Native Function Definition

Defining native functions in Cuneiform is done by giving the function name, its signature, and a body expression in curly braces:

def identity( x : Str ) -> Str {

identity( x = "bar" );

In the first line we define the function identity which consumes an argument x of type Str and produces a return value of type Str. In the second line, the body expression is just the argument x. In the last line we call the function binding the argument x to the value "bar".

Foreign Function Definition

Defining foreign functions is done by giving the function name, its signature, the foreign language name, and the function body in mickey-mouse-eared curly braces.

def greet( person : Str ) -> <out : Str> in Bash *{
  out="Hello $person"

greet( person = "Peter" );

The first line defines a foreign function greet taking one argument person of type Str and returning a tuple with a single field out of type Str. The foreign function body is given in Bash code. In the last line we call the foreign function, binding the argument person to the string value "Peter".

Iterating over Lists using For

To perform an operation on each element of a list, one can iterate using for:

let xs : [Bool] =
  [true, false, true, true : Bool];

for x : Bool <- xs do
  not x
  : Bool

Here, we define a list of four Booleans and negate each element.

Aggregating Lists using Fold

We can aggregate over lists using fold:

def add( a : Str, b : Str ) -> <c : Str> in Python *{
  c = int( a )+int( b )

let xs : [Str] = [1, 2, 3 : Str];

let sum : Str =
  fold acc : Str = 0, x : Str <- xs do
    ( add( a = acc, b = x )|c )


Here, we first define the function add which lets us add two numbers in Python and then the string list xs containing the numbers from one to three. We aggregate the sum of the numbers in xs and store it the result in the variable sum. Lastly, we query the sum variable.

System Requirements




Apache 2.0

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].