benzap / Eden

#+TITLE: Eden - lua-based scripting language in clojure #+AUTHOR: Benjamin Zaporzan #+DATE: 2018-07-01 #+EMAIL: [email protected] #+LANGUAGE: en #+OPTIONS: H:2 num:t toc:t \n:nil ::t |:t ^:t f:t tex:t

[[https://travis-ci.org/benzap/eden][https://travis-ci.org/benzap/eden.svg?branch=master]]

[[https://clojars.org/eden][https://img.shields.io/clojars/v/eden.svg]]

[[./doc/logo.png]]

eden is a language akin to traditional scripting languages like lua, python or ruby. It is embeddable, extensible, sandboxed, and familiarly simple.

eden is unique, in that it uses only valid EDN data values for data representation. This has the added benefit of ridiculously easy clojure interoperability.

/eden is still in beta development, so things are going to be broken, undocumented, and error messages are close to non-existent./

#+BEGIN_SRC (require '[eden.core :as eden])

(eden/eval println("Hello World!")) ;; Hello World! ;;

(eden/eval local x = 2 + 2

function add2(x) return x + 2 end

println("The value of x plus 2 equals" add2(x))) ;; The value of x plus 2 equals 6 ;;

#+END_SRC

Almost all of clojures core libraries work out-of-the-box within eden

#+BEGIN_SRC

(eden/eval println(rest([1 2 3 4]))) ;; (2 3 4)

(eden/eval println(conj([1 2 3] 4))) ;; [1 2 3 4]

(eden/eval local x = list(1 2 3 4) for i in x do println(i) end) ;; 1 ;; 2 ;; 3 ;; 4

#+END_SRC

Even higher-level clojure functions work in eden

#+BEGIN_SRC

(eden/eval local result = map(inc vector(1 2 3 4)) println(result)) ;; (2 3 4 5)

(eden/eval local sum = reduce(function(a b) return a + b end list(1 2 3 4)) println(sum)) ;; 10

#+END_SRC

Functions written in eden can be used within clojure

#+BEGIN_SRC

(eden/eval function addfirst2(xs) return first(xs) + second(xs) end)

(def addfirst2 (eden/get-var 'addfirst2)) (println (addfirst2 [1 2 3 4])) ;; 3

#+END_SRC

Functions written in clojure can be used within eden

#+BEGIN_SRC

(defn hello [name] (str "Hello " name "!"))

(eden/set-var! 'hello hello) (eden/eval hello("Ben")) ;; "Hello Ben!"

#+END_SRC

eden uses dot notation for retrieving and assigning to EDN collections, like vectors and hash maps.

#+BEGIN_SRC

(eden/eval local person = {} person.first-name = "John" person.last-name = "Doe" person.age = 12 println(person)) ;; {:first-name John, :last-name Doe, :age 12}

;; ;; similarly, vectors can be accessed using square bracket notation ;;

(eden/eval local list-of-letters = ["a" "b" "c"] println(list-of-letters[1])) ;; b

#+END_SRC

The getter syntax makes it much easier to manipulate more complex collections.

#+BEGIN_SRC

(eden/eval local default-person = {} default-person.first-name = "John" default-person.last-name = "Doe"

local display = function(p) println(p.first-name "-" p.last-name) end

local person-list = [ default-person default-person ]

person-list[0].first-name = "Ben" person-list[0].last-name = "Z" person-list[1].first-name = "Jane" person-list[1].last-name "M"

println(person-list) display(person-list[0])) ;; [{:first-name Ben, :last-name Z} {:first-name Jane, :last-name Doe}] ;; Ben - Z

#+END_SRC

• Rationale

  eden was developed to be a embedded language within a natively compiled clojure application (GraalVM'a native-image). It can be used to expose the application API so that a userbase can create plugins in a sandboxed environment. The applications of eden within clojure are very similar to the applications of lua within c/c++.

  eden can also be used as a standalone scripting language. A natively compiled commandline tool has been developed, and can be used to manipulate EDN files similar to how you would implement JSON files in javascript. Everything is still in its early stages, so I would not recommend using it in a production setting.

  I also plan on compiling eden to clojurescript, although the applications of eden within clojurescript are not of interest to me at the moment.

• Requirements

  eden requires clojure 1.9+

• Installation ** Native Executable Native Executables can be found on the [[https://github.com/benzap/eden/releases][releases page]]

  There are currently native executables generated for debian-based linux systems, and for rpm-based systems.

  If you would like to generate your own native executable, please follow the configuration instructions included in the Makefile.

  An example use:

  #+BEGIN_SRC sh $ eden -e "println("Hello World!")" Hello World! $ #+END_SRC

  #+BEGIN_SRC ;; hello.eden

  function hello(name) return str("Hello " name "!") end

  local name = system.args[0] or "there" println(hello(name)) #+END_SRC

  #+BEGIN_SRC sh $ eden hello.eden ben Hello ben! #+END_SRC

** Uberjar Alternatively, the uberjar can be found on the [[https://github.com/benzap/eden/releases][releases page]], which can be run as follows with java

#+BEGIN_SRC sh java -jar eden--standalone.jar #+END_SRC

** Clojure Installation

For the latest version, please visit [[https://clojars.org/eden][clojars.org]]

Leiningen/Boot

#+BEGIN_SRC clojure

[eden "0.9.0"]

#+END_SRC

Clojure CLI/deps.edn

#+BEGIN_SRC clojure

eden {:mvn/version "0.9.0"}

#+END_SRC

Gradle

#+BEGIN_SRC groovy

compile 'eden:eden:0.9.0'

#+END_SRC

Maven

#+BEGIN_SRC xml

#+END_SRC

** Docker Image Execution If you wish to just try out eden, and you have docker installed, give this a shot:

#+BEGIN_SRC $ docker run --rm -ti benzap/eden:0.8.0-1 -e '"Hello Eden!"' #+END_SRC

Using eden with docker has its issues, as you have to mount volumes in order to execute scripts. Here is a small example, which mounts my current working directory under the volume /mount, so that it can execute an eden script that also resides in my current working directory.

#+BEGIN_SRC $ docker run --rm -v pwd:/mount -ti benzap/eden:0.8.0-1 /mount/my_script.eden #+END_SRC

• Introduction

  eden is an imperative language, so it embraces the idea of mutable values being passed around. However, eden re-uses the persistent data collections that make up clojure, which makes eden copy-on-write when performing operations on collections.

  #+BEGIN_SRC

  function people-eq?(p1 p2) if p1 == p2 then println("Are Equal!") else println("Not Equal!") end end

  local person1 = {:first-name 12 :age 12} local person2 = person1

  people-eq?(person1 person2) ;; Are Equal!

  person2.age = 13

  people-eq?(person1 person2) ;; Not Equal!

  #+END_SRC

  In a more traditional language like lua, person2 would hold a reference to the same data structure as person1. However, eden uses copy-on-write semantics. They never share a reference. If you want to share a reference between variables, use a clojure atom.

  #+BEGIN_SRC

  local person1 = atom({:first-name "Ben" :age 12}) local person2 = person1

  swap!(person2 function(p) p.age = 13 return p end) println(deref(person1)) ;; {:first-name Ben, :age 13}

  #+END_SRC

• Installing the Eden Console for Administrative Tooling

  eden can be run from the commandline, which can make it suitable for commandline scripting. It can evaluate expressions with the -e commandline flag, or can evaluate files, which are usually designated with the suffix *.eden.

  The easiest way to try out eden is to clone the project and run it within the project directory.

  #+BEGIN_SRC sh $ git clone https://github.com/benzap/eden.git $ cd eden $ lein run ./examples/eden/get_project_version.eden 0.8.0 #+END_SRC

** Debian Installation Tested in Ubuntu 17.10, 18.10.

/May require stdc++ lib dependencies for other distributions./

#+BEGIN_SRC wget https://github.com/benzap/eden/releases/download/0.8.0/eden-0.8.0-amd64.deb sudo dpkg -i eden-0.8.0-amd64.deb #+END_SRC

** Redhat Installaion Tested on Fedora 28

#+BEGIN_SRC wget https://github.com/benzap/eden/releases/download/0.8.0/eden-0.8.0-1.x86_64.rpm sudo rpm -i eden-0.8.0-1.x86_64.rpm #+END_SRC

** Running the standalone uberjar Grab a pre-generated uberjar from the [[https://github.com/benzap/eden/releases][releases page]], and run it directly:

#+BEGIN_SRC

$ java -jar eden-0.8.0-standalone.jar ./examples/eden/basic_http_server.eden

#+END_SRC

The best use of eden as a standalone tool is to either build your own native executable, or grab one of the pre-compiled ones provided on the [[https://github.com/benzap/eden/releases][releases page]]

** Installing on Windows experimental

Download the *.exe from the [[https://github.com/benzap/eden/releases][releases page]]

/Note: Has issues with the current working directory, *.jar recommended/

• Programming in Eden This is a short manual explaining the Eden programming language. ** Values and Types Eden is a /dynamically typed/ language based on data types provided in the [[https://github.com/edn-format/edn][EDN data format]]. The types include:

  • Integers / BigIntegers :: 0, 13, -13, 14N
  • Floats / Doubles :: 3.14, 10e+3
  • Strings :: "Hello"
  • Keywords :: :foo, :bar
  • Booleans :: true, false
  • Symbols (vars) :: x, y
  • Lists :: list(1 2 3), list(4 5 6)
  • Vectors :: [1 2 3], vector(1 2 3)
  • Maps :: {:a 123 :b 456}
  • Set :: #{:a :b :c}, set(:a :b :c)
  • Nil :: nil

  I encourage you to review the EDN data format for additional types you might experience when using Eden.

** Variables Variables in Eden are presented in the form of Symbols. The extent of allowed variable names consists of what is allowed in the EDN format, but also restricts you from using keywords used by the Eden language, and symbols beginning with a dot (ex. ~.name ~.foo), since this is used to access collection properties.

#+BEGIN_SRC ;; Allowed

x y foo bar is-value? set-value! $hallo

;; Not Allowed

for in .foo .bar end #+END_SRC

** Statements *** Assignment Assignment in Eden is similar to languages like lua, or python. The most basic form of assignment is assigning a value to a global variable:

#+BEGIN_SRC
<identifier> = <value>

x = 12
foo = :bar
chk? = false
#+END_SRC

Global variables live for the duration of the program, and can be
accessed from anywhere in the program.

The second form of assignment is assigning to a local variable:

#+BEGIN_SRC
local <identifier> = <value>

local x = 12
local foo = :bar
local chk? = false
#+END_SRC

*** Conditional Structure The first and most often used control structure is the if control structure:

#+BEGIN_SRC
;; if structure
if <condition> then
  <truthy body statements...>
end

;; if-else structure
if <condition> then
  <truthy body statements...>
else
  <falsy body statements...>
end

;; if-elseif-else structure
if <condition> then
  <body...>

elseif <condition> then
  <body...>

[elseif <condition> then
  <body...>]...

else
  <body...>
end
#+END_SRC

Examples

#+BEGIN_SRC
local age = 12
if age < 21 then
  println("You are underage")
end

chk? = true
if chk? then
  println("Value is true")
else
  println("Value is false")
end
#+END_SRC

*** While Statement While statements check its condition, and upon determining that it's true, will run the block of statements contained in its body. Each time, it will check the condition and call the statment block forever until the condition becomes false.

#+BEGIN_SRC
while <condition> do
  <body...>
end
#+END_SRC

Examples

#+BEGIN_SRC

;; keep looping until `i` is greater than or equal to 10
local i = 0
while i < 10 do
  println("i: " i)
  i = i - 1
end    

;; this will loop forever
while true do
  println("Never gonna give you up")
end

#+END_SRC

*** Repeat-Until Statement Repeat statements are similar to the while statement, with the differences being that the body is guaranteed to always be called at least once, and the body will be looped over only if the condition is false. #+BEGIN_SRC repeat <body...> until #+END_SRC

Examples

#+BEGIN_SRC


;; keep looping until 'i' is greater than or equal to 10
local i = 1
repeat
  println("i: " i)
  i = i + 1
until i >= 10

;; this will loop forever
repeat
  println("Never gonna let you down")
until false

#+END_SRC

*** For Statement The first for statement representation closely resembles the for statement seen in C-based programming languages:

#+BEGIN_SRC
for <iter-var> = <start> <end> [step] do
  <body...>
end
#+END_SRC

Examples

#+BEGIN_SRC

;; loop from 0 to 10
for i = 0 10 do
  println("i: " i)
end

;; loop from first index to the length of the vector
local x = [1 2 3]
for i = 0 count(x) do
  println(i "-" x[i])
end

;; provide a step
for i = 0 10 2 do
  println("i: " i)
end
#+END_SRC

The second type of ~for~ statement is called the ~for-each~
statement. This is the more popular, and more often used loop
conditional.

#+BEGIN_SRC
for <iter-var> in <collection> do
  <body...>
end
#+END_SRC

Examples

#+BEGIN_SRC

;; Print out each element of xs
local xs = [1 2 3]
for x in xs do
  println("Element: " x)
end

#+END_SRC

** Expressions *** Arithmetic Operators - Addition :: + - Subtraction :: - - Multiplication :: * - Division :: /

#+BEGIN_SRC

println(2 + 2)
println(2 + 2 - 2)
println(2 * 2 - 5)
println((2 + 2) * 4)
println((2 / 2) * 5)

#+END_SRC

*** Coercions & Conversions Arithmetic performed between integer values will remain as integers. It is only if you include a float within an arithmetic operation that it is automatically converted into a float value.

Almost all types can be converted into a string using the ~str~
function.

#+BEGIN_SRC
println(2 + 2) ;; 4 (integer)
println(2 + 2.) ;; 4. (float)
#+END_SRC

*** Relational Operators - Equality :: == - Inequality :: != - Less Than :: < - Greater Than :: > - Less Or Equal Than :: <= - Greater Or Equal Than :: >=

#+BEGIN_SRC
println(2 == 2) ;; true
println(2 != 1) ;; true

local age = 12

println(age < 18) ;; true
println(age > 18) ;; false
#+END_SRC

*** Logical Operators - And Operator :: and - Or Operator :: or

#+BEGIN_SRC
10 or 20 ;; 10
nil or "a" ;; "a"
nil and 10 ;; nil
nil or 10 ;; 10
10 and 20 ;; 20
#+END_SRC

*** Length and Concatenation Unlike Lua, eden does not make use of special operators for length or concatenation. Instead, length can be obtained by using the function count, and concatentation can be performed by using concat.

#+BEGIN_SRC
count([1 2 3]) ;; 3
count("test") ;; 4

concat([1 2 3] [4 5 6]) ;; (1 2 3 4 5 6)
#+END_SRC

*** Precedence Precendence is in this order (similar to lua): + or, and + <, >, <=, >=, !=, == + +, - + *, / + unary operators (-, not)

#+BEGIN_SRC
println(2 + 2 * 4) ;; 10
#+END_SRC

*** Collection Construction Constructing each of the main collections is straightforward **** Vector - [1 2 3] - vector(1 2 3) - apply(vector list(1 2 3)) - to convert a collection to a vector, use vec **** Map - {:a 123 :b (2 + 2)} - to convert a collection to a map, use into #+BEGIN_SRC into({} [[:a 123] [:b "test"]]) #+END_SRC **** Set - #{:a :b :c} - to convert a collection to a set, use set **** List - list(1 2 3) - apply(list [1 2 3]) - to convert a collection to a list, use into #+BEGIN_SRC into(list() vector(1 2 3 4)) #+END_SRC *** Function Calls Function calls are similar to lua: #+BEGIN_SRC ([arguments...]) #+END_SRC

Examples
#+BEGIN_SRC
x("test") ;; call the function in variable 'x' with the argument "test"
#+END_SRC

*** Function Definitions Function creation can either be done standalone, or an anonymous function be assigned to a variable:

#+BEGIN_SRC
function add(x y)
  return x + y
end

add = function(x y)
  return x + y    
end
#+END_SRC

In both cases, they can be assigned to a local variable

#+BEGIN_SRC
local function add(x y)
  return x + y
end

local add = function(x y)
  return x + y
end
#+END_SRC

** Module System

eden has a simple module system. eden will look for files in order of increasing precedence:

• If on linux, in /usr/share/eden/libs
• If on linux, it will look in each of the colon separated paths in the Environment Variable EDEN_MODULE_PATH
• if on windows, it will look in each of the /semi-colon separated/ paths in the Environment Variable EDEN_MODULE_PATH
• In your home folder, located at ~/.eden/libs (%HOME%/.eden/libs on windows)
• The current working directory

As an example, assuming I have a file named test.eden in the current working directory:

#+BEGIN_SRC ;; test.eden

local print-hello = function(name) println(str("Hello " name "!")) end

export {:hello print-hello}

#+END_SRC

importing the module is simple:

#+BEGIN_SRC ;; another_file.eden

test = require "test"

test.hello("Ben")

#+END_SRC

#+BEGIN_SRC sh $ eden another_file.eden Hello Ben! $ #+END_SRC

** Standard Libraries Most of the standard libraries are handpicked community libraries that i've used in my other projects. The core libraries closely resemble the libraries seen in clojure *** Core Library Most of the clojure core library has been implemented in Eden as a core library. The complete list of clojure.core can be found [[http://clojuredocs.org/clojure.core][here]]. Any dynamic variables, or macros have not been included from the core library. *** system Library - system.env(s) :: Get the Environment Variable by the name s - system.exit(n) :: Return from program with Exit Code n - system.get-globals() :: Return all of the Eden program's global variables. Note that the keys are represented as symbols. - system.set-global(name value) :: Set global variable programmatically. *** string Library The Eden string library is a direct mirror of the [[https://funcool.github.io/cuerdas/latest/][cuerdas]] string library. Please refer to the provided page for the list of functions, which can be accessed via the string variable.

Examples
#+BEGIN_SRC
string.caseless=("Hello There!" "HELLO There!") ;; true
string.human(:great-for-csv-headers) ;; "great for csv headers"
#+END_SRC

*** filesystem Library The Eden filesystem library is a direct mirror of the [[https://github.com/Raynes/fs/][Raynes.fs]] filesystem library. The library api can be found [[https://raynes.github.io/fs/][here]]. *** io Library The Eden io library is a copy of the [[http://clojuredocs.org/clojure.java.io][clojure.java.io]] library. *** $, Specter Library For data transformations, the popular [[https://github.com/nathanmarz/specter][specter]] library has been included in the variable $

Examples
#+BEGIN_SRC
$.setval([:a $.END] [4 5] {:a [1 2 3]}) ;; {:a [1 2 3 4 5]}

$.transform([$.filterer(odd?) $.LAST] inc range(1 9)) ;; (1 2 3 4 5 6 8 8)

$.transform($.ALL inc #{1 2 3}) ;; #{2 3 4}
#+END_SRC

*** Parsing Libraries **** html Library - html.parse(s) :: Parses HTML string using [[https://github.com/davidsantiago/hickory][hickory]] (generates hiccup style collection) - html.stringify(coll) :: Creates HTML string from collection using [[https://github.com/weavejester/hiccup][hiccup]]

 Also has css generator provided by [[https://github.com/noprompt/garden][garden]]
 + ~html.css.gen-css(coll)~ :: provided by garden.core/css
 + ~html.css.gen-style(coll)~ :: provided by garden.core/style
 + ~html.css.color~ :: several color functions from garden.color
 + ~html.css.units~ :: several unit functions from garden.units

**** json Library JSON library provided by [[https://github.com/dakrone/cheshire][cheshire]] - json.parse(s [opts]) :: provided by cheshire.core/parse-string - json.stringify(coll [opts]) :: provided by cheshire.core/generate-string **** edn Library EDN library parser provided by [[https://github.com/clojure/tools.reader][tools.reader]] - edn.parse(s [opts]) :: provided by clojure.tools.reader.edn/read-string - edn.stringify(coll [opts]) :: clojure.core/pr-str **** markdown Library Markdown library stringifier provided by [[https://github.com/yogthos/markdown-clj][markdown-clj]] - ~markdown.stringify(s) :: provided by markdown-clj.core/md-to-html-string **** transit Library Transit Reader and Generator provided by [[https://github.com/cognitect/transit-clj][transit-clj]]

 - transit.parse(s)
 - transit.write(x)

*** http Library HTTP Server provided by [[http://www.http-kit.org/][http-kit]] HTTP Client provided by [[https://github.com/martinklepsch/clj-http-lite/][clj-http-lite]] HTTP Router provided by [[https://github.com/juxt/bidi][bidi]]

- ~http.router.make-handler(coll)~ :: provided by bidi.ring/make-handler
- ~http.server.run-server(handler [opts])~ :: provided by org.httpkit.server/run-server
- ~http.client.get(url [opts])~ :: provided by clj-http.lite.client/get

Example found at ./examples/eden/http_server.eden

*** shell Library Shell library provided by [[https://github.com/Raynes/conch][conch]], unfortunately it doesn't work due to reflection issues that will be resolved in future native executables

*** operator Library Includes all of the clojure equivalent operators. Useful for additional performance in certain applications * operator.add(x ...) :: clojure.core/+ * operator.sub(x ...) :: clojure.core/- * operator.mult(x ...) :: clojure.core/* * operator.div(x ...) :: clojure.core// * operator.not(x) :: clojure.core/not * operator.and(x y) :: clojure.core/and, 2-arity wrapped macro * operator.or(x y) :: clojure.core/or, 2-arity wrapped macro

• Dark-corners of Eden

  Since eden uses EDN data values directly, it does mean some funky things can happen unexpectedly.

** Vectors get confused as indexes

#+BEGIN_SRC

map(inc [1 2 3])

#+END_SRC

This says /get the index [1 2 3] of inc/. The equivalent in clojure would be (get-in inc [1 2 3]), which is not what we want. The solution is to use the vector function.

#+BEGIN_SRC

map(inc vector(1 2 3))

#+END_SRC

Note that indexing is only in effect after identifiers and function calls

#+BEGIN_SRC x[1] ;; indexing list([1 2 3] x) ;; not indexing list([1 2 3] x [2]) ;; x[2] is an index! #+END_SRC

** The EDN parser gets confused with complex map hashes

#+BEGIN_SRC

local x = { :x 2 + 2 :y 3 - 2 }

#+END_SRC

The parser will fail, since the resulting map within eden appears as {:x 2, '+ 2, :y 3, '- 2}. The solution is to group each expression in round brackets:

#+BEGIN_SRC

local x = { :x (2 + 2) :y (3 - 2) }

;; similarly for functions local y = { :hello (function(name) return str("Hello " name "!") end) }

#+END_SRC

• Differences between Lua and Eden ** Array Indexing eden uses zero-indexing for array types, whereas lua uses one-indexing for array types.

  #+BEGIN_SRC lua -- Lua x = {"A", "B", "C"} print(x[1]) -- A #+END_SRC

  #+BEGIN_SRC clojure ;; eden x = ["A" "B" "C"] println(x[1]) ;; B #+END_SRC ** Equality Symbols Lua uses ~= to represent inequality, whereas Eden uses !=

  #+BEGIN_SRC lua -- Lua print(true ~= false) -- true #+END_SRC

  #+BEGIN_SRC ;; eden println(true != false) ;; true #+END_SRC

** Module Systems eden adopts a module system with the special keyword export for exporting, whereas Lua reuses return to represent the module export.

#+BEGIN_SRC lua -- Lua local x = {} x.test = function() print("test!") end

 return x

#+END_SRC

#+BEGIN_SRC clojure ;; eden local x = {} x.test = function() println("test!") end

 export x

#+END_SRC

• Development ** Uberjar To generate a standalone uberjar file, run lein uberjar

  The generated jar file will be located in ./target/eden--standalone.jar

** Native Executable Distribution Please read the Makefile for instructions on how to native compile eden using GraalVM. *** Debian DPKG (Tested on Ubuntu 17.10) make dpkg *** Redhat RPM (Tested on Fedora 28) make rpm *** Tar Archive (Tested on Ubuntu 17.10) make tar ** Testing Tests can be run with lein test

• Features for Version 1.0.0 Stable Release

  • +Test Coverage for the entire standard language+
  • +Better parser errors (might require a parser rewrite)+
  • +support 'elseif clause in if conditionals+ (added in 0.4.0-SNAPSHOT)
  • +Additional standard libraries.+ (Several libraries have been added since 0.5.0-SNAPSHOT)
  • +clojure.string (or use funcool.cuerdas, can it native compile?)+ (added in 0.3.0-SNAPSHOT)
  • +json parse and stringify libs (one that native compiles)+ (added in 0.3.0-SNAPSHOT)

• Future Unreachable(?) Goals

  • eden repl
  • clojurescript build with passing tests
  • metafunctions
  • lua table implementation
  • natively compiled database interface (sqlite, psql)