All Projects → am-kantox → Elixir Iteraptor

am-kantox / Elixir Iteraptor

Licence: mit
Handy enumerable operations implementation.

Programming Languages

elixir
2628 projects

Labels

Projects that are alternatives of or similar to Elixir Iteraptor

Guitar
A Simple and Efficient Distributed Multidimensional BI Analysis Engine.
Stars: ✭ 86 (+56.36%)
Mutual labels:  mapreduce
Bigslice
A serverless cluster computing system for the Go programming language
Stars: ✭ 469 (+752.73%)
Mutual labels:  mapreduce
Mobius
C# and F# language binding and extensions to Apache Spark
Stars: ✭ 929 (+1589.09%)
Mutual labels:  mapreduce
Tdigest
t-Digest data structure in Python. Useful for percentiles and quantiles, including distributed enviroments like PySpark
Stars: ✭ 274 (+398.18%)
Mutual labels:  mapreduce
Data Science Ipython Notebooks
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
Stars: ✭ 22,048 (+39987.27%)
Mutual labels:  mapreduce
Cdap
An open source framework for building data analytic applications.
Stars: ✭ 509 (+825.45%)
Mutual labels:  mapreduce
dtail
DTail is a distributed DevOps tool for tailing, grepping, catting logs and other text files on many remote machines at once.
Stars: ✭ 112 (+103.64%)
Mutual labels:  mapreduce
Mare
MaRe leverages the power of Docker and Spark to run and scale your serial tools in MapReduce fashion.
Stars: ✭ 11 (-80%)
Mutual labels:  mapreduce
Bdp Dataplatform
大数据生态解决方案数据平台:基于大数据、数据平台、微服务、机器学习、商城、自动化运维、DevOps、容器部署平台、数据平台采集、数据平台存储、数据平台计算、数据平台开发、数据平台应用搭建的大数据解决方案。
Stars: ✭ 456 (+729.09%)
Mutual labels:  mapreduce
Yandex Big Data Engineering
Stars: ✭ 17 (-69.09%)
Mutual labels:  mapreduce
Redisson
Redisson - Redis Java client with features of In-Memory Data Grid. Over 50 Redis based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Tomcat, Scheduler, JCache API, Hibernate, MyBatis, RPC, local cache ...
Stars: ✭ 17,972 (+32576.36%)
Mutual labels:  mapreduce
Cascading
Cascading is a feature rich API for defining and executing complex and fault tolerant data processing flows locally or on a cluster. See https://github.com/Cascading/cascading for the release repository.
Stars: ✭ 318 (+478.18%)
Mutual labels:  mapreduce
Corral
🐎 A serverless MapReduce framework written for AWS Lambda
Stars: ✭ 648 (+1078.18%)
Mutual labels:  mapreduce
data-algorithms-with-spark
O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian
Stars: ✭ 34 (-38.18%)
Mutual labels:  mapreduce
Coursera Uw Machine Learning Clustering Retrieval
Stars: ✭ 25 (-54.55%)
Mutual labels:  mapreduce
st-hadoop
ST-Hadoop is an open-source MapReduce extension of Hadoop designed specially to analyze your spatio-temporal data efficiently
Stars: ✭ 17 (-69.09%)
Mutual labels:  mapreduce
Bigdata
💎🔥大数据学习笔记
Stars: ✭ 488 (+787.27%)
Mutual labels:  mapreduce
Data Algorithms Book
MapReduce, Spark, Java, and Scala for Data Algorithms Book
Stars: ✭ 949 (+1625.45%)
Mutual labels:  mapreduce
Bigdata Interview
🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
Stars: ✭ 857 (+1458.18%)
Mutual labels:  mapreduce
Distributed Computing
distributed_computing include mapreduce kvstore etc.
Stars: ✭ 654 (+1089.09%)
Mutual labels:  mapreduce

Iteraptor

Build Status Inline docs Hex.pm

Handy enumerable operations

bonus:

  • Iteraptor.jsonify/2 to prepare the term for JSON interchange; it basically converts keys to strings and keywords to maps because JSON encoders might have issues with serializing keywords.

  • Iteraptor.Extras.bury/4 to store the value deeply inside nested term (the intermediate keys are created as necessary.)

HexDocs

Usage

Iterating, Mapping, Reducing

# each
iex> %{a: %{b: %{c: 42}}} |> Iteraptor.each(&IO.inspect/1, yield: :all)
# {[:a], %{b: %{c: 42}}}
# {[:a, :b], %{c: 42}}
# {[:a, :b, :c], 42}
%{a: %{b: %{c: 42}}}

# map
iex> %{a: %{b: %{c: 42}}} |> Iteraptor.map(fn {k, _} -> Enum.join(k) end)
%{a: %{b: %{c: "abc"}}}

iex> %{a: %{b: %{c: 42}}}
...> |> Iteraptor.map(fn
...>      {[_], _} = self -> self
...>      {[_, _], _} -> "YAY"
...>    end, yield: :all)
%{a: %{b: "YAY"}}

# reduce
iex> %{a: %{b: %{c: 42}}}
...> |> Iteraptor.reduce([], fn {k, _}, acc ->
...>      [Enum.join(k, "_") | acc]
...>    end, yield: :all)
...> |> :lists.reverse()
["a", "a_b", "a_b_c"]

# map-reduce
iex> %{a: %{b: %{c: 42}}}
...> |> Iteraptor.map_reduce([], fn
...>      {k, %{} = v}, acc -> {{k, v}, [Enum.join(k, ".") | acc]}
...>      {k, v}, acc -> {{k, v * 2}, [Enum.join(k, ".") <> "=" | acc]}
...>    end, yield: :all)
{%{a: %{b: %{c: 42}}}, ["a.b.c=", "a.b", "a"]}

# filter
iex> %{a: %{b: 42, e: %{f: 3.14, c: 42}, d: %{c: 42}}, c: 42, d: 3.14}
...> |> Iteraptor.filter(fn {key, _} -> :c in key end, yield: :none)
%{a: %{e: %{c: 42}, d: %{c: 42}}, c: 42}

Flattening

iex> %{a: %{b: %{c: 42, d: [nil, 42]}, e: [:f, 42]}}
...> |> Iteraptor.to_flatmap(delimiter: "_")
#⇒ %{"a_b_c" => 42, "a_b_d_0" => nil, "a_b_d_1" => 42, "a_e_0" => :f, "a_e_1" => 42}

iex> %{"a.b.c": 42, "a.b.d.0": nil, "a.b.d.1": 42, "a.e.0": :f, "a.e.1": 42}
...> |> Iteraptor.from_flatmap
#⇒ %{a: %{b: %{c: 42, d: [nil, 42]}, e: [:f, 42]}}

Extras

iex> Iteraptor.jsonify([foo: [bar: [baz: :zoo], boo: 42]], values: true)
%{"foo" => %{"bar" => %{"baz" => "zoo"}, "boo" => 42}}

iex> Iteraptor.Extras.bury([foo: :bar], ~w|a b c d|a, 42)
[a: [b: [c: [d: 42]]], foo: :bar]

**As of version 1.2.0 there is an experimental AST traversal feature:

iex> Iteraptor.AST.reduce((quote do: 42), [], fn e, acc -> [e | acc], yield: :all)
'*'

Installation

Add iteraptor to your list of dependencies in mix.exs:

def deps, do: [{:iteraptor, "~> 1.5"}]

Changelog

1.13.0

keys: :reverse configuration option in all traversion functions to simplify pattern matching on leaf keys

1.10.0

Iteraptor.jsonify/2 for deep conversion of keyword lists to maps.

1.8.0

Iteraptor.Config for deep substitution of {:system, "VAR"} tuples with the values taken from the system environment in runtime.

1.7.0

Iteraptor.Array with Access support. Basically, Array is the list with Access support.

  • 1.7.2 → fixed bug with type recognition for MapSet and Iteraptor.Array.

1.6.0

Iteraptor.jsonify/2.

1.5.0

All iterators do now accept structs: :values keyword argument to prevent nested iteration into structs.

Experimental support for Iteraptable protocol.

1.4.0

Extended support for Iteraptor.Iteraptable:

1.3.0

We now support MapSets.

1.0.0-rc1

Better documentation, Iteraptor.Extras.bury/3.

0.9.0

Complete refactoring, Iteraptor.map/3, Iteraptor.reduce/4, Iteraptor.map_reduce/4.

0.5.0

NB: This functionality is experimental and might not appear in 1.0.0 release.

use Iteraptor.Iteraptable inside structs to make them both Enumerable and Collectable:

defmodule Iteraptor.Struct do
  @fields [field: nil]

  def fields, do: @fields
  defstruct @fields

  use Iteraptor.Iteraptable
end

iex> %Iteraptor.Struct{field: 42}
...>   |> Enum.each(fn e -> IO.inspect(e) end)
#⇒   {:field, 42}

0.4.0

Experimental: support for structs on input. Structs will be automagically created on |> Iteraptor.from_flatmap from keys like StructName%field if a) this structure is known to the system and b) keys are consistent (e. g. there are no subsequent elements, belonging to different structs: ["S1%f" => 42, "S2%f" => 3.14].)

0.3.0

Support for Keyword on input, but it will be output as map for |> Iteraptor.to_flatmap |> Iteraptor.from_flatmap back and forth transformation.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].