ExMarshal

ExMarshal encodes and decodes Elixir terms according to Ruby Marshal format.

Currently supported Ruby types are nil, false, true, Fixnum, Bignum, BigDecimal, Float, Symbol, String, Array, Hash as well as user-defined types such as Date.

Why?

Once you decide to integrate small Elixir tool into big-old-legacy Ruby system, chances are that you need to interact with Memcached. As soon as Ruby code writes something into Memcached, most likely Ruby uses dalli gem. And Dalli uses Ruby Marshal by default.

Another use case is to decode Rails session encoded using Ruby Marshal. For this you can use plug_rails_cookie_session_store or plug_session_redis if you want to store session data in Redis.

Installation

Add ExMarshal as a dependency to your mix.exs file:

def deps do
  [{:ex_marshal, "0.0.13"}]
end

Usage

iex(1)> ExMarshal.decode(<<4, 8, 91, 8, 105, 6, 105, 7, 105, 8>>)
[1, 2, 3]
iex(2)> ExMarshal.encode([1, 2, 3])
<<4, 8, 91, 8, 105, 6, 105, 7, 105, 8>>
iex(3)>

It is also possible to decode user objects such as Ruby dates using custom parsers which can be specified via the user_object_parsers option:

iex(1)> value = <<4, 8, 85, 58, 9, 68, 97, 116, 101, 91, 11, 105, 0, 105, 3, 72, 136, 37, 105, 0, 105, 0, 105, 0, 102, 12, 50, 50, 57, 57, 49, 54, 49>>
<<4, 8, 85, 58, 9, 68, 97, 116, 101, 91, 11, 105, 0, 105, 3, 72, 136, 37, 105,
  0, 105, 0, 105, 0, 102, 12, 50, 50, 57, 57, 49, 54, 49>>
iex(2)> ExMarshal.decode(value, user_object_parsers: %{Date: fn [_, julian_day, _, _, _, _] -> Date.from_gregorian_days(julian_day - 1721425) end})
~D[2021-05-20]

Nullify Ruby Objects

The default behaviour of ExMarshal is to raise an error when trying to decode an serilized ruby object. This config option can be used to nullify the ruby object without raising an error:

config :ex_marshal,
  nullify_objects: true

Use Case Rails 3.2 puts Ruby Objects into session(i.e. Flash messages wrapped in an object). When Phoenix is trying to read Rails session, this will cause ExMarshal to raise an error and a 500 error in Phoenix. If you don't need to read these values from Elixir/Phoenix, this option allows ExMarshal to skip objects.

ExMarshal with Memcache.Client

One of the reasons why ExMarshal was created is to work with Memcached. Here is how ExMarshal can be used with Memcache.Client:

defmodule Memcache.Client.Transcoder.Ruby do
  @behaviour Memcache.Client.Transcoder

  @ruby_type_flag 0x0001

  def encode_value(value) do
    {ExMarshal.encode(value), @ruby_type_flag}
  end

  def decode_value(value, @ruby_type_flag) do
    ExMarshal.decode(value)
  end

  def decode_value(_value, data_type) do
    {:error, {:invalid_data_type, data_type}}
  end
end

Then tell Memcache.Client to use this transcoder:

config :memcache_client,
  transcoder: Memcache.Client.Transcoder.Ruby

Examples

Read

Ruby side

:1 > dc = Dalli::Client.new('localhost:11211')
:2 > dc.set("str", "hello elixir")
 => true

Elixir side

iex(1)> Memcache.Client.get("str")
%Memcache.Client.Response{cas: 184, data_type: 1, extras: <<0, 0, 0, 1>>,
 key: "", status: :ok, value: "hello elixir"}

Write

Elixir side

iex(1)> Memcache.Client.set("str", "hello ruby")
%Memcache.Client.Response{cas: 185, data_type: nil, extras: "", key: "",
 status: :ok, value: ""}

Ruby side

:1 > dc = Dalli::Client.new('localhost:11211')
:2 > dc.get("str")
 => "hello ruby"

Kudos

Thanks to @tsharju for ability to use custom transcoders in Memcache.Client

Thanks to @shepmaster for series of posts about Ruby Marshal format.

Special thanks to @lexmag for help in writing this tool and for guiding me through Elixir world.

License

This software is licensed under the ISC license.