ETS stands for Erlang Term Storage. It's a means of storing large quantities of data in an Erlang Runtime System with the ability to access it in constant-time.

Let's play with them a little:

Project

Start a new project called ets_playground:

mix new ets_playground
cd ets_playground

Now we're going to see how to create a table and get its info. Open up test/ets_playground_test.exs, and we'll simulate a fairly poorly designed auto dealership inventory application for illustration purposes:

defmodule EtsPlaygroundTest do
  use ExUnit.Case

  setup do
    # The ets module has a function, `new`, that takes two arguments: the name
    # of the table and a list of options.  We'll tell it to create a table of
    # type 'bag' - the possible types are :set, :bag, :ordered_set, and
    # :duplicate_bag
    # We'll also make it a named table, which just means that we don't have to
    # pass around the table identifier to refer to it.  I'm doing this entirely
    # because it makes writing the tests easier - in general, it feels weird to me to
    # use named tables.
    cars = :ets.new(:cars, [:bag, :named_table])
    :ok
  end

  teardown do
    # We'll also delete this table after each test.  Besides manually deleting
    # them, ETS tables will also be destroyed when their parent process terminates.
    :ets.delete(:cars)
    :ok
  end

  test "creating a table and getting its info" do
    # Next, we can use the ets module's `info` function to get information on
    # the table
    info = :ets.info(:cars)
    # We'll print out the info just so you can see it when we run the tests:
    IO.inspect info
    # Finally, we'll make sure that we created the right kind of table, just to
    # prove that we created a table successfully
    assert info[:type] == :bag
  end
end

Run the tests, and they pass - so now you can create an ETS table.

What good is a table if we can't put stuff in it and get it out? Let's look at that:

  setup do
    #...
    # We'll insert a new car in.  The first element in the tuple is the key for
    # this data by the way
    :cars |> :ets.insert({"328i", "BMW", "White", 2011})
    :ok
  end

  test "inserting and retrieving data" do
    # Next, we'll fetch the data out of the table by key.  It returns a list, so
    # we'll pattern match on that to bind our data to local variables
    [{_model, make, _color, _year}|_tail] = :ets.lookup(:cars, "328i")
    assert make == "BMW"
  end

Run the tests. That was easy enough. Next we'll look at traversing the table sequentially. For that, we'll need to add some more data to the table:

  setup do
    #...
    :cars |> :ets.insert({"328i", "BMW", "White", 2011})
    :cars |> :ets.insert({"335i", "BMW", "Black", 2013})
    :cars |> :ets.insert({"528i", "BMW", "White", 2012})
    :ok
  end

Then we'll add our test:

  test "traversing the table sequentially" do
    # You can get the first key in a table with the `first` function
    first = :ets.first(:cars)
    # You can get the next key after a given key by passing the key as the
    # second argument
    second = :ets.next(:cars, first)
    third = :ets.next(:cars, second)
    assert third == "528i"
    # If you try to traverse past the end of the table, you'll get a
    # notification that that happened in the form of a special atom
    assert :"$end_of_table" == :ets.next(:cars, third)
  end

Run the tests. Alright, so now it would be nice to be able to get all of the 2012 cars out of the table - what good is a database without the ability to selectively query?

To do this, you'll use a "match pattern", which looks a lot like a tuple you might write for pattern matching, but has to be built in a bit of a 'string style' due to some limitations in Erlang. At any rate, it should be fairly easy to understand what's happening:

  test "querying the table for data that matches a pattern" do
    query = {:_, :_, :_, 2012}
    cars_from_2012 = :ets.match_object(:cars, query)
    [{model, _, _, _}|_tail] = cars_from_2012
    assert model == "528i"
  end

Run the tests, and they pass. It's important to know that the placeholders must be atoms. Unfortunately, this is pretty limiting - how would we find all cars made between 2011 and 2012, for instance?

Luckily, there's a solution. Sadly, it looks ridiculous and unwieldy. It's called 'match specifications'. I'll just show you how you can use it to make this date range query, and then I've provided a link to the full specification in the Resources. You'll also want to look into :ets.fun2ms for an easier way to write a match specification using erlang parse transforms. There are also inevitably some Elixir-specific tools to write match specs, but I've not yet found them.

Anyway, without further ado, here's how you could query just cars from 2011 to 2012:

  test "querying using match specs" do
    # So a match spec is just a list of three element tuples
    # - The first element is a core pattern to match and to bind local variables.
    # - The second element is a list of guard clauses
    # - The third element is the description of how you want the data returned -
    #   dollar underscore means give me the whole thing.
    query = [{
               {:_, :_, :_, :"$1"},
               [{:andalso,
                   {:'>=', :"$1", 2011},
                   {:'=<', :"$1", 2012},
               }],
               [:"$_"]
            }]
    selected_cars = :ets.select(:cars, query)
    IO.inspect selected_cars
    assert Enum.count(selected_cars) == 2
  end

Alright, so that's certainly a bit crazy, and there's a kind of hidden secret knowledge in there regarding a kind of odd way that "less than or equal to" is written in Erlang as opposed to Elixir, but there you have it.

Summary

In today's episode we learned what ETS was and how to use it. There's plenty more to learn - for instance, what all the other types are. In the next episode, we'll look at DETS, which is a Disk-based version of ETS. See you soon!

Resources