Hello again, and welcome to Elixir Sips Episode 8: Dynamo, Part 2. In the last episode, we built a basic web application using the Dynamo framework. It intends to serve as a simplistic twitter clone so that we have a testbed project that will let us get a bit deeper than one-off Elixir scripts, so we can get a feel for building a project in Elixir.

Today we're going to persist some data for our application. In a typical webapp (either in ruby or javascript) you would expect this to be roughly equivalent to the question of 'how do I talk to postgres' and so I'll briefly cover a project that aims to answer that question. But for kicks, and to get a good feeling for how things might end up being structured in this strange-new-world, we're actually going to use a non-SQL database for persistence.

Ecto

So first off, let me just show you the bleeding-edge database wrapper for Elixir. This comes straight out of the elixir-lang organization on GitHub, and it's called Ecto. Its first release was on September 5th, 2013.

Ecto

It's "a Domain Specific Language for writing queries and interacting with databases in Elixir." You can see some examples of what writing in it looks like from the GitHub page.

...peruse github...

I just wanted to show that stuff to you so you'd be aware of it, but as I said, we aren't going to implement it. So what are we going to implement?

Amnesia

Amnesia is an Elixir wrapper for the Erlang mnesia database. mnesia is "a distributed Database Management System, appropriate for ... Erlang applications which require continuous operation and exhibit soft real-time properties."

Getting started with Amnesia

The first thing we'll want to do to get started with Amnesia is to add the dependency to our mix file. I tagged the repo as it stood after last week's episode (sans some minor text changes) as v1.0. You can get this code by doing the following:

git clone https://github.com/knewter/dwitter.git
cd dwitter
git checkout v1.0

Next, open up mix.exs and add the amnesia dependency:

  defp deps do
    [ { :cowboy, github: "extend/cowboy" },
      { :dynamo, "0.1.0-dev", github: "elixir-lang/dynamo" },
      { :amnesia, "~> 0.1.0", github: "meh/amnesia" }]
  end

Now, pull in the dependency with:

mix deps.get

That's all it takes to get the dependency in place. Now it's possible to Just Use It. Let's get started by writing some tests, because that's what we do.

Learning Amnesia Through Testing

So all I did to get comfortable with Amnesia was to write some tests to exercise the behaviour provided in the README. Let's go ahead and do the same in this project, by building out a persistence layer to store Users and Dweets.

Open up test/amnesia_test.exs in your editor and let's get some tests written.

First, because we have to do some test_helper setup in order to get it available for our tests, we'll add a require statement:

Code.require_file "../test_helper.exs", __FILE__

Next:

use Amnesia

Let's step back and modify the test_helper. Open that file up and add this bit before ExUnit.start.

defmodule Amnesia.Test do
  def start do
    :error_logger.tty(false)

    Amnesia.Schema.create
    Amnesia.start

    :ok
  end

  def stop do
    Amnesia.stop
    Amnesia.Schema.destroy

    :error_logger.tty(true)

    :ok
  end
end

That will do basic startup and shutdown of the Amnesia system before and after each run of the suite. Now let's go ahead and start building out our test suite. Open test/amnesia_test.exs again and let's get started.

So first we'll write some basic Amnesia boilerplate to set up a table:

defdatabase Dwitter.Database do
  deftable Dweet, [:id, :content], type: :ordered_set do
    @type t :: Dweet[id: integer, content: String.t]
  end
end

This defines a record, called Dweet, inside the Dwitter.Database module, that has attributes id and content. With Amnesia the first attribute you define becomes a primary key, so we defined an id field. It's not required that you have an id, but strict lookups by that field will be much faster than other lookups.

Alright, let's go ahead and write a test. We'll just make sure that our database definition works, generally, and that we can store things in the database and fetch them back.

  test "saving dweets" do
    Amnesia.transaction! do
      dweet = Dweet[id: 1, content: 'some things happened']
      dweet.write
    end

    assert 'some things happened.' == Dweety.read!(1).content
  end

Now, before we can run it we have one more tiny thing to take care of. Add this to your test as well:

  setup_all do
    Amnesia.Test.start
  end

  teardown_all do
    Amnesia.Test.stop
  end

  setup do
    Dwitter.Database.create!

    :ok
  end

  teardown do
    Dwitter.Database.destroy

    :ok
  end

Alright, with that we can go ahead and run our tests.

... do it ...

Sweet, they all passed. Now that we've got the raw setup out of the way, we can build out domain functionality.

So it would be nice if someone could post a reply to someone else's post. Let's go ahead and see what that would look like with Amnesia. First, let's write some tests:

  test "finding replies" do
    Amnesia.transaction! do
      dweet = Dweet[id: 1, content: 'some things happened.']
      dweet.write
      reply = Dweet[id: 2, content: 'tell me more.', in_reply_to_id: 1]
      reply.write
    end

    Amnesia.transaction! do
      assert 'some things happened.' == Dweet.read!(2).in_reply_to.content
      [reply] = Dweet.read!(1).replies
      assert 'tell me more.' == reply.content
    end
  end

So let's do TDD. Run the tests, and see what fails.

... run tests ...

Alright, so we need to add the in_reply_to_id field to the database. Easy enough.

... hack hack hack ...

defdatabase Dwitter.Database do
  deftable Dweet, [:id, :content, :in_reply_to_id], type: :ordered_set do
    @type t :: Dweet[id: integer, content: String.t, in_reply_to_id: integer]

    def in_reply_to(self) do
      Dweet.read(self.in_reply_to_id)
    end

    def replies(self) do
      Dweet.where(in_reply_to_id == self.id).values
    end
  end
end

Summary

Alright, so now we have an actual, honest-to-god persistence layer for our data, with at least some minimal amount of richness. In the next episode, we'll extract our database model out of the test layer and wire it into the rest of the Dynamo application. See you soon!

Links

  • Ecto is a database wrapper and query builder for Elixir. At present it supports postgres.
  • Amnesia is an Elixir wrapper for the Erlang mnesia database.
  • mnesia is the Erlang DBMS Amnesia wraps.
  • The LYSE chapter on mnesia has some nice (if schizophrenic) discussion about the benefits it offers :)