Implement a forum data model in Elixir

Building an Elixir data layer for our Forum with Ecto.

Intro

We're going to build a Forum GraphQL API, but the API is not our application - it's just an interface to our application. We'll build the core business logic in a firestorm_data application inside of an umbrella.

Getting Started

We can start by building a new umbrella project:

mix new firestorm --umbrella
cd firestorm

We have a new umbrella app - let's create the firestorm_data app to hold our business logic and database interface:

cd apps
mix new firestorm_data --sup
cd firestorm_data

Let's review the Forum data model:

  • A Forum has many Categories
  • A Category has many Threads
  • A Thread has many Posts
  • A Post belongs to a Thread
  • A Post belongs to a User

Creating Categories (and setting up Ecto)

We'll begin with categories. We should be able to create a category:

# test/categories_test.exs
defmodule FirestormData.CategoriesTest do
  # We'll create a DataCase momentarily
  use FirestormData.DataCase

  import FirestormData.Categories
  alias FirestormData.Categories.Category

  test "create_category/1 with valid data creates a category" do
    attrs = %{title: "some title"}
    assert {:ok, %Category{} = category} = create_category(attrs)
    assert category.title == "some title"
  end
end

We haven't created the ExUnit case that we're deriving from for this. We'll set up the project to load modules from the support directory when running tests:

defmodule FirestormData.MixProject do
  use Mix.Project
  # ...
  def project do
    [
      # ...
      elixirc_paths: elixirc_paths(Mix.env()),
      # ...
    ]
  end

  # ...
  defp elixirc_paths(:test), do: ["lib", "test/support"]
  defp elixirc_paths(_), do: ["lib"]
  # ...
end

Now we will open up test/support/data_case.ex and create our case template:

defmodule FirestormData.DataCase do
  @moduledoc """
  This module defines the setup for tests requiring
  access to the application's data layer.

  You may define functions here to be used as helpers in
  your tests.

  Finally, if the test case interacts with the database,
  it cannot be async. For this reason, every test runs
  inside a transaction which is reset at the beginning
  of the test unless the test case is marked as async.
  """

  use ExUnit.CaseTemplate

  using do
    quote do
      alias FirestormData.Repo

      import Ecto
      import Ecto.Changeset
      import Ecto.Query
      import FirestormData.DataCase
    end
  end

  setup tags do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(FirestormData.Repo)

    unless tags[:async] do
      Ecto.Adapters.SQL.Sandbox.mode(FirestormData.Repo, {:shared, self()})
    end

    :ok
  end

  @doc """
  A helper that transform changeset errors to a map of messages.

      changeset = Accounts.create_user(%{password: "short"})
      assert "password is too short" in errors_on(changeset).password
      assert %{password: ["password is too short"]} = errors_on(changeset)

  """
  def errors_on(changeset) do
    Ecto.Changeset.traverse_errors(changeset, fn {message, opts} ->
      Enum.reduce(opts, message, fn {key, value}, acc ->
        String.replace(acc, "%{#{key}}", to_string(value))
      end)
    end)
  end
end

If you try to run the test now, they will fail because we haven't set up Ecto yet. We use Ecto to manage our interactions with the database. Let's bring it in:

defmodule FirestormData.MixProject do
  # ...
  defp deps do
    [
      {:ecto_sql, "~> 3.0"},
      {:postgrex, "~> 0.14"}
    ]
  end
end

Here we're bringing in ecto and postgres support.

We'll fetch the dependencies:

mix deps.get

Now we need to configure ecto. Open up config/config.exs, configure the repo and make it use microseconds for migrations, and enable per-mix-environment configuration:

# ...
config :firestorm_data,
  ecto_repos: [FirestormData.Repo]

config :firestorm_data, FirestormData.Repo, migration_timestamps: [type: :naive_datetime_usec]

import_config "#{Mix.env()}.exs"

Then make a file for each environment:

# config/dev.exs
use Mix.Config

config :firestorm_data, FirestormData.Repo,
  adapter: Ecto.Adapters.Postgres,
  database: "firestorm_data_repo_dev",
  hostname: "localhost"
# config/test.exs
use Mix.Config

config :firestorm_data, FirestormData.Repo,
  adapter: Ecto.Adapters.Postgres,
  database: "firestorm_data_repo_test",
  hostname: "localhost",
  pool: Ecto.Adapters.SQL.Sandbox,
  ownership_timeout: 30_000

# Print only warnings and errors during test
config :logger, level: :warn
# config/prod.exs
use Mix.Config

config :firestorm_data, FirestormData.Repo,
  adapter: Ecto.Adapters.Postgres,
  hostname: "localhost",
  # if you haven't seen this before, later we'll take advantage of REPLACE_OS_VARS to replace these with env vars at runtime.
  username: "${DB_USER}",
  password: "${DB_PASSWORD}",
  database: "${DB_NAME}",
  pool_size: 10

Now if we try to run the tests, they'll fail because there is no categories module. We'll create it at lib/firestorm_data/categories/categories.ex:

defmodule FirestormData.Categories do
  @moduledoc """
  Threads exist within categories.
  """

  alias FirestormData.{
    Repo,
    Categories.Category
  }

  @doc """
  List categories
  """
  @spec list_categories() :: [Category.t()]
  def list_categories() do
    Category
    |> Repo.all()
  end

  @doc """
  Creates a category.

  ## Examples

      iex> create_category(%{title: "Elixir"})
      {:ok, %Category{}}

      iex> create_category(%{field: bad_value})
      {:error, %Ecto.Changeset{}}

  """
  @spec create_category(map()) :: {:ok, Category.t()} | {:error, Ecto.Changeset.t()}
  def create_category(attrs \\ %{}) do
    %Category{}
    |> Category.changeset(attrs)
    |> Repo.insert()
  end

  @doc """
  Gets a single category by id.

  ## Examples

      iex> get_category("123")
      {:ok, %Category{}}

      iex> get_category!("456")
      {:error, :no_such_category}

  """
  @spec get_category(String.t()) :: {:ok, Category.t()} | {:error, :no_such_category}
  def get_category(id) do
    case Repo.get(Category, id) do
      nil -> {:error, :no_such_category}
      category -> {:ok, category}
    end
  end

  @doc """
  Updates a category.

  ## Examples

      iex> update_category(category, %{field: new_value})
      {:ok, %Category{}}

      iex> update_category(category, %{field: bad_value})
      {:error, %Ecto.Changeset{}}

  """
  @spec update_category(Category.t(), map()) :: {:ok, Category.t()} | {:error, Ecto.Changeset.t()}
  def update_category(%Category{} = category, attrs) do
    category
    |> Category.changeset(attrs)
    |> Repo.update()
  end

  @doc """
  Deletes a Category.

  ## Examples

      iex> delete_category(category)
      {:ok, %Category{}}

      iex> delete_category(category)
      {:error, %Ecto.Changeset{}}

  """
  @spec delete_category(Category.t()) :: {:ok, Category.t()} | {:error, Ecto.Changeset.t()}
  def delete_category(%Category{} = category) do
    Repo.delete(category)
  end
end

We need to define the Category schema. Categories just have a title field for now.

defmodule FirestormData.Categories.Category do
  @moduledoc """
  Schema for forum categories.
  """

  use FirestormData.Data, :model

  alias FirestormData.Categories.{
    Category
  }

  @type t :: %Category{
          id: String.t(),
          title: String.t(),
          inserted_at: DateTime.t(),
          updated_at: DateTime.t()
        }
  schema "firestorm_categories_categories" do
    field(:title, :string)

    timestamps()
  end

  def changeset(%__MODULE__{} = category, attrs \\ %{}) do
    category
    |> cast(attrs, [:title])
    |> validate_required([:title])
  end
end

I tend to namespace my tables, so we'll have the categories context's categories table, and I also am scoping the database tables with the firestorm prefix to make it easier for people that want to co-locate this data in an existing database, so we end up with the table name firestorm_categories_categories. Yes it looks goofy, but it's valuable in larger projects in my experience.

We use a module named FirestormData.Data. This is because we want to have a common configuration for our schemas. Here's what that looks like:

# lib/firestorm_data/data.ex
defmodule FirestormData.Data do
  def model do
    quote do
      use Ecto.Schema
      import Ecto
      import Ecto.Changeset
      import Ecto.Query

      # We use uuid primary and foreign keys, and we want microseconds in our timestamps
      @primary_key {:id, :binary_id, autogenerate: true}
      @foreign_key_type :binary_id
      @timestamps_opts [type: :utc_datetime, usec: true]
    end
  end

  @doc """
  When used, dispatch to the appropriate controller/view/etc.
  """
  defmacro __using__(which) when is_atom(which) do
    apply(__MODULE__, which, [])
  end
end

We'll create the migration for categories:

mix ecto.gen.migration create_categories

This won't work because we haven't yet created our repo.

# lib/firestorm_data/repo.ex
defmodule FirestormData.Repo do
  use Ecto.Repo, otp_app: :firestorm_data
end

Now we can create a migration:

mix ecto.gen.migration create_categories

We'll make our first table:

defmodule FirestormData.Repo.Migrations.CreateCategories do
  use Ecto.Migration

  def change do
    create table(:firestorm_categories_categories, primary_key: false) do
      add(:id, :uuid, primary_key: true)
      add(:title, :string)

      timestamps()
    end
  end
end
MIX_ENV=test mix ecto.migrate

We can run our tests:

mix test

They fail because we haven't started our repo. Let's open up the application and add it as a child:

# lib/firestorm_data/application.ex
defmodule FirestormData.Application do
  # ...
  def start(_type, _args) do
    # List all child processes to be supervised
    children = [
      FirestormData.Repo
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: FirestormData.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Now we can actually run the tests:

mix test

And they pass. Now we have a single schema, Category, that has a single field, title.

Creating Threads

A Category should have a list of threads. We'll make a new context for each of Categories, Threads, and Posts, just to give us some room to grow later. These aren't bounded contexts - generally, they know about each other. However, it's nice to have each piece have its own module where its API lives.

We'll add a test for threads:

# test/threads_test.ex
defmodule FirestormData.ThreadsTest do
  use FirestormData.DataCase

  import FirestormData.Threads

  alias FirestormData.{
    Categories,
    Threads.Thread
  }

  setup do
    {:ok, category} = Categories.create_category(%{title: "New Category"})
    {:ok, category: category}
  end

  test "create_thread/2 with valid data creates a thread", %{
    category: category
  } do
    attrs = %{title: "Some title"}
    assert {:ok, %Thread{} = thread} = create_thread(category, attrs)
    assert thread.title == attrs.title
  end

  test "get_thread returns the thread with given id", %{category: category} do
    {:ok, %Thread{id: id}} = create_thread(category, %{title: "New Title"})
    assert {:ok, %Thread{id: ^id}} = get_thread(category, id)
  end

  test "create_thread/2 with invalid data returns error changeset", %{
    category: category
  } do
    assert {:error, changeset} = create_thread(category, %{title: nil})
    assert "can't be blank" in errors_on(changeset).title
  end

  test "update_thread/2 with valid data updates the thread", %{category: category} do
    {:ok, thread = %Thread{}} = create_thread(category, %{title: "New Title"})
    updated_attrs = %{title: "some updated title"}
    assert {:ok, %Thread{} = thread} = update_thread(thread, updated_attrs)
    assert thread.title == updated_attrs.title
  end

  test "update_thread/2 with invalid data returns error changeset", %{
    category: category
  } do
    {:ok, thread = %Thread{}} = create_thread(category, %{title: "New Title"})
    assert {:error, changeset} = update_thread(thread, %{title: nil})
    assert "can't be blank" in errors_on(changeset).title
  end

  test "delete_thread/1 deletes the thread", %{category: category} do
    {:ok, thread = %Thread{}} = create_thread(category, %{title: "New Title"})
    assert {:ok, %Thread{}} = delete_thread(thread)
    assert {:error, :no_such_thread} = get_thread(category, thread.id)
  end
end

This verifies that we can create threads, update them, and delete them. They require a title and a category.

Let's create the threads context:

mkdir lib/firestorm_data/threads
# lib/firestorm_data/threads/threads.ex
defmodule FirestormData.Threads do
  @moduledoc """
  A Thread has a series of posts within it.
  """

  alias FirestormData.{
    Repo,
    Categories.Category,
    Threads.Thread
  }

  @doc """
  Gets a thread within a category.

  ## Examples

      iex> get_thread(category, "123")
      {:ok, %Thread{}}

      iex> get_thread(category, "456")
      {:error, :no_such_thread}

  """
  @spec get_thread(Category.t(), String.t()) :: {:ok, Thread.t()} | {:error, :no_such_thread}
  def get_thread(%Category{id: category_id}, id) do
    case Repo.get_by(Thread, id: id, category_id: category_id) do
      nil -> {:error, :no_such_thread}
      thread -> {:ok, thread}
    end
  end

  @doc """
  Gets a thread by id

  ## Examples

      iex> get_thread("123")
      {:ok, %Thread{}}

      iex> get_thread("nope")
      {:error, :no_such_thread}

  """
  @spec get_thread(String.t()) :: {:ok, Thread.t()} | {:error, :no_such_thread}
  def get_thread(id) do
    case Repo.get_by(Thread, id: id) do
      nil -> {:error, :no_such_thread}
      thread -> {:ok, thread}
    end
  end

  @doc """
  Creates a thread.

  ## Examples

      iex> create_thread(category, %{title: "OTP is cool"})
      {:ok, %Thread{}}

      iex> create_thread(category, %{title: nil})
      {:error, %Ecto.Changeset{}}

  """
  def create_thread(category, attrs \\ %{}) do
    thread_attrs =
      attrs
      |> Map.take([:title])
      |> Map.put(:category_id, category.id)

    %Thread{}
    |> Thread.changeset(thread_attrs)
    |> Repo.insert()
  end

  @doc """
  Updates a thread.

  ## Examples

      iex> update_thread(thread, %{title: "new title"})
      {:ok, %Thread{}}

      iex> update_thread(thread, %{title: nil})
      {:error, %Ecto.Changeset{}}

  """
  @spec update_thread(Thread.t(), map()) :: {:ok, Thread.t()} | {:error, Ecto.Changeset.t()}
  def update_thread(%Thread{} = thread, attrs) do
    thread
    |> Thread.changeset(attrs)
    |> Repo.update()
  end

  @doc """
  Deletes a Thread.

  ## Examples

      iex> delete_thread(thread)
      {:ok, %Thread{}}

      iex> delete_thread(thread)
      {:error, %Ecto.Changeset{}}

  """
  @spec delete_thread(Thread.t()) :: {:ok, Thread.t()} | {:error, Ecto.Changeset.t()}
  def delete_thread(%Thread{} = thread) do
    Repo.delete(thread)
  end
end

Now we can try to run the test:

mix test

It fails because there is no Thread schema, so we'll make that:

# lib/firestorm_data/threads/thread.ex
defmodule FirestormData.Threads.Thread do
  @moduledoc """
  Schema for forum threads.
  """

  use FirestormData.Data, :model

  alias FirestormData.{
    Threads.Thread,
    Categories.Category
  }

  @type t :: %Thread{
          id: String.t(),
          title: String.t(),
          category: Category.t() | %Ecto.Association.NotLoaded{},
          inserted_at: DateTime.t(),
          updated_at: DateTime.t()
        }
  schema "firestorm_threads_threads" do
    field(:title, :string)
    belongs_to(:category, Category)

    timestamps()
  end

  def changeset(%__MODULE__{} = thread, attrs \\ %{}) do
    thread
    |> cast(attrs, [:title, :category_id])
    |> validate_required([:title, :category_id])
  end
end

If we run the tests now, they fail because there's no database table for threads. We'll add it:

mix ecto.gen.migration create_threads
defmodule FirestormData.Repo.Migrations.CreateThreads do
  use Ecto.Migration

  def change do
    create table(:firestorm_threads_threads, primary_key: false) do
      add(:id, :uuid, primary_key: true)
      add(:title, :string)

      add(
        :category_id,
        references(:firestorm_categories_categories, type: :binary_id, on_delete: :delete_all)
      )

      timestamps()
    end

    create(index(:firestorm_threads_threads, [:category_id]))
  end
end
MIX_ENV=test mix ecto.migrate

The tests pass.

Creating Posts

We now have Categories and Threads. Threads have many posts, which are the actual things we care about in our system. We'll add a test:

# test/posts_test.ex
defmodule FirestormData.PostsTest do
  use FirestormData.DataCase

  import FirestormData.Posts

  alias FirestormData.{
    Categories,
    Threads,
    Posts.Post
  }

  setup do
    {:ok, category} = Categories.create_category(%{title: "Category"})
    {:ok, thread} = Threads.create_thread(category, %{title: "Thread"})
    {:ok, thread: thread}
  end

  test "creating a post in a thread", %{thread: thread} do
    assert {:ok, %Post{} = post} = create_post(thread, %{body: "Post body"})
    assert post.thread_id == thread.id
    assert post.body == "Post body"
  end
end

We create a category and a thread, and ensure that we can create a post in that thread.

Let's add the Posts context:

mkdir lib/firestorm_data/posts
# lib/firestorm_data/posts/posts.ex
defmodule FirestormData.Posts do
  @moduledoc """
  Posts exist on Threads
  """

  import Ecto.Query

  alias FirestormData.{
    Repo,
    Posts.Post,
    Threads.Thread
  }

  @doc """
  Creates a post within a thread.

  ## Examples

      iex> create_post(thread, %{body: "don't you think?"})
      {:ok, %Post{}}

      iex> create_post(thread, %{body: nil_value})
      {:error, %Ecto.Changeset{}}

  """
  def create_post(%Thread{} = thread, attrs) do
    attrs =
      attrs
      |> Map.put(:thread_id, thread.id)

    %Post{}
    |> Post.changeset(attrs)
    |> Repo.insert()
  end

  def list_posts(%Thread{id: thread_id}) do
    Post
    |> where([p], p.thread_id == ^thread_id)
    |> order_by([p], desc: p.inserted_at)
    |> Repo.all()
  end
end

We need to add the Post schema:

defmodule FirestormData.Posts.Post do
  @moduledoc """
  Schema for forum posts.
  """

  use FirestormData.Data, :model

  alias FirestormData.{
    Threads.Thread,
    Posts.Post
  }

  @type t :: %Post{
          id: String.t(),
          body: String.t(),
          thread: Thread.t() | %Ecto.Association.NotLoaded{},
          inserted_at: DateTime.t(),
          updated_at: DateTime.t()
        }
  schema "firestorm_posts_posts" do
    field(:body, :string)
    belongs_to(:thread, Thread)

    timestamps()
  end

  def changeset(%__MODULE__{} = post, attrs \\ %{}) do
    post
    |> cast(attrs, [:body, :thread_id])
    |> validate_required([:body, :thread_id])
  end
end

We'll run the tests, but of course we expect them to fail because there's no database table yet. We'll create the migration:

mix ecto.gen.migration create_posts
defmodule FirestormData.Repo.Migrations.CreatePosts do
  use Ecto.Migration

  def change do
    create table(:firestorm_posts_posts, primary_key: false) do
      add(:id, :uuid, primary_key: true)
      add(:body, :text)

      add(
        :thread_id,
        references(:firestorm_threads_threads, type: :binary_id, on_delete: :delete_all)
      )

      timestamps()
    end

    create(index(:firestorm_posts_posts, [:thread_id]))
  end
end
MIX_ENV=test mix ecto.migrate
mix test

The tests pass.

Creating Users

So now we have Categories, Threads, and Posts. Of course, users make posts. Let's add a Users context to handle them. We'll start with a test:

# test/users_test.exs
defmodule FirestormData.UsersTest do
  use FirestormData.DataCase

  import FirestormData.Users
  alias FirestormData.Users.User

  @attrs %{
    email: "some email",
    name: "some name",
    username: "some username"
  }

  test "create_user/1 with valid data creates a user" do
    assert {:ok, %User{} = user} = create_user(@attrs)
    assert user.email == "some email"
    assert user.name == "some name"
    assert user.username == "some username"
  end

  test "create_user/1 with invalid data returns error changeset" do
    assert {:error, changeset} = create_user(Map.delete(@attrs, :username))
    assert "can't be blank" in errors_on(changeset).username
  end

  test "get_user returns the user with given id" do
    {:ok, user} = create_user(@attrs)
    assert {:ok, %User{}} = get_user(user.id)
  end

  test "update_user/2 with valid data updates the user" do
    {:ok, user} = create_user(@attrs)
    updated_attrs = %{email: "updated@example.com", name: "New name", username: "new_username"}
    assert {:ok, user = %User{}} = update_user(user, updated_attrs)
    assert user.email == updated_attrs.email
    assert user.name == updated_attrs.name
    assert user.username == updated_attrs.username
  end

  test "update_user/2 with invalid data returns error changeset" do
    {:ok, user} = create_user(@attrs)
    assert {:error, changeset} = update_user(user, %{username: nil})
    assert "can't be blank" in errors_on(changeset).username
  end

  test "delete_user/1 deletes the user" do
    {:ok, user} = create_user(@attrs)
    assert {:ok, %User{}} = delete_user(user)
    assert {:error, :no_such_user} = get_user(user.id)
  end
end

We can run the test, but of course it will fail because there's no Users module. We'll create it, much like the other context modules we've created so far:

mkdir lib/firestorm_data/users
# lib/firestorm_data/users/users.ex
defmodule FirestormData.Users do
  @moduledoc """
  Users can interact with the forum in an authenticated manner.
  """

  alias FirestormData.{
    Repo,
    Users.User
  }

  @doc """
  Creates a user.

  ## Examples

      iex> create_user(%{field: value})
      {:ok, %User{}}

      iex> create_user(%{field: bad_value})
      {:error, %Ecto.Changeset{}}

  """
  @spec create_user(map()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
  def create_user(attrs \\ %{}) do
    %User{}
    |> User.registration_changeset(attrs)
    |> Repo.insert()
  end

  @spec get_user(String.t()) :: {:ok, User.t()} | {:error, :no_such_user}
  def get_user(id) do
    case Repo.get(User, id) do
      nil -> {:error, :no_such_user}
      user -> {:ok, user}
    end
  end

  @spec delete_user(User.t()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
  def delete_user(user = %User{}) do
    Repo.delete(user)
  end

  @spec update_user(User.t(), map()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
  def update_user(user = %User{}, attrs) do
    user
    |> User.changeset(attrs)
    |> Repo.update()
  end
end

If we try to run the tests now, they will fail because there's no User schema. We'll add it:

# lib/firestorm_data/users/user.ex
defmodule FirestormData.Users.User do
  @moduledoc """
  Schema for forum users.
  """

  use FirestormData.Data, :model
  alias FirestormData.Users.User

  @type t :: %User{
          email: String.t(),
          name: String.t(),
          username: String.t(),
          password_hash: String.t(),
          password: nil | String.t(),
          inserted_at: DateTime.t(),
          updated_at: DateTime.t()
        }

  schema "firestorm_users_users" do
    field(:email, :string)
    field(:name, :string)
    field(:username, :string)
    field(:password_hash, :string)
    field(:password, :string, virtual: true)

    timestamps()
  end

  @required_fields ~w(username)a
  @optional_fields ~w(email name)a

  def changeset(%__MODULE__{} = user, attrs \\ %{}) do
    user
    |> cast(attrs, @required_fields ++ @optional_fields)
    |> validate_required(@required_fields)
    |> unique_constraint(:username)
  end

  def registration_changeset(%__MODULE__{} = user, attrs) do
    user
    |> changeset(attrs)
    |> cast(attrs, [:password])
    |> validate_length(:password, min: 6)
    |> put_password_hash()
  end

  defp put_password_hash(changeset) do
    case changeset do
      %Ecto.Changeset{valid?: true, changes: %{password: pass}} ->
        changeset
        |> put_change(:password_hash, Comeonin.Bcrypt.hashpwsalt(pass))

      _ ->
        changeset
    end
  end
end

That's our user schema. We'll create the migration for the corresponding database table:

mix ecto.gen.migration create_users

We see a warning about the missing comeonin library we referenced - we'll add it in a moment. Let's finish the database migration:

defmodule FirestormData.Repo.Migrations.CreateUsers do
  use Ecto.Migration

  def change do
    create table(:firestorm_users_users, primary_key: false) do
      add(:id, :uuid, primary_key: true)
      add(:username, :string)
      add(:email, :string)
      add(:name, :string)
      add(:password_hash, :string)
      add(:api_token, :string)

      timestamps()
    end

    create(unique_index(:firestorm_users_users, [:username]))
  end
end

We'll run the migration:

MIX_ENV=test mix ecto.migrate

Now we can try to run the tests; they'll fail because we haven't added comeonin. Right?

mix test

They passed. What gives?They pass because we actually allow creating users without passwords, for the purposes of supporting OAuth registration later. The code that hashes the password isn't exercised in this test. Let's add a test regarding creating a user with a password and ensure it's stored correctly:

defmodule FirestormData.UsersTest do
  # ...
  test "create_user/1 with valid data creates a user with a password" do
    assert {:ok, %User{} = user} = create_user(Map.put(@attrs, :password, "somepassword"))
    assert user.email == "some email"
    assert user.name == "some name"
    assert user.username == "some username"
    assert user.password_hash
  end
  # ...
end

Now if we run the tests, they fail because of the missing dependency. Let's add it:

# mix.exs
defmodule FirestormData.MixProject do
  # ...
  defp deps do
    [
      # ...
      # We use bcrypt to hash passwords
      {:bcrypt_elixir, "~> 1.1.1"},
      # We interact with bcrypt via comeonin
      {:comeonin, "~> 4.0"}
    ]
  end
end
mix deps.get
mix test

Now the tests pass. To speed up testing a bit, we can reduce the number of rounds bcrypt uses:

# config/test.exs
config :bcrypt_elixir, log_rounds: 4

Posts should belong to Users

Now, we have Categories, Threads, Posts, and Users - the core data models for the app. However, it's still a bit hobbled:Posts should belong to users. Let's tweak Posts.create_post to require a user. We'll start with the tests:

# test/posts_test.exs
defmodule FirestormData.PostsTest do
  use FirestormData.DataCase

  import FirestormData.Posts

  alias FirestormData.{
    Categories,
    Threads,
    Users,
    Posts.Post
  }

  setup do
    {:ok, category} = Categories.create_category(%{title: "Category"})
    {:ok, thread} = Threads.create_thread(category, %{title: "Thread"})

    {:ok, user} =
      Users.create_user(%{
        username: "knewter",
        name: "Josh Adams",
        email: "josh@smoothterminal.com"
      })

    {:ok, thread: thread, user: user}
  end

  test "creating a post in a thread", %{thread: thread, user: user} do
    assert {:ok, %Post{} = post} = create_post(thread, user, %{body: "Post body"})
    assert post.thread_id == thread.id
    assert post.user_id == user.id
    assert post.body == "Post body"
  end
end

The test will fail of course, as we added a parameter to the function. Let's update the create_post function to require a %User{} as the second argument:

defmodule FirestormData.Posts do
  # ...
  alias FirestormData.{
    # ...
    Users.User
  }

  @doc """
  Creates a post within a thread.

  ## Examples

      iex> create_post(thread, user, %{body: "don't you think?"})
      {:ok, %Post{}}

      iex> create_post(thread, user, %{body: nil_value})
      {:error, %Ecto.Changeset{}}

  """
  def create_post(%Thread{} = thread, %User{} = user, attrs) do
    attrs =
      attrs
      |> Map.put(:thread_id, thread.id)
      |> Map.put(:user_id, user.id)

    %Post{}
    |> Post.changeset(attrs)
    |> Repo.insert()
  end
  # ...
end
mix test

It fails because there's no user_id field in %Post{}. We'll add the association:

defmodule FirestormData.Posts.Post do
  # ...
  alias FirestormData.{
    # ...
    Users.User
  }

  @type t :: %Post{
          # ...
          user: User.t() | %Ecto.Association.NotLoaded{},
          # ...
        }
  schema "firestorm_posts_posts" do
    # ...
    belongs_to(:user, User)
    # ...
  end

  def changeset(%__MODULE__{} = post, attrs \\ %{}) do
    post
    |> cast(attrs, [:body, :thread_id, :user_id])
    |> validate_required([:body, :thread_id, :user_id])
  end
end
mix test

It fails because there's no column user_id in the Post schema's table, so we'll add that in a migration:

mix ecto.gen.migration add_user_id_to_posts
defmodule FirestormData.Repo.Migrations.AddUserIdToPosts do
  use Ecto.Migration

  def change do
    alter table(:firestorm_posts_posts) do
      add(:user_id, references(:firestorm_users_users, type: :binary_id, on_delete: :nilify_all))
    end

    create(index(:firestorm_posts_posts, [:user_id]))
  end
end
MIX_ENV=test mix ecto.migrate
mix test

The tests pass. Now posts are associated with users.

Threads without any Posts are dumb

There's one other thing we'd like to resolve: we don't want to be able to create a thread without a first post. That means we need to specify the user when creating a thread, as well as pass the body of the first post. We'll update the test:

defmodule FirestormData.ThreadsTest do
  use FirestormData.DataCase

  import FirestormData.Threads

  alias FirestormData.{
    Categories,
    Threads.Thread,
    Users
  }

  setup do
    {:ok, category} = Categories.create_category(%{title: "New Category"})

    {:ok, user} =
      Users.create_user(%{
        username: "knewter",
        name: "Josh Adams",
        email: "josh@smoothterminal.com"
      })

    {:ok, category: category, user: user}
  end

  test "create_thread/3 with valid data creates a thread and its first post", %{
    category: category,
    user: user
  } do
    attrs = %{title: "Some title", body: "First post"}
    assert {:ok, %Thread{} = thread} = create_thread(category, user, attrs)
    assert thread.title == attrs.title
    first_post = hd(thread.posts)
    assert first_post.thread_id == thread.id
    assert first_post.body == "First post"
  end

  test "get_thread returns the thread with given id", %{
    category: category,
    user: user
  } do
    {:ok, %Thread{id: id}} =
      create_thread(category, user, %{title: "New Title", body: "First post"})

    assert {:ok, %Thread{id: ^id}} = get_thread(category, id)
  end

  test "create_thread/3 with invalid data returns error changeset", %{
    category: category,
    user: user
  } do
    assert {:error, changeset} = create_thread(category, user, %{title: nil})
    assert "can't be blank" in errors_on(changeset).title
  end

  test "update_thread/2 with valid data updates the thread", %{
    category: category,
    user: user
  } do
    {:ok, thread = %Thread{}} =
      create_thread(category, user, %{title: "New Title", body: "First post"})

    updated_attrs = %{title: "some updated title"}
    assert {:ok, %Thread{} = thread} = update_thread(thread, updated_attrs)
    assert thread.title == updated_attrs.title
  end

  test "update_thread/2 with invalid data returns error changeset", %{
    category: category,
    user: user
  } do
    {:ok, thread = %Thread{}} =
      create_thread(category, user, %{title: "New Title", body: "First Post"})

    assert {:error, changeset} = update_thread(thread, %{title: nil})
    assert "can't be blank" in errors_on(changeset).title
  end

  test "delete_thread/1 deletes the thread", %{
    category: category,
    user: user
  } do
    {:ok, thread = %Thread{}} =
      create_thread(category, user, %{title: "New Title", body: "First Post"})

    assert {:ok, %Thread{}} = delete_thread(thread)
    assert {:error, :no_such_thread} = get_thread(category, thread.id)
  end
end

If we run the tests, they show us what we need to fix in the Threads.create_thread function. Let's require a user and a body:

defmodule FirestormData.Threads do
  # ...
  @doc """
  Creates a thread.

  ## Examples

      iex> create_thread(category, user, %{title: "OTP is cool", body: "First Post"})
      {:ok, %Thread{}}

      iex> create_thread(category, user, %{title: nil})
      {:error, %Ecto.Changeset{}}

  """
  def create_thread(category, user, attrs \\ %{}) do
    post_attrs =
      attrs
      |> Map.take([:body])
      |> Map.put(:user_id, user.id)

    thread_attrs =
      attrs
      |> Map.take([:title])
      |> Map.put(:category_id, category.id)

    %{thread: thread_attrs, post: post_attrs}
    |> Thread.new_changeset()
    |> Repo.insert()
  end
  # ...
end

Here we're extracting the data we care about for the post and the thread, and passing them into a new changeset function we'll create, Thread.new_changeset. Let's see how that's implemented using put_assoc to put the association into the database as we insert the thread. We'll also need to add the posts association to Thread:

defmodule FirestormData.Threads.Thread do
  # ...
  alias FirestormData.{
    Posts.Post,
    # ...
  }

  @type t :: %Thread{
          # ...
          posts: [Post.t()] | %Ecto.Association.NotLoaded{},
          # ...
        }
  schema "firestorm_threads_threads" do
    # ...
    has_many(:posts, Post)
    # ...
  end
  # ...

  def new_changeset(%{thread: thread_attrs, post: post_attrs}) do
    post_changeset =
      %Post{}
      |> cast(post_attrs, [:body, :user_id])
      |> validate_required([:body, :user_id])

    %__MODULE__{}
    |> changeset(thread_attrs)
    |> put_assoc(:posts, [post_changeset])
  end
end

If we run the tests, we see that we also need to update the PostsTest to handle the API change when creating a thread:

defmodule FirestormData.PostsTest do
  # ...
  setup do
    # ...
    {:ok, thread} = Threads.create_thread(category, user, %{title: "Thread", body: "First post"})

    {:ok, thread: thread, user: user}
  end
  # ...
end

With that, if we run the tests, they pass.

Categories have threads

At present, our data model still has a gap - because we built Categories before Threads, we haven't added the has_many association to the Category schema. Let's write a test first:

defmodule FirestormData.CategoriesTest do
  # ...
  alias FirestormData.{
    Categories.Category,
    Repo,
    Threads,
    Threads.Thread,
    Users
  }
  # ...
  test "categories have many threads" do
    {:ok, user} =
      Users.create_user(%{
        username: "knewter",
        name: "Josh Adams",
        email: "josh@smoothterminal.com"
      })

    attrs = %{title: "some title"}
    {:ok, %Category{} = category} = create_category(attrs)

    {:ok, %Thread{id: thread_id}} =
      Threads.create_thread(
        category,
        user,
        %{title: "Thread", body: "First post"}
      )

    category = Repo.preload(category, [:threads])
    assert thread_id in Enum.map(category.threads, & &1.id)
  end
end

If we run the tests, we see that there's no threads field on %Category{}. We can fix that!

defmodule FirestormData.Categories.Category do
  # ...
  alias FirestormData.{
    Categories.Category,
    Threads.Thread
  }

  @type t :: %Category{
          # ...
          threads: [Thread.t()] | %Ecto.Association.NotLoaded{},
          # ...
        }
  schema "firestorm_categories_categories" do
    # ...
    has_many(:threads, Thread)
    # ...
  end
  # ...
end

We can run the tests again, and they pass.

Summary

When we started out, we described our desired data model:

- A Forum has many Categories
- A Category has many Threads
- A Thread has many Posts
- A Post belongs to a Thread
- A Post belongs to a User

We built this data model from scratch. We also tweaked our API to enforce an inability to create a thread without also simultaneously creating a first post. We did this by writing tests that described the desired behaviour, then letting those tests inform the next steps we took.

Resources