Elixir Makes Testing Hard Things Easy

I love Elixir. If we’ve talked at a conference or you work with me, you know this. I constantly find small nuggets when working with Elixir that simply bring joy to me. One of these nuggets recently emerged that I thought worth sharing.

Backstory

I’m working on some learning material right now for real-time system development. One of the key aspects of a section is about how important measurement is. Without measurement, how do we have any confidence that our running system is healthy?

I’m using StatsD for this section because it’s fairly well-adopted and is conceptually simple to understand. However, I kept bumping into small inefficiencies in my flow. An example of this is that I use a Node program called statsd-logger to view my metrics in development. It felt a bit strange to have a reader install this Node program to view logs in our Elixir app.

The same problem came up today when I was writing tests for the StatsD metrics. I would typically use Mockery to test that Statix, my client, is correctly called. If I went down this path, I’d be putting a lot on the reader in terms of understanding what the heck a mock is and when to use it. Ideally, we’ll have 1 line of code max additional setup.

Okay, enough of the backstory. I wanted to fix these two very specific problems. Let’s get into the more interesting bits.

Elixir is Many Programs

Elixir runs processes in a way that makes them feel like little independent programs. We can spin up tens of thousands of these programs without any issue at all. We can even spin up programs that we may consider expensive, such as a webserver.

The real jewel of these many programs is that they are given a simple and efficient way to communicate with each other, message passing. To take the web server example to the extreme, we could spin up 5 different servers on 5 ports and interface with them through our browser. Behind the scenes, these “servers” would be in the same Elixir VM and could be passing messages back and forth nearly freely.

This is conceptually interesting, to me, because it means that traditionally complex or standalone applications can be run in the same program (very easily) and can share information.

Let’s see how this applies to StatsD.

Just run a Server

What if we just ran a StatsD server in our Elixir application? It could listen on a port and accept messages. We could do whatever we wanted with these messages, like log them out or store them. Let’s do that in a few lines of code:

$ iex
iex(1)> :gen_udp.open(8126, active: true)

We’ve started a “UDP server” in our terminal. Let’s send it some messages (open new terminal):

# Your mileage may vary, fingers crossed
$ echo -n "hello:1" | nc -4u -w0 localhost 8126
$ echo -n "hello:2" | nc -4u -w0 localhost 8126

Now let’s go back to the original iex terminal:

iex(2)> flush()
{:udp, #Port<0.5>, {127, 0, 0, 1}, 61814, 'hello:1'}
{:udp, #Port<0.5>, {127, 0, 0, 1}, 61515, 'hello:2'}
:ok

Woah! We received the UDP packets as messages to our process. That’s super neat to me because of how simple it was (due to the great backing work of Erlang/OTP.)

So what if this was the solution to problem 1 (statsd-logger)? We could start a UDP server and log out messages. I threw together a quick library for that purpose called StatsDLogger.

That worked out nicely. What about testing, though? Let’s take what we just wrote and apply it to tests.

Test Anything Easily

Testing can sometimes be hard. Integration testing across multiple servers is usually hard. What level do we start to mock at? We can’t actually run all of our services all of the time without having very slow tests. To help solve this, we can take what we just did and apply it to tests. There is a key question though, how do we actually write an assertion against our server?

In Elixir, message passing is champion. If you have a process’s ID, you can send it a message and it will go into that process’s mailbox. With ExUnit, this mailbox can be asserted against by using assert_receive. This means that if we can send our test process a message, we can write a test for it.

It’s slightly harder to boil this concept down into an example as simple as the last. Let’s look at some code I wrote to handle this for StatsD:

  def handle(:valid, name, value, opts) do
    pid = Keyword.fetch!(opts, :send_to)
    send(pid, {:statsd_recv, name, value})
  end

We can handle a metric by sending a message to a pid. Now the question is, “which pid?” The process that calls start_link can be referenced with self(). This means we can do something like this:

  def start_link(opts) do
    opts = Keyword.merge([send_to: self()], opts)
    GenServer.start_link(__MODULE__, opts)
  end

We can put all of this together to write a test like this:

  test "valid / invalid messages are handled" do
    StatsDLogger.start_link(port: @port, formatter: :send)

    send_event("a:1")
    send_event("a:2|c")
    send_event("invalid")

    assert_receive {:statsd_recv, "a", "1"}
    assert_receive {:statsd_recv, "a", "2|c"}
    assert_receive {:statsd_recv_invalid, "invalid"}
  end

Elixir has made testing this multi-server integration a piece of cake.

In the Wild

I’m definitely not finding any new technique here. The reason I love Elixir is that gems like this are all over open-source libraries. The one that comes to mind here is Bypass.

Bypass works by starting an anonymous HTTP server that runs a function when it’s invoked. That is a great example of “start a server” to solve how to test that an application makes the right request.

Phoenix Channels are a great example of “send a message” in the wild. When you write assert_push, you’re actually just checking for a specific message. The Channel test is wired up so that the test process becomes the “socket transport.” This allows it to be at the edge of the system and capture a lot of test for very little code.

Wrapping Up

If you’re here, thanks for making it through. I love Elixir because of small things like this that would actually be huge in some other languages I’ve worked with. If you’re looking for a way to test complex code, try out message passing as a potential solution.

Shameless Plug for ElixirConf

My colleague Grant and I are going to give a great training session at ElixirConf Tuesday course. You can find more details at https://elixirconf.com/2019/training-classes/2. We’ll be focused on writing real-time systems, and will specifically be leveraging a lot of real world lessons from doing this at decent scale on several pieces of our SaaS app. Come check it out!