Understanding gen_server with Elixir and Ruby

Recently, I've been spending some time working in Erlang and Elixir. I had tried to break into Erlang in the past but I was always stymied by the steep learning curve of OTP. gen_server in particular always seemed like black magic to me. However, after attending an Erlang workshop at Lambda Jam this year it finally clicked for me. After I finally "got it" I had another realization: it isn't that complicated, but there aren't very many good explanations. In this post I'm going to attempt to explain gen_server using Elixir and Ruby code and some simple diagrams.

I chose to use Elixir rather than Erlang because I've really enjoyed working in it recently and I think the syntax is approachable for those new to Erlang/Elixir as well as those already familiar with Erlang.

Disclaimer: My goal is to help you understand the concepts around gen_server not how the actual underlying implementation works. Specifically, the Ruby code I link to at the end of the post is meant to help you understand the idea of what's happening. It is in no way a real implementation of gen_server in Ruby.

Concepts

The two key concepts we're going to focus on in this blog post are state and behavior.

In traditional OO languages classes store behavior and instances store state. You write a class definition with methods that specify the behavior of an object and then you instantiate that class to create an instance of an object that holds its own state.

           +-----------------+                   +-----------------+
           |      Class      |                   |     Instance    |
           |-----------------|                   |-----------------|
           |                 |                   |                 |
           |                 |                   |                 |
           |    Behavior     +------------------>|      State      |
           |                 |                   |                 |
           |                 |                   |                 |
           |                 |                   |                 |
           |                 |                   |                 |
           +-----------------+                   +-----------------+

Let's write a simple ruby class that has behavior and state.

class Greeter  
  def initialize(name)
    @name = name
  end

  def greet
    "Hello, #{@name}"
  end
end

drew = Greeter.new("Drew")  
puts drew.greet  

In the example above, the Greeter class defines some behavior, namely the greet and initialize methods. The instance of the Greeter class, drew, is instantiated with some state (the string "Drew") and this state can be referenced from behavior that the class defines. The greet method uses the instance variable @name. Those of us who have spent any time with an OO language understand these concepts intuatively. Objects are the fusion of behavior and state, classes and instances.

But there's a problem. Elixir doesn't have state. You can't (really) store stuff anywhere. We can fake something like the code above using modules and records.

defrecord Person, greet: ""

defmodule Greeter do  
  def new(name) do
    Person.new(greet: "Hello, #{name}")
  end
end

drew = Greeter.new("Drew")  
IO.puts drew.greet  

But there's a key difference here. We're creating records to hold values rather than state. These values can never change over time, so if we wanted to modify the greet attribute of our Person record we would actually create a totally new record. What if we want to have a true concept of state that can change over time and that we can use as part of the behavior we describe in a module?

Well, as you might have guessed, gen_server fills this role in Elixir. We will use gen_server to create processes that will be associated with our modules. Modules will contain behavior and processes will contain state.

           +-----------------+                   +-----------------+
           |      Module     |                   |     Process     |
           |-----------------|                   |-----------------|
           |                 |                   |                 |
           |                 |                   |                 |
           |    Behavior     +------------------>|      State      |
           |                 |                   |                 |
           |                 |                   |                 |
           |                 |                   |                 |
           |                 |                   |                 |
           +-----------------+                   +-----------------+

An Example

We're going to build a simple stack using Elixir and gen_server. We'll be able to push values onto the stack and pop values off of the stack. Our module will define the behvior of the stack but gen_server will store the state.

Because our module isn't responsible for storing the state, the methods we define inside of it must assume that the state will be passed in as arguments. The return values from these methods will need to include a new value for the state after the call. This makes the return values from our methods look a little weird because we have to both return the result of the method call (if there is one) as well as the new state. But don't worry, it's not too bad.

It will be gen_server's responsibility to find the state associated with our process and pass it to our methods as well as storing the resulting state after the method call.

First, let's take a look at our Stack module.

defmodule Stack do  
  use GenServer.Behaviour

  def handle_call(:pop, _from, []) do
    {:reply, nil, []}
  end

  def handle_call(:pop, _from, state) do
    [head|new_state] = state

    {:reply, head, new_state}
  end

  def handle_cast({:push, value}, state) do
    {:noreply, [value|state]}
  end
end  

Ok, so there's some weird stuff going on here. First, why are all our methods named handle_call and handle_cast? It's because when gen_server finds our processes state for us, it calls these callback methods on our module and passes along the arguments we provided, the pid of the caller (the _from argument, which we ignore) and the current state. Note that there are two (for now) types of callbacks: call and cast. Call is synchronous, it updates the state and sends a reply. Cast is asynchronous, it updates the state but doesn't send a reply.

Next, let's look at how we actually create a gen_server process (our equivilent of an instance) associated with this module and interact with it.

iex(1)> {:ok, pid} = :gen_server.start_link(Stack, [], [])  
{:ok, #PID<0.40.0>}
iex(2)> :gen_server.call(pid, :pop)  
nil  
iex(3)> :gen_server.cast(pid, {:push, 1})  
:ok
iex(4)> :gen_server.cast(pid, {:push, 2})  
:ok
iex(5)> :gen_server.call(pid, :pop)  
2  
iex(6)> :gen_server.call(pid, :pop)  
1  

First, we start a new gen_server process associated with the Stack module using the start_link method on gen_server. The first argument is the module with our behavior and the second is the initial state. Ignore the third. This method call returns the symbol :ok along with the pid of the process. Think of this as our instance or a pointer to our state.

Now, when we want to actually call (or cast) a function defined in our module we just use the call and cast functions on gen_server. We always pass our pid as the first argument and then the arguemnts to our Stack function as the second argument. If we want to pass anything other than the function name we use a tuple.

Behind the scenes gen_server gets our pid, finds the state associated with it and then calls the appropriate method on our module passing the state along as an argument. That's it!

"But Drew!", you shout, "That looks hideous!" Well, you're correct. But we can hide the complexity of gen_server from our callers by writing a few more functions on our module.

defmodule Stack do  
  use GenServer.Behaviour

  def start_link(state) do
    {:ok, pid} = :gen_server.start_link(Stack, state, [])
    pid
  end

  def pop(pid) do
    :gen_server.call(pid, :pop)
  end

  def push(pid, value) do
    :gen_server.cast(pid, {:push, value})
  end

  def handle_call(:pop, _from, []) do
    {:reply, nil, []}
  end

  def handle_call(:pop, _from, state) do
    [head|new_state] = state

    {:reply, head, new_state}
  end

  def handle_cast({:push, value}, state) do
    {:noreply, [value|state]}
  end
end  

Now when we interact with our Stack, it looks nice.

iex(1)> pid = Stack.start_link([])  
#PID<0.45.0>
iex(2)> Stack.push(pid, 1)  
:ok
iex(3)> Stack.push(pid, 2)  
:ok
iex(4)> Stack.pop(pid)  
2  
iex(5)> Stack.pop(pid)  
1  

Wrapping Up

I hope this helps you gain a better understanding what gen_server provides and, on a conceptual level, how it works. If you're interested in digging further, take a look at this demonstration of implementating the spirit of gen_server in Ruby. Again, this is not a true implemetation but a learning tool to help those who understand Ruby get a better feel for what gen_server is doing.

Now go out there and write some Elixir (or Erlang, if you must).