Tennis Kata in Erlang

I’ve recently been reading ploeh’s series on doing the Tennis Kata in F#. I decided to have a go in Erlang, we might not have a fancy type system but we do have pattern matching and partial functions.

You can find the source code to follow along here.

I decided to implement a “game” as a gen_server (when you have a hammer…), I refer you to a previous post for the pre-amble.

Love all

The first test is to check the initial score:

initial_score(Pid) ->
    fun() ->
        ?assertEqual(#{p1=>love,p2=>love}, gen_server:call(Pid, get_score))
    end.

I decided to return a map, rather than a tuple or record, and to use the atom “love” rather than 0. The implementation is pretty trivial:

init([]) ->
    {ok, #{p1=>love,p2=>love}}.

handle_call(get_score, _From, State) ->
    {reply, State, State};

If there was more information in the state map, you might want to return a projection rather than the whole object.

Fifteen love

fifteen_love(Pid) ->
    fun() ->
        ?assertEqual(#{p1=>15,p2=>love}, gen_server:call(Pid, {won_point, p1}))
    end.

This time we need to calculate the new score when a player has won a point:

handle_call({won_point, p1}, _From, State = #{p1:=CurrentScore}) ->
    NewState = State#{p1:=new_score(CurrentScore)},
    {reply, NewState, NewState};

...

new_score(love) -> 15.

Love fifteen

love_fifteen(Pid) ->
    fun() ->
        ?assertEqual(#{p1=>love,p2=>15}, gen_server:call(Pid, {won_point, p2}))
    end.

We can re-use the new_score method from before:

handle_call({won_point, p2}, _From, State = #{p2:=CurrentScore}) ->
    NewState = State#{p2:=new_score(CurrentScore)},
    {reply, NewState, NewState};

Fifteen all

fifteen_all(Pid) ->
    fun() ->
        won_point(Pid, p1),
        ?assertEqual(#{p1=>15,p2=>15}, won_point(Pid, p2))
    end.

won_point(Pid, Player) ->
    gen_server:call(Pid, {won_point, Player}).

No need for any new code, but it seemed like time for a little test refactoring.

Thirty fifteen

thirty_fifteen(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>30,p2=>15}, won_point(Pid, p1))
    end.

Now we need to add another function head, to handle the new score:

new_score(love) -> 15;
new_score(15) -> 30.

Thirty all

thirty_all(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>30,p2=>30}, won_point(Pid, p1))
    end.

No new code required.

Forty thirty

forty_thirty(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>40,p2=>30}, won_point(Pid, p1))
    end.

Again, we need to handle a new score:

...
new_score(30) -> 40.

P1 win

p1_wins(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p1),
        ?assertEqual({game_over, p1}, won_point(Pid, p1))
    end.

This requires a special case, for when the player has 40 points:

handle_call({won_point, p1}, _From, State = #{p1:=40}) ->
    {stop, normal, {game_over, p1}, State};

But now we have a problem, as I skipped over the troublesome “deuce”.

Deuce

deuce(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>deuce,p2=>deuce}, won_point(Pid, p2))
    end.

Again, I decided to use an atom, rather than 40-40. But now we need to take both player’s scores into account when determining the new score:

handle_call({won_point, p1}, _From, State = #{p1:=P1Score,p2:=P2Score}) ->
    {NewP1Score, NewP2Score} = new_score(P1Score, P2Score),
    NewState = State#{p1:=NewP1Score, p2:=NewP2Score},
    {reply, NewState, NewState};

...

new_score(love, Score) -> {15, Score};
new_score(15, Score) -> {30, Score};
new_score(30, 40) -> {deuce, deuce};
new_score(30, Score) -> {40, Score}.

Advantage

advantage_p1(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>advantage,p2=>40}, won_point(Pid, p1))
    end.

Another case to handle:

...
new_score(deuce, deuce) -> {advantage, 40}.

Back to Deuce

advantage_p1_back_to_deuce(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p2),
        won_point(Pid, p1),
        ?assertEqual(#{p1=>deuce,p2=>deuce}, won_point(Pid, p2))
    end.

Now we need to add a guard to the win case:

handle_call({won_point, p2}, _From, State = #{p1:=P1Score, p2:=40}) when P1Score =/= advantage ->
    {stop, normal, {game_over, p2}, State};

...

new_score(40, advantage) -> {deuce, deuce}.

Advantage, then win

advantage_p1_then_win(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p2),
        won_point(Pid, p1),
        ?assertEqual({game_over, p1}, won_point(Pid, p1))
    end.

Finally, we need to allow a win from Advantage:

handle_call({won_point, p1}, _From, State = #{p1:=advantage}) ->
    {stop, normal, {game_over, p1}, State};

I think that’s everything covered. As Mark says, “it’s easy enough that it’s fun to play with, but difficult enough that it’s fun to play with”. I personally feel that the use of pattern matching made the intent of the code clearer, but of course YMMV.

Buffered channels in Erlang

Golang has the handy concept of a buffered channel (a similar concept to .NET’s blocking collections). It allows you to throttle your code, by blocking when adding an item to a collection that is already full.

Erlang doesn’t offer anything similar, out of the box (mailboxes will happily increase in size until the VM topples over), but it’s pretty easy to implement using a gen_server. You only need to handle 2 messages, adding an item:

init([Limit]) ->
    State = #{
        limit => Limit,
        count => 0,
        add_q => queue:new(),
        get_q => queue:new()
    },
    {ok, State}.

handle_call(add, From, State = #{count:=Count, limit:=Limit, add_q:=AddQ, get_q:=GetQ}) ->
    UpdatedCount = Count + 1,
    case UpdatedCount > Limit of
        false ->
            case queue:is_empty(GetQ) of
                true ->
                    {reply, ok, State#{count => UpdatedCount}};
                _ ->
                    {{value, Client}, UpdatedQ} = queue:out(GetQ),
                    gen_server:reply(Client, ok),
                    {reply, ok, State#{get_q => UpdatedQ}}
            end;
        _ ->
            {noreply, State#{add_q => queue:in(From, AddQ)}}
    end;

The process is initialised with a limit, the number of items that can be added to the queue before it will block. This collection is FIFO, but another data structure, e.g. a stack, could be used.

When an add message is received, the current size of the collection is checked. If adding another item would breach the limit, then the pid of the caller is added to the wait queue. Otherwise, the number of items is incremented (this version only keeps a count of items, rather than an actual list, but adding that would be trivial).

If any process is blocked waiting to get an item from the collection, then it is unblocked and an item removed immediately. We also need to handle a get message:

handle_call(get, From, State = #{count:=Count, name:=_Name, add_q:=AddQ, get_q:=GetQ}) ->
    case Count > 0 of
        true ->
            case queue:is_empty(AddQ) of
                true ->
                    UpdatedCount = Count - 1,
                    {reply, ok, State#{count => UpdatedCount}};
                _ ->
                    {Client, UpdatedQ} = queue:out(AddQ),
                    gen_server:reply(Client, ok),
                    {reply, ok, State#{add_q => UpdatedQ}}
            end;
        _ ->
            {noreply, State#{get_q => queue:in(From, GetQ)}}
    end;

which is pretty much the opposite, check if the collection is empty. Block if it is, or return an element to the caller.

Once your collection is up & running (and supervised!), using it is as simple as calling any other gen_server:

ok = gen_server:call(queue_1, get, infinity),
do_something(),
ok = gen_server:call(queue_2, add, infinity),

I’ve set the timeout to infinity, but that might not be a good idea for production code.

It would also be pretty simple to back the collection with disk storage, e.g. using a DETS table.

Automatic bidding

Moving on from the excitement of tic-tac-toe, I wanted to try my hand at something a little more complex. I decided to have a look at implementing ebay style “automatic bidding“, which turned out to be a lot more interesting than I expected.

It’s a good exercise because it’s realistic, the behaviour is well documented, and there are plenty of examples. I chose to implement it as a gen_server, but it would work in any language.

I started out by just implementing basic bidding with a starting price, if the bid is higher than the current bid then it is accepted:

-record(state, {starting_price, bids=[]}).
-record(bid, {bidder, amount, time}).

-type money() :: {integer(), integer()}.
-spec start_link([money()]) -> {ok, pid()}.
start_link(StartingPrice) ->
    gen_server:start_link(?MODULE, [StartingPrice], []).

init([StartingPrice]) ->
    {ok, #state{starting_price = StartingPrice}}.

handle_call({bid, _UserId, Bid}, _From, State) when Bid < State#state.starting_price ->
    {reply, bid_too_low, State};

handle_call({bid, UserId, BidAmount}, _From, State) when State#state.bids =:= [] ->
    Bid = #bid{bidder = UserId, amount = BidAmount, time = calendar:universal_time()},
    NewState = State#state{bids = [Bid | State#state.bids]},
    {reply, bid_accepted, NewState};

handle_call({bid, UserId, BidAmount}, _From, State) ->
    CurrentHighBid = hd(State#state.bids),
    case BidAmount > CurrentHighBid#bid.amount of
        true ->
            Bid = #bid{bidder = UserId, amount = BidAmount, time = calendar:universal_time()},
            NewState = State#state{bids = [Bid | State#state.bids]},
            {reply, bid_accepted, NewState};
        false ->
            {reply, bid_too_low, State}
    end;

handle_call(list_bids, _From, State) ->
    {reply, State#state.bids, State};

I decided to handle currency as a tuple of pounds & pennies, due to the way Erlang term comparisons work they are ordered correctly.

With this in place, I started moving towards automatic bidding (the commit history is available, but I’ll skip ahead). The algorithm is fairly simple, but as always the devil is in the details!

-type user_id() :: integer().
-record(bid, {bidder :: user_id(), amount :: eb_money:money(), time :: calendar:datetime(), automatic = false :: boolean()}).
-record(state, {starting_price :: eb_money:money(), bids = [] :: [#bid{}], high_bid :: #bid{}}).

-spec start_link(eb_money:money(), calendar:datetime()) -> {ok, pid()}.
start_link(StartingPrice, EndTime) ->
    gen_server:start_link(?MODULE, [StartingPrice, EndTime], []).

init([StartingPrice, EndTime]) ->
    lager:info("Bidding begins. Starting price: ~p", [StartingPrice]),
    TimeRemaining = calculate_time_remaining(EndTime),
    lager:info("Time remaining: ~p", [TimeRemaining]),
    _TimerRef = erlang:send_after(TimeRemaining, self(), bidding_ends),
    {ok, #state{starting_price = StartingPrice}}.

handle_call({bid, _UserId, MaxBid}, _From, State) when MaxBid < State#state.starting_price ->
    {reply, bid_too_low, State};

handle_call({bid, UserId, MaxBid}, _From, State) when State#state.bids =:= [] ->
    BidTime = calendar:universal_time(),
    Bid = #bid{bidder = UserId, amount = State#state.starting_price, time = BidTime},
    NewState = State#state{bids = [Bid | State#state.bids], high_bid = #bid{bidder = UserId, amount = MaxBid, time = BidTime}},
    {reply, bid_accepted, NewState};

handle_call({bid, UserId, MaxBid}, _From, State) ->
    HighBid = State#state.high_bid,
    case HighBid#bid.bidder =:= UserId of
        true ->
            update_bid_for_high_bidder(State, UserId, MaxBid);
        false ->
            handle_new_bid(State, MaxBid, UserId)
    end;

handle_call(list_bids, _From, State) ->
    {reply, lists:reverse(State#state.bids), State};

handle_info(bidding_ends, State) when State#state.bids =:= [] ->
    lager:info("Bidding ends", []),
    lager:info("No winner", []),
    {stop, normal, State};

handle_info(bidding_ends, State) ->
    lager:info("Bidding ends", []),
    WinningBid = hd(State#state.bids),
    lager:info("Winner: ~p, Bid: ~p", [WinningBid#bid.bidder, WinningBid#bid.amount]),
    {stop, normal, State};

update_bid_for_high_bidder(State, UserId, MaxBid) ->
    HighBid = State#state.high_bid,
    case MaxBid > HighBid#bid.amount of
        true ->
            NewState = State#state{high_bid = #bid{bidder = UserId, amount = MaxBid, time = calendar:universal_time()}},
            {reply, bid_accepted, NewState};
        false ->
            {reply, bid_too_low, State}
    end.

handle_new_bid(State, MaxBid, UserId) ->
    CurrentBid = hd(State#state.bids),
    case MaxBid =< CurrentBid#bid.amount of
        true ->
            {reply, bid_too_low, State};
        false ->
            handle_valid_new_bid(State, MaxBid, UserId)
    end.

handle_valid_new_bid(State, MaxBid, UserId) ->
    HighBid = State#state.high_bid,
    BidTime = calendar:universal_time(),
    case MaxBid =< HighBid#bid.amount of
        true ->
            bid_lower_than_or_equal_to_high_bid(State, MaxBid, UserId, BidTime, HighBid);
        false ->
            bid_higher_than_current_high_bid(State, MaxBid, UserId, BidTime, HighBid)
    end.

bid_lower_than_or_equal_to_high_bid(State, MaxBid, UserId, BidTime, HighBid) ->
    LosingBid = #bid{bidder=UserId, amount=MaxBid, time=BidTime},
    update_bid_list(State, LosingBid, HighBid).

bid_higher_than_current_high_bid(State, MaxBid, UserId, BidTime, HighBid) ->
    WinningBid = #bid{bidder=UserId, amount=MaxBid, time=BidTime},
    update_bid_list(State, HighBid, WinningBid).

update_bid_list(State, LosingBid, WinningBid) ->
    NewAmount = eb_money:add(LosingBid#bid.amount, eb_bid_increments:get(LosingBid#bid.amount)),
    NewHighBid = case NewAmount < WinningBid#bid.amount of
        true ->
            #bid{bidder = WinningBid#bid.bidder, amount = NewAmount, time = WinningBid#bid.time, automatic = true};
        false ->
            #bid{bidder = WinningBid#bid.bidder, amount = WinningBid#bid.amount, time = WinningBid#bid.time, automatic = false}
    end,
    NewState = State#state{bids = [NewHighBid | [LosingBid | State#state.bids]]},
    {reply, bid_accepted, NewState}.

calculate_time_remaining(EndTime) ->
    Now = calendar:universal_time(),
    (calendar:datetime_to_gregorian_seconds(EndTime) - calendar:datetime_to_gregorian_seconds(Now)) * 1000. %% milliseconds!

Once you get used to the convenience of features like tuples and pattern matching, it can be quite painful to go back to a language without them. The next step was reserve price auctions, and some day I might get round to implementing the lowering of them.

If you want to play along, the tests can be found here.

Handling a POST with Cowboy

It’s not immediately obvious how to handle different HTTP methods using Cowboy.
This SO answer, and one of the examples show the way:

handle(Req, State) ->
	{Method, Req2} = cowboy_req:method(Req),
	{ok, Req3} = process_req(Method, Req2),
	{ok, Req3, State}.

process_req(<<"POST">>, Req) ->
	{ok, Body, Req2} = cowboy_req:body_qs(Req),
	_Foo = proplists:get_value(<<"foo">>, Body),
	cowboy_req:reply(200, [
		{<<"content-type">>, <<"text/plain; charset=utf-8">>}
	], <<"ohai">>, Req);

process_req(_, _, Req) ->
	%% Method not allowed.
	cowboy_req:reply(405, Req).

Basic auth with Hackney

Hackney is an Erlang http client library. The homepage states that it supports basic authentication, but documentation of this feature is a little light.

The only evidence of it’s existence seems to be this test, but usage is pretty simple:

Url = <<"http://localhost:8000/basic-auth/username/password">>,
Options = [{basic_auth, {<<"username">>, <<"password">>}}],
{ok, StatusCode, _, _} = hackney:request(get, URL, [], <<>>, Options)

Using erlydtl with Cowboy

Erlydtl seems to be the most popular Erlang templating library, and using it with Cowboy is fairly simple; but doesn’t seem to be terribly well documented.

As always, I’m assuming you’re using erlang.mk and know how to set up a Cowboy project. First, you need to add erlydtl as a dependency in your Makefile:

PROJECT = cowboy_stormpath
DEPS = cowboy erlydtl
include erlang.mk

and as a dependency in your .app.src:

{application, cowboy_erlydtl, [
    {description, ""},
    {vsn, "0.1.0"},
    {id, "git"},
    {modules, []},
    {registered, []},
    {applications, [
        kernel,
        stdlib,
        cowboy,
        erlydtl
    ]},
    {mod, {cowboy_erlydtl_app, []}},
    {env, []}
]}.

Next, create a folder called “templates” in your project root, any .dtl files in here will be compiled when you run “make app”:

<html><body>Your favourite Smurf is {{ smurf_name }}.</body></html>

If you look in ebin/ you should see a file named smurfin_dtl.beam (or whatever), this is the compiled version of your template. Finally, you need a handler to render the template:

-module(smurf_handler).
-behaviour(cowboy_http_handler).

-export([init/3]).
-export([handle/2]).
-export([terminate/3]).

-record(state, {}).

init(_, Req, _Opts) ->
    {ok, Req, #state{}}.

handle(Req, State=#state{}) ->
    {ok, Body} = smurfin_dtl:render([{smurf_name, "Smurfette"}]),
    {ok, Req2} = cowboy_req:reply(200, [{<<"content-type">>, <<"text/html">>}], Body, Req),
    {ok, Req2, State}.

terminate(_Reason, _Req, _State) ->
    ok.

and add a route:

    Dispatch = cowboy_router:compile([
        {'_', [
            {"/smurfin", smurf_handler, []}
        ]}
    ]),

et voila, you’ve rendered your first template!

Testing a gen_server with eunit

Of all the strange corners of Erlang, eunit can be one of the hardest to get your head around. Particularly if you’re used to one of the XUnit frameworks.

It is very powerful, especially once you get into generating tests; but it can also be very confusing, and sometimes the error messages are not that helpful.

I wanted to write some tests for a gen_server, using a new instance for each test, and it took me a couple of attempts to get what I wanted working. It’s very important to make sure your test can actually fail, either in advance or by mutating it, or to add some output (?debugMsg) that proves it ran.

-module(foo_server_tests).

-include_lib("eunit/include/eunit.hrl").

foo_server_test_() ->
    {foreach, fun setup/0, fun cleanup/1, [
        fun(Pid) -> fun() -> server_is_alive(Pid) end end,
        fun(Pid) -> fun() -> something_else(Pid) end end
    ]}.

setup() ->
    ?debugMsg("setup"),
    process_flag(trap_exit, true),
    {ok, Pid} = foo_server:start_link(),
    Pid.

server_is_alive(Pid) ->
    ?assertEqual(true, is_process_alive(Pid)).

something_else(Pid) ->
    ?assertEqual(bar, gen_server:call(Pid, foo)).

cleanup(Pid) ->
    ?debugMsg("cleanup"),
    exit(Pid, kill), %% brutal kill!
    ?assertEqual(false, is_process_alive(Pid)).

We need to trap exits in the setup fun, as the process is linked to the gen_server when it starts. The tricky bit is the nested funs in the generator, if you try to call the “test” directly:

foo_server_test_() ->
    {foreach, fun setup/0, fun cleanup/1, [
        fun(Pid) -> server_is_alive(Pid) end
    ]}.

you’ll get an error like this:

*** result from instantiator foo_server_tests:'-foo_server_test_/0-fun-2-'/1 is not a test ***

because eunit is trying to use the output of the called fun as a test. And if you return multiple tests from one block:

foo_server_test_() ->
    {foreach, fun setup/0, fun cleanup/1, [
        fun(Pid) -> [
             fun() -> server_is_alive(Pid) end,
             fun() -> something_else(Pid) end
        ] end
    ]}.

then the setup & teardown will only be called once for those tests (which may, or may not, be what you want).

UPDATE: actually this is a better approach:

-module(foo_server_tests).

-include_lib("eunit/include/eunit.hrl").

foo_server_test_() ->
    {foreach, fun setup/0, fun cleanup/1, [
        fun server_is_alive/1,
        fun something_else/1
    ]}.

server_is_alive(Pid) ->
    fun() ->
        ?assertEqual(true, is_process_alive(Pid))
    end.

something_else(Pid) ->
    fun() ->
        ?assertEqual(bar, gen_server:call(Pid, foo))
    end.

as the test names are included in the output:

$ SKIP_DEPS=true make eunit
 APP    foo.app.src
 GEN    test-dir
 GEN    eunit
======================== EUnit ========================
directory "ebin"
  module 'foo_server'
    module 'foo_server_tests'
      foo_server_tests: server_is_alive...ok
      foo_server_tests: something_else...[0.001 s] ok
      [done in 0.012 s]
    [done in 0.012 s]
  [done in 0.028 s]
=======================================================
  All 2 tests passed.

UPDATE 2: in case it wasn’t clear, this is effectively an “integration” test or black box test. The server is started, and messages are passed to it. It is also possible to “unit” test individual methods, if you don’t mind being more tightly coupled to the internal state of the server (white box). Both kinds of testing can provide value, pick whichever works for you for that scenario.