Erlang

❤️ the State Machine

Graham HayLeave a comment

Having recently read a post about using state machines in JS, I decided to try implementing the same logic using the (relatively) new gen_statem.

As ever, the code is available here.

The initial example is a naive representation of a bank account:

account-events-withself

The first step is to fill out the gen_statem boilerplate:

-module(bank_statem).

-behaviour(gen_statem).

-export([init/1, callback_mode/0]).
-export([open/3]).

init([]) ->
    {ok, open, #{balance=>0}}.

callback_mode() -> state_functions.

open({call, From}, get_balance, #{balance:=Balance} = Data) ->
    {keep_state, Data, [{reply, From, Balance}]}.

We return an initial state of open, and a map containing the initial balance of 0.

I also added a get_balance call, so we can inspect the current data (which would normally be referred to as the “state” in a gen_server, confusingly).

The first state transition is to close an open account:

open({call, From}, close, Data) ->
    {next_state, closed, Data, [{reply, From, closed}]};

The function name tells you which state this can be called from (open), and the return value tells you that the state machine would transition to a new state (closed), as well as returning a value (the atom closed) to the caller.

Re-opening an account is pretty similar:

closed({call, From}, reopen, Data) ->
    {next_state, open, Data, [{reply, From, open}]}.

As close is only defined for the open state, and reopen is only defined for the closed state, any inappropriate calls will cause the process to crash.

Deposit & withdraw are a little different:

open({call, From}, {deposit, Amount}, #{balance:=Balance} = Data) when is_number(Amount) andalso Amount > 0 ->
    NewBalance = Balance + Amount,
    {keep_state, Data#{balance:=NewBalance}, [{reply, From, deposit_made}]};

open({call, From}, {withdraw, Amount}, #{balance:=Balance} = Data) when is_number(Amount) andalso (Balance - Amount > 0) ->
    NewBalance = Balance - Amount,
    {keep_state, Data#{balance:=NewBalance}, [{reply, From, withdrawal_made}]}.

In that they retain the current state, but the data (in this case, the balance) is updated. Guards have also been used to validate the arguments; again, any other calls will cause an error.

This will seem pretty familiar, if you’ve ever used gen_fsm. Next time, we’ll look at the new alternative callback mode: handle_event_function.

Using rebar3 with Docker

Graham HayLeave a comment

I prefer to avoid installing dev tools on my laptop. I used to create a separate Vagrant instance for each project, but using Docker should provide a lighter weight alternative.

Assuming you already have the Docker tooling installed, the first thing to do is download the base container:

docker pull erlang

Next, download the latest rebar3 binary:

curl -OL https://s3.amazonaws.com/rebar3/rebar3 && chmod +x ./rebar3

At this point, you’re ready to create the app skeleton:

docker run --name $APP_NAME -it --rm -v ${PWD}:/app -w /app erlang ./rebar3 new app $APP_NAME

Annoyingly, rebar assumes that it should create the folder for the project, so you’ll need to move all the generated files up a level (or put up with an extra subdir). Also, the generated files are owned by root, so you probably want to chown the entire dir.

If you’re going to use any rebar plugins, you probably want to put it’s cache dir somewhere that will survive container restarts. Add this to your rebar.config:

{global_rebar_dir, "/app/.cache"}.

And then you can compile your app:

docker run --name $APP_NAME -it --rm -v ${PWD}:/app -w /app -e REBAR_CACHE_DIR=/app erlang ./rebar3 compile

Tennis Kata in Erlang

Graham HayLeave a comment

I’ve recently been reading ploeh’s series on doing the Tennis Kata in F#. I decided to have a go in Erlang, we might not have a fancy type system but we do have pattern matching and partial functions.

You can find the source code to follow along here.

I decided to implement a “game” as a gen_server (when you have a hammer…), I refer you to a previous post for the pre-amble.

Love all

The first test is to check the initial score:

initial_score(Pid) ->
    fun() ->
        ?assertEqual(#{p1=>love,p2=>love}, gen_server:call(Pid, get_score))
    end.

I decided to return a map, rather than a tuple or record, and to use the atom “love” rather than 0. The implementation is pretty trivial:

init([]) ->
    {ok, #{p1=>love,p2=>love}}.

handle_call(get_score, _From, State) ->
    {reply, State, State};

If there was more information in the state map, you might want to return a projection rather than the whole object.

Fifteen love

fifteen_love(Pid) ->
    fun() ->
        ?assertEqual(#{p1=>15,p2=>love}, gen_server:call(Pid, {won_point, p1}))
    end.

This time we need to calculate the new score when a player has won a point:

handle_call({won_point, p1}, _From, State = #{p1:=CurrentScore}) ->
    NewState = State#{p1:=new_score(CurrentScore)},
    {reply, NewState, NewState};

...

new_score(love) -> 15.

Love fifteen

love_fifteen(Pid) ->
    fun() ->
        ?assertEqual(#{p1=>love,p2=>15}, gen_server:call(Pid, {won_point, p2}))
    end.

We can re-use the new_score method from before:

handle_call({won_point, p2}, _From, State = #{p2:=CurrentScore}) ->
    NewState = State#{p2:=new_score(CurrentScore)},
    {reply, NewState, NewState};

Fifteen all

fifteen_all(Pid) ->
    fun() ->
        won_point(Pid, p1),
        ?assertEqual(#{p1=>15,p2=>15}, won_point(Pid, p2))
    end.

won_point(Pid, Player) ->
    gen_server:call(Pid, {won_point, Player}).

No need for any new code, but it seemed like time for a little test refactoring.

Thirty fifteen

thirty_fifteen(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>30,p2=>15}, won_point(Pid, p1))
    end.

Now we need to add another function head, to handle the new score:

new_score(love) -> 15;
new_score(15) -> 30.

Thirty all

thirty_all(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>30,p2=>30}, won_point(Pid, p1))
    end.

No new code required.

Forty thirty

forty_thirty(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>40,p2=>30}, won_point(Pid, p1))
    end.

Again, we need to handle a new score:

...
new_score(30) -> 40.

P1 win

p1_wins(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p1),
        ?assertEqual({game_over, p1}, won_point(Pid, p1))
    end.

This requires a special case, for when the player has 40 points:

handle_call({won_point, p1}, _From, State = #{p1:=40}) ->
    {stop, normal, {game_over, p1}, State};

But now we have a problem, as I skipped over the troublesome “deuce”.

Deuce

deuce(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>deuce,p2=>deuce}, won_point(Pid, p2))
    end.

Again, I decided to use an atom, rather than 40-40. But now we need to take both player’s scores into account when determining the new score:

handle_call({won_point, p1}, _From, State = #{p1:=P1Score,p2:=P2Score}) ->
    {NewP1Score, NewP2Score} = new_score(P1Score, P2Score),
    NewState = State#{p1:=NewP1Score, p2:=NewP2Score},
    {reply, NewState, NewState};

...

new_score(love, Score) -> {15, Score};
new_score(15, Score) -> {30, Score};
new_score(30, 40) -> {deuce, deuce};
new_score(30, Score) -> {40, Score}.

Advantage

advantage_p1(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p2),
        ?assertEqual(#{p1=>advantage,p2=>40}, won_point(Pid, p1))
    end.

Another case to handle:

...
new_score(deuce, deuce) -> {advantage, 40}.

Back to Deuce

advantage_p1_back_to_deuce(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p2),
        won_point(Pid, p1),
        ?assertEqual(#{p1=>deuce,p2=>deuce}, won_point(Pid, p2))
    end.

Now we need to add a guard to the win case:

handle_call({won_point, p2}, _From, State = #{p1:=P1Score, p2:=40}) when P1Score =/= advantage ->
    {stop, normal, {game_over, p2}, State};

...

new_score(40, advantage) -> {deuce, deuce}.

Advantage, then win

advantage_p1_then_win(Pid) ->
    fun() ->
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p1),
        won_point(Pid, p1),
        won_point(Pid, p2),
        won_point(Pid, p2),
        won_point(Pid, p1),
        ?assertEqual({game_over, p1}, won_point(Pid, p1))
    end.

Finally, we need to allow a win from Advantage:

handle_call({won_point, p1}, _From, State = #{p1:=advantage}) ->
    {stop, normal, {game_over, p1}, State};

I think that’s everything covered. As Mark says, “it’s easy enough that it’s fun to play with, but difficult enough that it’s fun to play with”. I personally feel that the use of pattern matching made the intent of the code clearer, but of course YMMV.

Buffered channels in Erlang

Graham HayLeave a comment

Golang has the handy concept of a buffered channel (a similar concept to .NET’s blocking collections). It allows you to throttle your code, by blocking when adding an item to a collection that is already full.

Erlang doesn’t offer anything similar, out of the box (mailboxes will happily increase in size until the VM topples over), but it’s pretty easy to implement using a gen_server. You only need to handle 2 messages, adding an item:

init([Limit]) ->
    State = #{
        limit => Limit,
        count => 0,
        add_q => queue:new(),
        get_q => queue:new()
    },
    {ok, State}.

handle_call(add, From, State = #{count:=Count, limit:=Limit, add_q:=AddQ, get_q:=GetQ}) ->
    UpdatedCount = Count + 1,
    case UpdatedCount > Limit of
        false ->
            case queue:is_empty(GetQ) of
                true ->
                    {reply, ok, State#{count => UpdatedCount}};
                _ ->
                    {{value, Client}, UpdatedQ} = queue:out(GetQ),
                    gen_server:reply(Client, ok),
                    {reply, ok, State#{get_q => UpdatedQ}}
            end;
        _ ->
            {noreply, State#{add_q => queue:in(From, AddQ)}}
    end;

The process is initialised with a limit, the number of items that can be added to the queue before it will block. This collection is FIFO, but another data structure, e.g. a stack, could be used.

When an add message is received, the current size of the collection is checked. If adding another item would breach the limit, then the pid of the caller is added to the wait queue. Otherwise, the number of items is incremented (this version only keeps a count of items, rather than an actual list, but adding that would be trivial).

If any process is blocked waiting to get an item from the collection, then it is unblocked and an item removed immediately. We also need to handle a get message:

handle_call(get, From, State = #{count:=Count, name:=_Name, add_q:=AddQ, get_q:=GetQ}) ->
    case Count > 0 of
        true ->
            case queue:is_empty(AddQ) of
                true ->
                    UpdatedCount = Count - 1,
                    {reply, ok, State#{count => UpdatedCount}};
                _ ->
                    {Client, UpdatedQ} = queue:out(AddQ),
                    gen_server:reply(Client, ok),
                    {reply, ok, State#{add_q => UpdatedQ}}
            end;
        _ ->
            {noreply, State#{get_q => queue:in(From, GetQ)}}
    end;

which is pretty much the opposite, check if the collection is empty. Block if it is, or return an element to the caller.

Once your collection is up & running (and supervised!), using it is as simple as calling any other gen_server:

ok = gen_server:call(queue_1, get, infinity),
do_something(),
ok = gen_server:call(queue_2, add, infinity),

I’ve set the timeout to infinity, but that might not be a good idea for production code.

It would also be pretty simple to back the collection with disk storage, e.g. using a DETS table.

Automatic bidding

Graham HayLeave a comment

Moving on from the excitement of tic-tac-toe, I wanted to try my hand at something a little more complex. I decided to have a look at implementing ebay style “automatic bidding“, which turned out to be a lot more interesting than I expected.

It’s a good exercise because it’s realistic, the behaviour is well documented, and there are plenty of examples. I chose to implement it as a gen_server, but it would work in any language.

I started out by just implementing basic bidding with a starting price, if the bid is higher than the current bid then it is accepted:

-record(state, {starting_price, bids=[]}).
-record(bid, {bidder, amount, time}).

-type money() :: {integer(), integer()}.
-spec start_link([money()]) -> {ok, pid()}.
start_link(StartingPrice) ->
    gen_server:start_link(?MODULE, [StartingPrice], []).

init([StartingPrice]) ->
    {ok, #state{starting_price = StartingPrice}}.

handle_call({bid, _UserId, Bid}, _From, State) when Bid < State#state.starting_price ->
    {reply, bid_too_low, State};

handle_call({bid, UserId, BidAmount}, _From, State) when State#state.bids =:= [] ->
    Bid = #bid{bidder = UserId, amount = BidAmount, time = calendar:universal_time()},
    NewState = State#state{bids = [Bid | State#state.bids]},
    {reply, bid_accepted, NewState};

handle_call({bid, UserId, BidAmount}, _From, State) ->
    CurrentHighBid = hd(State#state.bids),
    case BidAmount > CurrentHighBid#bid.amount of
        true ->
            Bid = #bid{bidder = UserId, amount = BidAmount, time = calendar:universal_time()},
            NewState = State#state{bids = [Bid | State#state.bids]},
            {reply, bid_accepted, NewState};
        false ->
            {reply, bid_too_low, State}
    end;

handle_call(list_bids, _From, State) ->
    {reply, State#state.bids, State};

I decided to handle currency as a tuple of pounds & pennies, due to the way Erlang term comparisons work they are ordered correctly.

With this in place, I started moving towards automatic bidding (the commit history is available, but I’ll skip ahead). The algorithm is fairly simple, but as always the devil is in the details!

-type user_id() :: integer().
-record(bid, {bidder :: user_id(), amount :: eb_money:money(), time :: calendar:datetime(), automatic = false :: boolean()}).
-record(state, {starting_price :: eb_money:money(), bids = [] :: [#bid{}], high_bid :: #bid{}}).

-spec start_link(eb_money:money(), calendar:datetime()) -> {ok, pid()}.
start_link(StartingPrice, EndTime) ->
    gen_server:start_link(?MODULE, [StartingPrice, EndTime], []).

init([StartingPrice, EndTime]) ->
    lager:info("Bidding begins. Starting price: ~p", [StartingPrice]),
    TimeRemaining = calculate_time_remaining(EndTime),
    lager:info("Time remaining: ~p", [TimeRemaining]),
    _TimerRef = erlang:send_after(TimeRemaining, self(), bidding_ends),
    {ok, #state{starting_price = StartingPrice}}.

handle_call({bid, _UserId, MaxBid}, _From, State) when MaxBid < State#state.starting_price ->
    {reply, bid_too_low, State};

handle_call({bid, UserId, MaxBid}, _From, State) when State#state.bids =:= [] ->
    BidTime = calendar:universal_time(),
    Bid = #bid{bidder = UserId, amount = State#state.starting_price, time = BidTime},
    NewState = State#state{bids = [Bid | State#state.bids], high_bid = #bid{bidder = UserId, amount = MaxBid, time = BidTime}},
    {reply, bid_accepted, NewState};

handle_call({bid, UserId, MaxBid}, _From, State) ->
    HighBid = State#state.high_bid,
    case HighBid#bid.bidder =:= UserId of
        true ->
            update_bid_for_high_bidder(State, UserId, MaxBid);
        false ->
            handle_new_bid(State, MaxBid, UserId)
    end;

handle_call(list_bids, _From, State) ->
    {reply, lists:reverse(State#state.bids), State};

handle_info(bidding_ends, State) when State#state.bids =:= [] ->
    lager:info("Bidding ends", []),
    lager:info("No winner", []),
    {stop, normal, State};

handle_info(bidding_ends, State) ->
    lager:info("Bidding ends", []),
    WinningBid = hd(State#state.bids),
    lager:info("Winner: ~p, Bid: ~p", [WinningBid#bid.bidder, WinningBid#bid.amount]),
    {stop, normal, State};

update_bid_for_high_bidder(State, UserId, MaxBid) ->
    HighBid = State#state.high_bid,
    case MaxBid > HighBid#bid.amount of
        true ->
            NewState = State#state{high_bid = #bid{bidder = UserId, amount = MaxBid, time = calendar:universal_time()}},
            {reply, bid_accepted, NewState};
        false ->
            {reply, bid_too_low, State}
    end.

handle_new_bid(State, MaxBid, UserId) ->
    CurrentBid = hd(State#state.bids),
    case MaxBid =< CurrentBid#bid.amount of
        true ->
            {reply, bid_too_low, State};
        false ->
            handle_valid_new_bid(State, MaxBid, UserId)
    end.

handle_valid_new_bid(State, MaxBid, UserId) ->
    HighBid = State#state.high_bid,
    BidTime = calendar:universal_time(),
    case MaxBid =< HighBid#bid.amount of
        true ->
            bid_lower_than_or_equal_to_high_bid(State, MaxBid, UserId, BidTime, HighBid);
        false ->
            bid_higher_than_current_high_bid(State, MaxBid, UserId, BidTime, HighBid)
    end.

bid_lower_than_or_equal_to_high_bid(State, MaxBid, UserId, BidTime, HighBid) ->
    LosingBid = #bid{bidder=UserId, amount=MaxBid, time=BidTime},
    update_bid_list(State, LosingBid, HighBid).

bid_higher_than_current_high_bid(State, MaxBid, UserId, BidTime, HighBid) ->
    WinningBid = #bid{bidder=UserId, amount=MaxBid, time=BidTime},
    update_bid_list(State, HighBid, WinningBid).

update_bid_list(State, LosingBid, WinningBid) ->
    NewAmount = eb_money:add(LosingBid#bid.amount, eb_bid_increments:get(LosingBid#bid.amount)),
    NewHighBid = case NewAmount < WinningBid#bid.amount of
        true ->
            #bid{bidder = WinningBid#bid.bidder, amount = NewAmount, time = WinningBid#bid.time, automatic = true};
        false ->
            #bid{bidder = WinningBid#bid.bidder, amount = WinningBid#bid.amount, time = WinningBid#bid.time, automatic = false}
    end,
    NewState = State#state{bids = [NewHighBid | [LosingBid | State#state.bids]]},
    {reply, bid_accepted, NewState}.

calculate_time_remaining(EndTime) ->
    Now = calendar:universal_time(),
    (calendar:datetime_to_gregorian_seconds(EndTime) - calendar:datetime_to_gregorian_seconds(Now)) * 1000. %% milliseconds!

Once you get used to the convenience of features like tuples and pattern matching, it can be quite painful to go back to a language without them. The next step was reserve price auctions, and some day I might get round to implementing the lowering of them.

If you want to play along, the tests can be found here.

Handling a POST with Cowboy

Graham Hay1 Comment

It’s not immediately obvious how to handle different HTTP methods using Cowboy.
This SO answer, and one of the examples show the way:

handle(Req, State) ->
	{Method, Req2} = cowboy_req:method(Req),
	{ok, Req3} = process_req(Method, Req2),
	{ok, Req3, State}.

process_req(<<"POST">>, Req) ->
	{ok, Body, Req2} = cowboy_req:body_qs(Req),
	_Foo = proplists:get_value(<<"foo">>, Body),
	cowboy_req:reply(200, [
		{<<"content-type">>, <<"text/plain; charset=utf-8">>}
	], <<"ohai">>, Req);

process_req(_, _, Req) ->
	%% Method not allowed.
	cowboy_req:reply(405, Req).

Basic auth with Hackney

Graham HayLeave a comment

Hackney is an Erlang http client library. The homepage states that it supports basic authentication, but documentation of this feature is a little light.

The only evidence of it’s existence seems to be this test, but usage is pretty simple:

Url = <<"http://localhost:8000/basic-auth/username/password">>,
Options = [{basic_auth, {<<"username">>, <<"password">>}}],
{ok, StatusCode, _, _} = hackney:request(get, URL, [], <<>>, Options)