Stok Footage

Continually experimenting with new ideas and techniques — Reconstructing, Developing, Modernising.

A Young Person’s Guide to Dialyzer — Fixing a Straightforward Problem

Sometimes a project’s external dependencies change their interface, and if our code either isn’t tested properly or has paths which are never run then you have a problem (or three 😉.) This post will show how dialyzer can help reveal the problem, how you choose to address the problem is up to you and the context you’re in. This example was culled from a production application with reasonable test coverage, but where dialyzer hadn’t been run for a while. dialyzer detected an error which has never been triggered in production and wasn’t exposed by the test suite. It shows how dialyzer can bring the benefits of static analysis to an existing code base.

The code which demonstrates the problem can be found in https://github.com/mikestok/a-young-persons-guide-to-dialyzer. To get to the “right place” to play along you can do the following:

git clone https://github.com/mikestok/a-young-persons-guide-to-dialyzer
cd a-young-persons-guide-to-dialyzer/straightforward
git co straightforward-with-problem
mix deps.get
mix test

After everything has been downloaded and compiled you should eventually see something like:

.

Finished in 0.02 seconds
1 test, 0 failures

Randomized with seed 129223

Seeing the Problem

This example is free of surrounding context, so it’s pretty simple to see what the problem is. But imagine yourself arriving at a post-IPO start-up to maintain the code base, and all the original developers have cashed in their shares and are now travelling the world and not checking slack! How are you going to give yourself some measure of confidence that the code is in good shape? The tests all seem to run, but what about code paths which haven’t been tested?

Here’s the code in lib/straightforward.ex:

1
2
3
4
5
6
7
defmodule Straightforward do
  def string_to_date(date_string) do
    date_string
    |> Timex.parse("{YYYY}-{0M}-{D}")
    |> DateTime.to_date()
  end
end

To see the problem in action we can use iex to explore. First we try to change the string "banana" into a Date, and get an expected, if not gracefully handled error. Then we try a much more reasonable looking string "2017-07-09" and it fails.

$ iex -S mix
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false]

Interactive Elixir (1.4.5) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Straightforward.string_to_date("banana")
** (FunctionClauseError) no function clause matching in DateTime.to_date/1
    (elixir) lib/calendar.ex:1587: DateTime.to_date({:error, "Expected `1-4 digit year` at line 1, column 1."})
iex(1)> Straightforward.string_to_date("2017-07-09")
** (FunctionClauseError) no function clause matching in DateTime.to_date/1
    (elixir) lib/calendar.ex:1587: DateTime.to_date({:ok, ~N[2017-07-09 00:00:00]})

Using mix dialyzer as a Diagnostic Tool

dialyzer uses a persistent lookup table (PLT) to store information about functions, and it will usually generate this the first time you run dialyzer. The generation of the PLT can take some minutes, and to illustrate this let’s perform it as a separate step and then all subsequent runs of mix dialyzer will be faster.

To get dialyzer set up the PLT and then stop you can run mix dialyzer --plt, then we can check the code:

$ time mix dialyzer --plt
Checking PLT...
[:asn1, :certifi, :combine, :compiler, :crypto, :elixir, :gettext, :hackney,
 :idna, :kernel, :logger, :metrics, :mimerl, :public_key, :ssl, :ssl_verify_fun,
 :stdlib, :timex, :tzdata, :unicode_util_compat]
Finding suitable PLTs
Looking up modules in dialyxir_erlang-20.0_elixir-1.4.5_deps-dev.plt
Finding applications for dialyxir_erlang-20.0_elixir-1.4.5_deps-dev.plt
Finding modules for dialyxir_erlang-20.0_elixir-1.4.5_deps-dev.plt
Checking 383 modules in dialyxir_erlang-20.0_elixir-1.4.5_deps-dev.plt
Adding 268 modules to dialyxir_erlang-20.0_elixir-1.4.5_deps-dev.plt

real    3m36.929s
user    10m14.977s
sys 0m42.806s
$ time mix dialyzer
Checking PLT...
[:asn1, :certifi, :combine, :compiler, :crypto, :elixir, :gettext, :hackney,
 :idna, :kernel, :logger, :metrics, :mimerl, :public_key, :ssl, :ssl_verify_fun,
 :stdlib, :timex, :tzdata, :unicode_util_compat]
PLT is up to date!
Starting Dialyzer
dialyzer args: [check_plt: false,
 init_plt: '/Users/mike/tmp/a-young-persons-guide-to-dialyzer/straightforward/_build/dev/dialyxir_erlang-20.0_elixir-1.4.5_deps-dev.plt',
 files_rec: ['/Users/mike/tmp/a-young-persons-guide-to-dialyzer/straightforward/_build/dev/lib/straightforward/ebin'],
 warnings: [:unknown]]
done in 0m2.0s
lib/straightforward.ex:2: Function string_to_date/1 has no local return
lib/straightforward.ex:5: The call 'Elixir.DateTime':to_date({'error',_} | {'ok',#{'__struct__':='Elixir.DateTime' | 'Elixir.NaiveDateTime', 'calendar':=atom(), 'day':=integer(), 'hour':=byte(), 'microsecond':={char(),0 | 1 | 2 | 3 | 4 | 5 | 6}, 'minute':=byte(), 'month':=integer(), 'second':=byte(), 'year':=integer(), 'std_offset'=>integer(), 'time_zone'=>binary(), 'utc_offset'=>integer(), 'zone_abbr'=>binary()}}) will never return since it differs in the 1st argument from the success typing arguments: (#{'__struct__':='Elixir.DateTime', 'calendar':=_, 'day':=_, 'month':=_, 'year':=_, _=>_})
done (warnings were emitted)

real    0m2.751s
user    0m2.429s
sys 0m0.632s

I discount the warning for lib/straightforward.ex:2 because there are more useful warnings in the function body I can deal with (see How to Ignore Some Errors.)

The way I interpret the line describing the errors for lib/straightforward.ex:5 is that 'Elixir.DateTime':to_date (Erlang’s name for Elixir’s DateTime.to_date) will get a wrong first argument. Using Elixir notation, the function’s success typing of its first argument is %DateTime{}, but dialyzer’s analysis shows that it will get one of {:error, _} or {:ok, %DateTime{} | %NaiveDateTime{}}.

What to do?

Looking at the suspect code we can see that Timex.parse/2 is providing the first argument to DateTime.to_date/1, so we can go and look at the documentation for Timex.parse/2 on hex.pm. It returns tuples, and I notice that DateTime.to_date/1 will not handle a %NaiveDateTime{}.

The tuple problem can be got around using Timex.parse!/2, we’ll get an error if the supplied string can’t be parsed and it will just return a %NaiveDateTime{} or a %DateTime{} which can be passed into the next function.

As I couldn’t tell easily what type to expect back from Timex.parse!/2 I decided to use Timex.to_date/1 which takes a Timex.Types.valid_datetime as an argument and turns it into a %Date{}.

With the code changed and some tests added both mix test and mix dialyzer pass happily. You can look at the new code by using git co straightforward-solved in your clone of the repository. The code ended up looking like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
defmodule Straightforward do
  @doc """
  Parse a string containing a date (yyyy-mm-dd) and return a Date if it is
  valid. Raise an error if it is not valid.

      iex> Straightforward.string_to_date("2017-07-09")
      ~D[2017-07-09]
  """
  @spec string_to_date(String.t) :: Date.t | no_return
  def string_to_date(date_string) when is_binary(date_string) do
    date_string
    |> Timex.parse!("{YYYY}-{0M}-{D}")
    |> Timex.to_date()
  end
end

Postscript

I was going to add a test to see if "2015-02-29" caused a problem. It didn’t, and I’ve rambled on enough for now…

Tags: ,

Leave a Reply

Your email address will not be published. Required fields are marked *