28 Days - Exploring Elixir Map access

I missed a day…damn. I did have a great day with my dad before he left to go home, so I’m letting it slide. I have a last minute trip coming up as well, so I’ll be working extra hard over the next few days to come up with some ideas and execute on them while flying over the pond to Spain.

Today, I want to dive deep into a common part of Elixir: map access. This topic is one that I have taken for granted until today, and discovering how it is all handled was quite an interesting adventure. While the end result of today’s post won’t be ground breaking new ideas, it should give some idea on how I approach a new foreign code base.

Digging in for the first time (and failing)

I started this adventure because my co-worker Ben brought up that Access is an interesting module that we take for granted. Even further than Access, the Kernel module has some interesting functions related to access. I have never used these before, but I could see them being pretty useful.

I started my journey by pulling down the elixir-lang/elixir github repo. Having this locally allows me to do some more powerful searches than Github allows, and also works offline. I started with some basic greps for how Access.get is exposed:

➜  elixir git:(master) grep -R "Access\.get(" lib/elixir/lib
lib/elixir/lib/access.ex:  Internally, `data[key]` translates  to `Access.get(term, key, nil)`.
lib/elixir/lib/kernel.ex:  def get_in(data, [h]), do: Access.get(data, h)
lib/elixir/lib/kernel.ex:  def get_in(data, [h | t]), do: get_in(Access.get(data, h), t)

Alright. So nothing new is really here. The Elixir language itself only uses Access.get via Kernel.get_in/2. Okay…so let’s find out how get_in is used. I’ve excluded iex> here as that is for docs and wasn’t useful for the search:

➜  elixir git:(master) grep -R "get_in(" lib/elixir/lib | grep -v "iex"
lib/elixir/lib/kernel.ex:  @spec get_in(Access.t(), nonempty_list(term)) :: term
lib/elixir/lib/kernel.ex:  def get_in(data, keys)
lib/elixir/lib/kernel.ex:  def get_in(data, [h]) when is_function(h), do: h.(:get, data, & &1)
lib/elixir/lib/kernel.ex:  def get_in(data, [h | t]) when is_function(h), do: h.(:get, data, &get_in(&1, t))
lib/elixir/lib/kernel.ex:  def get_in(nil, [_]), do: nil
lib/elixir/lib/kernel.ex:  def get_in(nil, [_ | t]), do: get_in(nil, t)
lib/elixir/lib/kernel.ex:  def get_in(data, [h]), do: Access.get(data, h)
lib/elixir/lib/kernel.ex:  def get_in(data, [h | t]), do: get_in(Access.get(data, h), t)

Alright….so now I’ve hit my dead end. I know that Access.get is what is used for accessing a map like map[:key] as the access.ex file documents as such, but I simply can’t find it.

Start from beginning

This is when I remembered something that I saw Chris McCord write in Slack once, he likes to use Macro.expand to see what happens when macros are processed. This gives the AST that Elixir is going to compile into erlang, so should give a good lead. I’ve formatted the following to read more easily:

iex(1)> Macro.expand((quote do: (map[:a])), __ENV__)
{
  {:., [], [Access, :get]},
  [],
  [{:map, [], Elixir}, :a]
}

Okay, this is what we expected to see but also tells us a lot. When map[:a] is expanded by the Elixir compiler, somehow Access.get is added in. We can see that this also works for map[:a][:b]:

iex(2)> Macro.expand((quote do: (map[:a][:b])), __ENV__)
{
  {:., [], [Access, :get]},
  [],
  [
    {
      {:., [], [Access, :get]},
      [],
      [{:map, [], Elixir}, :a]
    },
    :b
  ]
}

This revealed to me that I missed something crucial in searching the Elixir code: I didn’t search the compiler source code, which lives in lib/elixir/src. Let’s try a search there:

➜  elixir git:(master) grep -R "Access" lib/elixir/src
lib/elixir/src/elixir_erl_pass.erl:translate_remote('Elixir.Access' = Mod, get, Meta, [Container, Value], S) ->
lib/elixir/src/elixir_parser.yrl:%% Access
lib/elixir/src/elixir_parser.yrl:  {{'.', Meta, ['Elixir.Access', get]}, Meta, [Expr, List]}.
lib/elixir/src/elixir_rewrite.erl:-define(access, 'Elixir.Access').

Ahhhh, there we go. The Elixir compiler expands on map access. We can then follow the dots backwards to get how this is codified:

  1. build_access is used in bracket_expr
  2. parser notation for different access_expr
  3. Much more parser notation by looking at each variable

I won’t pretend to know the parser expressions and how they’re put together. I suspect that I could spend a lot of time researching that topic in particular. However, at this point we have our answer: the bracket expressions are handled by the Elixir compiler, and expanded into Access.get calls. They are not handled as normal macros and are engrained into the language.

Thanks for reading the 17th post in my 28 days of Elixir. Keep up through the month of February to see if I can stand subjecting myself to 28 days of straight writing. I am looking for new topics to write about, so please reach out if there’s anything you really want to see!

View other posts tagged: engineering elixir 28 days of elixir