Useful Elixir Patterns from Real-world Side Project

I believe that one of the best ways to push new practices is to work on a real-world project that we can afford to experiment on. We can push the boundary in the toy project while still seeing the results of the decisions in a production environment with real users. I’ve been fortunate to be able to ship several applications like this at SalesLoft. The latest one that we’ll look at today is our internal OKR (Objective-Key Result) app that we use for goal tracking and alignment.

We’ll walk through this project and see several different patterns that I like. Some of these patterns have been used in several different projects of mine and have held up well. Others were new experiments that I hope to use again in the future.

The source code for the application can be found on Github. The useful part is the code and tests, less so the specific functionality.

Context Usage

Contexts, at least as I use them, are all about establishing an application domain and keeping the interfaces small and useful for other parts of the system to use. I’ve done this in the past but really pushed it for this project by having no function usage other than top level modules. This pattern allows us to have a well-defined interface for the application, which makes it simpler for both us and others to modify the code and understand the consequences.

As an example, the following would not be acceptable:

MyApp.SomeContext.ASpecificQuery.execute(params)

Instead, I would prefer this:

MyApp.SomeContext.a_query(params)

All of my contexts live under the OkrApp module. You can see a list of them in the okr_app folder.

I started off the application by writing code directly in the context. Some of this is still seen because I didn’t go back through and change everything once I solidified on what I wanted. In the end, I found that function delegates allowed the context to be very easy to digest and I could write tests for the underlying modules in distinct files.

The hardest part of this pattern, that I haven’t figured out yet, is how to handle Ecto schemas between contexts. Sometimes, a context needs to leak out. For example, a User schema may need to be referenced by the AnalyticsEvent schema (as seen here). I found this acceptable because I wasn’t invoking logic outside of the context interface.

The advantage of Context came out when I built the mailer. I found it very simple to add in this new code without worrying about anything breaking. I was also to extract concepts specific to the mailer (such as Recipient) rather than relying on generic modules prone to change such as User.

Isolated Logic (Context) for SCIM

One of the requirements for this project was SCIM (System for Cross-domain Identity Management). We use Okta internally and they have a lot of docs on how to write a SCIM integration. I didn’t want the details of SCIM leaking into the system too heavily. I was able to leverage a SCIM Context to achieve this goal in a way that I am pretty satisfied with.

SCIM has all of its web functionality extracted into a module called Scim.Web. Phoenix.Router is used to provide the appropriate endpoint definitions, but the integration happens in a simple Plug which is used by OkrAppWeb.Router.

The entire SCIM controller leverages a behavior passed into it. This is the strategy pattern at work. I didn’t create a specific Behaviour requirement for this module but probably should have, because it has an expected interface.

The actual SCIM application integration happens in UsersScim. This code is a bit ugly since I didn’t clean it up too much, but it’s nice that the implementation doesn’t leak into my application API nor does it leak into other contexts.

ListQuery Module

I really like Ecto’s API for querying data, but I have found it cumbersome to build up dynamic queries from API params easily. I wrote a ListQuery some time ago that I brought into this project.

The ListQuery is used to take provided params and opts and turns it into an Ecto compatible query. Sometimes advanced queries are required and I devised a small way to do that by stripping certain params and then reapplying them. You can see that here.

I’m able to leverage a ListQuery powered function in various controllers by passing in the user params. If you use this pattern in an environment with non-friendly users, you should probably sanitize the input to not leak any information to the client.

SimpleEctoStore Module

My contexts use defdelegate to send queries to a different module. I called the module a “store” and found myself delegating to them often. Writing the same code again and again in the store became a bit cumbersome. Often, I would be writing the most simple code possible with just the module name changed.

My solution to this copy/paste problem was to build a macro powered SimpleEctoStore. Each usage of SimpleEctoStore involved passing in the schema module as well as what methods were desired to be pulled in. You can see this here.

This allowed me to remove the boiler plate for a large number of ecto stores. If I used a concept like stores again, I would definitely repeat this one.

Things that didn’t work as well

One of the biggest pain points I had was defining how I wanted my JSON API to work. At work, I follow the philosophy of very focused endpoints that don’t preload many models together. This is for performance and flexibility reasons. Doing that in this project made it much more complex for me and was going to take time that I didn’t want to spend. The end result here is that I have a massive endpoint called “Okr” that embeds many associations.

The preload for my Okr endpoint is pretty gnarly as well. However, I was thrilled to be able to pass queries into the preload clauses which means that I removed the risk of accidentally leaking data that is queried out of the system but then not handled properly by me in Elixir.

Another thing that I still feel awkward about is a context’s schema referencing a different context’s schema. I wanted to remove this but couldn’t figure out how to do so in a way I liked.

Summary

Contexts allowed for a focused API between parts of an application. Using them saved me some mental gymnastics as I was developing the application. I was particularly happy with how I was able to isolate the sort of whacky SCIM API via focused context modules.

ListQuery and SimpleEctoStore both made my life easier while defining my ecto based queries and stores. These are generically applicable modules that could be brought into other applications if needed.

I am really happy overall with how this project turned out. It is easy to read through nearly 6 months after I’ve originally built it and has been running without any Elixir issues since that time. I’ll be applying these patterns to future projects, for sure, and hope you are able to get some value from it!

Thanks for reading! I’ll be teaching a class at ElixirConf US about building scalable real-time systems in Elixir. I’ve been focused a lot on this topic so I’m pretty thrilled to share what I’ve learned. I believe that registration will open soon.