28 Days - Sourcing Libraries from Private Github

Hex is currently working on private organizations which are in beta. Until the full details of those are public and available, you may find yourself needing to source Elixir libraries privately. You may also just desire a quick fix and not setting up private packages. Enter git sourced packages.

Mix and Git

At the simplest form, you can think of git sourced dependencies as a clone of the provided repo using system git. In fact, the implementation is a shell out to git. This means any repo which a system can access will be accessible via git, including repos which are locked behind private access.

I have seen tags work as a good way to mark the versions in the git repo. The form looks something like:

{:my_dependency_name, git: "[email protected]:Organization/repo.git", tag: "v1.0.0"},

The above mix.exs entry will fetch the mix package named :my_dependency_name from [email protected]:Organization/repo.git, commit tagged v1.0.0. Github SSH access is used due to the repo being [email protected] format.

Trickery with Docker and CI

This all works well locally. In fact, just following the standard tutorial will lead to a working private dependency, locally. Things get a bit trickier when something like Docker or a build server is used to fetch your dependency. In this situation, the SSH key of each system would need to be included on Github. Another difficulty is that Docker containers have to have SSH keys added to them, they don’t come with them standard. All of this leads to another possible solution: accessing Github via API keys.

API Key Setup

Github personal access tokens can be used to access certain parts of Github, such as repository access. In order to use one with mix, just switch out the git url from [email protected]:Organization/repo.git to https://#{access_token}@github.com/Organization/repo. With this solution, any access token which has read access to the repo will serve as the access into Git.

There does exist a drawback with this solution, the access token would need to be provided to every user of the application. This is non-desirable as the access token should really be kept secret, even to the point of being encrypted docker arguments. With a small tweak, it is possible to achieve the best of both worlds, SSH locally but access tokens for Docker.

defp deps do
  [
    {:phoenix, "~> 1.3.0"},
  ] ++ private_deps(System.get_env("MIX_GITHUB_ACCESS_TOKEN"))
end

@my_dependency_version "v1.0.0"

defp private_deps(nil) do
  [
    {:my_dependency_name, git: "[email protected]:Organization/repo.git", tag: @my_dependency_version},
  ]
end

defp private_deps(access_token) do
  [
    {:my_dependency_name, git: "https://#{access_token}@github.com/Organization/repo", tag: @my_dependency_version},
  ]
end

This script will append the private dependencies into the deps list, but do so with either SSH or access token access. Dependency versions should always be constant, so it is pulled into an attribute.

One caveat with this approach is that each mix invocation must contain the ENV, not just deps.get. This is due to the mix.lock being checked when mix runs, and the correct dependencies are pulled each time through the above code path.

With the presented approach, there should be little trouble getting private repos both locally and in CI, without compromise on security of your keys. If the straight access token route is taken, take extra care regarding the permissions of the account behind the key.

Thanks for reading the fourth post in my 28 days of Elixir. Keep up through the month of February to see if I can stand subjecting myself to 28 days of straight writing. I anticipate the posts to get more and more technical as the time goes on!