Proposal for umbrella projects / apps in Mix

[josevalim]

One common pattern when building Erlang applications is to have an umbrella directory that contains many applications. For example:

apps/
  decoder/
    lib/decoder.ex
    mix.exs
  encoder/
    lib/encoder.ex
    mix.exs
  web/
    lib/dynamo.ex
    mix.exs
mix.exs

This proposal is to outline the details to support this kind of projects in Mix:

1. The umbrella project does not actually have any compilable/running code
2. Most mix commands executed at the umbrella project will just be executed recursively in the children applications. For example, mix test will go to each child and execute its tests. Since the umbrella project does not contain any code, we don't need to worry about tasks having double semantics or supporting options like mix compile --recursive
3. Dependencies in development will be per-application. This allows each application to run on its own without the knowledge of the parent application. I am aware that child applications may have common dependencies in between them and fetching it many times may be tiring. If this actually becomes an issue, it should be solved by having a local cache (~/.mix) and work transparently regardless if it is an umbrella project or not
4. A handful of tasks may actually run in the umbrella application and not in the children ones. Tasks that come to mind are: mix run and release related tasks. Since such tasks are rather the exception than the rule, they will need to be marked as non-recursive explicitly by setting @recursive false
5. In order to configure a mix project to work as an umbrella, all that needs to be set is the :apps_path configuration
6. Differently from Rebar, configuration set in the parent project will not propagate to children. It is a goal to have children applications running and tested in isolation is such is desired.

[yrashk]

I would rather disagree with ideas in (3) on the grounds of building a release and having two applications (for whatever reason) having two versions of the same library fetched in their private deps.

I have mixed feelings about (6)

[josevalim]

Regarding 3, we can ensure you won't have different versions of the same dependency. So I am less concerned about it.

Regarding 6, since mix.exs file is actually executable code, you can create a shared.exs in the umbrella project that all child projects can read and get configurations from. This same technique could be used to share common dependencies.

# shared.exs
defmodule Shared do
  def deps do
    [ ... ]
  end
end

And then:

# mix.exs
Code.require_file "../../../shared.exs", __FILE__

def project do
  [deps: Shared.deps ++ deps]
end

The main point about these options is to have applications being able to run standalone and not depend on the umbrella mix.exs file to being loaded before hand by mix. This makes the applications more sane and decoupled, and mix code gets cleaner too, because we don't need to support things like mix compile --app decoder if you want to run just decoder tests.

[yrashk]

Also regarding (3), things WILL get slow. Believe me, nothing is worse than a slowly compiling project.

[yrashk]

Agree on the shared thing.

[alexrp]

This all makes sense to me, but I do agree with @yrashk regarding dependencies. That should be as fast as possible.

[josevalim]

Closing this in favor of the pull request #1016.