Elixir Comprehensions: Learn Enough to be Dangerous

When you first encounter a for comprehension in Elixir, it looks like a standard loop. But Elixir doesn't have traditional for loops. Instead, comprehensions are powerful pipelines for iterating, filtering, and transforming data all at once.

To master comprehensions, we need to understand their anatomy. Let's look at the basic blueprint, and then scale it up to the advanced, infinite chain.

1. Simple Comprehension (like a simple Enum.map)

The absolute simplest comprehension has one <generator> and an <expression>.

for <generator> do <expression> end

Or, in its single-line form:

for <generator>, do: <expression>

Example:

nums = [1, 2, 3]
for x <- nums do
  x * 10
end
# Result: [10, 20, 30]

Or

for x <- nums, do: x * 10
# Result: [10, 20, 30]

Think of it like a simple Enum.map:

nums = [10, 20, 30]
Enum.map(nums, fn num -> num * 10 end)
# Result: [10, 20, 30]

# Or

Enum.map(nums, & &1 * 10)
# Result: [10, 20, 30]

2. Adding Filters

You can add a <filter> immediately after your <generator> to discard items you don't want. You aren't limited to just one; you can add multiple <filter>s in a row.

for <generator>, <filter>, <filter> do
  <expression>
end

As long as every <filter> evaluates to "truthiness" (anything other than false or nil), the comprehension continues to the next step. If any <filter> evaluates to false or nil, the item is immediately skipped.

🤔 How does Elixir know it's a <filter>?

Elixir figures it out by process of elimination. If an expression in your comma-separated list
(1) doesn't have an <generator> arrow (<-),
(2) isn't a predefined option (like into:, uniq:, or reduce:),
(3) and isn't the do block,
Elixir automatically treats it as a <filter>!

result = for x <- 1..10, rem(x, 2) == 0, x > 5 do
  x * 10
end
# Result: ~c"<Pd"

IO.inspect(result, charlists: :as_lists)
# Result: [60, 80, 100]

If IEx sees a list where every single number is a printable character, it tries to be "helpful" by printing it as a string of characters (prefixed with ~c).
Because 60, 80, and 100 all fall within the "printable" range, Elixir shows you <Pd instead of the numbers.

3. Adding Options

You can also add <options> at the very end of your setup (just before the do block) to change the behavior of the loop or format the final output.

for <generator>, <option> do <expression> end

🤔 How does Elixir recognize <option>s?

Unlike <generator>s (which use <-) or <filter>s (which are arbitrary expressions), <option>s are recognized because they belong to a strictly predefined set of keywords built into the Elixir language. The most common ones are into:, uniq:, and reduce:.

For example, using the predefined into: keyword turns the output from a standard list into a map:

for x <- [1, 2, 3], into: %{} do
  {x, x * 10}
end
# Result: %{1 => 10, 2 => 20, 3 => 30}

We can mix and match <option>s (more than one) to change how data is collected. However, not all <option>s play well together.

The most common combination is using into: and uniq: together.

for n <- [1, 1, 2, 2, 3], uniq: true, into: %{} do
  {n, n * n}
end
# Result: %{1 => 1, 2 => 4, 3 => 9}

reduce: is the "lone wolf". You cannot combine reduce: with into: because they are two different ways of handling the accumulation.

words = ["apple", "banana", "apple", "cherry", "banana", "apple"]

for word <- words, reduce: %{} do
  acc -> Map.update(acc, word, 1, & &1 + 1)
end
# Result: %{"apple" => 3, "banana" => 2, "cherry" => 1}

for <generator>, reduce: <initial_accumulator> do
  <current_accumulator> -> <new_accumulator>
end

In the reduce: case, Elixir swaps the default collection behavior for a manual accumulation process.

<initial_accumulator>:

• We provide an initial value (like 0, [], or %Base{}) which acts as the starting state for the accumulator.

<current_accumulator> -> <new_accumulator>:

• Instead of the do block simply returning a value to be added to a list, it acts as a reducing function that takes the current accumulator on the left side of the -> arrow.
• Whatever the expression on the right side returns becomes the new state of the accumulator for the very next iteration.
• Elixir skips the do block entirely and passes the unchanged accumulator forward, ensuring that by the time the <generator> is exhausted, you are left with a single, final value.

4. Adding Filters + Options

Before jumping into Complex Chain, let's combine these concepts. You can shape the data with a <filter> and change the output format with an <option>:

for <generator>, <filter>, <option> do <expression> end

Here are some examples that walk through the increasing complexity of Elixir comprehensions:

Example 4.1.

for n <- [1, 1, 2, 2, 3, 4], # generator
  rem(n, 2) == 0, # filter: find even numbers
  uniq: true, # option: remove duplicates
  do: n * 10 # expression
# Result: [20, 40]

In this example, we use one <filter> to find even numbers and the uniq: true option to remove duplicates.

Example 4.2.

for n <- 1..10, # generator
  rem(n, 2) == 0, # filter: find even numbers
  n > 5, # filter: find numbers > 5
  into: %{}, # option: collect the results in a Map
  do: {n, "Number #{n}"}
# Result: %{6 => "Number 6", 8 => "Number 8", 10 => "Number 10"}

Here, we use two <filter>s to narrow down the list before collecting the results into a Map.

Example 4.3.

for n <- [2, 4, 6, 8, 8, 10, 12], # generator
  n > 5, # filter: find numbers > 5
  n < 11, # filter: find numbers < 11
  uniq: true, # option: remove duplicates
  into: [], # option: collect the result in a List
  do: n * 2
# Result: [12, 16, 20]

This shows the uniq: and into: working together. It filters the data and ensures the destination collection only receives unique keys.

Example 4.4.

for x <- [1, 2], # generator 1
  y <- [10, 20], # generator 2: every single x, it will try every possible value of y
  x + y > 12, # filter 1
  y < 25, # filter 2
  reduce: 0, # option
  do: (acc -> acc + x + y)

# Result: 43

5. The Complex Chain

The true power of Elixir comprehensions lies in the Complex Chain. A comprehension isn't limited to just one <generator> or one <filter>. You can string together as many steps as you want, in almost any order.

The blueprint can take many forms depending on how you want to process your data. You can group all your data extraction first, followed by all your filtering:

for <generator>, <generator>, <generator>,
    <filter>, <filter>, <filter>,
    <options> do 
  <expression> 
end

Or, you can interleave them step-by-step (which is highly recommended for performance):

for <generator>, <filter>,
    <generator>, <filter>,
    <generator>, <filter>,
    <options> do 
  <expression> 
end

As long as you start the entire comprehension with at least one <generator>, you can mix and match <generator> and <filter> combinations infinitely!

6. How Elixir Decodes the Syntax (The Clues)?

With so many possible combinations, looking at a massive block of for code can be intimidating. How does the engine know which part is a <generator> and which is a <filter>?

Elixir's compiler reads it very simply. It looks for specific "clues" in your comma-separated list to know exactly what a piece of code is supposed to do, regardless of the order you place them in:

7. Execution Flow

Because Elixir uses these simple clues, you don't have to put all <generator>s first and all <filter>s last. You can interleave them.

Let's look at how Elixir processes the chain: <generator 1>, <filter 1>, <generator 2>, <filter 2>

1. Generator 1 pulls the first item.
2. Filter 1 evaluates that item. (a) If the <filter>'s truthiness is false or nil, Elixir throws the item away, skips the rest of the chain, and asks Generator 1 for the next item. (b) If it evaluates to truthy (e.g., true, a number, a string), it passes the gate.
3. Generator 2 is now triggered. Because it sits after Generator 1, it runs a full loop for every single item that survives Filter 1. This creates a nested loop, also known as a Cartesian Product.
4. Filter 2 evaluates the combined result of Generator 1 and 2.

Example 7.1. Finding valid mixed colors from RGB channels

for r <- [0, 255], g <- [0, 255], b <- [0, 255], # 3 Generators Grouped
  r + g + b > 0, # Filter 1: No pure black
  r + g + b < 765 do # Filter 2: Nor pure white
    {r, g, b} 
  end
# Result: [{0, 0, 255}, {0, 255, 0}, {0, 255, 255}, {255, 0, 0}, {255, 0, 255}, {255, 255, 0}]

This approach is useful when you need to calculate the entire Cartesian product first before you have enough context to filter it.

Example 7.2. Finding senior staff on high-performing teams at profitable companies.

companies = [
  %{
    name: "TechCorp",
    profit: 100_000,
    teams: [
      %{
        name: "Alpha",
        rating: 5,
        members: [
          {:senior, "Alice"},
          {:junior, "Bob"}
          ]
      }
    ]
  },
  %{
    name: "BankInc",
    profit: -50_000,
    teams: []
  } # Unprofitable company
]

for company <- companies,
    company.profit > 0, # Filter 1: Skip losing companies instantly
  team <- company.teams,
    team.rating >= 4, # Filter 2: Skip low-rated teams instantly
  { role, member_name } <- team.members,
    role == :senior do # Filter 3: Only keep senior members
  "#{member_name} from #{team.name} at #{company.name}"
  end
# Result: ["Alice from Alpha at TechCorp"]

This is where Elixir shines. By placing a <filter> immediately after its relevant <generator>, you prune bad branches early. If a company isn't profitable, Elixir skips iterating over its teams entirely!

As long as you start the entire comprehension with at least one <generator>, you can mix and matchand` combinations infinitely!

to be continued...