Stop fetching everything just to process one element at a time

12 min. read |
Article

Let's start with a familiar piece of code.

const longList = await fetchAllElementsInMemory(); 

for (const item of longList) {              
  process(item);
}

Looks innocent, right?

Fetch the data, store it in an array, iterate over it, and do whatever you need to do.

But here's the problem.

What if fetchAllElementsInMemory() returns 10 million records?

Do you really need all 10 million records sitting in memory before you can start processing the first one?

Probably not.

And yet, this is exactly what we do in a lot of applications.

We fetch everything, allocate memory for everything, wait for everything to finish loading, and then start processing it.

It's a little like ordering an entire restaurant's menu before taking your first bite.

What if there was a better way?

Meet iterators and generators.

First, what exactly is an iterator?

An iterator is a mechanism that gives you access to elements of a collection one at a time, without requiring you to load the entire collection upfront.

Think of it as a vending machine.

You press a button, get one item, consume it, and press the button again for the next one.

You don't need to empty the entire vending machine onto the floor before choosing what you want.

That's essentially what an iterator does.

Instead of returning an entire collection:

function fetchUsers(): User[] { 
  // Fetch and return 1 million users
}

You expose a way to retrieve the next user whenever the consumer is ready.

Conceptually:

function getNextUser(): User | null {
    // Return the next user
}

The consumer controls when to request the next element.

This introduces an important distinction:

  • Eager evaluation: Calculate or fetch everything upfront.
  • Lazy evaluation: Calculate or fetch an element only when it's needed.

And that's where iterators become interesting.

How does this actually improve space and time complexity?

Let's take an example.

You have 10 million users in your database, and you need to calculate how many of them are active.

The traditional approach might look something like this:

const users = await fetchAllUsers(); 

let activeCount = 0; 
for (const user of users) { 
  if (user.isActive) {                     activeCount++; 
      } 
}

What's happening here?

  1. Fetch all 10 million users.
  2. Deserialize them.
  3. Allocate memory for the entire collection.
  4. Start processing.
  5. Calculate the result.

If each user object occupies approximately 1 KB in memory, you're potentially looking at around 10 GB of object data alone.

That's before accounting for JavaScript object overhead, arrays, temporary allocations, and other application memory.

Now imagine doing this across multiple concurrent requests.

Your server's RAM is going to have a very interesting day.

Let's look at the complexity.

PortableText [components.type] is missing "table"

Notice something important?

Iterators do not magically improve the overall time complexity of processing all n elements.

If you need to inspect every record, you still need O(n) time.

What they improve is:

  • Additional memory consumption.
  • Time to first result.
  • Ability to stop processing early.
  • Avoidance of unnecessary intermediate collections.

That distinction matters.

If you're only interested in the first 100 matching records, why fetch and process all 10 million?

An iterator allows you to stop as soon as you've got what you need.

Iterators and generators in TypeScript

JavaScript (and therefore TypeScript) has a built-in iteration protocol.

There are two important concepts:

Iterable: An object that knows how to provide an iterator.

Iterator: An object that knows how to produce the next value.

An iterator implements a next() method that returns an object containing:

{ value: something, done: false }

When there are no more values:

{ value: undefined, done: true }

TypeScript provides two interfaces to understand this:

interface Iterator<T> {
    next(): IteratorResult<T>;
}

interface Iterable<T> {
    [Symbol.iterator](): Iterator<T>;
}

You don't usually need to implement these interfaces manually because JavaScript already provides them for arrays, maps, sets, and other built-in collections.

Generators: The easiest way to create iterators

This is where things get really nice.

Instead of manually implementing next(), you can use generator functions.

A generator function is declared using function* and produces values using yield.

Let's see:

function* generateNumbers() {
    yield 1;
    yield 2;
    yield 3;
}

const numbers = generateNumbers();

console.log(numbers.next());
// { value: 1, done: false }

console.log(numbers.next());
// { value: 2, done: false }

console.log(numbers.next());
// { value: 3, done: false }

console.log(numbers.next());
// { value: undefined, done: true }

Every time next() is called, execution resumes from where the previous yield left off.

This is the important part.

The generator doesn't calculate all the values and store them somewhere before returning.

It pauses, gives you a value, and waits until you ask for the next one.

You can also use it directly with for...of:

function* generateNumbers() {
    yield 1;
    yield 2;
    yield 3;
}

for (const number of generateNumbers()) {
    console.log(number);
}

Much cleaner.

Let's solve our original problem

Remember our 10 million users?

Let's create a generator that produces users one at a time.

function* getUsers() {
    for (let i = 1; i <= 10_000_000; i++) {
        yield {
            id: i,
            isActive: i % 2 === 0
        };
    }
}

let activeCount = 0;

for (const user of getUsers()) {
    if (user.isActive) {
        activeCount++;
    }
}

console.log(activeCount);

We never create an array containing 10 million user objects.

We generate one user, process it, and move to the next one.

The generator maintains only the state required to resume execution.

Additional memory for this example is O(1), excluding output and runtime overhead.

What if fetching the data itself is asynchronous?

This is a very common scenario in backend applications.

Imagine fetching records from an external API or reading data from a paginated endpoint.

You don't want to fetch every page upfront.

TypeScript has something for this too: Async Iterators and Async Generators.

Notice the difference:

  • function* creates a synchronous generator.
  • async function* creates an asynchronous generator.

Here's an example:

async function* fetchUsers() {
    let page = 1;

    while (true) {
        const response = await fetch(
            `https://api.example.com/users?page=${page}`
        );

        const users = await response.json();

        if (users.length === 0) {
            break;
        }

        for (const user of users) {
            yield user;
        }

        page++;
    }
}

And consuming it:

let activeCount = 0;

for await (const user of fetchUsers()) {
    if (user.isActive) {
        activeCount++;
    }
}

Look at what's happening:

  1. Fetch the first page.
  2. Yield its users individually.
  3. Process them.
  4. Fetch the next page when the current page has been consumed.
  5. Continue until there are no more pages.

We never need to maintain all 10 million records in memory.

One important detail: this example buffers one API page at a time. If an API returns 10,000 records per page, those 10,000 records still occupy memory.

The memory requirement is therefore O(page size), not literally zero.

That's lazy evaluation in practice.

Iterators in Go

Now let's switch to Go.

Go has always had a straightforward way of iterating over collections:

for _, user := range users {            process(user)
}

But here's the catch.

If users is already a slice containing 10 million records, you've already paid the memory cost of loading those records.

The range keyword itself doesn't make iteration lazy.

A slice is still a slice.

Go introduced a more interesting approach with range-over-function iterators in Go 1.23.

This allows us to write custom iterator functions and consume them using the familiar range syntax.

Understanding the iter package

Go 1.23 introduced the standard library's iter package.

It provides two important iterator types:

type Seq[V any] func(yield func(V) bool)

type Seq2[K, V any] func(yield func(K, V) bool)

Let's simplify this.

  • Seq produces one value at a time.
  • Seq2 produces two values at a time, commonly a key-value or index-value pair.

The yield function is how the iterator hands values to the consumer.

If yield returns false, it means the consumer wants to stop iterating.

Let's implement one:

package main

import (
    "fmt"
    "iter"
)

func GenerateNumbers() iter.Seq[int] {
    return func(yield func(int) bool) {
        for i := 1; i <= 5; i++ {
            if !yield(i) {
                return
            }
        }
    }
}

func main() {
    for num := range GenerateNumbers() {
        fmt.Println(num)
    }
}

Output:

1 2 3 4 5

What's interesting here?

We aren't returning a slice.

We're returning a function that knows how to produce values and pass them to the consumer.

The consumer controls the iteration.

What happens when we want to stop early?

Let's say we only want the first three numbers.

func main() {
    count := 0

    for num := range GenerateNumbers() {
        fmt.Println(num)

        count++

        if count == 3 {
            break
        }
    }
}

Output:

1 2 3

When we use break, the generated iteration machinery stops the sequence.

The iterator receives false from yield and should stop producing values.

This is especially useful when processing large datasets where you don't necessarily need every record.

A practical backend example

Let's imagine we're reading millions of users from PostgreSQL using Go.

The traditional approach might look like this:

users, err := fetchAllUsers(db)
if err != nil {
    return err
}

for _, user := range users {
    process(user)
}

The problem?

fetchAllUsers() has to construct and return a complete slice before processing begins.

Instead, we can expose an incremental database iterator.

package main

import (
    "database/sql"
    "iter"
)

type User struct {
    ID       int
    IsActive bool
}

func Users(db *sql.DB) iter.Seq2[User, error] {
    return func(yield func(User, error) bool) {
        rows, err := db.Query(
            "SELECT id, is_active FROM users",
        )

        if err != nil {
            yield(User{}, err)
            return
        }

        defer rows.Close()

        for rows.Next() {
            var user User

            if err := rows.Scan(
                &user.ID,
                &user.IsActive,
            ); err != nil {
                yield(User{}, err)
                return
            }

            if !yield(user, nil) {
                return
            }
        }

        if err := rows.Err(); err != nil {
            yield(User{}, err)
        }
    }
}

And consuming it:

func processUsers(db *sql.DB) {
    for user, err := range Users(db) {
        if err != nil {
            // Handle error
            return
        }

        if user.IsActive {
            // Process active user
        }
    }
}

Notice what we achieved:

  • We didn't construct a slice of all users.
  • We scanned a row when the consumer requested the next value.
  • We allowed early termination.
  • We kept database resource cleanup inside the iterator.

A caveat: database drivers and databases have their own buffering and cursor behavior. The iterator avoids materializing the entire result set in our application code, but it doesn't guarantee zero buffering throughout the system.

Also, the iterator must be consumed for its deferred cleanup to execute. If an iterator is created but never started, its body never runs and no query is opened.

TypeScript vs Go: What's actually different?

Both languages solve the same fundamental problem, but their implementations have some differences.

One interesting difference is that TypeScript has supported generators and async generators for quite some time, whereas Go introduced its standardized iterator package and range-over-function support in Go 1.23.

Also, Go's iterator model isn't a direct equivalent of JavaScript's next()-based iterator protocol.

Go uses a push-style callback mechanism: the iterator calls yield, rather than the consumer explicitly calling next().

Different implementation, same fundamental idea.

What about Python? Does it have iterators?

Absolutely.

In fact, Python has a very explicit iterator protocol, and it's one of the core concepts behind how its for loops work.

There are two important methods:

__iter__() and __next__()

An iterable provides an iterator through __iter__().

An iterator provides the next element through __next__().

When there are no more values, it raises StopIteration.

Python also has generators, which make implementing iterators extremely convenient.

Python generators

Consider the traditional approach:

def get_numbers():
    return [i for i in range(10_000_000)]

numbers = get_numbers()

for number in numbers:
    process(number)

This creates a list containing 10 million integers.

Now compare that with:

def get_numbers():
    for i in range(10_000_000):
        yield i

for number in get_numbers():
    process(number)

That's it.

One keyword changes the behaviour significantly.

yield suspends the function and preserves its execution state.

When the consumer requests another value, execution resumes from where it stopped.

Python has something even more interesting: Generator expressions

Look at this:

numbers = [i * i for i in range(10_000_000)]

This is a list comprehension.

It materializes the entire list.

Now:

numbers = (i * i for i in range(10_000_000))

This is a generator expression.

It produces values lazily.

You can even use it directly:

total = sum( i * i for i in range(10_000_000) )

You don't need to construct an intermediate list containing 10 million squared numbers.

The sum() function consumes the iterator and maintains its running total.

Python's itertools

Python takes this idea further with its standard library module itertools.

It provides building blocks for composing memory-efficient iteration pipelines.

For example:

from itertools import islice

numbers = range(10_000_000)

first_hundred = islice(numbers, 100)

for number in first_hundred:
    process(number)

Or:

from itertools import chain

combined = chain(
    range(1_000_000),
    range(1_000_000, 2_000_000)
)

for number in combined:
    process(number)

Neither operation needs to materialize a new list containing all the values.

Python's itertools is particularly useful for composing lazy transformations.

Async iterators in Python

Just like TypeScript, Python supports asynchronous iteration.

You use:

  • async def
  • yield
  • async for

Example:

async def fetch_users():
    page = 1

    while True:
        users = await fetch_api_page(page)

        if not users:
            break

        for user in users:
            yield user

        page += 1

Consumption:

async for user in fetch_users(): 
  await process(user)

This is useful for asynchronous APIs, network streams, and other I/O-bound workloads.

Do we actually need generators in production?

This is where things get interesting.

We have iterators in Go, generators in TypeScript, and generators in Python.

But we also have pagination, database cursors, batching, garbage collection, worker pools, and other memory-management techniques.

Are generators really necessary when these other techniques already exist?

My take 🌶️:

Generators deserve a place in production codebases, but they shouldn't be the default solution for every large-data problem.

Let's understand why.

What problem do generators solve that pagination doesn't?

Consider this function:

async function fetchUsers(): Promise<User[]> {
    let page = 1;
    const users: User[] = [];

    while (true) {
        const response = await fetchPage(page);

        if (response.length === 0) break;

        users.push(...response);
        page++;
    }

    return users;
}

We have pagination.

But we haven't solved the memory problem.

We're still accumulating every record in the users array.

Now imagine exposing a generator instead:

async function* fetchUsers() {
    let page = 1;

    while (true) {
        const response = await fetchPage(page);

        if (response.length === 0) break;

        for (const user of response) {
            yield user;
        }

        page++;
    }
}

The consumer can now process records as they arrive:

for await (const user of fetchUsers()) { 
  await processUser(user); 
}

The generator provides a clean interface between the data producer and the data consumer.

And importantly, the producer doesn't need to know whether the consumer is:

  • Writing records to another database.
  • Sending emails.
  • Generating a CSV.
  • Calculating statistics.
  • Filtering records.

That's a valuable separation of concerns.

The strongest argument for generators: Composable data pipelines

This is where I think generators really justify themselves.

Imagine a production workflow:

  1. Fetch users.
  2. Filter active users.
  3. Filter users who have subscribed to premium.
  4. Transform their data.
  5. Send them notifications.

Without lazy processing, you might end up with:

const users = await fetchAllUsers();

const activeUsers = users.filter(
    u => u.isActive
);

const premiumUsers = activeUsers.filter(
    u => u.isPremium
);

const notificationData = premiumUsers.map(u => ({
    email: u.email,
    name: u.name
}));

for (const user of notificationData) {
    await sendNotification(user);
}

Look at the intermediate arrays.

Depending on the implementation, you're maintaining multiple collections containing references to the same underlying user objects.

Even when you're not duplicating every user object, you're allocating additional arrays containing potentially millions of references.

Now let's turn this into a lazy pipeline:

async function* getNotificationUsers() {
    for await (const user of fetchUsers()) {
        if (!user.isActive || !user.isPremium) {
            continue;
        }

        yield {
            email: user.email,
            name: user.name
        };
    }
}

Consumption:

for await (const user of getNotificationUsers()) { 
    await sendNotification(user); 
}

Now each record flows through the pipeline individually.

No full-sized intermediate arrays.

No need to wait for all users to be fetched.

The consumer determines the pace of processing.

This is the architectural benefit I'd emphasize.

Generators allow us to build reusable, composable transformations without materializing every intermediate result.

Python is particularly good at this because of its itertools ecosystem.

For example:

def get_notification_users(users):
    for user in users:
        if user.is_active and user.is_premium:
            yield {
                "email": user.email,
                "name": user.name
            }

notifications = get_notification_users(fetch_users())

for user in notifications:
    send_notification(user)

The interesting thing is that fetch_users() itself can be a generator.

So the entire pipeline can remain lazy, from the source to the final consumer.

7.3 Generators are not always better

This is where I'd generally avoid them.

Consider three scenarios.

Scenario A: You have 500 records.

const users = await fetchUsers();

Just return the array.

There is no meaningful memory-management benefit in introducing a generator here.

The additional abstraction might actually make the code harder to understand.

Scenario B: You have 10 million records and need to process them sequentially.

A generator is a very good fit.

for await (const user of fetchUsers()) { 
  await processUser(user); 
}

You avoid building an enormous array and can start processing early.

Scenario C: You have 10 million records and need high throughput.

This is where a simple generator might not be enough.

Suppose each record requires an independent network request.

Sequential processing:

for await (const user of fetchUsers()) { 
  await processUser(user); 
}

This might be memory-efficient but painfully slow.

What you absolutely shouldn't do is this:

await Promise.all( users.map(user => processUser(user)) );

This can create millions of pending promises and retain millions of associated objects.

Even if users originally came from a generator, converting it into a massive array or creating millions of pending promises defeats the original purpose.

A generator controls production. Backpressure and bounded concurrency control consumption.

You often need both.

How does a programming language actually manage large data structures in memory?

Now we're getting to the more fundamental question.

Forget iterators for a moment.

Suppose you write:

users = fetch_all_users()

Or:

users := fetchAllUsers()

Or:

const users = await fetchAllUsers();

What actually happens underneath?

Regardless of the language, there are a few common stages:

  1. Allocation: The runtime reserves memory for the data structure.
  2. Representation: Data is stored as arrays, objects, pointers, buffers, etc.
  3. Access and mutation: Your application reads, modifies and processes the data.
  4. Lifetime management: Memory is reused or reclaimed when it is no longer needed.

The interesting part is that different languages implement these stages differently.

How Go manages memory

Go uses a combination of stack allocation, heap allocation, escape analysis, and garbage collection.

Consider:

users := make([]User, 1000000)

A slice has a descriptor containing:

  • Pointer to its backing array.
  • Length.
  • Capacity.

The backing array stores the actual elements contiguously.

This is important because a slice is not itself the underlying array.

For example:

users := make([]int, 1000000) 

small := users[:10]

You might think small only occupies memory for 10 integers.

But it still references the original backing array containing one million integers.

Even if you discard users, keeping small alive can keep the entire backing array alive.

This is a subtle memory-retention problem.

A senior Go developer might solve it by copying the required elements into a smaller backing array:

small := append([]int(nil), users[:10]...)

Now the new slice has its own smaller backing array.

This is the kind of optimization that actually matters in production.

How Python manages memory

Python is interesting because its memory model differs quite a bit from Go.

In CPython, the two important mechanisms are:

  1. Reference counting.
  2. Cyclic garbage collection.

Consider:

users = [1, 2, 3, 4, 5] 
another_reference = users del users

Does the underlying list get destroyed?

No.

Because another_reference still points to it.

Once the last reference disappears, CPython can generally reclaim the object through reference counting.

Cyclic garbage collection handles objects that reference each other in cycles and would otherwise remain alive despite being unreachable from the application.

There's another interesting detail.

A Python list is not the same as a Go slice.

A Python list stores references to Python objects in a dynamically sized array. Those objects generally live separately.

So:

numbers = [1, 2, 3, 4, 5]

isn't necessarily five contiguous machine integers sitting inside the list's backing array.

This introduces object and pointer overhead.

For large numerical datasets, this is one reason developers use libraries such as NumPy.

Instead of millions of individual Python integer objects, a NumPy array can store fixed-width numerical values in a contiguous memory buffer.

For example:

import numpy as np 

numbers = np.arange(10_000_000)

This creates a compact numerical array rather than a Python list of individually allocated integer objects.

And if you're processing millions of numerical values, the difference can be substantial.

One qualification: reference counting describes CPython, not every Python implementation.

When should we actually use generators?

If I were establishing engineering guidelines for a backend team working in TypeScript, Python and Go, I'd use something like this:

PortableText [components.type] is missing "table"

The idea behind iterators and generators is surprisingly simple:

Don't make the consumer wait for all the data when it only needs the next piece of data.

But generators aren't a replacement for pagination, batching, database cursors, garbage collection or concurrency management.

They complement these techniques by providing a reusable abstraction for lazy data production and consumption.

My verdict: I would absolutely encourage generators in TypeScript and Python backend codebases, particularly for data-access layers, ETL jobs, large exports and transformation pipelines.

Not because other memory-management techniques are inadequate, but because generators let us design APIs that don't force consumers to materialize entire collections unnecessarily.

And the next time you see a function named fetchAllSomething(), maybe ask yourself:

Do I actually need all of it, or do I just need the next item?

That’s it for this one, folks.

Until Next Time,

Happy Debugging Coding👋

References

Back To Articles