Large pull requests create a problem that is easy to underestimate.
A pull request with a few hundred changed lines is usually straightforward to display. A pull request containing hundreds of thousands or even millions of changed lines is different. Loading, calculating, and rendering that much diff data can put significant pressure on both the server and the browser.
GitHub has worked on its pull request architecture to handle very large changes without requiring the browser to load and render the entire diff at once.
The interesting part is not simply that a million-line pull request can exist. The more important engineering problem is how a system can make such a large change set usable without turning the browser into a bottleneck.
This article looks at the engineering ideas behind large diff rendering, including pagination, virtualization, incremental loading, data processing, and the trade-offs involved in building a system that can handle extremely large pull requests.
Why Rendering a Huge Diff Is Difficult
A code diff is more than plain text.
For every changed file, a system may need to determine:
Added lines
Removed lines
Context lines
File names
Renames
Binary files
Syntax highlighting
Comments
Review threads
Inline suggestions
File-level metadata
Now imagine doing this for a very large pull request.
Suppose a pull request contains:
10,000 files
+
1,000,000 changed lines
A naive implementation could attempt to send the complete diff to the browser:
Server
↓
Generate complete diff
↓
Serialize everything
↓
Send huge response
↓
Browser parses everything
↓
Browser creates thousands of DOM elements
↓
Browser renders everything
This approach does not scale well.
The browser must allocate memory for the response, parse it, create UI elements, calculate layout, and maintain the resulting DOM.
The result can be slow loading, excessive memory usage, and poor scrolling performance.
The Key Idea - Do Not Render Everything
One of the most important principles behind scalable interfaces is simple:
If the user cannot see it, do not render it yet.
Consider a pull request with 10,000 changed files.
A developer might initially see:
Files changed
src/Orders.cs
src/Customers.cs
tests/OrdersTests.cs
...
There is no reason for the browser to create every line of every file immediately.
Instead, the application can load and render only the content required for the current view.
This general approach is commonly known as virtualization or windowed rendering.
What Is Virtualized Rendering?
Virtualized rendering means that the UI maintains only the elements needed for the visible portion of a large dataset.
Imagine a list containing:
1,000,000 rows
A browser does not need to create one million DOM nodes just because the user can eventually scroll through them.
Instead, it can render something closer to:
Rows 1 - 50
When the user scrolls:
Rows 51 - 100
replace the previous visible window.
The user experiences a continuous list, while the browser handles a much smaller number of UI elements.
The same principle can be applied to large code diffs.
Why Traditional DOM Rendering Breaks Down
Consider this simplified HTML:
<div id="diff"></div>
A naive implementation might generate an element for every line:
for (const line of millionLines) {
const element = document.createElement("div");
element.textContent = line;
diff.appendChild(element);
}
This creates an enormous DOM.
The problem is not just the number of characters.
Each DOM element has overhead involving:
Memory
Layout
Style calculation
Event handling
Painting
Browser bookkeeping
A million-line diff can therefore become much more expensive than the raw text size suggests.
A Better Approach
Instead of rendering every line, render a window.
For example:
const visibleLines = lines.slice(startIndex, endIndex);
for (const line of visibleLines) {
renderLine(line);
}
When the user scrolls, calculate a new range:
Current viewport
↓
Start line
↓
Visible lines
↓
End line
Only that range needs to be represented by actual UI elements.
The rest can remain as data.
Server-Side Processing Matters Too
Browser optimization is only one part of the problem.
The server also has to process the pull request.
Before a diff reaches the browser, the backend may need to:
Read repository information
↓
Compare revisions
↓
Calculate changed files
↓
Generate diff information
↓
Process metadata
↓
Return data to the client
For very large changes, doing all of this in one request can create excessive CPU and memory pressure.
A scalable architecture therefore tries to avoid doing unnecessary work upfront.
Incremental Loading
One approach is incremental loading.
Instead of requesting everything:
GET /pull-request/123/diff
and returning the complete million-line result, the client can request smaller pieces.
Conceptually:
Request
↓
File list
↓
Selected file
↓
Selected section
↓
Visible diff
This allows the application to delay expensive operations until the user actually needs them.
For example, if a pull request contains 5,000 changed files and the developer opens only three, there is little reason to fully render all 5,000 immediately.
Pagination and Virtualization Solve Different Problems
These concepts are related but not identical.
Technique | Main Goal | Example |
|---|---|---|
Pagination | Split data into manageable requests | Load 100 files at a time |
Virtualization | Limit rendered UI elements | Render only visible lines |
Lazy loading | Delay loading until needed | Load a file when opened |
Caching | Avoid repeated work | Reuse previously loaded diff |
Streaming | Process data progressively | Display data as it arrives |
A large pull request viewer may use several of these techniques together.
File-Level Virtualization
Consider a pull request with:
2,000 changed files
The file list itself can become large.
Instead of rendering all file entries, the application can virtualize the file navigation list.
For example:
Visible viewport
---------------------
File 101
File 102
File 103
File 104
File 105
---------------------
The UI maintains the correct scroll height while rendering only the visible entries.
This reduces DOM size and improves scrolling.
Line-Level Virtualization
The same idea can be applied inside a file.
Suppose one generated file contains:
200,000 lines
Rendering all 200,000 lines immediately would be expensive.
Instead, the viewer can maintain a logical position for the complete file while rendering only the portion currently visible.
Conceptually:
Logical file
0 -------------------------------- 200,000
Rendered window
12,450 -------- 12,500
As the user scrolls, the rendered window moves.
Why Syntax Highlighting Is Expensive
Code editors and diff viewers often provide syntax highlighting.
For example:
public async Task<User?> GetUserAsync(int id)
{
return await repository.GetByIdAsync(id);
}
A syntax highlighter may need to tokenize the code and determine which portions represent:
Keywords
Types
Methods
Strings
Comments
Operators
Doing this for one small file is inexpensive.
Doing it for millions of lines can become expensive.
A scalable implementation should avoid syntax-highlighting content that the user has not requested to view.
This is another reason lazy rendering is important.
Review Comments Add Another Layer
A modern pull request interface is not just a diff viewer.
A changed line can have:
Code
+
Review comment
+
Reply
+
Reaction
+
Suggested change
+
Resolved state
Imagine thousands of comments spread across a huge pull request.
The system has to associate comments with the correct file and line while the user scrolls through the diff.
This creates another reason to avoid building a massive DOM tree containing every review element.
Anchoring Comments to Changed Lines
A review comment is normally associated with a particular location in a diff.
Conceptually:
File: UserService.cs
Line: 42
Comment: Handle the null result
But code changes over time.
If lines are added or removed, the original position may no longer exist.
The review system therefore needs a way to associate comments with the relevant code location rather than relying only on a static screen position.
This becomes more complicated when a large pull request contains thousands of changes.
The Browser Is Often the Real Bottleneck
Developers sometimes assume that if the server can generate the data, the browser can display it.
That is not necessarily true.
Suppose the server generates:
100 MB of diff data
The browser may need to:
Download 100 MB
↓
Parse the response
↓
Create objects
↓
Build UI
↓
Apply styles
↓
Calculate layout
↓
Paint pixels
The actual memory requirement can become considerably larger than the original response.
This is why frontend architecture is critical for large code review interfaces.
A Simple Performance Model
Consider a naive diff viewer.
If the number of changed lines is N, rendering every line creates approximately:
DOM elements = N
If each line requires additional child elements for syntax highlighting, the actual element count can become:
DOM elements > N
For example:
1,000,000 lines
×
3 DOM elements per line
=
3,000,000 DOM elements
This is only an illustrative calculation, not a benchmark.
The exact number depends on the implementation.
The important lesson is that DOM complexity can grow rapidly when every piece of content becomes a separate UI element.
A Virtualized Alternative
With virtualization, suppose the viewport requires only 100 lines:
Total logical lines = 1,000,000
Rendered lines = approximately 100
The browser can maintain a logical scroll area while keeping the active DOM much smaller.
A simplified architecture looks like this:
Million-line diff
|
+---------+---------+
| |
Data model Scroll position
|
↓
Visible line range
|
↓
Render small window
This architecture allows the amount of rendered UI to remain relatively stable even as the total dataset grows.
Why Large Pull Requests Are Still Difficult
Virtualization solves rendering problems, but it does not solve every problem.
A million-line pull request can still be difficult because:
Code Context Is Huge
Understanding the relationship between thousands of files is challenging.
Review Quality Can Decline
Important changes can become difficult to distinguish from generated or repetitive changes.
Navigation Becomes Harder
Finding the important part of a huge change can take significant effort.
Backend Processing Still Matters
The system still needs to calculate and organize the underlying data.
Human Review Does Not Scale Linearly
A reviewer cannot simply spend 1,000 times longer reviewing a 1,000-times-larger change.
This is one reason smaller, logically organized pull requests remain valuable.
Production Architecture for Large Diff Systems
A scalable implementation can separate responsibilities.
For example:
Pull Request
|
↓
Diff Processing
|
+--------+--------+
| |
File Index Metadata
| |
+--------+--------+
|
↓
API Layer
|
↓
Client Browser
|
+--------+--------+
| |
File List Diff Viewer
| |
+--------+--------+
|
Virtualized UI
The important design principle is to avoid forcing every layer to process everything immediately.
Caching Can Reduce Repeated Work
Large pull requests can involve repeated access to the same information.
For example, a developer may:
Open file
↓
Leave file
↓
Return to file
If the system recalculates everything each time, resources are wasted.
Caching can allow previously processed data to be reused.
Possible cache targets include:
File metadata
Diff segments
Syntax information
Review metadata
Repository information
Caching strategy depends on the system and consistency requirements.
Handling Generated Files
Generated files should receive special treatment in large repositories.
For example:
GeneratedClient.cs
GeneratedModels.cs
GeneratedSerializer.cs
might contain tens of thousands of lines.
If those files are generated deterministically from a schema, reviewing every generated line may provide little value.
A better workflow can focus review attention on:
Source schema
Generation configuration
Generator changes
Handwritten integration code
This reduces noise without pretending that generated changes do not exist.
Common Mistakes When Building Large Diff Viewers
Rendering Everything at Once
This is the most obvious scalability problem.
Loading Every File Before the User Opens It
This increases unnecessary network and processing costs.
Performing Syntax Highlighting for Hidden Content
Only visible or requested content should normally require expensive rendering work.
Creating Large DOM Trees
A massive DOM can create memory and layout problems.
Ignoring Mobile and Low-Memory Devices
A solution that works on a powerful development machine may behave differently on less capable hardware.
Treating Server Performance and Browser Performance as the Same Problem
Both need separate optimization strategies.
Troubleshooting Large Diff Performance
The Page Takes Too Long to Open
Check whether the client is downloading the complete diff before displaying anything.
Incremental loading can reduce initial work.
Scrolling Is Slow
Inspect DOM size and rendering frequency.
Virtualization can help when too many elements are being rendered simultaneously.
Browser Memory Usage Is High
Look for:
Huge response objects
Large DOM trees
Duplicate data
Cached content
Syntax-highlighted content that is not visible
Opening a Large File Freezes the Browser
Avoid processing the entire file synchronously.
Load and render the visible portion first.
Server Response Is Slow
Measure separately:
Diff generation
Serialization
Network transfer
Client parsing
Client rendering
Do not assume the browser is always responsible for the delay.
Advantages of Scalable Diff Rendering
Better browser performance - Only the necessary content is rendered.
Lower initial loading cost - Users can see useful content sooner.
Better memory usage - The browser does not need to maintain millions of active DOM elements.
Improved navigation - Large datasets can be loaded progressively.
Supports very large repositories - The architecture can scale beyond small pull requests.
Disadvantages and Trade-Offs
Implementation is more complex.
Virtualized interfaces require careful scroll and layout handling.
Lazy loading introduces additional client-server requests.
Caching increases system complexity.
Very large changes can still be difficult for humans to understand.
Rendering optimization does not solve code-review quality problems.
What Developers Can Learn From This
The engineering lessons behind large pull request rendering apply far beyond GitHub.
The same principles are useful when building:
Log viewers
Database management tools
IDEs
Monitoring dashboards
Large data tables
File explorers
Code search interfaces
Test result viewers
Whenever an application needs to display millions of records, ask:
Do I need to load everything?
Do I need to process everything?
Do I need to render everything?
Does the user need everything immediately?
The answer is usually no.
Best Practices for Large Code Review Systems
Load only the information required for the current view.
Use virtualization for large lists and code regions.
Separate file indexing from diff rendering.
Delay syntax highlighting until content is visible.
Cache expensive operations where appropriate.
Avoid unnecessary generated-file processing.
Measure server and browser performance separately.
Keep the initial response small.
Test with realistic large datasets.
Test memory consumption as well as response time.
Keep pull requests logically focused when possible.
Conclusion
Rendering a million-line pull request is not primarily a problem of displaying more text. It is a systems problem involving backend processing, data transfer, browser memory, DOM management, syntax highlighting, navigation, and review metadata.
The most important architectural idea is to avoid treating the entire pull request as something that must be loaded and rendered at once.
Incremental loading, lazy processing, caching, and virtualization allow a system to work with extremely large datasets while keeping the amount of active UI relatively small.
There is also an important lesson for developers: a system being capable of handling a million-line pull request does not mean a million-line pull request is easy to review.
Good engineering practices still matter. Smaller logical changes, focused pull requests, clear commit boundaries, and separation of generated code can make the actual review process much easier.
Large-scale rendering technology makes huge pull requests more manageable, but it does not remove the need for thoughtful software design or careful human review.
Join the conversation! Your thoughts help the community grow.