Failure Museum.Made with ❤️ by Aniruddh Jangid
Submit a case

EXHIBIT 038 · IT & SOFTWARE

GitHub: a brief partition, a long recovery

A network disruption triggered database failover and inconsistent database state.

OrganizationGitHub
Period2018
LocationGlobal
StatusRecovered
ImpactHigh
EvidenceTechnical postmortem
Compare

Illustrative draft · Source-backed synopsis · Museum analysis marked separately · Updated 5 October 2026

A GitHub workshop; contextual photograph of software collaboration, not the 2018 outage.
A GitHub workshop; contextual photograph of software collaboration, not the 2018 outage.Lisa Risager from Denmark Photograph · Context photograph; not the incident itself. · CC BY-SA 2.0 · Resized for display.
01 / EXECUTIVE SUMMARY

The outcome is only the beginning.

A network disruption triggered database failover and inconsistent database state. [1]

THE PRINCIPLE TO CARRY FORWARD

Test stateful failover under partial network partitions, and define consistency priorities before an incident.

02 / THE PROMISE

What the system was trying to do.

Keep code collaboration available across distributed infrastructure.

Museum interpretation

The assumption beneath the promise

Museum hypothesis to examine: Restoring connectivity is sufficient to restore consistency after distributed failover. This is an analytical proposition to test against the record, not an attributed statement by the organization.

Success should be assessed against the intended service and its safety, reliability, or integrity constraints. A headline outcome cannot tell us whether those constraints were visible, tested, or owned before the event.

03 / WHAT HAPPENED

A sequence, not a single moment.

The intended system

Keep code collaboration available across distributed infrastructure.

Source-backed synopsis

The event or review

A network disruption triggered database failover and inconsistent database state. [1]

Source-backed synopsis

The documented aftermath

GitHub reconciled data and published a detailed incident analysis. [1]

Source-backed synopsis
Timeline of documented milestones. Gaps are not evidence that nothing happened. Sources: [1]
04 / WARNING SIGNS

What could be seen at the time?

Documented in cited record

Restoring consistency was more complex than simply restoring connectivity. [1]

Who could see it?

See the cited investigation for named teams and the information they held. The synopsis does not infer awareness by every stakeholder.

Why was it not enough?

Signals need interpretation, authority, and a route to action. This is an analytical question, not proof that a warning was deliberately ignored.

05 / KEY DECISIONS

Choices made within constraints.

Museum analysis: Failover behavior needed to account for latency, location, and state reconciliation.

Available alternatives

Pause, test, narrow the operating envelope, seek independent review, or build a recovery option. These are proposed analytical alternatives; feasibility at the time is not established.

Information available

Restoring consistency was more complex than simply restoring connectivity. [1]

Incentives & constraints

Delivery pressure, cost, authority, and incomplete knowledge may shape a decision. Their specific weight is not established by this synopsis.

Risk accepted

Ask whether the relevant risk was recognized, who had authority to accept it, and whether affected people understood its implications.

06 / SYSTEM MAP

How conditions connected.

Select a node to inspect its evidence. Links show a proposed analytical relationship, not a measured causal effect.

05 / External context04 / Warning signal03 / Decisions02 / Technical conditions01 / Trigger06 / Consequences
Triggering event

A network disruption triggered database failover and inconsistent database state. [1]

Legend: numbered nodes = analytical categories; dashed arrows = proposed influence or propagation. Museum interpretation informed by the sources below. Feedback and omitted influences require further investigation.
07 / CONSEQUENCES

Beyond the immediate event.

Users experienced prolonged degradation of collaboration features. [1]

Technical impactOperational impactCustomer impact

Categories identify documented or relevant consequences. Financial, human, and environmental totals are not estimated here when the source base does not support them.

08 / WHAT THE MODELS EXPLAIN

More than one lens.

Analytical lens

Fault Tree Analysis

Work backward from a defined loss using logical combinations of conditions.

In this case, use the lens to examine state divergence. This application is museum interpretation, not a finding of the original investigation.

Limit: Results depend on the chosen top event and completeness of branches.

Analytical lens

FMEA

Identify failure modes, their effects, and controls before or during system development.

In this case, use the lens to examine state divergence. This application is museum interpretation, not a finding of the original investigation.

Limit: Lists can miss interactions and unanticipated operating conditions; scores are not exact probabilities.

Analytical lens

Systems thinking

Examine relationships, boundaries, feedback, and incentives across a whole system.

In this case, use the lens to examine state divergence. This application is museum interpretation, not a finding of the original investigation.

Limit: A broad lens needs explicit boundaries and evidence to avoid explaining everything after the fact.

09 / COUNTERFACTUALS

What could have changed the outcome?

Counterfactuals are hypotheses. They identify possible intervention points without claiming that a different choice would certainly have prevented the event.

INTERVENTION 2 / During early testing

Test the operating envelope

Exercise realistic boundary conditions and shared dependencies rather than validating components only in isolation.

Cost & feasibility

Requires time and independent review before commitment.

Likely effectiveness

May reduce exposure or consequences. Not quantified; feasibility and effect must be assessed against evidence available at the time.

10 / LESSONS

From hindsight to a usable practice.

FOR FOUNDERS

Test stateful failover under partial network partitions, and define consistency priorities before an incident.

Before accelerating hiring or spending, name the assumptions that remain untested and set evidence thresholds for the next commitment.

11 / AFTERWARD

Recovery is another investigation.

GitHub reconciled data and published a detailed incident analysis. [1]

A corrective action is evidence of a response; it is not, by itself, evidence that the wider pattern has disappeared.

12 / SOURCES & EVIDENCE

Follow the record.

Documented claims are linked to sources. Analytical applications, lessons, and intervention proposals are museum interpretation. This draft does not establish motives or a single complete causal explanation.

  1. 1
    github.blog — GitHub case record

    Technical postmortem · Accessed 4 October 2026

Editorial history & limits

Version 1 · 4 October 2026 — source-backed illustrative synopsis created. Independent editorial review is pending. No claim of exhaustive investigation. New sources may alter the analysis.