The Problem: Crashes During Writes
Appending a block to a file changes more than its contents. The filesystem must also update the inode's size and block references, and mark the new block as allocated in a bitmap. These structures live in different places on disk. A power failure between their writes can leave them disagreeing about who owns a block.
A consistency checker can scan the filesystem to reconcile those structures, but the work grows with the filesystem. Journaling records a recoverable transaction so ordinary crash recovery can focus on recent updates. The goal is consistent filesystem structures; preserving every recent application write is a separate requirement.
The Journaling Solution
Write-ahead logging puts recovery information on stable storage before overwriting the corresponding home metadata. A complete transaction has a durable commit record. After that point, the filesystem can copy its logged updates to their home locations, a process called checkpointing.
If power fails during checkpointing, recovery replays the committed records. In the redo-journal model used here, records without a valid commit are ignored. They do not require undoing home metadata because those updates were not allowed to reach home before commit. OSTEP's crash-consistency chapter develops this ordering from the inode-and-bitmap problem.
How Journaling Works: Interactive Exploration
Follow an 8 KiB file growing to 12 KiB, using three 4 KiB blocks. In Crash Recovery, compare failures before commit, after commit, and during checkpointing. Watch the journal, inode, allocation bitmap and file payload separately.
What survives a crash?
Follow one append through the journal and its home locations. The durable commit determines what recovery can replay.
Start with a consistent file
doc.txt occupies blocks 100 and 101. Append 4 KiB by reserving block 102 in memory.
Transaction journal
No active records
- Inode after-image
- No active record
- Bitmap after-image
- No active record
- Commit record
- No active commit
Home locations
doc.txt · 4 KiB blocks
- Inode size
- 8 KiB
- Inode block references
- 100101
- Bitmap for block 102
- free
- Payload in block 102
- New data not durable
No transaction has reached stable storage yet.
Journal Modes: Safety vs Performance
The explorer's transaction uses ext4's ordered-mode rule. ext4 also offers full data journaling and writeback mode; the choice changes which bytes are recoverable and which writes commit must wait for. These names describe ext4 options, not a universal set of modes shared by all filesystems.
1. Journal Mode (Full Journaling)
With data=journal, file contents and metadata pass through the journal. Committed journaled contents can be replayed before their home copies are complete. The additional payload writes increase I/O traffic, but do not make arbitrary application operations atomic.
2. Ordered Mode (Default)
With data=ordered, metadata is journaled and associated file data must reach its home blocks before the metadata transaction commits. Data and journal I/O can overlap; the requirement is the dependency at commit, not one fixed serial schedule. File data itself is not available in the journal for replay, and in-place overwrites are not made atomic.
3. Writeback Mode
With data=writeback, metadata is still journaled, but file-data writeback need not finish before metadata commit. A crash can therefore leave unexpected or stale contents in recently written files. Metadata still obeys write-ahead ordering. See the ext4 journal format and ext4 mount options for the guarantees and tradeoffs.
The Transaction Lifecycle
The important milestones are different kinds of completion:
- Record: describe the metadata update in the journal.
- Flush prerequisites: make those records durable and satisfy the mode's data-ordering requirement.
- Commit: make a valid commit record durable, establishing that recovery may replay the transaction.
- Checkpoint: write the committed metadata to its home locations.
- Reclaim: after those home writes are durable, reuse the transaction's journal space.
A successful buffered write() is not the same as a durable journal commit. Applications use operations such as fsync() and must check their results. Persisting a newly created or renamed directory entry can also require syncing its containing directory. The Linux fsync(2) documentation explains that boundary.
Recovery After Crash
Recovery examines the active journal, validates its records, and replays complete transactions that still need checkpointing. Incomplete transactions are discarded. Replay must tolerate a home write having already completed before the crash; restoring the same logged metadata value again should have the same result.
That is why the explorer's partial-checkpoint case ends with the same inode and bitmap as its after-commit case. The before-commit case keeps the old file, even though new payload bytes may already exist in an unreferenced block.
Recovery work depends on the active log and storage performance. Journaling reduces the need for a full consistency scan after an ordinary crash; it does not repair every form of corruption or eliminate filesystem repair tools.
Journaling in Different Filesystems
ext4
ext4 normally uses JBD2 with an internal journal, and can use a separate journal device. Its journal size depends on filesystem configuration. Fast commits can log a smaller description of eligible metadata changes; the explorer illustrates the conventional transaction path rather than that optimization. See the JBD2 documentation.
XFS
XFS logs filesystem metadata using its own mix of physical records, logical updates and intent records. It is not ext4 running in data=ordered mode. An external XFS log is also distinct from an XFS realtime data device, as described in xfs(5). The XFS logging design explains its transactions and recovery machinery.
NTFS
NTFS uses a transaction log and checkpoint information to recover filesystem consistency. Its USN change journal serves applications tracking changes and is a separate mechanism. Microsoft's NTFS overview describes recovery and when repair may still be necessary.
Btrfs / ZFS
Copy-on-write filesystems build changed structures in new locations and then publish a new committed tree. That does not exclude logging: Btrfs has a tree log, and ZFS has an intent log for synchronous operations. Their recovery paths differ from the conventional metadata journal shown here. Continue with the Copy-on-Write mechanism for the tree-update model.
Performance Impact
Journal Placement
A separate journal device can reduce contention between log writes and other I/O, especially on seek-bound storage. The effect depends on the device, queueing, workload and durability requirements; an external log is not automatically faster. Recovery depends on the journal device as well as the main data device.
Before tuning, inspect the filesystem and effective mount options for the path you care about:
# Inspect the filesystem containing the current directory.
findmnt -T . -o TARGET,SOURCE,FSTYPE,OPTIONS
Journal Size Tuning
More journal space can accommodate more outstanding metadata work before space pressure forces checkpointing. It does not directly change the periodic commit interval. The useful size depends on the workload; throughput and latency claims need measurements with the same durability settings.
For an ext4 filesystem, inspect the configured journal using its actual block-device path:
# Replace this placeholder with the ext4 device identified above.
sudo tune2fs -l /dev/your-ext4-device
This lists configuration; it does not monitor journal wraps. dumpe2fs -h provides another read-only view of an ext4 device's superblock and journal configuration. The tune2fs(8) manual documents journal options and their constraints.
Best Practices
- Choose a mode for its guarantees. Start from the filesystem's defaults and evaluate changes against a specific workload.
- Request application durability explicitly. Use the appropriate sync operations, check errors, and follow the application's recovery protocol.
- Keep storage ordering intact. Journal recovery assumes that the storage stack honors flush and ordering requirements.
- Measure before tuning. Compare latency and throughput under equivalent durability settings.
- Keep backups. A consistent filesystem can still contain unwanted changes or suffer device failure.
Journaling vs Copy-on-Write
| Aspect | Conventional journaling | Copy-on-write trees |
|---|---|---|
| Update strategy | Log changes, then update home locations | Build new blocks, then publish a new tree |
| Extra writes | Journal traffic; which bytes are duplicated depends on mode | New data and metadata paths; additional logging may apply |
| Crash recovery | Replay valid committed log records | Select committed roots and replay relevant logs |
| Snapshots | Not supplied by journaling itself; other layers may provide them | Shared tree blocks support snapshots, with later space costs |
| Application durability | Requires the appropriate sync protocol | Requires the appropriate sync protocol |
Neither technique guarantees that every recent application write survives. The distinction is how filesystem updates become recoverable. See the Btrfs design and OpenZFS copy-on-write design for how tree updates and logging coexist.
Related concepts
Follow filesystem copy-on-write from a changed block to a committed root. Explore snapshot sharing, space reclamation, and the trade-offs in Btrfs and ZFS.
Trace how Btrfs shares extents, creates snapshots, distributes copies, compresses writes, and repairs checksum errors when a good replica exists.
Explore ext4, the default Linux filesystem with journaling, extents, and proven reliability. Learn how ext4 protects your data.
Learn FAT32 and exFAT filesystems for cross-platform USB drives and SD cards. Understand file size limits and compatibility.
Explore Linux filesystems through interactive visuals. Learn VFS, compare ext4 vs Btrfs vs ZFS, and understand file operations.
Understand Linux inodes - the metadata structures behind every file. Learn about hard links, soft links, and inode limits.
