How to Benchmark fsync Latency for a Cloud Database

Peak IOPS do not tell you how long a durable database commit takes. A transaction may pass through the database engine, write-ahead log, filesystem, kernel and storage service before the client receives an acknowledgement. Benchmarking only one layer can make a fast-looking system fail the actual workload.

TL;DR

• Measure raw synchronization latency, database WAL synchronization and end-to-end transaction latency as separate layers.
• Keep durability enabled during the baseline; changing fsync or synchronous commit changes the question rather than answering it.
• Report distributions, especially p95 and p99, under idle and mixed load, not only an average.
• Run every destructive or write-heavy test against dedicated test files and isolated database instances, never live data or WAL paths.

What does fsync actually measure?

On a POSIX system, fsync() asks the operating system to flush a file’s modified data and metadata to the storage device. fdatasync() focuses on the data and metadata required for later retrieval. The call boundary is important, but it is not the entire database commit path.

A database can group several transactions into one log flush, choose different WAL synchronization methods, or wait for replication before acknowledging a commit. Filesystem mount options, virtualization and the cloud block-storage implementation can also affect the tail. The fio documentation defines its synchronization controls; your database documentation defines its separate durability contract.

Which benchmark layers should you run?

Use three layers so that a regression has somewhere to point.

• Storage and filesystem: a single-job fio synchronization test against a dedicated file reveals the latency distribution seen at the file interface.
• Database WAL: a database-native tool such as PostgreSQL pg_test_fsync compares the synchronization methods supported on that system.
• Transactions: a representative workload such as pgbench or an application replay shows what clients experience after database concurrency, grouping and locks.

Record the machine type, filesystem, mount options, volume configuration, dataset size, queue depth and test duration. Without that context, results are anecdotes rather than a comparison.

How do you build a safe fio baseline?

Create a dedicated test directory on the target filesystem and confirm that no production path is involved. A starting point for a small synchronous-write probe is:

fio --name=sync-latency   --filename=/mnt/benchmark/fsync.test   --size=4G --rw=write --bs=4k   --ioengine=psync --iodepth=1   --direct=0 --fdatasync=1   --time_based=1 --runtime=120 --ramp_time=15   --percentile_list=50:95:99:99.9   --group_reporting

This is an example configuration, not a universal standard. Buffered writes plus fdatasync=1 make every completed write pay a synchronization cost through the filesystem path. Increase the file size beyond effective cache and extend the runtime for the system under test. Do not point fio at a raw device, database data directory or WAL directory unless you have intentionally provisioned a disposable benchmark target.

Run the same command when the host is otherwise idle, then during the background work that matters: checkpoints, compaction, snapshots or backup reads. Keep the fio parameters constant. Inspect the separate synchronization latency statistics as well as write completion latency and throughput. With buffered writes followed by fdatasync, write clat alone is not the durable-operation latency. Do not add independently calculated write and sync percentiles and label the sum transaction p99. Measure the complete operation at the application boundary.

How do you validate the real database commit path?

For PostgreSQL, run pg_test_fsync on the intended filesystem to compare supported WAL synchronization methods. PostgreSQL explicitly notes that this tool reports average file synchronization time and does not predict database transaction throughput. Treat it as a diagnostic layer, not the final verdict.

Then benchmark transactions with pgbench or a representative application replay. Keep fsync=on and use the durable synchronous_commit=on baseline unless the production durability policy says otherwise. Report client count, run duration, scale factor, transaction rate and latency percentiles. Re-run after a restart to distinguish a warmed cache from repeatable behavior.

If the application uses MySQL, ClickHouse® or another database, use its documented durability settings and workload tool. Do not translate PostgreSQL flags by analogy. For ClickHouse®, also separate synchronous inserts from asynchronous insert settings and background merge pressure; those are different paths.

How should you interpret tails under contention?

Start with the median to understand the common case, but make the decision from the worst latency the application can tolerate. In an illustrative example, a database with a 2 ms median and 200 ms p99 can still breach a request-level service objective when commits sit on the critical path.

Correlate p95 and p99 spikes with device latency, CPU steal, filesystem pressure, WAL bytes, checkpoint activity and queue depth. Repeat the test across several windows. A one-minute run can miss periodic work; one unusually good run can be as misleading as one unusually bad run.

Also separate documented behavior from measured results. Vendor IOPS limits, included performance and volume-size rules describe the configuration envelope. Only your controlled test on a specified configuration measures the application path.

Where Nirvana fits

Nirvana high performance block storage is designed for stateful, latency-sensitive workloads. The evaluation method above lets teams test its durable write path with the same fio, database and application layers they use elsewhere. Keep the files, workload and durability settings identical; publish the complete configuration with any result.

For adjacent database work, see keeping p99 latency in check during backups and the ClickHouse® small-insert and merge-pressure guide.

FAQ

Is fsync latency the same as storage write latency?

No. It includes the operating system and filesystem path and may include device cache flush behavior. Database commit latency adds more layers.

Should I disable fsync to see peak throughput?

Not for a durable baseline. Disabling durability can answer a capacity question, but it does not measure the system you rely on after a crash.

How long should the benchmark run?

Long enough to include periodic background work and reach a stable distribution. Use several repeatable windows rather than one short sample, and disclose the duration.

Documentation reviewed October 26, 2026. Commands are starting points for isolated test environments; no benchmark result is claimed here.

About Nirvana Labs

Nirvana Labs is a high-performance storage cloud built for blockchain, AI and databases: demanding, real-time, stateful workloads. Its high performance block storage, compute, private networking, Nirvana Kubernetes Service and Agent Sandboxes help teams build and operate data-intensive applications.

Learn more at Nirvana Labs.

Nirvana Cloud | Pricing | Blog | Docs | Changelog | LinkedIn | Twitter | Telegram | YouTube