Back to Blog
Education

ClickHouse® Storage Architecture: Hot Data, Object Storage and Merge Pressure

April WongApril Wong
5 min read
ClickHouse® Storage Architecture: Hot Data, Object Storage and Merge Pressure

ClickHouse® storage design is not a choice between putting everything on fast disks or putting everything in a bucket. Separate three decisions: where authoritative table data lives, which data needs a fast serving path, and how the system will recover. Object storage can hold queryable data, not just archives. Block storage can support an active database, not just a cache. Choose the combination against your query, ingestion and recovery targets.

TL;DR

  • Distinguish native MergeTree storage, external data-lake files and backups. A bucket can serve different roles with different recovery requirements.
  • Define hot data by repeated access and latency requirements, not simply by its age.
  • Budget for merges, cache population, metadata and temporary work alongside retained table bytes.
  • Test cold reads, concurrent ingestion and recovery before accepting a cheaper storage layout.

Technical sources reviewed 18 November 2026. The procedure below is a proposed evaluation, not a benchmark performed by Nirvana Labs.

What does “ClickHouse on object storage” actually mean?

Start by naming the architecture. These are related but not interchangeable:

Pattern What the application reads Decision to make
Native tables on mounted storage ClickHouse-managed table parts Can the storage sustain this database's reads and background writes?
Native tables on an object-storage disk ClickHouse-managed table parts through the configured storage layer What metadata, cache and recovery design accompanies the bucket?
Queries over external files Files such as Parquet, accessed through a table function or lake integration Is direct file querying sufficient, or should frequently served data be ingested into native tables?
Backups in object storage A recovery copy, not the normal serving path Can you restore the required data and configuration within the recovery target?

The official S3 table-function documentation describes reading and writing external files, including streaming data into a destination table. That is different from configuring an object-storage disk for a MergeTree table.

ClickHouse Cloud is another specific architecture. Its stateless-compute explanation describes SharedMergeTree coordinating shared object storage through metadata in Keeper. Do not assume that pointing a self-managed server at an S3-compatible endpoint reproduces that service's replication, metadata or operational design.

Which data should stay on the fast path?

Ask what readers repeatedly touch. Yesterday's incident data may be hot during an investigation; today's raw logs may barely be queried. A useful working set includes the columns and ranges needed by the actual query mix, not necessarily every byte in the most recent partition.

For a self-managed deployment, storage policies can organize disks into volumes and control placement. MergeTree moves whole parts, not arbitrary individual rows. A TTL movement rule therefore depends on the part's contents, not just a calendar intention. The MergeTree reference explains these policy and movement boundaries.

Caching is a separate lever. ClickHouse supports a local filesystem cache over remote storage. The cache has its own size and eviction settings; write caching is configurable and should not be assumed from a read-cache configuration. Check your deployed version against the external-disk reference.

A recent example shows why “warm” needs a precise definition. The 26.10 release, published 6 November 2026, introduced an experimental in-memory columns cache for deserialized data, initially supporting wide MergeTree parts. That is not the same as a filesystem cache of remote bytes, and it is not a reason to enable an experimental feature blindly. Record which caches are enabled when comparing runs.

Why does merge pressure change the storage calculation?

Incoming bytes are only part of the work. Background merges read existing parts, combine data and write new parts. Old parts remain temporarily before deletion. CPU, memory and storage activity all participate in this process, as the official part-merges guide explains.

That creates two practical requirements. First, a capacity plan needs room for operations in flight, not just the final compressed dataset. Second, a throughput test must include maintenance work while readers and producers are active. An idle read test does not establish that the same layout can sustain ingestion and dashboard traffic together.

If many tiny inserts are creating avoidable work, address that first using our batching and merge-pressure guide. This article addresses the next decision: where to place and serve the remaining workload after the ingestion pattern is understood.

Before changing placement, capture the deployed configuration with read-only diagnostics:

SELECT version();
SHOW CREATE TABLE analytics.events;
SELECT * FROM system.disks;
SELECT * FROM system.storage_policies;

Replace the example table name. Review configuration output privately before sharing it. The system.disks and system.storage_policies references describe these views. They are a configuration inventory, not proof of sustained performance or a complete cluster-wide measurement.

What must survive when a node or storage path fails?

Write down the authoritative state before writing the recovery runbook. In a self-managed object-storage configuration, the default local metadata mode stores mappings to remote files locally. A bucket containing data does not, by itself, prove that a replacement server has everything needed to interpret it. Other metadata modes have different behavior; verify the exact configuration in the external-disk documentation.

Separate three tests: replacing a compute node, recovering missing metadata, and restoring after an erroneous deletion. They are different incidents. ClickHouse's backup guidance explicitly distinguishes replication from protection against human error, because a mistake can affect every replica.

For each test, record the last acknowledged input recovered, missing or duplicated logical records, elapsed time until representative queries work, and time until normal latency returns. A server accepting connections is an intermediate milestone. A recoverable database serving the expected data is the outcome.

How should you compare block, object and hybrid layouts?

Use one representative dataset and preserve schema, sorting key, partitioning, query mix, concurrency and producer behavior. The following acceptance matrix is a starting point, not a claim that any layout wins:

Test Keep visible Decision it informs
First read on a fresh test node Query p95/p99, remote bytes, errors Whether cold access meets the user-facing target
Repeated representative queries Cache behavior and memory use Whether the useful working set benefits from caching
Queries during sustained ingestion Freshness, latency, active parts, merge activity Whether serving and maintenance can coexist
Burst followed by normal traffic Time for the backlog to drain Whether the layout recovers from peak demand
Restore or node replacement Correctness and time to useful service Whether the recovery plan matches the business requirement

Do not clear production caches to manufacture a cold test. Use an isolated evaluation environment and document its starting state. Compare complete costs: storage, requests where applicable, transfer, compute, cache capacity, backups and operational effort. A lower retained-byte price is useful only if the full design meets the workload's targets.

For the mounted-storage path, Nirvana's high performance block storage and Nirvana Kubernetes Service documentation provide a starting point for infrastructure evaluation. Keep database design, backup ownership and recovery testing explicit. This guide does not establish compatibility, performance or availability for an untested object-storage integration.

FAQ

Is object storage only for cold ClickHouse data?

No. It can be the authoritative store in an architecture designed for it, with an appropriate cache and metadata layer. “Cold” describes access behavior, not a universal restriction of the storage category.

Will a larger cache fix every slow query?

No. A slow query may be limited by CPU, memory, schema, concurrency or work that cannot be reused. Compare the same query under documented cache states before assigning the cause.

Can I delete a local disk after moving data into a bucket?

Not safely from that fact alone. Confirm whether it still holds metadata, other tables or required configuration. Test recovery in a separate environment before removing anything.

Does replication replace a backup?

No. Replication and backup solve different failures. Keep an independently tested restore procedure for erroneous changes as well as a plan for infrastructure failure.

About Nirvana Labs

Nirvana Labs is a high performance storage cloud built for blockchain, AI and databases: demanding, real-time, stateful workloads. Its high performance block storage, compute, private networking, Nirvana Kubernetes Service and Agent Sandboxes help teams build and operate data-intensive applications.

Learn more at Nirvana Labs.

Nirvana Cloud | Pricing | Blog | Docs | Changelog | LinkedIn | Twitter | Telegram | YouTube

Powering AI, blockchain, and
databases

Talk to Sales