AI News
Real Time

Can ClickHouse Become the Real-Time Analytics Engine for AI Logs?

Over the past 18 months, few open-source data infrastructure companies have generated as much momentum as ClickHouse, rivaled perhaps only by Supabase...

Over the past 18 months, few open-source data infrastructure companies have generated as much momentum as ClickHouse, rivaled perhaps only by Supabase. ClickHouse Cloud's annual recurring revenue (ARR) mAIntained a 250% year-over-year growth rate through 2025, with third-party estimates placing it at roughly 160millionbyyearend,upfromapproximately15 million in mid-2024. Its paying cloud customer base has surpassed 3,000, up from 1,000, and in January 2026, the company closed a 400millionSeriesDroundata15 billion Valuation.

Yet CLIckHouse is not a new company. It began in 2009 as an internal project at the Russian search engine Yandex, was open-sourced in 2016, and only spun out for independent commercialization in 2021. Why would a columnar database originally built for real-time user analytics on a search engine experience a surge in demand during the AI boom?

The answer lies in an architectural alignment. ClickHouse excels at high-cardinality column scans, filtering, and aggregation queries on APPend-only event streams. LLM Inference logs—each API call generating dozens of fields including Token Consumption, latency, model version, GPU identifiers, and safety filtering—fit this workload profile exACTly. During the launch of claude 3 and 3.5, anthropic found its existing APM systems buckling under petabytes of daily log volume. The company ultimately migrated to ClickHouse, co-building an air-gapped private deployment with the ClickHouse team. OpenAI, meanwhile, runs its internal observability stack atop a multi-shard ClickHouse cluster spanning dozens of nodes. These AI labs, along with numerous leading AI application companies, have discovered that ClickHouse is one of the few engines capable of delivering millisecond-level queries on petabyte-scale data.

For ClickHouse as a company, the most significant recent shift is its product strategy. Through three key acquisitions, it has evolved from a Standalone OLAP engine into a broader data platform. In early 2025, it acquired HyperDX and launched ClickStack, directly entering the observability market dominated by players like Datadog, Splunk, and Elastic. In January 2026, it acquired Langfuse, securing a leading position in the open-source LLM observability space. That Same day, it also introduced a Postgres Service, bringing OLTP workloads into the fold. The logic behind these three acquisitions is consistent: ClickHouse's proprietary columnar stoRAGe engine serves as the foundation, with application-layer products layered on top through acquisition. The company is transitioning from an open-source engine vendor toward platform-level competition against snowflake, Databricks, and others.


01. What Happened in 2025

ClickHouse was originally designed by Yandex for real-time analytics on user behavior data. It has been open-source for a decade and saw widespread adoption among consumer internet companies between 2020 and 2022. It spun out of Yandex in 2021 to pursue commercial growth independently.

From mid-2025 through early 2026, ClickHouse rapidly completed three new funding rounds. Investor enthusiasm was driven by several milestone-level changes:

1. Platformization and TAM Expansion: From real-time OLAP into Observability

ClickHouse's architectural decisions make it particularly suited for aggregation queries on append-only event streams—high-velocity writes with minimal deletions or updates. LLM log data fits this pattern precisely. ClickHouse now holds open-source leadership positions across three markets:

  • Real-Time OLAP (TAM: $4–5 billion, core stronghold): Analytics databases capable of querying data within seconds of ingestion and returning results in milliseconds. This is ClickHouse's validated base and traditional use case.

  • Observability (TAM: ~$20 billion, second growth curve): Unified monitoring of logs, traces, and metrics. Following the HyperDX acquisition, ClickHouse officially launched ClickStack, directly competing against Datadog, Splunk, and Grafana.

  • AI observability (TAM: ~$2 billion, emerging category): Tracking token consumption and latency for every API call made by LLM applications. This market remains nascent and highly fragmented. ClickHouse's January 2026 acquisition of Langfuse gives it a leadership position as an open-source player.

All three product lines share ClickHouse's columnar storage engine underneath. This platform expansion follows an acquisition-driven strategy, stacking application-layer products atop the core engine to pursue vertical integration.

2. Leading AI Labs Shifted to ClickHouse

The data characteristics of AI Inference logs closely match ClickHouse's strengths. As leading AI labs scaled their Operations, daily log volumes reached petabyte scale, and their existing solutions either failed to scale or became prohibitively expensive. They began incorporating ClickHouse into their infrastructure stacks. After Anthropic's existing APM system broke down following the Claude 3.5 launch, the company migrated to ClickHouse, where a team of just three engineers now manages the entire observability infrastructure. openai partially moved away from an overly costly Datadog deployment in 2025, adopting ClickHouse as the underlying engine for its internally built observability solution.

3. Commercial Acceleration

ClickHouse's ARR grew from approximately 15millioninmid2024toanestimated160 million by the end of 2025—a 10x increase in 18 months. Paying customers grew 3x from 1,000 to over 3,000, while average contract value expanded 3–4x. In January 2025, the company implemented a ~30% price increase for ClickHouse Cloud. Former Atlassian CSO Kevin Egan joined as CRO, ending the era of purely product-led growth with no sales development representatives and initiating a more systematic enterprise sales motion.


02. ClickHouse's Products

ClickHouse currently has three core product lines:

ClickHouse Cloud

ClickHouse Cloud is the core offering: the open-source ClickHouse database delivered as a managed cloud service, designed to solve real-time aggregation queries on massive datasets. Three tiers serve everything from development and testing to large-scale enterprise deployments.

A typical use case: an AI company generates hundreds of thousands of inference log entries per second, each containing fields like token consumption, latency, GPU identifier, and model version. This data streams into ClickHouse Cloud via ClickPipes in real time. Engineers can then query a dashboard showing, for instance, "P99 latency for the Claude 3.5 model in us-east-1 over the last five minutes." The window from data ingestion to queryability is a few seconds, and queries themselves return in milliseconds. This "queryable immediately upon write, results in milliseconds" experience is architecturally unattainable for Snowflake or Databricks.

Two companion services complement this core offering:

  • ClickPipes (ETL/ELT Engine): A fully managed continuous data ingestion engine, originating from the 2024 PeerDB acquisition. It supports streaming data ingestion from a wide range of sources.

  • Postgres Service (OLTP Extension): Announced alongside the Series D round in January 2026, this is an enterprise-grade managed Postgres service built in partnership with Ubicloud. Currently in private preview (free), official pricing has not yet been announced.

This allows customers to have both an OLTP database (Postgres) and an OLAP database (ClickHouse) under a single ClickHouse Cloud bill and control plane. Two core integrations power this:

  1. pg_clickHouse Extension: Enables querying ClickHouse directly from Postgres. Every Postgres instance includes this extension, allowing developers to use SQL against ClickHouse data without application code changes. The extension performs automatic query pushdown to ClickHouse—in TPC-H benchmarks, 14 of 22 queries were fully pushed down, yielding over 60x performance improvements. Developers can use Postgres as a single query entry point while analytical queries execute on ClickHouse.

  2. Built-in CDC with Real-Time Replication: Based on the ClickPipes/PeerDB engine, this replicates transactional data from Postgres to ClickHouse with sub-second latency, eliminating the need for users to set up Debezium or Kafka Connect pipelines themselves.

Among open-source real-time OLAP database solutions, ClickHouse holds a dominant leadership position.

ClickHouse Cloud includes several exclusive features—SharedMergeTree, SharedCatalog, Lightweight UPDATEs, and serverless architecture—that are not available in the open-source version.

ClickStack

ClickStack is an open-source, unified observability platform composed of three components: the OpenTelemetry Collector (data ingestion) + ClickHouse (storage engine) + HyperDX UI (frontend). Launched following the HyperDX acquisition in 2025, it is available in a self-hosted version (open-source and free) and as Managed ClickStack (a managed service on ClickHouse Cloud, currently in beta).

Its four core capabilities/use cases:

  • Log Management: Real-time ingestion, storage, and search across all application and infrastructure logs. Engineers can use Lucene-style search syntax (e.g., (level:error AND service:payment-api AND latency:>500ms)) or write SQL for complex aggregation queries. Columnar compression reduces log storage costs by an order of magnitude compared to Elasticsearch or Splunk. OpenTelemetry-format log data can achieve up to 90% compression, eliminating the need for sampling or short retention Windows due to cost constraints. Petabyte-scale data can be retained in full.

  • Distributed Tracing: Tracks the complete journey of a user request across frontend, backend microservices, and databases. For an e-commerce order flowing through API gateway → Order Service → Inventory Service → Payment Service → Notification Service, ClickStack records the latency, status, and Metadata for each hop, allowing engineers to pinpoint exactly which method in which service caused a slow request.

  • Metrics Monitoring: Collects infrastructure metrics (CPU, mEMOry, disk, network) and custom business metrics (QPS, order conversion rates, token consumption rates). Supports dashboard construction and alert threshold configuration. Charts can be built using simple dropdown menus without learning query languages like PromQL.

  • Session Replay: Records actual user actions (clicks, scrolls, keystrokes) in the browser. When a user reports a bug, engineers can replay the complete interaction sequence, which is automatically correlated with the backend logs and traces generated during that session.

Compared to the similarly open-source Grafana LGTM Stack, ClickStack's primary advantage is cross-signal correlation. In traditional observability tools, logs, traces, and metrics are typically stored across separate backends (Grafana's Loki, Tempo, and Mimir operate independently). When an engineer detects a metric anomaly, they must manually toggle between three systems and align timestamps to pinpoint the issue. Because ClickStack stores all data in a single ClickHouse engine, engineers can click an error log, jump directly to the full distributed trace that produced it, and then navigate to relevant service metric dashboards for that time window—all without switching between multiple tabs. The path from anomaly detection to root cause identification is compressed by an order of magnitude.

Compared to Datadog, ClickStack can be 50% or more cost-effective. Datadog's pricing scales with host count and custom metric volume, causing monthly bills to inflate rapidly in Kubernetes environments. ClickStack's pricing is based on actual data consumption (storage + compute). Columnar compression rates of up to 90% mean storage costs are Dramatically lower than Elasticsearch or Splunk's inverted-index storage. The net result: at equivalent monitoring loads, ClickStack's cost is approximately 30–50% of Datadog and 15–20% of Splunk.

ClickStack's primary shortcomings center on ecosystem maturity, particularly its integration catalog, which lags far behind Datadog. It also currently lacks APM functionality. Still in beta, ClickStack needs to further strengthen RBAC and audit logging to meet compliance requirements.

ClickHouse has been used for observability long before ClickStack. Expert interviews reveal that Nike and Cisco previously used ClickHouse as a Splunk replacement for near-real-time infrastructure and network security monitoring, respectively. ClickStack represents a more fully productized delivery of these homegrown solutions built around ClickHouse.

Langfuse

Langfuse is an open-source LLM engineering platform acquired in January 2026 alongside the Series D round. Licensed under MIT, it covers the complete LLM application lifecycle from development to production. Prior to acquisition, it had over 2,000 paying customers, 19 Fortune 50 clients, more than 20,000 GitHub stars, and 26 million monthly SDK installations. Langfuse was already built on ClickHouse storage.

Its six core capabilities and use cases:

  • LLM Tracing: Tracks the full journey of a user request within a system: user query → retrieval of relevant documents (RAG) → Prompt construction → LLM invocation → post-processing → response. Each step's input, ouTPUt, latency, and token consumption are recorded. For Agents, Langfuse uses the concept of Sessions to trace multi-step workflows.

  • Cost Tracking: Automatically calculates token consumption and cost for every LLM call, with dimensional breakdowns by model, user, functional module, and time period.

  • Prompt Management: Centralized management, version control, and collaborative editing of all prompt templates. Each prompt version is automatically associated with production traces, enabling direct visibility into online performance (latency, cost, quality scores) for that version.

  • LLM Playground: Test prompts and model configurations directly within the Langfuse interface, viewing output results in real time.

  • Datasets: Create and manage test sets and benchmarks for continuous LLM application improvement, pre-deployment testing, and structured experimentation.

  • Eval: This is Langfuse's deepest functional module and its core differentiator relative to competitors. It supports flexible combination of four evaluation approaches: LLM-as-a-judge, human annotation, custom scoring functions, and retrieval evaluation.

Langfuse differentiates primarily through its evaluation capabilities and its open-source approach. Its product experience is very well-regarded within the community.


03. AI Tailwind?

ClickHouse itself is not an "AI-native" company. AI currently serves primarily as a business tailwind: AI inference logs are a data type that structurally matches ClickHouse's architecture, and leading AI labs rely on ClickHouse at production scale. The growth in AI-generated data is structural, driven by increasing dimensionality and complexity of telemetry data.

Beginning in the second half of 2025, ClickHouse introduced the concept of Agent-Facing Analytics. Built on three acquisitions—HyperDX, LibreChat, and Langfuse—the company has constructed a complete open-source Agentic Data Stack. This stack enables AI Agents to execute the full SQL query workflow on behalf of human users: an agent inspects schemas via MCP, understands business semantics through a business glossary, generates and executes SQL, and returns results to users in natural language.

ClickHouse itself deployed this architecture to build DWAINE (Data Warehouse AI Natural Expert) for its internal data warehouse. The system serves over 250 internal users, processes 200+ data query messages daily, handles approximately 70% of data warehouse query requests, and has reduced analyst workload by 50–70%. As of October 2025, it had processed 33 million LLM tokens.

On the other hand, when evaluating ClickHouse through a public-market software lens, several points challenge the AI beneficiary narrative:

  • ClickHouse does not do AI; it stores and queries AI-generated data. The company has no native AI/ML capabilities. It has competent vector search functionality but does not perform model training or inference. Its benefit pattern mirrors Datadog's: it does not enable customers to "do more with AI"; it simply helps them observe, in real time, what their AI systems are doing.

  • Agent-Facing Analytics represents the idea that AI agents will replace human users for BI operations, but at present there is no fundamental philosophical distinction from similar transformation attempts by Snowflake and others.

  • When leading customers like Anthropic and OpenAI use ClickHouse to build internal observability solutions, ClickHouse often does not sell its standard cloud-managed product. These deployments require co-created solutions, and OpenAI, in particular, is likely not a paying cloud customer.

ClickHouse's collaboration with Anthropic resembles a BYOC (Bring Your Own Cloud) model. The deployment took place in 2024, before ClickHouse Cloud's BYOC offering reached General availability.

Anthropic evaluated three approaches: open-source self-hosting (high operational burden requiring manual management of re-sharding, replicas, etc.), ClickHouse Cloud managed service (offering dynamic scaling and blob storage advantages but requiring deployment in ClickHouse's cloud), and a hybrid solution. Neither of the first two fully met requirements, as Anthropic mandated that all data remain within its own secure computing environment. The final approach: collaborating with the ClickHouse team to deploy a customized, air-gapped ClickHouse Cloud architecture within Anthropic's own infrastructure—a private, customized deployment of ClickHouse Cloud.

OpenAI cited ClickHouse's open-source nature, lack of vendor lock-in, and the ability to debug the source code as reasons for selection. OpenAI operates dozens of sharded ClickHouse clusters entirely in-house and is almost certainly not a ClickHouse Cloud customer.


04. ClickHouse's Go-To-Market

Traction

ClickHouse is undergoing a transition from pure PLG to a PLG + SLG hybrid model. Before 2025, the company had no sales development representatives, relying entirely on open-source users and inbound conversion, resulting in extremely low customer acquisition costs.

It currently operates three core sales motions:

  • Self-Serve: Self-service signup with monthly credit card billing. Developers can register directly on the website, create databases, and begin using the service without any interaction with sales.

  • Direct Enterprise Sales: Large customers typically purchase "ClickHouse Credits" (similar to Snowflake's capacity contracts) upfront to secure 15–40% discounts. This channel represents only 20–30% of customers but contributes over 50% of revenue. In July 2025, ClickHouse hired CRO Kevin Egan (formerly of Atlassian, Slack, Salesforce, Dropbox) to build an enterprise sales team—the necessary path from 100millionARRto1 billion ARR.

  • Cloud Marketplaces (AWS/GCP/Azure): Customers can purchase ClickHouse Cloud directly through AWS, GCP, or Azure Marketplaces, with billing consolidated into their cloud invoices. ClickHouse is currently categorized under the new "AI agents and Tools" category in the AWS Marketplace.

Historically, infrastructure companies that establish strong SLG practices experience revenue growth acceleration. MongoDB introduced a CRO around 100millionARR,thengrewrevenue19xoverthesubsequentsixtosevenyears.Datadogbuiltitsenterprisesalesteamaround300 million ARR and grew 8x over the next five years.

usage-based pricing

ClickHouse Cloud charges exclusively based on actual resource consumption. The bill comprises several dimensions:

  • Compute (estimated 65–75% of revenue): Billed per compute unit per hour, with per-minute granularity. On the Scale tier, compute costs approximately 0.69perunithour(oneunit=24GiBRAM+6vCPU).Forequivalentcomputepower,Snowflakechargesapproximately6.00 per hour, making ClickHouse roughly 88% cheaper. Services can auto-suspend when idle, incurring zero compute charges.

  • Storage (estimated 15–20% of revenue): Billed per compressed data volume per month. The Scale tier charges approximately 47perTBpermonth.WhilethenominalunitpriceisdoubleSnowflakes( 23/TB), ClickHouse's columnar compression typically achieves 90–98% ratios (Snowflake: ~50–75%), meaning ClickHouse storage is actually cheaper when normalized to raw data volume. One TB of raw data may only incur charges for 20–100 GB in ClickHouse.

  • data transfer/Egress (estimated 5–8% of revenue): Free before January 2025, subsequently priced at $115 per TiB. This represents a lock-in mechanism; once data volumes reach terabyte scale, the cost of migrating data becomes non-trivial.

Snowflake's gross margins exceed 70%. ClickHouse's blended gross margin may be slightly lower (its compute unit pricing is significantly cheaper), but because ClickHouse is natively developed in C++ with extreme optimization of every compute cycle, the same query workload consumes fewer underlying cloud resources (VMs, storage, network). ClickHouse Cloud is in its first pricing cycle: January 2025 brought a ~30% blended price increase plus the introduction of egress charges, and by July 2025 the transition period for legacy customers had concluded, with new pricing fully in effect.


05. Enterprise Sentiment

Every database's architectural decisions involve trade-offs. Analysis across 15 expert interviews reveals the following consensus on ClickHouse's strengths:

  • Unmatched Real-Time OLAP Performance: Customers have benchmarked speed advantages of 5–20x, with tens of billions of rows scanned in single-digit milliseconds. This was the highest-consensus dimension across all 15 experts (mean score +1.7). Competitor-side experts attempted to downplay this, but customer-reported quantitative data proved overwhelming.

  • Cost Advantages as the Primary Procurement Driver: Deutsche Bank reduced annual costs from 3millionto200,000 after migrating from kdb. Nike confirmed a 5x+ cost advantage over Splunk. Goldsky found storage costs of $47/TB/month significantly lower than Snowflake or BigQuery. This resonates strongly with mid-to-large organizations possessing engineering capabilities.

  • Structural Alignment with Modern Data Paradigms: The append-only, immutable event stream design structurally fits three high-growth scenarios: observability, financial time series, and product analytics. This is a structural advantage, not merely a functional one.

  • Petabyte-Scale Proven at Production: Goldsky (2 PB), Walmart (40 TB+), and Deutsche Bank (20+ years of time-series data) all validate ClickHouse's scalability. This was the second-highest consensus dimension (σ=0.67).

Expert feedback on ClickHouse's limitations/weaknesses includes:

  • Missing Platform Completeness as the Valuation Ceiling Determinant: No native AI/ML, limited ETL, no marketplace ecosystem. If ClickHouse remains positioned as a specialized tool, its TAM ceiling could be as low as 3–5% of total data analytics spend (Cisco VP estimate). The go-to-market execution of ClickPipes and ClickStack is a critical observation point.

  • Enterprise Readiness as the Largest Bottleneck to Cloud ARR Growth: Walmart explicitly stated that ClickHouse Cloud cannot pass their sensitive Data Security approval (SSP). Customer support has suffered from turnover and slow response times (Prefect). RBAC, IAM, and audit logging require customers to build their own solutions (Nike). Large enterprises remain stuck on open-source self-hosting. BYOC has been launched but awaits broad GA.

  • Moat Under Erosion: Goldsky's CTO warns that core advantages are being rapidly replicated by SingleStore, StarTree, and others, making products increasingly interchangeable. The most noteworthy signal: even highly satisfied customers are closely monitoring competitor pricing changes.

  • Polarizing Developer Experience Limiting GTM Ceiling: Deeply technical engineers view ClickHouse as a "paradise," while enterprise and non-technical users find the barrier to entry too high. The bottom-up adoption model inherently discourages CIO-level, top-down procurement.

Additionally, feedback from SMB IT decision-makers reveals a structural challenge: ClickHouse delivers exceptional performance but historically lacked auto-scaling, optimal compute-storage separation, and disaster recovery capabilities, making self-management highly complex. ClickHouse Cloud and Managed ClickStack solve these issues well, but akin to challenges faced by TiDB, some decision-makers harbor security concerns about using officially managed services due to ClickHouse's historical association with Yandex and Russia.

As Walmart's Senior Engineering Manager stated: "ClickHouse Cloud will never pass our SSP because it requires moving data to an external cloud." BYOC theoretically addresses this (data stays in the customer's own AWS account), but BYOC is currently GA only on AWS for the Enterprise tier. GCP and Azure support remains absent, and BYOC entails higher operational complexity and lower gross margins.


06. Team

  • Aaron Katz (CEO): Co-founder, former executive at Elastic and Salesforce. His Splunk background is strategically critical to ClickHouse's observability push; he personally lived through Elastic's journey from open-source search engine to a multi-billion-dollar observability platform. ClickStack directly targets the ELK Stack. Katz appears to be replicating Elastic's path but with a columnar engine offering a cost structure an order of magnitude lower.

  • Alexey Milovidov (CTO / Founder): The original creator of ClickHouse, who began its development at Yandex in 2009. He is among the world's foremost engineers in the columnar database space. GitHub commit history shows he remains the most active core contributor. A technical founder remaining as CTO with sustained high contribution levels is a strong positive signal for an engine-driven company.

  • Kevin Egan (CRO, joined July 2025): Former Atlassian CSO, Slack VP of Sales, with prior roles at Dropbox and Salesforce. Each chapter of his career represents a textbook case of PLG-to-Enterprise transformation. During his Atlassian tenure, revenue grew from 2billionto5 billion, with the core mechanism being the layering of enterprise sales atop a sustained self-service growth engine. His Slack experience validates his ability to drive large-customer expansion under a usage-based pricing model.


07. Valuation

At a 15billionvaluationagainst160 million ARR, ClickHouse trades at approximately 94x multiple. If ClickHouse can match Grafana's 2025 ARR trajectory and sustain high growth rates, this valuation could compress to approximately 11x by 2028.

For context: Grafana reached 400millionARRbySeptember2025ata9 billion valuation, corresponding to 22.5x. Databricks' 54billionARRand134 billion valuation correspond to approximately 24x. Public-market comparables like Snowflake and Datadog trade in the 8–15x range.


08. Key Takeaways

  • As an open-source data engine, ClickHouse's standout speed and cost advantages make it exceptionally well-suited for analytics and observability use cases in the AI era.

  • Beyond Databricks, ClickHouse is among the most complete players for the data infrastructure platformization thesis in the AI era. Its primary battleground is the mid-market and early enterprise segment, with current use cases concentrated in analytics and observability. Growth in token consumption drives AI inference demand in both areas.

  • As a private-market competitor, ClickHouse represents the largest obstacle to Snowflake and Databricks as they seek to expand into the real-time OLAP market. It is also an emerging threat to observability incumbents like Datadog, Splunk, and Elastic.

  • Fully open-source with a Yandex heritage, these factors have historically constrained ClickHouse's commercial progress. While the company has significantly expanded its TAM, its absolute market share across OLAP, observability, and AI observability markets remains low.

  • Currently, players including both Snowflake and ClickHouse are heavily emphasizing Agent-Facing Analytics, though this remains primarily focused on enabling indiVidual agents to replace human analysts in SQL-writing tasks. The evolution of analytics demand as agents progressively replace deeper human behaviors warrants ongoing observation.


09. Appendix: ClickHouse's Technical Characteristics

For most audiences, "ClickHouse" registers not as a company but as an open-source, columnar, real-time analytical distributed database using standard SQL, supporting distributed deployment, and capable of handling petabyte-scale data daily. Below is an overview of ClickHouse's core technical design and use cases as an open-source database.

Core Workload Type: Real-Time Analytics

OLAP can be divided into offline and real-time categories, each with fundamentally different technical requirements and business needs. Real-time analytics demands that a system simultaneously (1) ingest new data writes at high throughput and (2) make that data immediately queryable upon write. Across the 15 expert interviews in the appendix, the strongest consensus on ClickHouse was its query performance (scanning tens of billions of rows in single-digit milliseconds).

Technical Design: Columnar Storage + Vectorized Execution + MergeTree

Expert feedback indicates ClickHouse is 5–13x faster than Snowflake and 5–19x faster than Elasticsearch on real-time analytical queries, with an order-of-magnitude cost advantage. These advantages derive from its technical design:

  1. Columnar Storage: Traditional databases (MySQL, PostgreSQL) store data by row, with all fields for a given row stored contiguously on disk. When answering a query like "What is the average spend of 100 million users?", row-based storage must read every field of every row, even though only the "spend" column is needed. If a table has 100 columns, 100x more data than necessary gets read. ClickHouse stores data by column, so a query involving only three columns reads only 3% of the data. Furthermore, same-column data shares identical types and adjacent values are often similar, yielding excellent compression. ClickHouse's typical compression ratio is 10:1 to 20:1; a 1 TB raw dataset may occupy only 50–100 GB after compression (compared to Elasticsearch's ~1.5:1). This makes ClickHouse ideal for wide tables with many columns, avoiding the table-splitting and JOIN patterns typical of MySQL.

  2. Vectorized Execution: Most databases process data row by row. ClickHouse operates on blocks (typically 8,192 rows per block), applying operations to entire blocks simultaneously, fully leverAGIng modern CPU SIMD instruction sets. While many engines choose either columnar storage or vectorized execution, ClickHouse was an early open-source database to implement both—a combination that later became standard for modern analytical databases.

  3. MergeTree Engine: ClickHouse's core storage engine is designed around a philosophy: data is rarely modified or deleted after write; it is primarily continuously appended. Each write is organized into a small "part" rapidly flushed to disk, with background Threads aSynchronously merging parts. This makes writes extremely efficient, at the cost of relatively cumbersome deletions and updates.

Beyond these three characteristics, the ClickHouse open-source maintenance team and ecosystem developers demonstrate exceptional engineering velocity, continuously forking and re-forking the engine to provide optimized implementations for specific scenarios, data types, cardinalities, and more. ClickHouse itself evolves rapidly, making it difficult for forks (e.g., ByteDance's ByConity and ByteHouse, TiDB's TiFlash) to maintain comparable engineering pace.

The above technical decisions make ClickHouse nearly unbeatable in its sweet spot—"large-scale, append-only, real-time aggregation queries"—but inherently unsuited for frequent single-row updates/deletes, OLTP transactional operations, or simple queries on small datasets. This append-only characteristic aligns perfectly with observability data (append-only event streams), which is why ClickHouse is increasingly recognized as an observability platform beyond just OLAP.

Use Cases: From Internet to AI

ClickHouse was born at Yandex, Russia's largest search engine, originally designed for real-time website behavior analytics—analyzing billions of daily user clicks, searches, and page views. This workload type first exploded in the consumer internet industry. China represents one of the largest ClickHouse user bases; ByteDance was already operating the country's largest ClickHouse clusters by 2022, subsequently developing significant forks and customizations. Its use cases progressed from T+1 offline analysis of user behavior toward real-time BI, real-time feedback for A/B testing, and real-time advertising performance monitoring. Where engineering teams could internally address consistency challenges, ClickHouse also found use in low-latency operational product scenarios, such as coupon eligibility decisions.

In theory, any workload characterized by "large-scale, append-only, real-time aggregation queries" suits ClickHouse—including observability log data, financial time-series data, and IoT sensor data. Beyond user behavior analytics, another clearly growing ClickHouse domain is observability and infrastructure monitoring. For instance, Nike's Observability Platform Director disclosed that they use ClickHouse as a Splunk replacement for near-real-time infrastructure monitoring; Splunk's annual costs were in the tens of millions of dollars, while ClickHouse was over 5x cheaper. Cisco's VP of Data Analytics and AI uses it for network security threat detection.

The AI era has introduced massive volumes of structured log data. Every inference request theoretically generates dozens of log fields: token consumption, model version, inference latency, GPU cluster identifier, safety filtering results, cost attribution, and more. Consequently, companies across the AI ecosystem have begun adopting ClickHouse.

★★★★★
★★★★★
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!