Chaos and Order

💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

Introduction — What the 72-Hour Number Actually Is
Where 432 Comes From
From a Hard Limit to a Soft One
72 Hours Is the Best Case
What Happens When You Hit the Limit
Why Not Just Raise It — the Object-Storage Bill
The Compactor Is the Answer, and the Compactor Is Paid
- 2026: Deletion Becomes the Compactor's Job Too
The Real News of 2026 — A New Engine on Top of Parquet
What the Preview Is Worth — Honestly
So, When to Use It, and When Not To
Closing
References

Introduction — What the 72-Hour Number Actually Is

Anyone who has evaluated InfluxDB 3 has probably run across the line "the open-source Core edition can only query the last 72 hours" somewhere. It shows up repeatedly in community forums and in comparison posts.

But read the source, and there is no code that counts 72 hours. The logic to check a time range simply does not exist. What does exist is exactly one thing — a cap on the number of Parquet files a single query is allowed to touch. The default is 432.

This distinction is not wordplay. The fact that the limit's unit is files, not time, explains the limit's character, when it triggers earlier, and why it exists in the first place. And that answer leads straight to one architectural decision — the decision to build the storage layer on Parquet.

This post reads that bill. The landscape across the whole time-series database camp was covered in Time Series Databases 2026 Deep Dive; here I dig into InfluxDB 3 alone, in depth.

Where 432 Comes From

InfluxDB 3 Core is published under MIT or Apache-2.0 in the influxdata/influxdb repository. So this isn't a question you need to guess at — you can just read it.

First, the flag definition. The comment in influxdb3/src/commands/serve.rs reads:

/// Set the limit for number of parquet files allowed in a query. Defaults
/// to 432 which is about 3 days worth of files using default settings.
/// This number can be increased to allow more files to be queried, but
/// query performance will likely suffer, RAM usage will spike, and the
/// process might be OOM killed as a result. It would be better to specify
/// smaller time ranges if possible in a query.
#[clap(long = "query-file-limit", env = "INFLUXDB3_QUERY_FILE_LIMIT", action)]
pub query_file_limit: Option<usize>,

The flag is an Option type, and when no value is given, the default is hard-coded in influxdb3_write/src/write_buffer/mod.rs. In release tag v3.10.0, that's line 437.

query_file_limit: query_file_limit.unwrap_or(432),

So why 432? InfluxDB 3 buckets incoming data by timestamp into fixed-length time blocks and flushes each as a Parquet file. This block is called gen1, and its length is set by --gen1-duration.

/// Duration that the Parquet files get arranged into. The data timestamps will land each
/// row into a file of this duration. 1m, 5m, and 10m are supported. These are known as
/// "generation 1" files. The compactor in Pro can compact these into larger and longer
/// generations.
#[clap(long = "gen1-duration", env = "INFLUXDB3_GEN1_DURATION", default_value = "10m", action)]
pub gen1_duration: Gen1Duration,

The default is 10 minutes, and only three values are allowed: 1, 5, or 10 minutes. In the code, too, Duration::from_secs(600) is the default, and the parser only accepts 60, 300, or 600 seconds.

Now the arithmetic lines up.

gen1 block = 10 minutes
files per hour = 60 / 10 = 6
files per day  = 6 x 24 = 144
432 / 144 = 3 days = 72 hours

In other words, 72 hours is not an input — it's a derived value. It's simply the result of multiplying the 432-file cap by the 10-minute block length. The official docs say the same thing — with the default of 432 and the default gen1-duration of 10 minutes, a query can reach up to 72 hours' worth of data.

What happens if you lower --gen1-duration to 1 minute? 432 files becomes 432 minutes, about 7.2 hours. Same 432, but the queryable window shrinks to a tenth. That's the practical meaning of "it's a file limit, not a time limit."

From a Hard Limit to a Soft One

It really was a time limit at first. Early InfluxDB 3 Core had a hard 72-hour limit on both queries and writes, and the change that removed it is PR #25890. By the metadata from the GitHub API, it was opened on January 21, 2025 and merged on January 23.

The PR description is practically a summary of this post.

Remove Core's 72-hour query/write limit
Instead, cap queries by the default number of Parquet files. 432 was chosen because it corresponds to roughly 72 hours under the default gen1 time-block settings
This file cap can be raised, but the help text and error messages warn that query performance will degrade
If you hit this error, use a narrower time range where possible

And the last line is the crux — remove the hard limit and replace it with a soft limit users can opt into at the cost of performance, but if they can't accept that cost, they're pointed toward Enterprise, which ships with a built-in compactor.

That was written in January 2025, and as of July 2026 it still holds.

72 Hours Is the Best Case

This is where practitioners tend to be surprised. The docs state that multiplying 432 by 10 minutes yields 72 hours, then immediately attach a caveat — it can be shorter, depending on whether all the data in a given 10-minute block was ingested at the same time.

Why becomes clear from the code that builds file paths, in influxdb3_write/src/paths.rs.

{host_prefix}/dbs/{db_id}/{table_id}/{YYYY-MM-DD}/{HH-MM}/{wal_seq}.parquet

Everything up to the directory is the gen1 block (date + hour-minute), and the file name is the WAL sequence number. That means multiple files created under different WAL sequences can coexist within a single 10-minute block. If a snapshot happens twice for the same 10 minutes, you get two files, and that block now eats 2 of the 432-file budget.

Under default settings, this is tuned not to happen often. The default --wal-flush-interval is 1 second, the default --wal-snapshot-size is 600, and per the flag description, multiplying the two sets the snapshot cadence. 600 seconds is 10 minutes, so the snapshot cadence lines up exactly with the gen1 block length, yielding one file per block.

The problem is data doesn't always arrive so obediently. Backfilling to a past point in time, or late-arriving data landing in a block that's already been snapshotted, adds an extra file to that block. That burns through the 432-file budget faster than 72 hours. The more backfill-heavy the workload, the more "72 hours" becomes a lie.

To sum up: 72 hours is not a guarantee, it's a ceiling — and even then, only the ceiling for data that arrives strictly in real time.

What Happens When You Hit the Limit

The enforcement point is where write_buffer/mod.rs counts the file list. The logic itself fits in three lines — gather the Parquet files that pass the filter for that table, and if the count exceeds the cap, throw a DataFusion error and stop.

Three things are worth flagging here.

First, the limit is per table. Because the file list is filtered by database ID and table ID, 432 is not a budget for "the whole query" but for "the single table the query touches."

Second, this is a rejection at planning time, not a failure after scanning. It doesn't open the files and then slow down — it counts the files and refuses to run at all. There are no partial results.

Third, there's a minor bug in the error message. The value passed into the message's format placeholder is not actually the number of files the query would have scanned — it's the cap itself.

format!(
    "Query would scan {} Parquet files, exceeding the file limit. \
     InfluxDB 3 Core caps file access to prevent performance degradation \
     and memory issues. Use a narrower time range, or increase the limit \
     with --query-file-limit ...",
    self.query_file_limit
)

There's only one argument, and its value is the cap. So even a query that would scan 5,000 files always prints "Query would scan 432 Parquet files." The error doesn't tell you how much you exceeded it by. For anyone trying to gauge how much to shrink their time range, that's a fairly unhelpful behavior.

The message body doesn't recommend raising the cap. It steers you toward a narrower time range, or toward Enterprise, which ships with the compactor. The error message even includes the note that non-commercial/home use is free and there's a free trial for commercial evaluation.

Why Not Just Raise It — the Object-Storage Bill

Could you just raise --query-file-limit to 4320 and query 30 days? You could. But the docs list side effects.

Degraded performance for queries that read more Parquet files
Increased memory use
A chance the influxdb3 process gets OOM-killed
If you're on object storage, a large number of GET requests to access the data — in the docs' own words, up to 2 per file

That last item is the quiet killer. Even at the default of 432, a single query can issue up to 864 GETs. Raise it to 4320, and every time a dashboard auto-refreshes, a single query can issue up to 8,640 GETs. S3 GET pricing is billed per 1,000 requests, so before this is a performance problem, it's a billing problem.

And this is a direct consequence of Parquet's design. Reading one Parquet file means reading the footer metadata first, then reading the row groups you need. The smaller and more numerous the files, the worse the ratio of metadata round-trips to actual data. A structure that drops one file every 10 minutes is exactly that worst case.

The docs' recommendation is candid — keep the default settings and query shorter time ranges. If a query touching more than an hour needs a longer range or faster performance, use Enterprise.

The Compactor Is the Answer, and the Compactor Is Paid

The correct fix for the problem of small immutable files piling up is compaction — rewriting small files into larger files spanning longer time ranges, reducing file count, sorting by series, and attaching indexes.

InfluxDB 3 has that compactor. Only in Enterprise.

This shows up directly in the docs' own structure. The example configuration for spinning up a dedicated compactor node with --mode=compact sits inside an Enterprise-only block in the docs source, and the mode option itself is classified as an Enterprise-only item. Core has no concept of a compactor node at all.

The Core docs' opening sentence says this obliquely — InfluxDB 3 Core is an open-source time-series database designed and optimized for real-time and recent data. That's an architectural description before it's marketing copy. Without a compactor, you can only see recent data, so it's positioned for recent data.

The same pattern repeats elsewhere. The performance numbers Core's README boasts about — sub-10ms last-value queries, sub-30ms distinct metadata — aren't general query performance; they're the performance of the last-value cache and the distinct-value cache. Both caches are separate structures bolted on in memory. They were built separately because a point lookup like "the last value for this series" can't be fast on top of a Parquet layout, and the very fact that a cache was needed is evidence that the storage layer is weak at that question.

2026: Deletion Becomes the Compactor's Job Too

v3.10.0, released June 17, 2026, brought row-level deletes to Enterprise. But the implementation repeats this post's theme exactly.

Per the docs, deletion is asynchronous. Running influxdb3 delete rows records a delete request and stores it in object storage, and the request is applied when the compactor next rewrites that data. By default, the delete may not take effect for up to 24 hours after the request, and that delay is tuned with the --pt-row-delete-min-age flag. The docs are explicit — because deletion is applied during compaction, rows remain queryable until the compactor processes the request, and you should not expect rows to vanish immediately just because the command returned.

The predicate only supports tag equality comparisons. You cannot delete by field value.

And v3.10.0's known-issues list for the docs includes this — after row deletion is reported "complete," rows in the not-yet-compacted ingest tail can survive and keep showing up in queries. The workaround is to re-issue the delete request after that data has been compacted and verify the row count.

This isn't an accidental bug — it's the logical consequence of the design. If deletion is implemented as file rewriting, then deletion doesn't exist in regions that haven't been rewritten yet. Parquet files are immutable.

What about Core? In the official docs repository, Core's admin docs list has no delete-data.md at all. It's Enterprise-only. No compactor means no way to delete rows. If you're putting data that draws GDPR deletion requests into open-source Core, it's worth checking on this now.

The Real News of 2026 — A New Engine on Top of Parquet

That's the background. Now for this year's news.

In v3.9.0, released April 2, 2026, InfluxData opened a performance upgrade preview (beta). Turned on with the --use-pacha-tree flag, it includes, in the release notes' own words, a new columnar file format (.pt files), automatic Parquet migration with a hybrid query mode, column families for wide-table I/O, and bounded compaction.

In short, InfluxDB is partly stepping off of Parquet. v3.0.0, InfluxDB 3's first GA release, shipped April 16, 2025 — so this is roughly a year after GA.

You don't have to guess why. The vendor wrote it down directly. The preview docs' "Why these upgrades" section opens like this — the existing InfluxDB 3 storage layer uses Apache Parquet and is optimized for analytical workloads. And customers running high-cardinality, wide-schema, query-intensive workloads need better single-series query performance, more predictable resource usage, and the schema flexibility that made InfluxDB v1 and v2 popular.

That last phrase is worth rereading. It's a sentence about reclaiming what made v1 and v2 popular. v1 and v2's storage engine was TSM, built specifically for time series — and TSM is exactly what 3.0 threw away.

The description of the .pt format makes that regression clearer still. Per the docs, data within a file is sorted by column-family key, then series key, then timestamp, and uses type-specific compression algorithms matched to the data's characteristics — delta-delta RLE for timestamps, Gorilla encoding for floats, dictionary encoding for low-cardinality strings, and so on.

Gorilla stands out here. Check the Parquet format spec's encoding list directly, and that's the complete set: PLAIN, dictionary (PLAIN_DICTIONARY / RLE_DICTIONARY), RLE, BIT_PACKED (deprecated), DELTA_BINARY_PACKED, DELTA_LENGTH_BYTE_ARRAY, DELTA_BYTE_ARRAY, and BYTE_STREAM_SPLIT. The XOR family of float encodings — Gorilla — is not in the spec. Using the standard technique for time-series float compression meant stepping outside Parquet.

On compression ratio, the docs state it typically achieves 5–20x. But this is a vendor's own claim, and the docs don't say against which dataset, which schema, or which baseline. It isn't even specified whether the comparison is against Parquet or raw line protocol, so this figure shouldn't be used for capacity planning. Likewise, the phrase "single-digit-millisecond response for high-selectivity time-series queries" doesn't disclose its measurement conditions. It's safer to read it as a target than a measurement.

Column families are specified using a :: separator in the line-protocol field name. Whatever comes before the first :: is the family name, and unspecified fields are grouped 100 at a time into auto-generated families. The structure means that when you query only specific fields in a wide table, you skip reading the other family blocks entirely.

Compaction changes too. Per the docs, compacted data is organized into 24-hour UTC windows, Gen0 files progress from L1 through L4, and it runs continuously in the background within a byte-based memory budget (default: half of system RAM).

What the Preview Is Worth — Honestly

Before getting excited, let me carry over the warnings the docs themselves attach.

It's beta, and production use is off the table. The docs' warning box states plainly that this preview is offered as beta to Enterprise trial and paid customers, is subject to change, and must not be used for production workloads. It says explicitly that it's for staging and test environments only.

v3.10.0's known issues include data loss. Running with --use-pacha-tree enabled and two or more shards (--pt-shard-count) can trigger data loss and a bootstrap deadlock. The workaround is to keep the shard count at 1. In other words, this preview is effectively usable only as a single shard right now.

Downgrading is not undoing. influxdb3 downgrade-to-parquet updates the catalog and deletes every .pt file from object storage. In the docs' words, this deletes even data written after the upgrade, and only the original Parquet files that existed before the upgrade are preserved. Turn the preview on, accumulate a month of data, then change your mind, and that month is gone.

The 3.10 upgrade itself is one-way. The first time you start 3.10, the on-disk catalog auto-migrates from v2 to v3, and after that, binaries at 3.9.x or earlier fail to start against the same cluster data. The release notes say to back up the catalog directory and the catalog checkpoint object beforehand, and that restoring that object is the only way to roll back to 3.9.x. If you enabled --use-pacha-tree, data written in the .pt format also can't be read by 3.9.x.

And all of this is Enterprise-only. Core gets none of .pt, column families, or bounded compaction. Core keeps living inside its 432-file budget.

The temperature difference between Core and Enterprise shows up bluntly if you read the release notes back to back. v3.9.6 on June 25, 2026, and v3.10.2 plus v3.9.7 on June 30 — all three releases' Core entries are the identical sentence: a maintenance release containing only build and dependency updates, with no user-facing changes. The Enterprise entries for the same dates include a compaction deduplication fix and a processing-engine trigger-cancellation fix.

So, When to Use It, and When Not To

When Core is enough. Core isn't a bad product. It does what it says it does.

Real-time dashboards and alerting where the query window is recent hours to a few days
Buffering at edge/IoT ingestion points before forwarding upstream
Workloads centered on latest-value lookups — the last-value cache exists for exactly this
Pipelines where data only arrives in real time, with no backfill

When Core won't do. If even one of the following applies, open-source Core isn't the answer. This isn't the kind of wall you tune your way past.

You need to regularly query stretches of history beyond a few days — raising the file cap raises GET requests and memory together
Backfill or late-arriving data is frequent — files pile up overlapping in the same gen1 block, and 72 hours collapses faster
You need to delete rows — Core has no way to do that
You have a reason to shrink --gen1-duration to 1 minute — your queryable window drops to around 7.2 hours

When it's worth trying the Enterprise preview. The docs recommend it for high-cardinality, wide tables; frequent backfill spanning time ranges; low-latency, query-intensive access; sparse schemas where columns appear dynamically; and environments where a memory/CPU usage ceiling matters. But start in staging, with a single shard, on data you can afford to lose.

When you should look elsewhere. If you need long-term retention, deletion, and historical queries all in open source, this choice might not be InfluxDB 3 Core to begin with. Whether you go to a Postgres extension or a column store, the starting point for comparison is laid out in Time Series Databases 2026 Deep Dive.

Closing

To sum up: "InfluxDB 3 Core has a 72-hour limit" is an inaccurate summary. More precisely, a single query is capped at scanning a default of 432 Parquet files in a single table; multiplying that by the default 10-minute gen1 block yields 72 hours; and even that 72 hours is the best case, valid only when data arrives strictly in real time.

This isn't an arbitrary feature restriction. Building a time-series DB on top of Parquet inevitably produces a large volume of small, immutable files, the only fix for that is compaction, and InfluxData drew the commercial line at that compactor. Open-source Core inherits every consequence of that decision as-is — the file cap, up to two GETs per file, and the absence of a delete feature.

And in 2026, the bill for this architecture arrived for the vendor too. That Parquet is optimized for analytical workloads, and therefore falls short for single-series queries and wide schemas — that's not my claim, it's written in InfluxData's own preview docs. The result is .pt, and it carries Gorilla encoding, which isn't in the Parquet spec — the very technique that v1 and v2's TSM had.

If there's a lesson, it seems to be about the nature of choosing a general-purpose format. By choosing Arrow, Parquet, and DataFusion, InfluxDB 3 got SQL, Flight SQL, and ecosystem tooling almost for free. That was a real win. But a general-purpose format is optimized for the general case, and the further your workload drifts from that average, the more you end up paying the difference somewhere. For InfluxDB 3, that difference was billed as a paid component named the compactor, and as rebuilding its own file format a year after GA.

When you're evaluating a technology choice, I'd suggest looking at this before the feature list in the release notes — what this system's fundamental constraint is, and which license the component that lifts that constraint sits behind.