satus.
§POST|~/blog||8 min read

I shipped bad defaults, and you fixed them

A founder post-mortem on the three satus defaults I got wrong in the first 90 days, what the run telemetry can and cannot say about them, and what changes and when.

This is a founder post, not a release note. Nothing here has shipped yet; the current published CLI is @passkeybridge/satus@0.3.2. What follows is an honest accounting of the defaults I chose when I first cut the CLI, which of them were wrong, what the run telemetry actually says versus what I initially claimed it said, and which changes are queued for v0.3.3 and v0.4.0.

The short version: three defaults were wrong in ways that were obvious in hindsight and invisible on my laptop. --rows 50 is too high for a first run, temperature 0.7 is too creative for schemas with tight CHECK constraints, and the CLI has no --seed flag at all, which is the single most common thing people ask for after their first successful run. The evidence for the first two is a mix of telemetry and support tickets; the evidence for the third is entirely inbox and GitHub issues. I want to be careful about which is which.

The three defaults

Default Set to Where it lives Problem
--rows 50 packages/cli/src/commands/generate.ts Too high for a "just trying this out" first run. Cost surprise, long feedback loop.
temperature 0.7 packages/cli/src/generate/providers/openai.ts Fine for prose, wrong for rows a Postgres CHECK will accept.
deterministic seed not exposed (no flag exists) Reproducing a run requires re-running the LLM. Every reviewer asks for this.

All three are verifiable in the tree today, at the file paths above. None of the numbers I give below about the fixes have shipped; they are the design targets for the next two releases.

What the telemetry actually captures

Before I say what the telemetry "told me", the honest surface of what it records, from packages/cli/src/generate/telemetry.ts:

  • profile, provider, model
  • target_schema (the Postgres schema name, not the schema contents)
  • per-table rows_generated, and the run total
  • total_cost_usd, input_tokens, output_tokens, duration_ms
  • environment (dev or live) and a coarse status
  • run id and CLI version

That is it. The telemetry does not record the value of --temperature, whether a run was --dry-run, whether the user re-ran with the same inputs, or whether the run ended in a Postgres CHECK violation on insert. So any claim of the form "telemetry told me temperature was wrong" is not a claim I can honestly make. Temperature evidence is qualitative, from support threads and reproductions on the customer schemas we hold. I will flag which is which for each default.

Default 1: --rows 50

generate.ts line 91 sets the default row count per table to 50. I picked 50 because it is enough to exercise a foreign-key graph in a way that 10 is not, and small enough that a three-table schema costs cents rather than dollars. Both halves of that are still true on the schemas I tested it against, which had between three and eight tables.

The telemetry we do have shows a wide range of total_rows per run, skewed toward the low end. Rather than cite a precise percentile that I would have to invent, the honest observation is this: the modal first run on a new schema is smaller than 50 rows per table, sometimes by an order of magnitude, and the runs that do use 50 tend to be second or third runs by the same operator against the same schema. First-run cost surprises are real and they arrive as support email, not as telemetry.

The queued change, targeted for v0.3.3 alongside the GitHub Action:

  • Default drops to --rows 10.
  • satus init writes an explicit rows field into satus.config.json so the value is visible, not hidden.
  • The satus generate cost estimate line prints the row count and total table count on the same line as the estimate, so "why is this $2.40?" answers itself before the LLM is called.

None of that requires new telemetry to justify. The current default is a laptop default, not a first-encounter default, and first encounters dominate.

Default 2: temperature 0.7

The OpenAI provider hard-codes temperature = 0.7 for non-GPT-5 models (providers/openai.ts:74). GPT-5 pins temperature at 1 and the code skips the field for that family, which is correct.

0.7 is the OpenAI cookbook default for "creative but coherent" text. It is the wrong default for structured JSON that has to pass a CHECK (amount > 0 AND amount < 1000000) or a varchar(24). In the reproductions I have run against customer schemas that hit us with validation failures on insert, lowering temperature from 0.7 to 0.2 does not fix every failure but it removes the entire class of "the model wrote a plausible-sounding value that happens to be 12 characters over the column limit" issues. It also makes the dry-run validator findings more stable between runs, which is what a reviewer expects.

I will not claim a percentage reduction in validation failures. Telemetry does not record failure-class counts today, and I would rather ship the fix than manufacture a stat to justify it.

Queued change, also targeted for v0.3.3:

  • New CLI flag --temperature <n> with a default of 0.2.
  • The OpenAI provider reads the flag and passes it through unchanged; GPT-5 detection stays as-is.
  • The Anthropic provider (providers/anthropic.ts) gains the same flag with the same default. Anthropic accepts temperature 0–1 with the same "lower is more deterministic" semantics (Anthropic API reference).
  • Run telemetry gets a temperature field so future posts about this can cite an actual distribution.

The last bullet is the one that lets me stop apologising for the shape of my evidence.

Default 3: no --seed flag

There is no way today to make two satus generate runs produce the same rows. The simulate.ts path used for --dry-run is deterministic by construction, so --dry-run is reproducible; the live path is not.

Every reviewer who has looked at a satus PR has asked some version of "can I re-run this and get the same rows?" The answer today is no, and the workaround (commit the generated SQL, replay it) works but is not what people want. They want to commit a seed number and a schema hash and know that a colleague on a fresh checkout gets the same data.

This is the change I am least sure how to scope, and I want to be honest about that. There are two shapes:

  1. Cheap shape. Add --seed <n> that only seeds the local RNG (row shuffling, batch ordering, fixture-name choice). Cheap to build, honest about its limits, does not touch LLM output. Postgres inserts are still non-deterministic because provider responses are.
  2. Correct shape. --seed <n> also gets forwarded to the provider (seed on OpenAI, best-effort on Anthropic which does not expose one), and the CLI records the provider system_fingerprint in the run manifest so a mismatch is visible. Correct but larger, and Anthropic parity is not free.

Neither shape lands in v0.3.3. The v0.3.3 window is reserved for the GitHub Action and the two default changes above. --seed is queued for v0.4.0 (the release formerly called v0.5), scoped as the cheap shape first with a follow-up post if we ship the correct shape after.

What I am not changing

Three things I considered and left alone.

  • Default profile stays saas. The telemetry does show profile distribution and saas is the top choice by a comfortable margin. Changing this would be churn.
  • Default provider stays openai. Auto-detection from OPENAI_API_KEY vs ANTHROPIC_API_KEY is doing the right thing in every support thread I have opened; when both keys are set, the CLI errors and asks the user to be explicit, and nobody has complained about that error.
  • Default --batch-size 25. No signal that this is wrong. It is the value the Anthropic provider was designed around (comment in providers/anthropic.ts) and OpenAI is comfortable with it.

The uncomfortable meta-lesson

The pattern in all three defaults is the same: I picked a value that was correct for the environment I ship from (my laptop, my schemas, my tolerance for cost) and shipped it as the value for the environment users encounter first (unfamiliar CLI, unfamiliar cost model, unfamiliar failure modes). "Correct on my machine" is a very specific kind of wrong.

The fix is not "add more telemetry" as a reflex. The fix is to record the values that would let a future post like this one cite a distribution instead of a vibe, then to change the defaults on the strength of that plus the support inbox, and to write the follow-up post the same day the change ships so the record is public. That is what v0.3.3 will look like when it lands the week of 2026-07-13. A /blog/v0-3-3-release-notes post will replace the current roadmap post on the day of the npm publish, and the numbers in this post become checkable.

References

—the satus.sh team

published 2026-07-08 · satus.sh · ·meta ·product
← all posts