The quiet cost of the nightly full refresh
It worked when the table had a million rows. Now it has eight hundred million, the nightly job rebuilds all of it, and the warehouse bill has a hockey-stick shape nobody can explain.
The full refresh is the most natural pattern in the world: every night, drop the table and rebuild it from scratch. It's simple, it's obviously correct, and it scales linearly with your data — which is exactly the problem. As the table grows, the cost and runtime grow with it, long after the simplicity stopped being worth it.
Incremental is the upgrade
Most of last night's data didn't change. An incremental model processes only the new and changed partitions and merges them into the existing table, so the work scales with what changed, not with total history. For an append-mostly table, that's the difference between processing one day and processing three years, every night.
Full refresh costs grow with the size of your history. Incremental costs grow with the size of your change. Pick the one that matches reality.
Keep a full rebuild on hand
Incremental introduces a risk: if logic changes, old partitions still reflect the old logic. So keep the full-refresh path available as a deliberate operation — run it when you change the model, not every night. Idempotent partitions make this a clean re-run rather than a special procedure.
When full refresh is still right
Small tables, or ones where most rows change nightly, don't benefit — full refresh stays simpler and cheaper. The point isn't dogma; it's noticing when a table outgrew the pattern, instead of paying the growing bill out of habit.