What is schema evolution in LanceDB?

Schema evolution refers to LanceDB's ability to add, rename, retype, or drop columns on an existing table without needing to rewrite the entire dataset from scratch, which is possible specifically because of the Lance format's columnar storage design.

table.add_columns({"category": "cast(NULL as string)"})
table.alter_columns({"path": "category", "rename": "product_category"})
table.drop_columns(["obsolete_field"])

Because data is stored column-by-column rather than row-by-row, adding a new column doesn't require touching the storage for every existing row's other columns — the new column's data (often just null placeholders for existing rows) can be added as new column-level storage without rewriting unrelated data that hasn't changed.

This matters a great deal in iterative machine learning workflows, where adding a new feature column, changing an embedding's dimensionality, or removing an obsolete field are common, recurring needs; a traditional row-oriented database might require a costly migration or downtime for the same kind of change, while LanceDB's columnar design keeps these operations comparatively cheap.