The future of SQL gets argued as if the language were on trial, and the verdict keeps arriving the same way. In the 2025 Stack Overflow Developer Survey, SQL ranked third among all programming, scripting and markup languages, used extensively by 58.6% of respondents over the previous year, behind only JavaScript and HTML/CSS. The syntax people learned a decade ago still runs most of the analytics being shipped today.
What changed is where it runs. The same SELECT now executes against a cloud warehouse, against files in object storage through an open table format, against a federated engine reaching three systems at once, and against an unbounded stream of events. Each of those reads the statement the same way and then executes it, bills it and fails it differently.
That gap is the part worth planning for. The language travels between engines almost unchanged, while the behaviour around it does not, so a query that was cheap and correct on one platform can be expensive and quietly wrong on the next. What follows covers what stays portable, what breaks, and how teams keep both under control.
Why SQL became the default interface for distributed data
Two things happened at once, and they reinforced each other.
The first is that the standard kept absorbing the work that alternatives were invented to do. SQL:2023, published by ISO as the ninth edition of ISO/IEC 9075, added a JSON data type to the core language and introduced an entirely new part, SQL/PGQ, for querying property graphs with pattern matching inside a SQL statement. Document and graph workloads that once required a separate database and a separate query language now have a standardised path back into SQL.
The second is that storage stopped deciding the engine. Open table formats put ACID transactions, schema evolution and time travel on top of ordinary files in object storage, and the Apache Iceberg REST catalog specification standardised how any engine discovers those tables and commits changes to them. Databricks, Snowflake and BigQuery all read and write Iceberg tables through that protocol today, which means several engines can point at one copy of the data. The consequence is that SQL became the interface shared across the stack, because every engine already implemented it.
Underneath both is a property of the language itself. SQL is declarative, so a query describes the result and leaves the execution strategy to the planner. That separation lets one statement run on a warehouse, a lake engine and a stream processor, and it is also what makes the differences between them easy to miss. It is why the skills that separate senior data engineers in 2026 now lean harder on execution and cost than on syntax.
The future of SQL: portable syntax, non-portable behaviour
Here is the practical version of the shift. The text of a query moves between engines with modest edits. The economics, the data movement and the semantics of "finished" stay behind.
Cost follows the engine, not the query
Google's BigQuery cost documentation states that on-demand billing counts the bytes read across the columns a query references, and that on a non-clustered table a LIMIT clause does not reduce that number at all: the query is billed for reading the whole table even though it returns ten rows. Clustering changes the answer, because scanning stops once enough blocks have been read. Same query text, two table layouts, two invoices.
Snowflake prices the identical workload on a different axis. Its warehouse documentation puts a 60-second minimum charge on every warehouse start, with per-second billing after that, and notes that each resume triggers that minimum again. A workload made of hundreds of short bursts against a warehouse that keeps suspending can therefore cost more than one long-running job touching the same rows.
Neither behaviour is visible in the SQL. Both are visible in the bill. Understanding how a platform turns a statement into scanned bytes or warm compute is what separates a query that scales from one that merely runs, and BigQuery architecture explained for data teams is a good place to see that mechanism end to end.
| What the engine charges for | Typical platform | What makes the bill jump | What to change in the query |
|---|---|---|---|
| Bytes read per column referenced | BigQuery on-demand | Wide SELECT *, unpartitioned tables, LIMIT on non-clustered data | Name the columns, partition and cluster on the filter keys |
| Compute time while a warehouse is warm | Snowflake, Redshift Serverless | Many short bursts with frequent suspend and resume | Batch small jobs, tune auto-suspend to the real gap in the workload |
| Cluster resources held for the duration | Trino, Spark SQL, self-managed engines | Cross-source joins that cannot be pushed down | Filter and aggregate at the source, land intermediate results |
| Continuous state held between events | Flink SQL, streaming engines | Unbounded joins and aggregations over long windows | Bound the window, set state retention deliberately |
Federated queries push down only what the connector allows
Federation promises one statement across many systems without moving anything first. The mechanism that delivers on it is pushdown, and the Trino documentation is explicit that support for pushdown is specific to each connector and the database behind it. When a predicate, a projection or an aggregation can be pushed down, the remote system does the filtering and only the result crosses the network. When it cannot, the engine fetches the full dataset and performs the operation itself on its own workers.
One constraint deserves particular attention, because it sits against the way federation is usually pitched. Trino requires that the tables in a join come from the same catalog for join pushdown to happen at all. A join across two different sources, which is the headline use case, is executed inside the cluster by definition, with both sides pulled in.
None of that makes federation a bad idea. It makes it an architectural decision with a known cost profile, well suited to exploration and cross-system reporting, and poorly suited to a dashboard refreshing every minute over billions of rows. Running federated queries with Trino across multiple sources and the wider case for querying data without moving it both show where the line falls.
A streaming query never ends, so its state has to be bounded
Streaming SQL looks like batch SQL and behaves like a long-lived process. Apache Flink models it through dynamic tables and continuous queries: a query over a stream never terminates and produces a result that keeps updating. Because those queries run for weeks or months over unbounded input, Flink's documentation is direct that a continuous query has to restrict the size of the state it maintains, or it runs out of storage.
The control for that is state retention, and it changes results rather than just resource usage. Once the state for a key is dropped after an idle period, the query has no record of having seen that key, so a late event arrives as if it were the first. A batch query producing a wrong number is a bug. A streaming query producing a different number because its retention window expired is working exactly as configured. That distinction is the one most teams meet the hard way when they choose between streaming and batch processing, and it is worth settling before the first continuous query reaches production alongside an event-driven architecture built on the Kafka API.
What semantic layers and AI assistance actually change
Two shifts get sold as the future of SQL, and both are real with narrower edges than the pitch suggests.
A semantic layer moves metric definitions out of individual queries and into one governed place, so revenue, active user and churn are computed once and reused by every dashboard and downstream model. The portable asset stops being the query text and becomes the definition, which is the version of portability that survives a platform migration. How that works in practice is covered in how metrics work in the dbt semantic layer.
AI assistance is further along for writing queries than for owning them, and there is a benchmark that shows the gap cleanly. Spider 2.0, presented at ICLR 2025, evaluates models on 632 real enterprise text-to-SQL workflows across databases with more than a thousand columns, multiple dialects including BigQuery and Snowflake, and solutions that often exceed a hundred lines of SQL. GPT-4o scored 86.6% on the original Spider benchmark and around 13% on Spider 2.0. Agent frameworks have since pushed the top of the Snowflake split above 95%, while the split built on dbt projects, the one closest to a real analytics codebase, still tops out in the mid-60s. Generating a query is close to solved. Maintaining a warehouse of dependent models is not.
Dialect portability sits in the same category. SQLGlot, the open-source parser most transpilation tooling is built on, supports more than 30 SQL dialects, which is a fair measure of how much divergence exists between engines that all claim to speak the same language. Skills transfer across platforms; code transfers with a translation step and a test suite. The performance model diverges too, since the same aggregation can run an order of magnitude faster on a vectorised engine, as the Databricks Photon engine shows without a single character of the query changing.
How to future-proof a SQL codebase
Four habits carry most of the weight, and none of them depend on picking the right engine.
- Treat SQL as production code. Version control, code review, tests, documentation, lineage and a named owner turn a collection of queries into a system that outlives the person who wrote it. Transformation tooling makes this cheap, and automating data quality and cleansing with dbt is the usual entry point.
- Write for the cost model in front of you. Name columns instead of selecting everything, partition and cluster on the keys people actually filter by, and batch small jobs so a warehouse stays warm rather than resuming forty times an hour.
- Define metrics once. A governed semantic layer, or even documented canonical tables with consistent naming, removes the class of problem where two dashboards disagree because two analysts wrote the same logic twice.
- Learn the planner, not the dialect. Join strategies, cardinality and statistics, partitioning, materialisation and the difference between a view and a temporary table transfer everywhere. Function-name trivia does not.
Distributed architectures made analytics more capable and more complicated in the same movement, and SQL survived because it reduces that complexity to something a whole organisation can read. The useful reading of the future of SQL is that the language is now the stable part of the stack while everything underneath it keeps moving, so the leverage sits in understanding what each engine does with a statement rather than in memorising syntax. Teams that instrument cost per query, bound their streaming state on purpose and keep their definitions in one place get to change engines without rewriting their analytics.
SQL is the analytics default, but teams also pick languages for services and applications, and the criteria for that choice are in how to choose a programming language and framework.
If your team is deciding how much of its analytics should stay in SQL, or paying more than expected for queries that looked identical on a different platform, our specialists can review the architecture and the trade-offs before you commit. Talk to our team and move your data maturity forward. ⬇️
FAQ: frequently asked questions about the future of SQL
What is the future of SQL in a distributed data world?
SQL is becoming the shared interface across warehouses, lakehouses, federated engines and streams rather than the language of a single database. The standard keeps expanding, with SQL:2023 adding a JSON data type and property graph queries, while open table formats let several engines query one copy of the data.
Will SQL be replaced by Python or by AI tools?
Neither replaces it, because they solve different problems. Python handles procedural logic and machine learning that SQL expresses poorly, and AI assistants generate and explain queries. On the Spider 2.0 enterprise benchmark, the split modelled on real dbt projects still tops out in the mid-60s, so query generation assists ownership rather than removing it.
Is SQL portable between BigQuery, Snowflake and Databricks?
The skill is portable and the code needs translation. SQLGlot, the parser behind most transpilation tooling, supports more than 30 dialects, which indicates how much syntax diverges. Cost and execution diverge further: BigQuery bills bytes read per column referenced, while Snowflake bills compute time with a 60-second minimum per warehouse start.
Why are federated SQL queries slower than expected?
Because pushdown is connector-specific. When a filter or aggregation cannot be pushed to the source, the engine pulls the full dataset and processes it locally. Trino additionally requires both tables of a join to live in the same catalog for join pushdown, so cross-source joins are always executed inside the cluster.
How is streaming SQL different from batch SQL?
A batch query reads a finite table and terminates, while a streaming query runs continuously over unbounded input and keeps updating its result. Because it may run for months, it has to bound the state it holds, and state retention settings decide whether a late-arriving event updates an existing key or starts a new one.
