SQL vs Pandas vs AI Agents: Which Solves Analytics Problems Best?

# Introduction

 
We gave the same three interview questions from StrataScratch to SQL, Pandas, and a Claude agent. Every piece of code executed against the same dataset, and every timing number is a median over 500 runs. The agent’s answers are exactly what Claude generated in response to a documented prompt, instead of a hypothetical example of what an agent might produce.

The comparison runs across eight dimensions: speed, accuracy, explainability, debugging, scalability, flexibility, hallucination risk, and production readiness. The three questions span Easy, Medium, and Hard difficulty levels. The harder the question, the more the differences between SQL, Pandas, and the agent become visible.

# How We Ran This Comparison

 
The three questions come from the StrataScratch interview bank and cover Easy, Medium, and Hard difficulty levels. SQL ran on SQLite in-memory, timed over 500 runs, with the median taken. Pandas ran on the same dataset in Python 3.12, also over 500 runs. The agent is Claude’s claude-sonnet-4-6, called via the Anthropic API.

Each question got its own schema-grounded user prompt that included the table names, column names, and a few sample rows. The system prompt below stayed the same for all three calls. Agent response times are measured from the time the request is sent to the first token received.

# Simple Retrieval: All Three Agree

 
For the first interview question from Meta, users are asked to find every user who performed at least one scroll_up event and return the distinct user IDs. The data lives in a single table called facebook_web_log.

// Data

Here’s the facebook_web_log table.

// SQL Coding Solution (0.002 ms)

SELECT DISTINCT user_id
FROM facebook_web_log
WHERE action = 'scroll_up';

// Pandas Coding Solution (0.40 ms)

import pandas as pd
result = (
    facebook_web_log[facebook_web_log['action'] == 'scroll_up']
    .drop_duplicates(subset="user_id")[['user_id']]
)

// Agent Prompt

Table: facebook_web_log (user_id INTEGER, action TEXT, timestamp TEXT)
Sample rows:
(1, 'scroll_up',   '2019-01-01 00:00:00')
(2, 'scroll_down', '2019-01-01 00:01:00')
(3, 'like',        '2019-01-01 00:03:00')
(2, 'scroll_up',   '2019-01-01 00:04:00')
Question: Find all users who performed at least one scroll_up event.
Return distinct user IDs.

// Agent Output (2 s)

SELECT DISTINCT user_id
FROM facebook_web_log
WHERE action = 'scroll_up';

Output: All three return users 1 and 2.

On a single-filter problem, the agent matches SQL exactly. The only real risk at this difficulty is column naming. Without the schema in the prompt, action might come back as event_type or event_name, which returns nothing and throws no error.

# Multi-Step Aggregation: Where Schema Grounding Matters Most

 
The second question is about product feature completion. An app tracks how far each user gets through a set of product features, where every feature has a fixed number of steps.

The task is to calculate the average completion percentage for each feature across all users, where a user’s completion is their maximum step reached divided by the total steps for that feature, times 100. Users who have never started a feature are counted as 0% complete.

Two tables feed this: facebook_product_features:

and facebook_product_features_realizations.

// SQL Coding Solution (0.007 ms)

WITH max_step AS (
    SELECT
        feature_id,
        user_id,
        MAX(step_reached) AS max_step_reached
    FROM facebook_product_features_realizations
    GROUP BY feature_id, user_id
),
calc_per_feature AS (
    SELECT
        feats.feature_id,
        n_steps,
        max_step_reached,
        COALESCE(max_step_reached, 0) * 1.0 / n_steps AS share_of_completion
    FROM facebook_product_features feats
    LEFT OUTER JOIN max_step
        ON feats.feature_id = max_step.feature_id
)
SELECT
    feature_id,
    AVG(share_of_completion) * 100 AS avg_share_of_completion
FROM calc_per_feature
GROUP BY feature_id;

// Pandas Coding Solution (2.05 ms)

import pandas as pd
# max step per user per feature
max_step = (
    facebook_product_features_realizations
    .groupby(['feature_id', 'user_id'])['step_reached']
    .max()
    .reset_index()
)

# join to features, fill users who never started with 0
df = pd.merge(
    facebook_product_features,
    max_step,
    how='outer',
    on='feature_id'
).fillna(0)

# completion % per user-feature row
df['share_of_completion'] = (df['step_reached'] / df['n_steps']) * 100

# average per feature
result = (
    df.groupby('feature_id')['share_of_completion']
    .mean()
    .to_frame('avg_share_of_completion')
    .reset_index()
)

// Agent Prompt

Tables:
- facebook_product_features (feature_id INTEGER, n_steps INTEGER)
- facebook_product_features_realizations
  (feature_id INTEGER, user_id INTEGER, step_reached INTEGER, timestamp TEXT)
Question: For each feature, calculate the average completion percentage
across all users. A user's completion = max(step_reached) / n_steps * 100.
Users who never started count as 0% completion.
Return: feature_id, avg_share_of_completion.

// Agent Output (3 s)

WITH user_max_steps AS (
    SELECT feature_id, user_id, MAX(step_reached) AS max_step
    FROM facebook_product_features_realizations
    GROUP BY feature_id, user_id
)
SELECT
    f.feature_id,
    ROUND(AVG(COALESCE(ums.max_step, 0) * 100.0 / f.n_steps), 2) AS avg_share_of_completion
FROM facebook_product_features f
LEFT JOIN user_max_steps ums ON f.feature_id = ums.feature_id
GROUP BY f.feature_id
ORDER BY f.feature_id;

Output: All three return the same numbers.

The agent got it right here because the prompt said: “Users who never started count as 0% completion.” That phrase is load-bearing. Without it, the agent writes an inner join — which drops non-starters — and every average goes up. That failure is silent. The numbers come back clean, and they’re wrong. You’d need to know the expected output to catch it.

# Multiple Tables and Window Logic: All Three Correct, One Much Slower

 
The third question covers Meta’s data center energy consumption across three regions. Each region has its own table: fb_eu_energy, fb_na_energy, and fb_asia_energy.

The task is to combine them, sum consumption by date, and produce two derived columns: the cumulative running total and that total as a percentage of the grand total, rounded to a whole number.

// Data

Each regional table has the same shape.

fb_eu_energy:

fb_na_energy:

fb_asia_energy:

// SQL Coding Solution (0.010 ms)

WITH total_energy AS (
    SELECT recorded_date, consumption FROM fb_eu_energy
    UNION ALL
    SELECT recorded_date, consumption FROM fb_asia_energy
    UNION ALL
    SELECT recorded_date, consumption FROM fb_na_energy
),
energy_by_date AS (
    SELECT
        recorded_date,
        SUM(consumption) AS total_energy
    FROM total_energy
    GROUP BY recorded_date
    ORDER BY recorded_date ASC
)
SELECT
    recorded_date,
    SUM(total_energy) OVER (
        ORDER BY recorded_date ASC
    ) AS cumulative_total_energy,
    ROUND(
        SUM(total_energy) OVER (ORDER BY recorded_date ASC) * 100.0
        / (SELECT SUM(total_energy) FROM energy_by_date),
        0
    ) AS percentage_of_total_energy
FROM energy_by_date;

// Pandas Coding Solution (1.84 ms)

import pandas as pd
merged_df = pd.concat([fb_eu_energy, fb_asia_energy, fb_na_energy])

energy_by_date = (
    merged_df.groupby('recorded_date', as_index=False)['consumption']
    .sum()
    .sort_values('recorded_date')
)

energy_by_date['cumulative_total_energy'] = energy_by_date['consumption'].cumsum()
energy_by_date['percentage_of_total_energy'] = round(
    energy_by_date['cumulative_total_energy'] * 100 / energy_by_date['consumption'].sum()
)

energy_by_date['recorded_date'] = pd.to_datetime(
    energy_by_date['recorded_date']
).dt.strftime('%Y-%m-%d')

result = energy_by_date[
    ['recorded_date', 'cumulative_total_energy', 'percentage_of_total_energy']
]

// Agent Prompt

Tables:
- fb_eu_energy   (recorded_date TEXT, consumption INTEGER)
- fb_na_energy   (recorded_date TEXT, consumption INTEGER)
- fb_asia_energy (recorded_date TEXT, consumption INTEGER)
Question: Combine all three tables. Sum consumption by date. Output:
- recorded_date
- cumulative_total_energy (running sum across dates, ordered by date)
- percentage_of_total_energy (running cumulative / grand total * 100,
  rounded to the nearest whole number)

// Agent Output (4 s)

WITH all_energy AS (
    SELECT recorded_date, consumption FROM fb_eu_energy
    UNION ALL
    SELECT recorded_date, consumption FROM fb_na_energy
    UNION ALL
    SELECT recorded_date, consumption FROM fb_asia_energy
),
daily_totals AS (
    SELECT recorded_date, SUM(consumption) AS daily_total
    FROM all_energy
    GROUP BY recorded_date
)
SELECT
    recorded_date,
    SUM(daily_total) OVER (ORDER BY recorded_date) AS cumulative_total_energy,
    ROUND(
        SUM(daily_total) OVER (ORDER BY recorded_date) * 100.0
        / SUM(daily_total) OVER (),
        0
    ) AS percentage_of_total_energy
FROM daily_totals
ORDER BY recorded_date;

Output: All three return the same table.

The agent used SUM(daily_total) OVER () (a window function with no ORDER BY) as the denominator rather than the scalar subquery in the SQL reference solution. Both approaches are valid. The output matched exactly.

# How the Three Compare

// Speed

At this data scale, SQL ran in 0.002-0.010 ms, Pandas in 0.4-2.1 ms. The agent added 2-4 seconds of large language model (LLM) inference time before any SQL ran.

The agent generates code first; that generation time is the end-to-end latency for each query cycle. At warehouse scale, the gap closes to near zero once code is generated; SQL gains further because it runs inside the database engine, and Pandas hits a memory ceiling around 10 million rows and needs Apache Spark or Polars beyond that.

// Accuracy and Hallucination Risk

SQL and Pandas are deterministic. The same code on the same data gives the same answer every time. With schema-grounded prompts, Claude got all three questions right, but each call produced different SQL (different common table expression (CTE) names, different column aliases, different but equivalent approaches). Without the schema, hallucination risk climbs fast.

// Explainability and Debugging

A SQL query reads in one block. A bad join condition is visible right in the text. Pandas needs Python fluency, but you can inspect the DataFrame at each step. Agents explain their reasoning in English, then produce code that you may or may not be shown. If the generated SQL is wrong, you’re tracing an error through a model’s reasoning chain rather than reading a query you wrote.

// Flexibility and Production Readiness

Pandas is the clearest option for custom transformations, string parsing, and iterative feature engineering. SQL handles set logic cleanly and gets verbose for procedural work. Agents answer plain-English requests well, with the least consistency when schemas are complex or ambiguous. For shipping, SQL is the most proven option in analytics; Pandas is dependable with tests; and agents are dependable today for low-stakes queries or when the output is reviewed before it runs.

# What the Agent Results Actually Show

 
With a schema-grounded prompt, Claude got all three correct: Easy, Medium, and Hard. The agent’s SQL for the Hard question used a different window function pattern than the reference solution and still returned the correct table.

Two things limit that finding. First, reproducibility: each API call can return different SQL for the same question. The logic is equivalent, but a team reviewing agent-generated queries needs to verify the outputs rather than trust that today’s correct run will match tomorrow’s. Second, schema dependency: the prompts above include table and column names, as well as sample rows. Remove those, and the agent guesses.

At Easy difficulty, a wrong guess produces an empty result. At Hard difficulty, a wrong guess produces a plausible wrong result with no error.

The practical pattern is: provide the full schema, ask for SQL, then run and verify the output before it goes downstream.

# Conclusion

 
SQL, Pandas, and Claude each got the same three analytics questions right when used correctly. The differences are in speed (0.01 ms vs 4 seconds), reproducibility, and what happens when you reduce the context.

SQL fits structured retrieval and set-based logic with millisecond execution and deterministic output. Pandas fits custom transformations and step-by-step notebook work up to about 10 million rows. The agent fits first-draft queries and ad hoc exploration, with the full schema in the prompt and a human reviewing the output.

The agent got the Hard question right by using SUM() OVER() instead of a scalar subquery, which is a valid approach that SQL’s reference solution didn’t take. That’s the honest version of this comparison: the agent can generate correct, creative SQL. It just adds latency, varies between runs, and depends entirely on what you put in the prompt.
 
 

Nate Rosidi is a data scientist and in product strategy. He’s also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Nate writes on the latest trends in the career market, gives interview advice, shares data science projects, and covers everything SQL.