Skip to main content

Data Analysis Skill

Overview

This skill analyzes user-uploaded Excel/CSV files using DuckDB — an in-process analytical SQL engine. It supports schema inspection, SQL-based querying, statistical summaries, and result export, all through a single Python script.

Core Capabilities

  • Inspect Excel/CSV file structure (sheets, columns, types, row counts)
  • Execute arbitrary SQL queries against uploaded data
  • Generate statistical summaries (mean, median, stddev, percentiles, nulls)
  • Support multi-sheet Excel workbooks (each sheet becomes a table)
  • Export query results to CSV, JSON, or Markdown
  • Handle large files efficiently with DuckDB’s columnar engine

Workflow

Step 1: Understand Requirements

When a user uploads data files and requests analysis, identify:
  • File location: Path(s) to uploaded Excel/CSV files under /mnt/user-data/uploads/
  • Analysis goal: What insights the user wants (summary, filtering, aggregation, comparison, etc.)
  • Output format: How results should be presented (table, CSV export, JSON, etc.)
  • You don’t need to check the folder under /mnt/user-data

Step 2: Inspect File Structure

First, inspect the uploaded file to understand its schema:
This returns:
  • Sheet names (for Excel) or filename (for CSV)
  • Column names, data types, and non-null counts
  • Row count per sheet/file
  • Sample data (first 5 rows)

Step 3: Perform Analysis

Based on the schema, construct SQL queries to answer the user’s questions.

Run SQL Query

Generate Statistical Summary

This returns for each numeric column: count, mean, std, min, 25%, 50%, 75%, max, null_count. For string columns: count, unique, top value, frequency, null_count.

Export Results

Supported output formats (auto-detected from extension):
  • .csv — Comma-separated values
  • .json — JSON array of records
  • .md — Markdown table

Parameters

[!NOTE] Do NOT read the Python file, just call it with the parameters.

Table Naming Rules

  • Excel files: Each sheet becomes a table named after the sheet (e.g., Sheet1, Sales, Revenue)
  • CSV files: Table name is the filename without extension (e.g., data.csvdata)
  • Multiple files: All tables from all files are available in the same query context, enabling cross-file joins
  • Special characters: Sheet/file names with spaces or special characters are auto-sanitized (spaces → underscores). Use double quotes for names that start with numbers or contain special characters, e.g., "2024_Sales"

Analysis Patterns

Basic Exploration

Aggregation & Grouping

Cross-file Joins

Window Functions

Pivot-style Analysis

Complete Example

User uploads sales_2024.xlsx (with sheets: Orders, Products, Customers) and asks: “Analyze my sales data — show top products by revenue and monthly trends.”

Step 1: Inspect the file

Step 2: Top products by revenue

Step 4: Statistical summary

Present results to the user with clear explanations of findings, trends, and actionable insights.

Multi-file Example

User uploads orders.csv and customers.xlsx and asks: “Which region has the highest average order value?”

Output Handling

After analysis:
  • Present query results directly in conversation as formatted tables
  • For large results, export to file and share via present_files tool
  • Always explain findings in plain language with key takeaways
  • Suggest follow-up analyses when patterns are interesting
  • Offer to export results if the user wants to keep them

Caching

The script automatically caches loaded data to avoid re-parsing files on every call:
  • On first load, files are parsed and stored in a persistent DuckDB database under /mnt/user-data/workspace/.data-analysis-cache/
  • The cache key is a SHA256 hash of all input file contents — if files change, a new cache is created
  • Subsequent calls with the same files will use the cached database directly (near-instant startup)
  • Cache is transparent — no extra parameters needed
This is especially useful when running multiple queries against the same data files (inspect → query → summary).

Notes

  • DuckDB supports full SQL including window functions, CTEs, subqueries, and advanced aggregations
  • Excel date columns are automatically parsed; use DuckDB date functions (DATE_TRUNC, EXTRACT, etc.)
  • For very large files (100MB+), DuckDB handles them efficiently without loading everything into memory
  • Column names with spaces are accessible using double quotes: "Column Name"