Skills

Spark CLI

Manages Fabric Spark work, including notebook cell code with %%configure, %%sql...

Install

npx skills add microsoft/skills-for-fabric --skill spark-cli -g -a claude-code -y
Category
Data and ML
Updated
6d ago
License
MIT
Context cost
Mediumabout 2,479 tokens
Runs scripts
No, instructions only

What it does

Manages Fabric Spark work, including notebook cell code with %%configure, %%sql, PySpark and notebookutils, named notebook runs, Livy sessions, triage of failed or OOM notebook and pipeline Spark runs, and the Materialized Lake View (MLV) lifecycle. Load it before writing MLV SQL, since CREATE MATERIALIZED LAKE VIEW and its CONSTRAINT clause are Fabric-only. KQL materialized views belong to eventhouse-cli.

Telemetry, MANDATORY. Every api.fabric.microsoft.com call must carry x-ms-fabric-skill: spark-cli (az rest: --headers "x-ms-fabric-skill=spark-cli"), including every LRO poll, fabric_lro and retry. Snippets omit it, add it anyway.

CRITICAL NOTES

  1. To find the workspace details (including its ID) from workspace name: list all workspaces and, then, use JMESPath filtering
  2. To find the item details (including its ID) from workspace ID, item type, and item name: list all items of that type in that workspace and, then, use JMESPath filtering
  3. Skill disambiguation: use spark-cli for notebook cell code (including %%sql cells), named notebook runs, Livy sessions, Spark failure triage, and everything about a Materialized Lake View -- writing the definition, reviewing a query for incremental-refresh readiness, and scheduling, refreshing, monitoring or diagnosing an existing one. A KQL materialized view in an Eventhouse is eventhouse-cli; plain read-only T-SQL against a Warehouse or Lakehouse SQL endpoint is sqldw-cli.
  4. Hard routing boundary: never execute an Eventhouse/KQL materialized-view request from this skill. Route it to eventhouse-cli; if that skill is unavailable, state that the request cannot be completed in the current skill context and stop without calling Fabric APIs or creating artifacts.

Fabric Spark and Materialized Lake Views -- CLI Skill

This one skill owns Fabric Spark: notebook cell authoring, notebook runs, Livy-session analysis, Spark failure diagnostics, and the whole Materialized Lake View lifecycle.

It is a mode dispatcher and contains NO procedures. Pick the mode that matches the request from the table below, then read the matching references/<mode>.md file end to end with your file-reading tool BEFORE issuing a single command. That file holds the endpoints, payload shapes, templates and gotchas; acting without it produces wrong payloads and wrong results.

Mode selection

Read the whole SKILL.md on GitHub

More in Data and ML