# Dask

Scales pandas, NumPy, and custom Python research workflows beyond memory or across...

- Repository: https://github.com/K-Dense-AI/scientific-agent-skills (55651 stars)
- Category: Data and ML
- License: BSD-3-Clause license
- Last updated: 2026-10-26
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add K-Dense-AI/scientific-agent-skills --skill dask -g -a claude-code -y`
Codex: `npx skills add K-Dense-AI/scientific-agent-skills --skill dask -g -a codex -y`
Cursor: `npx skills add K-Dense-AI/scientific-agent-skills --skill dask -g -a cursor -y`
Gemini CLI: `npx skills add K-Dense-AI/scientific-agent-skills --skill dask -g -a gemini-cli -y`

## Description

Scales pandas, NumPy, and custom Python research workflows beyond memory or across clusters with Dask. Covers DataFrames, Arrays, Bags, Futures, chunking, schedulers, and distributed diagnostics. Use for partitioned file processing, scientific array computation, or parallel tasks whose memory and dependency structure require Dask.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Dask

## Overview

Dask is a Python library for parallel and distributed computing that enables three critical capabilities:
- **Larger-than-memory execution** on single machines for data exceeding available RAM
- **Parallel processing** for improved computational speed across multiple cores
- **Distributed computation** supporting terabyte-scale datasets across multiple machines

Capacity depends on partition sizes, intermediate data, concurrency, storage, and available memory.

**Reviewed release:** dask/distributed **2026.8.0** (September 30, 2026 review). Docs: [docs.dask.org](https://docs.dask.org/en/stable/). Since **2025.1.0**, the expression-based DataFrame API with query planning is the only implementation, do not install `dask-expr` separately or set `dataframe.query-planning: False`.

## Quick Start

### Installation

```bash
uv pip install "dask[array,dataframe]==2026.8.0"
```

For a typical pandas/NumPy workflow with the distributed scheduler and dashboard:

```bash
uv pip install "dask[complete]==2026.8.0"
```

Remote object storage (requires provider credentials for private data):

```bash
uv pip install s3fs    # s3:// paths
uv pip install gcsfs   # gs:// paths
```

Requires **Python 3.10+**, pandas 2+, PyArrow 16+. Zarr, HDF5, SciPy, Xarray, Dask-ML, and cluster deployment packages are separate optional dependencies. See [review and validation](references/review.md) for tested versions, source links, and limits. File paths and undefined application functions below are illustrative. Put process scheduler/`Client()` execution in a `main()` guarded by `if __name__ == "__main__":` in scripts; close clients and clusters afterward.

## When to Use This Skill

The rest: https://github.com/K-Dense-AI/scientific-agent-skills/blob/HEAD/skills/dask/SKILL.md