# Datalad

Retrieves, versions, and publishes scientific datasets with DataLad and git-annex, and...

- Repository: https://github.com/K-Dense-AI/scientific-agent-skills (55651 stars)
- Category: Data and ML
- License: MIT
- Last updated: 2026-10-26
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add K-Dense-AI/scientific-agent-skills --skill datalad -g -a claude-code -y`
Codex: `npx skills add K-Dense-AI/scientific-agent-skills --skill datalad -g -a codex -y`
Cursor: `npx skills add K-Dense-AI/scientific-agent-skills --skill datalad -g -a cursor -y`
Gemini CLI: `npx skills add K-Dense-AI/scientific-agent-skills --skill datalad -g -a gemini-cli -y`

## Description

Retrieves, versions, and publishes scientific datasets with DataLad and git-annex, and captures computational provenance with datalad run, rerun, and containers-run. Use when cloning or fetching data from OpenNeuro, DANDI, datasets.datalad.org, or any DataLad dataset; when a file in a dataset reads as a broken symlink or a small pointer instead of real data; when an analysis needs a machine-readable record of how each output was produced so it can be re-executed; or when publishing a dataset to siblings such as a GitHub repository plus a storage remote. Also use to decide between DataLad and plain Git for a data-carrying repository.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# DataLad

## Overview

DataLad is a data management layer over Git and git-annex. Git tracks the dataset
structure, small text files, and the history. git-annex tracks the *content* of large
files, storing each file as a key and keeping the bytes somewhere that is not necessarily
the local repository.

A normal clone retrieves Git history and the top-level file listing while leaving
annexed bytes unfetched. Installed subdatasets have their own histories; a clone does not
automatically populate them. Clone cost depends on Git history and file count, not just
the data volume. Retrieve annexed bytes selectively with `datalad get`.

The second thing DataLad adds is provenance. `datalad run` executes a command and commits
the result together with a machine-readable record of the command, its inputs, and its
outputs. `datalad rerun` reads that record back and re-executes it. This turns "how was
this figure produced" from an archaeology problem into a command.

## When to use DataLad instead of plain Git

Use DataLad when any of the following holds:

- Files are too large for Git to handle comfortably, or the total exceeds what every
  collaborator wants on disk.
- Data lives in more than one place (a lab server, a cluster scratch, S3, a supercomputer)
  and you need to know which copies exist.
- The analysis must be re-executable, and a plain commit message is not enough evidence.
- You are consuming published datasets from OpenNeuro, DANDI, or `datasets.datalad.org`,
  which are distributed as DataLad datasets.
- The project nests other datasets inside it and you want each one to keep its own
  independent history.

Use plain Git when the repository is code and text only, everything fits comfortably in
Git, and nobody needs partial checkouts. DataLad on top of a small pure-code repository
adds indirection without buying anything.

## Installation

The rest: https://github.com/K-Dense-AI/scientific-agent-skills/blob/HEAD/skills/datalad/SKILL.md