# Kermt Add Cmim Pretrain

Convert a grover_base checkpoint (encoder-only or encoder + vocab heads) into a hybrid...

- Repository: https://github.com/NVIDIA/skills (3812 stars)
- Category: Engineering
- License: Apache-2.0
- Last updated: 2026-10-30
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add NVIDIA/skills --skill kermt-add-cmim-pretrain -g -a claude-code -y`
Codex: `npx skills add NVIDIA/skills --skill kermt-add-cmim-pretrain -g -a codex -y`
Cursor: `npx skills add NVIDIA/skills --skill kermt-add-cmim-pretrain -g -a cursor -y`
Gemini CLI: `npx skills add NVIDIA/skills --skill kermt-add-cmim-pretrain -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add NVIDIA/skills`

## Description

Convert a grover_base checkpoint (encoder-only or encoder + vocab heads) into a hybrid checkpoint by adding a randomly-initialized cMIM decoder + latent_dist, then continue pretraining on the user's corpus as hybrid (vocab + contrast). Effectively kermt-continue-pretrain with a one-time ckpt-conversion step prepended.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# kermt-add-cmim-pretrain

Convert a grover_base checkpoint (legacy original-GROVER `grover.encoders.*`
or modern `kermt.encoders.*`, with or without vocab heads) into a fully-formed
hybrid (cMIM + vocab) checkpoint, then continue pretraining on the user's
corpus as hybrid.

This is a thin wrapper: `upgrade_to_hybrid.py` produces a new ckpt that
classifies as `model_type: hybrid` via `check_checkpoint.py`, and the rest of
the workflow is identical to `kermt-continue-pretrain`.

> **Status: experimental.** This workflow is functional end-to-end but has not
> been benchmarked against the manuscript's from-scratch hybrid training (which
> produces the released checkpoint). Use as an experimental alternative to
> `kermt-pretrain-scratch` when you want to extend an existing grover_base
> checkpoint rather than restart from random init. Validate downstream
> performance on your own benchmark before relying on the upgraded ckpt for
> production work.

## Skill and runtime paths

Set `SKILL_DIR` to the absolute path of this installed skill directory. Export
`KERMT_REPO` as the absolute path to the KERMT checkout used for model
execution. The bundled container helper mounts that checkout at
`/workspace` and this skill at `/skill` (read-only). Commands inside
the container use `/skill/scripts/`; defaults are bundled in `config/`.

## Hardware requirements

Same as `kermt-continue-pretrain` (the cMIM decoder adds parameters but not
substantially; VRAM headroom should be fine). The upgrade step itself is
fast (~5 s) and CPU-only, only the subsequent continue-pretrain consumes
GPU.

## When to invoke

- User has a grover_base checkpoint (encoder-only or with vocab heads) and
  wants to extend it into a hybrid (vocab + cMIM contrastive) pretrain.
- Useful for adding the SMILES-reconstruction contrastive objective to a
  pretrained encoder without restarting pretraining from scratch (which
  `kermt-pretrain-scratch` would do at days-scale).

The rest: https://github.com/NVIDIA/skills/blob/HEAD/skills/bionemo-kermt-add-cmim-pretrain/SKILL.md