<!-- examples/README-clip.md -->

# CLIP: offline mechanics and genuine released inference

`clip_training.py --smoke` needs only PyTorch and never accesses a checkpoint,
network or native-source directory. It trains reduced random paired encoders
on synthetic inputs. It does not train on WIT or produce semantic predictions.

For the separate genuine export, download these public files into one directory:

- [Training program](https://www.abhik.ai/examples/clip_training.py)
- [Exporter](https://www.abhik.ai/examples/clip_export.py)
- [Pinned requirements](https://www.abhik.ai/examples/requirements-clip.txt)
- [Native source ZIP](https://www.abhik.ai/examples/clip_native.zip)
- [Source hash record](https://www.abhik.ai/examples/clip-source-hashes.json)

Extract the ZIP there. It contains `clip_native/`, the unchanged official native
model, reference tokenizer and preprocessing source, the original BPE merge
asset, upstream README, MIT license and source manifest. The repository is
[openai/CLIP](https://github.com/openai/CLIP/tree/d05afc436d78f1c48dc0dbf8e5980a9d471f35f6),
revision `d05afc436d78f1c48dc0dbf8e5980a9d471f35f6`. Each file's original URL and
SHA256 are in the manifest and independent public hash record. The exporter
checks all of them before loading native code. No ignored research cache is
needed by these downloaded programs.

The source ZIP SHA256 is
`25f08313679a9a6766e60040e245f0742e5e0b1027b858f7bc8183b22bdbfb37`.

Download the [official ViT-B/32 checkpoint](https://openaipublic.azureedge.net/clip/models/40d365715913c9da98579312b702a82c18be219cc2a73407c4526f58eba950af/ViT-B-32.pt)
as `ViT-B-32.pt`. Its complete SHA256 must be
`40d365715913c9da98579312b702a82c18be219cc2a73407c4526f58eba950af`.
The exporter constructs the full native architecture, verifies its 151,277,313
parameters and loads every checkpoint tensor with `strict=True`. Native model
bodies are unchanged. The original scripted archive is used only for its state
dictionary; execution uses CPU float32 native PyTorch rather than JIT CUDA code.

```bash
python clip_training.py --smoke
python clip_export.py --source-dir clip_native --checkpoint ViT-B-32.pt --output clip-demo
```

The exporter retrieves two pinned CC0 scikit-image photos and verifies their
hashes. For an entirely local photo path, add `--photo-dir photo-directory`,
containing `source-cat.png` and `source-coffee.png` from the public artifact
directory. No checkpoint is downloaded by the program. Checkpoint and photo
paths are explicit; outputs remain in the selected directory.

The released input transform uses PIL bicubic short-side resize to 224, a
224×224 center crop, RGB conversion after crop, tensor conversion and CLIP
normalization: mean `(0.48145466, 0.4578275, 0.40821073)`, standard deviation
`(0.26862954, 0.26130258, 0.27577711)`. The displayed crop is a quantized PNG;
the NPZ's float32 `input` preserves the exact normalized tensor used by the model.

The tokenizer adapter supports **printable ASCII only** (U+0020 through U+007E),
both before and after two HTML-unescape passes. It rejects Unicode, all control
characters including tabs/newlines/DEL, embedded SOT/EOT strings and text that
exceeds context. It also rejects deeper HTML entity nesting that still changes
after those two passes, since unrestricted ftfy cleanup is not available.
It collapses spaces, lowercases, matches contractions before
ASCII letter runs, emits each digit separately and groups punctuation. Native
byte-to-Unicode mapping and the unchanged native BPE algorithm use the exact
official sliced merge table. Context is 77, vocabulary size 49,408, SOT 49,406,
EOT 49,407 and padding zero. This is a narrow adapter, not unrestricted parity
with the original `ftfy` and `regex` normalization. Those packages are not
required or installed. Upstream copyright and MIT license remain in the bundle.

Five candidate descriptions are `cat`, `dog`, `cup of coffee`, `bowl of soup`
and `car`. Each uses exactly two templates: `a photo of a {}.` and
`a close-up photo of a {}.`. Ten text features are normalized individually,
averaged in class-major/template-minor order and renormalized into five
prototypes. This is an inspection set, not the 80-template ImageNet protocol.
Released inference uses raw `exp(logit_scale)`; original training capped the
multiplier at 100. Softmax depends on the selected descriptions and templates
and is uncalibrated. No target-dataset accuracy is measured here.

Each NPZ saves normalized input, token IDs, raw and normalized full image/text
features, prototypes, ten native prompt logits, five candidate cosine/logit/
softmax outputs and the learned scale. `predictions.json` records exact prompts,
source/checkpoint/producer hashes, Python and package versions, image and array
hashes and native arithmetic parity receipts. Public artifact inspection can
recompute all candidate scores from the full feature arrays without re-running
the model. Reproducing floating-point values requires the recorded environment;
displayed rounded percentages are not the authoritative arrays.

Primary explanation sources: [paper §§2.3–2.5, 3.1.2 and 3.1.4](https://arxiv.org/html/2103.00020v1),
[native encoders and released scale](https://github.com/openai/CLIP/blob/d05afc436d78f1c48dc0dbf8e5980a9d471f35f6/clip/model.py),
[native tokenization](https://github.com/openai/CLIP/blob/d05afc436d78f1c48dc0dbf8e5980a9d471f35f6/clip/simple_tokenizer.py),
and [official prompt ensemble notebook](https://github.com/openai/CLIP/blob/d05afc436d78f1c48dc0dbf8e5980a9d471f35f6/notebooks/Prompt_Engineering_for_ImageNet.ipynb).
