ONNX WebGPU Model Converter

Convert any HuggingFace model to ONNX and run it in the browser with Transformers.js + WebGPU.

Workflow Overview

1. Check if ONNX version already exists on HuggingFace
2. Set up Python environment with optimum
3. Export model to ONNX with optimum-cli
4. Quantize for target deployment (WebGPU vs WASM)
5. Upload to HuggingFace Hub (optional)
6. Use in Transformers.js with WebGPU

Step 1: Check for Existing ONNX Models

Before converting, check if the model already has an ONNX version:

• Search onnx-community/<model-name> on HuggingFace Hub
• Check the model repo for an onnx/ folder
• Browse https://huggingface.co/models?library=transformers.js (1200+ pre-converted)

If found, skip to Step 6.

Step 2: Environment Setup

# Create venv (recommended)
python -m venv onnx-env && source onnx-env/bin/activate

# Install optimum with ONNX support
pip install "optimum[onnx]" onnxruntime

# For GPU-accelerated export (optional)
pip install onnxruntime-gpu

Verify installation:

optimum-cli export onnx --help

Step 3: Export to ONNX

Basic Export (auto-detect task)

optimum-cli export onnx --model <model_id_or_path> ./output_dir/

With Explicit Task

optimum-cli export onnx \
  --model <model_id> \
  --task <task> \
  ./output_dir/

Common tasks: text-generation, text-classification, feature-extraction, image-classification, automatic-speech-recognition, object-detection, image-segmentation, question-answering, token-classification, zero-shot-classification

For decoder models, append -with-past for KV cache reuse (default behavior): text-generation-with-past, text2text-generation-with-past, automatic-speech-recognition-with-past

Full CLI Reference

Flag	Description
-m MODEL, --model MODEL	HuggingFace model ID or local path (required)
--task TASK	Export task (auto-detected if on Hub)
--opset OPSET	ONNX opset version (default: auto)
--device DEVICE	Export device, cpu (default) or cuda
--optimize {O1,O2,O3,O4}	ONNX Runtime optimization level
--monolith	Force single ONNX file (vs split encoder/decoder)
--no-post-process	Skip post-processing (e.g., decoder merging)
--trust-remote-code	Allow custom model code from Hub
--pad_token_id ID	Override pad token (needed for some models)
--cache_dir DIR	Cache directory for downloaded models
--batch_size N	Batch size for dummy inputs
--sequence_length N	Sequence length for dummy inputs
--framework {pt}	Source framework
--atol ATOL	Absolute tolerance for validation

Optimization Levels

Level	Description
O1	Basic general optimizations
O2	Basic + extended + transformer fusions
O3	O2 + GELU approximation
O4	O3 + mixed precision fp16 (GPU only, requires --device cuda)

Step 4: Quantize for Web Deployment

Quantization Types for Transformers.js

dtype	Precision	Best For	Size Reduction
fp32	Full 32-bit	Maximum accuracy	None (baseline)
fp16	Half 16-bit	WebGPU default quality	~50%
q8 / int8	8-bit	WASM default, good balance	~75%
q4 / bnb4	4-bit	Maximum compression	~87%
q4f16	4-bit weights, fp16 compute	WebGPU + small size	~87%

Using optimum-cli quantization

# Dynamic quantization (post-export)
optimum-cli onnxruntime quantize \
  --onnx_model ./output_dir/ \
  --avx512 \
  -o ./quantized_dir/

Using Python API for finer control

from optimum.onnxruntime import ORTQuantizer, ORTModelForSequenceClassification
from optimum.onnxruntime.configuration import AutoQuantizationConfig

model = ORTModelForSequenceClassification.from_pretrained("./output_dir/")
quantizer = ORTQuantizer.from_pretrained(model)
config = AutoQuantizationConfig.avx512_vnni(is_static=False, per_channel=False)
quantizer.quantize(save_dir="./quantized_dir/", quantization_config=config)

Producing Multiple dtype Variants for Transformers.js

To provide fp32, fp16, q8, and q4 variants (like onnx-community models), organize output as:

model_onnx/
├── onnx/
│   ├── model.onnx              # fp32
│   ├── model_fp16.onnx         # fp16
│   ├── model_quantized.onnx    # q8
│   └── model_q4.onnx           # q4
├── config.json
├── tokenizer.json
└── tokenizer_config.json

Step 5: Upload to HuggingFace Hub (Optional)

# Login
huggingface-cli login

# Upload
huggingface-cli upload <your-username>/<model-name>-onnx ./output_dir/

# Add transformers.js tag to model card for discoverability

Step 6: Use in Transformers.js with WebGPU

Install

npm install @huggingface/transformers

Basic Pipeline with WebGPU

import { pipeline } from "@huggingface/transformers";

const pipe = await pipeline("task-name", "model-id-or-path", {
  device: "webgpu",    // GPU acceleration
  dtype: "q4",         // Quantization level
});

const result = await pipe("input text");

Per-Module dtypes (encoder-decoder models)

Some models (Whisper, Florence-2) need different quantization per component:

const model = await Florence2ForConditionalGeneration.from_pretrained(
  "onnx-community/Florence-2-base-ft",
  {
    dtype: {
      embed_tokens: "fp16",
      vision_encoder: "fp16",
      encoder_model: "q4",
      decoder_model_merged: "q4",
    },
    device: "webgpu",
  },
);

For detailed Transformers.js WebGPU usage patterns: See references/webgpu-usage.md

Troubleshooting

For conversion errors and common issues: See references/conversion-guide.md

Quick Fixes

• "Task not found": Use --task flag explicitly. For decoder models try text-generation-with-past
• "trust_remote_code": Add --trust-remote-code flag for custom model architectures
• Out of memory: Use --device cpu and smaller --batch_size
• Validation fails: Try --no-post-process or increase --atol
• Model not supported: Check supported architectures — 120+ architectures supported
• WebGPU fallback to WASM: Ensure browser supports WebGPU (Chrome 113+, Edge 113+)

Supported Task → Pipeline Mapping

Task	Transformers.js Pipeline	Example Model
text-classification	sentiment-analysis	distilbert-base-uncased-finetuned-sst-2
text-generation	text-generation	Qwen2.5-0.5B-Instruct
feature-extraction	feature-extraction	mxbai-embed-xsmall-v1
automatic-speech-recognition	automatic-speech-recognition	whisper-tiny.en
image-classification	image-classification	mobilenetv4_conv_small
object-detection	object-detection	detr-resnet-50
image-segmentation	image-segmentation	segformer-b0
zero-shot-image-classification	zero-shot-image-classification	clip-vit-base-patch32
depth-estimation	depth-estimation	depth-anything-small
translation	translation	nllb-200-distilled-600M
summarization	summarization	bart-large-cnn