HuggingFaceModelDownloader

📸 Screenshots

✨ Features

🚀 Quick Start

One-Liner Commands (Linux / macOS / WSL)

# 🌐 Instant Web UI - opens browser automatically!
bash <(curl -sSL https://g.bodaay.io/hfd) -w

# 📦 Download a model
bash <(curl -sSL https://g.bodaay.io/hfd) download TheBloke/Mistral-7B-GGUF

# 💾 Install for regular use
bash <(curl -sSL https://g.bodaay.io/hfd) -i

After Installation

# Download a GGUF model
hfdownloader download TheBloke/Mistral-7B-Instruct-v0.2-GGUF

# Download specific quantizations
hfdownloader download TheBloke/Mistral-7B-Instruct-v0.2-GGUF:q4_k_m,q5_k_m

# Download a dataset
hfdownloader download facebook/flores --dataset

# Start web interface
hfdownloader serve

That's it! Files are saved to ./Storage/<repo-name>/ by default.

📥 Installation

Option 1: One-Liner Script (Recommended)

# 🌐 Start web UI immediately (no installation, opens browser)
bash <(curl -sSL https://g.bodaay.io/hfd) -w

# 💾 Install to /usr/local/bin
bash <(curl -sSL https://g.bodaay.io/hfd) -i

# 📂 Install to custom location
bash <(curl -sSL https://g.bodaay.io/hfd) -i -p ~/.local/bin

# 📦 Download without installing
bash <(curl -sSL https://g.bodaay.io/hfd) download TheBloke/Mistral-7B-GGUF

# 🌐 Web UI on custom port
bash <(curl -sSL https://g.bodaay.io/hfd) -w 3000

Option 2: Download Binary

Download from Releases:

Option 3: Build from Source

git clone https://github.com/bodaay/HuggingFaceModelDownloader
cd HuggingFaceModelDownloader
go build -o hfdownloader ./cmd/hfdownloader

Option 4: Docker 🐳

# Build the image
docker build -t hfdownloader .

# Run CLI in container
docker run --rm -v ./models:/data hfdownloader download TheBloke/Mistral-7B-GGUF -o /data

# Run web server
docker run --rm -p 8080:8080 -v ./models:/data hfdownloader serve \
  --models-dir /data/Models \
  --datasets-dir /data/Datasets

# With HuggingFace token
docker run --rm -e HF_TOKEN=hf_xxx -p 8080:8080 hfdownloader serve

🌐 Web UI

Start the web server for a browser-based experience:

hfdownloader serve
# Open http://localhost:8080

Features:

• 📊 Real-time download progress via WebSocket
• 📁 Separate pages for Models and Datasets
• ⚙️ Settings management
• 📜 Download history

Server Options:

hfdownloader serve \
  --port 3000 \
  --models-dir ./Models \
  --datasets-dir ./Datasets \
  --token hf_xxx \
  --endpoint https://hf-mirror.com  # Optional: use mirror

💻 CLI Reference

hfdownloader [command] [flags]

Commands

Download Examples

# Basic download
hfdownloader download TheBloke/Mistral-7B-GGUF -o ./Models

# With authentication (private/gated repos)
hfdownloader download meta-llama/Llama-2-7b -t hf_xxxxx
# Or use environment variable
export HF_TOKEN=hf_xxxxx

# Filter specific files (case-insensitive)
hfdownloader download TheBloke/Mistral-7B-GGUF:q4_k_m,q5_k_m

# Exclude files by pattern
hfdownloader download TheBloke/Mistral-7B-GGUF -E .md,fp16,onnx

# Combine filters and excludes
hfdownloader download owner/repo -F q4_k_m,q5_k_m -E .md

# Organize filtered files into subdirs
hfdownloader download TheBloke/Mistral-7B-GGUF:q4_0,q8_0 --append-filter-subdir
# Result: ./Storage/Mistral-7B-GGUF/q4_0/file.gguf
#         ./Storage/Mistral-7B-GGUF/q8_0/file.gguf

# Download a dataset
hfdownloader download facebook/flores --dataset -o ./Datasets

# Specific revision/branch
hfdownloader download owner/repo -b v1.0

# Dry run (preview files)
hfdownloader download owner/repo --dry-run --plan-format json

# Use HuggingFace mirror (e.g., for China)
hfdownloader download owner/repo --endpoint https://hf-mirror.com

Flags Reference

⚙️ Configuration

Create a config file to set defaults:

# Generate config file
hfdownloader config init        # JSON
hfdownloader config init --yaml # YAML

# Show current config
hfdownloader config show

Example Config (YAML)

# ~/.config/hfdownloader.yaml
output: ~/Models
connections: 12
max-active: 4
multipart-threshold: 64MiB
verify: sha256
retries: 6
token: hf_xxxxxxxxxx

Example Config (JSON)

{
  "output": "~/Models",
  "connections": 12,
  "max-active": 4,
  "multipart-threshold": "64MiB",
  "verify": "sha256",
  "retries": 6,
  "token": "hf_xxxxxxxxxx"
}

📚 Go Library

Use as a Go library for programmatic downloads:

package main

import (
  "context"
    "fmt"
  "log"

    "github.com/bodaay/HuggingFaceModelDownloader/pkg/hfdownloader"
)

func main() {
  job := hfdownloader.Job{
        Repo:     "TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
        Revision: "main",
        Filters:  []string{"q4_k_m"},
  }

    cfg := hfdownloader.DefaultSettings()
    cfg.OutputDir = "./Models"
    cfg.Concurrency = 8
    cfg.Token = os.Getenv("HF_TOKEN")

    err := hfdownloader.Download(context.Background(), job, cfg, 
        func(e hfdownloader.ProgressEvent) {
            if e.Event == "file_progress" {
                fmt.Printf("\r%s: %.1f%%", e.Path, 
                    float64(e.Bytes)/float64(e.Total)*100)
    }
        })

    if err != nil {
    log.Fatal(err)
  }
    fmt.Println("\n✅ Download complete!")
}

Progress Events

🔌 REST API

When running hfdownloader serve, a REST API is available:

Example: Start Download via API

curl -X POST http://localhost:8080/api/download \
  -H "Content-Type: application/json" \
  -d '{"repo": "TheBloke/Mistral-7B-GGUF", "filters": ["q4_k_m"]}'

🔄 How Resume Works

┌─────────────────────────────────────────────────────────────┐
│                    Resume Decision Flow                      │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│   File exists locally?                                      │
│        │                                                    │
│        ├── No  → Download                                   │
│        │                                                    │
│        └── Yes → Is it LFS with SHA-256?                   │
│                      │                                      │
│                      ├── Yes → SHA match? → Skip ✓          │
│                      │              │                       │
│                      │              └── No → Re-download    │
│                      │                                      │
│                      └── No → Size match? → Skip ✓          │
│                                    │                        │
│                                    └── No → Re-download     │
│                                                             │
└─────────────────────────────────────────────────────────────┘

No metadata files — resume is purely filesystem-based. If a file exists and matches, it's skipped.

🛠️ Troubleshooting

export HF_TOKEN=hf_xxxxx
hfdownloader download meta-llama/Llama-2-7b

hfdownloader download repo/name -c 16 --max-active 4

📝 License

Apache 2.0 — use freely in personal and commercial projects.