← Journal · · Deep dive
How to store HuggingFace model weights in S3
Hey everyone, I just ran into the task of caching HF weights in a local S3 to use with vLLM. Got some interesting results.
What you need to know about the HF model cache? The cache is usually stored in ~/.cache/huggingface/hub
blobs/: contains the binary files of models and datasets. refs/: stores info about the latest revisions of repos. snapshots/: contains snapshots of specific repo revisions as soft links.
There's also chunk-based caching (Xet) being rolled out now - a caching approach where large files (like model weights) get split into small fixed-size chunks. But not all repos have moved from Git LFS to Xet yet, so let's skip this option for now.
The problem with loading the cache into S3 The snapshots folder stores soft links, and you can't just dump those into S3 storage.
Options
1️⃣ Copying the HF cache via rclone 👎 download the weights
huggingface-cli download gpt2
and upload to S3 via rclone with the --copy-links flag
rclone copy --copy-links ~/.cache/huggingface/hub/models--gpt2 <s3_config>:<bucket>/path
this uploads copies of the files instead of soft links.
But this doubles the size, so this option doesn't work.
We could pack the cache into a tar archive. Then we could upload to S3 without issues, but that adds archiving/unarchiving operations, which eats time. Worth trying the zstd format.
2️⃣ Direct weight download and import via local path 👍 Download the weights into a specific folder. Don't forget the fast method.
HF_HUB_ENABLE_HF_TRANSFER=True huggingface-cli download --local-dir hf_cache/ Qwen/Qwen3-1.7B
Then sync to S3
rclone copy hf_cache/ <s3_config>:<bucket>/path
in vLLM, instead of the HF path Qwen/Qwen3-1.7B, use the local path to the weights downloaded from S3
{ "model":"/root/models/hf_cache", "served_model_name": "Qwen3-1.7B" }
Simple and elegant :)
3️⃣ Tensorizing 😩 An approach based on CoreWeave's Tensorizer. Here we're solving not just the problem of storing weights in S3, but also fast loading into the GPU from the local file system!
It deserves its own post, so stay tuned :)