Memory Calculator: Estimate LLM Memory

Memory Calculator: Estimate LLM Memory

Memory Calculationlink image 1

Disclaimer: This post has been translated to English using a machine translation model. Please, let me know if you find any mistakes.

If you want to calculate the memory you need to run a model, use this space from HuggingFace.

Frequently asked questions

What tool can I use to calculate how much GPU memory I need before downloading a model?

The tip links to the Hugging Face Space derek-thomas/transformer_calculator (https://huggingface.co/spaces/derek-thomas/transformer_calculator), embedded directly on the page via iframe (https://derek-thomas-transformer-calculator.hf.space). You just enter the model (or its parameters) and the tool estimates the memory required to run it, no install or manual math needed.

Does this calculator only cover inference, or can it also estimate training memory?

The derek-thomas Space breaks down memory for both inference and training (weights, activations, gradients and optimizer states), so you can use it both to check whether a model fits your GPU for serving and to plan how much VRAM you'll need for fine-tuning.

Continue reading

Last posts -->

Have you seen these projects?

Gymnasia

Gymnasia Gymnasia
React Native
Expo
TypeScript
FastAPI
Next.js
OpenAI
Anthropic

Mobile personal training app with AI assistant, exercise library, workout tracking, diet and body measurements

Horeca chatbot

Horeca chatbot Horeca chatbot
Python
LangChain
PostgreSQL
PGVector
React
Kubernetes
Docker
GitHub Actions

Chatbot conversational for cooks of hotels and restaurants. A cook, kitchen manager or room service of a hotel or restaurant can talk to the chatbot to get information about recipes and menus. But it also implements agents, with which it can edit or create new recipes or menus

View all projects -->
>_ Available for projects

Do you have an AI project?

Let's talk.

maximofn@gmail.com

Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.

Do you want to watch any talk?

Last talks -->

Do you want to improve with these tips?

Last tips -->

Use this locally

Hugging Face spaces allow us to run models with very simple demos, but what if the demo breaks? Or if the user deletes it? That's why I've created docker containers with some interesting spaces, to be able to use them locally, whatever happens. In fact, if you click on any project view button, it may take you to a space that doesn't work.

Flow edit

Flow edit Flow edit

FLUX.1-RealismLora

FLUX.1-RealismLora FLUX.1-RealismLora
View all containers -->
>_ Available for projects

Do you have an AI project?

Let's talk.

maximofn@gmail.com

Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.

Do you want to train your model with these datasets?

short-jokes-dataset

HuggingFace

Dataset with jokes in English

Use: Fine-tuning text generation models for humor

231K rows 2 columns 45 MB
View on HuggingFace →

opus100

HuggingFace

Dataset with translations from English to Spanish

Use: Training English-Spanish translation models

1M rows 2 columns 210 MB
View on HuggingFace →

netflix_titles

HuggingFace

Dataset with Netflix movies and series

Use: Netflix catalog analysis and recommendation systems

8.8K rows 12 columns 3.5 MB
View on HuggingFace →
View more datasets -->