See the memory usage of a script

30 of january of 2025

View the memory usage of a script

Disclaimer: This post has been translated to English using a machine translation model. Please, let me know if you find any mistakes.

If a script is consuming a lot of RAM, we can monitor it to see what is consuming it.

Installation

To monitor it, we are going to install memory_profiler. We can install it with Conda.

conda install memory_profiler```
      
      or with pip
      ``` bash
      pip install memory_profiler```

Usage

Decorator

We can put a decorator on the function we want to monitor.

from memory_profiler import profile
      @profiledef my_function():pass```

	
		
			< >
			Input
		
		
			Python
			
		
	
	
		%%writefile memory_profiler_decorator.py
from memory_profiler import profile
@profile
def my_function():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a
if __name__ == '__main__':
    my_function()
	
	Copied

>_ Output

			
				Overwriting memory_profiler_decorator.py

Now we run it using -m memory_profiles memory_profiler_decorator.py

	
		
			< >
			Input
		
		
			Python
			
		
	
	
		!python -m memory_profiler memory_profiler_decorator.py
	
	Copied

>_ Output

			
				Filename: memory_profiler_decorator.py
Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     3     49.5 MiB     49.5 MiB           1   @profile
     4                                         def my_function():
     5     57.0 MiB      7.5 MiB           1       a = [1] * (10 ** 6)
     6    209.6 MiB    152.6 MiB           1       b = [2] * (2 * 10 ** 7)
     7     57.2 MiB   -152.4 MiB           1       del b
     8     57.2 MiB      0.0 MiB           1       return a

As we can see, we get the memory usage of each line of the script.

Memory usage over time

We can obtain a memory usage graph over time by doing

mprof run memory_profiler_decorator.pymprof plot```

	
		
			< >
			Input
		
		
			Python
			
		
	
	
		!mprof run memory_profiler_decorator.py
	
	Copied

>_ Output

			
				mprof: Sampling memory every 0.1s
running new process
running as a Python program...
Filename: memory_profiler_decorator.py
Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     3     49.5 MiB     49.5 MiB           1   @profile
     4                                         def my_function():
     5     57.0 MiB      7.5 MiB           1       a = [1] * (10 ** 6)
     6    209.6 MiB    152.6 MiB           1       b = [2] * (2 * 10 ** 7)
     7     57.2 MiB   -152.4 MiB           1       del b
     8     57.2 MiB      0.0 MiB           1       return a

	
		
			< >
			Input
		
		
			Python
			
		
	
	
		!mprof plot
	
	Copied

>_ Output

			
				Using last profile data.
Figure(1260x540)

When running it, this graph is created memory_profiler plot

Frequently asked questions

How do I interpret the "Increment" column in memory_profiler's output?

When you decorate a function with @profile (from from memory_profiler import profile) and run it with python -m memory_profiler script.py, each line shows "Mem usage" (total process memory at that point) and "Increment" (the change from the previous line). In the example, b = [2] * (2 * 10 ** 7) has an Increment of 152.6 MiB (what that list occupied), and after del b the Increment is -152.4 MiB, showing the memory that got freed.

How can I get a graph of a script's memory usage over time instead of just a line-by-line table?

Instead of (or in addition to) python -m memory_profiler, use mprof run script.py followed by mprof plot. mprof samples memory every 0.1s ("mprof: Sampling memory every 0.1s") and produces a time-series plot of the whole process's RAM usage, which is useful for spotting spikes the line-by-line table doesn't make as obvious.

Continue reading

Gymnasia: architecture of an AI agent that runs on the phone

How the Gymnasia AI agent is built: a BYOK conversational coach with 12 local tools and a vision subagent that estimates macros from photos, both running entirely on the phone with no backend.

Deep Research with LangGraph (3/3): the Writer agent and final report

Third and final part of the Deep Research with LangGraph series. Implement the Writer agent that drafts the final report...

Deep Research with LangGraph (2/3): the multi-agent Research Supervisor

Second part of the Deep Research with LangGraph series. Build the Research Supervisor that coordinates several Researche...

Last posts -->

Have you seen these projects?

Gymnasia

LangGraph Deep Researcher

Tau

View all projects -->

>_ Available for projects

Do you have an AI project?

Let's talk.

maximofn@gmail.com

Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.

Write me LinkedIn

Do you want to watch any talk?

Tomorrow's Agents: Deciphering the Mysteries of Planning, UX and Memory

AI agents, powered by LLMs, promise to transform applications. But are they simple executors today or future intelligent collaborators? To reach their...

Create your own Apple intelligence

Learn to create an IA system to execute efficiently on a device

Last talks -->

Do you want to improve with these tips?

Best practices building agents with Claude Code

Technical talk: skills, subagents, slash commands and MCPs in Claude Code

o1 prompt engineering

Create better prompts for o1 following an example

DataLoader with pin_memory and num_workers

Increase DataLoader performance with pin_memory and num_workers

Last tips -->

Use this locally

Hugging Face spaces allow us to run models with very simple demos, but what if the demo breaks? Or if the user deletes it? That's why I've created docker containers with some interesting spaces, to be able to use them locally, whatever happens. In fact, if you click on any project view button, it may take you to a space that doesn't work.

Flow edit

FLUX.1-RealismLora

token_hmr

View all containers -->

>_ Available for projects

Do you have an AI project?

Let's talk.

maximofn@gmail.com

Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.

Write me LinkedIn

Do you want to train your model with these datasets?

short-jokes-dataset

HuggingFace

Dataset with jokes in English

Use: Fine-tuning text generation models for humor

231K rows 2 columns 45 MB

View on HuggingFace →

opus100

HuggingFace

Dataset with translations from English to Spanish

Use: Training English-Spanish translation models

1M rows 2 columns 210 MB

View on HuggingFace →

netflix_titles

HuggingFace

Dataset with Netflix movies and series

Use: Netflix catalog analysis and recommendation systems

8.8K rows 12 columns 3.5 MB

View on HuggingFace →

View more datasets -->