py-smi: NVIDIA GPU Monitor in Python

14 of december of 2024

py-smi

Disclaimer: This post has been translated to English using a machine translation model. Please, let me know if you find any mistakes.

Do you want to be able to use nvidia-smi from Python? Here you have a library to do it.

Installation

To install it, run:

pip install python-smi```

Usage

We import the library

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		from py_smi import NVML
	
	Copied

We create an object of pynvml (the library behind nvidia-smi)

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		nv = NVML()
	
	Copied

We get the version of the driver and CUDA

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		nv.driver_version, nv.cuda_version
	
	Copied

	
		>_
		Output
	
	
		('560.35.03', '12.6')

Since in my case I have two GPUs, I create a variable with the number of GPUs

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		num_gpus = 2
	
	Copied

I get the memory of each GPU

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		[nv.mem(i) for i in range(num_gpus)]
	
	Copied

	
		>_
		Output
	
	
		[_Memory(free=24136.6875, total=24576.0, used=439.3125),
 _Memory(free=23509.0, total=24576.0, used=1067.0)]

The utilization

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		[nv.utilization() for i in range(num_gpus)]
	
	Copied

	
		>_
		Output
	
	
		[_Utilization(gpu=0, memory=0, enc=0, dec=0),
 _Utilization(gpu=0, memory=0, enc=0, dec=0)]

The power used

This works very well for me, because when I was training a model and had both GPUs full, sometimes my computer would shut down, and looking at the power, I see that the second one consumes a lot, so it may be what I already suspected, which was due to power supply.

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		[nv.power(i) for i in range(num_gpus)]
	
	Copied

	
		>_
		Output
	
	
		[_Power(usage=15.382, limit=350.0), _Power(usage=40.573, limit=350.0)]

The clocks of each GPU

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		[nv.clocks(i) for i in range(num_gpus)]
	
	Copied

	
		>_
		Output
	
	
		[_Clocks(graphics=0, sm=0, mem=405), _Clocks(graphics=540, sm=540, mem=810)]

PCI Data

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		[nv.pcie_throughput(i) for i in range(num_gpus)]
	
	Copied

	
		>_
		Output
	
	
		[_PCIeThroughput(rx=0.0, tx=0.0),
 _PCIeThroughput(rx=0.1630859375, tx=0.0234375)]

And the processes (I'm not running anything now)

	
		
			<>
			Input
			Python
		
		
			
		
	
	
		[nv.processes(i) for i in range(num_gpus)]
	
	Copied

	
		>_
		Output
	
	
		[[], []]

Continue reading

Deep Research with LangGraph: Build an AI Research Assistant from Scratch

Learn how neural networks work from scratch with a practical linear regression example. This beginner-friendly tutorial explains artificial neurons, parameter initialization, loss functions, and mean squared error (MSE) with step-by-step code examples in Python.

MCP Elicitation: Implementing Elicitation in Servers with FastMCP and Python

Learn how to implement elicitation in MCP (Model Context Protocol) servers with FastMCP. Complete step-by-step tutorial ...

MCP Durability: Server and Client with Persistence for Long-Running Tasks

Learn to build durable MCP server and client for long-running tasks with persistence. Complete Model Context Protocol tu...

Last posts -->

Have you seen these projects?

Gymnasia

Horeca chatbot

Naviground

View all projects -->

>_ Available for projects

Do you have an AI project?

Let's talk.

maximofn@gmail.com

Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.

Write me LinkedIn

Do you want to watch any talk?

Tomorrow's Agents: Deciphering the Mysteries of Planning, UX and Memory

AI agents, powered by LLMs, promise to transform applications. But are they simple executors today or future intelligent collaborators? To reach their...

Create your own Apple intelligence

Learn to create an IA system to execute efficiently on a device

Last talks -->

Do you want to improve with these tips?

o1 prompt engineering

Create better prompts for o1 following an example

Memory profiler

See the memory usage of a script

DataLoader with pin_memory and num_workers

Increase DataLoader performance with pin_memory and num_workers

Last tips -->

Use this locally

Hugging Face spaces allow us to run models with very simple demos, but what if the demo breaks? Or if the user deletes it? That's why I've created docker containers with some interesting spaces, to be able to use them locally, whatever happens. In fact, if you click on any project view button, it may take you to a space that doesn't work.