Today, I played around with local LLMs using Ollama. I ran Qwen 3.8 27B with Q3_K_S quantization, which fits easily into my RTX 5080 with 16GB VRAM.

Unfortunately, the downside was maxed-out GPU/CPU utilization (100%), pushing temperatures up to 70–80°C and spinning the fans into jet-engine mode. To keep the thermals in check, I decided to cap the power limit using the nvidia-smi utility

The command is quite simple

sudo nvidia-smi -pl 250

Where the 250 is a desired power limit maximum.

The response will be something like this

$ sudo nvidia-smi -pl 250
Power limit for GPU 00000000:02:00.0 was set to 250.00 W from 250.00 W.

Warning: persistence mode is disabled on device 00000000:02:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.

You also will observe that the maximum now set to 250 in the nvidia-smi stat header

$ nvidia-smi
Tue Aug 25 15:49:50 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.84                 Driver Version: 595.84         CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5080        Off |   00000000:02:00.0  On |                  N/A |
|  0%   58C    P1             51W /  250W |    1106MiB /  16303MiB |      5%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

To get the default and recomended values you can use

nvidia-smi -q -d POWER

The response will be something like

==============NVSMI LOG==============

Timestamp                                              : Tue Aug 25 15:46:28 2026
Driver Version                                         : 595.84
CUDA Version                                           : 13.2

Attached GPUs                                          : 1
GPU 00000000:02:00.0
    GPU Power Readings
        Average Power Draw                             : 52.41 W
        Instantaneous Power Draw                       : 54.99 W
        Current Power Limit                            : 250.00 W
        Requested Power Limit                          : 250.00 W
        Default Power Limit                            : 360.00 W
        Min Power Limit                                : 250.00 W
        Max Power Limit                                : 396.00 W
    Power Samples
        Duration                                       : 2.36 sec
        Number of Samples                              : 119
        Max                                            : 63.99 W
        Min                                            : 43.56 W
        Avg                                            : 54.07 W

Where you can see

• Requested Power Limit = 250.00 W
• Default Power Limit = 360.00 W

If you want to persist the setting after the reboot you need to enable nvidia-persistenced service

# Check is already enabled
sudo systemctl status nvidia-persistenced

# Enable for autostart and start the service
sudo systemctl enable nvidia-persistenced
sudo systemctl start nvidia-persistenced

You can also configure sudoers with the following rule if you don’t want to type your sudo password every time you change or reset the power limits:

sudo nano /etc/sudoers.d/nvidia-smi

Rules

vo    ALL=(ALL)    NOPASSWD: /usr/bin/nvidia-persistenced
vo    ALL=(ALL)    NOPASSWD: /usr/bin/nvidia-smi

!!! change “vo” to your username in the system

Set the limit on system startup

Create systemd service file

sudo nano /etc/systemd/system/nvidia-power-limit.service

with the contents

[Unit]
Description=Set NVIDIA GPU Power Limit
After=multi-user.target

[Service]
Type=oneshot
ExecStart=/usr/bin/nvidia-smi -pm 1
ExecStart=/usr/bin/nvidia-smi -pl 275
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Now just enable the service

sudo systemctl daemon-reload
sudo systemctl enable --now nvidia-power-limit.service

Check service status

systemctl status nvidia-power-limit

● nvidia-power-limit.service - Set NVIDIA GPU Power Limit
     Loaded: loaded (/etc/systemd/system/nvidia-power-limit.service; enabled; preset: enabled)
     Active: active (exited) since Mon 2026-09-07 12:01:07 CEST; 15s ago
 Invocation: a169d55e4ea148cd91ff76983a7ecbc3
    Process: 8075 ExecStart=/usr/bin/nvidia-smi -pm 1 (code=exited, status=0/SUCCESS)
    Process: 8078 ExecStart=/usr/bin/nvidia-smi -pl 275 (code=exited, status=0/SUCCESS)
   Main PID: 8078 (code=exited, status=0/SUCCESS)
   Mem peak: 17.4M
        CPU: 31ms

Check the power limit

nvidia-smi

Mon Sep  7 12:02:21 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.84                 Driver Version: 595.84         CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5080        On  |   00000000:02:00.0  On |                  N/A |
|  0%   53C    P1             40W /  275W |    1051MiB /  16303MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

As you can see here “40W / 275W” the maximum set to 275W