Gpu Sentry

Overview

GPU-Sentry is a Flask-based package designed for efficiently monitoring the utilization of nVidia GPUs across multiple machines. Instead of manually checking each machine using the nvidia-smi command, GPU-Sentry automates the process by sending GPU memory utilization statistics to a central server, which then aggregates and displays this information on a single website. This not only makes monitoring easier but also enhances productivity by providing real-time insights into GPU performance.

The package is straightforward to install and configure, making it accessible even for those who may not have extensive technical expertise. By leveraging this tool, users can keep track of their GPU resources effectively, ensuring that they are utilized optimally and helping to diagnose any potential issues promptly.

Features

• Real-Time Monitoring: Collects GPU memory utilization statistics from multiple machines and displays them on a single dashboard.

• Automated Data Collection: Avoids manual checks by having clients send information to the central server periodically.

• Simplified Installation: Quickly set up the package using a simple pip install command, integrating all necessary dependencies.

• Configurable Client Timeout: Easily adjust the frequency of data reporting from GPU machines by changing the CLIENT_TIMEOUT setting.

• Selective Machine Display: Use the PERMIT_CLIENTS setting to filter which machine statistics are shown, allowing for a more organized overview.

• Flexible Execution: Although not designed as a background process, the package can be run using tools like screen or tmux, ensuring flexibility in how you manage resources.

• User-Friendly Interface: Aggregates statistics in a coherent format on a web interface, making it easy to access and interpret GPU utilization data.