The cluster uses OpenPBS to schedule work across the compute racks. Please submit anything compute-heavy through the scheduler — the login node is shared by everyone.
| Command | Purpose |
|---|---|
| qsub | Submit a job |
| qstat | Show status of your jobs |
| qdel | Delete (cancel) a job |
| qrerun | Rerun a job |
| pbsnodes | List the status of all nodes in the cluster |
A job is a normal bash script with a few #PBS directives at the top telling
the scheduler the job's name, resource request, and where to write output/errors. The
compute racks currently expose one queue, stampede.
#!/bin/sh #PBS -N my_job #PBS -q stampede #PBS -l select=1:ncpus=4 #PBS -o output_file.txt #PBS -e error_file.txt cd $PBS_O_WORKDIR echo "server is $PBS_O_HOST" echo "working directory is $PBS_O_WORKDIR" module load python/3.11 python script.py
Once submitted, the job runs as if you'd just logged into a compute node — it
starts in your home directory unless you cd into $PBS_O_WORKDIR
first, as above. A few environment variables PBS sets automatically for every job:
| Variable | Meaning |
|---|---|
| PBS_O_PATH | The value of PATH on the machine you submitted from |
| PBS_O_HOST | The host you ran qsub from |
| PBS_O_WORKDIR | The absolute path of the directory you ran qsub from |
qsub my_job.pbs
Resource requests use PBS Pro/OpenPBS select syntax: chunks of nodes, each
with a number of CPUs.
#PBS -l select=1:ncpus=4 # 4 cores on one node #PBS -l select=2:ncpus=16 # 16 cores on each of 2 nodes #PBS -l select=1:ncpus=4:mem=8gb # also request memory #PBS -l walltime=08:00:00 # cap the run time at 8 hours
If you find older examples online using -l nodes=1:ppn=4, that's classic
Torque syntax and won't work here — this cluster runs OpenPBS, which uses
select=N:ncpus=M as shown above.
qstat -u <username>
pbsnodes -a
qdel <job_id>
For quick testing or debugging, request an interactive session instead of writing a script — this drops you into a shell on a compute node once resources are available.
qsub -I -l select=1:ncpus=4 -q stampede
Jupyter Lab submits a stampede job for
you automatically, and Remote Desktop (VNC) gives you a full
desktop to launch GUI-based analysis tools from.