Quick Start Information

Condor Topics

Condor Command Summary

This page gives details of a few of the most used Condor commands. For futher information on these and other Condor commands please refer to the official Condor manual command reference. The full manual contains far more information than the casual user will need but it is well worth reading the first two chapters (Overiew and Users' Manual) as an introduction to Condor.

condor_submit	- submit one or more Condor jobs using a job submission file
condor_q	- list the state of jobs in the Condor queue
condor_status	- list the state of machines making up the Condor pool
condor_rm	- remove single or multiple Condor jobs
condor_vacate_job	- force a running job to vacate a pool machine
condor_userprio	- list Condor users' current priorities
condor_prio	- set job priority

local command additions

mws_submit	- submit one or more Condor jobs using a simplified job submision file
matlab_build	- create a standalone executable from a MATLAB M-file
matlab_submit	- submit MATLAB jobs using a simplified job submision file
python_submit	- submit Python jobs using a simplified job submision file
fortran_submit	- submit Fortran jobs using a simplified job submision file
r_submit	- submit R jobs using a simplified job submision file

condor_submit

The condor_submit command can be used to submit one or more jobs to the Condor pool using a job submission file viz:

$ condor_submit <job_submission_file>

where <job_submission_file> is the name of the file. A simple example of a job submission file for a single job is:

universe = vanilla
executable = myexec.exe
input_files = myinput
output_files = myoutput
transfer_files = always
notification = never
requirements = ( Arch=="X86_64" ) && ( OpSys="WINNT61" )
queue

The queue attribute must come last and e-mail notifications should be switched off as e-mail cannot be delivered from the Condor server. The requirements attribute ensures that jobs will only run on the 64-bit Windows PCs (there may be some 32-bit machines available but these are very low spec and are best avoided).

It is often useful to include a log file which will Condor will use to provide details of the job's progress. Capturing the standard output and error can also be useful in debugging. This leads to the following job submission file:

universe = vanilla
executable = myexec.exe
input_files = myinput
output_files = myoutput
output = mystdout
error  = mystderr
log = mylog
transfer_files = always
notification = never
requirements = ( Arch=="X86_64" ) && ( OpSys="WINNT61" )
queue

To submit a batch or cluster of jobs with different input and output fils, the $(PROCESS) macro can be used to index the filenames. For example:

universe = vanilla
executable = myexec.exe
input_files = myinput$(PROCESS)
output_files = myoutput$(PROCESS)
output = mystdout$(PROCESS)
error  = mystderr$(PROCESS)
log = mylog$(PROCESS)
transfer_files = always
notification = never
requirements = ( Arch=="X86_64" ) && ( OpSys="WINNT61" )
queue 10

Condor substitutes a unique index (in the range [0..N-1]) for the $(PROCESS) macro in each job as it runs so each job will work with a different input file. It is often easier to use the simplified job submission files than the actual Condor ones. See the sections on mws_submit, matlab_submit and r_submit for further details.

condor_q

The condor_q command provides information on the state of jobs that have been submitted to the Condor pool. The information is divided into nine columns as follows

The first column shows the Job ID which is a unique identifier for each job. These identifiers are useful in removing single or multiple jobs from the queue. The integer part contains the cluster ID which is essentially the batch number and the decimal part contains the process ID which identifies a job within a cluster.
The owner of the job is shown in column two.
Columns three and four show the date and time at which the job was submitted.
The fifth column indicates how long a job has been running for since it was first submitted in the form days+hours:mins:secs. This will include any badput as well as goodput. To see how long a job has been running since it was last evicted, use the -currentrun option.
The sixth column contains probably the most useful information - namely the job state. An I in this column means that a job is idle waiting for a machine to become available to run on, The R code indicates that a job is running on a pool PC. Finally the H code means that a job has been held because of an error (see the section on troubleshooting for more information about this).
The seventh column contains the process priority - this is not of immediate interest.
The amount of memory used by a job running on a PC (in MB) is shown in column eight.
Column nine contains the name of the command being run by Condor on a pool PC. This will usually be a .bat file "wrapper" script.

Often there will be far more information displayed than will fit into the window and most of it will scroll past quickly. To display the output, one screenfull at a time use the more program e.g.

$ condor_q | more

The Condor server has been configured in an attempt to give each user their own Condor scheduler however since there are more users than schedulers, these may need to be shared somtimes. This means that you may sometimes see the details of others jobs. To list only your own jobs, just add your username to the condor_q command e.g.

$ condor_q <my_username> | more

Conversely if you want to see all of the jobs submitted by different users, the -global option is needed:

$ condor_q -global | more

Using the -analyze option is good way to find out why a job is not running:

$ condor_q -analyze <JobID>

If you get the following message, then there is most likely a mistake in the requirements specification:

WARNING:  Be advised:
   No resources matched request's constraints

There are two other options which can come in handy. The -run option displays only those jobs which are running (and where they are running) and -held option shows only those jobs in the held state.

condor_status

The condor_status command is used to list the state of all of the PCs in the Condor pool. The output is divided into eight columns as follows:

The first column lists the names of the job slots in the Condor pool. On each PC, a Condor job can be run on each processor core. The dual core machines therefore have two job slots and the quad core machines have four.
The operating system running on the PC is listed in column two. WINNT61 is Condor's idea of Windows 7.
Column three gives the processor architecture for each PC. The 32-bit machines have "INTEL" as the architecture and the 64-bit PCs have "X86_64". Since only the lowest specification machines still use 32-bit Windows these are best avoided for Condor jobs.
The fourth column contains probably the most useful information which is the machine state. The Owner state indicates that a logged in teaching centre users is occupying the PC making it unavailable to Condor. Machines in the Unclaimed state are available to run Condor jobs and once a job is running on a PC, the state will change to Claimed.
The current activity on the pool machines is shown in the fifth column. Roughly speaking, Idle indicates that a Condor job is not making use of a PC and Busy indicates that a Condor job is running on the PC.
Columns six and seven show the current load average and available memory for each job slot and are not of immediate interest.
The eighth column shows how long the job slot has been engaged in the current activity.

Often there will be far more information displayed than will fit into the window and most of it will scroll past quickly. To display the output, one screenfull at a time use the more program e.g.

$ condor_status | more

You can also get a summary of the overall use with:

$ condor_status -totals

There are few other options which are useful. The -available option lists only those machine available to run jobs and the -run option lists only those machines running Condor jobs.

condor_rm

The condor_rm command is used to remove jobs from the Condor queue. To remove a single job (i.e. a process) use:

$ condor_rm <JobID>

and to remove an entire cluster of jobs:

$ condor_rm <ClusterID>

If you wish to remove all of your Condor jobs use:

$ condor_rm -all

The command will not give you a second chance to change your mind so multiple jobs should be removed with care.

Condor will attempt to shutdown jobs gracefully which may take a siginificant amount of time. If your jobs do not disappear then the -f (force) option can be used to speed things up e.g.

$ condor_rm -f <JobID>

If jobs resolutely refuse to go away, please contact the Condor administrator.

condor_vacate_job

This command can be used to force a checkpointing job to stop running on a given PC and will cause any checkpoint files to be sent to the Condor server (for details see the section on long running jobs). The job will stay in the queue and eventually run elsewhere. Usage is straightforward:

$ condor_vacate_job <JobID>

condor_userprio

The condor_userprio command lists the current "pecking order" for Condor users, for example:

$ condor_userprio
Last Priority Update: 10/22 15:22
                                    Effective
User Name                           Priority 
------------------------------      ---------
eg0u9254@ulgp5.liv.ac.uk                 5.45
edrom@ulgp5.liv.ac.uk                    6.68
ee0u61a9@ulgp5.liv.ac.uk              1087.59
                                1100.16
------------------------------      ---------
Number of users shown: 3

Small numbers imply that the user's jobs are more likely to run. Here the first two users have roughly the same (high) priority but the jobs submitted by the third user are over 200 times less likely to run than those of the first two. Condor adjusts the relative priorities over time in attempt to give all users their fair share of resources.

condor_prio

The condor_prio command can be used to set the priority of a given job or cluster of jobs using:

$ condor_prio <JobID> -setprio <priority>

For more information, see the FAQ on: How do I get some of my jobs to run before others ?.

mws_submit

The mws_submit command provides a replacement for condor_submit and allows job submission files to be used which are far simpler than the standard Condor ones. The syntax is straighforward:

$ mws_submit <simplified_job_submission_file>

An example of a simplified job submission file for a single job, similar to that shown above is:

executable = myexec.exe
input_files = myinput
output_files = myoutput
stdout = mystdout
stderr = mystderr
log = mylog
total_jobs = 1

For multiple jobs, the simplified job submission files do away with the need to use the $(PROCESS) macro. For example:

executable = myexec.exe
indexed_input_files = myinput
indexed_output_files = myoutput
indexed_stdout = mystdout
indexed_stderr = mystderr
indexed_log = mylog
total_jobs = 10

Any of the attributes can be overidden using command line options without the need to edit the submission file. So to change the number of jobs to 20:

$ mws_submit <simplified_job_submission_file> -total_jobs=20

For more information on using simplified job submission files see the Simplified Job Submission page.

matlab_build

The matlab_build command can be used to create a standalone MATLAB executable by submitting a Condor job to do the compilation work. The command takes a single M-file as an argument:

$ matlab_build <M-file>

No syntax checks are peformed and an incorrect M-file will still be compiled into an executable. These errors are can be very difficult to spot so it worth checking that the M-file is correct first. For more information see the section on MATLAB Applications.

matlab_submit

The matlab_submit command can be used to submit standalone MATLAB jobs to the Condor pool. The command takes a simplified job submission file as an argument: $ matlab_submit <simplified job submission file> An example of a simplified job submission file is:

executable = standalone.exe
indexed_input_files = myinput
indexed_output_files = myoutput
indexed_stdout = mystdout
indexed_stderr = mystderr
indexed_log = mylog
total_jobs = 10

Any of the attributes can be overidden using command line options without the need to edit the submission file. So to change the number of jobs to 20:

$ mws_submit <simplified_job_submission_file> -total_jobs=20

For more information see the section on MATLAB Applications.

python_submit

The python_submit command can be used to submit jobs based on Python applications to the Condor pool. The command takes a simplified job submission file as an argument:

$ python_submit <simplified job submission file>

An example of a simplified job submission file is:

python_script = myscript.py
indexed_input_files = myinput
indexed_output_files = myoutput
indexed_stdout = mystdout
indexed_stderr = mystderr
indexed_log = mylog
total_jobs = 10

Any of the attributes can be overidden using command line options without the need to edit the submission file. So to change the number of jobs to 20:

$ python_submit <simplified_job_submission_file> -total_jobs=20

For more information see the section on Python Applications.

fortran_submit

The fortran_submit command can be used to submit jobs based on Fortran applications to the Condor pool. The command takes a simplified job submission file as an argument:

$ fortran_submit <simplified job submission file>

An example of a simplified job submission file is:

executable = myapp.exe
indexed_input_files = myinput
indexed_output_files = myoutput
indexed_stdout = mystdout
indexed_stderr = mystderr
indexed_log = mylog
total_jobs = 10

Any of the attributes can be overidden using command line options without the need to edit the submission file. So to change the number of jobs to 20:

$ python_submit <simplified_job_submission_file> -total_jobs=20

For more information see the section on Fortran Applications.

r_submit

The r_submit command can be used to submit jobs based on R applications to the Condor pool. The command takes a simplified job submission file as an argument:

$ r_submit <simplified job submission file>

An example of simplified job submission file is:

r_script = myscript.r
indexed_input_files = myinput
indexed_output_files = myoutput
indexed_stdout = mystdout
indexed_stderr = mystderr
indexed_log = mylog
total_jobs = 10

Any of the attributes can be overidden using command line options without the need to edit the submission file. So to change the number of jobs to 20:

$ r_submit <simplified_job_submission_file> -total_jobs=20

For more information see the section on R Applications.

This website will look much better in a web browser that supports web standards, but it is accessible to any browser or Internet device.