High Throughput Computing using HTCondor

HTCondor Registration

HTCondor is provided completely free of charge to staff (including associates) and postgraduate students but we do ask that all users register for the service using Service Now (or go direct to this form). Please include a brief description of your project and try to keep us informed of any published research which has benefited from using the HTCondor service (including conference proceedings and dissertations) - this will help secure future support for HTCondor at Liverpool.

It will help us if you can briefly answer at least some of the questions below in the registration form. Even if you do not have definite responses it is worth thinking about these points as they will influence how well particular problems are suited to using HTCondor. If in doubt please contact the HTCondor administrator Ian C. Smith (email: i.c.smith@liverpool.ac.uk)

What application software is needed ?
Are you planning to use software that you (or maybe a colleague) has written - if so which language does it use e.g. C/C++, FORTRAN, MATLAB, R or Python. If third party software is needed, can it be obtained free of charge or is it a commercial package ? Are there any licensing restrictions associated with it ?
How long will the individual jobs run for ?
HTCondor works extremely well with jobs running for around 15-30 minutes but longer jobs are less efficient. It is generally not possible to run jobs for longer than about 10-12 hours unless the job can restart from where it left off (see the section on checkpointing).
Will the jobs require a lot of memory ?
Since HTCondor jobs are run on commodity PCs, memory is quite restriced. Around 16 GB is the absolute maximum which any job can use. If more memory is needed then the application may be better suited to the High Performance Computing Service.
How much disk space will be needed by individual jobs ?
HTCondor jobs should aim not to use more than a few GB of disk space on the HTCondor pool PCs as the availability of large amounts of storage cannot be guaranteed.