Tue, 03 May 2011
Rethinking Cron
cron(8) is one of the oldest tools in UNIX. Despite of that,
I think cron is not something to be proud of. In my opinion, it
falls to the unfixable designs
category.
The recent attempts to fix it (factoring out atd(8),
a dirty hack that is anacron(8), etc.) show some of the problems
of cron. My recent experience confirms it:
This is the load average graph from our server, which runs periodical jobs of
IS MU. Around 2 pm, I have rewritten the
main crontab joining several similar tasks to one line, and adding several
seconds delay between their startup. The groups of tasks are now started by a
simple Perl script which handles redirecting STDOUT and STDERR, and handling
the return code. The Perl script is started using exec in the
crontab line, saving one more process.
This way, I have managed to get the number of jobs which are
simultaneously started in the peak minutes of an hour from 155 to 13.
The system does exactly the same amount of work as before, but most of the
work is evenly distributed across the whole timeframe, not started in parallel
the first second of a minute.
This is one of the big weaknesses of cron. I think the future
cron will need to support the following use cases:
- Starting jobs approximately in a given period, but not exactly at the beginning of a minute.
- Starting jobs the given period after the previous instance has finished (and maybe warn if the previous instance keeps running for too long time).
- Run the job weekly, near the beginning of the weekend (not at some random
time as
anacronand/etc/cron.weeklydoes). See Fedora bug #671076. - Start a job several times in parallel (depending on number of CPUs or something like that), and restart them after some of them finishes.
What periodical and semi-periodical tasks scheduler do you use? Will
systemd
be the answer to these problems?