Use Multiple Cores in IRAF
January 11th, 2007 by admin
Generally speaking, IRAF tasks are single-threaded, meaning they don’t take advantage of multiple cores or multiple processors. So, wanting to use all the power in my new Mac Pro – I discovered one can run tasks in the background in IRAF with an &, just like in UNIX.
So, for those of us with Intel processors that have 2 cores, or Mac Pros with 2 2-core chips, this is a good way to speed up tasks by running multiple processes simultaneously. For data reduction, this can speed things up, since each extraction of say a spectra is independent of the next one.
For example, I automate my extractions of echelle spectra. Usually I make doit files and just run them all sequentally. But now I can set up a “doit” file that contains lines like so
myspec file1 output1 &
myspec file2 output2 &
myspec file3 output3 &
myspec file4 output4
....
and then four more lines similarly. The first 3 are run in the background, the fourth is run normally… so the fifth line of this doit file doesn’t get run until the fourth (and presumably at about the same time the first 3) tasks finish. Feed this into the cl like so…
cl> cl < doit
And the end result? IRAF uses nearly 400% of my CPU… check out the screen cap below…
Update Ok this does work. If I want to log what’s happening in each “myspectask” I just need to do…
myspectask input1 output1 > log1 &
making sure the ampersand is after the redirect to a log file.
Update Number 2 Apparently, some times the next batch of 3 background tasks tries to fire up before the first three are done, and then for some reason I get a “no more background slots” error, even though I’m told that IRAF can have 20 background tasks. So, I stuck a sleep 3 after each non-background task to let the background tasks “catch up” and finish before the next four simultaneous tasks are started. That seems to work…
