Hi,
this post (and a few more) is the reason why I created this whole blog. I would like to write something about analysis of astronomical data in IRAF software and to point out steps which were most problematic for me as a first-time-user of this program.
Websites you should see before or during reading this blog (or before your data reduction):
***WARNING***
This is not a manual and I do not guarantee that this is exactly what you should do with your data. As the sources of all wisdom in this post were used IRAF manuals obtained mainly from this link but there is also a lot of other websites, blogs, forums etc. where I was looking for help. I hope this post will be helpful to you and if there is anything wrong/something you are not certain of feel free to send me an e-mail or leave a comment.
***
***NOTE***
If you have already installed IRAF and know the basics like what is package and how to edit task parameters you can skip the introduction and basics parts right to my next post - 'Data Analysis in IRAF - First reduction steps'.
***
***NOTE***
If you have already installed IRAF and know the basics like what is package and how to edit task parameters you can skip the introduction and basics parts right to my next post - 'Data Analysis in IRAF - First reduction steps'.
***
Introduction
WTF is IRAF, where and how can I get it?
IRAF - the Image Reduction and Analysis Facility is a general purpose software written by NOAO and operated by other important institutions. The most important thing you should know about this program is that it can analyse astronomical data from (nearly??) all observatories worldwide (though, a lot of them have their own specialized software).
The next most important information about it is that it do not run under the Windows! If you have Mac you are lucky guys and if you have Linux you are even luckier but if you use only Windows you will have a hard time.
***NOTE***
I was that only one unlucky case half a year ago and I managed to do it so it can´t be that hard.
***
***NOTE***
I was that only one unlucky case half a year ago and I managed to do it so it can´t be that hard.
***
The best way to get the IRAF is to look at IRAF Website where you can find installation files to download.
IRAF Installation (A short story of my beginning with IRAF)
When I first heared about the IRAF it was something like: "That program can do everything you want but you will have only one wish after the installation - never use it!" So for me, as never-before-used-linux-terminal-user-of-windows, it was pretty scary and I was really fearing this software (important poit: the latest version of IRAF in that time was 2.15.1). I installed Ubuntu 12.04 and I got used to it pretty fast. After a while I thought it is the time to install IRAF. I opened IRAF Website and downloaded corresponding version. I opened README file, read it, closed it and I knew I was not prepared yet.
Fortunately, I found this great youtube installation guide Really great installation manual and slit spectra reduction guide on youtube so I tried it and I got ONLY ONE ERROR in the installation and I WAS ABLE to solve the problem (now you can imagine how amazingly I had to feel at that moment).
OK. That was my story... You have one big disadvantage now when new version 2.16 was released because you will not experience anything like this. A new version has as simple installation as installation can be (it is even easier than any installation on windows). All you have to do is to download installation package, move it to Iraf directory (or any directory you want to have your IRAF installed in), unpack downloaded file and in terminal type ./install (more detailed description can be found in the description of first video on youtube link). There are some additional libraries you need to install but it can be done in a few minutes depending of the speed of your internet connection.
There is also one more thing I have to say in this section. You can install a huge number of additional packages into IRAF depending on what you want and need to do with your data. Instead of installing all of them manually there is one really simple way to install a lot of them with IRAF and some more useful software. It is called Ureka. Installation of Ureka is even simpler than the IRAF v.2.16. All you need to know about Ureka can be found on Ureka Website.
***
NOTE: At first I have installed IRAF and only then I have installed Ureka for additional packages so I have both of them in my computer. There is no need for this but I found it simpler this way because there was an accident with root privileges while installing Ureka.
***
A really useful software you probably know - DS9
This is probably best known astronomical software. It is used for displaying fits files. Just in case you do not know about it yet, this is the best time to download it from this site SAOImage DS9.
And what now...
Great. So now I assume you have installed IRAF and you know how to run it (I really recommend you to watch first three videos on youtube link - they really helped me!!!) and also you have installed DS9. And of course you know basic commands in Linux terminal (cd/cp/mk/rm...)
When you start IRAF (I am talking about v.2.16 by default) you are in the directory from which you started it. If you want to change directory use basic task
cd
***NOTE***
When you are writing full path you can´t use the '~' symbol instead of home directory as it is in Linux terminal!
***
All commands of Linux terminal can be run only with '!' in front of it so instead of:
cl> rm file_to_remove
you have to write it this way:
cl> !rm file_to_remove
Now, when you are located in the directory which contains data, you want to start using tasks. All IRAF tasks are located in packages. The list of all loaded packages will be print after typing
cl> package
***NOTE***
You do not type cl>. It is just the beginning of command line.
***
and list of all tasks currently loaded and tasks which are present in current package will be displayed after
cl> ??
and
cl> ?
respectively. When you want to go to specific package, you just type its name. For example, if I want to work in imred package which is located in noao package I will type:
cl> noao
noao> imred
***NOTE***
Again, you do not type cl> or noao>. It is just the beginning of command line.
***
Another useful thing is (but also something you should take care of) that you do not have to type whole name of package (or task) to run it. All you need to type is unambiguous abbreviation of given package. For example, when I want to work in echelle package located in imred package you just type:
imred> ec
echelle>
Editing tasks
When you get to needed package it is time to use specific tasks. Firstly, you want to edit some parameters in task for the reduction/analysis to be done correctly. This can be done in a few ways.
ccdred> epar ccdlist
and you get list of all parameters with their values and a short description:
_______________________________________________________
PACKAGE = ccdred
TASK = ccdlist
images = CCD images to listed
(ccdtype= ) CCD image type to be listed
(names = no) List image names only?
(long = no) Long format listing?
(ccdproc= ) CCD processing parameters
(mode = ql)
_______________________________________________________
You can move up and down and you can change values in parameter you need. When you are done with editing, you close the window by typing
:q
This method is used mainly for editing tasks which are used for a lot of data obtained by the same instrument or for data which need to be treated in the same way.
However, I really recommend you to check the values of all parameters in task before every run by epar.
If you need to get back to the default values of some task (for example ccdlist) you can use:
ccdred> unlearn ccdlist
If you need to know more about the specific task and its parameters, run:
cl> help task_name
You can also change values of parameters from the command line. This is done by writing.
task_name.parameter_name="value"
So if you want to change parameter ccdtype in ccdlist task you can do it like this:
cl> ccdlist.ccdtype=zero
Another way to set needed values is to set them while executing task. Big advantage of this method is that it uses given values only once and do not change the default values of task permanently. This is also a best way to set the list of images which should be processed by the task (if you want to run task only on one image just type its name). For example, I want to list all bias (zero) data in directory (ccdlist lists data and output some important header parameters):
ccdred> ccdlist * ccdtype=zero
As you can see, the first value after task name is list of input images and all other parameters have to be write in the form 'parameter_name'='value'. Here you can also use the unambiguous abbreviations:
ccdred> ccdl * ccdt=zero
Finaly, some parameters can be edited right after the task is run. For example, every time you do not specify the input in command line, you will be asked for input list:
ccdred> epar ccdlist
_______________________________________________________
PACKAGE = ccdred
TASK = ccdlist
images = * CCD images to listed
(ccdtype= zero) CCD image type to be listed
(names = no) List image names only?
(long = no) Long format listing?
(ccdproc= ) CCD processing parameters
(mode = ql)
CCD images to listed (*):
and if you are happy with value in brackets you just confirm it by enter.
***
I hope it is all. If I find anything I forgot to write about I will add it instantly. If there is ANYTHING unclear (any you-think-they-are-stupid questions are welcome - I know exactly what you are going through) leave a comment or send me e-mail.
Enjoy
Jakub
Fortunately, I found this great youtube installation guide Really great installation manual and slit spectra reduction guide on youtube so I tried it and I got ONLY ONE ERROR in the installation and I WAS ABLE to solve the problem (now you can imagine how amazingly I had to feel at that moment).
OK. That was my story... You have one big disadvantage now when new version 2.16 was released because you will not experience anything like this. A new version has as simple installation as installation can be (it is even easier than any installation on windows). All you have to do is to download installation package, move it to Iraf directory (or any directory you want to have your IRAF installed in), unpack downloaded file and in terminal type ./install (more detailed description can be found in the description of first video on youtube link). There are some additional libraries you need to install but it can be done in a few minutes depending of the speed of your internet connection.
There is also one more thing I have to say in this section. You can install a huge number of additional packages into IRAF depending on what you want and need to do with your data. Instead of installing all of them manually there is one really simple way to install a lot of them with IRAF and some more useful software. It is called Ureka. Installation of Ureka is even simpler than the IRAF v.2.16. All you need to know about Ureka can be found on Ureka Website.
***
NOTE: At first I have installed IRAF and only then I have installed Ureka for additional packages so I have both of them in my computer. There is no need for this but I found it simpler this way because there was an accident with root privileges while installing Ureka.
***
A really useful software you probably know - DS9
This is probably best known astronomical software. It is used for displaying fits files. Just in case you do not know about it yet, this is the best time to download it from this site SAOImage DS9.
And what now...
Great. So now I assume you have installed IRAF and you know how to run it (I really recommend you to watch first three videos on youtube link - they really helped me!!!) and also you have installed DS9. And of course you know basic commands in Linux terminal (cd/cp/mk/rm...)
Basics
What I needed most at this stage was some introduction of commands I can use in IRAF terminal. There is that Beginners guide but I do not have time to read all of those pages full of unnecessary (at least for me in that stage of learning to use IRAF) information.
Changing directories and packages
When you start IRAF (I am talking about v.2.16 by default) you are in the directory from which you started it. If you want to change directory use basic task
cd
***NOTE***
When you are writing full path you can´t use the '~' symbol instead of home directory as it is in Linux terminal!
***
All commands of Linux terminal can be run only with '!' in front of it so instead of:
cl> rm file_to_remove
you have to write it this way:
cl> !rm file_to_remove
Now, when you are located in the directory which contains data, you want to start using tasks. All IRAF tasks are located in packages. The list of all loaded packages will be print after typing
cl> package
***NOTE***
You do not type cl>. It is just the beginning of command line.
***
and list of all tasks currently loaded and tasks which are present in current package will be displayed after
cl> ??
and
cl> ?
respectively. When you want to go to specific package, you just type its name. For example, if I want to work in imred package which is located in noao package I will type:
cl> noao
noao> imred
***NOTE***
Again, you do not type cl> or noao>. It is just the beginning of command line.
***
Another useful thing is (but also something you should take care of) that you do not have to type whole name of package (or task) to run it. All you need to type is unambiguous abbreviation of given package. For example, when I want to work in echelle package located in imred package you just type:
imred> ec
echelle>
Editing tasks
When you get to needed package it is time to use specific tasks. Firstly, you want to edit some parameters in task for the reduction/analysis to be done correctly. This can be done in a few ways.
- eparam task
- in command line
- as part of task execution
- answering questions after task is run
ccdred> epar ccdlist
and you get list of all parameters with their values and a short description:
_______________________________________________________
PACKAGE = ccdred
TASK = ccdlist
images = CCD images to listed
(ccdtype= ) CCD image type to be listed
(names = no) List image names only?
(long = no) Long format listing?
(ccdproc= ) CCD processing parameters
(mode = ql)
_______________________________________________________
:q
This method is used mainly for editing tasks which are used for a lot of data obtained by the same instrument or for data which need to be treated in the same way.
However, I really recommend you to check the values of all parameters in task before every run by epar.
If you need to get back to the default values of some task (for example ccdlist) you can use:
ccdred> unlearn ccdlist
If you need to know more about the specific task and its parameters, run:
cl> help task_name
You can also change values of parameters from the command line. This is done by writing.
task_name.parameter_name="value"
So if you want to change parameter ccdtype in ccdlist task you can do it like this:
cl> ccdlist.ccdtype=zero
Another way to set needed values is to set them while executing task. Big advantage of this method is that it uses given values only once and do not change the default values of task permanently. This is also a best way to set the list of images which should be processed by the task (if you want to run task only on one image just type its name). For example, I want to list all bias (zero) data in directory (ccdlist lists data and output some important header parameters):
ccdred> ccdlist * ccdtype=zero
As you can see, the first value after task name is list of input images and all other parameters have to be write in the form 'parameter_name'='value'. Here you can also use the unambiguous abbreviations:
ccdred> ccdl * ccdt=zero
ccdred> epar ccdlist
_______________________________________________________
PACKAGE = ccdred
TASK = ccdlist
images = * CCD images to listed
(ccdtype= zero) CCD image type to be listed
(names = no) List image names only?
(long = no) Long format listing?
(ccdproc= ) CCD processing parameters
(mode = ql)
_______________________________________________________
ccdred> ccdlistCCD images to listed (*):
and if you are happy with value in brackets you just confirm it by enter.
Preparations
Before any data reduction, you need to obtain data you want to analyze. All data which were made more than one year before are publicly available and can be downloaded from corresponding science archive (this is probably not complete list of Science archives of the biggest ground and space observatories - software means if there is any specialized program in which data can be processed and analyzed. NOTE - if there is no software written, there is no recommended software on the archive webpage).
After the downloading, you need to unzip/untar/uncompress your data (depending on how they were compressed). I also recommend you to save all data of one dataset in one directory.
After the downloading, you need to unzip/untar/uncompress your data (depending on how they were compressed). I also recommend you to save all data of one dataset in one directory.
***IMPORTANT***
When you are downloading scientific data for further processing and analysis you need to download also all associated files (bias frames, dark frames (optional), flat frames, lamp spectra (ThAr, CuAr depending on specific observatory and instrument) and, if exist, files for flux calibration)
***NOTE***
If you have any problems with downloading all associated files feel free to contact me.
If you have any problems with downloading all associated files feel free to contact me.
***
There is one more thing left before we start with actual analysis. IRAF is able to work with specific kind of data (for example only with bias data, flat data or scientific frames) but it has to know which data are which. This information is written in fits headers under specific keyword. This keyword is IMAGETYP and its value is translated by the task
setinstrument
To find out, if IRAF recognizes value of your IMAGETYP run the task ccdlist on all your data:
What you need to do is to edit source file of this task. At first you need to check values of IMAGETYP keyword in headers of all types of your data. Then go to the folder where are stored source files of all instruments:
ccdred> ccdlist *
You will get list of data in directory in this form:
0332.fits[1028,1500][real][unknown][][OTZF]:NGC6720
Translation of IMAGETYP is in the third square brackets. If there is unknown value, IRAF do not recognize it.
Translation of IMAGETYP is in the third square brackets. If there is unknown value, IRAF do not recognize it.
What you need to do is to edit source file of this task. At first you need to check values of IMAGETYP keyword in headers of all types of your data. Then go to the folder where are stored source files of all instruments:
cd Iraf/noao/imred/ccdred/ccddb/kpno
For every instrument there are two files - .cl and .dat. I was working on spectroscopic data so I was using specphot instrument which is default for this type of data. (If you want to know specifications of each instrument, in IRAF terminal type: 'ccdred> setinstrument ?' and you get list of all instruments). All you need to do is to insert values of IMAGETYP keyword into .dat file of instrument you want to use.
***
***
EXAMPLE
Scientific frames have value of given keyword OBJECT,POINT and flat frames LAMP,FLAT. I want to use specphot instrument.
Scientific frames have value of given keyword OBJECT,POINT and flat frames LAMP,FLAT. I want to use specphot instrument.
cd ~/Iraf/noao/imred/ccdred/ccddb/kpno
nano specphot.dat // for editing this file you can use any editor
I got this table:
subset filters
DARK dark
BIAS zero
OBJECT object
'DOME FLAT' flat
'PROJECTOR FLAT' flat
'COMPARISON' comp
'SKY FLAT' object
and under the last line I insert keyword value from my data
'OBJECT,POINT' object
'LAMP,FLAT' flat
Now you save it (in nano editor: F3 save; F2 exit). If you want to check if you done it well, open IRAF terminal, go to directory with your data and type:
ccdred> setinstrument ?
direct Current headers for Sun plus CCDPROC setup for direct CCD
specphot Current headers for Sun plus CCDPROC setup for spectropho-
tometry, ie GoldCam, barefoot CCD
hydra WIYN Hydra with Arcon
foe Current headers for Sun plus CCDPROC setup for FOE
fibers Current headers for Sun plus CCDPROC setup for fiber array
coude Current headers for Sun plus CCDPROC setup for Coude
cyrocam Current headers for Sun plus CCDPROC setup for Cryo Cam
echelle Current headers for Sun plus CCDPROC setup for Echelle
kpnoheaders Current headers with no changes to CCDPROC parameters
fits Mountain FITS header prior to Aug. 87 (?)
camera Mountain CAMERA header for IRAF Version 2.6 and earlier
Instrument ID (type q to quit) (q): specphot
It will want you to edit ccdred and ccdproc tasks but for now you can quit both epar by ':q'
Now type:
ccdred> ccdlist *
You will get list of data in directory in this form:
0332.fits[1028,1500][real][object][][OTZF]:NGC6720
It starts with the name of fits file, first brackets contain dimensions of ccdimage, second brackets contains 'I have no idea what it means', third brackets contain image type, fourth and fifth brackets contains 'Again I have no idea what they mean' and in the end is value of IMAGETYP keyword. If you have specific value in the third brackets (anything except blank brackets or unknown sign) you have done a great job.
***NOTE***
In this stage of processing you need this value only for flat frames, bias frames, dark frames and scientific observations. Any other data can contain unknown in the third brackets.
In this stage of processing you need this value only for flat frames, bias frames, dark frames and scientific observations. Any other data can contain unknown in the third brackets.
I hope it is all. If I find anything I forgot to write about I will add it instantly. If there is ANYTHING unclear (any you-think-they-are-stupid questions are welcome - I know exactly what you are going through) leave a comment or send me e-mail.
Enjoy
Jakub
No comments:
Post a Comment