- 1 - Numeric MLab module not found
- 2 - Plotutils compilation failed
- 3 - RADIAL VELOCITY, giCrossC.py RV erroneous helio/barycentric correction
- 4 - RADIAL VELOCITY giCrossC.py binary mask
- 5 - ThAr wavelength; giCrossC.py and giRecNWcal2.py
- 6 - SKY SUBTRACTION till rel 1.12
- 7 - giArithm.py, Signs '+' '-' '*' in KW 'ESO OBS TARG NAME'
- 8 - Use of giCrossC.py with data produced by Paranal pipeline
- 9 - Where to start if I know nothing about the data structure and reduction
- 10 - Gawk parse error, problem with scripts (Wed Mar 23 06:15:21 CLT 2005)
- 11 - Best accuracy in RV
- 12 - FLAT FIELDING in ARGUS
- 13 - PIPE, Word too long, csh
- 14 - Transfering Localisation and Simultaneous calibration from other science frame
- 15 - ARGUS, orientation on the sky, image reconstruction
Numeric MLab module not found
depending on existing system wide python Numeric installation,
it happens that the MLab module is not found.
To get rid of this problem you have to install the girBLDRS own
python Numeric module, that is the one, and precisely THIS ONE
available at
http://obswww.unige.ch/Instruments/GIRAFFE/Files/girBLDRS/required/Numeric-23.3.tar.gz
Download the preceding URL to the REQUIRED directory in the girBLDRS
source tree, where you find the install_Numeric.sh script, and run
this script, then re-run the buildall.sh installation script
[004 00NumericMLab.txt
]
Plotutils compilation failed
To compile plotutils, your system need to have the X11 and Xaw
packages installed, and also their development part: X11-dev
and Xaw-dev.
The real name of these packages 'rpm' or 'deb' is different
depending on your installed Linux distribution, RedHat, Sebian,
Suse, etc...
[004 00PlotutilsCompile.txt
]
RADIAL VELOCITY, giCrossC.py RV erroneous helio/barycentric correction
QUESTION: deangeli@pd.astro.it
What sign should be Paranal geographical longitude. It seems that +70 rather than -70
given by image header is the correct value. ?
ANSWER (AB):
It appears that the (general) convention used for Geographical Longitude and assumed in
computation of the Geocentric Radial Velocity correction in giCrossC.py assuming +sign for
West longitude is in DESAGREEMENT with ESO KW 'ESO TEL GEOLON' which gives for Paranal
-70.403
Impact on the GIRAFFE BLDRS:
Unless you modifed KW 'ESO TEL GEOLON', all releases till 1.13 included are affected Worst
case error reaches +-0.75 km/sec for object at the equator and vary with date and time.
The error is mostly systematic for observations taken with similar hour angles while it
produces random scatter for observations taken at distant dates
Correction in release 1.13
Edit installed $GIR_ROOT/python/share/bin/giCrossC.py
go to line 1373 which should be:
"if verbose: print 'MJD,EXPOSURE,LON,LAT,ELE ..."
add (beware indentation):
if LON is not None:
if abs(70.403+LON)<0.001:
if verbose>=0: print '# Setting longitude +sign for Paranal'
LON=abs(LON)
#endif
check the giCrossC.py syntax:
giCrossC.py -h
[2004 50RadialVelocity.txt
]
RADIAL VELOCITY giCrossC.py binary mask
villanova@pd.astro.it Wed, 27 Oct 2004 12:10:07:
Is it possible to create a binary mask for radial velocities measurements to be used in
pipe-line giCrossC.py task?
Answer (AB):
That is something marginally supported; you may proceed as follows:
cd $GIR_CALIB
Get ASCII format of any H9A table:
fitstotable girG2H9A.tfits | justify > girG2H9Atest.r
Edit girG2H9Atest.r and remove lines with unsupported keys:
# SIMPLE =T / file does conform to FITS standard
# OGL PRO GIRTABLE TYPE='LINEMASK' / GIRAFFE BLDRS table type
# ESO INS GRAT NAME='HR316' / Grating common name (hls).
# ESO INS GRAT WLEN=525.8 / Grating central wavelength [nm] (hs).
# OGL PRO REBIN LAMBDA STEP=0.0 / wlen step used for rebinning
Use your tools (good editor - better to make tabs visible) to adjust WLEN1 WLEN2 columns;
other columns has no effect on giCrossC.py but should be there. Be careful to keep
separators
convert to tfits using:
tabletofits -b girG2H9test.r > girG2H9test.tfits
use giCrossC.py with -T girG2H9test.tfits option
[2004 51RVcreatePrivatMask.txt
]
ThAr wavelength; giCrossC.py and giRecNWcal2.py
QUESTION: deangeli@pd.astro.it
What is the source of reference wavelength used to build the binary masks for
cross-correlation ?
ANSWER (PN):
Palmer B.A. & Engleman R. Jr., 1983, Atlas of the Thorium spectrum, Los Alamos National
Laboratory.
[004 52ThArWavelengthSource.txt
]
SKY SUBTRACTION till rel 1.12
randich@arcetri.astro.it 22 Nov 2004 10:39:36
when trying to subtract the sky (both with the pipe command skyss
with the python command giSky.py) I obtain the message of error:
Not enough valid bins in sky spectra: [ 3 15 17 19 25 ...]
[0 0 0 0 0 0 0 0 .....
Answer (AB):
The reason is the presence of one ore more inconsistent sky spectra (very different from
others). It will be properly solved in 1.13. In mean time several ways to get around.
1. Give list of good sky spectra using '-S' option; proceed as follows:
- run giSky in verbose mode; '-v' option
- and look for line: 'Rejected sky spectra: [bad1,bad2,..]'
this line appears after 'Model-rawdata for Integrated Sky; nspec ...'
- run giSky with option -S sp1,sp2,sp3,.... dropping spectra numbers bad1,bad2,...
2. Increase the sigma-clipping limit to prevent the rejection of a complete spectrum as
follows:
- run 'giSky.py ---skInt -,-,-5 ...' -5 stands for nsigma = 5 (default is 3)
While method 2. is simple, for obvious reasons the 1. is better
[2004 53SkySubtractionNot_enough_valid_bins.txt
]
giArithm.py, Signs '+' '-' '*' in KW 'ESO OBS TARG NAME'
QUESTION: emsellem@obs.univ-lyon1.fr
giArithm.py produces error '**ERROR** Input file of the expression do not exists'
while the referenced file, NGC-3623ArgusL4Seq01bSub1??.fits, DOES exits. The problems
seems to be related to the symbol '-' present in the filename which has been generated
by the previous step within the pipe.
ANSWER (AB):
The pipe uses the KW 'ESO OBS TARG NAME' to generate default for the first part of the
intermediate filenames governed by the pipe parameter 'OBJ'. If one of characters '+'
'-' '*' is present within the KW it is interpreted by giArithm.py as arithmetic
operation leading to pipe failure since the file is not found.
You should therefore specify pipe parameter 'OBJ'. In your case you may add to the pipe
command 'OBJ=ngc3623' so as to have for exemple:
pipe extract RAW=../raw/GIRAF.2003-01-28T05:49:49.211.fits OBJ=ngc3623
[004 54TargetNameWithForbidenCharacters.txt
]
Use of giCrossC.py with data produced by Paranal pipeline
QUESTION: ebernard@iac.es, lucatello@pd.astro.it and others
I have been trying to use girBLDRS, in particular the function giCrossC.py, on
pipeline-reduced data in order to calculate radial velocities but kept obtaining error
messages starting by:
***WARNING: absent or empty ozPoz table, we asume all spectra listed in Fibre table ...
ANSWER (AB):
The Paranal pipeline does not produce the data in the format compatible with the BLDRS
python release till 1.12. The main difference concerns the changes in KW names and the
absence of the binary tables in reduced data. Though the BLDRS release 1.13 in
preparation will support at least at some basic level the output from Paranal pipeline,
unfortunatelly we must accept that Python and Paranal version diverge since even the low
level c-code evolved independently during the past 18 months. We strongly recommend to
users desiring to use correlation, sky subtraction or image reconstruction to fully
reprocess the data using the Python version.
[2005 55useBldrsWithDataFromParanalPipeline.txt
]
Where to start if I know nothing about the data structure and reduction
QUESTION: miguel.santander@iac.es Thu, 10 Mar 2005 17:05:15
I recently installed the latest girBLDR software in order to reduce some data taken
with Giraffe in ARGUS mode. I have taken a look to the Reference Manual as well. The
problem is that it is my first time reducing this kind of data and I'm quite a bit
lost.
ANSWER (AB):
I would recommend followings:
Run the complete test 'time ptest' after the girbldrs_girbldrs-pipe pipe installations
(see file INSTALL_EXAMPLE in the distribution which gives full example of the
installation including test data download).
Declare test calibration as current calibration directory
setenv GIR_CALIB ../calib
Get list of 'ptest' by typing: 'ptest -2'
Execute test 1-by-1 and try to understand what is it doing. Use your usual tools to
look on produced fits files (rtd, ...) then biggExt.py graphic tool specific to
girbldrs.
Would be useful to read in parallel sub-sections 4.1 (pipe INTRODUCTION) - 4.4 (DATA
REDUCTION COOK_BOOK) [4.5] of the Reference Manual
Install the calibrations you need for your data from:
http://obswww.unige.ch/Instruments/GIRAFFE/Files/girBLDRS/calib2.3current/
In case of problem have a look on FAQ at: http://girbldrs.sourceforge.net/FAQ/index.cgi
[2005 56whereToStart.txt
]
Gawk parse error, problem with scripts (Wed Mar 23 06:15:21 CLT 2005)
QUESTION: zaggia@ts.astro.it. I have the following problem with fhead
I cannot get any suitable answer anymore.
dfits ../raw/*fits | fhead
gawk: compute.gawk.23212:7: $2=sprintf("%5i",$2);
DATE=substr(DATE,1,length(DATE)-7); if (length(FILT)>0) {s1=substr(FILT,1,1);
s2=substr(FILT,3); FILT=sprintf("%s%2.2i",s1,s2)}$3substr$31length$3
7iflength$90s1substr$911s2substr$93$9sprintf"%s%2.2i"s1s2;print $0
gawk: compute.gawk.23212:7:
^ parse errorI cannot get any suitable answer anymore.
ANSWER (AB):
Must be a conflict between your local SW and girbldrs-pipe. Check followings:
dfits ../raw/*fits
# must be a list of KWs something like
====> ../raw/GIRAF.2004-04-14T07:53:47.450.fits (main) <====
SIMPLE =T / Standard FITS format (NOST-100.0)
BITPIX =16 / # of bits storing pix values
..
# check /rdb scripts used within commands by piping most of scripts toogether
echo "1" | number nu 5 | addcol nn | compute 'nn=10*nu' | column nu | rename nu NU \
| sorttable -nr NU | addccol addedstring str | tabletolist | listtotable | rmcol str \
| row 'NU<3' |headoff
# the output should be '1'
In case of problem run the /rdb test partially till you localise the offending
command and consult your local guru. I would suspect that your path and/or gawk is
the source of problem.
[2005 57gawk_parse_error_problem_with_scripts.txt
]
Best accuracy in RV
QUESTION: Claudio Melo Tue Apr 12 13:44:47 MEST 2005
ANSWER (AB):
INTRODUCTION/SUMMARY
Here is the description what I will do (with some comments). The data reduction
strategy and final accuracy will depend on many factor such as time coverage, setup,
fibre assignment and scientific goal.I will just make few hints on what should be
looked at. An example of reduction in Medusa1H12 setup is given. It is assumed that
master calibrations exists in $GIR_CALIB.
A. SIMPLE REDUCTION
The standard calibration if available in:
http://obswww.unige.ch/Instruments/GIRAFFE/Files/girBLDRS/calib2.3current/ )
provides the elements of the standard solution:
$GIR_CALIB/girSlitGeoMedusa1H12.tfits # slit geometry
ThArMedusa1H12dbSub.extsp.xores.tfits # optical+Cheby W-solution
For science exposure with simultaneous wavelength calibration, this solution is adjusted
using the giCrossC.py during the standard extraction and therefore YOU NEED THE
CALIBRATION only as accuracy check. The accuracy with your actual data could be checked
using the standard extraction:
pipe extract OBJ=ThAr EMETHOD=SUMM NAM=Seq01 RAW=../raw/WcalFile#1
pipe extract OBJ=ThAr EMETHOD=SUMM NAM=Seq02 RAW=../raw/WcalFile#2
giCrossC.py -N 0:-0 --txts -g 5 -T \
ThArMedusa1H12Seq01dbSub.extsp.rebinlin.fits \
ThArMedusa1H12Seq02dbSub.extsp.rebinlin.fits \
The giCrossC.py output ends by:
# Average fluxes - CALSIM: 17465, CAL: 2246
# Medusa1H12: average RV -37 +- 111[m/sec] using 131 RV determination out of 132 valid
RV peaks, \
preliminary list of invalid spectra: 24C
...
In the above case (2 calibration 1 year apart) the noise added by calibration (i.e
fibre-to-fibre) is 111/(2**0.5)=78[m/sec] while exposure-to-exposure accuracy obtained
by comparison of several files is ~100m/sec
For science exposure use standard extraction:
pipe extract RAW=../raw/file_name
The simultaneous calibration is detected and processed automatically
B. REDUCTION USING YOUR W-CALIBRATIONS
#1. Make new empty directory for reduced data. If you have several images in the same
setup you would better make 1 directory/image since intermediate calibration data will be
located there and the naming scheme of intermediate data does not support multiple
instances of the same setup.
#2. Run locMast with 'MAST=no' option which will not modify data in $GIR_CALIB
pipe locMast RAW=../raw/GIRAF.2004-05-15T15:09:52.586.fits MAST=no
#3. Run wcalMast with 'MAST=no' option
pipe wcalMast RAW=../raw/GIRAF.2004-05-15T15:13:54.653.fits MAST=no
It produces the W-solution using the slit geometry from the $GIR_CALIB. At this stage
you can check the accuracy:
dfits -x0 ThArMedusa1H12dbSub.extsp.rebinlin.fits | egrep QC
OGL PRO QC LOC COURBMAX = 7.86811007897 / max ydiff within any spect
OGL PRO QC PSF COURBMAX = 8.08508513408 / max ydiff of 2 adjacent pi
OGL PRO QC WSRMS OPT = '1.04' / Wcal: RMS[pixel] optical model
OGL PRO QC WSRMS CHEB = '0.16' / Wcal: RMS[pixel] after chebycheff
OGL PRO QC WSLINE PERCENT = 99 / Wcal: Percentage of accepted lines
OGL PRO QC SLIT RV AVERAGE = 3 / CC: Average RV[m/s] of residuals
OGL PRO QC SLIT RV RMS = 63 / CC: RMS[m/s] of residuals
OGL PRO QC SLIT RV NSP = 132 / CC: nombre of fibres used
OGL PRO QC SLIT RV SPOUT = '42L,27M' / CC: list of rejected fibres
Mainly OGL PRO QC WSRMS CHEB gives the accuracy of single line in pixels. In the
above case it would mean approximately that we have
DRV=BAND*0.16*300000/(4000.*WLEN)=0.5[km/sec/line] accuracy. You can expect on
final data using this calibration the random error due to the calibration of
0.5[km/sec]/(n_lines**0.5).
You can either run wcalMast once per run using one or several calibration files or
take for each observation the closest calibration. The later becomes compulsory if
you do not have simultaneous W-calibration, but I am not sure that for a run of few
days WITH simultaneous W-calibration it will bring any improvement.
#4. Adjust slit Geometry (only once per setup)
pipe wcalSlit RAW=../raw/GIRAF.2004-05-15T15:13:54.653.fits
This produces the new slit geometry and new rebinned ThAr file. The graphic shows
the residual RV of spectra rebinned using the standard girSlitGeoMedusa1H12.tfits
taken from $GIR_CALIB. If you repeat the pipe wcalSlit you will see the residuals
decrease to almost arbitrary low level which does not reflect the actual accuracy.
In most cases 2 iterations are enough to reach few m/sec.
This step (#4) produces new $GIR_CALIB/girSlitGeoMedusa1H12.tfits which will be
used for all subsequent W-calibrations of the same setup within the observing run.
This is also the only operation which modify the archive in $GIR_CALIB. In case of
doubt you can check with results of other calibrations (you will have to rename
'girSlitGeoMedusa1H12.tfits' by hand), but very likely the use of the same
girSlitGeoMedusa1H12.tfits for all data will lead to the best accuracy.
You completed the global W-calibration. The result used for subsequent rebinning of
science data is in 2 files:
$GIR_CALIB/girSlitGeoMedusa1H12.tfits # slit geometry
ThArMedusa1H12dbSub.extsp.xores.tfits # optical+Cheby W-solution
You can check the accuracy of rebinned data by the same method as above:
pipe extract OBJ=ThAr EMETHOD=SUMM NAM=Seq01 RAW=../raw/WcalFile#1
pipe extract OBJ=ThAr EMETHOD=SUMM NAM=Seq02 RAW=../raw/WcalFile#2
giCrossC.py -N 0:-0 --txts -g 5 -T \
ThArMedusa1H12Seq01dbSub.extsp.rebinlin.fits \
ThArMedusa1H12Seq02dbSub.extsp.rebinlin.fits \
Typical systematic error (exposure-to-exposure) RMS is 60 m/sec and
fibre-to-fibre RMS 50m/sec.
#5. Extract and rebin science data
pipe extract RAW=../raw/GIRAF.2004-05-15T08:17:51.045.fits
C. RV DETERMINATION ON SCIENCE FRAMES
If you want a set of RVs to compare objet-to-object RV the use of binary mask is
best solution; for example:
giCrossC.py -N 1:30:1 -O ngc --txts -V -200,200,1 -T girK0 -R 1 \
NGC6253_center_fieldMedusa1H11dbSub.oxtsp.rebinlin.fits
If you want optimize the accuracy of RV curves for individual objects you may use the CC
against the selected spectrum of each object (within the same setup):
giCrossC.py -N 1:-0 -O ngc --txts -T Selected_exposureframe. \
NGC6253_center_fieldMedusa1H11dbSub.oxtsp.rebinlin.fits
Do not omit '.' after Selected_exposureframe which indicates that same spectra should
be taken. If you do not have same fibre assignment for all exposure you must give the
list of correspondence after the dot (see giCrossC.py -h). You can check the quality
of this correlation by correlating frame against itself as:
giCrossC.py -N 1:-0 -O ngc --txts -T . --Vcorr n \
NGC6253_center_fieldMedusa1H11dbSub.oxtsp.rebinlin.fits
The column rv should have |average value| and RMS ~ few 10m/sec on good spectrum.
COMPARING RESULTS IN DIFFERENT SETUPS
The RV obtained are values relative to the difference between average position of
ThAr lines of the setup and average of science object spectra lines within mask.
Moreover there is an additional unknown shift between 5 simultaneous spectra and
science spectra due to the (usually very) different illumination of simultaneous
calibration fibres and other fibres on the W-calibration ThAr exposures. These are
main reason why EACH SETUP SHOULD BE CONSIDERED AS A SEPARATE SET OF MEASUREMENT
which could be several 100 m/sec apart. If you want to merge 2 or several setup RVs
you must use an extra piece of information to adjust the difference (standard star,
time overlap ...). This is also the best way to evaluate the final accuracy you got.
Created Thu Apr 14 09:27:42 MEST 2005
[2005 58HowtogetbestRV.txt
]
FLAT FIELDING in ARGUS
QUESTION: Which FF is better, Nasmyth screen or Calibration unit (Wed Nov 16 10:39:06 CLST 2005)
ANSWER (AB):
INTRODUCTION/SUMMARY
It appears that the calibration unit does not provide the uniform illumination of the
ARGUS field. Data taken in October 2005 illustrates the situation.
http://obswww.unige.ch/Instruments/GIRAFFE/Files/girBLDRS/FAQimag/ArgusL4FlatNasmythAndCalib.gif
On the right, reconstructed image of FF taken with Calibration Unit shows the
gradient in illumination on horizontal axis, while the left image reconstructed from
Nasmyth screen illuminated exposure has only the fibre-fibre random variation. Note
that uppermost row and rightmost column of Nasmyth image seems to suffer also some systematic effect.
RECOMMENDATION
Use Nasmyth screen data or remove smooth gradient from Calibration Unit data (for
example using the average row to divide all rows).
Ask ESO to improve the Calibration Unit
Created Wed Nov 16 11:14:52 CLST 2005
[2005 59ArgusFFVignettingProblem.txt
]
PIPE, Word too long, csh
QUESTION: hector.flores@obspm.Fr
In my new machine I've installed Knoppix (debian like)
Linux 2.6.12 #2 SMP Tue Aug 9 23:20:52 CEST 2005 i686 GNU/Linux
when running the pipe for installation test
pipe -t locMast
I got:
....
Word too long.
[1]: No match.
After looking for "Word too long" error on the network, I found a possible source of
the problem on the csh and/or tcsh shell command,
ANSWER (AB):
You are right, this a problem of c-shell
Standard recent installations reserve 10240 characters for the word which is 10-times
more then necessary for GIRAFFE SW. You can check how much your installation supports
using:
echo `echo "qq" | gawk '{imax=1024; for (i=1;i<=imax;i++) {o=o "Q"}; print o}'`
Modify "imax=.." till you have/have_not the message "Word too long". If you are below
1024 you should install a better version of c-shell.
Fri Feb 10 12:54:40 CLST 2006
[2006 60Wordtoolong.txt
]
Transfering Localisation and Simultaneous calibration from other science frame
QUESTION: Pierre.North@obs.unige.ch (Mon, 10 Apr 2006)
I have, within the night, only one science exposure with simultaneous calibration. I
would like to use the localisation and wavelength solution adjusted on the exposure
with simultaneous calibration for other exposures within the night. Any possibility to
do it ?
ANSWER (AB):
Yes, you should proceed as follows:
1. Extract the exposure with simultaneous calibration:
pipe extract RAW=your_science_frame_with_simcal_exposure_Name
EX: pipe extract RAW=../raw/GIRAF.2005-04-30T05:16:30.215.fits
in data product you will find (ls *.clocy.fits *.wlenRes.tfits):
a. adjusted localisation named something like: ObjectSetup...dbSub.clocy.fits
EX: HE1503_fieldArgusL7dbSub.clocy.fits
b. fibre-to-fibre correction table : ObjectSetup...dbSub.wlenRes.tfits
EX: HE1503_fieldArgusL7dbSub.wlenRes.tfits
2. Make the a) master localisation
mv ObjectSetup...dbSub.clocy.fits SetupMast.clocy.fits
EX: mv HE1503_fieldArgusL7dbSub.clocy.fits ArgusL7Mast.clocy.fits
3. Run "extract" pipe on all science data in the setup WITHOUT simultaneous calibration:
pipe extract RAW=your_science_frameName
EX: pipe extract RAW=../raw/GIRAF.2005-04-30T05:17:39.741.fits
4. One more rebin/science_file manualy using the correction 1.b:
type: "extract -t your_science_frameName"
pick the last line and add after "giRecBinn2.py " the option:
--wres ObjectSetup...dbSub.wlenRes.tfits
EX: --wres HE1503_fieldArgusL7dbSub.wlenRes.tfits
the command in the example will be:
giRecBinn2.py --wres HE1503_fieldArgusL7dbSub.wlenRes.tfits -w --bnstep - --bnrange=setup \
--sgtable $GIR_CALIB/girSlitGeoArgusL7.tfits --bnlin --bnmethod linear \
HE1503_fieldArgusL7dbSub.oxtsp.fits - HE1503_fieldArgusL7dbSub.clocy.fits -
Tue Apr 11 03:19:05 CLT 2006
[2006 61LocalisationTransfer.txt
]
ARGUS, orientation on the sky, image reconstruction
QUESTION:
How is the ARGUS reconstructed image oriented on the sky ?
ANSWER (AB):
The ARGUS position angle from North to East is given by the KW ARGPOSAN in the header
of the OzPoz table
to see the angle use: dfits -x0 fits_file | egrep ARGPOSAN
The reconstructed image has the major axis along the ARGPOSAN angle and is right-left
inverted along the minor axis. To get the 1-st pixel in the S-W corner you should flip
the image around the minor axis and (clockwise) rotate of ARGPOSAN angle.
EXAMPLE: for ARGPOS=330 I would rotate image 30 degrees counter-clockwise then flip
around the major axis.
Wed Aug 2 06:21:47 CLT 2006
[006 62Argusorientation.txt
]