Main Page | Modules | Data Structures | File List | Data Fields | Globals | Related Pages

The SNifs Useful Reduction Package

developer

snurp_logo.png

Introduction

This document is all about the SNIFS (hopefully useful) Data Reduction Package (hereafter the SNURP), the reduction software dedicated to the SuperNovŠ Integral Field Spectrograph (SNIFS) used in the Nearby SupernovŠ Factory (SNFactory).

The SNIFS instrument is a lenslet-based integral field spectrograph developped at CRAL-Observatoire de Lyon, with an optical concept very similar to the one used in the previous 3D spectrographs TIGER (Bacon et al., 1995A&AS..113..347B), OASIS (e.g. Bacon et al., 2001A&A...371..409B) and SAURON (Bacon et al., 2001MNRAS.326...23B).

As a consequence, the SNURP implements reduction algorithms very similar to the SAURON ones (see e.g. Copin, PhD). Once the CCD frames have been properly preprocessed (e.g. with E. Gangler's project Ccd), the main steps are:

While the Ccd package is supposed to remove the detector signature from the CCD frames, the SNURP take cares of the data reduction up to the fully calibrated (including flux) datacube, i.e. removes all the SNIFS instrumental signatures. Note the computation of the precise flux calibration curve (including use of the photometric channel) is not supposed to be part of the SNURP; however, a crude extract_flux program (simple aperture photometry without ADR correction) is provided.

The cube is then ready to be used as input to other pieces of analysis softwares (e.g. supernova/galaxy separation and analysis), which are not part of the SNURP.

Useful links:
Contact:
y.copin@ipnl.in2p3.fr

News

Taggued versions:

Installation

Getting the code

Snifs

The SNIFS project code is maintained under CVS (https://cvs.in2p3.fr/snovae-IFU/Snifs/). You can check it out with the setcvs.pl script (http://supernova.in2p3.fr/doc/ccin2p3/help_user/cvs_1.html.en), but you need special privileges to commit it back.

For a typical front-end user (anonymous access) who wants to get taggued release SNIFS-0-1 in directory Snifs-oldies:

 eval `setcvs.pl -a IFU`
  cvs export -d Snifs-oldies -r Snifs-0-1 Snifs 
There is no easy way to get the list of taggued releases. However, you can try:
 cvs checkout -d Snifs-tmp Snifs/ChangeLog 
  cvs status -v Snifs-tmp/ChangeLog
  cvs release -d Snifs-tmp/ 

If you feel brave, you can use the latest version (HEAD):

 cvs checkout Snifs 
which can then be regularly updated with
 cvs update -d -P Snifs 

IFU libraries

Two libraries are compulsory to compile the SNIFS Reduction Package: the IFU_C_iolibs (providing an 3D-oriented I/O library, see The IFU/Euro3D-LCL libraries) and the IFU_C_mathlibs (including the GNU Scientific Library, http://sources.redhat.com/gsl/). Both are downloadable from the main CVS repository (https://cvs.in2p3.fr/snovae-IFU/). Please read the specific documentation for installation and setup procedures.

DISLIN

Few programs (see Graphical output) can produce a graphical output on their own, using the scientific data plotting library DISLIN (http://www.linmpi.mpg.de/dislin/). See Compilation to see how to compile your project with DISLIN support.

Compilation

Once you have your Snifs-xxx/ directory (hereafter called TOPDIR), you can proceed with the compilation. The project is built upon automake and autoconf, so that it should be pretty portable.

The first command to issue in TOPDIR is

 ./autogen.sh 
This script will generate all kind of technical files you don't want to know about. Then, you can proceed with the standard
 ./configure 

The configure script accepts some non-standard options:

  --with-dislin=DIR       prefix to DISLIN graphical library (use \$DISLIN)
  --enable-debug[=format] debug mode compilation (e.g. stabs+)
  --enable-profile        profiling mode compilation (gprof)
  --enable-verbose        mega-verbose mode compilation
  --with-optim=level      optimized compilation (level=1,2,3) 

--with-dislin
Few programs can produce a graphical output on their own, using the DISLIN. If you have this library installed (and therefore have the environment variable $DISLIN set to the proper path), you can compile the SNIFS project with the DISLIN support. See Graphical output for the list of programs with a graphical output, and the list of available devices.
--enable-debug
Set this option if you want to compile in debug mode (option -g). Note that the DWARF debug information generated by gcc 3.x (at least up to 3.3.2) is flaky. As a way-out, you can specify your own debugging information format (e.g. stabs+).
--enable-profile
Set this option if you want to profile your code using e.g. gprof.
--enable-verbose
Turn on this option is you want some programs to be extremely verbose in debug mode.
--with-optim
Set this option to allow optimization during the compilation.
Some environment variables have an influence on the configure script. See
 ./configure --help 
for more.

Execution

C-programs

The standard options of the programs are:
Parameters:
-h|help Print-out a complete list of possible options (specific and standard)
-version Print-out version (major.minor) of program and I/O library (prog.vers-lib.vers)
-debug Run the program in DEBUG mode, i.e. with a higher level of verbosity and/or intermediate steps. The standard and error outputs are also copied in a executable.dbg file (Note that it has nothing to do with the debug-mode compilation.)
-quiet Run the level with a minimum level of verbosity
-noask Don't ask confirmation for overwriting existing files
-inputformat 
-outputformat Specify input/output format, overring the default format set by the environment variable $IFU_DEFAULT_FMT if any.
-tk Arrange standard output to be used by the GUI.

Graphical output

Few programs (e.g. focus_spectro, plot_optics, center_gauss) can produce a graphical output on their own if the compilation (see Compilation) used the DISLIN.

These programs have a device option -dev, which default is ASCII (text output written to standard output). If compiled with DISLIN, the device can be:

External calibration files

Some calibration programs make use of external calibration files: If you want to use your own version, the format of these tables is described in module External calibration files. The Snifs project provides some standard calibrations under pkg/pipeline/data/:

You can produce your own reference flux correction spectrum from quick_flux on your favorite photometric standard star (wrapper to quick_cube + quick_calib -x + quick_extract -x). If photometric precision is not (yet) your goal, the default quick_calib use the standard flux_B.fits and flux_R.fits from pkg/pipeline/data/.

Quick scripts

The SNURP provides a handful of bash scripts (aka quick_scripts) to be used as the building blocks of a quick & dirty pipeline. Each script has its own documentation (e.g. quick_cube) under the Quick scripts module.

quick_graph.png

These quick scripts are the basic elements of the pythonic quick_pipeline.

Python scripts

Some python scripts are also present in the SNURP. They need a recent version of python (from 2.2) to import the OptionParser module. They also have special requirements:

Development

Documentation

The SNIFS code is natively documented using the inline documentation system Doxygen (http://www.stack.nl/~dimitri/doxygen/). Provided your doxygen program is recent enough (check out the ./configure output), a voluminous cross-referenced manual is produced in the Doc/ directory of the TOPDIR, both in LaTeX/PDF (latex/refman.pdf) and in plain HTML (html/index.html).

Furthermore, for the emacs addicts, a TAGS file is produced for the project, including info from the two IFU libraries, providing you have Exuberant ctags (http://ctags.sourceforge.net).

Some extra info (on-line documentation, metrics, sample files, etc.) is available at http://snfactory.in2p3.fr/soft/snifs/

Coding and debugging

If you plan to put your dirty hands in this clean code (could be the opposite?), you're advised to strictly follow the E3D Developer's Guide and the other helpful documents from http://www-obs.univ-lyon1.fr/Euro3D/LCL/

Here are some tools that could be useful to help you to debug the code:

Valgrind

Valgrind is really easy to use (it doesn't need any special includes, not even a specific compilation), but makes of course you program much slower (x10). RTFM, but the typical use of Valgrind is the following:
 valgrind --leak-check=yes [--show-reachable=yes] executable -usual-args 

You can also add some Valgrind macros to the code. E.g. to check if the particular variable xsup is defined at a given point, include memcheck.h and use the macro VALGRIND_CHECK_DEFINED(xsup); where Valgrind thinks there is a problem.

ElectricFence

There's no need to recompile your code against the efence library, thanks to the LD_PRELOAD environment variable. Use the ef script provided along with the efence library, or just set LD_PRELOAD=libefence.so.0.0 before lauching your executable.

Splint

Splint can statically check the code for ``security vulnerabilities and coding mistakes''. E.g. to test program extract_spec.c, one can use:
 splint -weak -nestcomment -fixedformalarray +relaxtypes \
  -I ../incl -I ../../incl \
  -I ../../../../IFU_C_iolibs-6.2/incl -I ../../../../IFU_C_mathlibs-6.2/incl \
  extract_spec.c
  

CVS notes

If you are a developper, and want to contribute to the Snifs code, please help yourself! You need however special privileges to commit code back to the CVS repository.

The IFU/Euro3D-LCL libraries

The Euro3D-LCL I/O library (PÚcontal-Rousset, Copin, Ferruit, 2004AN....325..163P) is the official Euro3D I/O library, implementing the Euro3D data format (Kissler-Patig et al., 2004AN....325..159K) and offering a variety of tools for efficient and portable programming (option parsing, session handling, standard and error I/O, etc.). It is based on the so-called IFU library, developped over the years at the CRAL-Observatoire de Lyon (hence the LCL name, for `Lyon C Library') by the TIGER team, and mainly by Arlette Rousset-Pecontal, and heavily used and tested in various integral field spectrography projects (TIGRE, OASIS, SAURON).

The latest stable release of the library can be found on the `Official' Euro3D-LCL I/O library webpage, while the latest (unsupported, potentially unstable) release can be found on the Lyon Euro3D-LCL website. If you are new in the Euro3D/IFU programmation, this page also provides:

Links:

Generated on Wed Oct 26 23:59:38 2005 for Snifs by doxygen 1.3.5