Updated: 14 March 2016

Precious Metals in SDSS Quasar Spectra II:
Tracking the Evolution of
Strong, 0.4 < z < 2.3 Mg II Absorbers with Thousands of Systems

Supplementary Data

Eduardo N. Seyffert, Kathy L. Cooksey, Robert A. Simcoe, John M. O'Meara, Melodie M. Kao, and J. Xavier Prochaska

Go directly to: Paper and Abstract | Results Summary | Data Products | Codes and Calls

Paper and Abstract

Reference: Seyffert et al. (2013, ApJ, 779, 161: online, PDF)

Abstract: We have performed an analysis of over 34,000 Mg II doublets at 0.36 < z < 2.29 in Sloan Digital Sky Survey (SDSS) Data-Release 7 quasar spectra. The catalog was divided into 14 small redshift bins with roughly 2,500 doublets in each, and from Monte-Carlo simulations, we estimate 50% completeness at rest equivalent width Wr ≈ 0.8 Å. The equivalent-width frequency distribution is described well by an exponential model for all redshifts, and the distribution becomes flatter with increasing redshift, i.e., there are more strong systems relative to weak ones. Direct comparison with previous SDSS Mg II surveys reveal that we recover at least 70% of the doublets in these other catalogs, in addition to detecting thousands of new systems. We discuss how these surveys come by their different results, which qualitatively agree but, due to the extremely small uncertainties, differ by a statistically significant amount. The estimated physical cross-section of Mg II-absorbing galaxy halos increased three-fold, approximately, from z = 0.4 → 2.3, while the Wr ≥ 1 Å absorber line density grew, dNMg II/dX, by roughly 45%. Finally, we explore the different evolution of various absorber populations—damped Lyman-$\alpha$ absorbers, Lyman-limit systems, strong C IV absorbers, and strong and weaker Mg II systems—across cosmic time (0 < z < 6).

Mg II Results Summary

(1)(2)(3)(4)(5) (6)(7)(8)(9)(10) (11)
zzlim NobsΔXmax W50% dNMg II/dz dNMg II/dX Nfitk αχ2red
          −1) −1) 
1.10749[0.36539,2.28259] 34254 183071 0.81 0.293+0.002/−0.002 0.123+0.001/−0.001 224140.71+0.02/− 0.02−1.40+0.01/− 0.01 0.535
0.45524[0.36539,0.51988] 2414 17037 1.04 0.162+0.004/−0.004 0.096+0.002/−0.002 15090.96+0.11/− 0.10−1.73+0.06/− 0.06 1.519
0.57586[0.52003,0.62992] 2402 13739 0.82 0.191+0.005/−0.005 0.102+0.003/−0.003 14850.87+0.10/− 0.09−1.63+0.06/− 0.06 2.422
0.68065[0.63003,0.72993] 2491 13048 0.74 0.220+0.006/−0.005 0.110+0.003/−0.003 15100.75+0.08/− 0.08−1.51+0.06/− 0.06 1.327
0.77562[0.73005,0.81998] 2445 12057 0.72 0.251+0.006/−0.006 0.118+0.003/−0.003 15080.83+0.09/− 0.08−1.54+0.06/− 0.06 1.617
0.86369[0.82002,0.90992] 2575 12320 0.71 0.274+0.007/−0.007 0.123+0.003/−0.003 16270.83+0.09/− 0.08−1.50+0.05/− 0.05 1.240
0.94890[0.91002,0.99998] 2288 11944 0.80 0.295+0.007/−0.007 0.126+0.003/−0.003 14800.86+0.10/− 0.09−1.51+0.06/− 0.06 1.658
1.04931[1.00003,1.10999] 2557 14924 0.93 0.292+0.006/−0.006 0.119+0.003/−0.003 17490.75+0.08/− 0.07−1.44+0.05/− 0.05 1.535
1.15652[1.11001,1.20000] 2356 11850 0.76 0.332+0.008/−0.008 0.130+0.003/−0.003 15900.67+0.07/− 0.06−1.35+0.05/− 0.05 1.836
1.25157[1.20005,1.29999] 2521 11904 0.70 0.330+0.008/−0.008 0.125+0.003/−0.003 15780.64+0.06/− 0.06−1.32+0.05/− 0.05 1.060
1.34541[1.30009,1.38999] 2515 10176 0.53 0.369+0.010/−0.009 0.135+0.004/−0.003 15500.72+0.07/− 0.07−1.36+0.05/− 0.05 1.237
1.43499[1.39003,1.48998] 2431 10394 0.58 0.374+0.010/−0.010 0.133+0.003/−0.003 15430.65+0.06/− 0.06−1.31+0.05/− 0.05 2.267
1.54058[1.49024,1.59994] 2321 10060 0.63 0.400+0.010/−0.010 0.138+0.004/−0.003 15270.64+0.06/− 0.06−1.27+0.05/− 0.05 1.652
1.67796[1.60001,1.77974] 2467 13671 0.82 0.398+0.009/−0.009 0.132+0.003/−0.003 17710.52+0.05/− 0.04−1.19+0.04/− 0.04 1.064
1.91822[1.78010,2.28259] 2471 23091 1.22 0.360+0.007/−0.007 0.111+0.002/−0.002 19870.42+0.04/− 0.04−1.16+0.04/− 0.04 2.489

Note −  Summary of the most common redshift bins and data used for the various analyses. Columns 1−2 give the median, minimum, and maximum redshifts for the observed number of doublets (Column 3), and the maximum co-moving pathlength in the redshift bin is given in Column 4. The 50% completeness from the Monte Carlo tests is in Column 5. The redshift and co-moving absorber line densities for Wr ≥ 1.0 Å are in Columns 6−7. The frequency distribution was fit with an exponential ƒ(Wr) = k exp(αWr) for Nfit absorbers with Wr ≥ 0.8 Å (Column 8), and the best-fit parameters are given in Columns 9−10. The reduced χ2 from the best fit and ƒ(Wr) (in bins with ≈100 doublets each) is given in Column 11.

Data Products

Bundle Description
dr7qso_MgII_noBAL.fit.gz QSO structure, a subset of the Schneider et al. (2010) catalog.
dr7qso_MgII_noBAL.list Formatted list of spectra, in order of the QSO structure, commonly used in the various codes.
dr7qso_MgII_noBAL_SNR.fit.gz Signal-to-noise structure, created by SDSS_MKSNRTAB, in order of the QSO structure.
spectro_0266-0531.tar spectro_0532-0783.tar spectro_0784-1054.tar spectro_1055-1365.tar spectro_1366-1688.tar spectro_1689-1955.tar spectro_1956-2233.tar spectro_2234-2579.tar spectro_2580-2973.tar SDSS spectra of the sightlines surveyed for Mg II.
hybconti_0266-0531.tar hybconti_0532-0783.tar hybconti_0784-1054.tar hybconti_1055-1365.tar hybconti_1366-1688.tar hybconti_1689-1955.tar hybconti_1956-2233.tar hybconti_2234-2579.tar hybconti_2580-2973.tar Hybrid-continuum fits generated with SDSS_DBLFITCONTI.PRO (see below).
abslin_0266-0531.tar abslin_0532-0783.tar abslin_0784-1054.tar abslin_1055-1365.tar abslin_1366-1688.tar abslin_1689-1955.tar abslin_1956-2233.tar abslin_2234-2579.tar abslin_2580-2973.tar Continuum and automatically detected line structure (SDSSCONTISTRCT__DEFINE.PRO), generated with SDSS_FNDLIN.PRO (see below).
vpstrmgii72422le500perEW.fit.gz Simulated Mg II Voigt profiles used for the Monte Carlo completeness tests.
mcmgii_0266-0531.tar mcmgii_0532-0783.tar mcmgii_0784-1054.tar mcmgii_1055-1365.tar mcmgii_1366-1688.tar mcmgii_1689-1955.tar mcmgii_1956-2233.tar mcmgii_2234-2579.tar mcmgii_2580-2973.tar Basic Monte Carlo completeness tests for a sub-sample of the Mg II sightlines, generated with SDSS_COMPLETENESS.PRO (see below).
sdss_mgii_seyffertetal13.fit.gz The full Mg II systems structure from the paper.
cmpltrec_MgII_0.3z2.3cum.fit.gz cmpltrec_MgII_0.36z2.29n2500cum.fit.gz Combined completeness curves for redshift bins used in paper, generated with SDSS_COMPLETENESS_CZW or SDSS_COMPLETENESS_CUM (see below).
cmpltrec_userbias_noBAL_0.63zdiv1.78.fit.gz Estimate of the true negative and accepted false positive rates from SDSS_COMPLETENESS_USERBIAS; used by SDSS_APPLYUSERBIAS() and SDSS_GETAFPDNDX.PRO.
fwfit_MgII.tar Equivalent width frequency distribution fit structures presented in the paper, generated with SDSS_FITFXW.PRO (see below).

Codes and Calls

Go directly to: Directory Structure | Setting-up Codes | Working with Mg II Structure | Continuum Fitting, Identifying Candidates, Generating Completeness Curves

Directory Structure

You will need to set the environment variables SDSSPATH and SDSSDR; for example, in bash:


The codes will expect the following directories under $SDSSPATH/$SDSSDR/:

Setting-up Codes

The main data processing and analysis codes are IDL-based and in the XIDL package. The majority of the custom codes are in $XIDL_DIR/SDSS/CIV. Several general programs and functions are in SDSS_FUNCTIONS.PRO, and this needs to be compiled before any XIDL SDSS codes are run. This can be done automatically at startup by adding the following lines to your .idlstartup file:

!quiet = 1 ; suppress messages
.r sdss_functions
!quiet = 0

The list and description of programs and functions in SDSS_FUNCTIONS.PRO can be seen by calling sdss_functions at the IDL prompt:


Below, basic calls that other researchers may find useful are described. Other program and function options can often be seen by making the program or function calls without parameters or with the /help option. Several calls can return values in redshift (as opposed to co-moving pathlength) space when /dz is included as an option.

Working with Mg II Structure

The absorber structure is described in SDSSCIVSTRCT__DEFINE.PRO. Here are basic ways of manipulating the structure. First, SDSS_GETCIVSTRCT() (in SDSS_FUNCTIONS.PRO) can be used to grab various sub-samples. For example, to extract the doublets with redshifts 1.5 ≤ z < 2 and rest equivalent widths Wr ≥ 1.0 Å that are more than 5000 km s−1 from the background QSO (i.e., δvQSO < −5000 km s−1) and not in visual BAL QSO sightlines, the call would be:

IDL>mgiistrct = sdss_getcivstrct('sdss_mgii_seyffertetal13.fit', zlim=[1.5,2], ewlim=[1.0,!values.f_infinity], /noBAL, dvqso=-5000)

More options for SDSS_GETCIVSTRCT() can be found by using the /help option. The redshift, equivalent widths, and column densities (from the apparent optical depth method) are stored in the arrays tagged as ZABS_ORIG, EW_ORIG, and NCOLM_ORIG, with errors in the SIGZABS_ORIG, SIGEW_ORIG, and SIGNCOLM_ORIG arrays. The zeroth array element is for the Mg II 2796 line and the first is for the 2803 line. The 2796 and 2803 column density measurements can be combined into a system column density estimate as follows:

IDL>ncolm = sdss_calncolmdblt(mgiistrct, 'MgII', signcolm=signcolm, /log, ncolmflg=ncolmflg)

The column density (limit) flags can be estimated by forcing all doublets with ratio Wr,2796/Wr,2803 < (2 − combined error) to be lower limits, assuming they are saturated.

A quick way to view a lot of information about the systems is to use the rating GUI SDSS_CHKCIV.PRO, which needs the directory structure defined above. The program can accept a structure or file name as an input and needs an output variable or file:

IDL>sdss_chkciv, mgiistrct, outstrct, dblt_name='MgII'

If you have a completeness curve for 1.5 ≤ z < 2 and δvQSO < −5000 km s−1 called cmpltrec_MgII_1.5z2.0.fit (see below), you can view the completeness curve and errors, if the IDLUTILS package is installed (available e.g., here), with the following command:

IDL>cxw = sdss_getdxw('cmpltrec_MgII_1.5z2.0.fit', mgiistrct.zabs_orig[0], mgiistrct.ew_orig[0], sigewabs=mgiistrct.sigew_orig[0], sigma=sigcxw, /czw)
IDL>plot, mgiistrct.ew_orig[0], dxw, psym=4, xtitle='EW', ytitle='C(W)'
IDL>oploterror, mgiistrct.ew_orig[0], mgiistrct.sigew_orig[0], sigcxw[*,0], /lobar
IDL>oploterror, mgiistrct.ew_orig[0], mgiistrct.sigew_orig[0], sigcxw[*,1], /hibar
If you exclude the /czw option, SDSS_CALCDXW() returns the co-moving pathlength available to detect each given equivalent width. To correct the completeness or pathlength obscured by detected absorbers with greater equivalent width, include the civobs_corr=mgiistrct option.

You can generate an equivalent width frequency distribution ƒ(Wr) as follows:

IDL>fw = sdss_calcfxw(mgiistrct, 'cmpltrec_MgII_1.5z2.0.fit', [1.5,2])

This returns a structure that can be plotted with the errors as follows:

IDL>plot, fw.ewcenter, fw.fxw, psym=4, xtitle='EW', ytitle='f(W)', /ylog
IDL>oploterror, fw.ewcenter, fw.fxw, fw.sigewcenter[*,0], fw.sigfxw[*,0], /lobar
IDL>oploterror, fw.ewcenter, fw.fxw, fw.sigewcenter[*,1], fw.sigfxw[*,1], /hibar

You can fit the 1.5 ≤ z < 2 frequency distribution with an exponential and overplot the results on the previous plot with the following calls:

IDL>sdss_fitfxw, 'cmpltrec_MgII_1.5z2.0.fit', intlim=[0.8, 8.5, 8.5], /exp, civstrct_fil=mgiistrct, ostrct_fil='fwfit_MgII_1.5z2.0.fit', psfil='fwfit_MgII_1.5z2.0.ps', /plot, sigma=1.
IDL>fwfit = sdss_calcfxwfit(fw.ewcenter, fitstrct='fwfit_MgII_1.5z2.0.fit')
IDL>oplot, fw.ewcenter, fwfit, thick=2, linestyle=2

You can generate and plot a structure with the absorber line density dNMg II/dX for an equivalent width range, as a function of redshift, given a completeness curve with multiple redshift bins cmpltrec_MgII_0.3z2.3cum.fit.

IDL>dndx = sdss_calcdndx(mgiistrct_all, 'cmpltrec_MgII_0.3z2.3cum.fit', [1.0, !values.f_infinity]
IDL>plot, dndx.zcenter, dndx.dndx, psym=4, xtitle='z', ytitle='dN/dX'
IDL>oploterror, dndx.zcenter, dndx.dndx, dndx.sigzcenter[*,0], dndx.sigdndx[*,0], /lobar
IDL>oploterror, dndx.zcenter, dndx.dndx, dndx.sigzcenter[*,1], dndx.sigdndx[*,1], /hibar
To subtract off the false positives, include /afp_flg, biasuser_fil='cmpltrec_userbias_noBAL.fit' in the call.

Continuum Fitting, Identifying Candidates, and Generating Completeness Curves

Continuum Fitting:

There are three flavors of continuum fits: eigen, spline, and hybrid. The latter depends on the former, and the order of the following calls will lead to the hybrid-fit. The required file is a structure with the information for the QSOs to be fit, dr7qso_MgII.fit (formatted like the Schneider et al. (2010) catalog, see example above).

  1. First fit the eigen-continua, which can be accomplished by running one line in IDL or using a shell script to divide up the task between multiple "processors" via multiple IDL sessions. (Make sure the shell script is executable.) The output will be written to the conti/ directory:
    XIDL>runeigqsoconti, sdsssum='dr7qso_MgII.fit'
    TERM>$XIDL_DIR/SDSS/CIV/eig_runparallelfit.sh 4 dr7qso_MgII.fit
  2. Search the eigen-normalized spectra of absorption features and create the SDSSCONTISTRCT__DEFINE.PRO structure in the abslin/ directory. This code will require a text file that lists the directory to write the output to on the first line, followed by the spectra names, in the same order as the QSO structure, e.g., dr7qso_MgII.list (see example above). This again can be accomplished by a single IDL call or by running a shell script:
    IDL>sdss_fndlin, 'dr7qso_MgII.list', 'dr7qso_MgII.fit', /noBAL, only_cflg=sdss_getcflg(/eig)
    TERM>$XIDL_DIR/SDSS/CIV/sdss_runparallelsrch.sh 4 dr7qso_MgII.list dr7qso_MgII.fit "/noBAL, only_cflg=sdss_getcflg(/eig)"
  3. The hybrid-fitting algorithm will use the lines found in the first eigen-normalized spectra to make a mask. Whether a given spectrum is corrected by a spline-fit to the eigen-normalized spectrum is determined by the S/N, and it is faster to just query the S/N structure, generated by SDSS_MKSNRTAB. This too can be done with one IDL call or via a shell script:
    IDL>sdss_dblfitconti, 'dr7qso_MgII.list', 'dr7qso_MgII.fit', snrstrct_fil='dr7qso_MgII_SNR.fit', dblt_name='MgII'
    TERM>$XIDL_DIR/SDSS/CIV/hyb_runparallelfit.sh 4 dr7qso_MgII.list dr7qso_MgII.fit "snrstrct_fil='dr7qso_MgII_SNR.fit', dblt_name='MgII'"
  4. [Optional] You can also use only a b-spline to fit the continua, via an IDL call or a shell script:
    IDL>sdss_fndlin_fitallspl, 'dr7qso_MgII.list', 'dr7qso_MgII.fit', pixlim=7
    TERM>$XIDL_DIR/SDSS/CIV/spl_runparallelfit.sh 4 dr7qso_MgII.list dr7qso_MgII.fit "pixlim=7"

Identifying Candidates:

To generate a candidate structure, you first must search the normalized spectra for automatically detected lines and then pair these into candidate Mg II doublets. You can generate candidates from any three of the continua stored in the abslin/ structure by changing the cflg. Focusing on the hybrid-normalized spectra:

  1. To search the hybrid-normalized spectra and store the results in the abslin/ structures, use either of the following calls:
    IDL>sdss_fndlin, 'dr7qso_MgII.list', 'dr7qso_MgII.fit', /noBAL, only_cflg=sdss_getcflg(/hyb)
    TERM>$XIDL_DIR/SDSS/CIV/sdss_runparallelsrch.sh 4 dr7qso_MgII.list dr7qso_MgII.fit "/noBAL, only_cflg=sdss_getcflg(/hyb)"
  2. Now you can actually search for the candidates. Below, a secondary line search at the 2.5 σ level is called to find partners for un-matched, 3.5 σ lines:
    IDL>sdss_civsearch, 'dr7qso_MgII.list', 'dr7qso_MgII.fit', 'sdss_mgiicand_hyb.fit', dblt_name='MgII', lsnr2=2.5, cflg=sdss_getcflg(/hyb)
    TERM>$XIDL_DIR/SDSS/CIV/sdss_runparallelsrch.sh 4 dr7qso_MgII.list dr7qso_MgII.fit sdss_mgiicand_hyb.fit "dblt_name='MgII', lsnr2=2.5, cflg=sdss_getcflg(/hyb)"
  3. To rate your own candidates, you can use the GUI:
    IDL>sdss_chkciv, 'sdss_mgiicand_hyb.fit', 'sdss_mgiirate_hyb.fit', dblt_name='MgII'

Generating Completeness Curves:

There is a completeness file for each sightline tested. The information in these files need to be combined into the redshift bin or bins of interest, including any δvQSO cut. This can be accomplished for an explicit redshift range as follows:

IDL>.r sdss_completeness
IDL>sdss_completeness_czw, 'dr7qso_MgII.list', 'dr7qso_MgII_SNR.fit', 'cmpltrec_MgII_0.3z2.3cum.fit', zbinsize=2.1, zlim=[0.3, 2.3], dvqso=-5000, /rec_param, civobs_corr=mgiistrct

Since only a subset of sightlines have Monte Carlo tests, the un-tested sightlines are accounted for by the average of the tested sightlines in the same S/N and zQSO bin. The zbinsize= is set to something large to make just one bin in the same file, for the full redshift range 0.3 < z < 2.3. The /rec_param option signals that the redshifts and equivalent widths used for combining the completeness curves are not the input, Voigt-profile values but the ones automatically measured from the simulated spectra. The civobs_corr=mgiistrct option corrects the completeness or pathlength obscured by detected absorbers with greater equivalent width and folds in the error.

You can also combine the Monte Carlo files into little redshift bins either by forcing an approximate number per bin (first line below) or by explicitly setting the bins (second). You input the root of the output files that will be made.

IDL>sdss_completeness_cum, 2500, 'cmpltrec_MgII_', dvqso=-5000, /rec_param, dblt_name='MgII'
IDL>zlim=fltarr(10,2) & zlim[*,0] = 0.2*lindgen(10)+0.3 & zlim[*,1] = zlim[*,0] + 0.2
IDL>sdss_completeness_cum, zlim, 'cmpltrec_MgII_', /z, dvqso=-5000, /rec_param, dblt_name='MgII'

To actually run Monte Carlo tests yourself, you can use a single IDL call or the shell script:

IDL>sdss_completeness, 'dr7qso_MgII.list', 'dr7qso_MgII.fit', 1000, dblt_name='MgII', config_fil=getenv('XIDL_DIR')+'/SDSS/MgII/sdss_mgiiprof.config', vpstrct_fil='vpstrctmgii72422le500perEW.fit'
TERM>$XIDL_DIR/SDSS/CIV/sdss_runparallelcmplt.sh 4 dr7qso_MgII.list dr7qso_MgII.fit 1000 "dblt_name='MgII', config_fil=getenv('XIDL_DIR')+'/SDSS/MgII/sdss_mgiiprof.config', vpstrct_fil='vpstrctmgii72422le500perEW.fit'"

The use of the given simulated profile library (vpstrctmgii72422le500perEW.fit) is optional. The code can call SDSS_GENPROF_MC() and use the sdss_mgiiprof.config file to generate profiles on the fly, though this is slower.

igmabsorbers.info is funded by the National Science Foundation Astronomy & Astrophysics Postdoctoral Fellowship (AST-1003139).