Simultaneous analog and photon counting detection for Raman lidar

Simultaneous analog and photon countingdetection for Raman lidar

Rob K. Newsom,1,* David D. Turner,2 Bernd Mielke,3 Marian Clayton,4 Richard Ferrare,5

and Chitra Sivaraman1

1Pacific Northwest National Laboratory, PO Box 999, MSIN K9-30, Richland, Washington, 99352, USA2University of Wisconsin, Madison, Wisconsin 53706, USA

3Licel GmbH, Gustav-Meyer-Allee 25, 13355 Berlin, Germany4SSAI/NASA Langley Research Center, Hampton, Virginia 23681, USA

5NASA Langley Research Center, Hampton, Virginia 23681, USA

*Corresponding author: [email protected]

Received 30 October 2008; revised 29 April 2009; accepted 20 May 2009;posted 1 June 2009 (Doc. ID 103375); published 1 July 2009

The Atmospheric Radiation Measurement program Raman lidar was upgraded in 2004 with a new datasystem that provides simultaneous measurements of both the photomultiplier analog output voltage andphoton counts. We describe recent improvements to the algorithm used to merge these two signals into asingle signal with improved dynamic range. The effect of modifications to the algorithm are evaluated bycomparing profiles of water vapor mixing ratio from the lidar with radiosonde measurements over a sixmonth period. The modifications that were implemented resulted in a reduction of the mean bias in thedaytime water vapor mixing ratio from a 3% dry bias to well within 1%. This improvement was obtainedby ignoring the temporal variation of the glue coefficients and using only the nighttime average gluecoefficients throughout the entire diurnal cycle. © 2009 Optical Society of America

OCIS codes: 010.3640, 010.7340, 040.5250, 280.3640, 290.5860, 120.0280.

1. Introduction

Since 1998 the US Department of Energy’s (DOE)Atmospheric Radiation Measurement (ARM) pro-gram has operated a Raman lidar at its SouthernGreat Plains (SGP) site near Billings, Oklahoma,USA (97:487° W, 36:609° N). This site contains an ex-tensive suite of instrumentation dedicated to long-term climate observations [1]. The primary role ofthe ARM Raman lidar is to provide continuousheight- and time-resolved measurements of watervapor, aerosol, and clouds [2,3].The ARM Raman lidar is an autonomous, turn-key

system that transmits at a wavelength of 355nmwith 300mJ, ∼5ns pulses, and a pulse repetitionfrequency of 30Hz. The detection system currently

consists of 10 channels. These include two water va-por channels at 408nm, two nitrogen channels at387nm, three elastic channels, two temperaturechannels at 354 and 353nm, and one liquid waterchannel at 403nm. The lidar has two fields of view(FOVs). Three channels (water vapor, nitrogen,and elastic) have a wide FOV of 2mrad, and the re-maining channels have a narrow FOV of 0:3mrad.All channels use Electron Tube 9954B photomulti-plier tubes (PMTs). Data products derived from theARM Raman lidar include water vapor mixing ratio,aerosol scattering ratio, aerosol extinction, and line-ar depolarization ratio [4].

In 2004 the system underwent a major upgrade[5,6], which included replacement of the existingdata system with new transient data recorders fromLicel GmbH (Berlin, Germany). Also, in 2005, threenew channels (and three new Licel recorders) were

0003-6935/09/203903-12$15.00/0© 2009 Optical Society of America

10 July 2009 / Vol. 48, No. 20 / APPLIED OPTICS 3903

added for temperature and liquid water profiling[7,8]. The Licel recorders, as installed in the ARMRaman lidar, are currently configured to output dataat a range resolution of 7:5m and a temporal resolu-tion of 10 s. These recorders are unique in the sensethat they provide simultaneous measurements ofboth analog photomultiplier current and photoncounts. By optimally merging these two signals itis possible to achieve improved dynamic range. In-deed, the new detection electronics have enabledwater vapor profiling up to 5–6km above ground le-vel (AGL) during the day, which is amarked improve-ment over the original version of this lidar (∼3km in1999) [5].To enable downstream processing the photon

counting and analog signals must first be combinedinto a single signal through a process that has be-come known as “gluing” [9]. The so-called MERGEvalue added procedure (VAP) was developed in orderto optimally combine the raw analog and photoncounting signals from the Licel electronics into a sin-gle signal that can be ingested by the other Ramanlidar VAPs. These other VAPs include procedures forcalculating water vapor mixing ratio, relative humid-ity, aerosol scattering ratio, aerosol extinction, aero-sol optical depth, depolarization ratio, temperature,and liquid water content. After the addition of theLicel recorders in 2004, the MERGE VAP became thefirst step in the data processing chain for the ARMRaman lidar.Earlier versions of the MERGE VAP tended to

cause large biases in daytime mixing ratios. As a re-sult, these versions were never implemented opera-tionally within ARM’s data management facility, andwith time the backlog of unprocessed raw data grew.The problem was recently elevated to critical statusby the ARM community, and an intensive effort waslaunched to identify and remedy the problem in theMERGE VAP [10,11]. This paper summarizes thoseefforts and the modifications that have been madethat have resulted in a significant reduction of thedaytime water vapor mixing ratio bias. A goal of thispaper is to examine how the gluing procedure affectsthe lidar’s ability to maintain its calibration throughthe entire diurnal cycle, particularly during thedaytime.This paper is organized as follows. First we

describe the basic MERGE algorithm and improve-ments to the regression technique for determinationof the so-called glue coefficients. Second, we discussthe method that was used to determine the systemdead-time parameter for each detection channel.The methods described in these sections differ in sev-eral important aspects from the approach outlined byWhiteman et al. [9], and these differences are alsodiscussed in those sections. Finally, the effect ofchanges to the MERGE VAP are assessed by compar-ing profiles of water vapor mixing ratio from thelidar with simultaneous and collocated radiosondemeasurements.

2. MERGE VAP

The MERGE VAP ingests the raw photon countingand analog signals from each of the ten detectionchannels for one 24 h period. The output of thisVAP consists of photon count rates that have beencorrected for nonlinearity.

The first step in the MERGE algorithm is to con-vert photon counts to count rates in units of mega-hertz, and the raw accumulated digitized analogvalues to millivolts. Next, measurements of the elec-tronic background are interpolated to the times ofeach profile and subtracted from the signals. Theelectronic background level is measured automati-cally by the system every hour by blocking all detec-tion channels. A correction for the relative temporaloffset between the analog and photon count rate sig-nals is then applied. Electronic processing of the ana-log signal induces a slight delay relative to thephoton counting signal. This delay is a property ofthe electronics and is fixed for a given data recorder.The delay is corrected for by shifting the analog sig-nal along the range axis in order to obtain the bestmatch with the photon count rate signal. The nextstep in the MERGE VAP involves correcting thephoton count rate for pulse pileup effects. Assumingthat the PMT and associated electronics obey thenonparalyzable assumption [12], the pulse pileupcorrection takes the following form:

C0ij ¼

Cij

1 − τCij; ð1Þ

where Cij is the measured count rate, and τ is the re-sponse time. The indices i and j are used to denotetime and height such that Cij ¼ Cðti; zjÞ.

Above its inherent noise floor the analog signal isassumed to be proportional to the “true” count rate.Thus, we define the so-called “virtual” count rate asCij ¼ siAij þ oi;where Aij is the analog signal. Theslope si, and offset oi, are referred to as the glue co-efficients (GCs). A single set of GCs is determined foreach profile. Thus, si and oi are in general time de-pendent and height independent.

As mentioned previously, the electronic back-ground level has been removed from the analog sig-nal in Eq. (2). However, this background level ismeasured only once every hour, and it does in factchange on time scales shorter than this. These short-time scale fluctuations are typically much smallerthan longer-time scale fluctuations. Nevertheless,there is some residual electronic background in theanalog signal that must be taken into account. Thus,the offset parameter is included in the expression forthe virtual count rate.

A. Glue Coefficients

The basic idea behind the MERGE algorithm is tofind si and oi that results in the best fit betweenthe dead-time-corrected count rate, C0

ij, and the vir-tual count rate, Cij. The fit range is constrainedsuch that

3904 APPLIED OPTICS / Vol. 48, No. 20 / 10 July 2009

C0bi þ δC < C0

ij < Cmax; ð3Þwhere C0

bi is the mean background of the dead-time-corrected count rate, and δC and Cmax are prescribedparameters that set lower and upper thresholds onthe fit range, respectively. The lower threshold is spe-cified by a fixed increment, δC, above the solar back-ground. Thus, the lower limit of the fit range varieswith the solar background, whereas the upper limitremains fixed. The lower limit is imposed to preventbackground Aij–C0

ij sample pairs from influencing theregression result. In our current implementation weuse δC ¼ 0:5MHz, which is sufficient to eliminatemuch of the background for C0

bi less than about5MHz.It is necessary to impose a fixed upper limit on the

fit range because Eq. (1) represents an approxima-tion that is only strictly applicable for small signals,i.e., for τCij ≪ 1. Thus, Cmax should be set to a smallvalue in order to prevent application of Eq. (1) to ar-bitrarily large count rates. On the other hand, Cmaxmust also be large enough to ensure an adequate fitrange as the solar background increases above zero.These conflicting requirements create problems forthe determination of the GCs during the daytimefor the solar-sensitive channels (water vapor and ni-trogen channels) because the fit range shrinks as thesolar background increases.In our current implementation we use Cmax ¼

50MHz. Even with this relatively large value ofCmax there are long periods during the day whenCbi þ δC ≥ Cmax for the water vapor channel. Duringthese periods it is not possible to derive the GCs, andwe are forced instead to simply interpolate throughthe daytime voids.Once the GCs are determined, the merged signal is

obtained by combining the virtual and correctedcount rate data such that

Cmergeij ¼

(C0

ij forC0ij < Cmax

Cij forC0ij ≥ Cmax

: ð4Þ

The merged signal contains the virtual count ratedata above the upper threshold, and the dead-time-corrected count rate data below this level. Bycombining the signals in this manner we take advan-tage of the superior sensitivity of the photon countingdata in the weak signal regime, and the improved lin-earity of the analog data in the strong signal regime.

B. Baseline Regression Method

In the original or baseline implementation of theMERGE VAP the GCs are determined by performinga least-squares fit between the dead-time-correctedcount rates,C0

ij, and the virtual count rates, Cij, whileconstraining the fit range according to Eq. (3). This isachieved by minimizing

Jbaselinei ¼

Xj

ðC0ij − CijÞ2σ2ij

ð5Þ

with respect to the slope, si, and offset, oi, in the ex-pression for CijðAijÞ. The fluctuations in C0

ij are welldescribed by a Poisson distribution. Thus, the mea-surement error, σij, is taken to be proportional

toffiffiffiffiffiffiC0

ij

q.

In Eq. (5) the dead-time-corrected count rates, C0ij,

are regarded as measurements with correspondingabscissa values Aij. Thus, C0

ij is treated as the depen-dent variable and Aij is treated as the independentvariable. The fit range is constrained by imposinglower and upper thresholds on the dependent vari-able. This creates a problem because the regressionalgorithm assumes that the dependent variablecontains normally distributed noise, while thethresholds impose an artificial truncation of thesefluctuations. This has the effect of biasing the esti-mates of si and oi. Instead, the fit range should beconstrained by applying thresholds on the indepen-dent variable.

C. Modified Regression Method

The problem described above can be remedied in oneof two ways. We can either minimize Eq. (5) by spe-cifying limits on the analog data, Aij, or we could re-formulate Eq. (5) and retain Eq. (3) as the constraint.Since it seems more natural to us to define the limitsof the fit range in terms of count rate, we chose thelatter approach.

In the modified MERGE VAP the GCs are deter-mined by performing a least-squares fit betweenthe analog voltage, Aij, and the so-called virtual ana-log signal

Aij ¼ s0iC0ij þ o0i: ð6Þ

The objective is then to minimize

Ji ¼Xj

ðAij − AijÞ2σ2ij

ð7Þ

by adjusting s0i and o0i. The measurement error inEq. (7) is assumed to be proportional to the squareroot of the analog voltage.

The only significant difference between the modi-fied and baseline approach is in the assignment ofthe independent and dependent variables in the lin-ear regression. In the baseline method C0

ij is assignedthe role of dependent variable and Aij is the indepen-dent variable. In the modified method these roles arereversed. Once s0i and o0i are determined, the GCscan be obtained by inverting Eq. (6). The resultsare given by

si ¼ 1=s0i; ð8Þ

oi ¼ −o0i=s0i: ð9Þ

In the modified method Eqs. (8) and (9) are used tocompute the virtual count rate, Cij, from Eq. (2). The


dead-time-corrected count rates and virtual countrates are then merged, as in the baseline method,using Eq. (4).Figure 1 illustrates the effect that this modifica-

tion has on the virtual count rate, Cij. This particularexample shows results from the wide FOV water va-por channel during a late morning period on 16 Au-gust 2007. The solar background in this case is about43MHz, and the fit range is between 43:5MHz and50MHz. The dashed and solid lines in Fig. 1 are thelinear fits produced by the baseline and modified re-gression techniques, respectively. Figure 1(a) showsthe baseline result in which C0

ij is treated as the de-pendent variable, while Fig. 1(b) shows the modifiedregression result in which the analog signal is trea-ted as the dependent variable. In both cases, thesame data are used, and the fit region is constrainedby imposing lower and upper thresholds on C0

ij. Inthis particular example, the baseline regressionmethod yields slope and offset values of 17:19�1:43MHzmV−1 and 22:50� 1:90MHz, respectively.After inversion, the corresponding slope and offsetvalues from the modified regression method are42:03� 2:26MHzmV−1 and −10:47� 2:50MHz, re-spectively. We note that the offset computed fromthe modified method is closer to zero, as it shouldbe. This example illustrates the large differencesthat can occur as a result of constraining the fit re-gion based on the dependent variable as opposed tothe independent variable. Indeed, the modified re-gression method yields a much better fit to the databecause the fit region is defined by thresholds im-posed on the independent variable.Figure 2 shows a time series of the slope, si, com-

puted from the baseline and modified regressionmethods for the wide FOV water vapor and nitrogenchannels on 16 August 2007. The baseline and mod-ified methods give nearly the same results during thenighttime. The difference between the two methodsis most apparent in the slope for the water vaporchannel as the solar background increases. Duringthe daytime the slope for the water vapor channelas computed by the baseline method decreases by

∼85% compared to its nighttime value. By contrast,the slope for the same channel as computed from themodified method decreases by ∼15% during the day-time compared to its nighttime value. Thus, it is clearthat the modified regression method exhibits muchless sensitivity to changing solar backgrounds.

A time series of the deviation between the baselineand modified slope is shown in Fig. 2(c). During thenighttime there is a ∼2% difference in the water va-por slopes, and virtually no difference between thetwo nitrogen slopes. Since the dynamic range of

Fig. 1. Representative examples of results from (a) the baseline and (b) the modified regression methods. Panel (c) shows both resultssuperimposed, where the dashed line is from the baseline regression and the solid line is the inverted modified regression result. Darkpoints fall within the lower and upper thresholds of the fit region as indicated by the dotted lines. Light gray points fall outside the fitregion and are not used in the regression analysis. This example is taken from the wide FOVwater vapor channel at about 1600UTC on 16August 2007.

Fig. 2. Time series of the slope, s, computed using (a) the baselineregression method and (b) the modified regression method. Panel(c) shows the difference between the baseline and modified slopes,normalized by the modified slope. The light gray dotted verticallines indicate the times of sunset and sunrise. These exampleswere taken from the wide FOV water vapor (solid) and nitrogen(dashed) channels on 16 August 2007. Linear interpolation is usedto fill the daytime voids (between 16 and 22 UTC for the watervapor). The dotted vertical lines at about 01:15 and 12:00 UTCrepresent the times of sunset and sunrise, respectively.


the water vapor signal is approximately 10MHz, thisnighttime difference is due entirely to the effect ofthe lower threshold. The upper threshold does nothave a significant effect until the difference betweenCmax and the solar background approaches 10MHz.By contrast, the nitrogen signal exhibits more dy-namic range (∼50MHz) and is therefore affectedby the upper and lower thresholds simultaneouslyregardless of the solar background level. However,the nitrogen signal exhibits less photon noise, andso the slopes show less sensitivity to the changing so-lar background.It is interesting to note that in Fig. 2 both regres-

sion methods show a slow drift in the slope duringthe night. This drift is most apparent in the solarsensitive channels, i.e., the water vapor channeland to a lesser extent the nitrogen channel. This driftis also more pronounced during summer than in win-ter. This behavior, which does not appear to be anartifact of the processing, suggests that the PMT ex-periences a slow relaxation process during the night-time in response to strong solar illumination duringthe daytime. We are currently investigating thiseffect.Whiteman et al. [9] describes an approach to the

computation of the GCs that differs in several re-spects to the methods described in this paper. Theprimary difference is that Whiteman removes the so-lar background level from both the count rate and theanalog signal prior to performing the linear regres-sion, whereas the background levels are retainedin both regression methods described in this paper.Whiteman observed that with his system, subtract-ing the backgrounds prior to regression resulted inless variability in the time series of the GCs [13]. An-other important difference is that in Whiteman’s ap-proach the fit range is restricted to photon countrates between 1 and 20MHz. Otherwise, the regres-sion analysis is similar to the baseline method de-scribed above, i.e., the photon count rate is treatedas the dependent variable.Whiteman’s results also show that as the solar

background changes, the computed slopes for thewater channel change quickly. This is consistent withour results using the baseline method, and suggestsa biasing of the GCs due in part to thresholds im-posed on the dependent variable. To overcome thisproblem, Whiteman applies the mean of the night-time GCs to the entire data record, and uses onlythe virtual count rate whenever the solar back-ground exceeds 1MHz.

3. System Dead-Time Estimation

The gluing procedure that is implemented within theMERGE VAP treats the slope and offset as adjusta-ble parameters, and the response time, τ, as aprescribed parameter. Thus, it is necessary to deter-mine τ for each detection channel prior to runningthe MERGE VAP. Once established, the responsetime is assumed to be a fixed system parameter thatremains constant over periods lasting several days or

longer. In actuality, the response time, τ, appearingin Eq. (1) represents the combined effect of deadtimes associated with the transient recorder andthe pulse shape, which in turn depends on the highvoltage settings and the PMT characteristics. Any ofthese properties may slowly vary with time. Thus, itis necessary to periodically monitor the system deadtime, and to update the prescribed τ values whenneeded. This section describes the analysis methodthat is used to estimate τ. This analysis is performedindependent of the MERGE VAP, and is only used tomonitor the system dead time.

Prior to the addition of the Licel data recorders thestandard technique for estimating τ involved analy-sis of full- and reduced-strength signals [3]. The re-duced strength signal was recorded by placing aneutral density filter in front of the PMT duringthe lidar’s calibration cycle. One shortcoming to thisapproach was that the full- and reduced-strengthsignals could not be recorded simultaneously. Itwas also difficult to apply this technique to noisy sig-nals with very limited dynamic range, as in the caseof the water vapor signal. Thus, the response timefor the water vapor channel was determined by tak-ing the mean response time of the other detectionchannels.

With the addition of the Licel electronics it becamepossible to use the simultaneous measurements ofCij and Aij sample pairs to estimate the responsetime. The basic premise is that the analog signalhas a linear response and that Eq. (1) is a valid de-scription of the pulse pileup effect for counting ratesless than Cmax.

Whiteman et al. [9] describes a method in whichthe response time is estimated from individual pro-files by requiring that the offset, o, be zero, as shouldbe the case in the absence of any electronically in-duced background in the analog and photon countingsignals. In this method the photon counting rate sig-nal is first corrected for pulse pileup effects usingEq. (1). Then the background levels are subtractedfrom each signal and a linear regression is performedin amanner similar to our baseline approach, i.e., thecounting rate signal is treated as the dependentvariable. The above steps are then repeated by sys-tematically varying the response time until o ¼ 0is found.

The approach used in this study involves estimat-ing the response time by minimizing the root-mean-squared difference between the measured analogand the virtual analog data. This represents a slightgeneralization of the modified regression techniquein which the slope, offset, and response time are trea-ted as adjustable parameters. The problem is then tominimize a modified version of Eq. (7) with respect tos0i, o

0i, and τ. The response time obtained from this

analysis gives the best linear fit between the cor-rected counting rate and analog signals.

When this analysis is applied to a single profile, itis difficult to obtain reliable estimates of τ from thewater vapor signal due to its limited dynamic range


(typically less than 10MHz). However, since thewater vapor channel is also sensitive to solar radia-tion, it is possible to increase the range of signalavailable to the analysis by using many profiles witha range of solar background levels. The approach ta-ken here involves using profiles that span an entirediurnal cycle. This so-called “multiple-profile” meth-od enables improved estimation of τ, assuming thatthe relationship between the analog and countingrate signals does not change with time.The process of estimating τ using the multiple-

profile method begins by selecting several days withlittle or no cloud cover. A separate estimate of τ iscomputed for each 24 h period using all cloud-freeprofiles within that period. This typically resultsin a very large number of Cij and Aij sample pairs(on the order of 107).The first step in the multiple-profile method is to

reduce the volume of measurements by computing asingle curve that describes the relationship betweenthe count rate and the analog voltage. This isachieved using a conditional averaging procedurein which the count rates are divided into N equallyspaced bins from 0 to Cmax. The kth bin includes allCij and Aij sample pairs that satisfy Eq. (3) andkΔC ≤ Cij < ðkþ 1ÞΔC, where ΔC ¼ 1MHz. The re-duced data set is obtained by computing the mediancount rate, ck, the median analog voltage, ak, and thestandard deviation of the analog voltage δak withineach bin. This procedure, which is illustrated inFig. 3, establishes a single curve in which ak andck are regarded as the dependent and independentvariables, respectively.Once ck, ak, and δak have been determined, the

problem is then to minimize

JðτÞ ¼ N−1XN−1

k¼0

ðak − akÞ2δa−2k ð10Þ

with respect to τ. As before, the virtual analog dataare given by ak ¼ s0c0k þ o0, and the dead-time-corrected count rates are given by c0k ¼ ckð1 − τckÞ−1.The minimum of JðτÞ is determined by systemati-cally varying τ from 0 to 10ns in steps of 0:1ns. Ateach step, s0 and o0 are determined by linear regres-sion. Theminimum value of JðτÞ is then estimated byfitting a parabola to the smallest value of JðτÞ and itstwo neighboring values. The value of τ correspondingto the minimum of the parabola fit is taken to be theoptimal response time.

When performing this minimization, it is impor-tant to properly normalize JðτÞ. As τ is increasedfrom zero, it is likely that c0k will exceed Cmax for someof the higher bins. Since we wish to prevent this, werequire that c0k ≤ Cmax. Thus, as τ is varied, the num-ber of bins used in the evaluation of JðτÞ will, ingeneral, change. Accordingly, Eq. (10) includes a nor-malization parameter, N, that prevents biasing theresult based purely on the number of bins includedin the evaluation of JðτÞ.

Figure 4 illustrates the procedure described aboveusing Cmax ¼ 50MHz, which is currently the valueused in the operational software. The data used inthis example were obtained from the narrow FOVnitrogen channel (387nm) on 31 August 2007.Figure 4(a) shows that JðτÞ exhibits a well definedminimum at τ ¼ 5:04ns, in this case. Figure 4(b) dis-plays the conditionally-averaged analog voltage, ak,versus the conditionally-averaged corrected (dashed)and uncorrected (solid) count rates. Clearly, the pulsepileup correction helps to linearize the relationshipbetween the analog voltages and the count rates.Last, Fig. 4(c) shows the fit residual as a functionof the analog voltage, ak. Here the fit residual is ex-pressed as the percent deviation of the analog datafrom the virtual analog data, i.e., 100ðak − akÞ=ak.

The analysis described above has been performedon many days and through all four seasons. Consis-tent results were obtained for a given detection chan-nel and for a given value of Cmax. The estimates do,however, exhibit some sensitivity to Cmax, and theminimum value of JðτÞ tends to increase withCmax. Figure 5 illustrates the effect that Cmax hason the estimate of τ and the corresponding valueof minðJðτÞÞ for the narrow FOV water vapor chan-nel. The data in Fig. 5 were obtained by averagingresults from 29 clear days between 1 April and30 September 2007. The optimal response time[Fig. 5(a)] initially increases before reaching a con-stant value for Cmax greater than about 50MHz.The initial increase in the optimal τ indicates thata stronger dead-time correction is required when lar-ger count rates are included in the fit. There comes apoint at which further increasing Cmax has little or noeffect on the estimate of τ. The corresponding valuesof minðJðτÞÞ increase monotonically with increasing

Fig. 3. Example illustrating the process of conditional averagingas applied to the counting rate and analog data. The boundary ofeach bin is indicated by the dotted vertical lines. For clarity, the binsize used in this example is ΔC ¼ 2MHz (the bin size used in theactual estimation of τ was 1MHz). Light gray points indicate ac-tualmeasurements. This example was taken from the narrowFOVnitrogen channel on 19 April 2007.


Cmax, implying that the fits degrade with increasingCmax. This is not surprising, since the nonparalyzableassumption breaks down at higher count rates.Figures 6 and 7 are also included to illustrate the

effect thatCmax has on the behavior of JðτÞ and the fitresiduals. Figures 6 and 7 show results for Cmax ¼ 20and 100MHz, respectively. As in Fig. 4, the data usedin these examples were obtained from the narrowFOV nitrogen channel (387nm) on 31 August 2007.For Cmax ¼ 20MHz, JðτÞ exhibits a minimum atτ ¼ 5:29ns; however, this curve is broader and theminimum is shallower than the same curve forCmax ¼ 50MHz [compare Figs. 5(a) and 7(a)]. Never-theless, the residual [Fig. 6(c)] is quite small, andminðJðτÞÞ is about the same as for Cmax ¼ 50MHz.For Cmax ¼ 100MHz, JðτÞ exhibits a well defined

minimum at τ ¼ 5:68ns [see Fig. 7(a)]; however,the value of minðJðτÞÞ is approximately an order ofmagnitude higher than for Cmax ¼ 20MHz. Also,the residual for Cmax ¼ 100MHz is quite large andexhibits a distinctive systematic variation as a func-tion of the analog voltage, ak [see Fig. 7(c)]. This ex-ample clearly illustrates just how poorly Eq. (1)describes the behavior of the pulse pileup correctionwhen high count rates are included in the fit. For thisreason, we limit the use of the corrected photoncounting data to count rates less than 50MHz, anduse the virtual count rates derived from the analogdata in the portions of the profile where the countrates are larger than this.

Table 1 displays response times and correspondinguncertainties for the water vapor and nitrogen detec-tion channels as obtained using the multiple-profilemethod with Cmax ¼ 50MHz. These values were ob-tained by averaging results from 29 clear days be-tween 1 April and 30 September 2007. The quoteduncertainties represent the day-to-day variabilityin the estimated response times. As indicated inTable 1, the uncertainties in τ for the two water vaporchannels are ∼10%, while the uncertainties for thetwo nitrogen channels are much smaller (∼1%to 2%).

The effects of uncertainties in τ on the GCs and thewater vapor mixing ratios were evaluated by runningthe MERGE VAP using a range of response times forthe wide and narrowwater vapor channels. Responsetimes for the wide and narrow FOV water vaporchannels were perturbed by �10% from the meanvalues shown in Table 1. Using raw data from 16August 2007, we found that a 10% change in the re-sponse time resulted in a mean percent difference(averaged over one 24 h period) in the slope of∼0:5%, and a mean percent difference in the offsetof about 5%. Output from the water vapor mixing ra-tio VAP (described later) was found to be less sensi-tive to changes in τ. A 10% change in the responsetimes for both the wide and narrow FOV water chan-nels resulted in a 0.4% mean percent difference inthe water vapor mixing ratio.

(a) (b) (c)

Fig. 4. Examples illustrating the behavior of (a) JðτÞ versus τ, (b) analog voltage versus corrected (dashed) and uncorrected (solid) countrate, and (c) the fit residual (100ðak − akÞ=ak) versus analog voltage for Cmax ¼ 50MHz. The optimal value of τ is determined from theminimum of Jmod, as shown in panel (a). This value is then used in panels (b) and (c) to correct the count rate for pulse pileup effects.These results were computed from data acquired by the narrow FOV nitrogen channel on 31 August 2007.

(a)

(b)

Fig. 5. (a) Dependence of the estimated response time, τ, on Cmax,and (b) the minimum value of JðτÞ, minðJðτÞÞ, as a function of Cmaxfor the narrow FOV water vapor channel. These results were ob-tained by averaging over 29 clear nights between 1 April and 30September 2007.


4. Lidar–Sonde Comparisons

The effects of changes to the MERGE VAP were as-sessed by comparing profiles of water vapor mixingratio from the lidar with simultaneous and collocatedradiosonde measurements. In order to develop goodstatistics, the lidar-to-sonde comparisons were con-ducted over a six month period from 1 April to 30September 2007. Sondes are typically launched fourtimes daily from ARM’s SGP central facility. Thus,the comparisons are performed using hundreds ofsoundings.

A. Calibration of the Lidar Water Vapor Mixing Ratio

Water vapor mixing ratios from the lidar are ob-tained from the ratio of the water vapor Ramansignal to the nitrogen Raman signal [2,12,14]. Theprocessing of mixing ratio data from the ARMRaman lidar is accomplished using the so-called mix-ing ratio (MR) VAP. The MR VAP ingests the outputfrom the MERGE VAP and then averages the data toa coarser height and time grid. Currently the fullheight and time resolutions are 7:5m and 10 s, re-spectively. For this study, the data were processedusing 10 min averages and a variable resolutionheight grid. Below 200m the vertical resolution is37:5m (five raw bins), from 200m to 5500m AGLthe vertical resolution is 75m (ten raw bins), andabove 5500m AGL the vertical resolution increasesincrementally up to a maximum of 600m at 14kmAGL. The variable resolution height grid helps to

mitigate the increase in random error with height.Once the averaging is complete, the MR VAP per-forms background subtraction and computes the ra-tio of the water vapor to the nitrogen signals for boththe wide and narrow FOVs.

After the initial ratios have been computed, over-lap corrections are applied to both the wide and nar-row FOV profiles. Overlap corrections are derivedmanually by first inspecting the ratio of the un-corrected wide FOV to the sonde mixing ratioprofile. Typically, the correction resembles a stronglydamped oscillation that approaches one as the heightapproaches infinity. For the wide FOV channels, com-plete overlap is achieved at a height of about 800m,and thus the correction is essentially unity above thislevel. Once a reasonable overlap correction is ob-tained for the wide FOV mixing ratio, the overlap-corrected wide FOV data are used to establish thecorrection for the narrow FOV. Here again, the cor-rection curve is developed manually by inspectingthe ratio of the (corrected) wide FOV to the (uncor-rected) narrow FOV mixing ratio profiles.

Lidar mixing ratio profiles are also corrected fordifferential molecular and aerosol transmission. Dif-ferential molecular transmission is computed from astandard atmospheric model, whereas differentialtransmission due to particle extinction is determinedfrom the output of the extinction VAP [4] prior to run-ning the mixing ratio VAP. Once the overlap and dif-ferential transmission corrections have been applied,

(a) (b) (c)

Fig. 6. Same as in Fig. 4, except Cmax ¼ 20MHz.

(a) (b) (c)

Fig. 7. Same as in Fig. 4, except Cmax ¼ 100MHz.


the narrow and wide FOVs are merged into a singleprofile.The final step in the mixing ratio VAP involves the

determination of a height-independent calibrationfactor. This is done by adjusting the calibration valueto achieve agreement in total precipitable watervapor (PWV) with that retrieved from a dual-channelmicrowave radiometer (MWR) [4]. The MWR wasalso nearly collocated with the lidar (within 300m)at ARM’s SGP central facility. It is important to pointout that the PWV calibration technique is onlyapplied during the nighttime when the lidar is ableto sense over 98% of the PWV. A single calibrationfactor is determined from nighttime measurementsduring a given 24 h period and then applied to allprofiles within that same 24 h period.

B. Sonde Calibration

The ARM program uses Vaisala sondes at the SGPcentral facility. The sondes are launched four timesdaily and provide measurements of basic atmo-spheric state variables, such as pressure, tempera-ture, and humidity.It well known that Vaisala sonde humidity profiles

suffer from diurnal biases, with the daytime mea-surements being 4% to 8% drier than nighttime mea-surements [15–17]. However, these biases are to firstorder height-independent in the lowest 4km [18]. Forthis study, the sonde humidity data were scaled toforce agreement with theMWR PWVmeasurements,which do not exhibit diurnal biases [15]. The detailsof the MWR retrieval algorithm are discussed byTurner et al. [13].The calculation of PWV from the sonde data is per-

formed over the same altitude range as the lidar.Thus, the sonde data is subject to the same heightlimitation as the nighttime lidar data. This removesbiases between the sonde and the lidar that mayarise due to differences in sensing height.

C. Results

As described above, both the sonde and the lidar mix-ing ratio data are calibrated against the same source,i.e., the MWR PWV measurements. One importantdifference is that a single calibration constant is de-termined for the lidar using only nighttime measure-ments, and then applied to the entire diurnal cycle.Thus, a key concern here is the lidar’s ability to main-tain its calibration through the daytime period.

Figures 8, 9, and 10 show profiles of the mean nor-malized difference in the sonde and lidar water vapormixing ratio using the baseline and two versions ofthe modified MERGE data. Differences are normal-ized by the sonde mixing ratios, and positive valuesimply a wet bias of the lidar relative to the sonde.Daytime and nighttime profiles were separatelyaveraged based on the count rate of the solar back-ground in the wide FOV water vapor channel.Daytime periods were defined as having a solar back-ground level greater than 1MHz, and nighttime per-iods were defined as having a solar background lessthan 0:01MHz. Also, only soundings during cloud-free periods were used in the comparisons. After ap-plying these criteria we were left with 140 daytimeand 120 nighttime soundings during the period from1 April to 30 September 2007.

Figure 8 shows that the baseline MERGE data re-sults in a small wet bias of less than 1% during thenighttime, and a fairly significant dry bias of nearly3% during the daytime. It is this relatively large day-time bias that is the primary motivation behind thisstudy. Figure 9 shows that when the modifiedMERGE data are used, the daytime dry bias is re-duced to about 2%, while the nighttime bias remainsat approximately 1%. This improvement is due to thereduced sensitivity of the modified regression meth-od to changing solar background levels.

Although the above results are encouraging, therestill remains some question as to whether further im-provements can be gained by completely eliminatingthe diurnal variation in the GCs. As a result, themodifiedMERGEVAPwas configured with an optionto use time-independent GCs. When the modifiedMERGE VAP is run with this option, the slope andoffset parameters are computed using the modifiedregressionmethod, as before, and then averaged overthe nighttime period. The average slope and offsetare then applied to the entire diurnal cycle. Figure 10shows the result of using the modified MERGE VAPwith time-independent GCs. In this case, the night-time bias remains approximately 1%, but the day-time bias has been reduced to well within 1%.

The difference between the daytime and nighttimebias is approximately 1% for the modified methodwith time-independent GCs. This compares to 3%for the time-dependent modified method, and 4%for the baseline method. The modified approachreduces differences between the daytime and night-time biases. More importantly, the best performanceis achieved when the nighttime average GCs are ap-plied throughout the entire diurnal cycle, which isconsistent with the way in which the PWV calibra-tion is performed.

5. Summary

Recent modifications to the so-called MERGE VAPhave been shown to reduce the daytime biases inwater vapor mixing ratios derived from the ARMRaman lidar. The modified regression algorithmsignificantly reduced the diurnal variation in the

Table 1. Response Times Used to Correct RawPhoton Count Rates for System Dead Timea

Detection Channel τ (ns)

Narrow FOV H2O 5:53� 0:63Wide FOV H2O 5:67� 0:44Narrow FOV N2 5:07� 0:06Wide FOV N2 4:73� 0:10

a These values were obtained by averaging results from 29 cleardays between 1 April and 30 September 2007 using the multiple-profile method with Cmax ¼ 50MHz.


GCs, and resulted in a smaller daytime water vaporbias. However, the best results were obtained by com-pletely suppressing the temporal variation of theGCs and using only the nighttime average GCsthroughout the entire diurnal cycle.The error in the baseline regression algorithm was

caused by constraining the fit range with respect tothe dependent variable. This resulted in biased esti-mates of the GCs. At night these biases were small(typically <2%); however, during the day the biasesbecame quite large. A simple fix was implemented inthe modified regression algorithm that involved es-sentially swapping the roles of dependent and inde-pendent variable in the linear regression, whileretaining the same constraint on the fit range. Thismodification significantly reduced the diurnal varia-tion of the GCs, and, in turn, reduced the daytimebias in water vapor mixing ratio.In the end, however, the best results were obtained

using constant GCs derived by averaging over a timeperiod when both regression methods produce simi-lar results, i.e., the nighttime. Applying the averagenighttime GCs to the entire diurnal cycle resulted ina mean daytime bias of less than 1%, relative to thesonde measurements. By contrast, the baselineMERGE VAP resulted in a relatively large dry biasof approximately 3% during the daytime.The question remains as to why the time-

independent approach worked best. Our less solar-sensitive channels, i.e., elastic and rotationalRaman, exhibit essentially no temporal variation inthe GC. However, we consistently observe a slow driftin the water vapor GCs during the nighttime that we

speculate is caused by a relaxation process occurringin the PMT. This drift is more pronounced in thesummer than in the winter. The change in the slopeduring the nighttime can be as large as ∼8% duringthe summer. Both regression algorithms are able tocapture this nighttime variation. However, our at-tempts at deriving the GCs during the daytime areconfounded by the elevated solar background, mak-ing it difficult if not impossible to bridge the daytimeperiod. The modified regression algorithm is an im-provement over the baseline approach in this regard;however, we suspect that processing artifacts maystill be obscuring the true daytime variation. Thebest practical solution is then to simply average themore trustworthy data, i.e., the nighttime data, andapply those averaged values to the entire diurnal cy-cle This approach works well as long as the true diur-nal variation in the GCs remain small.

In our current implementation we use δC ¼0:5MHz, which is sufficient to eliminate much ofthe background for C0

bi less than about 5MHz. How-ever, as the background increases above this level,the regression becomes increasingly more affectedby the background samples, and we suspect that thismay contribute to biases in the GCs during the day-time. A better approach may be to set δC equal tosome multiple of the background standard deviation.We are currently experimenting with this approach.

The modified MERGE VAP uses response times, τ,that are estimated using the so-called multiple-profile method. Our results indicate that this methoddecreases the uncertainty in the estimate of theresponse time compared to a similar analysis using

(a) (b)

Fig. 8. Results using the baseline MERGE VAP. Profiles of the mean percent difference between the lidar and sonde water vapor mixingratio for (a) nighttime and (b) daytime soundings. The solid lines indicate the value of the vertically averaged lidar-to-sonde ratio over thedisplayed height range, and the dotted line indicates zero bias. These results are based on 140 daytime soundings and 120 nighttimesoundings between 1 April and 30 September 2007.


single profiles. By taking advantage of the range ofsignals resulting from changing solar background le-vels, the multiple-profile method enables better esti-mation of the response time for weak solar sensitivesignals such as the water vapor signal. This methodassumes that the relationship between the analogand counting rate signals does not change with time.It was also pointed out that estimates of the re-

sponse time are sensitive to the choice of the upper

threshold, Cmax, that determines the maximumcount rate allowed in the fit. Our results showed thatas Cmax increases, the minimum of JðτÞ becomesmore well-defined while its value increases. ForCmax, at less than about 20MHz the day-to-day esti-mates of the response time tend to exhibit largervariations than for Cmax greater than about60MHz. However, the fit residuals generally in-crease as Cmax increases. Our analysis indicates that

(a) (b)

Fig. 9. Same as in Fig. 8, except these results are from the modified MERGE VAP with time-dependent GCs.

(a) (b)

Fig. 10. Same as in Fig. 8, except these results are from the modified MERGE VAP with time-independent GCs.


a reasonable trade-off between these two effects isachieved with Cmax ¼ 50MHz, which is the valuethat is currently used in the operational version ofthe modified MERGE VAP.The modified MERGE VAP (with time-

independent GCs) was recently implemented opera-tionally within the ARM data management facility atPacific Northwest National Laboratory. This re-moved a bottleneck in the Raman lidar data proces-sing chain that had existed since the new datasystem was installed in 2004. MERGE data arecurrently being generated and made available tothe user community through the ARM programweb site (http://www.arm.gov/). We are currentlyworking to bring the other Raman lidar VAPs backinto production.

We thank John Goldsmith, Sandia National La-boratories, for his invaluable expertise and continu-ing assistance with the lidar hardware, and DianaPetty for helping to lay the groundwork for theMERGE algorithm. We also thank Chris Martin,SGP Site Operations, for the day-to-day maintenanceof the ARM Raman lidar. The ARM Raman lidar issponsored by the U.S. Department of Energy, Officeof Energy Research, Office of Health and Environ-mental Research, Environmental Science Division.

References1. R. R. Stokes and S. E. Schwartz, “The Atmospheric Radiation

Measurement (ARM) Program: programmatic backgroundand design of the Cloud and Radiation Test Bed,” Bull. Am.Meteorol. Soc. 75, 1201–1221 (1994).

2. J. E. M. Goldsmith, F. H. Blair, S. E. Bisson, and D. D. Turner,“Turn-key Raman lidar for profiling atmospheric water vapor,clouds and aerosols,” Appl. Opt. 37, 4979–4990 (1998).

3. D. D. Turner and J. E. M. Goldsmith, “Twenty-four-hour Ra-man lidar water vapor measurements during the AtmosphericRadiation Measurement Program’s 1996 and 1997 water va-por intensive observation periods,” J. Atmos. Ocean. Technol.16, 1062–1076 (1999).

4. D. D. Turner, R. A. Ferrare, L. A. Heilman Brasseur,W. F. Feltz, and T. P. Tooman, “Automated retrievals of watervapor and aerosol profiles from an operational Raman lidar,”J. Atmos. Ocean. Technol. 19, 37–49 (2002).

5. R. A. Ferrare, D. D. Turner, M. Clayton, B. Schmid,J. Redemann, D. Covert, R. Elleman, J. Ogren, E. Andrews,J. E. M. Goldsmith, and H. Jonsson, “Evaluation of daytimemeasurements of aerosols and water vapor made by an opera-tional Raman lidar over the Southern Great Plains,” J. Geo-phys. Res. 111, D05S08 (2006).

6. D. D. Turner and J. E. M. Goldsmith, “Refurbishment and up-grade of the Atmospheric Radiation Measurement Raman li-dar,” presented at the 15th ARM Science Team Meeting,Daytona Beach, Florida, USA, 14–18 March 2005.

7. D. D. Turner, R. A. Ferrare, J. E. M. Goldsmith, D. Petty,M. B. Clayton, L. A. Brasseur, T. Tooman, Z. Wang, andJ. M. Comstock, “A decade of Raman lidar water vapor, aero-sol, and cloud observations at the ARM Southern Great Plainssite,” presented at the 3rd Symposium on Lidar AtmosphericApplications and AMS Annual Meeting, San Antonio, Texas,USA, 14–18 January 2007.

8. F. Russo, D. N. Whiteman, D. D. Turner, and R. M. Hoff, “Mea-surements of the aerosol indirect effect using a Raman lidar,”presented at the 24th International Laser Radar Conference,Boulder, Colorado, USA, 23–27 June 2008.

9. D. N. Whiteman, B. Demoz, P. Di Girolamo, J. Comer,I. Veselovskii, K. Evans, Z. Wang, M. Cadirola, K. Rush,G. Schwemmer, B. Gentry, S. H. Melfi, B. Mielke, D. Venable,and T. Van Hove, “Raman water vapor lidar measurementsduring the International H2O Project. I. Instrumentationand analysis techniques,” J. Atmos. Ocean. Technol. 23,157–169 (2006).

10. D. Petty and D. D. Turner, “Combine analog-to-digital andphoton counting detection utilized for continuous Raman lidarmeasurements,” presented at the 23rd International LaserRadar Conference, Nara, Japan, 24–28 July 2006).

11. R. K. Newsom, D. D. Turner, M. Clayton, and R. Ferrare, “Pro-gress on the use of combined analog and photon counting de-tection for Raman lidar,” presented at the 24th InternationalLaser Radar Conference, Boulder, Colorado, USA, 23–27June 2008.

12. D. N. Whiteman, S. H. Melfi, and R. A. Ferrare, “Raman lidarsystem for the measurement of water vapor and aerosols inthe Earth’s atmosphere,” Appl. Opt. 31, 3068–3082 (1992).

13. D. N. Whiteman, NASA Goddard Space Flight Center, Green-belt, MD 20771 (personal communication, 2008).

14. A. Ansmann, M. Riebesell, U. Wandinger, C. Weitkamp,E. Voss, W. Lahmann, and W. Michaelis, “Combined Ramanelastic-backscatter lidar for vertical profiling of moisture,aerosol extinction, and lidar ratio,” Appl. Phys. B 55, 18–28(1992).

15. D. D. Turner, B. M. Lesht, S. A. Clough, J. C. Liljegren,H. E. Revercomb, and D. C. Tobin, “Dry bias and variabilityin Vaisala radiosondes: the ARM experience,” J. Atmos.Ocean. Technol. 20, 117–132 (2003).

16. K. E. Cady-Pereira, M. W. Shephard, D. D. Turner,E. J. Mlawer, S. A. Clough, and T. J. Wagner, “Improved day-time column-integrated precipitable water vapor from Vaisalaradiosonde humidity sensors,” J. Atmos. Ocean. Technol. 25,873–883 (2008).

17. L. M. Miloshevich, A. Paukkunen, H. Vömel, andS. J. Oltmans, “Development and validation of a time-lag cor-rection for Vaisala radiosonde humidity measurements,” J. At-mos. Ocean. Technol. 21, 1305–1327 (2004).

18. H. E. Revercomb, D. D. Turner, D. C. Tobin, R. O. Knuteson,W. F. Feltz, J. Barnard, J. Bösenberg, S. Clough, D. Cook,R. Ferrare, J. Goldsmith, S. Gutman, R. Halthore, B. Lesht,J. Liljegren, H. Linné, J. Michalsky, V. Morris, W. Porch,S. Richardson, B. Schmid, M. Splitt, T. Van Hove, E. West-water, and D. Whiteman, “The ARM Program's water vaporintensive observation periods,” Bull. Am. Meteorol. Soc. 84,217–236 (2003).


Simultaneous analog and photon counting detection for Raman lidar

Documents

Transcript of Simultaneous analog and photon counting detection for Raman lidar