Post on 28-Dec-2015
Physics Division
The South Pole The South Pole TelescopeTelescope
Readout SystemReadout SystemMarch 4, 2004March 4, 2004
SPT EAB Meeting, U. ChicagoSPT EAB Meeting, U. Chicago
OUTLINEOUTLINE Overview of SystemOverview of System Technical ChallengesTechnical Challenges Current StatusCurrent Status
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 2
SPT Readout TeamSPT Readout Team Matt Dobbs + Helmuth SpielerMatt Dobbs + Helmuth Spieler
overall coordination, design, integration, hands on implementationoverall coordination, design, integration, hands on implementation LBNL Engineers John Joseph and Chinh VuLBNL Engineers John Joseph and Chinh Vu
Design, Layout and Commissioning of Osc/Demod and SQUID Controller Design, Layout and Commissioning of Osc/Demod and SQUID Controller boardsboards
John Clarke (UCB) and Sherry Cho (postdoc)John Clarke (UCB) and Sherry Cho (postdoc) SQUID ExpertiseSQUID Expertise
UCB Grad Student Trevor Lanting (A. Lee)UCB Grad Student Trevor Lanting (A. Lee) Cold Frequency Domain SQUID Multiplexer commissioning and Cold Frequency Domain SQUID Multiplexer commissioning and
performanceperformance UCB Grad Student Martin Lueker (W. Holzapfel)UCB Grad Student Martin Lueker (W. Holzapfel)
SQUID shielding, SQUID Controller commissioning, softwareSQUID shielding, SQUID Controller commissioning, software Technician Dennis SeitzTechnician Dennis Seitz
SQUID Quality control and testing (experience from CDMS)SQUID Quality control and testing (experience from CDMS) much useful help and discussions from T. Crawford (Chicago)much useful help and discussions from T. Crawford (Chicago)
extended visits to LBNLextended visits to LBNL
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 3
Readout System Readout System RequirementsRequirements
TES Bolometer ReadoutTES Bolometer Readout resolve bolometer noise floor (~10 pA/√Hz)resolve bolometer noise floor (~10 pA/√Hz)
SQUID as 1st amplifier stageSQUID as 1st amplifier stage maintain voltage bias for low (0.5 Ω) impedance sensormaintain voltage bias for low (0.5 Ω) impedance sensor
low amplifier input impedance low amplifier input impedance shunt feedback shunt feedback 1/f knee low enough for drift scanning (~100 mHz)1/f knee low enough for drift scanning (~100 mHz)
AC biasAC bias
DiagnosticsDiagnostics operate SQUID open loopoperate SQUID open loop map SQUID output voltage vs. input currentmap SQUID output voltage vs. input current monitor SQUID DC level during data takingmonitor SQUID DC level during data taking full spectral distribution for each readout channelfull spectral distribution for each readout channel
Full computer controlFull computer control
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 4
APEX-SZ is SPT PrototypeAPEX-SZ is SPT Prototype SPT and APEX-SZ use same readoutSPT and APEX-SZ use same readout sharing of resources between APEX-SZ and SPT readout sharing of resources between APEX-SZ and SPT readout
systems provides SPT with early experiencesystems provides SPT with early experience APEX-SZ is 320 channelsAPEX-SZ is 320 channels APEX-SZ baseline: no multiplexingAPEX-SZ baseline: no multiplexing Readout includes all functionality req’d for multiplexingReadout includes all functionality req’d for multiplexing
APEX-SZ timeline (deployment end of this year) means APEX-SZ timeline (deployment end of this year) means electronics will be available early for SPTelectronics will be available early for SPT serves as proof-of-concept for SPTserves as proof-of-concept for SPT time for modifications / upgrades post-APEXtime for modifications / upgrades post-APEX
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 5
ImplementationImplementation 8-channel SQUID controller8-channel SQUID controller
mounted directly on cryostatmounted directly on cryostat full computer control + diagnosticsfull computer control + diagnostics one cable per 8-ch controller connects to demodulator boardone cable per 8-ch controller connects to demodulator board
Demodulator + digitizer for each bolometer channelDemodulator + digitizer for each bolometer channel rack-mounted in receiver cabinrack-mounted in receiver cabin 3 VME crates (9U) for 940 bolometers3 VME crates (9U) for 940 bolometers 16 demodulator channels per 9U board16 demodulator channels per 9U board each board includes digitization + computer interfaceeach board includes digitization + computer interface connections to computer optically isolatedconnections to computer optically isolated
Use standard commercial ICs throughoutUse standard commercial ICs throughout low cost per channellow cost per channel
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 6
MultiplexingMultiplexing reduce heat transfer through wiring to cold stagereduce heat transfer through wiring to cold stage scalability of system to larger arraysscalability of system to larger arrays minimize complexity of cold wiringminimize complexity of cold wiring reduce number of SQUIDsreduce number of SQUIDs
reduce costreduce cost reduce testing timereduce testing time minimize Prozac usageminimize Prozac usage
increased dynamic range requirements relative to single increased dynamic range requirements relative to single bolometer readoutbolometer readout
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 7
Frequency Domain Frequency Domain MultiplexingMultiplexing
complementary to time domain multiplexingcomplementary to time domain multiplexing bolometers are AC biased in superconducting transitionbolometers are AC biased in superconducting transition
ffcarrier carrier well above bolometer bandwidthwell above bolometer bandwidth
bolometers grouped in readout “modules”, 8-32 channelsbolometers grouped in readout “modules”, 8-32 channels each bolometer in module biased at unique feach bolometer in module biased at unique fcarrriercarrrier = 0.5-1 MHz = 0.5-1 MHz each channel has stand-alone capability (no shared oscillators)each channel has stand-alone capability (no shared oscillators)
incident radiation (sky signal) causes variation in Rincident radiation (sky signal) causes variation in RBOLOBOLO
amplitude modulates carrier amplitude modulates carrier transfers signal power to sidebands (~1kHz bandwidth)transfers signal power to sidebands (~1kHz bandwidth) bolometer signals at unique frequencies bolometer signals at unique frequencies can be summed in one wire can be summed in one wire
bias currents are applied as “comb” of carriersbias currents are applied as “comb” of carriers one wire for bias one wire for bias module requires 2 wires module requires 2 wires
frequency-selective demodulation to separate bolometer signalsfrequency-selective demodulation to separate bolometer signals
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 8
Frequency Domain Frequency Domain MultiplexerMultiplexer
f500 kHz
f500 kHz
200 Hz
80 dBc
Amp-Modulated by BoloDDS
Amplified
600 kHz 600 kHz f100 Hz
Demodulated
500 kHz
600 kHz
f100 Hz
f500 kHz
200 Hz
80 dBc
600 kHz
Shunt FBSQUID
fMUX Chip
DDS Oscillators
RB
R1
R2
-V2
V1
I
I1
RS
C1 C2 C3 Cn
L1 L2 L3 Ln
2
Vn(fn)
Demodulator
RSRSRS
Rfb
ADCSQUID Controller
TES Bolometers
Series Array SQUIDs
T=4 K T=0.3 K only 2 wires from coldstage
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 9
Frequency Domain Frequency Domain MultiplexerMultiplexer
fMUX Chip
DDS Oscillators
RB
R1
R2
-V2
V1
I
I1
RS
C1 C2 C3 Cn
L1 L2 L3 Ln
2
Vn(fn)
Demodulator
RSRSRS
Rfb
ADCSQUID Controller
TES Bolometers
Series Array SQUIDs
Bolometers are AC biased Bolometers are AC biased not sensitive to low frequency noise downstream of bolometernot sensitive to low frequency noise downstream of bolometer
Shunt feedback Shunt feedback provides low input impedance to maintain bolo voltage bias for AC carriersprovides low input impedance to maintain bolo voltage bias for AC carriers
Use of Digital Direct Synthesizers Use of Digital Direct Synthesizers programmable, cheap, and scalable.programmable, cheap, and scalable.
System is bolo noise limited.System is bolo noise limited.
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 10
Frequency Domain Frequency Domain MultiplexerMultiplexer
fMUX Chip
DDS Oscillators
RB
R1
R2
-V2
V1
I
I1
RS
C1 C2 C3 Cn
L1 L2 L3 Ln
2
Vn(fn)
Demodulator
RSRSRS
Rfb
ADCSQUID Controller
TES Bolometers
Series Array SQUIDs
test chip of 8-channel Multiplexer (LBNL, UCB, NGC)
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 11
Osc/Demod Readout Osc/Demod Readout BoardBoard 16 Channels / board (9u x 400mm)16 Channels / board (9u x 400mm)
Separate oscillators for bolo bias + nullingSeparate oscillators for bolo bias + nulling independent amplitude + phase controlindependent amplitude + phase control use DDS with common clock (both synchronous)use DDS with common clock (both synchronous)
harmonics, spurious signals < 80 dBc (typ < 100 dBc)harmonics, spurious signals < 80 dBc (typ < 100 dBc) noise sidebands low to 10 mHz.noise sidebands low to 10 mHz.
Sampling Demodulator (+35 dBm IP3)Sampling Demodulator (+35 dBm IP3) Parallel channel to monitor SQUID DC output Parallel channel to monitor SQUID DC output
independent of carrier amplitude (flag flux jumping)independent of carrier amplitude (flag flux jumping) 14 Bit ADC’s (4 per 16 channels)14 Bit ADC’s (4 per 16 channels)
oversampling to obtain dynamic rangeoversampling to obtain dynamic range one sample-and-hold per channel one sample-and-hold per channel
Opto-isolated RS485 to PC for ControlOpto-isolated RS485 to PC for Control SQUID Controller commands passed throughSQUID Controller commands passed through
Opto-isolated LVDS to Opto-isolated LVDS to Custom digital PCI I/O board in PC (FNAL design)Custom digital PCI I/O board in PC (FNAL design) data push at 7 Mbits /s data push at 7 Mbits /s
(about 1000 channels x 2.5 kHz = 6 Mbits/s)(about 1000 channels x 2.5 kHz = 6 Mbits/s) Power ~80W per board.Power ~80W per board.
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 13
Warm Cabling and Warm Cabling and MechanicsMechanics 9U Osc/Demod boards housed in VME 64x Crates with 9U Osc/Demod boards housed in VME 64x Crates with
custom backplanescustom backplanes 20 boards (16 ch ea.) / crate, 3 crates20 boards (16 ch ea.) / crate, 3 crates 1 clock distribution board per crate 1 clock distribution board per crate
Warm CablingWarm Cabling commercial 10’ SCSI differential cables commercial 10’ SCSI differential cables (Cryostat(Cryostat Readout boards), twisted pairs in common shield Readout boards), twisted pairs in common shield LVDS twisted pair to PCLVDS twisted pair to PC
Power:Power: 5 kW for 60 boards = 920 Channels5 kW for 60 boards = 920 Channels commercial 250 kHz switching power supplies, 1% Vcommercial 250 kHz switching power supplies, 1% Vp-pp-p Ripple Ripple
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 14
8ch SQUID 8ch SQUID ControllerController
8 channel SQUID controller8 channel SQUID controller• shunt feedbackshunt feedback• switch open loop/closed loop w. switchable switch open loop/closed loop w. switchable gain, gain, • on board FPGA / DAC for programming on board FPGA / DAC for programming SQUIDSQUID• mates directly to cryostatmates directly to cryostat• analog data out to demodulator boards.analog data out to demodulator boards.
Min
RFB
104VΦ
enoiseam
p
Lin
Signal IN
100
100
10
MOSFET
Cold Warm
Not Shown:• DC Squid bias• DC Flux bias• DC Amp Voltage Offset• SQUID Voltage measure lead
19mm pitch
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 15
SQUID pre-amplifersSQUID pre-amplifers Measure current in SC loop of LCRMeasure current in SC loop of LCRbolobolo
100 element Series array SQUIDs from 100 element Series array SQUIDs from NISTNIST AC biased AC biased 1/f noise unimportant 1/f noise unimportant Squid Noise: 0.1 μSquid Noise: 0.1 μΦΦ00/√Hz ≈ 2 pA/√Hz, /√Hz ≈ 2 pA/√Hz,
adequateadequate Expect to receive first two wafers from Expect to receive first two wafers from
NIST next week (destined for APEX-SZ)NIST next week (destined for APEX-SZ) Quality control and testing at Berkeley by Quality control and testing at Berkeley by
Dennis Seitz (experienced with NIST Dennis Seitz (experienced with NIST arrays for CDMS)arrays for CDMS)
SQUID Transimpedance
0
50
100
150
200
250
300
350
400
-0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0
Flux Bias (Phi0)
Ztr
(O
hm
s) ZTrans= 400 Ω
Flux BiasPUT N
IST DRAW
ING H
ERE.
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 16
SQUID Magnetic SQUID Magnetic ShieldingShielding
!" # $ %"" & '&
( "" ) %$ $ *+,-. # ' # )
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 17
SQUID Magnetic SQUID Magnetic ShieldingShielding
Two part shieldingTwo part shielding 0.056’ Cryoperm chimney style shield0.056’ Cryoperm chimney style shield
(attenuates B, by few 10(attenuates B, by few 1033
Nb film Nb film (pins B field lines, by >15)(pins B field lines, by >15)
Measured performance (prototype)Measured performance (prototype) DC attenuation of few 10DC attenuation of few 1055
improves to >10improves to >1066 for AC B fields for AC B fields Production Drawings near-readyProduction Drawings near-ready Finalizing Mechanical support/connetorsFinalizing Mechanical support/connetors
CalculatedAttenuationfrom Nb Foil
(Martin Lueker)
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 19
Excess Loading
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
400 500 600 700 800 900 1000 1100 1200
freq (kHz)
loa
din
g (
oh
ms
)
Excess Loading
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0 0.2 0.4 0.6 0.8 1 1.2
dielectric area (mm2)
loa
din
g (
oh
ms
) NbNb22OO55 capacitors capacitors
lossy, spoils lossy, spoils voltage bias.voltage bias.
Solution: use Solution: use external NP0 external NP0 ceramic chip caps.ceramic chip caps.
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 20
MUX LC ChipMUX LC Chip
2nd Generation MUX ChipNew Mask for MUX chip, inductors only
1/3 Q
Fabricated byA. Smith, Northrop Grumman Space Tech
1st Generation MUX ChipIntegrated LC’s. 8 channels
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 21
Excess Loading
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
400 500 600 700 800 900 1000 1100 1200
freq (kHz)
loa
din
g (
oh
ms
)
Measured Loading (L only with surface mount)
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0 200 400 600 800 1000 1200
freq (kHz)
load
ing
(o
hm
)
OLD LC Chip
New L chip, with Surf mount caps
Loss in LCLoss in LC
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 22
100000 1000000
1E-6
1E-5
1E-4BK2 L chip Inductive Coupling
k=0.01
k=0.009
k=0.008k=0.007
k=0.006neighbour
next nearest neighbour
co
up
led
cu
rre
nt (
A)
frequency (Hz)
LC Chip Cross TalkLC Chip Cross Talk
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 23
0.01 0.1 1 10 100 1000
1
10
100
1000
Demodulated Noise of Sensor J2 (sensor normal, T=0.3K)
expected level
demodulator floor
sensor
cu
rre
nt n
ois
e (
pA
/Hz1/
2 )
freq (Hz)
NOISE OF SENSOR JUST ABOVE ITS TRANSITION (HIGH VOLTAGE BIAS) WITH CMB ELECTRONICS (fbias=1.5 MHz)
100 Hz Sky signal
1/f dominated byactive filter in demodulator fixed in new Tayloe mixer
measured noise as calculated
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 24
0.1 1 10 100
1
10
100
1000
10000
Demodulated Noise of Sensor J2 (T=0.3K)
detector normal in transition in transition in transition in transition demodulator noise floor
curr
en
t no
ise
(p
A/H
z1/2 )
freq (Hz)
NOISE OF SENSOR IN TRANSITION WITH 100Hz SIMULATED SKY SIGNAL WITH CMB ELECTRONICS (fbias = 1.5 MHz)
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 25
Readout ChallengesReadout Challenges dynamic range: >80 dB for single bolometerdynamic range: >80 dB for single bolometer high Q LC resonatorhigh Q LC resonator
solved- custom Nb inductors + NP0 chip capacitorssolved- custom Nb inductors + NP0 chip capacitors intermodulation distortionintermodulation distortion
new Tayloe Mixer (35 dBm IP3)new Tayloe Mixer (35 dBm IP3) requires (passive) nulling of bolometer carriers at SQUID inputrequires (passive) nulling of bolometer carriers at SQUID input
low frequency responselow frequency response measured DDS stability, good to 10 mHzmeasured DDS stability, good to 10 mHz low noise Tayloe Mixer low noise Tayloe Mixer
cold SQUID + warm amplifier feedback loopcold SQUID + warm amplifier feedback loop requires short cold-warm wiring lengthsrequires short cold-warm wiring lengths
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 26
Intermodulation Distortion Intermodulation Distortion and Nullingand Nulling
Carrier inter-modulation products Carrier inter-modulation products troublesome.troublesome.
Can avoid them or NULL the Can avoid them or NULL the carriers to minimize their effectcarriers to minimize their effect
Carrier nulling at 60 dB Carrier nulling at 60 dB obtainable, accurate to 10 mHz. obtainable, accurate to 10 mHz.
New TAYLOE Mixer on New TAYLOE Mixer on demodulator board has excellent demodulator board has excellent IP3=35 dBm.IP3=35 dBm.
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 27
ColdColdWarm Wiring Warm Wiring LengthsLengths
SQUID locked in feedback loop w/ warm ampSQUID locked in feedback loop w/ warm amp Loop gain determines dynamic range.Loop gain determines dynamic range.
original goal: Aoriginal goal: ALOOPLOOP=100 at 1 MHz=100 at 1 MHz
AALOOPLOOP=1 at 100 MHz=1 at 100 MHz
need to maintain phase margin at freq. where Aneed to maintain phase margin at freq. where ALOOPLOOP>1 >1 limits wiring length between cold and warmlimits wiring length between cold and warm depends on max bias freq. x loop gaindepends on max bias freq. x loop gain Example: AExample: ALOOPLOOP=100 at 1 MHz =100 at 1 MHz
round trip wiring length <20 cm for 45 deg phase marginround trip wiring length <20 cm for 45 deg phase margin SQUID arrays relax dynamic range requirementsSQUID arrays relax dynamic range requirements Loop gain requirement set by SQUID non-linearity (intermodulation products) Loop gain requirement set by SQUID non-linearity (intermodulation products)
-- t.b.d. for NIST arrays-- t.b.d. for NIST arrays achieved wire length is 2x 15 cmachieved wire length is 2x 15 cm
can achieve Acan achieve ALOOP LOOP = 50 with some margin= 50 with some margin carrier nulling reduces max. signal -- essentialcarrier nulling reduces max. signal -- essential
COLD | WARM
I1Rfb
SQUID ControllerSeries Array SQUIDs
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 28
SPT Readout StatusSPT Readout Status SQUIDsSQUIDs
sample 8-turn NIST arrays tested with our systemsample 8-turn NIST arrays tested with our system first production SQUIDs arriving at Berkeley this monthfirst production SQUIDs arriving at Berkeley this month
SQUID ShieldsSQUID Shields prototypes fabricated at Amunealprototypes fabricated at Amuneal attenuation measured – adequateattenuation measured – adequate designing production cryoperm shields nowdesigning production cryoperm shields now
Cold wiring harnessesCold wiring harnesses prototypes ordered from TechDataprototypes ordered from TechData
SQUID ControllersSQUID Controllers 4-channel analog and 8-channel digitally controlled prototypes tested 4-channel analog and 8-channel digitally controlled prototypes tested
(testing of 8-ch board continues)(testing of 8-ch board continues) Layout revisions of production boards begun Layout revisions of production boards begun
Warm cablingWarm cabling off the shelf SCSI cables, samples procuredoff the shelf SCSI cables, samples procured
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 29
SPT Readout StatusSPT Readout Status Oscillator/Demodulator Readout BoardsOscillator/Demodulator Readout Boards
Prototype single channel board fabricated and testedPrototype single channel board fabricated and tested Tayloe Mixer tested as add-on boardTayloe Mixer tested as add-on board
Prototype 16 channel board fabricatedPrototype 16 channel board fabricated functionality verified, need to measure performance, test with systemfunctionality verified, need to measure performance, test with system
VME Crates selected (Wiener), off-the-shelf power supplies selectedVME Crates selected (Wiener), off-the-shelf power supplies selected Custom backplanes: power plane finished, signal/clock plane underway.Custom backplanes: power plane finished, signal/clock plane underway.
FirmwareFirmware slave board firmware written/testedslave board firmware written/tested SQUID Controller / Master board firmware will start soonSQUID Controller / Master board firmware will start soon
Clock Distribution BoardClock Distribution Board prototype fabricated and testedprototype fabricated and tested 9U prototype in the pipeline (very simple board)9U prototype in the pipeline (very simple board)
Digital IO BoardDigital IO Board ““borrowed” Custom IO boards from Fermilab, 2 boards in handborrowed” Custom IO boards from Fermilab, 2 boards in hand
Readout SoftwareReadout Software OO software in C++ on Linux OS, interfaced with experiment Software via TCP/IPOO software in C++ on Linux OS, interfaced with experiment Software via TCP/IP High Level Design Sketches/BrainstormingHigh Level Design Sketches/Brainstorming Many low level Hardware interfaces and tuning algorithms written and in use.Many low level Hardware interfaces and tuning algorithms written and in use.
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 30
Readout Status – SystemReadout Status – System individual elements of readout system individual elements of readout system
prototyped and testedprototyped and tested early prototype integrated readout early prototype integrated readout
system tested with bolometersystem tested with bolometer analog SQUID controller, single channel analog SQUID controller, single channel
demodulator…demodulator… noise properties and two channel noise properties and two channel
fMUX testedfMUX tested commissioning 8 channel system with commissioning 8 channel system with
bolometers nowbolometers now testing of “production ready” testing of “production ready”
integrated system forthcoming integrated system forthcoming (this spring/early summer)(this spring/early summer)
Physics Division
The South Pole The South Pole TelescopeTelescope
Readout System Readout System SoftwareSoftwareMarch 4, 2004March 4, 2004
SPT EAB Meeting, U. ChicagoSPT EAB Meeting, U. Chicago
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 32
Framework / Framework / InfrastructureInfrastructure
Programming in C++ on Linux OSProgramming in C++ on Linux OS Communication w/ Outside world via TCP/IP only.Communication w/ Outside world via TCP/IP only. Dedicated PC for Data Acquisition, receives digital data Dedicated PC for Data Acquisition, receives digital data
from readout boards, and makes it available on sockets for from readout boards, and makes it available on sockets for SPT control PC to archive (rates up to 100-400 Hz, TBD)SPT control PC to archive (rates up to 100-400 Hz, TBD) also supports streaming data directly to disk at rates up to 2.5 also supports streaming data directly to disk at rates up to 2.5
kHz, for debugging, etc.kHz, for debugging, etc. Readout Tuning and Monitoring PC Readout Tuning and Monitoring PC
algorithms to setup and tune SQUIDs/Bolometersalgorithms to setup and tune SQUIDs/Bolometers monitors noise and DC levels (flux jumping) in all channelsmonitors noise and DC levels (flux jumping) in all channels
knows algorithms to repair channelsknows algorithms to repair channels
provides quality control data on sockets for SPT archiveprovides quality control data on sockets for SPT archive
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 33
Sampling Rate of Bolo Sampling Rate of Bolo TimestreamTimestream
sampling pulse can be generated externally or internallysampling pulse can be generated externally or internally the maximum bandwidth is determined by the bolometer time the maximum bandwidth is determined by the bolometer time
constantconstant(hardware allows us to reduce bandwidth below 1 kHz)(hardware allows us to reduce bandwidth below 1 kHz) Bolometer time constant ~ 1 ms Bolometer time constant ~ 1 ms (not well known yet!)(not well known yet!) bandwidth 1/(2πτ) = 160 Hzbandwidth 1/(2πτ) = 160 Hz
by low pass filtering the bolometer timestream at 160 Hz, and sampling at the by low pass filtering the bolometer timestream at 160 Hz, and sampling at the Nyquist frequency 400 Hz, all the timestream information is encoded Nyquist frequency 400 Hz, all the timestream information is encoded
we sample with a 14 bit ADC, providing a LSB accuracy of ~10we sample with a 14 bit ADC, providing a LSB accuracy of ~10 -4-4.. Why use the full bandwidth when the sky signal will be modulated Why use the full bandwidth when the sky signal will be modulated
at slower rates?at slower rates? we can gain further accuracy by over-samplingwe can gain further accuracy by over-sampling Sampling beyond bolometer 3 dB bandwidth allows full reconstruction of Sampling beyond bolometer 3 dB bandwidth allows full reconstruction of
signals within MUX bandwidth for diagnostics. signals within MUX bandwidth for diagnostics. Data storage rate can be reduced by real time averaging during measurementData storage rate can be reduced by real time averaging during measurement
can be done in FPGA on demodulator boardcan be done in FPGA on demodulator board can keep high sample rate and reduce readout rate.can keep high sample rate and reduce readout rate.
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 36
Data Flow CartoonData Flow Cartoon
Oscillator/Demodulator Boards(optically isolated from PC / GPS)
DAQ Computer
Control and Monitoring Computer(might be same PC)
read/write
contro
ls
& time st
amp
Data pushed at few MBytes / s
subset of data foronline monitoring &SQUID / Bolo tuning
receiver housekeeping data
RS 485 Serial
To SPT Archive PC
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 37
Data RateData Rate Between readout boards and PCBetween readout boards and PC
size of frame for one sample: size of frame for one sample: 2.9 kBytes2.9 kBytes (of which about 40% is header information– see cartoon of frame on next slide)(of which about 40% is header information– see cartoon of frame on next slide)
400 Hz sampling rate 400 Hz sampling rate 1.1 MBytes / second 1.1 MBytes / second 2.5 kHz sampling rate 2.5 kHz sampling rate 7.1 MBytes / second 7.1 MBytes / second
Data rate written to disk (bolo data only)Data rate written to disk (bolo data only) 400 Hz 400 Hz 0.74 MBytes/s 0.74 MBytes/s (62 GBytes / 24 hrs)(62 GBytes / 24 hrs)
100 Hz 100 Hz 0.18 MBytes /s 0.18 MBytes /s (15.5 GBytes / 24 hrs)(15.5 GBytes / 24 hrs)
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 38
Size of Data Frame: 2.9 Size of Data Frame: 2.9 kByteskBytes
Data
Fra
me f
rom
O
sc/D
em
od
ula
tor
Data
fro
m o
ne
Cra
te1 Byte CRATE ID9 Bytes GPS Time
1 Byte BOARD ID
1 Byte CHANNEL ID2 Bytes 14 bit data sample
Data
fro
m o
ne
Osc/D
em
od
B
oard
3 cra
tes
16 chan
nels/b
oard
20 boar
ds/cr
ate
Matt Dobbs & Helmuth Spieler, SPT Readout System Mar 4, 2004 EAB 39
Block Diagram: Bolo Data Block Diagram: Bolo Data FlowFlow
SQUID Controller
8 MUX Modules / board
16 DDS Oscillatorsfor Carriers
16 DDS Oscillatorsfor Nulling
16 Demodulators ADC’s
FPGA
Op
to-
Isola
tors
Osc/Demodulator Boards
Digital IO BoardWith FPGA
2 MByte Buffer
Persistent Storageon Hard Disk& Ultrim Tape
Data Acquisition PC
PCI Interface
PC MemorySoftware
ProcessingDataReduction
Bolometer time
stream1 MHz BW
Comb of Carriers
Comb for Nulling
Bolometer time
stream1 kHz BW
Data Cable: Diff. Twist
Pair≤ few
MBytes/s
time interval of encoded data need only be larger than
signal bandwidth,~ 100 Hz.≈0.2 MB/s
we are free to average some number of data samples here, while still maintaining benefits of oversampling.
60 x 16ch Boards
in 3 9u Crates
15 x (8 Mod x 8ch) Boards
RS 485 IODifferential & Isolated
Control (and Monitoring?) PC
PC MemorySoftware
Processing
External SPT Sampling Clock
Control & Bolo / SQUID Tuning
Provides GPSTimestamp
Control
Subset of Data for
Monitoring
Readout Hardware accommodatesboth of these timing Possibilities.
Comment
Flow of Timing Info
Flow of Control Commands
Flow of Bolo Data
Legend
Housekeeping Data