DSP at SAO and a Study of Next Generation ALMA Digital Processing - - PowerPoint PPT Presentation

DSP at SAO and a Study of Next Generation ALMA Digital Processing

Andr´ e Young

Harvard-Smithsonian Center for Astrophysics ALMA Future Science Development Program Workshop Charlottesville, VA 24 August 2016

Overview

This talk is about a study of next generation ALMA digital processing...

Credit: Jonathan Weintroub Credit: EFE/Ariel Marinkovic

...but we begin the recent correlator upgrade at the SMA:

high-altitude site, challenging environment to keep equipment cool power is expensive, reduce consumption as far possible keep up with rapidly evolving technical landscape, while still supporting / running legacy systems in parallel

2

The Submillimeter Array Correlator Upgrade

Original ASIC correlator built on 90s tech

MIT Haystack MarkIV correlator board, 32 XF-corr ASICs

90 correlator boards: Process up to 2 GHz per Rx 2 GHz IF split into six 328 MHz blocks, each block into four 82 MHz chunks 812.5 kHz finest uniform spectral resolution without loss of bandwidth (various modes to allow finer resolution non-uniformly, or by trading bandwidth)

3

The Submillimeter Array Correlator Upgrade

SWARM: SMA Wideband Astronomical ROACH2 Machine

ROACH2, built around 1 Xilinx Virtex-6 FPGA

8x 1U rack components (1 quadrant): Process 2 GHz per Rx full usable 2 GHz in single baseband 2 channels per ROACH2 (2x 5Gsps ADC) 140 kHz spectral resolution, uniform across the entire band full Stokes without loss of resolution

4

The Submillimeter Array Correlator Upgrade

Forest of lines in Orion BN/KL, two quadrants of SWARM. Data from 14 Aug 2016. (Primiani+ 2016, in prep.) 5

The Submillimeter Array Correlator Upgrade

SMA correlator room digital racks (left) and analog racks (right)

ASIC correlator essentially replaced with single SWARM quadrant

digital equipment space reduced by a factor of 6 wideband digital system greatly reduces analog system 1x 2 GHz SWARM baseband = 24x 82 MHz ASIC basebands ⇒ analog equipment space reduced by factor ∼ 8 large reduction in power consumption from 25 kW (excl cooling) to 2 kW (incl cooling)

Full SWARM system, 8 GHz per Rx expected around end 2016

6

SWARM: FPGA Logic

SWARM built on the CASPER packetized correlator architecture, within each ROACH2

F-engine, 16384 channels per Rx from single antenna X-engine, 2048 channels per Rx for all baselines 10GbE crossbar switch for F-to-X data distribution

7

SWARM: FPGA Logic

Very high utilization of FPGA resources

full rate system clocked at 286 MHz, staged in 11ths to meet timing closure: 6/11, 8/11, 10/11 (compatibility with legacy clocking) currently at 11/11 rate, with timing score of ∼3000 using -2 speedgrade parts (picoseconds of negative slack required to route signals at given clock rate, operating conditions dependent, helps to run systems as cool as possible)

Xilinx Virtex-6 (XC6VSX475T) Occupied slices 97% Slice registers 44% Slice LUTs 82% DSP48Es 44% BRAM 70% IOBs 89%

8

SWARM: Built-in Phased Array & VLBI Capability

Combine connected-element array into equivalent larger single dish

in-situ phase calibration to correct time- and direction-dependent atmospheric delay signal-/noise-space decomposition of the correlation matrix (largest eigenvalue → eigenvector is the phasing solution)

Phasing feedback loop in SWARM Phasing efficiency during EHT observation, 4 April 2016 9

Next Generation ALMA Digital Processing Study

Present ALMA correlator technology is ∼10 years old. What ALMA digital processing system can be built with an upgrade to current / near-future technology?

ngALMA correlator & phased array kick-off meeting, May 2016 @ SAO LTR: J. Weintroub, M. Rupen, R. Lacasse, B. Carlson, M. Hecht, J. Hickish,

• R. Wilson, G. Crew, C. Langley, S. Doeleman, AY, L. Blackburn, A. Baudry,
• A. Saez, S. Ashton, R. Escoffier, L. Greenhill.

+ R. Primiani, D. Herrera, J. Test, K. Young, L. Matthews.

Smithsonian Astrophysical Observatory Joint ALMA Observatory Universit´ e de Bordeaux National Radio Astronomy Observatory National Research Council Canada University of California Berkeley MIT Haystack Observatory

10

ngALMA Study: Project Overview and Timeline

One-year study

Monthly teleconferences Monthly progress reports Bookend in-person meetings (May ’16 @ SAO, Feb ’17 @ NRCC) Mid-term two ‘busy weeks’ @ SAO (11/16 visit A. Saez & D. Herrera) Start-of-study Kick-off meeting Science requirements System architecture & breakdown Subsystems analyses Integration Concluding meeting Final reports & project end

04/16 05/16 06/16 07/16 08/16 09/16 10/16 11/16 12/16 01/17 02/17 03/17 04/17 11

ngALMA Study: Baseline Science Requirements

Identify requirements to enhance existing / enable new science Target requirements compared to existing system:

Proposed Existing* 1 Number of antennas 72–80 64 2 Maximum baseline 300 km 30 km 3 Total bandwidth 32 GHz / pol 8 GHz / pol 4 Baseband channel 12–14 GHz 2 GHz 5 Input / correlator sample resolution 4-bit / 4-bit 3-bit / 2- or 4-bit 6 Finest spectral resolution 1 kHz ≥3.8 kHz 7 Number of channels ≥1e6 / pol 8 Integration / read-out time 1 ms (auto-corr) 1 ms (auto-corr) 16 ms (cross-corr) 16 ms (cross-corr) 9 Independent subbands >16 10 Subarrays 5–7 11 Phased array beams 2–4 12 Built-in VLBI capability

*Escoffier et al 2007

12

ngALMA Study: System Architecture and Work Breakdown

Translate science requirements to system specifications, assuming

receive raw digitized baseband data as input deliver raw correlator / phased array data as output matched up/down-stream analog/digital capabilities will be available FX-architecture favored (over FXF-, XF-)

Project onto technology for start-of-construction by Q2 2022

cost & performance estimates

Break down into subsystems (work packages)

smaller working groups subsystem specifications investigate platforms & architectures

13

ngALMA Study: System Architecture and Work Breakdown

Work packages / subsystems

Science require- ments Define a set of baseline science requirements (two slides ago...) F-engine Identify platform, CPU vs GPU vs FPGA vs ASIC Determine architecture for given platform Corner-turn Identify platform, backplane vs ethernet Consider interface to F-/X-engine X-engine Identify platform, CPU vs GPU vs FPGA vs ASIC Determine architecture for given platform Phased Array Beamformer design and integration VLBI capability Staging Logistics of assembly, running in parallel with existing correlator, etc

14

ngALMA Study: Intermediate Outcomes

Only so much done in a year... ⇒ Identify prototyping study areas ...with limited person-power ⇒ Enlarge the collaboration Possible areas to fast track to project status include:

1 High-speed ADCs and interfacing 26 GSa/s & Virtex-7 port to SKARAB (a.k.a. ROACH3) [UCB] 2 F-/X-engine subsystems implementation Prototyping on latest high-performance tech (Xilinx Ultrascale+, NVIDIA Tesla P100, 100GbE, ASIC) 3 Proof-of-concept wideband pipeline SMA processes 4x 2 GHz, natural testbed for 1x 8 GHz Phased project towards 16 GHz (ngALMA)

Q2 16 Q2 17 Q2 18 Q2 19 Q2 20 Q2 21 Q2 22

ngALMA Prototyping / PoC studies Projected start-of-construction

15

Conclusions

Upgrade to SMA correlator yielded great performance benefits Can these benefits be reproduced for a future ALMA system? ...ngALMA digital processing study to answer that, plus Identify areas for fast-track prototyping Promote these to ‘project’ phase with more team members SMA receivers widebanded to deliver 14 GHz SMA a natural testbed of ngALMA digital systems High-altitude site, prototypes can be fielded and commissioned there for later deployment at ALMA

16