ARDA: Technology for Distributed Analysis http://cern.ch/arda - - PowerPoint PPT Presentation

▶
arda technology for distributed analysis
SMART_READER_LITE
LIVE PREVIEW

ARDA: Technology for Distributed Analysis http://cern.ch/arda - - PowerPoint PPT Presentation

Oct 15, 2023 373 likes •701 views

ISGC 2004 Taipei ARDA: Technology for Distributed Analysis http://cern.ch/arda Jakub Moscicki ARDA Project Jakub.Moscicki@cern.ch www.eu-egee.org cern.ch/lcg EGEE is a project funded by the European Union under contract IST-2003-508833

slide-1
SLIDE 1

ISGC 2004 Taipei

ARDA: Technology for Distributed Analysis

Jakub Moscicki

ARDA Project Jakub.Moscicki@cern.ch

http://cern.ch/arda www.eu-egee.org cern.ch/lcg

EGEE is a project funded by the European Union under contract IST-2003-508833

slide-2
SLIDE 2

ISGC 2004 Taipei- 2

Contents

  • Context of ARDA Project
  • Technology Discussions and ARDA Prototypes

Service Interaction and Architecture Data Management Files on the GRID Metadata Catalogs GRID Services and Databases Interactivity on the GRID Connectivity & Protocols

  • Outlook
slide-3
SLIDE 3

ISGC 2004 Taipei- 3

Evolution of ARDA

  • November 2003: ARDA RTAG Report
  • ARDA Blueprint = Architectural Roadmap for Distributed Analysis
  • Set of collaborating Grid services and their interfaces
  • January 2004: ARDA Workshop 1
  • ARDA Project = A Realisation of Distributed Analysis
  • Coordination and early integration between Generic Middleware (EGEE) and

LHC Experiments' Software

  • May 2004: EGEE protype
  • Glite = new-generation generic middleware
  • Very first prototype available internally to ARDA group
  • June 2004: ARDA Workshop 2
  • Hands-on meeting: “First 30 days of ARDA prototype”
  • Fall 2004: ARDA Workshop 3
  • Summary of the first phase of ARDA prototype
slide-4
SLIDE 4

ISGC 2004 Taipei- 4

Software Activities in ARDA

ARDA Glite / EGEE Atlas LHCb CMS Alice

ARDA-Alice ARDA-Atlas ARDA-CMS ARDA-LHCb

Regional Centers

Taiwan, Russia

Bio Science LCG

POOL, SEAL, ROOT

ARDA RTAG ARDA RTAG ARDA RTAG

slide-5
SLIDE 5

ISGC 2004 Taipei- 5

EGEE Middleware vs ARDA

  • Interfacing middleware to the experiment frameworks
  • Early deployment of (a series of) end-to-end prototypes to ensure

functionality and coherence

  • Middleware as a building block
  • Validation of the design
  • Feedback, discussions
  • Identification of common/generic components
  • New service decomposition
  • Strong influence of Alien system
  • the Grid system developed by the ALICE experiments and used by a wide

scientific community (not only HEP)

  • Role of experience, existing technology…
  • Web service framework

ARDA Prototype(s) EGEE Middleware

slide-6
SLIDE 6

ISGC 2004 Taipei- 6

GRID Service Framework

slide-7
SLIDE 7

ISGC 2004 Taipei- 7

Service Interaction

  • WSDL vs Client API

Regular Services (Metadata Catalog,....)

  • API expected by Clients may be not implementable in a performant way

if mapped one-to-one to the WSDL

– e.g. paging of query results

  • may require a client side library (stubs)

– langauge bindings, platform compatibility,...

“Interoperability Service”

  • Require dynamic interface discovery and composition: WSDL

Service

WSDL

Client Client

API

Service

WSDL

slide-8
SLIDE 8

ISGC 2004 Taipei- 8

Grid Access Service

  • Grid Access Service

Client Proxy

  • All client operations are channeled via a user-based GAS instance
  • GAS instance exports the interface based on user's certificate

– authorisation to access different services

Entry Point (Bootstraping)

  • Resolves references to other services and gives back the pointers

client File Catalog Metadata Replica Workload

GAS

slide-9
SLIDE 9

ISGC 2004 Taipei- 9

Grid Access Library: ARDA-Alice

  • Client Proxy : Grid Access Library
  • Service Factory (OGSA) vs Pool of Services

libgliteUI *Ssl-poolserver

ROOT gshell

gLite/AliEn Services

SHM-Bus session Authentication

gLite Authen. (GSI, PAM)

libgliteIO

Iod

File Access Services (via catalogue)

T

  • M

S S I/O UI

C2PERL gLite.pl *Mtpoolserver C2PERL gLite.pl *Mtpoolserver C2PERL gLite.pl *Mtpoolserver C2PERL gLite.pl Embedded Perl module

Shell commands

slide-10
SLIDE 10

ISGC 2004 Taipei- 10

Modularity & Interfaces

  • Architectural Issues

ability to replace implementation of components

  • No backdoor interfaces
  • Integration with middleware developed by experiments

Service

Interface

Client Service

Interface

X

slide-11
SLIDE 11

ISGC 2004 Taipei- 11

Files and Metadata

slide-12
SLIDE 12

ISGC 2004 Taipei- 12

Files on the GRID

  • Files on the GRID

Categories of files

  • 90% of files are WORM
  • Read/Write Files:

– File GUID lifecycle and management – Versioning

Role of LFN:

  • LFN is mutable and may point to different GUIDs over time
  • Files without LFNs?
  • High-level, VO specific optimization based on LFN?

Persistent references

  • To other files (via GUID, LFN)
  • To objects inside files
slide-13
SLIDE 13

ISGC 2004 Taipei- 13

Metadata

  • Metadata Flavours

Filesystem Metadata: property of File Catalog Physics Metadata: in a separate database

  • Physics Metadata

Many parallel developments in LHC experiments

  • AMI, RefDB, BKDB,...
  • Glite should (?) provide mechanisms to plug in these components

Generic metadata service

  • “Annotate any objects with key-value lists of metadata”

– any objects = datasets, applications, users,...

  • Isn't it purely a database problem?
slide-14
SLIDE 14

ISGC 2004 Taipei- 14

Data Management

slide-15
SLIDE 15

ISGC 2004 Taipei- 15

File Transfer

  • Data management transfer service needed

Reliable file transfer Misuse protection Example: TMDB (CMS)

RefDB McRunjob

T0 worker nodes GDB castor pool

Tapes Export Buffers Transfer agent RLS TMDB

Reconstruct ion instructions Reconstruction jobs Reconstructed data Reconstructed data Checks what has arrived Updates Updates Summaries of successful jobs

CMS DC04 production

slide-16
SLIDE 16

ISGC 2004 Taipei- 16

GridFTP

  • GridFTP (GT 3.3)
  • 1GB in 5-10 s : for files > 100MB

– 2x PIII, 2.40GHz, 1GB RAM

slide-17
SLIDE 17

ISGC 2004 Taipei- 17

Protocol Stack: SOAP

  • Control layer vs Transport Layer

Example: RFT (Reliable File Transfer, GT3)

Service

Interface

Service

Interface

SOAP GridFTP

slide-18
SLIDE 18

ISGC 2004 Taipei- 18

SOAP Encoding

  • SOAP problems

encoding of binary data and complex structures: overhead 10x floating point representation: parser dependant SOAP/https encryption: overhead 30%

  • Example:

Grid Access Library (ARDA-Alice)

  • ~1500 (plain) / 800 (encrypted) call per second

grid command encoding+encryption

SOAP Body

Session ID Session Crypto(t)

SOAP Body

Sym.Cipher SSLEncryption

SOAP String Call XY Arg1 = '...'; Arg2 = '...'; Arg3 = '...';

C O D E C / S e r i a l i z e r UU Encoding

UU

Session ID Session Crypto Session ID Session Crypto

slide-19
SLIDE 19

ISGC 2004 Taipei- 19

Database Access

slide-20
SLIDE 20

ISGC 2004 Taipei- 20

DB / SOAP: AMI

  • AMI : Atlas File Metadata Catalogue:
  • Simulation/Reconstruction-Software Version
  • Does not contain physical filenames
  • Implementation Features:
  • dynamic schema versioning
  • Java/AXIS Web Service Framework + MySQL Backend

SOAP-Proxy

Meta-Data (MySQL) User User User

slide-21
SLIDE 21

ISGC 2004 Taipei- 21

AMI Performance

10 20 30 40 50 60 20 40 60 80 100 120 140 Time to completion Number of rows selected

100 40 150 50 30 Clients 20 10 5 1

Clients per Second

100 150 Rows 50 20 10 5

0.1 1 10 100 20 40 60 80 100 120 140

AMI behaviour using many concurrent clients:

  • Large network traffic overhead due to schema independent tables
  • SOAP Web Services proxy supposed to provide DB access

– Note that Web Services are “stateless” (not automatic handles to have the concept of session, transaction, etc…): 1 query = 1 (full) response

  • Large queries might crash server
  • Shall SOAP front-end proxy re-implement all the database functionality?
slide-22
SLIDE 22

ISGC 2004 Taipei- 22

DB / PHP: RefDB

  • RefDB

CMS Metadata and Bookkeeping Database

  • Implementation Features:

MySQL backend PHP script frontend

  • User query encoded in URL
  • DB query logic in the PHP scripts

Result handling:

  • small queries: XML over HTTP
  • large queries: cached in a file to be fetched again

PHP proxy

Meta-Data (MySQL) User User User

slide-23
SLIDE 23

ISGC 2004 Taipei- 23

RefDB Performance

slide-24
SLIDE 24

ISGC 2004 Taipei- 24

DB / XML RPC : BKDB

  • LHCb Bookkeeping DB
  • Implementation Features:

Oracle backend XML-RPC frontend

  • Python client for tests
slide-25
SLIDE 25

ISGC 2004 Taipei- 25

CERN/Taiwan tests on LHCb metadata catalogue

5 10 15 20 25 30 50 100 150 200 250 Number of rows selected Time to completion(sec)

1 Client 5 Clients 10 Clients 20 Clients 30 Clients 40 Clients 50 Clients 100 Clients

10 20 30 40 50 50 100 150 200 250 300 350 400 450 500 550 The Number of Clients Response Time(s)

XML-RPC access

Unoptimized SQL access

200 rows = 180KB

slide-26
SLIDE 26

ISGC 2004 Taipei- 26

Database Access Summary

  • Service fronted to a DB

problems:

  • reproducing DB capabilities is very hard

– ACID – transactions – timeouts – large queries

  • additional conversion layer: significant performance drop

benefits

  • high-level query support: client is abstracted from the internal DB
  • ptimization techniques

– simple user query may map to a complex ‘optimized’ internal query

slide-27
SLIDE 27

ISGC 2004 Taipei- 27

Connectivity and Interactivity

slide-28
SLIDE 28

ISGC 2004 Taipei- 28

Connectivity

  • Asynchronous Messaging

SOAP over Jabber (IM)

  • “Agent Model”

Outbound Connectivity Required

  • direct/indirect

Examples

  • DIRAC, PROOF

Security models

  • Open ports for trusted inbound traffic
  • Parachute trusted agent which will initiate outbound connection
  • Connection Managament

SOAP over http is connectionless

slide-29
SLIDE 29

ISGC 2004 Taipei- 29

Connectivity: DIRAC

Application packager Workflow editor Production editor

Production manager

Production DB Production preparation

Edit Instantiate workflow Create application tar file

Central Services

Monitoring DB Bookkeeping DB

Production resources Agent A

Site A

Agent n

Site n

…

Job XML Job status Meta XML

Castor MSS CERN

Dataset replica

Agent B

Site B

Central Storage

Job request

Bookkeeping Service Monitoring Service Production Service

slide-30
SLIDE 30

ISGC 2004 Taipei- 30

Connectivity: PROOF

USER SESSION USER SESSION

PROOF PROOF SLAVES

SLAVES

TcpRouter

PROOF PROOF

PROOF PROOF SLAVES

SLAVES

PROOF MASTER PROOF MASTER SERVER

SERVER

PROOF PROOF SLAVES

SLAVES

TcpRouter TcpRouter TcpRouter

Site A Site C Site B

no support for hierarchical Grid infrastructure, only local cluster mode.

slide-31
SLIDE 31

ISGC 2004 Taipei- 31

Summary

  • ARDA provides a discussion forum

active feedback from beta-testers and users discussion plaform for development teams technology & ideas exchange medium

  • ARDA prototypes and tests

building technology knowledge-base sharing ideas and implementations Towards usable end-to-end prototype of Distributed Analysis ARDA

slide-32
SLIDE 32

ISGC 2004 Taipei- 32

http://cern.ch/arda