Subrata Chattopadhyay CDAC , Bangalore subratac@cdac.in 25-Oct-13 - - PowerPoint PPT Presentation

▶
subrata chattopadhyay cdac bangalore subratac cdac in 25
SMART_READER_LITE
LIVE PREVIEW

Subrata Chattopadhyay CDAC , Bangalore subratac@cdac.in 25-Oct-13 - - PowerPoint PPT Presentation

Jan 10, 2023 534 likes •1.02k views

Garuda : the unique platform for Innovative research collaboration Subrata Chattopadhyay CDAC , Bangalore subratac@cdac.in 25-Oct-13 Outline Grid & Garuda NGI Recent achievements Applications Services & support

slide-1
SLIDE 1

Garuda : the unique platform for Innovative research collaboration

Subrata Chattopadhyay CDAC , Bangalore subratac@cdac.in

25-Oct-13

slide-2
SLIDE 2

Outline

Grid & Garuda NGI Recent achievements Applications Services & support Conclusion

2

slide-3
SLIDE 3

Grid Computing

GG-BLR GG-CHE GG-HYD TF BLR TF PUNE IITD YUVA PRL

Climate Modeling Disaster Management Bio Informatics CFD Crypt analysis Grid Middleware

3

slide-4
SLIDE 4

Grid Computing

 Grid allows seamless access to information

AND computing resources

 Name derived from the “Power Grid”  You need not know or care where either

comes from

 It means

 Grid allows users to share resources in a secure,

seamless and transparent way.

 Grid also facilitates collaborative environment over

the high speed communication Fabric.

 The Super Computing power is accessible to the

end users without the need of huge investment on

4

slide-5
SLIDE 5

 Sharing of resources among the community  Seen as a collective pool  Heterogeneous  Geographically distributed  Different Administrative domains  Wide variety of Tools, Interfaces to choose with.

Grid Computing

slide-6
SLIDE 6

Components of Grid Middleware

6

slide-7
SLIDE 7

Grid Middleware - Definition

 Middleware is a layer between grid

applications and low level functionality

  • f grid.

 Grid Middleware is a layer of S/W to

enforce:

 Scalability, Transparency, Heterogeneity,

Fault Tolerance and Security of the Grid below the application layers.

 It provides an uniform interface to the

Grid users and handle all the complexity generated due to heterogeneous systems.

7

slide-8
SLIDE 8

8

Popular Middleware

Globus – Globus Alliance GridBus – University of Melbourne UNICORE - Uniform Interface to Computing Resource gLite – CERN / EGEE /EGI Legion – (Avaki - Corporate Distributor) Alchemi – (.NET Grid Computing Framework)

 Condor  SGE

slide-9
SLIDE 9

70 + Partners 10000 CPUs – 550TF EGI, CHAIN- REDS 220TB Storage 1500 + Certificates NKN

25-Oct-13

slide-10
SLIDE 10
  • High Capacity, Highly

Scalable Backbone

  • Provide Quality of Service

(QoS) and Security

  • Wide Geographical Coverage
  • Common Standard Platform
  • Bandwidth from Many NLD’s
  • Highly Reliable & Available by

Design

  • Test beds ( for various

implementation)

  • Dedicated and Owned.

25-Oct-13

slide-11
SLIDE 11

How to join Garuda grid ?

11

slide-12
SLIDE 12

Pre-requisites for Accessing Garuda

Network Connectivity to Garuda

Resources

Valid Grid Certificate MoU or LoE

12

slide-13
SLIDE 13
  • Mr. A
  • 1. CDAC

Bangalore BD 08-06-82

Govt Of India

Certificates can be compared to identity cards

Name Issuer Validity Signature

Valid Till: 01-02-2012

13

slide-14
SLIDE 14

– Indian Grid Certification Authority located at C-DAC, Knowledge Park, Bangalore, India. – IGCA is the accredited member of APGridPMA. – Issues X.509 Certificates to support the secure environment in

  • Grid. (for GARUDA, institutes that do research in grid from India

and foreign institutes that collaborates with GARUDA). – http://ca.garudaindia.in

1500 +

  • Certificates

Issued 338

  • Valid User

Certificates 41

  • Valid Host

Certificates 45

  • Registration

Authorities

25-Oct-13

slide-15
SLIDE 15

– Indian Grid Certification Authority located at C-DAC, Knowledge Park, Bangalore, India. – IGCA is the accredited member of APGridPMA. – Issues X.509 Certificates to support the secure environment in

  • Grid. (for GARUDA, institutes that do research in grid from India

and foreign institutes that collaborates with GARUDA).

IGCA

15

slide-16
SLIDE 16

GARUDA SLCS provides grid users an instant access to GARUDA grid for a trial period of 30days.

Highlights:

  • Hassle free registration
  • Get an access in less than 5mins.
  • Service over the internet.

Features:

  • GARUDA Job submission portal
  • GARUDA Compiler Service

Website: http://labs.garudaindia.in

25-Oct-13

slide-17
SLIDE 17

Public accessible

 CA root certificates  Certificate requests  Valid certificates  Revoked certificates  CRL  CP/CPS  User guides/FAQ’s  RA contact lists.  CA Contact info

http://ca.garudaindia.in

Web Repository

17

slide-18
SLIDE 18

VOMS

VOMS –Virtual Organization Management/Membership Service VOMS is a system to classify users that are part of a Virtual Organization (VO) on the: –Set of attributes that will be granted to them upon request. –include that information inside Globus-compatible proxy certificates 1. In addition to the standard X.509 based Grid certificates, the users should obtain separate VO credentials, from the VO Server.

  • 2. The VO credentials identify the membership of a Grid User to a specific VO.
  • 3. The VO credentials also identify the ‘Role’ of the Grid User in the corresponding

VO.

slide-19
SLIDE 19

VO Registration

http://voms.garudaindia.in/

19

santhosh119

slide-20
SLIDE 20

VO Registration

20

slide-21
SLIDE 21

Virtual User Community in Garuda

Group Name Description

Bioinformatics

application of statistics and computer science to the molecular biology

ClimateModelling

Deals with the dynamics of the climate system.

OSDD

Community dedicated to develop drugs for tropical infectious diseases like malaria, tuberculosis

GeoPhysis

Study related to physics of the Earth and its environment in space

CAE

usage of computer software to solve engineering problems

IndianHeritage

Focused on technology products for preserving & processing Heritage texts

HealthInformatics

Focused on utilizing compute power for health informatics

MaterialScience

interdisciplinary field applying the properties of matter to science and engineering

Euindia

The vision of a worldwide Grid for Research by both Europe and India

ToolsDeveloper

Forum to communicate and collaborate on developing Garuda Tools

21

slide-22
SLIDE 22

VO Registration

22

slide-23
SLIDE 23

Affiliating Mail from Garuda

23

slide-24
SLIDE 24

Programming Development Environment

Computing Resources and Virtual Organizations

Research Organizations Educational institutions Computing Centers

WSRF+GT4 + other Services + Cloud S/W (Nimbus/ VMware)

NKN

Grid PSE

Virtualization support

Workflows Grid Security and High-Performance Grid Networking

Data Grid

Resource Enabler & Monitoring

CDAC Resource centers

Access Portal CLI Visualization Federated Information Server Job Scheduler

Programming Environments

Security Resource Management User Environments Middleware Data Grid Resources

Hand held devices GARUDA – enabled Applications

Non – Research Organizations

Cloud Interface

25-Oct-13

slide-25
SLIDE 25

GSRM Garuda Information Registry Paryavekshanam

25-Oct-13

Garuda Access Portal

slide-26
SLIDE 26

Garuda GridFTP GUI Globus Online AGSG PSP

25-Oct-13

slide-27
SLIDE 27

Megha Scilab VRGeo Galaxy Workflow – OSDD Garuda Garuda User Forum

25-Oct-13

slide-28
SLIDE 28

 CDAC Resource :

  • 4TF HPC clusters each at

Bangalore, Chennai & Hyderabad

  • PARAM Yuva II at Pune and

PARAM Padma at Bangalore  Fourteen of the partner institutions are

also

contributing resources including satellite terminals.  Total computing power is more than 10000 CPUs equivalent to 550TF  Storage space 220 TB

25-Oct-13

slide-29
SLIDE 29

25-Oct-13

  • Serving applications of National Importance

‐

Alliance with the Open Source Drug Discovery (OSDD) project

  • f CSIR

‐

Disaster management applications

‐

Weather forecasting models & Earthquake engineering

‐

Applications from the fields of Bioinformatics, CAE & Material sciences

  • First in India

‐

Setting up of Indian Grid Certification Authority (IGCA) in 2009, to issue digital certificates for grid researchers in India

‐

Digital certificates trusted by other International Certification authorities

‐

Issued more than 1500 IGCA certificates

  • Global Integration

‐

Integrated with the European Grid Infrastructure through the EU-India Grid and CHAIN-REDS

‐

Achieved middleware interoperability between the European Glite middleware & Garuda middleware components

slide-30
SLIDE 30

25-Oct-13

slide-31
SLIDE 31

Bioinformatics : Open Source Drug Discovery

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

NKN Internet / NKN

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

NKN Internet / NKN

OSDD User Community

Grid Enabled Bioinformatics tools useful in drug discovery pipeline

Internet / NKN Internet / NKN

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

NKN Internet / NKN

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

NKN Internet / NKN

OSDD User Community

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

NKN Internet / NKN

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

NKN Internet / NKN

OSDD User Community

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

NKN Internet / NKN

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

OSDD HeadNode

Garuda Middleware Stack, login service, Gridway Metascheduler OSDD Customized Galaxy

DB Ext DB

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

GGHYD Cluster Yuva Cluster JNU Cluster Other OSDD Cluster

GARUDA Grid

Garuda Middleware Stack LRM- Torque OSDD Tools – weka, cdk,…

NKN Internet / NKN

OSDD User Community

Grid Enabled Bioinformatics tools useful in drug discovery pipeline

Internet / NKN Internet / NKN

  • OSDD users given access to

Garuda through OSDD VO

  • HPC clusters to run drug

discovery problems

  • Users connected through both

NKN and Internet

  • Galaxy Workflow for genomics

proteomics applications

  • Distributed job execution through

Gridway Project Team : OSSD community

25-Oct-13

slide-32
SLIDE 32

Climate modeling on Garuda

 Seasonal Forecast Model (SFM) implemented on

GARUDA to do ensemble forecasting

 A framework developed to do ensemble

forecasting using the existing grid middleware services

 Several simulations were done  Use GSRM for data storage  Model resolution: 40 Km x 40 Km  Needs 27 GB of disk space and 80hrs of wall

clock time (If I use 64 processors – Intel Xeon, 3.16 GHZ, 16GB RAM, Infiniband connectivity) for 1 season of 1 year with 1 ensemble member

slide-33
SLIDE 33

Collaborative Class Room

Project Team : CDAC Hyderabad

Supported Features:-

  • Interface to Access

grid

  • GSRM based data

storage for maintaining course repositories

  • Indexing of course

material based on key words

Website: http://ccr.garudaindia.in

25-Oct-13

slide-34
SLIDE 34

Bioinformatics: Taxogrid

Resources

gg-hyd.cdac.org.in gg-che.local gg-blr.tfg xn00.npsf.cdac.ernet.in yuva.pnq-cdac-hq-100.garuda

Software Versions

Globus – 4.0.7 GridWay – 5.4 DRMAA - 1

Dataset

Database: UniProtKB Mycobacterium tuberculosis H37Rv

  • No. of sequences: 3988

Size of input dataset: 1.65 MB Size of output file: 10.4 Method : Maximum Likelihood

TaxoGrid Benchmark (Time Scale) 200 400 600 800 1000 1200 1400 5 10 15 20 Number of cores Time (in minutes)

TaxoGrid Benchmark (Speed-up)

1.00 1.76 2.72 3.31 0.5 1 1.5 2 2.5 3 3.5 5 10 15 20 25
  • No. of pipelines (160 cores)

Speed-up

Time for speedup at value ‘1’ : 1144 minutes Time for speedup at value ‘3.31’ : 346 minutes

Project Team : CDAC Pune

25-Oct-13

slide-35
SLIDE 35

Aim: Optimize the noise generated by a 3-D wing with flaps in landing configuration by variation of flap location and orientation.

CAE: Aeroacoustics Optimization

Project Team : Zeus Numerix

  • Uses Kepler workflow Framework

integrated with native Globus job submission routines

  • Optimization Module uses OPT4J

framework

  • Optimization module includes

AFFG (Adaptive Fuzzy Fitness Granule) routine which can reduce the number of fitness function evaluations up to 50%.

  • Completion sucessful 40

simultaneous simulations (parallel + serial )e Fuzzy Fitness Granule) routine which can reduce the number of fitness function evaluations up to 50%.

  • Completion sucessful 40

25-Oct-13

slide-36
SLIDE 36

25-Oct-13

slide-37
SLIDE 37

New Middleware Stack

New Middleware Stack (v4.0)

released

 QoS Framework  Megha Cloud interface  Reservation framework  Improved services for login, accounting & compilation  Updated components of Globus Toolkit & Gridway

37

slide-38
SLIDE 38

Megha Cloud Interface

Cloud computing platform built on

top of the Garuda resources.

Convergence of Grid and Cloud

technologies

Following apps are available in

the SaaS model

 GROMACS, OpenFOAM, R, Octave, SFM, FASTA

38

slide-39
SLIDE 39

Scilab

 Open source,

cross-platform numerical computational package and a high-level, numerically

  • riented

programming language.

39

  • In collaboration with IITB
  • scilab.in accesses Megha for executing scilab code and

rendering graphics

  • Many textbooks examples are solved and available as

part of text book companion project

slide-40
SLIDE 40

VRGeo

40

Open-Source Collaborative Mapping Platform for Crowd- sourcing Geospatial information

slide-41
SLIDE 41

Garuda Challenge

 Called upon participants to showcase compute-

data intensive scalable applications on the GARUDA

 Received nearly 20 submissions  Shortlisted candidates benchmarked their

results on production resources of GARUDA

 Winner - Rs. 50,000/-  Runner up - Rs. 25,000/-  Results announced during PARCOMPTECH

conference – Feb 2013

41

slide-42
SLIDE 42

PARCOMPTECH India 2014

 Conference Dates: 27th & 28th February 2014  Venue: YASHADA, Baner Road, Pune  http://parcomptech.cdac.in/

42

slide-43
SLIDE 43

25-Oct-13

slide-44
SLIDE 44

44

The “Global” Grid…

CNGrid

NKN & Garuda

EUAsiaGrid

SAGrid & SANREN

GISELA Genesis II

and the “non-Global” middleware

courtesy : Roberto Barbera, INFN

slide-45
SLIDE 45

Conclusion

Garuda grid – stable and operational over

NKN

Many tools and services deployed to assist

the users

Huge potential to grow Garuda virtual

community

Unique opportunity for solving national

problems through collaborations

45

slide-46
SLIDE 46

Services & Support

Grid Operations : 080-66116474, 6511

Email : grid-help@cdac.in

Website : https://gridsupport.garudaindia.in http://www.garudaindia.in

46

slide-47
SLIDE 47

धनॎयवाद! ಧನ೎ಯವಾದಗಳು! ధన౎యవాదాలు!

நன௎றி!

നന്ധി!

Thank you!

ধনৎযবাদ!

25-Oct-13