The MNM-CloudLab -- Ideas, Concepts & Implemenation 17.7. 22.7. - - PowerPoint PPT Presentation

the mnm cloudlab ideas concepts implemenation
SMART_READER_LITE
LIVE PREVIEW

The MNM-CloudLab -- Ideas, Concepts & Implemenation 17.7. 22.7. - - PowerPoint PPT Presentation

DEUTSCH-FRANZSISCHE SOMMERUNIVERSITT UNIVERSIT DT FRANCO-ALLEMANDE FR NACHWUCHSWISSENSCHAFTLER 2011 POUR JEUNES CHERCHEURS 2011 CLOUD COMPUTING : CLOUD COMPUTING : CLOUD COMPUTING : CLOUD COMPUTING : DFIS ET OPPORTUNITS


slide-1
SLIDE 1

DEUTSCH-FRANZÖSISCHE SOMMERUNIVERSITÄT FÜR NACHWUCHSWISSENSCHAFTLER 2011 UNIVERSITÉ D’ÉTÉ FRANCO-ALLEMANDE POUR JEUNES CHERCHEURS 2011

CLOUD COMPUTING : CLOUD COMPUTING : DÉFIS ET OPPORTUNITÉS CLOUD COMPUTING : CLOUD COMPUTING : HERAUSFORDERUNGEN UND MÖGLICHKEITEN

17.7. – 22.7. 2011

The MNM-CloudLab -- Ideas, Concepts & Implemenation

Nils gentschen Felde MNM-Team, Ludwig-Maximilians-Universität München

slide-2
SLIDE 2

Nils gentschen Felde 2 The MNM-CloudLab

The MNM Team

Leibniz-Rechenzentrum der Bayerischen Akademie der Wissenschaften

slide-3
SLIDE 3

Nils gentschen Felde 3 The MNM-CloudLab

Agenda

  • The idea & concept/setup
  • The implementation: Eucalyptus
  • Deploying VMs
  • Inter-VM communication
  • Elastic Block Storage (EBS)
  • Network security: Concepts & their implementation
  • (High Performance?) Computing in the Cloud
  • Effects of concurrency
  • Outlook & further work
slide-4
SLIDE 4

Nils gentschen Felde 4 The MNM-CloudLab

Designing an infrastructure

  • Goal: Infrastructure-as-a-Service (IaaS) Cloud
  • Idea: Separation of functional areas

→ Mainly performance & security reasons

  • Separation of networks
  • Storage
  • Global file store (NAS/SAN)
  • Elastic block storage (EBS) as a service
  • Management
  • VM-initiated traffic
  • Inter-VM
  • Internet
  • Security aspects
  • Separation of networks & traffic
  • “Hiding” hosts
  • Sandboxing of VMs
slide-5
SLIDE 5

Nils gentschen Felde 5 The MNM-CloudLab

MNM-CloudLab – the idea

Mgmt. Host01

. . .

Storage/NFS: Storage/NFS:

  • Export NFS-shares to mgmt.
  • Mgmt. Network:
  • Mgmt. Network:
  • VM-image transfer

(for initial deployment)

  • Accessing network-based

storage from within VMs

  • Further monitoring & mgmt.

VM-bas VM-based communication: ed communication:

  • Multinet (!)
  • “Public” network
  • /24 subnet
  • Shared & routed
  • “Private” network
  • /27-subnets
  • One subnet per user

here: Layer-3 separation

Router,Firewall, NFS-Server To MWN/Internet

slide-6
SLIDE 6

Nils gentschen Felde 6 The MNM-CloudLab

Agenda

  • The idea & concept/setup
  • The implementation: Eucalyptus
  • Deploying VMs
  • Inter-VM communication
  • Elastic Block Storage (EBS)
  • Network security: Concepts & their implementation
  • (High Performance?) Computing in the Cloud
  • Effects of concurrency
  • Outlook & further work
slide-7
SLIDE 7

Nils gentschen Felde 7 The MNM-CloudLab

Weapon of Choice

  • Ubuntu Enterprise Cloud (UEC)
  • Based on Eucalyptus
  • Components
  • Cloud Controller (CLC)
  • Walrus Storage Controller (WS3)
  • Elastic Block Storage Controller (EBS)
  • Cluster Controller (CC)
  • Node Controller (NC)
  • All components…
  • … are implemented as Web Services
  • … expose Web Service Description Language (WSDL) documents

defining their API

  • Further information are taken from the…
  • Technical White Paper “Ubuntu Enterprise Cloud Architecture”
  • By Simon Wardley, Etienne Goyer & Nick Barcet
slide-8
SLIDE 8

Nils gentschen Felde 8 The MNM-CloudLab

The UEC architecture

Source: Technical White Paper “Ubuntu Enterprise Cloud Architecture” by Simon Wardley, Etienne Goyer & Nick Barcet

slide-9
SLIDE 9

Nils gentschen Felde 9 The MNM-CloudLab

Weapon of Choice: Eucalyptus (1/5)

  • Components
  • Cloud Controller (CLC)
  • Provides interface for users to interact with
  • SOAP-based API
  • Fully compatible to Amazon Elastic Compute Cloud (EC2)
  • CLC talks to the Cluster Controllers (CC)
  • Makes top level choices for allocating new VMs
  • Holds most information

linking users to running instances collection of available machines to be run view of the load of the entire system

  • Walrus Storage Controller (WS3)
  • Elastic Block Storage Controller (EBS)
  • Cluster Controller (CC)
  • Node Controller (NC)
slide-10
SLIDE 10

Nils gentschen Felde 10 The MNM-CloudLab

Weapon of Choice: Eucalyptus (2/5)

  • Components
  • Cloud Controller (CLC)
  • Walrus Storage Controller (WS3)
  • Implements Representational State Transfer (REST) and SOAP API
  • Fully compatible with Amazon Simple Storage Protocol (S3)
  • It is used for:

Storing machine images Accessing and storing data

  • File level storage system
  • Elastic Block Storage Controller (EBS)
  • Cluster Controller (CC)
  • Node Controller (NC)
slide-11
SLIDE 11

Nils gentschen Felde 11 The MNM-CloudLab

Weapon of Choice: Eucalyptus (3/5)

  • Components
  • Cloud Controller (CLC)
  • Walrus Storage Controller (WS3)
  • Elastic Block Storage Controller (EBS)
  • Runs on the same machine as the Cluster Controller
  • Allows for creating persistent block devices

Block devices can be mounted on running machines

  • Ability to create point-in-time snapshots of volumes stored on WS3

Starting point for new EBS volumes Protect data for long-term durability

  • At the network level

ATA over Ethernet (AoE)

  • > no routing possible!

iSCSI (SCSI over TCP or (unlikely) UDP)

  • > routing possible
  • Cluster Controller (CC)
  • Node Controller (NC)
slide-12
SLIDE 12

Nils gentschen Felde 12 The MNM-CloudLab

Weapon of Choice: Eucalyptus (4/5)

  • Components
  • Cloud Controller (CLC)
  • Walrus Storage Controller (WS3)
  • Elastic Block Storage Controller (EBS)
  • Cluster Controller (CC)
  • “Sits” between the NC and the CLC
  • Receives requests from the CLC to allocate VMs
  • Decides which NC will run the VM

Decision based upon status reports from NCs Different strategies possible

  • In charge of managing any virtual network
  • Routing traffic to and from VMs
  • Node Controller (NC)
slide-13
SLIDE 13

Nils gentschen Felde 13 The MNM-CloudLab

Weapon of Choice: Eucalyptus (5/5)

  • Components
  • Cloud Controller (CLC)
  • Walrus Storage Controller (WS3)
  • Elastic Block Storage Controller (EBS)
  • Cluster Controller (CC)
  • Node Controller (NC)
  • Runs on the physical machines on which VMs will be operated
  • Interacts with the OS and hypervisor
  • Instructed by the Cluster Controller

Start/stop VMs Reply to availability queries etc.

slide-14
SLIDE 14

Nils gentschen Felde 14 The MNM-CloudLab

Agenda

  • The idea & concept/setup
  • The implementation: Eucalyptus
  • Deploying VMs
  • Inter-VM communication
  • Elastic Block Storage (EBS)
  • Network security: Concepts & their implementation
  • (High Performance?) Computing in the Cloud
  • Effects of concurrency
  • Outlook & further work
slide-15
SLIDE 15

Nils gentschen Felde 15 The MNM-CloudLab

Deploying a VM (1/2)

  • Generate an ssh key-pair (this step is only required once!)
  • VMs do not grant access using username/password combinations
  • Only strong ssh-key-based authentication!
slide-16
SLIDE 16

Nils gentschen Felde 16 The MNM-CloudLab

Deploying a VM (2/2)

As easy as it is:

  • Choose VM-image (Eucalyptus Machine Image, EMI)
  • Deploy desired number of machines
  • Wait… Done.
slide-17
SLIDE 17

Nils gentschen Felde 17 The MNM-CloudLab

Deploying a VM: …and technically?

Mgmt. Host01

. . .

Router,Firewall, NFS-Server To MWN/Internet

  • Mgmt. holds database
  • stored on NFS-share
  • DB holds VM-images (EMIs)
  • Choose host
  • Different strategies

(Random, Round-Robin etc.)

  • Constraint: Hosts’ resources
  • Copy VM-image to host
  • Caching occurs
  • Deploy image locally
  • Adjust security settings on mgmt.
  • Inject ssh-key into VM
  • Connect VM to bridge on host
  • Launch VM
  • Network config via DHCP

(DHCP-server on mgmt. host)

  • User chooses public or private IP

in advance

VM VM

Storage/NFS

  • Mgmt. Network

VM-based communication

slide-18
SLIDE 18

Nils gentschen Felde 18 The MNM-CloudLab

Agenda

  • The idea & concept/setup
  • The implementation: Eucalyptus
  • Deploying VMs
  • Inter-VM communication
  • Elastic Block Storage (EBS)
  • Network security: Concepts & their implementation
  • (High Performance?) Computing in the Cloud
  • Effects of concurrency
  • Outlook & further work
slide-19
SLIDE 19

Nils gentschen Felde 19 The MNM-CloudLab

VM-based network traffic – an overview

VMA

eth0

VMB

eth0

br0 Host01 ethX VMY

eth0

VMZ

eth0

br0 HostNN ethX

switch Router ethX To MWN/Internet

  • private /27-subnet
  • private IP address

No IP address necessary!

slide-20
SLIDE 20

Nils gentschen Felde 20 The MNM-CloudLab

Considerations

  • Performance aspects:
  • Dedicated Inter-VM network
  • Communication via switch

(here: 1GBit/sec. Ethernet)

  • Communication via bridge device

(Kernel-based (mem copy) possible, depends on Hypervisor)

  • Drawbacks:
  • Shared network for all customers
  • Traffic demands CPU resources
  • Security aspects:
  • Hiding hosts from VMs

(no layer-3 config for hosts)

  • BUT:
  • VM-isolation up to Hypervisor
  • Shared network for all customers
  • Network isolation

Here: layer-3 basis only Others possible (!)

VMA

eth0

VMB

eth0

br0 Host01 ethX VMY

eth0

VMZ

eth0

br0 HostNN ethX

switch

slide-21
SLIDE 21

Nils gentschen Felde 21 The MNM-CloudLab

Agenda

  • The idea & concept/setup
  • The implementation: Eucalyptus
  • Deploying VMs
  • Inter-VM communication
  • Elastic Block Storage (EBS)
  • Network security: Concepts & their implementation
  • (High Performance?) Computing in the Cloud
  • Effects of concurrency
  • Outlook & further work
slide-22
SLIDE 22

Nils gentschen Felde 22 The MNM-CloudLab

Elastic Block Storage (EBS)

Repetition:

  • Walrus Storage Controller
  • Implements Representational State Transfer (REST) and SOAP API
  • Fully compatible with Amazon Simple Storage Protocol (S3)
  • It is used for:
  • Storing machine images
  • Accessing and storing data
  • File level storage system
  • Elastic Block Storage Controller
  • Runs on the same machine as the Cluster Controller
  • Allows for creating persistent block devices
  • Block devices can be mounted on running machines
  • Ability to create point-in-time snapshots of volumes stored on WS3
  • Starting point for new EBS volumes
  • Protect data for long-term durability
  • At the network level
  • ATA over Ethernet (AoE)
  • > no routing possible!
  • iSCSI (“SCSI oder TCP or unlikely UDP”)
  • > routing possible
slide-23
SLIDE 23

Nils gentschen Felde 23 The MNM-CloudLab

Implementation: Elastic Block Storage

Mgmt. Host01

. . .

Router,Firewall, NFS-Server To MWN/Internet

  • File on NFS-store allocated
  • File exported via NFS to Mgmt.
  • Mgmt. loop-mounts file
  • Re-export file via iSCSI
  • Host imports block device
  • Imported block device is passed

directly to VM

  • Possibilities: E.g. mount & re-

export via NFS to other VMs

  • Experience: Unexpectedly good!

VM

Storage/NFS

  • Mgmt. Network

VM-based communication

File NFS File iSCSI Block Device

/dev

Pass-through

/dev /dev

loop-mount

VM

NFS Filesystem

slide-24
SLIDE 24

Nils gentschen Felde 24 The MNM-CloudLab

Agenda

  • The idea & concept/setup
  • The implementation: Eucalyptus
  • Deploying VMs
  • Inter-VM communication
  • Elastic Block Storage (EBS)
  • Network security: Concepts & their implementation
  • (High Performance?) Computing in the Cloud
  • Effects of concurrency
  • Outlook & further work
slide-25
SLIDE 25

Nils gentschen Felde 25 The MNM-CloudLab

Networking & security models

  • 4 networking modes supported by Eucalyptus:
  • System Mode
  • Static Mode
  • Managed Mode
  • Managed-NoVLAN Mode
  • Different levels of security granted
  • Choice influenced by aspects of corporate network

setup/configuration

slide-26
SLIDE 26

Nils gentschen Felde 26 The MNM-CloudLab

Network models – System Mode

  • Considered the simplest networking mode
  • Assigns random MAC address to the VM before booting
  • Attaches the VM’s Ethernet device to the physical Ethernet

through node's local bridge

  • VM instances typically obtain IP address using DHCP

(same way any non-VM machine using DHCP would obtain address)

  • Obvious disadvantages:
  • Typically, VMs not separated from nodes
slide-27
SLIDE 27

Nils gentschen Felde 27 The MNM-CloudLab

Network models – Static Mode

  • Mapping of MAC address

IP address needed

  • Static entry in DHCP server is set up
  • At VM’s startup, next free MAC/IP pair is assigned to VM
  • VM’s Ethernet device connected to physical Ethernet through node’s

bridge (similar to SYSTEM mode).

  • Obvious advantages:
  • More control over VM IP address assignment
  • Disadvantages:

System and Static mode do not offer…

  • Security groups (see later)
  • Elastic IPs

(user-defined IP address assignments / IP reservations)

  • Isolation to network traffic between VMs
  • Availability of the metadata service

(to obtain instance specific information)

slide-28
SLIDE 28

Nils gentschen Felde 28 The MNM-CloudLab

Network models – Managed Mode

  • Admin defines large network
  • usually private and un-routable
  • Eucalyptus maintains DHCP server
  • static mappings for each VM
  • One VLAN per customer implemented!
  • VM’s Ethernet device connected to physical Ethernet through node’s

bridge

  • VLAN-tagging enabled
  • Prerequisite: reserved VLAN-range for Eucalyptus
  • Static/public IPs possible

(“Elastic IPs”)

  • Users can define “named networks”

(aka Amazon’s “security groups”)

  • Subnet out of previously defined IP range
  • Application of ingress rules for whole network possible
  • Most feature-full mode
slide-29
SLIDE 29

Nils gentschen Felde 29 The MNM-CloudLab

Pitfall: Initially no rule defined!

slide-30
SLIDE 30

Nils gentschen Felde 30 The MNM-CloudLab

Public vs. private IP addresses

  • Actually, only “private” IPs supplied
  • /XX-subnet on per-customer basis
  • Public IPs:

Static NAT von Mgmt. host

  • Advantage:
  • “One EMI suites them all”
  • Accounting of traffic made easy
  • Disadvantage:
  • “Inter-customer traffic” translated via
  • mgmt. host
  • Puts extra load on mgmt. host

VMA

eth0

VMB

eth0

br0 Host01 ethX VMY

eth0

VMZ

eth0

br0 HostNN ethX

switch Router ethX Mgmt. ethX

external request

slide-31
SLIDE 31

Nils gentschen Felde 31 The MNM-CloudLab

Network models – Managed-NoVLAN Mode

  • Identical to MANAGED mode, but…
  • does not provide VM network isolation
  • Layer-3 isolation granted
  • Advantage:
  • Alternative for not “VLAN-clean” setups
slide-32
SLIDE 32

Nils gentschen Felde 32 The MNM-CloudLab

Agenda

  • The idea & concept/setup
  • The implementation: Eucalyptus
  • Deploying VMs
  • Inter-VM communication
  • Elastic Block Storage (EBS)
  • Network security: Concepts & their implementation
  • (High Performance?) Computing in the Cloud
  • Effects of concurrency
  • Outlook & further work
slide-33
SLIDE 33

Nils gentschen Felde 33 The MNM-CloudLab

Motivation - Idea

HPC Cluster HPC Cloud Cluster Trend towards Manage Management be ment benefits of virtual nefits of virtual H HPC

  • Dynamical sizing / partitioning
  • Loadbalancing
  • Automation / scripting of management tasks
  • Fault tolerance without „Checkpoint and restart“
  • Better security due to sandboxing of processes
slide-34
SLIDE 34

Nils gentschen Felde 34 The MNM-CloudLab

Motivation - Challenges

But: What about performance?

  • Effects expected:
  • Virtualization layer will induce overhead
  • Concurrent effects appear when running several VMs in

parallel; e.g. effects on storage, network, memory access

  • Effects will depend on virtualization architecture,

implementations, OS, …

  • What scale will that effects be?
  • How to measure?
slide-35
SLIDE 35

Nils gentschen Felde 35 The MNM-CloudLab

Agenda

  • The idea & concept/setup
  • The implementation: Eucalyptus
  • Deploying VMs
  • Inter-VM communication
  • Elastic Block Storage (EBS)
  • Network security: Concepts & their implementation
  • (High Performance?) Computing in the Cloud
  • Effects of concurrency
  • Outlook & further work
slide-36
SLIDE 36

Nils gentschen Felde 36 The MNM-CloudLab

Benchmark Suite – Design Goals

  • Measure core components of virtualized systems
  • CPU
  • Main Memory
  • Network
  • Disk
  • Compare results of different hypervisors
  • Architectures (full-, native-, … , para-virtualization)
  • Implementations (VMware ESXi, Xen, MS Hyper-V, …)
  • Analyze impact of virtualization on different…
  • Operating Systems (Windows, Linux, …)
  • System architectures (32bit, 64bit, …)
  • Measure
  • Overhead of virtualization
  • Effects of concurrency
  • Take care about special time measurement in VMs
slide-37
SLIDE 37

Nils gentschen Felde 37 The MNM-CloudLab

Benchmark Suite – Time Measurement

Measurement of time difficult in virtualized environments:

  • Definition of time
  • Time synchronization

Example: Linpack-Benchmark static REAL second(void){ return ((REAL)((REAL)clock()/(REAL)CLOCKS_PER_SEC)); } # clock() : Wall-Clock-Time in time ticks since process start # CLOCKS_PER_SEC: Frequency of the hardware clock

External clocks may be necessary, depending on counters used by the benchmark tool

VM1 Hypervisor VM1 CPU1 CPU2 Zeit I/O VM1 Real duration Duration as seen by VM1

slide-38
SLIDE 38

Nils gentschen Felde 38 The MNM-CloudLab

Benchmark Suite - Implementation

  • Benchmarks used:
  • Run combinations of single benchmarks to test for

concurrent effects, e.g.

  • CPU + Network
  • Parallel network access
  • Different network setups (VM2VM, VM2PM, …)

Component Benchmark CPU Linpack Main Memory Ramspeed Disk Iometer Network Iometer

slide-39
SLIDE 39

Nils gentschen Felde 39 The MNM-CloudLab

Benchmark Suite - Implementation

  • Benchmarks used:
  • Run combinations of single benchmarks to test for

concurrent effects, e.g.

  • CPU + Network
  • Parallel network access
  • Different network setups (VM2VM, VM2PM, …)

Component Benchmark CPU Linpack Main Memory Ramspeed Disk Iometer Network Iometer

slide-40
SLIDE 40

Nils gentschen Felde 40 The MNM-CloudLab

Applying the Test Suite

Absolute runtime Relative runtime

Physical Linux 450,58 Xen Para 448,79 OpenVZ 452,53 MS Hyper‐V 485,76 VMware ESXi 472,00 0,00 100,00 200,00 300,00 400,00 500,00 600,00 Run‐time in seconds Physical Linux 100,00% Xen Para 99,60% OpenVZ 100,43% MS Hyper‐V 107,81% VWware ESXi 104,75% 0,00% 20,00% 40,00% 60,00% 80,00% 100,00% 120,00% Effectiveness

Linpack – Absolute/Relative runtime

slide-41
SLIDE 41

Nils gentschen Felde 41 The MNM-CloudLab

Applying the Test Suite

ESXi Xen Hyper-V Virtuozzo CPU – effects of concurrency

single VM two VMs three VMs VM3 466 481 728 VM2 481 739 VM1 728 500 1.000 1.500 2.000 2.500 Runtime in seconds single VM two VMs three VMs VM3 486 487 938 VM2 472 470 VM1 939 500 1.000 1.500 2.000 2.500 Runtime in seconds single VM two VMs three VMs VM3 213,21 331,18 519,7 VM2 333,2 518,74 VM1 524,77 500 1.000 1.500 2.000 2.500 Runtime in seconds single VM two VMs three VMs VM3 214,2 293,55 451,96 VM2 295,59 452,52 VM1 455,63 500 1.000 1.500 2.000 2.500 Runtime in seconds

slide-42
SLIDE 42

Nils gentschen Felde 42 The MNM-CloudLab Xen – receive data from network Virtuozzo – send data to network Xen – send data to network Virtuozzo – receive data from network

Network – effects of concurrency

single VM two VMs three VMs VM 3 17,17 VM 2 25,89 17,03 VM 1 49,26 25,41 16,93 10 20 30 40 50 60 70 Throughput in MByte/s single VM two VMs three VMs VM 3 20,85 VM 2 30,85 20,46 VM 1 60,00 30,40 20,27 10 20 30 40 50 60 70 Throughput in MByte/s single VM two VMs three VMs VM 3 17,511341 VM 2 24,436174 17,528381 VM 1 51,876715 25,291585 17,512457 10 20 30 40 50 60 70 Throughput in MByte/s single VM two VMs three VMs VM 3 3,769137 VM 2 5,652343 3,767896 VM 1 60,712871 5,651204 3,765671 10 20 30 40 50 60 70 Throughput in MByte/s

Applying the Test Suite

slide-43
SLIDE 43

Nils gentschen Felde 43 The MNM-CloudLab Hyper-V – receive data from network Hyper-V– send data to network ESXi– receive data from network ESXi– send data to network

single VM two VMs three VMs VM 3 17,61 VM 2 25,94 17,61 VM 1 51,79 25,94 17,61 10 20 30 40 50 60 70 Throughput in MByte/s single VM two VMs three VMs VM 3 3,77 VM 2 31,97 3,77 VM 1 63,98 32,84 3,77 10 20 30 40 50 60 70 Throughput in MByte/s single VM two VMs three VMs VM 3 15,36 VM 2 30,55 15,34 VM 1 46,02 30,24 15,31 10 20 30 40 50 60 70 Throughput in MByte/s single VM two VMs three VMs VM 3 21,44 VM 2 22,87 21,31 VM 1 62,26 22,87 21,33 10 20 30 40 50 60 70 Throughput in MByte/s

Network – effects of concurrency

Applying the Test Suite

slide-44
SLIDE 44

Nils gentschen Felde 44 The MNM-CloudLab

Xen Hyper-V ESXi Virtuozzo Network – Physical vs. virtual communication peers

Send Receive VM to PM 51,79 63,98 VM to VM 108,29 112,23 0,00 50,00 100,00 150,00 200,00 250,00 Throughput in MByte/s Send Receive VM to PM 46,02 62,26 VM to VM 129,31 129,01 0,00 50,00 100,00 150,00 200,00 250,00 Throughput in MByte/s Send Receive VM to PM 49,26 60,00 VM to VM 233,85 210,78 0,00 50,00 100,00 150,00 200,00 250,00 Throughput in MByte/s Send Receive VM to PM 51,88 60,71 VM to VM 12,23 9,26 0,00 50,00 100,00 150,00 200,00 250,00 Throughput in MByte/s

Applying the Test Suite

slide-45
SLIDE 45

Nils gentschen Felde 45 The MNM-CloudLab

Performance Requirements

Criterion High-Performance High-Throughput Coupling tight loose CPU impact co-determinant critical Interconnect impact critical less Main memory addressing sensitive; prefers contiguous, uniform latency addressing less sensitive Program structures inter-communicating program replicas workflows; pipelining of computing tasks Examples fluid dynamics problems, crash codes high-energy physics (e.g. CERN LHC experiments), general parameter variation studies

slide-46
SLIDE 46

Nils gentschen Felde 46 The MNM-CloudLab

Conclusion

  • CPU virtualization is efficient (close to 100%)
  • RAM access is fast

High Throughput Computing (HTC) Tasks can be virtualized efficiently

  • Network Access efficiency depends on
  • flow direction
  • concurrent use
  • Available CPU power
  • Topology changes (e.g. live migration) “confuses” MPI

High Performance Computing (HPC) Tasks should not be virtualized yet

slide-47
SLIDE 47

Nils gentschen Felde 47 The MNM-CloudLab

Agenda

  • The idea & concept/setup
  • The implementation: Eucalyptus
  • Deploying VMs
  • Inter-VM communication
  • Elastic Block Storage (EBS)
  • Network security: Concepts & their implementation
  • (High Performance?) Computing in the Cloud
  • Effects of concurrency
  • Outlook & further work
slide-48
SLIDE 48

Nils gentschen Felde 48 The MNM-CloudLab

Outlook & further work

  • Eucalyptus 3.0 Roadmap
  • High Availablity (HA)
  • Eucalyptus Identity Authorization and Management (EIAM)
  • Active Directory/LDAP integration
  • Windows Hosting Service
  • Boot from EBS (Elastic Block Storage)
  • Benchmarking parallel HPC applications in the Cloud
  • Different Hypervisors
  • Uniform Hypervisor per run
  • VMs powered by different Hypervisor during one run
  • Different locations of VMs
  • On the same host
  • On different hosts, but in the same LAN
  • Including host in foreign locations using VPN-tunnels
  • And much more… ;-)

Want to collaborate?

slide-49
SLIDE 49

Nils gentschen Felde 49 The MNM-CloudLab

Contact: felde@nm.ifi.lmu.de Thank you.

Have a “cloudy” day!

slide-50
SLIDE 50

Nils gentschen Felde 50 The MNM-CloudLab

Authentication and Authorization

  • Two type of “actors” need to be authenticated and authorized on UEC:
  • The users or administrators of the system which have specific rights to modify the system and start and stop instances;
  • The components of the system (NC, CLC, CC) which need to trust each other when transmitting requests.
  • On a general level, the authentication is performed using locally-generated X509 certificates, as cryptographic keys to

authenticate and secure communications between all actors with communication based on the WS-Security policy framework.

  • This is true for all internal communication within the cloud. Users authenticate either with an X509 certificate or a Query

type key.

  • The initial addition of Cluster Controller or Node Controller to the Cloud Controller environment requires using a

password to exchange the cryptographic keys from the lower controller to the upper one.

  • Once this is done, all operations rely on the trust provided by these keys in the communications between the controllers.
  • Authentication and authorization of users:
  • Initial user account creation of user is performed in a two-step operation:
  • First, any user with access to the Cloud Controller user interface can fill in a form to request an account;
  • When a request is received, it is up to the administrator to grant access to the user according to its own policy.
  • Users which have been granted access retrieve the certificate and query type key that have been generated for them and

which are used for all their requests performed through the APIs.

  • The type of authentication (query key or certificate) will vary based on the tool used to access the cloud, their password
  • nly being used to access the web console that allows them to retrieve their certificate and query key.
  • Note: a new certificate and query key is generated each time the user ask to retrieve them.
  • The Cloud Controller holds the central right repository for each user.
  • Every time a request arrives at a controller (CLC, CC or NC), it is its duty to verify that the user from which the request
  • riginated is duly authorized to perform it.
  • Users obviously only make requests to the Cloud Controller, so authentication is only performed there, but authorization is

verified at each level to prevent abuse.