Apache Libcloud Paul Querna, Chief Architect, Cloudkick June 1, - - PowerPoint PPT Presentation

▶
apache libcloud
SMART_READER_LITE
LIVE PREVIEW

Apache Libcloud Paul Querna, Chief Architect, Cloudkick June 1, - - PowerPoint PPT Presentation

Aug 14, 2022 317 likes •766 views

Apache Libcloud Paul Querna, Chief Architect, Cloudkick June 1, 2010 Tuesday, June 1, 2010 About Me Chief Architect at Cloudkick Developer on Apache HTTP Server Former VP Infrastructure @ ASF Libcloud developer! Tuesday, June

slide-1
SLIDE 1

Apache Libcloud

Paul Querna, Chief Architect, Cloudkick June 1, 2010

Tuesday, June 1, 2010

slide-2
SLIDE 2

About Me

  • Chief Architect at Cloudkick
  • Developer on Apache HTTP Server
  • Former

VP Infrastructure @ ASF

  • Libcloud developer!

Tuesday, June 1, 2010

slide-3
SLIDE 3

About the Cloud

Tuesday, June 1, 2010

slide-4
SLIDE 4

About the Cloud

  • Awesome.

Tuesday, June 1, 2010

slide-5
SLIDE 5

About the Cloud

  • Awesome.
  • Really.

Tuesday, June 1, 2010

slide-6
SLIDE 6

About the Cloud

  • Awesome.
  • Really.
  • Maybe not always.

Tuesday, June 1, 2010

slide-7
SLIDE 7

About the Cloud

  • Awesome.
  • Really.
  • Maybe not always.
  • But mostly.

Tuesday, June 1, 2010

slide-8
SLIDE 8

Services (SaaS)

  • GMail, Google Docs, etc
  • Most any website cloud be called SaaS.
  • Cloudkick

Tuesday, June 1, 2010

slide-9
SLIDE 9

Platforms (PaaS)

  • Google AppEngine
  • Heroku (Rails as PaaS)
  • SalesForce.com /

VMForce

Tuesday, June 1, 2010

slide-10
SLIDE 10

Storage

  • Amazon S3
  • Rackspace CloudFiles
  • Google Storage for Developers

Tuesday, June 1, 2010

slide-11
SLIDE 11

Compute

  • Amazon EC2
  • Rackspace Cloud
  • Linode
  • GoGrid
  • Voxel
  • And many more!

Tuesday, June 1, 2010

slide-12
SLIDE 12

I want a server.

Tuesday, June 1, 2010

slide-13
SLIDE 13

I want a server: right now.

Tuesday, June 1, 2010

slide-14
SLIDE 14

from libcloud.types import Provider from libcloud.providers import get_driver rs = get_driver(Provider.RACKSPACE)('rackspace-apikey') rs.create_node('serverA')

Enter Libcloud

Tuesday, June 1, 2010

slide-15
SLIDE 15

About Libcloud

  • Started in the summer of 2009
  • Easy to use.
  • Portable.
  • Pure Python (proposed ports to others)
  • Socket & HTTP interfaces exist today!

Tuesday, June 1, 2010

slide-16
SLIDE 16

Why?

  • API Styles:
  • Amazon: XML + Custom HMAC Auth
  • Rackspace: JSON + Auth Tickets
  • SoftLayer: XML RPC + User / Password

Tuesday, June 1, 2010

slide-17
SLIDE 17

Libcloud Today

  • In the Apache Incubator
  • 15 Providers:
  • Dreamhost, Amazon EC2, Enomaly ECP

, Eucalyptus, GoGrid, IBM Developer Cloud, Linode, OpenNebula, Slicehost, SoftLayer, Rackspace, RimuHosting, Terramark, VMWare vCloud, Voxel, VPS.net

Tuesday, June 1, 2010

slide-18
SLIDE 18

Libcloud APIs

  • Originally 6 Core APIs
  • List Nodes
  • List Images
  • List Sizes
  • Create / Destroy / Reboot Node

Tuesday, June 1, 2010

slide-19
SLIDE 19

list_nodes

foo = d.list_nodes() for node in foo: print node.id print node.public_ip

Tuesday, June 1, 2010

slide-20
SLIDE 20

list_images

images = d.list_images() ubuntu = [i for i in images if i.name.find('Ubuntu') != -1] print ubuntu[0].id print ubuntu[0].name

Tuesday, June 1, 2010

slide-21
SLIDE 21

list_sizes

sizes = d.list_sizes() print sizes[0].id print sizes[0].ram print sizes[0].disk print sizes[0].price

Tuesday, June 1, 2010

slide-22
SLIDE 22

Create Node

images = d.list_images() sizes = d.list_sizes() print d.create_node(name="test22", image=images[0], size=sizes[0])

Tuesday, June 1, 2010

slide-23
SLIDE 23

Reboot/Destroy

d.reboot(nodeA) d.destroy(nodeB)

Tuesday, June 1, 2010

slide-24
SLIDE 24

Locations!

loc = d.list_locations() print loc[0].name print loc[0].country

Tuesday, June 1, 2010

slide-25
SLIDE 25

Extended APIs

  • Providers inconsistent about services.
  • Have a “ex_” prefix, documented per-

driver.

  • Amazon Security Groups:
  • amz.create_node(‘foo’,

ex_securitygroup=‘groupA’)

Tuesday, June 1, 2010

slide-26
SLIDE 26

Getting Started

  • easy_install apache-libcloud

Tuesday, June 1, 2010

slide-27
SLIDE 27

Get your Provider Info

  • Amazon:
  • http://aws-portal.amazon.com/gp/aws/

developer/account/index.html? action=access-key

  • Rackspace:
  • https://manage.rackspacecloud.com/

APIAccess.do

Tuesday, June 1, 2010

slide-28
SLIDE 28

List your Machines

from libcloud.types import Provider from libcloud.providers import get_driver d = get_driver(Provider.RACKSPACE)("xxxxxxx") nodes = d.list_nodes() for node in nodes: print "id: %s name: %s public_ips: %s" % (node.id, node.name, node.public_ip)

Tuesday, June 1, 2010

slide-29
SLIDE 29

Cheapest 4 gig node

  • utside the US

possible = [] for d in drivers: loc = filter(lambda x: x.country != 'US', d.list_locations()) for l in loc: sizes = filter(lambda x: x.ram >= 4096, d.list_sizes(l)) for s in sizes: possible.append({'size': s, 'location': l, 'driver': d}) best = sorted(possible, lambda x,y: x['size'].price < y['size'].price)[0] print best

Tuesday, June 1, 2010

slide-30
SLIDE 30

Integrating with Fabric

env.hosts = [x.public_ip[0] for x in d.list_nodes()] def hostname(): run('hostname')

Tuesday, June 1, 2010

slide-31
SLIDE 31

$ fab hostname [173.45.245.33] run: hostname [173.45.245.33] out: lctest3.k1k.me [173.45.245.32] run: hostname [173.45.245.32] out: lctest2.k1k.me Done. Disconnecting from 173.45.245.33... done. Disconnecting from 173.45.245.32... done.

Tuesday, June 1, 2010

slide-32
SLIDE 32

One more thing!

  • deploy_node
  • Calls create_node
  • Consistent initial bootstrapping of

machines.

  • SSH Keys
  • Configuration Management

Tuesday, June 1, 2010

slide-33
SLIDE 33

Installing Puppet

skey = SSHKeyDeployment(key) sd = ScriptDeployment("apt-get install -y puppet") msd = MultiStepDeployment([skey, sd]) node = d.deploy_node(name="lc-test", deploy=msd)

Tuesday, June 1, 2010

slide-34
SLIDE 34

Up next for Libcloud

Tuesday, June 1, 2010

slide-35
SLIDE 35

Image Formats

  • Hazy world between Operating System,

Configuration Management and the Sysadmin.

  • People stick with Config Management,

because dealing with base Images is painful today

Tuesday, June 1, 2010

slide-36
SLIDE 36

Existing standards

  • Amazon AMI
  • VMWare Open

Virtualization Format (OVF)

Tuesday, June 1, 2010

slide-37
SLIDE 37

Hosting Provider Side

  • Technical challenges
  • Most commercial hosting is Xen based
  • Most hosting companies aren’t giant tech

companies

Tuesday, June 1, 2010

slide-38
SLIDE 38

User Side

  • Building Images is complicated.
  • Versioning Sucks
  • Time sink uploading

Tuesday, June 1, 2010

slide-39
SLIDE 39

Proposed Image Format

  • Based on Cloudlets Project
  • JSON Metadata in single file
  • Filesystem in a tarball
  • Versioned in DVCS
  • Includes building server-side support

infrastructure for Hosting providers!

  • More details on mailing list

Tuesday, June 1, 2010

slide-40
SLIDE 40

Multiple Languages

  • Hundreds of emails exchanged on the

mailing list.

  • Interest, something will happen.
  • If interested, join the lists, start hacking on

code!

Tuesday, June 1, 2010

slide-41
SLIDE 41

Contributing!

  • Open Community just as important as
  • pen code -- everything on list or IRC.
  • Hosting providers: Get your driver in!
  • Hackers: Make cool tools!
  • Sysadmins: Manage your infrastructure.

Tuesday, June 1, 2010

slide-42
SLIDE 42

Related Projects

  • JClouds
  • Java
  • Apache Deltacloud
  • Ruby, started by Redhat, just joined

Apache Incubator in May

  • Fog
  • Ruby

Tuesday, June 1, 2010

slide-43
SLIDE 43

Questions?

  • Apache Libcloud:

http://libcloud.org/ #libcloud on Freenode IRC

  • Slides online:

http://paul.querna.org/slides

Tuesday, June 1, 2010