SLIDE 1
INTERNET EVOLUTION AND THE ROLE OF SOFTWARE ENGINEERING Pamela - - PowerPoint PPT Presentation
INTERNET EVOLUTION AND THE ROLE OF SOFTWARE ENGINEERING Pamela - - PowerPoint PPT Presentation
INTERNET EVOLUTION AND THE ROLE OF SOFTWARE ENGINEERING Pamela Zave AT&T LaboratoriesResearch Florham Park, New Jersey, USA INTERNET EVOLUTION AND THE ROLE OF SOFTWARE ENGINEERING: OUTLINE WHAT IS THE STATE OF THE A INTERNET, AND WHY
SLIDE 2
SLIDE 3
STATE OF THE "CLASSIC" INTERNET ARCHITECTURE
THE "CLASSIC" INTERNET ARCHITECTURE defined in terms of layers with different functions THE REAL INTERNET designed to empower users and encourage innovation has succeeded beyond anyone's wildest dreams it does not meet current or future needs for . . . . . . convergence of data, telephone, and broadcast networks, . . . security and reliability, . . . quality of service, . . . resource management, . . . balancing the interests of diverse stakeholders the good properties of the classic architecture are eroded badly by workarounds the classic architecture has been made obsolete by explosive growth in users, traffic, and applications applications middleware transport (TCP,UDP) network (IP) link physical Internet core
SLIDE 4
THE INTERNET IS NOT APPLICATION-FRIENDLY THE COSTS AND BENEFITS OF NETWORK SECURITY NAT and firewalls are so tightly inter- twined with TCP and UDP that unless they know about an application, it is unlikely to work
THE STATE OF INTERNET APPLICATIONS
Network-level security is provided by Network Address Translation (NAT) and firewalls. private subnet public Internet machines here have no persistent public addresses no initiation of communication, even if wanted yet applications are not secure! this form of security is inappropriate for peer-to-peer applications, making them difficult to build and very inefficient i.e., unless it looks like a Web service many application needs are not supported (mobility, multihoming, anycast, security, privacy, etc.) because there is little separation of concerns, . . . . . . mechanisms to solve problems are limited in applicability and tend to break each other as a result, . . . . . . networked applications and middleware are complex, unreliable, . . . . . . and far too difficult to build, deploy, and maintain
SLIDE 5
THE ST THE STATE OF INTERNET EVOLUTION TE OF INTERNET EVOLUTION
[Handley 06] INTERNET "OSSIFICATION" there has been no important change in the transport layer (TCP/ UDP) since 1988 " . . . technologies get deployed in the core of the Internet when they solve an immediate problem
- r when money can be made"
there has been no important change in the network layer (IP) since 1993 it is very difficult for an Internet service provider to make money with improvements, because most have no effect until everyone else adopts them a crisis is the only way to get the global consensus required for real change
SLIDE 6
THE STATE OF THE IETF
"A Hitchhiker's Guide to SIP" is a snapshot of SIP RFCs and drafts . . . . . . which lists 142 documents, totaling many thousands of pages THE MEDIUM the base document (IETF RFC 3261) is 268 pages specifications are written in English, augmented only by message sequence charts that look like this (IETF macros): process1 process2 IETF philosopy is to standardize based on "rough consensus and working code" finite-state machines are rarely used note how this forces you to forget race conditions! THE MESSAGE it is continually being extended, bottom-up, in response to an endless series of new use cases
- pinions are based on two false
assumptions: more generality can only be obtained with more complexity a bad scenario can be ignored if you claim it is rare (a "corner case") . . . AS CAPTURED BY THE SPECIFICATION OF SIP it is now so difficult to build SIP applications that there is talk of abandoning it
SLIDE 7
THE ST THE STATE OF NETWORKING RESEARCH TE OF NETWORKING RESEARCH
[Rexford 10]
"In my college networking class I fell asleep at the start of the semester when the IP header was on the screen, and woke up at the end of the semester with the TCP header on the screen."
SLIDE 8
THE ST THE STATE OF NETWORKING RESEARCH TE OF NETWORKING RESEARCH
[Rexford 10]
"In my college networking class I fell asleep at the start of the semester when the IP header was on the screen, and woke up at the end of the semester with the TCP header on the screen." "Networking is all details and no principles."
SLIDE 9
THE ST THE STATE OF NETWORKING RESEARCH TE OF NETWORKING RESEARCH
[Rexford 10]
"In my college networking class I fell asleep at the start of the semester when the IP header was on the screen, and woke up at the end of the semester with the TCP header on the screen." "Networking is all details and no principles." "There is a tendency in our field to believe that everything we currently use is a paragon of engineering, rather than a snapshot of our understanding at the time. We build great myths of spin about how what we have done is the only way to do it . . . to the point that our universities now teach the flaws to students . . . who don't know better." —John Day
SLIDE 10
THE ST THE STATE OF NETWORKING RESEARCH TE OF NETWORKING RESEARCH
[Rexford 10]
"In my college networking class I fell asleep at the start of the semester when the IP header was on the screen, and woke up at the end of the semester with the TCP header on the screen." "Networking is all details and no principles." "There is a tendency in our field to believe that everything we currently use is a paragon of engineering, rather than a snapshot of our understanding at the time. We build great myths of spin about how what we have done is the only way to do it . . . to the point that our universities now teach the flaws to students . . . who don't know better." —John Day "So, these network research people today aren't doing theory, and yet they aren't the people who brought us the Internet. What exactly are they doing?"
SLIDE 11
A A
A A A A
INTERNET EVOLUTION AND THE ROLE OF SOFTWARE ENGINEERING: OUTLINE
WHAT IS THE STATE OF THE
INTERNET, AND WHY SHOULD SOFTWARE ENGINEERS BE CONCERNED?
WHAT CAN SOFTWARE-
ENGINEERING RESEARCHERS DO ABOUT IT? The Internet is not application-friendly. Every segment of society has a stake in the Internet, . . . . . . and this limits its usefulness to all of them. Businesses, networking researchers, and the IETF are not making progress toward an application-friendly Internet, . . . . . . so software researchers have to do it. [Clark et al. 05]
SLIDE 12
EVOLUTION THROUGH OVERLAYS AND VIRTUALIZATION
Common definition: An overlay is a custom-built network layer deployed over existing layers. OVERLAYS ARE THE MODULES OF NETWORK ARCHITECTURE AN OVERLAY IS A "CLEAN SLATE" FOR DESIGN FOR TODAY: FOR THE FUTURE: network resources (physical, link layers) Internet core (network, transport layers) applications OVERLAYS OVERLAYS frequently used to support applications better than the unvarnished Internet does (e.g., Akamai, VPNs, some middleware) slice of resources applications OVERLAYS slice of resources applications OVERLAYS network resources virtualization can experiment with new architectural ideas in the end, there may be no universal Internet layer [Roscoe 2006]
SLIDE 13
A A
A A A A
BRIDGING THE KNOWLEDGE GAP
SOFTWARE ENGINEERING + DISTRIBUTED SYSTEMS NETWORKING BETWEEN Using Day’s definition
- f an overlay [Day 08],
each overlay contains all the basic mechanisms
- f networking,
including naming, communication service, security, and resource management. The definition is a template that can be instantiated differently for different purposes, scopes, and levels. The definition makes clear how overlays compose in a hierarchy and what their relationships are. It is precise enough to be formalized. THIS IS THE TOOL I HAVE BEEN LOOKING FOR! deeply rooted in the history and practice of networking represents all the important concepts without unnecessary detail draws the module boundary in exactly the right place
SLIDE 14
DAY'S DEFINITION OF AN OVERLAY
A B C D E Membership: the members are processes; each has a unique and persistent name from the name space; enrollment protocol accepts and names new members Routing: any member can reach any
- ther through a path in the
- verlay; routing protocol
spreads knowledge of links and paths; forwarding protocol uses path knowledge THERE IS SECURITY AND RESOURCE MANAGEMENT THROUGHOUT Links: there is a link between two member processes if both are registered in the same lower
- verlay
Registration: user processes in a higher overlay can register their locations at member processes on the same machine; there is a directory of registrations Communication Service: the overlay provides a specified service for its users, e.g., point-to-point sessions
B E
session(B,E)
SLIDE 15
THE SESSION INTERFACE BETWEEN OVERLAYS
UPPER OVERLAY LOWER OVERLAY accept- Session(A,B,sb) send(sb,msg) receive(sb,msg) request- Session(A,B,sa) send(sa,msg) receive(sa,msg) A B session session API calls a b path of session initially a receives request, looks up location of B at b, caches b, and sends request from a to b routing and forwarding can cause congestion and loss (as in IP), . . . . . . so the overlay also needs flow and error control (as in TCP)
SLIDE 16
As A moves, it changes its registration from a1 to a2 because . . . . . . A has moved from one machine to another, or . . . in the new location, the lower process must have a different name, or . . . A's point of attachment to communication services is now through a different overlay.
MOBILITY
UPPER OVERLAY LOWER OVERLAYS mobility is always movement of processes in an upper overlay with respect to processes in lower overlays A B session a1 a2 b1 b2 registrations relate processes
- n the same
machine A process in an overlay is a protocol endpoint. It should be able to move with no effect on its current sessions. process migration VM migration physical mobility
SLIDE 17
A B a1 a2 b1 b2
MEMBER MOBILITY: THE DO-IT-YOURSELF IMPLEMENTATION
R S r r s s ATTACHMENT MOVES FROM ONE LOWER OVERLAY TO ANOTHER THE UPPER OVERLAY DOES THE WORK! AS A MEMBER MOVES ITS ATTACHMENTS TO LOWER OVERLAYS, ITS OWN OVERLAY MUST MAINTAIN ITS REACHABILITY as links change, the upper overlay must update its routing tables and recover from loss of misrouted messages most difficult problem: in a large overlay, names are location- dependent and routing works at this scale
- nly because
- f regional
aggregation— mobile members violate the basic
- perating
assumption
SLIDE 18
THE LOWER OVERLAY DOES THE WORK! A B session a1 a2 b1 b2
SESSION MOBILITY: EXPORTING MOBILITY AS A SERVICE
ATTACHMENTS MOVE WITHIN ONE LOWER OVERLAY when B moves from b1 to b2, both a1’s cache and the directory must be updated if there is a period when B has no attachment, messages to it could be routed through another process and buffered there for a smooth handoff update and handoff also needed when A moves from a1 to a2 to:b1 to:a1 to:b2 to:a1 this implementation relies on good directory technology initially, a1 and b1 cache each other’s names for this session
SLIDE 19
EXAMPLE: DEVICE MOBILITY, PROCESS MIGRATION, AND MULTIHOMING
the user of this handheld device is engaged in a multiplayer game game instance during the game session, the user moves across campus . . . . . . and the device uses both WiFi and cellular networks . . . . . . and the game provider migrates the game instance to a lightly loaded server THE STATE OF PRACTICE IS NOWHERE CLOSE TO ACHIEVING THIS: migration of the game- instance process mobility from one cell tower to another multihoming would allow the bandwidth from both edge networks to contribute to the same session multihoming is similar to mobility, replacing “movement” by “multiplicity” WiFi cell
SLIDE 20
EXAMPLE: DEVICE MOBILITY, PROCESS MIGRATION, AND MULTIHOMING
the user of this handheld device is engaged in a multiplayer game game instance during the game session, the user moves across campus . . . . . . and the device uses both WiFi and cellular networks . . . . . . and the game provider migrates the game instance to a lightly loaded server EASY STRAIGHTFORWARD CHALLENGING BUT ACCESSIBLE how can the solutions be implemented, configured, and deployed independently? understand how to satisfy each requirement with an overlay in each case, understand the issues of Internet scale and how to apply the known techniques understand how all the solutions can be combined without interfering with each other ? ? multi- homing migration
SLIDE 21
QUESTIONS WE CAN NOW ASK
Which communication services can be implemented in overlays, in addition to those already available as middleware? Is there a uniform approach to scaling, by splitting and bridging
- verlays, without the bad
side-effects of NAT? Is there a convincing approach to network and application security that allows policies appropriate to each application, unlike firewalls?
1 2 3 4 5 6 7 8
applications, user requirements server resources, network resources, and universal connectivity How do the principles of requirements engineering apply to this domain? What are the principles for
- rganizing “uses” hierarchies of
- verlays (including middleware)?
How can overlays be implemented independently and composed at run-time? What are the best techniques for specification and verification of
- verlay software?
When and how is it safe to
- ptimize by collapsing layers?
SLIDE 22
A A
A A A A
INTERNET EVOLUTION AND THE ROLE OF SOFTWARE ENGINEERING: OUTLINE
WHAT IS THE STATE OF THE
INTERNET, AND WHY SHOULD SOFTWARE ENGINEERS BE CONCERNED?
WHAT CAN SOFTWARE-
ENGINEERING RESEARCHERS DO ABOUT IT? The Internet is not application-friendly. Every segment of society has a stake in the Internet, . . . . . . and this limits its usefulness to all of them. Businesses, networking researchers, and the IETF are not making progress toward an application-friendly Internet, . . . . . . so software researchers have to do it. There are research opportunities in plenty. NO SOFTWARE SYSTEM IS MORE
IMPORTANT THAN THE INTERNET.
SLIDE 23