Virtual Environments: Networked Virtual Environments Anthony Steed - - PowerPoint PPT Presentation

Virtual Environments: Networked Virtual Environments

Anthony Steed

Department of Computer Science University College London http://www.cs.ucl.ac.uk/teaching/VE

• Part I: Goals
• Part II: The Internet
• Part III: Strategies
• Part IV: Latency and Scalability

2

3

DIVE “London Demonstrator”, UCL & colleagues 1999

4

IEEE Virtual Reality 2009 programme committee meeting, autumn 2008

5

Burnout Paradise, Electronic Arts

Nature of a Networked Virtual Environment

• Multiple user – geographically distributed
• More types of interactions

– Passive navigation – Single object/multiple user – Multiple objects/multiple users

• Voice link between participants
• Heterogeneous immersion devices

6

Demands

• Provide up-to-date info to all participants

– Users location – Support Interactions

• Minimal delay update
• Support multiple message type

– Simple state – Voice/video

7

Consistency and Plausibility

• Goal is the illusion of a consistent shared state
• Local plausibility is the appearance of consistency of
• nly local actions (e.g. physics & collision detection

appears to work)

• Shared plausibility is the appearance of properties

being the same as observed by users

– Objects that are in the background need not be consistent – Further: only things that might be the focus of joint attention can be discussed and be different

• A local implausibility might be an obvious thing to

talk about!

8

• Part I: Goals
• Part II: The Internet
• Part III: Strategies
• Part IV: Latency and Scalability

9

Introduction

• Networks at the heart of networked VR
• Many network protocols are out there

– TCP, UDP, multicast, RTP, etc – Choice based on needs

• Properties of the Internet

10

Internet Protocol - IP

11

Application Transport Network Link Physical DHCP, DIS, DNS, FTP, HTTP, IMAP, RTP, SMTP, SSH, Telnet TCP, UDP, RSVP IP, ICMP, IGMP Ethernet, 802.11, ADSL copper wires, fibre-optic cable, radio waves

Internet Protocol (IP)

• Any particular inter-connect might guarantee

delivery

– Routers can get congested, for example when incoming links carry more traffic than outgoing – Dropping packets is the right solution to congestion!

• Thus IP is a best effort protocol
• Every node an internet must understand IP
• Every node has an IP (IPv4) address of the form

– 128.16.6.8 (32 bits)

12

Routing

13

Application Transport Network Link Application Transport Network Link Network Link Network Link

Transport Protocols: UDP

• Connectionless

– No state is maintained – No setup time – Light-weight

• No Quality of Service

– No guarantee delivery – No order delivery

Postal Service

TCP

• Connection Oriented

– State is maintained at routers – Setup Handshake – Heavy-weight

• Quality of Service

– FIFO delivery – Flow Control – Error Recovery

• Mechanisms

– Timers, Acknowledgements, etc

15

16

Transport Network Link Transport Network Link Application Application

Source Port = 1901 Destination Port = 15001 narok (128.16.13.118 ) seychelles (128.16.3.52 ) Source Port = 1220 Destination Port = 15000

Network properties

• Latency (Round Trip Time)

– Devices take time to send data (e.g. Modems) – Data takes time to transmit (speed of light)

• Jitter

– Routers insert bandwidth

• Bandwidth (Capacity)

– Bandwidth costs money – 8Mbps is fairly standard @£8/month

• Loss (Congestion, Reliability)

– Routers drop packets, links do go down, routes do fluctuate

17

Interpacket arrival time Frequency of occurrence Correct spacing Gaussian distribution Observed distribution

Variance of inter-packet arrival times

Application Programming: Socket

• End-point for communication
• Network connections based on sockets
• Socket identified by

– Host address – numeric or name – Port number

• 0-1023 : well known services (e.g. http traffic on 80)
• 1024-49151: registered ports (less well known)
• 49152-65536: private applications or dynamic use

19

Socket communication

20

Listener Needs to create a socket to listen to incoming connection requests Client Needs to create a socket to connect to listening sockets

The Client/Server logic

21

Server Client

Create socket for listening incoming connections Create socket for connecting to Server Listen Request connection Accept and create thread Send/Receive message Send/Receive message

XVR functions

22

Server Client

NetCreateChannelTCP( ) NetConnectTCP( ) NetAcceptTCP ( ) Accept and create thread NetSendToTCP() / NetReceiveFromTCP() NetSendToTCP() / NetReceiveFromTCP()

• Part I: Goals
• Part II: The Internet
• Part III: Strategies
• Part IV: Latency and Scalability

23

Architectures

• Peer-to-Peer
• Client/Server
• Hybrid

24

Simple Peer-to-Peer

25

• Total replicated database
• Other user’s input treated locally like a second set of

keyboard and mouse

• Simple extension to single users
• One example is DOOM from id Software

DB DB

Multiple Peer-to-Peer

26

• Decoupled da

Client-Server Architecture

27

Client Server

• Database on Server with Shadow Copy on Clients
• Current Online Games
• Some VE (MASSIVE,Deva)

DB

Client-Server Architecture

28

Server Client Client Client Client

Hybrid Architectures

29

• Multiple servers serving different regions
• Multiple service types & service layers

DB

Server pool

How do these architecture measure up?

• Up-to-date information

– Synchronisation in peer-to-peer impossible – no absolute reality!

• Interaction

– Client/Server provides central locus of control – constraint resolution

• Consistency!

30

Consistency : Reality

• GOLDEN RULE
• Information propagation IS NOT instantaneous

31

It is not possible for EVERY user to share the EXACT same state at EVERY instance

State Consistency: Case 1 (Total Sync)

32

T = t Acknowledge every update Propagation delay is 100ms Client A Client B

State Consistency: Case 1

33

Client A Client B T = t + 50ms

State Consistency: Case 1

34

Delta T Client A Client B T = t + 50 ms + 100 ms

State Consistency: Case 1

35

T = t + 50ms + 100ms + 50ms + 100ms T = t + 300ms After 300ms Client A may move again!!! Client A Client B Delta T

State Consistency: Case 2 (Async)

36

T = t Local update every 150ms Propagation delay is 100ms Client A Client B

State Consistency: Case 2

37

T = t + 100ms Client A Client B

State Consistency: Case 2

38

T = t + 150ms Client A Client B

State Consistency: Case 2

39

T = t + 250ms Client A Client B Delta T

State Consistency: Case 2

40

T = t + 300ms Client A Client B

State Consistency: Case 2

41

T = t + 400ms Inconsistency!!! Client A Client B Delta T Delta T

State Consistency: Problem

• Would like to send data at the rate it is changing
• Network is congested → retransmission

– BUT this makes congestion more likely

42

Consistency Update Frequency

State Consistency: Total Consistency

Provided by Client/Server Architecture

• Server controls total consistency of

database

• Communication is reliable
• Each Client reads from Server
• Client sends updates to Server (dirty

cache)

• Events are totally ordered

43

State Consistency: Total Consistency

Pros

• Guaranteed ordered event processing
• Implicit ownership

Cons

• Latency penalty: 2 Round Trip Time
• Reliability not always compatible with real-time (msg +

ack)

• Poor scalability

– Server is bottleneck

• Server is single point of failure

44

State Consistency: Eventual Consistency

• Clients just send periodic updates
• Discrepancy is tolerated if:

– Does not exceed a given time threshold – Does not dramatically a perceptual error threshold

• Clients do not need acknowledgements
• A client is unawares of other clients
• Inconsistency inversely proportional update

frequency

45

State Consistency: Eventual Consistency

• Pros:

– Minimum latency – No infrastructure -> reduced processing overhead – Simple to upgrade a single user system to multi users – No bottleneck – Fault tolerant (no packets -> remove entity)

• Cons:

– Explicit ownership – Bandwidth hungry (high frequency for low inconsistency) – Better scalability but still poor

46

State Consistency: Predicted Consistency

Facts:

• Users are not aware of other users
• Total consistency is not a requirement
• But eventual consistency has problems

47

Possible solution:

• Dead reckoning
• High level behaviours

State Consistency: Predicted Consistency

• Dead reckoning I

48

Client A Client B Ghost A Ghost B Movement Model: P(t) = P(t0) + v*t P(t) = P(t0) + v*t + 0.5 * a * t2

State Consistency: Predicted Consistency

• Dead reckoning II

49

Client A Simulation If new pos does not exceed error threshold then do nothing If new pos does exceed error threshold then send new position

State Consistency: Predicted Consistency

• Dead reckoning III - convergence

50

T = t

OR

T = t T = t+1 T = t+2

State Consistency: Predicted Consistency

• High level behaviours (Games)

51

Position A Position B

• At each game interval send current position packet
• If path takes 30 seconds and game tick is 10 per second
• You have 300 packets being sent

State Consistency: Predicted Consistency

• High level behaviours (Games)

52

Position A Position B

• Just send Destination Position B
• Use path finding to compute best path
• Not all users will see same path
• But all will see final result
• Send 1 packet only with parameters

Summary

• Networked VR

– allows new possibilities for interaction – Poses new challenges to VR designers

• Three types of architectures exist and their

suitability depends on the target application

• Consistency is at the heart of networked VR

53

• Part I: Goals
• Part II: The Internet
• Part III: Strategies
• Part IV: Latency and Scalability

54

Issues

• Issues of Scale
• Area Of Interest Management (AOIM)
• Spatial Decomposition
• Functional and Temporal Decomposition
• Discussion

Issues of Scale

AOIM: Goals

• A user is NOT an omniscient being
• A user is NOT interested in every event
• A client is NOT able to process everything
• Just give each client enough to fool the

user

• Interest: visibility, awareness, functional, …
• Network and computational constraints

• Spatial Decomposition

– Based on locaton

• Functional Decomposition

– Based on function – Army fleet – airplanes

• Temporal Decomposition

– Based on update requirement (realtime vs delayed)

Environmental Decomposition

Spatial Decomposition

• Static Grid
• Hierarchical Grid
• Locales
• Aura
• Region
• Filtering
• Nearest Neighbours

AOIM: Static Grid

• 1 Cell = 1 Group
• Hexagon regular shape
• Tied into the grid – static
• Send current cell
• Receive neighbours
• Any architecture

(distributed)

• NPSNET (DIS)

AOIM: Hierarchal Grid

• 1 Cell = 1 Group
• Square cells
• Send current cell
• Receive current cell
• Any architecture

(distributed)

• Exceeds threshold, expand

Threshold = 5

AOIM: Locales

• 1 locale = 1 group
• Locale is arbitrary shape
• Locale placement is static
• Associated transform

matrix

• Any architecture

(distributed)

• MERL

AOIM: Locales

• 1 locale = 1 group
• Locale is arbitrary shape
• Locale placement is static
• Associated transform

matrix

• Any architecture

(distributed)

• MERL

AOIM: Locales

• 1 locale = 1 group
• Locale is arbitrary shape
• Locale placement is static
• Associated transform

matrix

• Any architecture

(distributed)

• MERL

AOIM: Aura

• Nimbus, Aura, Focus
• Auras intersect = group
• Focus, Nimbus =

awareness

• Aura Manager
• Client/Server
• MASSIVE

Nimbus Focus Aura

AOIM: Filtering

B

• Line of sight
• Entity visible = group
• Client/Server

A C

AOIM: Nearest Neighbours

• 1 group = quorum
• Computational/Network

constraints

• Client/Server
• Community Place

AOIM: Nearest Neighbours

• 1 group = quorum
• Computational/Network

constraints

• Client/Server
• Community Place

Distance function Layered nimbus

Level Of Detail

• Integrate distance
• 1 group = 1 send rate
• Layered multicast

Summary

• Not possible to send everything about everything
• Area of interest management

– Spatial, functional and temporal

• Discussion