VR everywhere consumer virtual reality for desktop, mobile and web - - PowerPoint PPT Presentation

VR everywhere

consumer virtual reality for desktop, mobile and web

tony parisi march, 2015

about me

http://www.tonyparisi.com 3/16/15

CONTACT

tparisi@gmail.com skype: auradeluxe http://twitter.com/auradeluxe http://www.tonyparisi.com/ http://www.learningwebgl.com/

GET THE BOOKS!

WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-Applications-HTML5- WebGL-Visualization/dp/1449362966

MEETUPS

http://www.meetup.com/WebGL-Developers-Meetup/ http://www.meetup.com/Web-VR/

BOOK CODE

https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications

GET GLAM

http://www.glamjs.org/ https://github.com/tparisi/glam/ WORK http://www.thirdeye.gl/ CREDS Co-creator, VRML and X3D

the promised land! consumer VR

3/16/15 http://www.tonyparisi.com

hardware explosion

3/17/15 http://www.tonyparisi.com

headsets input devices high performance laptops AFFORDABLE STEREOSCOPIC DISPLAYS, CRAZY-FAST GPUS, CHEAP MOTION TRACKING

IMUS… FTW!

PC HMDs

3/17/15 http://www.tonyparisi.com

Provide deepest, most immersive VR experiences

Desktop peripheral - acts as external monitor Position and orientation tracking Tethered – one or more cables to the computer; camera for position tracking

Oculus Rift – the Gold Standard

• f VR headsets

Developer Kit CHEAP $350 In production late 2015/early 2016

HTC/Valve VIVE OSVR – Razer open dev kit and SDK; ecosystem initiative

mobile HMDs

3/17/15 http://www.tonyparisi.com

Deluxe Mobile – Samsung GearVR

Optics and IMU licensed from Oculus Custom Android build w/ VR extensions and enhanced UI The best VR experience out there! (Look Ma, no wires) BUT $200 + requires new Phone (Note 4) NOT CHEAP $1,000 Support for S6 coming, price coming down Orientation tracking only So far… no competition… but EXPECT IT!

drop-in phone viewers

3/17/15 http://www.tonyparisi.com

Google Cardboard

Just Add SmartPhone Orientation tracking only Open specification for mobile VR Standard Android OS support, using simple stereo rendering and accelerometer tracking $2 in parts or $25 ready-to- assemble Kits WAY CHEAP Not as nice… but great for the price!

MergeVR, Wearality, Moggles…

Some of these have different rendering thus require custom SDK

• perating systems

3/16/15 http://www.tonyparisi.com

Desktop – Oculus SDK supports Windows/Mac/Linux; Vive SDK not yet released… GearVR - custom Android build w/Oculus mobile SDK Cardboard

Google has SDK for native Android with simple stereo render and head tracking; also adding first-class VR support into future OS Cardboard is just side-by side rendering and accelerometer tracking… so it works on iOS too Mobile browsers can run Cardboard apps! Just stereo render with WebGL and track w/DeviceOrientation API

The Oculus Store for GearVR

rendering

3/16/15 http://www.tonyparisi.com

Oculus Rift/GearVR

110 diagonal FOV Side-by-side stereo Barrel distortion

Cardboard

90 degree FOV Side-by-side stereo

head tracking

3/16/15 http://www.tonyparisi.com

Oculus Rift/GearVR

1000hz refresh rate

Cardboard

200hz refresh rate

Desktop IMUs built into HMDs track position and orientation. Use camera or wall-mounted sensors for position tracking Mobile IMUs track orientation only

INERTIAL MEASURE UNITS (IMUS)

the challenge: “presence”

3/16/15 http://www.tonyparisi.com

Camera must respond to head tracking at 75FPS+

higher latency risks motion sickness and breaks the illusion

Rendering, as always, must be at 60FPS+ “Motions-to-photons” latency goal is <20ms Desktop and GearVR can hit this, currently mobile can’t (though it’s in the works); most phone IMU motions-to-photons are are 50ms+ What’s wrong with this picture???

Classic targeting of 60FPS makes rendering the bottleneck If targeting 75FPS, that’s minimum 150FPS simulation (75FPS x 2 cameras), or <7ms frame time!!!

performance (1)

3/16/15 http://www.tonyparisi.com

Software techniques

Of course push everything you can onto the GPU New engine optimizations

Example: render each state change to each camera

• vs. whole scene to each camera

Mobile: create experiences with lower res than typical games... till the hardware gets faster and resolutions get higher; low latency trumps hi poly count

Time Warp

Oculus SDK technique, transforms image in Z-buffer at next refresh based on new IMU position/

• rientation… even if new rendered frame isn’t ready

Already supported in game engines and other runtimes

performance (2)

3/16/15 http://www.tonyparisi.com

First attempts at a comprehensive GPU approach to VR - NVIDIA VR Direct

New features enabled in Maxwell processors (GeForce GTX 980 and GTX 970) Overall attack on latency – whole pipeline Asynchronous Warp (NVIDIA’s version of Time Warp) Dynamic Super Resolution (DSR) – up to 4 megapixels per eye downsampled to HD Multi-frame sampled anti-aliasing (MFAA) Boasts motion-to-photons latency of 25ms … plus “GeForce Experience” control panel –

• ne end-user settings dialog instead of

several

ACCELERATING VR USING THE POWER OF THE GPU

developing desktop and mobile VR

3/16/15 http://www.tonyparisi.com

Game engines

Unity3D – targets all platforms: desktop Oculus, Gear VR, Cardboard. Free and open prefabs available on Unity store and from HMD vendors. Expect Vive and other platforms quickly; they move fast. Unreal - desktop now, mobile in development

Native SDKs

Oculus SDK C++ Oculus Mobile SDK C++ Android SDK for Cardboard

WebVR: fast, cheap and democratized

3/17/15 http://www.tonyparisi.com

• Experimental browser API
• Query browser for attached

HMD devices

• Render fullscreen stereo in

WebGL

• Track HMD position/
• rientation by polling HMD

device

• Already in Firefox and

Chrome nightly builds, uses same API

Build apps using open source JavaScript engines, or WebGL “exported” from Unity/Unreal via Emscripten cross-compiler Browser refresh currently throttled at 60FPS but they are lifting this limitation

Quake 3 WebVR demo, developed by Brandon Jones of Google http://media.tojicode.com/q3bsp/

NO-DOWNLOAD VR APPLICATIONS. INSTANT ACCESS IN THE BROWSER, SHARE IT WITH A URL! WRITTEN IN JAVASCRIPT!?

input

3/17/15 http://www.tonyparisi.com

We still don’t have the mouse

• f VR

State of the practice is blind use of keyboard or gamepad controller Motion controllers like Leap Motion, Sixense STEM system are gaining popularity for PC-based VR GearVR has a really innovative trackpad design, plus simple, elegant navigation UI Drop-in phone viewers are experimenting with side- mounted click devices and Bluetooth controllers

Leap Motion controller Sixense STEM System

image: https://www.attachmate.com/blogs/legacymodernization/legacy-data-the-elephant-in- the-room/

augmented reality

3/17/15 http://www.tonyparisi.com

Explosion of VR has renewed interest in AR Promising new hardware

Microsoft Hololens CastAR - table-top AR display for gaming, close to coming to market Magic Leap ????

A little farther out… most projects don’t have ship dates Another SDK babel looming…

THE NEXT NEXT BIG THING?

Hololens Sculpting Tool Demo CastAR Multi-Player Game

coming in May 2015

3/16/15 http://www.tonyparisi.com

keep in touch

http://www.tonyparisi.com 3/16/15

CONTACT

tparisi@gmail.com skype: auradeluxe http://twitter.com/auradeluxe http://www.tonyparisi.com/ http://www.learningwebgl.com/

MEETUPS

http://www.meetup.com/WebGL-Developers-Meetup/ http://www.meetup.com/Web-VR/

BOOK CODE

https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications

GET GLAM

http://www.glamjs.org/ https://github.com/tparisi/glam/ WORK http://www.thirdeye.gl/ CREDS Co-creator, VRML and X3D

GET THE BOOKS!

WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-Applications-HTML5- WebGL-Visualization/dp/1449362966

VR everywhere

consumer virtual reality for desktop, mobile and web

tony parisi march, 2015