Introduction to the XVR technology: the basic framework Franco - - PowerPoint PPT Presentation

Introduction to the XVR technology: the basic framework

Franco Tecchia franco@sssup.it

06/20/06 2

Why you should NOT use XVR

• It only works under Windowz

It only works under Windowz

• The kernel of XVR is closed source

The kernel of XVR is closed source

• Being developed around a scripting language, XVR is

Being developed around a scripting language, XVR is much slower than C++ for hard-core tasks (array sorting) much slower than C++ for hard-core tasks (array sorting)

• If you are writing some special module and need hard-core

If you are writing some special module and need hard-core access to the Win32API you need C/C++ access to the Win32API you need C/C++

• XVR has (yet) very poor debugging tools compared to C++

XVR has (yet) very poor debugging tools compared to C++

• Documentation is still weak

Documentation is still weak

• You are in love with C no matter what

You are in love with C no matter what

06/20/06 3

Why you should try XVR

• It’s free

It’s free

• It drastically and consistently reduce development time of

It drastically and consistently reduce development time of an application an application

• It lower the cultural barrier usually associated to develop

It lower the cultural barrier usually associated to develop VR applications. VR applications.

• It’s self-sufficient, very simple to install, very simple to use

It’s self-sufficient, very simple to install, very simple to use

• It’s a good common platform to work in teams

It’s a good common platform to work in teams

• You can show your applications on the web

You can show your applications on the web

• Your students will love it

Your students will love it

06/20/06 4

Please note: there are alternatives

• VirTools (Dassault Systems)

VirTools (Dassault Systems)

• Quest 3D (Act – 3D)

Quest 3D (Act – 3D)

06/20/06 5

XVR: An IDE for VR applications

06/20/06 6

XVR Architecture

Virtual Machine Bytecode execution (Multi-Threaded) HI- Performance Real-Time Scene-Graph Audio – Video Library VR Device Handling Real- Time Physics Network Renderer Network Manager

06/20/06 7

XVR Workflow

Compiler

Interpreter (Virtual Machine) Dedicated scripting language Output Binary ByteCode

06/20/06 8

What does XVR provide

• A complete and very fast 3D Scene Graph Manager

A complete and very fast 3D Scene Graph Manager

• Low level OpenGL wrapping for added flexibility

Low level OpenGL wrapping for added flexibility

• Vertex and pixel-shaders (OpenGL2.0 GLSL)

Vertex and pixel-shaders (OpenGL2.0 GLSL)

• Basic GDI and 2D GUI functions (menus, text)

Basic GDI and 2D GUI functions (menus, text)

• Sound management: MIDI, MP3, 3D Positional Sound

Sound management: MIDI, MP3, 3D Positional Sound

• Collision detection & Real-time Physics

Collision detection & Real-time Physics

• Haptic devices control

Haptic devices control

• Network data management

Network data management

• Basic streaming support

Basic streaming support

• Support for advanced devices: Trackers, Stereo

Support for advanced devices: Trackers, Stereo projection, HMDs, Haptic Interfaces, CAVE projection, HMDs, Haptic Interfaces, CAVE

06/20/06 9

Why: from demos to complex applications

• Demos as single-minded applications

Demos as single-minded applications

• Very limited interaction

Very limited interaction

• Problems can be ignored or postponed

Problems can be ignored or postponed

• Most of the work to pass from concept

Most of the work to pass from concept demo to application stage demo to application stage

• We learn from our work and our students

We learn from our work and our students Features VS. Complexity Features VS. Complexity Linear increase Exponential grow Linear increase Exponential grow

06/20/06 10

The simplest architecture: the single loop

One loop to manage everything: One loop to manage everything:

• Graphic Rendering

Graphic Rendering

• User Input

User Input

• Animation

Animation

• Collision detection

Collision detection

• Physical simulation

Physical simulation

~ 60 Hz

Refresh rate is usually the upper bound Refresh rate is usually the upper bound Collision and physics can give troubles Collision and physics can give troubles

06/20/06 11

Second option: separate rendering from the rest

• Animation independent from frame rate
• Animation independent from frame rate
• No more missed collisions!
• No more missed collisions!

~ 60 Hz 100-200 Hz

• Variables need to be shared between processes
• Variables need to be shared between processes
• Atomicity should be carefully addresses

Atomicity should be carefully addresses

• Two threads, one rendering context?

Two threads, one rendering context?

06/20/06 12

Third step: Haptic Devices get into the picture

• The Haptic Loop is *very* fast
• The Haptic Loop is *very* fast
• More representations for the same objects
• More representations for the same objects

~ 60 Hz

• Problem: How does Haptics influence Physics?
• Problem: How does Haptics influence Physics?

100-200 Hz >1kHz

06/20/06 13

Going further: loops are everywhere

• Network lag and dead reckoning
• Network lag and dead reckoning

~ 60 Hz 100-200 Hz >1kHz ~10Hz

• Can’t you poll for your
• Can’t you poll for your

tracking? tracking?

~30Hz

06/20/06 14

Events: loops can’t solve everything ~30Hz

VR_EVENT_1 VR_EVENT_1 VR_EVENT_3 VR_EVENT_3 VR_EVENT_2 VR_EVENT_2 VR_EVENT_4 VR_EVENT_4

06/20/06 15

The XVR approach: simple but not too simple ~ 60 Hz 100-1000 Hz

3 Main Call-backs 3 Main Call-backs OnFrame() OnFrame() OnTimer() OnTimer() All variables are All variables are shared shared OnEvent() OnEvent()

VR_EVENT_1 VR_EVENT_1 VR_EVENT_3 VR_EVENT_3 VR_EVENT_2 VR_EVENT_2 VR_EVENT_4 VR_EVENT_4

06/20/06 16

Why is XVR useful in VR development

• It’s a complete programming environment

It’s a complete programming environment

• A good implementation of what we need in our daily work

A good implementation of what we need in our daily work

• One single programming language (and it’s VR-oriented)

One single programming language (and it’s VR-oriented)

• Automatic handling of the infrastructure stuff

Automatic handling of the infrastructure stuff

• Plenty of built-in advanced function (Including GLSL

Plenty of built-in advanced function (Including GLSL support) and low-level OpenGL support for Graphics support) and low-level OpenGL support for Graphics Hackers Hackers

• It’s extendable: If you have special need, you can easily

It’s extendable: If you have special need, you can easily load and use external DLLs load and use external DLLs

06/20/06 17

The Language

06/20/06 18

The XVR scripting language (.S3D)

• It’s the fundamental glue that

It’s the fundamental glue that keeps everything together keeps everything together

• Similar syntax to JAVA and

Similar syntax to JAVA and BASIC BASIC

• Byte code – virtual machine

Byte code – virtual machine model model

• Type less

Type less

• Object oriented

Object oriented

06/20/06 19

Does it have an impact on performances?

• No scripting language is as fast as C or C++

No scripting language is as fast as C or C++

• XVR uses bytecode generation/execution –speed similar to

XVR uses bytecode generation/execution –speed similar to JAVA JAVA

• A script-only function (es Quicksort) is about 20 times

A script-only function (es Quicksort) is about 20 times slower than a C++ equivalent slower than a C++ equivalent

• Most of CPU/GPU workload outside the scripting language

Most of CPU/GPU workload outside the scripting language anyway anyway

• In practical application we experimented very small to no

In practical application we experimented very small to no impact impact

• A detailed overview of the virtual machine later

A detailed overview of the virtual machine later

06/20/06 20

But there are advantages

• Simplified syntax: script constructs tailored on VR needs

Simplified syntax: script constructs tailored on VR needs

• Light, self sufficient development environment -> easy to

Light, self sufficient development environment -> easy to set-up set-up

• Not much framework jargon: straight to the business

Not much framework jargon: straight to the business

• VERY fast compilation time (almost 0)

VERY fast compilation time (almost 0)

• Compact bytecode, web - enabled

Compact bytecode, web - enabled

• Dominant technologies are doing the same: see VirTools

Dominant technologies are doing the same: see VirTools

06/20/06 21

S3D Statements

IF..ELSE WHILE DO..WHILE FOR BREAK CONTINUE SWITCH..CASE SET

SCENE_FOV Field of view SCENE_NEAR Near clipping plane distance SCENE_FAR Far clipping plane distance AUDIO_MODE 3d Audio Setup mode

06/20/06 22

Preprocessor commands

#DEFINE #IFDEF #ENDIF #ELSE #INCLUDE #PRAGMA Preprocessor directives, such as #define and #ifdef, are typically used tomake source programs easy to change and easy to compile in different execution environments. Directives in the source file tell the preprocessor to perform specific actions. For example, the preprocessor can replace tokens in the text, insert the contents of other files into the source file, or suppress compilation of part of the file by removing sections of text. Preprocessor lines are recognized and carried out before macro expansion.

06/20/06 23

Macros

06/20/06 24

Functions

• Same as usual

Same as usual

• Parameter type is not specified

Parameter type is not specified

• Type of the return value is not specified

Type of the return value is not specified

06/20/06 25

S3D Variables

VAR i; VAR i; VAR j,k; VAR j,k; VAR x = 2; VAR x = 2; VAR y = 3.1; VAR y = 3.1; VAR s = “Don’t panic !!!” VAR s = “Don’t panic !!!” k = 4.3; k = 4.3; i = 8; i = 8; i = 9.2; i = 9.2; VAR v = [0.0, 1.0, 0.0]; VAR v = [0.0, 1.0, 0.0];

static var a = 10; … a = 7; … reset a; // a = 10

• Variable type is not explicitly declared

Variable type is not explicitly declared

• The same variable can assume different types as the

The same variable can assume different types as the program evolve program evolve

• The construct “reset” can be useful!

The construct “reset” can be useful!

06/20/06 26

S3D Types

INT, FLOAT, BOOLEAN, STRING VECTOR: (of floats - monodimensional)

Var1 = [ 0.2 , 2 , 3.33 , 4 ]; Var2 = Vector(16); // create a 16-elements vector Var3 = Var1[0]; // Var3 equals 0.2

MATRICES: Although not directly supported, often XVR functions return, or accept as input, 16-dim vectors representing matrices. In this case the matrix is stored in column-major order, like in OpenGL.

06/20/06 27

ARRAY

An Array is a set of elements which can be accessed through an index. The elements of a S3D array can be of different type. The array size can be changed at run-time. To specify the index in order to access the desired element, the operator [ ] is used.

Var1 = { 1, true, { 12, 3 }}; // Array made of different types Var2 = Array(12); // Create an array of 12 elements Var3 = Var1[0]; // Var3 equals 1 Var4 = Var1[2][0]; // Var4 equals 12 Var1[1] = false; // Now Var1[1] equals "false"

06/20/06 28

Classes

• S3D is heavily object-oriented
• Built-in objects
• Script-based classes
• Imported objects (DLLs)

06/20/06 29

Class declaration example

class COutput { print(p1); println(p1); }; class Engine: COutput { var m_fuel; var m_power; Init( f, p); Info(); ShowPower(); }; function Engine::Engine( f, p ) { Init(f,p); } function Engine::~Engine( f, p ) { m_fuel = m_power = void; } function Engine::Init( f, p ) { m_fuel = f; m_power = p; }

Classes use a C++-like sintax Simple declaration Declaration with inheritance

• Variable
• Method

Constructor (Optional) Destructor (Optional) Method implementation

06/20/06 30

Class declaration & constructors sequence

class Engine: COutput { var m_fuel; var m_power; Init( f, p); Info(); ShowPower(); }; function Engine::Engine( f, p ) { Init(f,p); } function Engine::Info() { print( "Engine fuel: "); println(m_fuel); ShowPower(); } function Engine::ShowPower() { print( "Engine power: "); print(m_power); println(" HP"); }

The constructor of the class Engine is equivalent to:

function Diesel::Diesel( f, p, n ) { Engine( f, p ) m_NumInjector = n; }

class Diesel : Engine { var m_NumInjector; ShowPower(); }; function Diesel::Diesel( f, p, n ) { m_NumInjector = n; } function Engine::ShowPower() { print( "Engine power: "); print(m_power); println(" HP"); }

• Parameters passed to the constructor
• f the derived class are passed to the

constructor(s) of the base class

06/20/06 31

Multiple inheritance is supported

Class Veicle: Engine, Bodywork { var m_Desc; var m_NumWeels; var m_MaxVel; Info(); }; function Veicle::Veicle( Desc, Weels, MaxVel, EngineFuel, EngineHP, BodyColor, BodyStuff ) { m_Desc = Desc; m_NumWeels = Weels; m_MaxVel = MaxVel; Engine::this.Init( EngineFuel, EngineHP ); Bodywork::this.Init( BodyColor, BodyStuff ); }

Multiple inheritance, comma separated Calling a base class method

06/20/06 32

All the methods are Virtual!

Example:

var e = Engine( "Gas", 55); e.info(); Output: Engine fuel: Gas Engine power: 55 HP var d = Diesel( "Gas Oil", 100, 4); d.info(); Output: Engine fuel: Gas Oil Number Injector: 4 Engine power: 74.5 KWatt

class Engine: COutput { var m_fuel; var m_power; Init( f, p); Info(); ShowPower(); }; function Engine::Engine( f, p ) { Init(f,p); } function Engine::Info() { print( "Engine fuel: "); println(m_fuel); ShowPower(); } function Engine::ShowPower() { print( "Engine power: "); print(m_power); println(" HP"); } class Diesel : Engine { var m_NumInjector; ShowPower(); }; function Diesel::Diesel( f, p, n ) { m_NumInjector = n; } function Diesel::ShowPower() { print( "Number Injector: "); println (m_NumInjector); print( "Engine power: "); print(m_power*0.745); println(" KWatt"); }

06/20/06 33

S3D classes: accessing functions by name

Any S3D class has these member functions built-in: __GetName(): returns the name of the class __Call( “name“, param1, param2, ...): calls the member function named „name“ __Set( “namevar“, value): set the value of a member variable __Get( “namevar“ ): get the value of a member variable

PRO: maximum flexibility PRO: Function name can be decided at run time

• For(i=0; i<8; i++)

if(joystick.__call( “Button“+str(i) ) ) ...

CONS: low efficiency. HASH table access.

06/20/06 34

S3D classes: accessing functions by name

PRO: very flexible PRO: Function name can be decided at run time

• For(i=0; i<8; i++)

if(joystick.__call( “Button“+str(i) ) ) ...

CONS: low efficiency. HASH table access.

06/20/06 35

Operators

06/20/06 36

Swizzles

Swizzles allow to access the single components of a vector or to generate a new vector with the components exchanged. <VECTOR NAME>.<SWIZZLE> x : first component y : second component z : third component w : fourth component _ or 0 : ignore component v1 = [ 1, 100, -10, -100]; v1.zwx corresponds to [ -10, -100, 1] v1.xxxy corresponds to [ 1, 1, 1, 100 ] v1.x_z or v1.x0z correspond to [ 1, 0, -10 ] v2 = [ -2, 20, 200, -20 ]

• ne component change:

v2.x = 5 --> v2 = [ 5, 20, 200, -20 ] multiple components change: v2.wy = [ 3,4 ] --> v2 = [ -2, 4, 200, 3 ] multiple components change with no changes on some of them: v2.x_z = [ 3,4,5 ] --> v2 = [ 3, 20, 5, -20 ]

06/20/06 37

Programming with XVR

06/20/06 38

How is an application organised?

• Any XVR application is based

Any XVR application is based

• n 6 fundamental function (built
• n 6 fundamental function (built

in in all the scripts) in in all the scripts)

• Currently there are 2 threads

Currently there are 2 threads calling these functions. There calling these functions. There could be more in the future. could be more in the future.

• Atomicity of the operations is

Atomicity of the operations is

• guaranteed. Global variables
• guaranteed. Global variables

are properly shared between are properly shared between threads threads

06/20/06 39

OnDownload()

• Every time an XVR script is

Every time an XVR script is executed it creates a temporary executed it creates a temporary working directory (inside the user working directory (inside the user temp) temp)

• OnDownload is used to move files

OnDownload is used to move files from their original location to the from their original location to the working directory working directory

• Zip Files are automatically exploded

Zip Files are automatically exploded

• Generic code can be executed

Generic code can be executed inside OnDownload inside OnDownload

06/20/06 40

OnInit()

• Once the download is complete, this

Once the download is complete, this is the right place for initialization code is the right place for initialization code

• Please note that OnInit accept a

Please note that OnInit accept a “params” value. This is the “params” value. This is the “UserParam” string reported in the “UserParam” string reported in the HTML code HTML code

• There is an implicit OpenGL

There is an implicit OpenGL “SwapBuffer” when OnInit ends “SwapBuffer” when OnInit ends

06/20/06 41

OnFrame()

• Function dedicated to rendering. By

Function dedicated to rendering. By default it gets called after each screen default it gets called after each screen refresh (60 Hz). refresh (60 Hz).

• Desired framerate can be changed

Desired framerate can be changed using SetFramerate(Value) using SetFramerate(Value)

• It’s the ONLY function that has

It’s the ONLY function that has access to the OpenGL frame buffer access to the OpenGL frame buffer

• All the rendering code must be inside

All the rendering code must be inside this function (or inside functions this function (or inside functions called by OnFrame) called by OnFrame)

• The maximum Frame Rate allowed

The maximum Frame Rate allowed is 100 Hz is 100 Hz

06/20/06 42

OnTimer()

• Function normally dedicated to

Function normally dedicated to mange an high speed loop. By default mange an high speed loop. By default it gets called at 100Hz it gets called at 100Hz

• Desired time-step can be changed

Desired time-step can be changed using SetTimestep(Value) using SetTimestep(Value)

• Minimum time-step = 1ms

Minimum time-step = 1ms

• Independent from framerate (unless

Independent from framerate (unless the system is overloaded) the system is overloaded)

• Excellent place for updating objects

Excellent place for updating objects positions, sound, real time physics, positions, sound, real time physics, haptic interfaces and network haptic interfaces and network programming. programming.

• It has no access to the framebuffer

It has no access to the framebuffer

06/20/06 43

OnEvent()

• Function under construction, coming soon

Function under construction, coming soon

• Used for Event-driven programming

Used for Event-driven programming

• It gets called when an EVENT arrives

It gets called when an EVENT arrives

• EVENTS can be generated inside XVR

EVENTS can be generated inside XVR (GenerateEvent(val, param)) or outside XVR (The (GenerateEvent(val, param)) or outside XVR (The Win32 Events will be intercepted) Win32 Events will be intercepted)

• Extremely useful in state machines programming

Extremely useful in state machines programming

06/20/06 44

The typical rendering code

• Handling Mouse (or keyboard) input
• SceneBegin(), SceneEnd(), a very

common XVR graphical construct useful for multipass rendering, picture in picture, stereo rendering

• Mixing low-level calls to scene graph

rendering

06/20/06 45

The XVR Scene graph

Scene Scene

Obj Obj

Mesh Mesh

Obj Obj

Billboar Billboar d d

Obj Obj Obj Obj Obj Obj

Mesh Mesh

Obj Obj

Callback Callback

• A flexible built-in scene graph
• Plenty of optimizations and tricks, good performances
• GLSL Vertex and Pixel Shaders support

06/20/06 46

The XVR Web Control

06/20/06 47

Knowing how XVR is organised

• XVR architecture is historically web-oriented

XVR architecture is historically web-oriented

• Applications normally run inside a webpage

Applications normally run inside a webpage

• The core module is an ACTIVEX control

The core module is an ACTIVEX control

06/20/06 48

Embedding the XVR ActiveX in a webpage

<OBJECT ID=“My3DCanvas“ WIDTH = 640 HEIGHT = 480 CLASSID="CLSID:5D589287-1496-4223-AE64-65FA078B5EAB" TYPE="application/x-oleobject" align="left" border="2" CODEBASE="http://client.vrmedia.it/XVRPlayer.cab#Version=1,0, 0,900"> <PARAM NAME="ScriptName" VALUE=" AppName.s3d.bin"> <PARAM NAME="EngineVersion" VALUE="0136"> <PARAM NAME="BackgroundColor" VALUE="#000066"> <PARAM NAME="Params" VALUE "DEPTH=24"> <PARAM NAME=“UserParams" VALUE "…"> </OBJECT>

06/20/06 49

The “Engine version” number

• A single ActiveX control for multiple engine

A single ActiveX control for multiple engine versions versions

• Each engine is self-sufficient

Each engine is self-sufficient

• A new engine released from time to time

A new engine released from time to time

• The sequence was 0100, 0101, … , 0142

The sequence was 0100, 0101, … , 0142 Naming convention used so far: Naming convention used so far:

0142 0142

Engine Family Engine Family (Always the same) (Always the same) Compiler Generation Number Compiler Generation Number Progress inside the Progress inside the same family same family

06/20/06 50

The Auto Update feature

• From time to time we upload bug fixes or minor changes to an

From time to time we upload bug fixes or minor changes to an engine (es. version 0140 had more than 20 releases) engine (es. version 0140 had more than 20 releases)

• When connected to the net XVR has an auto update

When connected to the net XVR has an auto update mechanism mechanism

• If a new version of the engine is detected the old one gets

If a new version of the engine is detected the old one gets updated updated

• When an autoupdate operation is taking place you can see a

When an autoupdate operation is taking place you can see a blue bar below the spinning cube blue bar below the spinning cube

• Auto-updates are very practical for bug-fix or quickly insert

Auto-updates are very practical for bug-fix or quickly insert new functions new functions

06/20/06 51

Exchanging data with other entities

06/20/06 52

Exchanging data with other entities

• From an ATL perspective, the XVR control exposes 3 public methods:

From an ATL perspective, the XVR control exposes 3 public methods:

• DataIn (“string”)

DataIn (“string”)

• DataOut (“string”)

DataOut (“string”)

• FunctionCall (“function name”,”params string”);

FunctionCall (“function name”,”params string”);

• DataIn and DataOut have an equivalent command in the S3D scripting

DataIn and DataOut have an equivalent command in the S3D scripting language language

• They are used to receive formatted strings or to send strings to the other

They are used to receive formatted strings or to send strings to the other elements of the webpage elements of the webpage

• With FunctionCall is possible to execute any function defined inside a

With FunctionCall is possible to execute any function defined inside a script, and to pass a string as a parameter script, and to pass a string as a parameter

06/20/06 53

Tips & Tactics

06/20/06 54

Use the built-in profiler

• After every execution of an S3D script, XVR

provides accurate profiling information

• You can use the info provided to isolate the

part of code (and the function calls) that are more time-consuming in your app.

06/20/06 55

Handling Large Projects (I)

• It’s possible to use

multiple source files

• Each file gets

compiled separately

06/20/06 56

Handling Large Projects (II)

• Main script file
• The main file

includes the others

• Secondary script file
• Secondary script file
• Please note the

“#define”s

06/20/06 57

Handling Large Projects (III)

• Secondary script file
• #define used to avoid

multiple compilations

06/20/06 58

It’s a good idea to collect utility functions

• S3D snippets are

simple to share

• Soon a dedicated

wiki section

06/20/06 59

A simple, script based GUI (EasyGUI)

06/20/06 60

A complex project needs a State Machine!

• Only simple demos can do

Only simple demos can do without without

• State Machines make life easier

State Machines make life easier

• Code is easier to read and

Code is easier to read and easier to manage easier to manage

• States and transitions can be

States and transitions can be added without the need to rewrite added without the need to rewrite everything everything

06/20/06 61

Our script-based state machine

• Transitions are scripted using

Transitions are scripted using MACROS and many syntax tricks MACROS and many syntax tricks

• Textual-only description

Textual-only description

• Not perfect, but useful

Not perfect, but useful

• MACROS provide a good-enough

MACROS provide a good-enough astraction from the plain S3D astraction from the plain S3D languade languade

06/20/06 62

There are also other options

• Marcello Carrozzino will show a graphical tool to manage

Marcello Carrozzino will show a graphical tool to manage “state machine scripts” “state machine scripts”

• Andrea Baroni wrote a much more sophisticate object-oriented

Andrea Baroni wrote a much more sophisticate object-oriented authoring infrastructure. A presentation will follow later authoring infrastructure. A presentation will follow later

• There is plenty of potential for a graphical language->script

There is plenty of potential for a graphical language->script

• converter. It’s a critical issue: proposals are welcome!
• converter. It’s a critical issue: proposals are welcome!

06/20/06 63

Other tools

06/20/06 64

XVR AAM Viewer

06/20/06 65

XVR Shader Exporter

06/20/06 66

XVR Texture Converter

06/20/06 67

XVR State Machine Visual Editor