Evolutionary Computation in Games: Dealing With Uncertainty Paolo - - PowerPoint PPT Presentation

▶
evolutionary computation in games dealing with uncertainty
SMART_READER_LITE
LIVE PREVIEW

Evolutionary Computation in Games: Dealing With Uncertainty Paolo - - PowerPoint PPT Presentation

Jan 30, 2023 145 likes •735 views

Evolutionary Computation in Games: Dealing With Uncertainty Paolo Burelli - Aalborg University Copenhagen pabu@create.aau.dk - www.paoloburelli.com Me Research in Artificial Intelligence and Computer Graphics (Intelligent User Interfaces)

slide-1
SLIDE 1

Evolutionary Computation in Games: Dealing With Uncertainty

Paolo Burelli - Aalborg University Copenhagen pabu@create.aau.dk - www.paoloburelli.com

slide-2
SLIDE 2

Me

  • Research in Artificial Intelligence

and Computer Graphics (Intelligent User Interfaces)

  • Focus on

Virtual Cinematography and Player Modelling

slide-3
SLIDE 3

Tutorial

  • Evolutionary Computation in Games
  • Uncertainty
  • Uncertainty in Games
  • Examples
slide-4
SLIDE 4

Evolutionary Computation In Games

  • Objective functions
  • Player: performance/human

likeness

  • Game: player experience, balance,

duration...

  • Domain
  • Player: controller/strategy
  • Game: content

configuration

Generate optimal player/game

slide-5
SLIDE 5

Galactic Arms Race

  • Evolving weapons
  • Interactive Evolutionary

Computation

  • Objective function is human evaluation
  • Compositional Pattern-

Producing Networks

  • E. J. Hastings, R. K. Guha and K. O. Stanley. Automatic Content Generation in the Galactic Arms Race Video Game. IEEE Transactions on Computational Intelligence

and AI in Games, 2009.

slide-6
SLIDE 6

Uncertainty

  • Noise
  • Robustness
  • Approximation
  • Dynamic Problem
slide-7
SLIDE 7

Noise

  • Noisy objective function evaluation
  • Same evaluation, different values
  • Genotype v.s. phenotype
  • Environment/Sensor noise
slide-8
SLIDE 8

Robustness

  • Variations of the design variables
  • Variations of the environment
slide-9
SLIDE 9

Approximation

  • Objective function is an

approximation of the real problem

  • Evaluation is time-consuming
  • No real fitness available
  • Additional evaluation necessary
  • Rugged fitness landscape
slide-10
SLIDE 10

Dynamic Problem

  • Optimum moves during optimization
  • Environment
  • Objectives
  • Representation
  • Linear/non-linear motion
  • Oscillation
  • Random jumps
slide-11
SLIDE 11

Uncertainty in Games

  • Affects the quality of content/agent
  • Sources?
slide-12
SLIDE 12

Uncertainty in Games

  • Affects the quality of content/agent
  • Sources:
  • Player
  • Sensors
  • Dynamic virtual environment
  • Complex virtual environment
  • Slow execution

My list

slide-13
SLIDE 13

Examples

  • Automatic Camera Control
  • Experience Driven Procedural Content Generation
  • Simulation Based Optimization
slide-14
SLIDE 14

Example 1: Automatic Camera Control

slide-15
SLIDE 15

Virtual Camera

Camera Action Frame

slide-16
SLIDE 16

Automatic Camera Control

  • Abstraction Layer
  • High Level Properties
  • Automatic Configuration
  • Automatic Animation

Camera Controller Virtual Environment Camera Visual and Motion Properties

slide-17
SLIDE 17

Inputs

Camera Controller Virtual Environment Came Visual and Motion Properties

Composition Properties Animation Properties Camera Properties

slide-18
SLIDE 18

Inputs

Camera Controller Virtual Environment Came Visual and Motion Properties

Subjects Environment Geometry

slide-19
SLIDE 19

CamOn

Camera Controller rtual ronment al and Motion erties

Objective Function Visual and Motion Properties Solver Animator Camera Virtual Env.

slide-20
SLIDE 20

Objective Function

Property 0 Property 1 Property N Object.

  • Func. 0

Object.

  • Func. 1

Object.

  • Func. N

Sum Objective Function

Weight 0 Weight 1 Weight N

slide-21
SLIDE 21

Objective Function: Properties

Visibility Vantage Angle Projection Size Frame Position

slide-22
SLIDE 22

Objective Function: Domain

X Y Z α β

Camera

Position Orientation

slide-23
SLIDE 23

Main source of uncertainty?

slide-24
SLIDE 24

Main source of uncertainty: Dynamic Problem

slide-25
SLIDE 25

Dynamic Problem

  • Subjects and other objects move in the virtual space
  • The frame properties might change
  • The geometry of the subjects might change
slide-26
SLIDE 26

Possible Solution

  • Restart
slide-27
SLIDE 27

Possible Solution

  • Restart
  • Simple
slide-28
SLIDE 28

Possible Solution

  • Restart
  • Simple
  • No time
slide-29
SLIDE 29

Possible Solution

  • Restart
  • Simple
  • No time
  • Waste of information
slide-30
SLIDE 30

Possible Solution

  • Restart
  • Simple
  • No time
  • Waste of information
  • Might be the only solution
slide-31
SLIDE 31

Challenges

Information Reuse

how to store and reuse information about the landscape?

Population Diversity

how to avoid premature population convergence?

slide-32
SLIDE 32

Information Reuse

  • Explicit memory
  • Data structure: landscape fingerprint, optima
  • Ruse part of the population
  • Implicit memory
  • Multiploidy/Diploidy
  • Information validity
  • Generational
slide-33
SLIDE 33

Population Diversity

  • Diversity after change
  • Hypermutation
  • Variable local search
  • Diversity throughout the optimization
  • Random immigrants
  • Multiple populations

Rasmus K. Ursen. Multinational GAs: Multimodal optimization techniques in dynamic environments. Evolutionary Computation Conference, 2000

slide-34
SLIDE 34

Hybrid Genetic Algorithm

  • Hybrid Lamarckian-Darwinian

evolution

  • Explore if early convergence
  • Early convergence if:
  • No improvement for one frame
  • Complete occlusion

Paolo Burelli. Interactive Virtual Cinematography. IT University Of Copenhagen, 2012

slide-35
SLIDE 35

Example 2: Experience Driven Procedural Content Generation

slide-36
SLIDE 36

EDPCG

Georgios N. Yannakakis and Julian Togelius. Experience-driven procedural content generation. IEEE Transactions on Affective Computing, 2011.

Op#mize ¡player ¡ experience Capture ¡player ¡ experience Model ¡the ¡effect ¡

  • f ¡game ¡content
slide-37
SLIDE 37

Challenges

  • How to capture Player Experience?
  • How to evaluate the quality of content?
  • How to optimize game content for Player Experience?
slide-38
SLIDE 38

Capturing Player Experience

  • Subjectively
  • Asking players: self-report questionnaires (ranking, preferences)
  • Objectively
  • Physiology (GCR, EEG, EMG, BVP

,…); eye-tracking; facial expression; speech

  • GamePlay-Based
  • Player game preferences (what players do relates to their experience)
slide-39
SLIDE 39

Content Quality

  • Direct utility/fitness
  • A direct mapping between content and quality; e.g. number of jumps in a platform game
  • Simulation-based
  • An AI agent (human-like?) plays the game for a while and content is evaluated through playing style
  • Interactive fitness
  • Real-time evaluation via a player or players
slide-40
SLIDE 40

Player Experience Model

Optimize Content

Content Optimizer Content Quality Content Representationn

slide-41
SLIDE 41

Main sources of uncertainty?

slide-42
SLIDE 42

Main sources of uncertainty: Noise, Robustness

slide-43
SLIDE 43

Noise

slide-44
SLIDE 44

Dealing with Noise

  • Explicit average
  • Multiple samples per evaluation
  • Average with neighborhud
  • Interpolation
  • Implicit average
  • Increase population size
  • Selection scheme
  • Threshold for selection
  • Noise might be useful...

Sandor Markon, Dirk V. Arnold, Thomas Back, Thomas Beielstein and Hans–Georg Beyer. Thresholding – a Selection Operator for Noisy ES, IEEE Congress on Evolutionary Computation, 2001

slide-45
SLIDE 45

Robustness

slide-46
SLIDE 46

Dealing With Robustness

  • Optimizing Expected Fitness
  • Average in the neighborhood
  • Average with similar previous values
  • Add noise and increase population
  • Multi-Objective Optimization
  • Fitness v.s. Robustness
  • Measure of robustness

Yaochu Jin and Bernhard Sendhoff. Trade-off between Performance and Robustness: An Evolutionary Multiobjective Approach. Evolutionary Multi-Criterion Optimization, 2003 Fitness Robustness

slide-47
SLIDE 47

Example 3: Simulation Based Optimization

slide-48
SLIDE 48

Evolving Strategy Game Units

  • Objective: complementarity
  • Balanced units sets stronger than

unbalanced ones

Tobias Mahlmann, Julian Togelius and Georgios N. Yannakakis. Towards Procedural Strategy Game Generation : Evolving Complementary Unit Types. European Conference on Applications of Evolutionary Computation, 2011.

<

slide-49
SLIDE 49

Problem Characteristics

  • 21 attributes in the gene
  • Objective function based on 6 matches player 200 times
  • 1 minute per evaluation
slide-50
SLIDE 50

Time Consuming Evaluation

  • Long experimental time
  • No possible “real-time” execution
  • Applies also to agent learning
slide-51
SLIDE 51

Main source of uncertainty: Approximation

slide-52
SLIDE 52

Motivations

  • Time consuming evaluation
  • No available analytical fitness
  • Noise Reduction
  • Rugged landscape
  • Smart population initialisation
slide-53
SLIDE 53

Approximation Methods

  • Simplified simulation
  • Data-driven functional approximation
  • Evaluations reduction
  • Fitness inheritance
  • Fitness imitation
  • Fitness assignment
slide-54
SLIDE 54

Dealing With Approximation

Combine approximated function with real-function Individual Based Control

  • Random
  • Best
  • Most uncertain
  • Most representative

Generation Based Control Whole population every N generations

Jürgen Branke and Christian. Faster convergence by means of fitness estimation. Soft Computing, 2005

slide-55
SLIDE 55

Future Work

  • Experiment these techniques in games
  • Use games as a benchmark for uncertainty
  • Other forms of uncertainty?
slide-56
SLIDE 56

References

  • Erin J. Hastings, Ratan K. Guha and Kenneth O. Stanley. Automatic Content Generation in the Galactic Arms Race

Video Game. IEEE Transactions on Computational Intelligence and AI in Games, 2009.

  • Paolo Burelli. Interactive

Virtual Cinematography. IT University Of Copenhagen, 2012

  • Rasmus K. Ursen. Multinational GAs: Multimodal optimization techniques in dynamic environments. Evolutionary Computation

Conference, 2000

  • Georgios N.

Yannakakis and Julian

  • Togelius. Experience-driven procedural content generation. IEEE

Transactions on Affective Computing, 2011.

  • Sandor Markon, Dirk
  • V. Arnold,

Thomas Back, Thomas Beielstein and Hans–Georg Beyer. Thresholding – a Selection Operator for Noisy ES, IEEE Congress on Evolutionary Computation, 2001

  • Yaochu Jin and Bernhard Sendhoff.

Trade-off between Performance and Robustness: An Evolutionary Multiobjective Approach. Evolutionary Multi-Criterion Optimization, 2003

  • Tobias Mahlmann, Julian

Togelius and Georgios N. Yannakakis. Towards Procedural Strategy Game Generation : Evolving Complementary Unit

  • Types. European Conference on Applications of Evolutionary Computation, 2011.
  • Jürgen Branke and Christian. Faster convergence by means of fitness estimation. Soft Computing, 2005
  • Jürgen Branke and

Yaochu Jin. Evolutionary Optimization in Uncertain Environments. IEEE Transaction on Evolutionary Computation, 2005

slide-57
SLIDE 57

Thank you! Questions?

EvoGAMES 2013

Bio-inspired Algorithms in Games

Submission ¡deadline: ¡1 ¡November ¡2012 Vienna, ¡3-­‑5 ¡April ¡2013