[PPT] - A Multi-Agent Prediction Market based on Raj Dasgupta Partially PowerPoint Presentation

SLIDE 1

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game

Janyl Jumadinova, Raj Dasgupta

C-MANTIC Research Group Computer Science Department University of Nebraska at Omaha, USA

ICEC 2011

1 / 37

SLIDE 2

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Outline

Problem: Traders’ behavior in a prediction market and its impact on the prediction market’s behavior Solution: A multi-agent system that formalizes the strategic behavior and decision making by market’s participants based on a partially observable stochastic game Experimental validation: Comparison with other trading approaches

1 / 37

SLIDE 3

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

A Prediction market is

a market-based mechanism used to

combine the opinions on a future event from different

people and

forecast the possible outcome of the event based on the

aggregated opinion

2 / 37

SLIDE 4

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

3 / 37

SLIDE 5

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

4 / 37

SLIDE 6

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

5 / 37

SLIDE 7

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

6 / 37

SLIDE 8

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

7 / 37

SLIDE 9

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

8 / 37

SLIDE 10

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

9 / 37

SLIDE 11

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

10 / 37

SLIDE 12

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

11 / 37

SLIDE 13

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

12 / 37

SLIDE 14

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

13 / 37

SLIDE 15

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Prediction Market

Main Features

A prediction market is run for a real-life unknown event Each event has a finite duration Each event’s outcome has a security associated with it Traders buy and sell the securities based on their beliefs about the outcome of the event Traders’ beliefs are expressed as probabilities Market maker aggregates the probabilities from all the traders into a single probability, market price Market price of a security represents the probability of the outcome of an event associated with that security happening Traders get paid according to their reported beliefs

14 / 37

SLIDE 16

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

A Multi-Agent Prediction Market

Software trading agents perform calculations and trade

n behalf of human traders

Provides testbed for modeling different strategic behaviors of traders through simulations with trading agents

15 / 37

SLIDE 17

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Do Prediction Markets Work?

Yes, evidence from real markets, laboratory experiments, and theory

I.E.M. beat political polls 451/596 [Forsythe 1999, Berg 2001, Pennock 2002] HP market beat sales forecast 6/8 [Plott 2000] Sports betting markets provide accurate forecasts of game outcomes [Debnath 2003, Schmidt 2002] Market games work [Pennock 2001] Laboratory experiments confirm information aggregation [Forsythe 1990, Plott 1997, Chen 2001] Theory of Rational Expectations [Lucas 1972, Grossman 1981] and more...

16 / 37

SLIDE 18

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Research Problem Addressed

Develop a formal, game-theoretic model of the trading agent behavior in prediction markets including

impact of information from external sources on trading

agent decisions/behavior,

a solution concept for calculating the equilibrium

strategies of the trading agents

Research Questions:

How does different traders’ behaviors affect market prices? What trading strategies give the highest utilities to the traders? How can prediction markets incentivize traders to participate and report their beliefs truthfully in order to achieve a higher prediction accuracy?

17 / 37

SLIDE 19

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Our Solution

A partially observable stochastic game with information (POSGI)-based model of the trading agent behavior A correlated equilibrium (CE)-based solution to determine equilibrium strategy in the POSGI representation

18 / 37

SLIDE 20

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Partially Observable Stochastic Game with Information (POSGI)

19 / 37

SLIDE 21

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Partially Observable Stochastic Game with Information (POSGI)

Logarithmic Market Scoring Rule (LMSR) [Hanson

2007] gives formula to calculate aggregate market price from the outstanding quantity of a security

19 / 37

SLIDE 22

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Partially Observable Stochastic Game with Information (POSGI)

Belief state is updated using a Bayesian model of past

beliefs, past actions and current observation

20 / 37

SLIDE 23

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Partially Observable Stochastic Game with Information (POSGI)

Information signal can be {−1, 0, 1} representing

positive, neutral, or negative information

21 / 37

SLIDE 24

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Partially Observable Stochastic Game with Information (POSGI)

22 / 37

SLIDE 25

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Two Agent Trading Scenario with 1 security

Example

Actions = to buy, sell, or hold one security Market state is denoted by q C(q) is the cost function calculated by LMSR, that reflects the total money collected by the market maker Can construct a normal form game to capture the decision problem for each agent

23 / 37

SLIDE 26

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Correlated Equilibrium (CE)

Set up

Let N be the set of trading agents Let Ai be the set of actions for agent i ∈ N Let Φi be the set of mixed strategy profiles defined over Ai Joint strategy space is then Φ = ×|N|

i=1Φi

Let φ ∈ Φ be a strategy profile and φi denote agent i’s component in φ Let ui(φ) be the utility of agent i from joint strategy φ

24 / 37

SLIDE 27

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Correlated Equilibrium (CE)

Correlated Equilibrium is a probability distribution p on Φ such that

for all agents i and all strategies φi, φ′

i

if all agents follow a strategy profile φ, agent i has no incentive to play another strategy φ′

i

instead

The Chicken-Dare Game Assume there is a trusted third party that draws a strategy for each player and announces it to each player separately Players agree to follow the strategy suggested by the third party Players get higher payoffs when use CE than NE

25 / 37

SLIDE 28

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Correlated Equilibrium (CE) Algorithm

CE algorithm applied to our POSGI model is based on the method proposed by Papadimitriou [2008] CE algorithm of the POSGI gives the action (buy or sell; quantity) for each agent at each time step Time complexity of our CE algorithm is O(|N| × |Φi|2)

26 / 37

SLIDE 29

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Correlated Equilibrium (CE)

with Trading Agents’ Risk Preferences

The beliefs and risk preferences of traders are directly correlated [Kadane 1988, Dimitrov 2008] Use constant relative risk averse (CRRA) utility function To find CE in the market with risk-averse trading agents:

Find the set of all Pareto optimal strategy profiles
Check whether a Pareto optimal strategy profile satisfies

CE constraints

27 / 37

SLIDE 30

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Correlated Equilibrium (CE)

with Trading Agents’ Risk Preferences

A strategy profile φP is Pareto optimal if there does not exist another strategy profile φ′ such that ui(φ′) ≥ ui(φP )∀i ∈ N with at least one inequality strict Pareto optimal strategy is found by maximizing weighted utilities and solving the maximization problem using Lagrangian method

Proposition

If p is a correlated equilibrium and φP is a Pareto optimal strategy profile calculated by p in a prediction market with risk averse agents, then the strategy profile φP is incentive compatible, that is each agent is best off reporting truthfully

28 / 37

SLIDE 31

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Set up

Default values for market related parameters came from Iowa Electronic Marketplace (IEM) movie market Assume there is one event with two outcomes Consider only disjoint events Report utilities and market price for the security corresponding to the outcome of the event being 1

29 / 37

SLIDE 32

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Set up

We use the following strategies for comparison of the trading agents’ and market’s behavior

1

ZI (Zero Intelligence) - each agent submits randomly calculated quantity to buy or sell

2

ZIP (Zero Intelligence Plus) - each agent selects a quantity to buy or sell that satisfies a particular level of profit

3

CP (by Preist and Tol) - each agent adjusts its quantity to buy or sell based on past prices and tries to choose that quantity so that it is competitive among other agents

4

GD (by Gjerstad and Dickhaut) - each agent maintains a history of past transactions and chooses the quantity to buy

r sell that maximizes its expected utility

5

DP (Dynamic Programming solution for POSG game) - each agent uses dynamic programming solution to find the best quantity to buy or sell [Hansen 2004]

30 / 37

SLIDE 33

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-neutral Agents

(a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with trading agents using CE strategy result in more accurate market prices (c) Trading agents using CE strategy obtain 38% more utility than the agents following the next best performing strategy DP

31 / 37

SLIDE 34

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-neutral Agents

(a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with trading agents using CE strategy result in more accurate market prices (c) Trading agents using CE strategy obtain 38% more utility than the agents following the next best performing strategy DP

31 / 37

SLIDE 35

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-neutral Agents

(a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with trading agents using CE strategy result in more accurate market prices (c) Trading agents using CE strategy obtain 38% more utility than the agents following the next best performing strategy DP

31 / 37

SLIDE 36

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-neutral Agents

(a) Trading agents using CE strategy obtain more utility than trading agents using ZIP strategy (b) Trading agents using CE strategy obtain more utility than trading agents using CP strategy

32 / 37

SLIDE 37

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-neutral Agents

(a) Trading agents using CE strategy obtain more utility than trading agents using ZIP strategy (b) Trading agents using CE strategy obtain more utility than trading agents using CP strategy

32 / 37

SLIDE 38

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-neutral Agents

(c) Trading agents using CE strategy obtain more utility than trading agents using GD strategy (d) Trading agents using CE strategy obtain more utility than trading agents using DP strategy

33 / 37

SLIDE 39

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-neutral Agents

(c) Trading agents using CE strategy obtain more utility than trading agents using GD strategy (d) Trading agents using CE strategy obtain more utility than trading agents using DP strategy

33 / 37

SLIDE 40

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-averse Agents

(a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with risk-averse trading agents using CE strategy result in more accurate market prices (c) Risk-averse trading agents using CE strategy obtain 41% more utility than the agents following the next best performing strategy DP

34 / 37

SLIDE 41

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-averse Agents

(a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with risk-averse trading agents using CE strategy result in more accurate market prices (c) Risk-averse trading agents using CE strategy obtain 41% more utility than the agents following the next best performing strategy DP

34 / 37

SLIDE 42

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Experimental Results

Risk-averse Agents

(a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with risk-averse trading agents using CE strategy result in more accurate market prices (c) Risk-averse trading agents using CE strategy obtain 41% more utility than the agents following the next best performing strategy DP

34 / 37

SLIDE 43

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Conclusions and Future Work

In this work we have:

described a novel multi-agent based representation of

the prediction market using POSG

developed a CE solution for solving POSG
empirically compared different agent behavior strategies

in the prediction market

showed how CE can be obtained in the prediction

market with risk averse agents

35 / 37

SLIDE 44

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Conclusions and Future Work

In this work we have:

described a novel multi-agent based representation of

the prediction market using POSG

developed a CE solution for solving POSG
empirically compared different agent behavior strategies

in the prediction market

showed how CE can be obtained in the prediction

market with risk averse agents In the future we plan to: Conduct experiments in an n-agent scenario using richer commercial data sets Investigate the dynamics evolving from multiple prediction markets that interact with each other

35 / 37

SLIDE 45

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

References

1 Y. Chen, D. Pennock. Utility Framework for

Bounded-Loss Market Maker. Proc. of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI 2007), pages 49-56, 2007.

2 E. Hansen, D. Bernstein, S. Zilberstein. Dynamic

programming for partially observable stochastic games. In Proceedings of the 19th National Conference on Artificial Intelligence, pages 709-715, 2004.

3 R. Hanson. Logarithmic Market scoring rules for

Modular Combinatorial Information Aggregation. Journal of Prediction Markets, 1(1):3-15, 2007.

4 Iowa Electronic Marketplace. URL:

www.biz.uiowa.edu/iem/

5 C. Papadimitriou, T. Roughgarden. Computing

correlated equilibria in multi-player games. Journal of ACM, 55(3):1-29, 2008.

36 / 37

SLIDE 46

A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work

Thank You! Questions?

jjumadinova@unomaha.edu C-MANTIC Research Group http://cmantic.unomaha.edu/

37 / 37