[PPT] - Duopolistic competition in an Duopolistic competition in an PowerPoint Presentation

SLIDE 1

CINEF - University of Genoa

Duopolistic competition in an Duopolistic competition in an artificial power exchange artificial power exchange with learning agents with learning agents

Speaker: ERIC GUERCI

Joint work with: Silvano Cincotti, Stefano Ivaldi, Marco Raberto “Complex Markets” Meeting, Marseille, 6-7th October 2006

SLIDE 2

10/20/2006 "Complex Markets" meeting, Marseille 2\35

Outline Outline

Electricity Market outlook

Electricity Market outlook

Agent

Agent-

based Artificial Power Exchange

based Artificial Power Exchange

Stochastic Games Framework

Stochastic Games Framework

Reinforcement Learning

Reinforcement Learning

Computational Results

Computational Results

Conclusions

Conclusions

SLIDE 3

10/20/2006 "Complex Markets" meeting, Marseille 3\35

Electricity Markets Electricity Markets

Worldwide deregulation in the 90’s: from monopolistic state

Worldwide deregulation in the 90’s: from monopolistic state-

wned suppliers to “competitive” electricity market
wned suppliers to “competitive” electricity market
Active

Active Eureopan Eureopan Power Power Exchanges Exchanges: : – – Nord Pool (Finnish countries), 1996 Nord Pool (Finnish countries), 1996 – – OMEL (Spain), 1998 OMEL (Spain), 1998 – – APX (The Netherlands), 1999 APX (The Netherlands), 1999 – – NETA (UK), 2000 NETA (UK), 2000 – – EEX Frankfurt (Germany), 2000 EEX Frankfurt (Germany), 2000 – – LPX LPX Lipsia Lipsia (Germany), 2000 (Germany), 2000 – – PPE (Poland ), 2000 PPE (Poland ), 2000 – – Opcom Opcom (Romania), 2001 (Romania), 2001 – – Powernext Powernext (France), 2001 (France), 2001 – – Borzen Borzen (Slovenia), 2002 (Slovenia), 2002 – – EXAA (Austria), 2003 EXAA (Austria), 2003 – – IPEX (Italy), 2004 IPEX (Italy), 2004

SLIDE 4

10/20/2006 "Complex Markets" meeting, Marseille 4\35

Actors in a Power Exchange Actors in a Power Exchange

Producers (generators and suppliers)

Producers (generators and suppliers) – – produce and have customers that consume physical quantities produce and have customers that consume physical quantities

f energy
f energy
Traders

Traders – – buy and sell electrical energy under contract buy and sell electrical energy under contract

Clients

Clients – – consuming energy customers consuming energy customers

Market Operator

Market Operator – – organization and management of the electricity market

rganization and management of the electricity market
System Operator

System Operator – – determines what actions need to be taken in order to maintain determines what actions need to be taken in order to maintain the required national and local balances of generation and the required national and local balances of generation and consumption consumption

SLIDE 5

10/20/2006 "Complex Markets" meeting, Marseille 5\35

Electricity Markets Electricity Markets -

Clearinghouse

Clearinghouse

Day

Day-

Ahead Market

Ahead Market – – DAM (Energy market) DAM (Energy market) – – Collection of offers and bids for next day hours Collection of offers and bids for next day hours – – Construction of demand and supply curves Construction of demand and supply curves – – Market clearing for every hour of the next day Market clearing for every hour of the next day – – Zonal splitting, in the case of congestion Zonal splitting, in the case of congestion

Adjustment Market

Adjustment Market – – AM (Energy market) AM (Energy market) – – Allows revision of trading activities Allows revision of trading activities – – Starts after DAM and considers separately every our of the next Starts after DAM and considers separately every our of the next day day

Ancillary Services Market

Ancillary Services Market – – ASM (Service market) ASM (Service market) – – Procures resources for dispatching, i.e. management, operation Procures resources for dispatching, i.e. management, operation and control of the power system and control of the power system – – Planned grid congestion relief, purchase of operating reserve Planned grid congestion relief, purchase of operating reserve for the next day, electricity for real for the next day, electricity for real-

time balancing of the system

time balancing of the system

SLIDE 6

10/20/2006 "Complex Markets" meeting, Marseille 6\35

Artificial Power Exchange Artificial Power Exchange

Main Aims and opportunities:

To understand and to simulate the micro

To understand and to simulate the micro-

structure of a

structure of a real power exchange real power exchange

To overcome analytical intractability

To overcome analytical intractability

To perform What

To perform What-

if Analysis

if Analysis

To develop a framework for market design and

To develop a framework for market design and validation validation

SLIDE 7

10/20/2006 "Complex Markets" meeting, Marseille 7\35

Stochastic Game Framework (I) Stochastic Game Framework (I)

1... 1...

( , , , , )

n n

n S A T R

Stochastic Game

Stochastic Game

n

Number of agents

S

Set of States i

A

Set of actions available to agent i

[ ]

0,1 S S × × → Α

i i A

= × Α

T

Transition function i

R

Reward function of agent i

SLIDE 8

10/20/2006 "Complex Markets" meeting, Marseille 8\35

Strategic form of the game Strategic form of the game

R-bimatrix game (one-shot game)

1 2

, actions of the matrix game istantaneous reward for player

j k i lm

a A b A R i ∈ ∈

1

a

2

a

1 2 11 11

, R R

1 2 12 12

, R R

1 2 21 21

, R R

1 2 22 22

, R R

1

b

2

b

SLIDE 9

10/20/2006 "Complex Markets" meeting, Marseille 9\35

Strategic form of the game Strategic form of the game

{ } { }

1 2

, ,..., ,... , , ,,..., ,... stationary policies expected sum of discounted delayed rewards for player

j j j jt k k k jt i lm

a a a b b b Q i α β = =

Q-bimatrix game (repeated games)

1 2 11 11

, Q Q

1 2 12 12

, Q Q

1 2 21 21

, Q Q

1 2 22 22

, Q Q

1

α

2

α

1

β

2

β

is a joint policy( , )

i t i lm t t l m

Q E R

π

γ π α β

∞ =

⎡ ⎤ = ⎢ ⎥ ⎣ ⎦

∑

SLIDE 10

10/20/2006 "Complex Markets" meeting, Marseille 10\35

Solution Solution Concepts Concepts

* * * *

Joint set of actions ( , ), is the generic payoff either

r

i i i i i i i

x a a a A R Q

−

= ∈ Π

Nash Nash equilibria equilibria: Competitive Solution : Competitive Solution

* * * * * *

( , ) is Nash if ( , ) ( , ),

i i i i i i i i

x a a a a a a i

− − −

= Π ≥ Π ∀

Pareto Pareto optima

ptima:

: Tacit Tacit collusive collusive solution solution

* *

is not Pareto if : ( ) ( ),

i i

x x x x i ∃ Π ≥ Π ∀

SLIDE 11

10/20/2006 "Complex Markets" meeting, Marseille 11\35

Why learning? Why learning?

Real electricity markets are characterized by:

Real electricity markets are characterized by:

– – Small number of suppliers, possible Small number of suppliers, possible oligopolistic

ligopolistic

scenario scenario – – Few big producers exercising market power Few big producers exercising market power – – Portfolio generators, with different market shares Portfolio generators, with different market shares – – Repeated interaction among the same sellers may Repeated interaction among the same sellers may produce collusive behavior produce collusive behavior

SLIDE 12

10/20/2006 "Complex Markets" meeting, Marseille 12\35

Reinforcement Learning Reinforcement Learning

1. 1. Adaptive evolutionary Adaptive evolutionary -

Marimon

Marimon and and McGrattan McGrattan (1995) (1995)

Matrix game payoffs (R

Matrix game payoffs (R-

matrix game)

matrix game)

Learning instantaneous rewards

Learning instantaneous rewards

Adaptive Stochastic algorithms

Adaptive Stochastic algorithms

2. 2. Q Q-

Learning

Learning – – Watkins (1989) Watkins (1989)

Repeated game payoffs (Q

Repeated game payoffs (Q-

matrix game)

matrix game)

Learning from delayed rewards

Learning from delayed rewards

Sequential decision task

Sequential decision task

Markov Decision Process Framework

Markov Decision Process Framework

SLIDE 13

10/20/2006 "Complex Markets" meeting, Marseille 13\35

Adaptive evolutionary learning Adaptive evolutionary learning

( (Marimon Marimon and and McGrattan McGrattan ‘95) ‘95)

, 1 , , 1

( ) 1 ( ) ( )

i t i i i t i i t i

a if i plays a a a

therwise

η η η

− −

+ ⎧ ⎪ = ⎨ ⎪ ⎩

, 1 , 1 , 1 , 1 , , 1

1 ( ) ( ) ( ) ( ) ( ) ( )

i t i i t i i t i i i t i i t i i t i

S a S a a if i plays a a S a S a

therwise

η

− − − − −

⎧ ⎡ ⎤ − ⋅ −Π ⎪ ⎣ ⎦ = ⎨ ⎪ ⎩

Strength vector Counter

, 1 , 1

( ) , 1 , ( ) , 1 , , 1 ,

( ) ( ) ( ) ( ) 1

i t i i t i i

S a i t i i t S a i t i i t i a i t i i t

e a with probability a e a a with probability σ ρ σ σ σ ρ

− −

− − −

⎧ ⋅ ⎪ ⎪ = ⎨ ⎪ − ⎪ ⎩

∑

INERTIA Updating mixed strategies EXPERIMENTATION Minimum probability bound over pure strategies

εi,t is the minimum probability value for mixed strategies

( )

, , , , , , , ,

( ) ( ) ( ) 1 ( ) ( )

i

i t i t i i t i t i i t i i t i t i i t i t i a

if a a a a

therwise

a

,

ε σ ε σ σ ε σ ε σ ≤ ⎧ ⎪ ⎪ = ⎨ − ⋅ ≤ ⎪ ⎪ ⎩∑

SLIDE 14

10/20/2006 "Complex Markets" meeting, Marseille 14\35

Q Q-

learning

learning (Watkins ‘89)

(Watkins ‘89)

* * ' * *

( , ) ( , ) ( , , ') ( ) ( ) max ( , )

s a

Q s a R s a s a s V s V s Q s a γ = + Ρ =

∑

Optimal value function Optimal Q-function Stochastic Iterative algorithm

( )

1( ,

) (1 ) ( , ) ( , ) max ( , )

t t t t t t t t t t t t t a

Q s a Q s a R s a Q s a η η γ

+

= − + +

SLIDE 15

10/20/2006 "Complex Markets" meeting, Marseille 15\35

Experimental assumption Experimental assumption

No network constraints

No network constraints

Demand: Inelastic and time

Demand: Inelastic and time-

invariant demand by

invariant demand by means of a representative buyer means of a representative buyer

Supply: Duopolistic competition, modeled by means of

Supply: Duopolistic competition, modeled by means of learning capabilities learning capabilities

Linear and constant cost functions with different

Linear and constant cost functions with different marginal costs marginal costs

Sellers’ action space is discrete and bi

Sellers’ action space is discrete and bi-

dimensional

dimensional

– – Couples of values for prices and quantities Couples of values for prices and quantities

Two key questions are issued

Two key questions are issued

– – What is the solution most frequently chosen and how What is the solution most frequently chosen and how seller profits evolve in time? seller profits evolve in time?

SLIDE 16

10/20/2006 "Complex Markets" meeting, Marseille 16\35

Experimental analysis Experimental analysis

Two Auction mechanism:

Two Auction mechanism:

– – Discriminatory Discriminatory vs vs Uniform Uniform

Two rationing rules:

Two rationing rules:

– – Equal Equal vs vs Cost Cost

Two market scenario:

Two market scenario:

– – Low Demand and High Demand Low Demand and High Demand

SLIDE 17

10/20/2006 "Complex Markets" meeting, Marseille 17\35

Clearinghouse double Clearinghouse double-

auction

auction

Discriminatory auction Uniform auction

2 4 6 8 30 32 34 36 38 40 42

Quantities (MWh)

System marginal price

2 4 6 8 30 32 34 36 38 40 42

Quantities (MWh) Prices (Euro/MWh)

Demand Supply

Pay as bid mechanism

Prices (Euro/MWh)

Demand Supply

SLIDE 18

10/20/2006 "Complex Markets" meeting, Marseille 18\35

Rationing Rules Rationing Rules

Equal rationing

Equal rationing

– – In case of same bid price the two sellers are In case of same bid price the two sellers are equally equally despatched despatched

Cost rationing

Cost rationing

– – In case of same bid price the most efficient In case of same bid price the most efficient sellers is preferred to be sellers is preferred to be despatched despatched

SLIDE 19

10/20/2006 "Complex Markets" meeting, Marseille 19\35

Low Low-

Demand

Demand Economic Economic Scenario Scenario

{ }

1 2

min ,

d s s

Q Q Q <

Low Demand characteristics A single seller can satisfy the whole demand

Computational experiments

1. 1. LD2 LD2-

MM

MM-

ER

ER: Low : Low-

demand situation with MM learning and ER

demand situation with MM learning and ER rule rule 2. 2. LD2 LD2-

QL

QL-

ER

ER: Low : Low-

demand situation with Q

demand situation with Q-

learning and ER rule

learning and ER rule 3. 3. LD2 LD2-

QL

QL-

ER

ER: Low : Low-

demand situation with Q

demand situation with Q-

learning and CR rule

learning and CR rule 4. 4. LD3 LD3-

MM

MM-

ER

ER: Low : Low-

demand situation with MM learning and ER

demand situation with MM learning and ER rule rule

SLIDE 20

10/20/2006 "Complex Markets" meeting, Marseille 20\35

Experimental Setting Experimental Setting

Seller 1 Seller 1 Seller 2 Seller 2 Demand Demand Max Q Max Q 6 6 MC MC 4 4 7 7 10 10 6 6 6 6 Max P Max P 10 10 10 10

Frequencies are evaluated as ensemble averages over all

Frequencies are evaluated as ensemble averages over all 10,000 experiments. 10,000 experiments.

SLIDE 21

10/20/2006 "Complex Markets" meeting, Marseille 21\35

LD2 LD2-

MM

MM-

ER (Nash

ER (Nash vs vs Pareto) Pareto)

1000 2000 3000 4000 5000 6000 7000 8000 9000

10000

0.2 0.4 0.6 0.8 1 Auction rounds

Frequency

Discriminatory Auction

Nash Pareto 1000 2000 3000 4000 5000 6000 7000 8000 9000

10000

0.2 0.4 0.6 0.8 1

Auction rounds Frequency

Uniform Auction

Nash Pareto

SLIDE 22

10/20/2006 "Complex Markets" meeting, Marseille 22\35

Payoff Matrix DA Payoff Matrix DA

ps ps 1 1 qs qs 1 1 ps ps 2 2 qs qs 2 2 Payoff Payoff 1 1 Payoff Payoff 2 2 Nash Nash Pareto Pareto 7 7 6 6 8 8 6 6 18.00 18.00 0.00 0.00 X X 7 7 6 6 8 8 5 5 18.00 18.00 0.00 0.00 X X 7 7 6 6 8 8 4 4 18.00 18.00 0.00 0.00 X X 7 7 6 6 8 8 3 3 18.00 18.00 0.00 0.00 X X 7 7 6 6 9 9 6 6 18.00 18.00 0.00 0.00 7 7 6 6 9 9 5 5 18.00 18.00 0.00 0.00 7 7 6 6 9 9 4 4 18.00 18.00 0.00 0.00 7 7 6 6 9 9 3 3 18.00 18.00 0.00 0.00

SLIDE 23

10/20/2006 "Complex Markets" meeting, Marseille 23\35

Ps 1 Qs 1 Ps 2 Qs 2 Payoff 1 Payoff 2 7 5 10 6 30.00 3.00 6 5 10 5 30.00 3.00 7 5 10 5 30.00 3.00 6 5 10 6 30.00 3.00 5 5 10 6 30.00 3.00

Payoff Matrix UA Payoff Matrix UA

ps ps 1 1 qs qs 1 1 ps ps 2 2 qs qs 2 2 Payoff 1 Payoff 1 Payoff 2 Payoff 2 Nash Nash Pareto Pareto

7 7 5 5 10 10 6 6 30.00 30.00 3.00 3.00 X X X X 6 6 5 5 10 10 5 5 30.00 30.00 3.00 3.00 X X X X 7 7 5 5 10 10 5 5 30.00 30.00 3.00 3.00 X X X X 6 6 5 5 10 10 6 6 30.00 30.00 3.00 3.00 X X X X 5 5 5 5 10 10 6 6 30.00 30.00 3.00 3.00 X X X X 6 6 5 5 10 10 4 4 30.00 30.00 3.00 3.00 X X X X 4 4 5 5 10 10 6 6 30.00 30.00 3.00 3.00 X X X X 5 5 5 5 10 10 4 4 30.00 30.00 3.00 3.00 X X X X 7 7 5 5 10 10 4 4 30.00 30.00 3.00 3.00 X X X X 3 3 5 5 10 10 6 6 30.00 30.00 3.00 3.00 X X X X 4 4 5 5 10 10 5 5 30.00 30.00 3.00 3.00 X X X X 5 5 5 5 10 10 5 5 30.00 30.00 3.00 3.00 X X X X 3 3 5 5 10 10 5 5 30.00 30.00 3.00 3.00 X X X X 7 7 5 5 10 10 3 3 30.00 30.00 3.00 3.00 X X X X 2 2 5 5 10 10 6 6 30.00 30.00 3.00 3.00 X X X X 5 5 10 10 6 6 30.00 30.00 3.00 3.00 X X X X 1 1 5 5 10 10 6 6 30.00 30.00 3.00 3.00 X X X X 1 1 5 5 10 10 5 5 30.00 30.00 3.00 3.00 X X X X 2 2 5 5 10 10 5 5 30.00 30.00 3.00 3.00 X X X X 3 3 5 5 10 10 4 4 30.00 30.00 3.00 3.00 X X X X 4 4 5 5 10 10 4 4 30.00 30.00 3.00 3.00 X X X X 2 2 5 5 10 10 4 4 30.00 30.00 3.00 3.00 X X X X 5 5 5 5 10 10 3 3 30.00 30.00 3.00 3.00 X X X X 5 5 10 10 5 5 30.00 30.00 3.00 3.00 X X X X 1 1 5 5 10 10 4 4 30.00 30.00 3.00 3.00 X X X X 4 4 5 5 10 10 3 3 30.00 30.00 3.00 3.00 X X X X 6 6 5 5 10 10 3 3 30.00 30.00 3.00 3.00 X X X X 2 2 5 5 10 10 3 3 30.00 30.00 3.00 3.00 X X X X 3 3 5 5 10 10 3 3 30.00 30.00 3.00 3.00 X X X X 5 5 10 10 4 4 30.00 30.00 3.00 3.00 X X X X 1 1 5 5 10 10 3 3 30.00 30.00 3.00 3.00 X X X X 8 8 5 5 10 10 6 6 30.00 30.00 3.00 3.00 X X 6 6 5 5 10 10 2 2 30.00 30.00 3.00 3.00 X X X X 5 5 10 10 3 3 30.00 30.00 3.00 3.00 X X X X 8 8 5 5 10 10 4 4 30.00 30.00 3.00 3.00 X X 8 8 5 5 10 10 5 5 30.00 30.00 3.00 3.00 X X 7 7 5 5 10 10 2 2 30.00 30.00 3.00 3.00 X X X X

SLIDE 24

10/20/2006 "Complex Markets" meeting, Marseille 24\35

LD2 LD2-

MM

MM-

ER (Profits

ER (Profits vs vs Costs) Costs)

00 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 10 20 30 40 50 60 70 80 90

Auction rounds Costs / Payments

Payments to suppliers UA Payments to suppliers DA Total generation costs

SLIDE 25

10/20/2006 "Complex Markets" meeting, Marseille 25\35

LD2 LD2-

QL

QL-

ER (Nash

ER (Nash vs vs Pareto) Pareto)

1000 2000 3000 4000 5000 6000 7000 8000 9000

10000

0.2 0.4 0.6 0.8 1

Auction rounds Frequency

Discriminatory Auction

Nash Pareto 1000 2000 3000 4000 5000 6000 7000 8000 9000

10000

0.2 0.4 0.6 0.8 1

Auction rounds Frequency

Uniform Auction

Nash Pareto

SLIDE 26

10/20/2006 "Complex Markets" meeting, Marseille 26\35

LD2 LD2-

QL

QL-

ER (Profits

ER (Profits vs vs Costs) Costs)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 10 20 30 40 50 60 70 80 90 100

Auction rounds Costs / Payments

Payments to suppliers UA Payments to suppliers DA Total generation costs

SLIDE 27

10/20/2006 "Complex Markets" meeting, Marseille 27\35

Example of Market Design Example of Market Design Testing LD2 Testing LD2-

QL case with

QL case with a different Rationing Rule: a different Rationing Rule: Cost Rationing Rule Cost Rationing Rule

SLIDE 28

10/20/2006 "Complex Markets" meeting, Marseille 28\35

LD2 LD2-

QL

QL-

CR (Nash

CR (Nash vs vs Pareto) Pareto)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0.2 0.4 0.6 0.8 1

Auction rounds Frequency

Discriminatory Auction

Nash Pareto 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0.2 0.4 0.6 0.8 1

Auction rounds Frequency

Uniform Auction

Nash Pareto

SLIDE 29

10/20/2006 "Complex Markets" meeting, Marseille 29\35

LD2 LD2-

QL

QL-

CR (Profits

CR (Profits vs vs Costs) Costs)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 10 20 30 40 50 60 70 80 90 100

Auction rounds Costs / Payments

Payments to suppliers UA Payments to suppliers DA Total generation costs

SLIDE 30

10/20/2006 "Complex Markets" meeting, Marseille 30\35

Example of Policy Design Example of Policy Design

Extension of the LD2 Extension of the LD2-

ER case where a third

ER case where a third producer/seller is assumed to entry into the producer/seller is assumed to entry into the market following an antitrust action drawn to market following an antitrust action drawn to reduce market power by distributing the total reduce market power by distributing the total productive capacity from the original two to three productive capacity from the original two to three sellers. sellers.

Seller 1 Seller 1 Seller 2 Seller 2 4 4 4 4 MC MC 4 4 5.5 5.5 7 7 10 10 10 10 Seller 3 Seller 3 Demand Demand Max Q Max Q 4 4 6 6 Max P Max P 10 10 10 10

SLIDE 31

10/20/2006 "Complex Markets" meeting, Marseille 31\35

LD3 LD3-

MM

MM-

ER (Nash

ER (Nash vs vs Pareto) Pareto)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0.2 0.4 0.6 0.8 1

Auction rounds

Uniform Auction

Nash Pareto

Frequency

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0.2 0.4 0.6 0.8 1

Auction rounds

Discriminatory Auction

Frequency

Nash Pareto

SLIDE 32

10/20/2006 "Complex Markets" meeting, Marseille 32\35

LD3 LD3-

MM

MM-

ER (Profits

ER (Profits vs vs Costs) Costs)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 5 10 15 20 25 30 35 40

Uniform Auction case

Auction rounds

Most efficient seller 2nd seller Least efficient seller

Profits

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 5 10 15 20 25 30 35 40

Discriminatory Auction case

Auction rounds

Most efficient seller 2nd seller Least efficient seller

Profits

SLIDE 33

10/20/2006 "Complex Markets" meeting, Marseille 33\35

Conclusions Conclusions

RESULTS

Market design experiment on auction efficiency:

Market design experiment on auction efficiency:

– – DA DA-

ER is the most efficient mechanism with both QL and

ER is the most efficient mechanism with both QL and MM algorithm MM algorithm – – UA UA-

CR is the most efficient mechanism with QL algorithm

CR is the most efficient mechanism with QL algorithm

Policy design experiment on auction efficiency:

Policy design experiment on auction efficiency:

– – The anti The anti-

trust action reduces market power, the payment to

trust action reduces market power, the payment to suppliers decreases. suppliers decreases.

Importance of bi

Importance of bi-

dimensional strategy space

dimensional strategy space

– – New competitive market solutions appear. New competitive market solutions appear.

SLIDE 34

10/20/2006 "Complex Markets" meeting, Marseille 34\35

Future Activities Future Activities

COMPUTATIONAL

COMPUTATIONAL

– – Extending the multi Extending the multi-

agent framework

agent framework

increasing the number of sellers

increasing the number of sellers

adopting multi

adopting multi-

agent learning algorithm (Nash

agent learning algorithm (Nash-

Q

Q-

learning)

learning)

THEORETICAL

THEORETICAL

– – Theoretical analysis of the bi Theoretical analysis of the bi-

dimensional case in the

dimensional case in the

ne
ne-
stage game hypothesis

stage game hypothesis

SLIDE 35

10/20/2006 "Complex Markets" meeting, Marseille 35\35

References

von der Fehr, N. and Harbord, D. (1993). Spot market competition in the UK electricity

industry. Economic Journal, 103:531–546.

Fabra, N. (2003). Tacit collusion in repeated auctions: uniform versus discrimatory. J Industrial Economics, 51(3):271–293. Fabra, N., von der Fehr, N. and Harbord, D. (2005). Designing electricity auctions. RAND J. Economics. Nicolaisen, J., Petrov, V., and Tesfatsion, L. (2001). Market power and efficiency in a computational electricity market with discriminatory double-auction pricing. IEEE T Evolut Comput, 5(5):504–523. Bower, J. and Bunn, D.W. (2001). Experimental analysis of the efficiency of uniform-price versus discriminatory auctions in the England and Wales electricity market. Journal of Economic Dynamics and Control, 25(3-4):561–592. Marimon, R. and McGrattan, E. (1995). On adaptive learning in strategic games. In Kirman,

A. and Salmon, M., editors, Learning and Rationality in Economics, pages 63–101. Blackwell.

SLIDE 36

10/20/2006 "Complex Markets" meeting, Marseille 36\35

THANKS for your attention

SLIDE 37

10/20/2006 "Complex Markets" meeting, Marseille 37\35

Economic Economic Scenarios Scenarios

1. 1. HD HD-

MM

MM-

ER:

ER: High

High-

demand situation with MM adaptive evolutionary

demand situation with MM adaptive evolutionary

2. 2. HD HD-

QL

QL-

ER:

ER: High

High-

demand situation with Q

demand situation with Q-

learning

learning

High Demand characteristics: Both sellers are involved in High Demand characteristics: Both sellers are involved in

rder to satisfy the demand
rder to satisfy the demand

{ }

1 2

max ,

d s s

Q Q Q >

SLIDE 38

10/20/2006 "Complex Markets" meeting, Marseille 38\35

HD HD-

MM (Nash

MM (Nash vs vs Pareto) Pareto)

1000 2000 3000 4000 5000 6000 7000 8000 9000

10000

0.2 0.4 0.6 0.8 1

Auction rounds Frequency

Discriminatory Auction

Nash Pareto 1000 2000 3000 4000 5000 6000 7000 8000 9000

10000

0.2 0.4 0.6 0.8 1

Auction rounds Frequency

Uniform Auction

Nash Pareto

SLIDE 39

10/20/2006 "Complex Markets" meeting, Marseille 39\35

Payoff Matrix DA Payoff Matrix DA

ps ps 1 1 qs qs 1 1 ps ps 2 2 qs qs 2 2 Payoff 1 Payoff 1 Payoff 2 Payoff 2 Nash Nash Pareto Pareto 10 10 6 6 10 10 6 6 33.00 33.00 16.50 16.50 X X X X 10 10 5 5 10 10 6 6 30.00 30.00 18.00 18.00 X X

SLIDE 40

10/20/2006 "Complex Markets" meeting, Marseille 40\35

Payoff Matrix UA Payoff Matrix UA

ps ps 1 1 qs qs 1 1 ps ps 2 2 qs qs 2 2 Payoff 1 Payoff 1 Payoff 2 Payoff 2 Nash Nash Pareto Pareto 9 9 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 10 10 6 6 9 9 6 6 30.00 30.00 18.00 18.00 X X X X 8 8 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 7 7 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 6 6 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 10 10 6 6 8 8 6 6 30.00 30.00 18.00 18.00 X X X X 5 5 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 4 4 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 3 3 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 1 1 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 2 2 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 6 6 10 10 6 6 36.00 36.00 15.00 15.00 X X X X 10 10 6 6 7 7 6 6 30.00 30.00 18.00 18.00 X X X X 10 10 6 6 6 6 6 6 30.00 30.00 18.00 18.00 X X X X 10 10 6 6 5 5 6 6 30.00 30.00 18.00 18.00 X X X X 10 10 6 6 3 3 6 6 30.00 30.00 18.00 18.00 X X X X 10 10 6 6 4 4 6 6 30.00 30.00 18.00 18.00 X X X X 10 10 6 6 2 2 6 6 30.00 30.00 18.00 18.00 X X X X 10 10 6 6 1 1 6 6 30.00 30.00 18.00 18.00 X X X X 10 10 6 6 6 6 30.00 30.00 18.00 18.00 X X X X

SLIDE 41

10/20/2006 "Complex Markets" meeting, Marseille 41\35

HD HD-

MM (Profits

MM (Profits vs vs Costs) Costs)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 20 40 60 80 100 120 Auction rounds Costs / Payments Payments to suppliers UA Payments to suppliers DA Total generation costs

SLIDE 42

10/20/2006 "Complex Markets" meeting, Marseille 42\35

HD HD-

QL (Nash

QL (Nash vs vs Pareto) Pareto)

1000 2000 3000 4000 5000 6000 7000 8000 9000

10000

0.2 0.4 0.6 0.8 1

Auction rounds Frequency

Discriminatory Auction

Nash Pareto 1000 2000 3000 4000 5000 6000 7000 8000 9000

10000

0.2 0.4 0.6 0.8 1

Auction rounds Frequency

Uniform Auction

Nash Pareto

SLIDE 43

10/20/2006 "Complex Markets" meeting, Marseille 43\35

HD HD-

QL (Profits

QL (Profits vs vs Costs) Costs)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 20 40 60 80 100 120

Auction rounds Costs / Payments

Payments to suppliers UA Payments to suppliers DA Total generation costs

SLIDE 44

10/20/2006 "Complex Markets" meeting, Marseille 44\35

Remarks Remarks HD HD

In the UA case

In the UA case Nash Nash equilibria equilibria are are also also Pareto Pareto optima

ptima
In the

In the long long-

run

run Nash Nash solutions solutions prevail prevail in the in the context context of

f

R R-

matrix

matrix game, game, as as expected expected

UA

UA mechanism mechanism is is less less efficient efficient than than the DA the DA mechanism mechanism SOLUTIONS: AUCTION MECHANISM:

SLIDE 45

10/20/2006 "Complex Markets" meeting, Marseille 45\35

Solution Concepts (II) Solution Concepts (II)

In the context of infinitely repeated games with rational

In the context of infinitely repeated games with rational players, the players, the Folk Theorem states that tacit collusive Folk Theorem states that tacit collusive behavior may arise, with the players repeatedly behavior may arise, with the players repeatedly selecting Pareto optima of the one selecting Pareto optima of the one-

shot game (R

shot game (R-

matrix game).

matrix game).

In the case of bounded rational players we expect that

In the case of bounded rational players we expect that joint stationary policy of playing Pareto optima of the joint stationary policy of playing Pareto optima of the R R-

matrix game are equilibrium solutions for the players

matrix game are equilibrium solutions for the players intertemporal intertemporal optimizing (Q

ptimizing (Q-
matrix game)