Application of Information Theory, Introduction Iftach Haitner Tel - - PowerPoint PPT Presentation

▶
application of information theory introduction
SMART_READER_LITE
LIVE PREVIEW

Application of Information Theory, Introduction Iftach Haitner Tel - - PowerPoint PPT Presentation

Oct 12, 2023 312 likes •593 views

Application of Information Theory, Introduction Iftach Haitner Tel Aviv University. October 28, 2014 Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 1 / 12 Section 1 Administration Iftach Haitner (TAU)

slide-1
SLIDE 1

Application of Information Theory, Introduction

Iftach Haitner

Tel Aviv University.

October 28, 2014

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 1 / 12

slide-2
SLIDE 2

Section 1 Administration

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 2 / 12

slide-3
SLIDE 3

Important Details

  • 1. Iftach Haitner. Schriber 20, email iftachh at gmail.com

Reception: Sundays 9:00-10:00 (please coordinate via email in advance)

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 3 / 12

slide-4
SLIDE 4

Important Details

  • 1. Iftach Haitner. Schriber 20, email iftachh at gmail.com

Reception: Sundays 9:00-10:00 (please coordinate via email in advance)

  • 2. Who are you?

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 3 / 12

slide-5
SLIDE 5

Important Details

  • 1. Iftach Haitner. Schriber 20, email iftachh at gmail.com

Reception: Sundays 9:00-10:00 (please coordinate via email in advance)

  • 2. Who are you?
  • 3. Mailing list: 0368-4486-01@listserv.tau.ac.il

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 3 / 12

slide-6
SLIDE 6

Important Details

  • 1. Iftach Haitner. Schriber 20, email iftachh at gmail.com

Reception: Sundays 9:00-10:00 (please coordinate via email in advance)

  • 2. Who are you?
  • 3. Mailing list: 0368-4486-01@listserv.tau.ac.il

◮ Registered students are automatically on the list (need to activate

the account by going to https://www.tau.ac.il/newuser/)

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 3 / 12

slide-7
SLIDE 7

Important Details

  • 1. Iftach Haitner. Schriber 20, email iftachh at gmail.com

Reception: Sundays 9:00-10:00 (please coordinate via email in advance)

  • 2. Who are you?
  • 3. Mailing list: 0368-4486-01@listserv.tau.ac.il

◮ Registered students are automatically on the list (need to activate

the account by going to https://www.tau.ac.il/newuser/)

◮ If you’re not registered and want to get on the list (or want to get

another address on the list), send e-mail to: listserv@listserv.tau.ac.il with the line: subscribe 0368-4486-01 <Real Name>

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 3 / 12

slide-8
SLIDE 8

Important Details

  • 1. Iftach Haitner. Schriber 20, email iftachh at gmail.com

Reception: Sundays 9:00-10:00 (please coordinate via email in advance)

  • 2. Who are you?
  • 3. Mailing list: 0368-4486-01@listserv.tau.ac.il

◮ Registered students are automatically on the list (need to activate

the account by going to https://www.tau.ac.il/newuser/)

◮ If you’re not registered and want to get on the list (or want to get

another address on the list), send e-mail to: listserv@listserv.tau.ac.il with the line: subscribe 0368-4486-01 <Real Name>

  • 4. Course website:

.../~iftachh/Courses/Info/Fall14/index.html (or just Google iftach and follow the link)

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 3 / 12

slide-9
SLIDE 9

Grades

  • 1. Class exam 80

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 4 / 12

slide-10
SLIDE 10

Grades

  • 1. Class exam 80
  • 2. Homework 20%: 5-6 exercises.

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 4 / 12

slide-11
SLIDE 11

Grades

  • 1. Class exam 80
  • 2. Homework 20%: 5-6 exercises.

◮ Recommended to use use L

A

T EX (see link in course website)

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 4 / 12

slide-12
SLIDE 12

Grades

  • 1. Class exam 80
  • 2. Homework 20%: 5-6 exercises.

◮ Recommended to use use L

A

T EX (see link in course website)

◮ Exercises should be sent to nisnis.levi at gmail.com or put in

mailbox ?, in time!

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 4 / 12

slide-13
SLIDE 13

and..

  • 1. Slides

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 5 / 12

slide-14
SLIDE 14

and..

  • 1. Slides
  • 2. English

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 5 / 12

slide-15
SLIDE 15

Section 2 Information Theory

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 6 / 12

slide-16
SLIDE 16

Information Theory

◮ Started in the paper A Mathematical Theory of Communication by

Claude Shannon.

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 7 / 12

slide-17
SLIDE 17

Information Theory

◮ Started in the paper A Mathematical Theory of Communication by

Claude Shannon.

◮ A paper in Electrical Engineering (EE).

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 7 / 12

slide-18
SLIDE 18

Information Theory

◮ Started in the paper A Mathematical Theory of Communication by

Claude Shannon.

◮ A paper in Electrical Engineering (EE). ◮ One of the most important scientific papers of the 20’th century.

Revolutionized EE, and ideas from this papers are central in almost all area of science: Physics, Mathematics, Computer Science,Chemistry, Biology.

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 7 / 12

slide-19
SLIDE 19

Information Theory

◮ Started in the paper A Mathematical Theory of Communication by

Claude Shannon.

◮ A paper in Electrical Engineering (EE). ◮ One of the most important scientific papers of the 20’th century.

Revolutionized EE, and ideas from this papers are central in almost all area of science: Physics, Mathematics, Computer Science,Chemistry, Biology.

◮ An amazing example of an amazing research done by mainly by

asking the right questions.

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 7 / 12

slide-20
SLIDE 20

Section 3 The Course

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 8 / 12

slide-21
SLIDE 21

Course Topics Information Theory is typically taught in EE. In this course we will focus on the point of view of CS and Math, and less on EE applications.

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 9 / 12

slide-22
SLIDE 22

Course Topics Information Theory is typically taught in EE. In this course we will focus on the point of view of CS and Math, and less on EE applications. First part of the course will follow Ran Raz course.

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 9 / 12

slide-23
SLIDE 23

Course Topics Information Theory is typically taught in EE. In this course we will focus on the point of view of CS and Math, and less on EE applications. First part of the course will follow Ran Raz course.

◮ Axiomatic derivation of Shanon’s entropy ◮ Conditional entropy and mutual information ◮ Relative entropy (Kullback-Leibler information) ◮ Entropy of a continuous random variable ◮ The maximum entropy principle ◮ Huffman coding ◮ The asymptotic equipartition theorem ◮ Data compression ◮ Channel capacity ◮ Shearer’s inequality ◮ Applications to combinatorics ◮ Kolmogorov complexity

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 9 / 12

slide-24
SLIDE 24

Course Topics cont. Second part of the course will focus on computational notions of entropy.

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 10 / 12

slide-25
SLIDE 25

Course Topics cont. Second part of the course will focus on computational notions of entropy.

◮ Parallel repetition of interactive argument ◮ Pseudo entropy and pseudorandom generators ◮ Accessible entropy and statistically hiding commitments

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 10 / 12

slide-26
SLIDE 26

Material

◮ Books:

Thomas Cover & Joy Thomas: Elements of Information Theory.

◮ Papers:

◮ Jaikumar Radhakrishnan: Entropy and Counting. ◮ Thomas Holenstein: Parallel Repetition: Simplifications and the

No-Signaling Case.

◮ Lecture notes:

◮ Anup Rao. ◮ Venkatesan Guruswami and Mahdi Cheraghchi . Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 11 / 12

slide-27
SLIDE 27

Prerequisites Basic probability and calculus.

Iftach Haitner (TAU) Application of Information Theory, Intro October 28, 2014 12 / 12