Models of Computation Z. Sawa (TU Ostrava) Theoretical Computer - - PowerPoint PPT Presentation

Models of Computation

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 1 / 63

Necessity of Formalization of the Notion of “Algorithm”

The above mentioned characterization of the notion of an algorithm was a little bit vague. If we would like to prove for a problem that there is no algorithm solving the given problem, it would probably not be possible with such vague definition of the notion of an algorithm. Intuitively, we understand what properties an algorithm should have: It should consist of simple steps that can be performed “mechanically” without understanding the problem. Objects, with which the algorithm manipulates, and also the performed operations should be finite.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 2 / 63

An Algorithm Performed by a Computer

For example a program in any programming language surely has these properties. Regardless of the programming language in which a program is written, its instructions are always performed on a hardware of some concrete computer on the level of instructions of a processor. So it should be clear that every program in every programming language could be written as a program consisting only of instructions of a machine code of some processor.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 3 / 63

Operation of a Computer

A typical architecture of most of computers looks as follows:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 R5 R4 R3 R2 R1 R7 R6 R8 IP flags

MEM CPU

A computer has a memory consisting of a big number of memory cells. Every cell contains a number of some size, typically 1 byte (8 bits), i.e., a number in the range 0 . . 255. Cells are numbered. The number of a cell is called its address.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 4 / 63

Operation of a Computer

A typical architecture of most of computers looks as follows:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 R5 R4 R3 R2 R1 R7 R6 R8 IP flags

MEM CPU

Instructions are stored in the memory (every instruction has its numerical code and the instructions are executed by a processor). The processor has so called instruction counter IP that contains an address of an instruction that is currently executed. The processor reads an instruction from the address specified by IP, increments IP by the length of the read instruction, and executes the given instruction.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 4 / 63

Operation of a Computer

A typical architecture of most of computers looks as follows:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 R5 R4 R3 R2 R1 R7 R6 R8 IP flags

MEM CPU

A processor usually contains several registers of some fixed length (e.g., 32 or 64 bits). Most of operations are performed on registers. A processor usually contains flags that can be used for testing of the result of the last performed operation (e.g., an overflow, if the result is zero, etc.)

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 4 / 63

Operation of a Computer

Typical instructions: Loading of a value of some memory cell (resp. several consecutive cells) into some register (LOAD). Storing of a value of some register into some memory cell (resp. several consecutive cells) (STORE). Remark: An address of a cell can be either direct (i.e., it is a part of the instruction), or indirect (stored in some register, or computed from values in one or several registers). Copying a value in one register into some other register (MOV). Arithmetic instructions (ADD, SUB, MUL, DIV, NEG, CMP, INC, DEC, . . . ). Logical instructions (AND, OR, XOR, NOT, . . . ). Bit shifts and rotations (SHL, SHR, . . . )

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 5 / 63

Operation of a Computer

Typical instructions (continued): Unconditional jump (JMP). Remark: A destination address can be direct or indirect. Conditional jumps (JZERO, JGTZ, . . . ). Invocation of subroutines (CALL, RET). Some special instructions – manipulation with input and output, an interrupt handling, processor modes, memory access control, paging, etc.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 6 / 63

Operation of a Computer

Example: An instruction written is a high-level language as x = y + 2 can be implemented by the following sequence of instructions of some (hypothetical) processor: LOAD R5,[0x001b7c44] ADD R5,2 STORE [0x001b7c38],R5 Remark: We assume that variable x is stored at address 0x001b7c38 and variable y at the address 0x001b7c44.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 7 / 63

Random Access Machine

A Random Access Machine (RAM) is an idealized model of a computer. It consists of the following parts: Program unit – contains a program for the RAM and a pointer to the currently executed instruction Working memory consists of cells numbered 0, 1, 2, . . . ; the content

• f the cells can be read and written to

Input tape – read-only Output tape – write-only

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 8 / 63

Random Access Machine

3 2 1 4 5 6 8 7 1 3 6 10 13 14 12 11 9 8 7 5 4 2 READ JZERO 10 STORE *3 ADD 2 STORE 2 ADD =1 JUMP 1 LOAD 2 DIV 1 STORE 2 STORE 1 LOAD =0 STORE 1 LOAD 1

7 5 2

IC ALU

input

• utput

working memory program unit

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 9 / 63

Random Access Machine

Cells 0 and 1 are special and are used as registers of the RAM: Cell 0 – a working register (accumulator) – the register that is one

• f operands for most of instructions and to which the result of most
• f operations is stored.

Cell 1 – an index register – it is used for indirect addressing. Forms of operands of instructions (i ∈ N): form the value of the operand =i the number i i the number stored in the cell at address i *i the number stored in the cell at address i + j, where j is the current value of the index register

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 10 / 63

Random Access Machine

Example: LOAD <op> reads the value of the operand <op> into the working register (i.e., into cell 0). LOAD =5 – loads 5 into the working register LOAD 5 – loads the value in cell 5 into the working register LOAD *5 – loads the value in cell 5 + j, where j is the cur- rent value of the index register, into the working register

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 11 / 63

Random Access Machine

Input and output instructions (they have no operands): READ – the value from the cell of input tape on which is the reading head is stored into the working register and the reading head is moved right WRITE – the value in the working register is written on the output tape and the output head is moved right by one Intructions working with memory: LOAD <op> – the value of the operand is loaded into the work- ing register STORE <op> – the value of the operand is rewritten with the content of the working register (an operand of the form =i is not allowed)

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 12 / 63

Random Access Machine

Instruction for arithmetic operations: ADD <op> – the value in the working register is increased by the value of the operand (i.e., the value of the

• perand is added to it)

SUB <op> – the value of the operand subtracted from the value of the working register MUL <op> – the value in the working register is multiplied by the value of the operand DIV <op> – the value in the working register is divided by the value of the operand (the result is truncated to an integer value)

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 13 / 63

Random Access Machine

Jump instructions: JUMP <label> – the computation will continue with an in- struction determined by the label JZERO <label> – if the working register contains value 0, the computation will continue with an instruc- tion determined by the label; otherwise it will continue with the following instruction JGTZ <label> – if the working register contains a positive value, the computation will continue with an instruction determined by the label; oth- erwise it will continue with the following in- struction A halting instruction: HALT – the computation is halted

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 14 / 63

Random Access Machine

Problem “Searching”

Input: An integer x and a sequence of integers a1, a2, . . . , an (where ai = 0) terminated by 0. Output: If ai = x then the output is i (if there are several such i then the smallest one), otherwise the output is 0. start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 15 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: Cell 1: Cell 2: Cell 3: Cell 4: . . . Output: Instructions: 0

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: Cell 1: Cell 2: Cell 3: Cell 4: . . . Output: Instructions: 0

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 9 Cell 1: Cell 2: Cell 3: Cell 4: . . . Output: Instructions: 1

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 9 Cell 1: Cell 2: Cell 3: 9 Cell 4: . . . Output: Instructions: 2

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 1 Cell 1: Cell 2: Cell 3: 9 Cell 4: . . . Output: Instructions: 3

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 1 Cell 1: Cell 2: 1 Cell 3: 9 Cell 4: . . . Output: Instructions: 4

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 13 Cell 1: Cell 2: 1 Cell 3: 9 Cell 4: . . . Output: Instructions: 5

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 13 Cell 1: Cell 2: 1 Cell 3: 9 Cell 4: . . . Output: Instructions: 6

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 4 Cell 1: Cell 2: 1 Cell 3: 9 Cell 4: . . . Output: Instructions: 7

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 4 Cell 1: Cell 2: 1 Cell 3: 9 Cell 4: . . . Output: Instructions: 8

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 1 Cell 1: Cell 2: 1 Cell 3: 9 Cell 4: . . . Output: Instructions: 9

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 2 Cell 1: Cell 2: 1 Cell 3: 9 Cell 4: . . . Output: Instructions: 10

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 2 Cell 1: Cell 2: 1 Cell 3: 9 Cell 4: . . . Output: Instructions: 11

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 2 Cell 1: Cell 2: 2 Cell 3: 9 Cell 4: . . . Output: Instructions: 12

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 5 Cell 1: Cell 2: 2 Cell 3: 9 Cell 4: . . . Output: Instructions: 13

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 5 Cell 1: Cell 2: 2 Cell 3: 9 Cell 4: . . . Output: Instructions: 14

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0:

• 4

Cell 1: Cell 2: 2 Cell 3: 9 Cell 4: . . . Output: Instructions: 15

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0:

• 4

Cell 1: Cell 2: 2 Cell 3: 9 Cell 4: . . . Output: Instructions: 16

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 2 Cell 1: Cell 2: 2 Cell 3: 9 Cell 4: . . . Output: Instructions: 17

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 3 Cell 1: Cell 2: 2 Cell 3: 9 Cell 4: . . . Output: Instructions: 18

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 3 Cell 1: Cell 2: 2 Cell 3: 9 Cell 4: . . . Output: Instructions: 19

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 3 Cell 1: Cell 2: 3 Cell 3: 9 Cell 4: . . . Output: Instructions: 20

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 9 Cell 1: Cell 2: 3 Cell 3: 9 Cell 4: . . . Output: Instructions: 21

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 9 Cell 1: Cell 2: 3 Cell 3: 9 Cell 4: . . . Output: Instructions: 22

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: Cell 1: Cell 2: 3 Cell 3: 9 Cell 4: . . . Output: Instructions: 23

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: Cell 1: Cell 2: 3 Cell 3: 9 Cell 4: . . . Output: Instructions: 24

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 3 Cell 1: Cell 2: 3 Cell 3: 9 Cell 4: . . . Output: Instructions: 25

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 3 Cell 1: Cell 2: 3 Cell 3: 9 Cell 4: . . . Output: 3 Instructions: 26

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

start: READ STORE 3 LOAD =1 loop: STORE 2 READ JZERO

• utput

SUB 3 JZERO found LOAD 2 ADD =1 JUMP loop found: LOAD 2

• utput:

WRITE HALT Input: 9, 13, 5, 9, 7, 2, 0 Cell 0: 3 Cell 1: Cell 2: 3 Cell 3: 9 Cell 4: . . . Output: 3 Instructions: 27

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 16 / 63

Random Access Machine

Main differences with respect to real computers: The size of memory is not limited (an address can be an arbitrary natural number). The size of a content of individual memory cells is not limited (a cell can contain an arbitrary integer). It reads data sequantially from an input that consists of a sequence of

• integers. The input is read-only.

It writes data sequantially on the output that consists of a sequence

• f integers. The output is write-only.
• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 17 / 63

Random Access Machine and Turing Machine

Any program in any programming language can be implemented as a program for a RAM. It is not difficult (although a little bit tedious) to realize that every algorithm performed by a RAM can also be implemented by a Turing machine. A Turing machine can implement any algorithm that can be written as a program in an arbitrary programming language.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 18 / 63

Simulation of a Computation

Explanation what it means that a machine M is simulated by a machine M′: A computation of machine M for input w is a (finite or infinite) sequence of configurations of machine M α0 − → α1 − → α2 − → · · · For this computation, there is a corresponding computation of machine M′ consisting of configurations α′

0 −

→ α′

1 −

→ α′

2 −

→ · · · where for every configuration αi there is some corresponding configuration α′

f (i) where f : N → N is a function, for which

f (i) ≤ f (j) for every i and j where i < j. There is a function mapping configurations of machine M to corresponding configurations of machine M′. There are functions mapping an input w to corresponding initial configurations α0 and α′

0 and analogously functions mapping final

configurations to a result of computation.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 19 / 63

Random Access Machine and Turing Machine

A Turing machine works with words over some alphabet, while a RAM works with numbers. But numbers can be written as sequences of symbols and conversely symbols of an alphabet can be written as numbers. For example the following input of a RAM 5 13 -3 0 6 can be represented for a Turing machine as # 1 0 1 # 1 1 0 1 # - 1 1 # 0 # 1 1 0 #

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 20 / 63

Multitape Turing Machine

When constructing a Turing machine for a given RAM, it can be simpler to construct at first a multitape Turing machine:

q

a b b a a a b a b b b a b b b b a a a b a a b a b b a

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 21 / 63

Multitape Turing Machine

A computation of a multitape Turing machine a b b a a a b a b b b a b b b b a a a b a a b a b b a can be simulated by a one tape Turing machine a b a a b a b b a b b a b b b b a a a b b a a a b a b

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 22 / 63

Multitape Turing Machine

Example: A machine that gets as an input two natural numbers written in binary and separated by symbols # (e.g., number 6 and 11 will be written as “#110#1011#”), and computes their sum.

q

1 # # 1 1 1 # # 1 1 #

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 23 / 63

Turing Machine Simulating RAM

A Turing machine simulating a computation of a RAM has several tapes: A tape representing the input tape of the RAM. A tape representing the output tape of the RAM. A tape containg the value of the working register. A tape containg the value of the index register. A tape containing the values of all other cells of the memory of the RAM. A tape containing the address of a memory cell on which the current

• peration is performed.

Some other auxiliary tapes (to store some intermediate values, etc.)

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 24 / 63

Turing Machine Simulating RAM

The Turing machine stores the information about the instruction of the RAM that is currently executed in its control unit. Execution of most of instructions is not difficult: Instruction READ – to copy the value (marked at the ends by symbols “#”) from the input tape to the tape representing the working register. Instruction WRITE – to copy the value of the working register to the

• utput tape.

Instruction JUMP – just changes the state of the control unit of the Turing machine. Instrukce JZERO a JGTZ – the content of the working register is tested and the state of the control unit is changed accordingly.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 25 / 63

Turing Machine Simulating RAM

Arithmetic instructions (ADD, SUB, MUL, DIV) are also quite simple: A value of the second operand is copied to an auxiliary tape. The operation (e.g., addition) is performed bit by bit, the result is stored to some other auxiliary tape. The result is copied to the tape containg the value of the working register. Remark: Multiplication and division can be done as a sequence of additions and bit shifts.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 26 / 63

Turing Machine Simulating RAM

Probably the most complex is the implementation of the RAM memory. One possibility is to store only values of those cells that were actually used so far in the computation of the RAM (we know that all other cells contain value 0). Example: The RAM worked so far only with cells 2, 3 and 6: Cell 2 contains value 11. Cell 3 contains value −1. Cell 6 contains value 2. The content of the tape of the Turing machine representing the content of the memory of the RAM will be as follows: $ # 1 0 : 1 0 1 1 # 1 1 : - 1 # 1 1 0 : 1 0 # $

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 27 / 63

Turing Machine Simulating RAM

The instruction LOAD: The respective address is written on the tape containing the current working address. The address is searched on the tape with the content of the memory. (If it is not found, it is appened at the end with value 0.) The value at the addess is copied to the tape containg the value of the working register.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 28 / 63

Turing Machine Simulating RAM

The instruction STORE: Similarly to the instruction LOAD, the position on the memory tape is found. The rest of the memory tape is copied to an auxiliary tape. The value of the working register is copied at the corresponding position. The rest of the tape, copied on the auxiliary tape, is copied back on the memory tape (after the newly written value).

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 29 / 63

Church-Turing Thesis

Church-Turing thesis

Every algorithm can be implemented as a Turing machine. It is not a theorem that can be proved in a mathematical sense – it is not formally defined what an algorithm is. The thesis was formulated in 1930s independently by Alan Turing and Alonzo Church.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 30 / 63

Church-Turing Thesis

Examples of mathematical formalisms modelling the notion of an algorithm: Random Access Machines Turing machines Lambda calculus Recursive functions . . . We can also mention: An arbitrary (general purpose) programming language (for example C, Java, Lisp, Haskell, Prolog, etc.). All these models are equivalent with respect to algorithms that can be implemented by them.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 31 / 63

Models of Computation

Such languages (resp. machines), which are general enough, so that programs written in any other programming language can be translated to them, are called Turing complete.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 32 / 63

Examples of Computational Models

All the following models have a finite control unit equipped with some sort

• f memory of unbounded size.

Such memory can constist of one of more structures such as: Tape — reading and writing a symbol on a current position, movement of the head to the left and to the right Remark: The tape can be infinite of one side or on both sides. Stack — push, pop, a test of emptiness of the stack Counter — a value is a natural number, operations of incrementing and decrementing by one, a test whether the value is equal to zero

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 33 / 63

Examples of Computational Models

A tape infinite on one side can be viewed as a special case of a tape infinite on both sides. A stack can be viewed as a special case of a tape infinite on one side. A counter can be viewed as a special case of a stack.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 34 / 63

Examples of Computational Models

A tape infinite of both sides can be simulated by a pair of stacks. A stack can be simulated using a pair of counters — a value of the first counter represents the content of the stack as a number of base k = |Γ| (where Γ is a stack alphabet).

A stack on the top of the stack — the remainder of division by k Pop — to divide by k Push — to multiply by k and to add the code of the given symbol

Division and multiplication by a constant can be implemented by successive addition and subtraction using the second counter.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 35 / 63

Examples of Computational Models

Any finite number of counters can be simulated by two counters. One counter represents values of all counters — e.g., values of counters x, y, z can be represented number 2x3y5z. Incrementing counter x by one is simulated as multiplying by 2, incrementing counter y by one is simulated as multiplying by 3, etc. In a similar way, decrementing by one is simulated by division by a corresponding constant. All these operations can be implemented by successive addition and subtraction using the second counter.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 36 / 63

Universal Turing Machine

A description of arbitrary Turing machine M (or other computation model) can be represented in a form of data — e.g., a word over some alphabet. Let Code(M) denote such representation of machine M (in some particular format). A universal Turing machine U is a machine that when obtains a word Code(M)#w as an input, where w is a word over alphabet Σ, starts to simulate the computation of machine M on input w. So the universal Turing machine is able to perform a computation of any

• ther Turing machine (whose description it obtains as a part of its input).

Remark: This corresponds to situation where we have a hardware of a computer (the machine U) that able to perform any algorithm given in a form of a program (software) — Code(M).

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 37 / 63

Computation of a Turing machine as Data

Not only a description of a Turing machine can viewed as data. Also a computation (resp. a desription of this computation) of a given machine M on some input w can be viewed as a certain sort of data. For example, configurations of a Turing machine M = (Q, Σ, Γ, δ, q0, F) can be encoded as words over alphabet ∆ = Γ ∪ (Q × Γ): In a representation of a configuration, there is always exactly one symbol from the set (Q × Γ) — it represents the control state and the position of the head. Remaining symbols (from set Γ) represent the content of the tape. A computation can be represented as a sequence of configurations separated by symbol # ∈ ∆.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 38 / 63

C0 C1 C2 C3 Ci Ct−1 Ct β β1 β2 β3 a2 a3 an

• q0

qacc a1 x 1 2 3 j n n+1

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 39 / 63

Undecidable Problems

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 40 / 63

Algorithmically Solvable Problems

Let us assume we have a problem P. If there is an algorithm solving the problem P then we say that the problem P is algorithmically solvable. If P is a decision problem and there is an algorithm solving the problem P then we say that the problem P is decidable (by an algorithm). If we want to show that a problem P is algorithmically solvable, it is sufficient to show some algorithm solving it (and possibly show that the algorithm really solves the problem P).

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 41 / 63

Algorithmically Unsolvable Problems

A problem that is not algorithmically solvable is algorithmically unsolvable. A decision problem that is not decidable is undecidable. Surprisingly, there are many (exactly defined) problems, for which it was proved that they are not algorithmically solvable.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 42 / 63

Halting Problem

Let us consider some general programming language L. Futhermore, let us assume that programs in language L run on some idealized machine where a (potentially) unbounded amount of memory is available — i.e., the allocation of memory never fails. Example: The following problem called the Halting problem is undecidable:

Halting problem

Input: A source code of a L program P, input data x. Question: Does the computation of P on the input x halt after some finite number of steps?

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 43 / 63

Halting Problem

Let us assume that there is a program that can decide the Halting problem. So we could construct a subroutine H, declared as Bool H(String code, String input) where H(P, x) returns: true if the program P halts on the input x, false if the program P does not halt on the input x. Remark: Let us say that subroutine H(P, x) returns false if P is not a syntactically correct program.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 44 / 63

Halting Problem

Using the subroutine H we can construct a program D that performs the following steps: It reads its input into a variable x of type String. It calls the subroutine H(x, x). If subroutine H returns true, program D jumps into an infinite loop loop: goto loop In case that H returns false, program D halts. What does the program D do if it gets its own code as an input?

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 45 / 63

Halting Problem

If D gets its own code as an input, it either halts or not. If D halts then H(D, D) returns true and D jumps into the infinite

• loop. A contradiction!

If D does not halt then H(D, D) returns false and D halts. A contradiction! In both case we obtain a contradiction and there is no other possibility. So the assumption that H solves the Halting problem must be wrong.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 46 / 63

Semidecidable Problems

A problem is semidecidable if there is an algorithm such that: If it gets as an input an instance for which the answer is Yes, then it halts after some finite number of steps and writes "YES" on the

• utput.

If it gets as an input an instance for which the answer is No, then it either halts and writes "NO" on the input, or does not halt and runs forever. It is obvious that for example HP (Halting Problem) is semidecidable. Some problems are not even semidecidable.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 47 / 63

Post’s Theorem

The complement problem for a given decision problem P is a problem where inputs are the same as for the problem P and the question is negation of the question from the problem P.

Post’s Theorem

If a problem P and its complement problem are semidecidable then the problem P is decidable.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 48 / 63

Reduction between Problems

If we have already proved a (decision) problem to be undecidable, we can prove undecidability of other problems by reductions. Problem P1 can be reduced to problem P2 if there is an algorithm Alg such that: It can get an arbitrary instance of problem P1 as an input. For an instance of a problem P1 obtained as an input (let us denote it as w) it produces an instance of a problem P2 as an output. It holds i.e., the answer for the input w of problem P1 is Yes iff the answer for the input Alg(w) of problem P2 is Yes.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 49 / 63

Reductions between Problems

Inputs of problem P1 Inputs of problem P2 Yes Yes No No

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 50 / 63

Reductions between Problems

Inputs of problem P1 Inputs of problem P2 Yes Yes No No Alg

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 50 / 63

Reductions between Problems

Let us say there is some reduction Alg from problem P1 to problem P2. If problem P2 is decidable then problem P1 is also decidable. Solution of problem P1 for an input x: Call Alg with x as an input, it returns a value Alg(x). Call the algorithm solving problem P2 with input Alg(x). Write the returned value to the output as the result. It is obvious that if P1 is undecidable then P2 cannot be decidable.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 51 / 63

Other Undecidable Problems

By reductions from the Halting problem we can show undecidability of many other problems dealing with a behaviour of programs: Is for some input the output of a given program Yes? Does a given program halt for an arbitrary input? Do two given programs produce the same outputs for the same inputs? . . .

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 52 / 63

Halting Problem

For purposes of proofs, the following version of Halting problem is often used:

Halting problem

Input: A description of a Turing machine M and a word w. Question: Does the computation of the machine M on the word w halt after some finite number of steps?

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 53 / 63

Other Undecidable Problems

We have already seen the following example of an undecidable problem:

Problem

Input: Context-free grammars G1 and G2. Question: Is L(G1) = L(G2)? respectively

Problem

Input: A context-free grammar generating a language over an alphabet Σ. Question: Is L(G) = Σ∗?

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 54 / 63

Other Undecidable Problems

An input is a set of types of cards, such as:

a aa abb bbab bab ab baba aa aba a

The question is whether it is possible to construct from the given types of cards a non-empty finite sequence such that the concatenations of the words in the upper row and in the lower row are the same. Every type of a card can be used repeatedly.

a aa abb bbab abb bbab baba aa abb bbab aba a

In the upper and in the lower row we obtained the word aabbabbbabaabbaba.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 55 / 63

Other Undecidable Problems

Undecidability of several other problems dealing with context-free grammars can be proved by reductions from the previous problem:

Problem

Input: Context-free grammars G1 and G2. Question: Is L(G1) ∩ L(G2) = ∅?

Problem

Input: A context-free grammar G. Question: Is G ambiguous?

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 56 / 63

Other Undecidable Problems

An input is a set of types of tiles, such as: The question is whether it is possible to cover every finite area of an arbitrary size using the given types of tiles in such a way that the colors of neighboring tiles agree. Remark: We can assume that we have an infinite number of tiles of all types. The tiles cannot be rotated.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 57 / 63

Other Undecidable Problems

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 58 / 63

Other Undecidable Problems

Problem

Input: A closed formula of the first order predicate logic where the

• nly predicate symbols are = and <, the only function

symbols are + and ∗, and the only constant symbols are 0 and 1. Question: Is the given formula true in the domain of natural numbers (using the natural interpretation of all function and predicate symbols)? An example of an input: ∀x∃y∀z((x ∗ y = z) ∧ (y + 1 = x)) Remark: There is a close connection with G¨

• del’s incompleteness

theorem.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 59 / 63

Other Undecidable Problems

It is interesting that an analogous problem, where real numbers are considered instead of natural numbers, is decidable (but the algorithm for it and the proof of its correctness are quite nontrivial). Also when we consider natural numbers or integers and the same formulas as in the previous case but with the restriction that it is not allowed to use the multiplication function symbol ∗, the problem is algorithmically decidable.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 60 / 63

Other Undecidable Problems

If the function symbol ∗ can be used then even the very restricted case is undecidable:

Hilbert’s tenth problem

Input: A polynomial f (x1, x2, . . . , xn) constructed from variables x1, x2, . . . , xn and integer constants. Question: Are there some natural numbers x1, x2, . . . , xn such that f (x1, x2, . . . , xn) = 0 ? An example of an input: 5x2y − 8yz + 3z2 − 15 I.e., the question is whether ∃x∃y∃z(5 ∗ x ∗ x ∗ y + (−8) ∗ y ∗ z + 3 ∗ z ∗ z + (−15) = 0) holds in the domain of natural numbers.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 61 / 63

Other Undecidable Problems

Also the following problem is algorithmically undecidable:

Problem

Input: A closed formula ϕ of the first-order predicate logic. Question: Is | = ϕ ? Remark: Notation | = ϕ denotes that formula ϕ is logically valid, i.e., it is true in all interpretations.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 62 / 63

Rice’s Theorem

Let P be an arbitrary prorperty of Turing machines. The property P is: nontrivial — if there exists at least one machine that has the property P, and at least one machine that does not have the property P input-output — if in every pair of machines that halt on the same inputs and that give the same outputs for the same inputs, either both machines have the property P or both do not have it

Theorem

Every problem of the form Input: A Turing machine M. Question: Does machine M have property P? where P is a nontrivial input-output property, is undecidable.

• Z. Sawa (TU Ostrava)

Theoretical Computer Science December 9, 2020 63 / 63