The mystery of the computer bits and data 10101011110101 Mikko - - PowerPoint PPT Presentation

01110111010110 11110101010101 00101011010011 01010111010101 01001010101010 10101010101010 10101011110101 01010101011101 01010111010110 10101101010110 10101110101010 11101010101101 01110111010110 10111011010101 11110101010101 00010101010101 01011010101110 10101010100101

The mystery of the computer –– bits and data

Mikko Kivelä

Department of Computer Science Aalto University

CS-A1120 Programming 2 2 March 2020

Lecture notes based on material created by Petteri Kaski

Billion computations per second

def test(m: Long) = { var i = 1L var s = 0L while(i <= m) { // s = 1 + 2 + ... + m s = s + i i = i + 1 } s } val NANOS_PER_SEC = 1e9 val test_start_time = System.nanoTime test(4000000000L) val test_end_time = System.nanoTime val test_duration = test_end_time - test_start_time println("test took %.2f seconds".format(test_duration/NANOS_PER_SEC))

1029: c4 e2 7d 19 02 vbroadcastsd (%rdx),%ymm0 102e: c4 e2 7d 19 0c 0a vbroadcastsd (%rdx,%rcx,1),%ymm1 1034: c4 e2 7d 19 14 4a vbroadcastsd (%rdx,%rcx,2),%ymm2 103a: 48 83 c2 08 add $0x8,%rdx 103e: c5 fd 28 18 vmovapd (%rax),%ymm3 1042: c4 e2 fd b8 e3 vfmadd231pd %ymm3,%ymm0,%ymm4 1047: c4 e2 f5 b8 eb vfmadd231pd %ymm3,%ymm1,%ymm5 104c: c4 e2 ed b8 f3 vfmadd231pd %ymm3,%ymm2,%ymm6 1051: c5 fd 28 58 20 vmovapd 0x20(%rax),%ymm3 1056: c4 e2 fd b8 fb vfmadd231pd %ymm3,%ymm0,%ymm7 105b: c4 62 f5 b8 c3 vfmadd231pd %ymm3,%ymm1,%ymm8 1060: c4 62 ed b8 cb vfmadd231pd %ymm3,%ymm2,%ymm9 1065: c5 fd 28 58 40 vmovapd 0x40(%rax),%ymm3 106a: c4 62 fd b8 d3 vfmadd231pd %ymm3,%ymm0,%ymm10 106f: c4 62 f5 b8 db vfmadd231pd %ymm3,%ymm1,%ymm11 1074: c4 62 ed b8 e3 vfmadd231pd %ymm3,%ymm2,%ymm12 1079: c5 fd 28 58 60 vmovapd 0x60(%rax),%ymm3 107e: c4 62 fd b8 eb vfmadd231pd %ymm3,%ymm0,%ymm13 1083: c4 62 f5 b8 f3 vfmadd231pd %ymm3,%ymm1,%ymm14 1088: c4 62 ed b8 fb vfmadd231pd %ymm3,%ymm2,%ymm15 108d: 48 01 c8 add %rcx,%rax 1090: 48 ff cb dec %rbx 1093: 75 94 jne 1029

Example: The innermost loop of a matrix multiplication subroutine implemented with Intel X86-64 machine code with AVX2 and FMA extensions supported by the Skylake architecture

Intel Skylake – machine code example (***)

https://github.com/pkaski/cluster-play/blob/master/haswell-mm-test/libmynative.c

1029: c4 e2 7d 19 02 vbroadcastsd (%rdx),%ymm0 102e: c4 e2 7d 19 0c 0a vbroadcastsd (%rdx,%rcx,1),%ymm1 1034: c4 e2 7d 19 14 4a vbroadcastsd (%rdx,%rcx,2),%ymm2 103a: 48 83 c2 08 add $0x8,%rdx 103e: c5 fd 28 18 vmovapd (%rax),%ymm3 1042: c4 e2 fd b8 e3 vfmadd231pd %ymm3,%ymm0,%ymm4 1047: c4 e2 f5 b8 eb vfmadd231pd %ymm3,%ymm1,%ymm5 104c: c4 e2 ed b8 f3 vfmadd231pd %ymm3,%ymm2,%ymm6 1051: c5 fd 28 58 20 vmovapd 0x20(%rax),%ymm3 1056: c4 e2 fd b8 fb vfmadd231pd %ymm3,%ymm0,%ymm7 105b: c4 62 f5 b8 c3 vfmadd231pd %ymm3,%ymm1,%ymm8 1060: c4 62 ed b8 cb vfmadd231pd %ymm3,%ymm2,%ymm9 1065: c5 fd 28 58 40 vmovapd 0x40(%rax),%ymm3 106a: c4 62 fd b8 d3 vfmadd231pd %ymm3,%ymm0,%ymm10 106f: c4 62 f5 b8 db vfmadd231pd %ymm3,%ymm1,%ymm11 1074: c4 62 ed b8 e3 vfmadd231pd %ymm3,%ymm2,%ymm12 1079: c5 fd 28 58 60 vmovapd 0x60(%rax),%ymm3 107e: c4 62 fd b8 eb vfmadd231pd %ymm3,%ymm0,%ymm13 1083: c4 62 f5 b8 f3 vfmadd231pd %ymm3,%ymm1,%ymm14 1088: c4 62 ed b8 fb vfmadd231pd %ymm3,%ymm2,%ymm15 108d: 48 01 c8 add %rcx,%rax 1090: 48 ff cb dec %rbx 1093: 75 94 jne 1029

https://github.com/pkaski/cluster-play/blob/master/haswell-mm-test/libmynative.c

?

Intel Skylake – machine code example (***)

Example: The innermost loop of a matrix multiplication subroutine implemented with Intel X86-64 machine code with AVX2 and FMA extensions supported by the Skylake architecture

NVIDIA Volta – machine code example (***)

https://github.com/pkaski/motif-localized

LOP3.LUT R8, R6, R8, R19, 0x96, !PT; /* 0x0000000806087212 */ /* 0x000fe400078e9613 */ LOP3.LUT R64, R11, R64, RZ, 0x3c, !PT; /* 0x000000400b407212 */ /* 0x000fc400078e3cff */ LOP3.LUT R62, R62, R5, R4.reuse, 0x96, !PT; /* 0x000000053e3e7212 */ /* 0x100fe400078e9604 */ LOP3.LUT R17, R17, R15.reuse, R7.reuse, 0x78, !PT; /* 0x0000000f11117212 */ /* 0x180fe400078e7807 */ LOP3.LUT R8, R8, R15, R7, 0x78, !PT; /* 0x0000000f08087212 */ /* 0x000fe400078e7807 */ LOP3.LUT R18, R19.reuse, R18, R4.reuse, 0x96, !PT; /* 0x0000001213127212 */ /* 0x140fe400078e9604 */ LOP3.LUT R5, R19, R10, R4, 0x96, !PT; /* 0x0000000a13057212 */ /* 0x000fe400078e9604 */ LOP3.LUT R9, R6, R9, R19, 0x96, !PT; /* 0x0000000906097212 */ /* 0x000fc400078e9613 */ LOP3.LUT R7, R64, R15, R7, 0x78, !PT; /* 0x0000000f40077212 */ /* 0x000fe400078e7807 */ LOP3.LUT R61, R61, R12, R19, 0x96, !PT; /* 0x0000000c3d3d7212 */ /* 0x000fe400078e9613 */ LOP3.LUT R59, R17, R59, RZ, 0x3c, !PT; /* 0x0000003b113b7212 */ /* 0x000fe400078e3cff */ LOP3.LUT R60, R60, R5, R6, 0x96, !PT; /* 0x000000053c3c7212 */ /* 0x000fe400078e9606 */ LOP3.LUT R58, R9, R58, RZ, 0x3c, !PT; /* 0x0000003a093a7212 */ /* 0x000fe400078e3cff */ LOP3.LUT R51, R18, R51, RZ, 0x3c, !PT; /* 0x0000003312337212 */ /* 0x000fc400078e3cff */ LOP3.LUT R50, R8, R50, RZ, 0x3c, !PT; /* 0x0000003208327212 */ /* 0x000fe400078e3cff */ LOP3.LUT R57, R7, R57, RZ, 0x3c, !PT; /* 0x0000003907397212 */ /* 0x000fe200078e3cff */ @P0 BRA 0x8d0; /* 0xfffff96000000947 */ /* 0x000fee000383ffff */

Example: Part an inner loop of an algorithm (vertex-localized graph motif search) with NVIDIA GV100
 GPU machine code (Compute Capability 7.0)

Example: Part an inner loop of an algorithm (vertex-localized graph motif search) with NVIDIA GV100
 GPU machine code (Compute Capability 7.0)

https://github.com/pkaski/motif-localized

LOP3.LUT R8, R6, R8, R19, 0x96, !PT; /* 0x0000000806087212 */ /* 0x000fe400078e9613 */ LOP3.LUT R64, R11, R64, RZ, 0x3c, !PT; /* 0x000000400b407212 */ /* 0x000fc400078e3cff */ LOP3.LUT R62, R62, R5, R4.reuse, 0x96, !PT; /* 0x000000053e3e7212 */ /* 0x100fe400078e9604 */ LOP3.LUT R17, R17, R15.reuse, R7.reuse, 0x78, !PT; /* 0x0000000f11117212 */ /* 0x180fe400078e7807 */ LOP3.LUT R8, R8, R15, R7, 0x78, !PT; /* 0x0000000f08087212 */ /* 0x000fe400078e7807 */ LOP3.LUT R18, R19.reuse, R18, R4.reuse, 0x96, !PT; /* 0x0000001213127212 */ /* 0x140fe400078e9604 */ LOP3.LUT R5, R19, R10, R4, 0x96, !PT; /* 0x0000000a13057212 */ /* 0x000fe400078e9604 */ LOP3.LUT R9, R6, R9, R19, 0x96, !PT; /* 0x0000000906097212 */ /* 0x000fc400078e9613 */ LOP3.LUT R7, R64, R15, R7, 0x78, !PT; /* 0x0000000f40077212 */ /* 0x000fe400078e7807 */ LOP3.LUT R61, R61, R12, R19, 0x96, !PT; /* 0x0000000c3d3d7212 */ /* 0x000fe400078e9613 */ LOP3.LUT R59, R17, R59, RZ, 0x3c, !PT; /* 0x0000003b113b7212 */ /* 0x000fe400078e3cff */ LOP3.LUT R60, R60, R5, R6, 0x96, !PT; /* 0x000000053c3c7212 */ /* 0x000fe400078e9606 */ LOP3.LUT R58, R9, R58, RZ, 0x3c, !PT; /* 0x0000003a093a7212 */ /* 0x000fe400078e3cff */ LOP3.LUT R51, R18, R51, RZ, 0x3c, !PT; /* 0x0000003312337212 */ /* 0x000fc400078e3cff */ LOP3.LUT R50, R8, R50, RZ, 0x3c, !PT; /* 0x0000003208327212 */ /* 0x000fe400078e3cff */ LOP3.LUT R57, R7, R57, RZ, 0x3c, !PT; /* 0x0000003907397212 */ /* 0x000fe200078e3cff */ @P0 BRA 0x8d0; /* 0xfffff96000000947 */ /* 0x000fee000383ffff */ ?

NVIDIA Volta – machine code example (***)

What are the principles

• f how computers work?

What is computing?

The mystery of the computer

Why is it important that a programmer understand the central principles of computers?

• Computer is a machine 


–– understanding the basic principles of how this machine works is a fundamental part of programmers professional competence

• Skills for applications where the computer needs

to be used at the limits of its performance

• Physical device (“hardware”) and programs

(“software”) are interacting all the way from design to execution

• Curiosity and the joy of finding out how things

work

(NVIDIA DGX-1, 8 x Tesla V100 GPU, 40960 cores, 3.2 kW, 170 teraflops)

dgx01.triton.aalto.fi (***)

Finland: Mahti & Puhti (***)

https://research.csc.fi/techspecs/

New Finnish supercomputer @ CSC Kajaani (Atos BullSequana) Puhti: 320 Nvidia V100 Volta GPUs (2.7 petaflops) ~27 000 Intel Xeon cores (2.5 petaflops) Mahti: ~180 000 AMD EPYC cores (7.5 petaflops)

Summit: #1 top500.org (***)

https://www.olcf.ornl.gov/olcf-resources/compute-systems/summit/

~4600 computational nodes, 
 every node has six
 NVIDIA Volta V100s (~4600 x 6 x 5120 32 bit cores 
 = ~140 million cores 
 1312 MHz clock rate, 15 MW) ~200 petaflops

Physical limits of computers (***)

Principles of operation

a) at the logical level b) at the physical level

The mystery of the computer (4 rounds)

• 2. Bits and data


• 3. Combinational logic


• 4. Sequential logic


• 5. Programmable computer

–– binary numbers,
 manipulating bits with Scala –– circuits built of logic gates simulated with Scala –– assembly language programming 
 Scala simulated “armlet” processor –– clock and feedback to the Scala simulation, processor data path

Round 2: Bits and data

• Data is sequence of bits
• Orders of magnitude
• Data has a format


(Formats of Scala data types)

• Binary numeral system


(Base 2 numeral system)

• Manipulating bits with Scala
• Floating point numbers(IEEE 754–2008) (*)

Bits

• Bit (short for “binary digit”)
• Basic unit for representing information

1

Claude E. Shannon [A mathematical theory of communication. Bell System

• Tech. J. 27 (1948) 379–423, 623–656]


reports that J. W. Tukey introduced “bit” to represent a number 0 or 1 in a binary numeral system.

Sequence of bits (“data”)

• All information in computing is represented

as sequence of bits (“data”)

• The amount of data is measured in bits


(= how many bits in the sequence)

1 0 1 1 1 1 1 1

Orders of magnitude

• The amount of data is typically measured in

bits or bytes [1 byte = 8 bits]


• It is customary to use the prefixes in the

International System of Units (kilo, mega, giga, etc.) to denote the orders of magnitude

• Magnitudes are typically given either as base 10

magnitudes or base 2 magnitudes

Base 10 prefixes

[See the ISO 80000-1 standard]

Base 2 prefixes

(“kilobinary”, “megabinary”, …)

[See the ISO 80000-1 standard]

Example: megabit and terabit

One megabit (106 bits) can be stored on one square metre One terabit (1012 bits) can be stored on one square kilometre Assume, that one bit can be stored on a surface area of one square millimetre

Otaniemi (Google Earth)

Terabinarybyte (TiB) –– performance example

Single core All 40 cores Read (consecutive 64-bit words) Read (random 512-bit cache lines) Read (random 64-bit words)

9.6 GiB/s 36.4 GiB/s 1.5 GiB/s 0.5 GiB/s 5.2 GiB/s 20.1 GiB/s dgx01.triton.aalto.fi Aalto University, “triton” computing cluster

2 x 2.2-GHz Intel Xeon E5-2698v4
 (Broadwell, 20 cores, 50 MiB last-level cache)

512 GiB main memory 
 (16 x 32 GiB DDR4-2133 DIMM)

NVIDIA DGX-1

CPU & main memory

On-device global memory bandwidth Host/device transfer bandwidth

720 GiB/s 12 GiB/s

8 x NVIDIA Tesla V100 
 SXM2 Accelerator devices
 (NVIDIA GV100 GPU @ 1312 MHz, 


Volta microarchitecture, 
 5120 32-bit cores, 80 SMs, 64 cores/SM, 16 GiB of on-device 4096-bit HBM2 memory)

GPU on-device memory

Single device All 8 devices

5760 GiB/s dgx01.triton.aalto.fi

NVIDIA DGX-1

Terabinarybyte (TiB) –– performance example

Aalto University, “triton” computing cluster

Petabyte

https://databricks.com/blog/2014/10/10/spark-petabyte-sort.html

$144.22 / 100 TB

https://databricks.com/blog/2016/11/14/setting-new-world-record-apache-spark.html

http://sortbenchmark.org

What does data represent ?

• Sequence of bits is only a boring

sequence of zeros and ones
 


• What makes the sequences of bits

interesting are conventions and standards on what these sequences represent

1 0 1 1 1 1 1 1

27550 = 145 x 190 bits

http://search.dilbert.com/search?w=planet+zorp&x=0&y=0

64 bits

0100000000001001001000011111101101010100010001000010110100011000

(3.141592653589793115997963468544185161590576171875)

http://dx.doi.org/10.1109%2FIEEESTD.2008.4610935

IEEE Std 754–2008 scala.math.Pi

256 bits

0010000101000111011100100110010101100101011101000110100101101110 0110011101110011001000000110011001110010011011110110110100100000 0111010001101000011001010010000001110000011011000110000101101110 0110010101110100001000000101101001101111011100100111000000100001

!Greetings from the planet Zorp!

http://unicode.org Unicode (UTF-8)

625 bits

http://www.qrcode.com/en/about/standards.html

Data and data format

• In addition to data (=sequence of bits)

we need a convention of what the data represents


• These conventions or standards of

what data represents are called formats


• r alternatively data types

Numerical and textual data

• Most programs involve at least one of

these types:

• numerical data


(= data representing numbers)

• textual data


(= data representing text)

• The basic data types in Scala can be divided

to numerical and textual types:

• numerical


Byte, Short, Int, Long, Float, Double

• textual


Char, String

Numerical and textual data

Word, word length

• Computers have been built to process data in units,

which have a fixed number of bits (for example 8, 16, 32, 64, or 128 bits)

• Units like this are called words
• In a processor architecture the word length is the

largest number of bits in a word for which the processor can do computations with a single instruction

• For example, “64 bit processor” typically means that

the word length in the processor architecture is 64 bits

Lengths of basic value types in Scala

• Scala is a hardware-independent language


(it is designed to be used with many types of processors)

• The basic value types in Scala have the following fixed lengths in

bits:

• Byte (8 bits)
• Short (16 bits)
• Int (32 bits)
• Long (64 bits)
• Char (16 bits)
• If the hardware does not support one of these lengths the

support for them is implemented with software in a way that is transparent to the (Scala) programmer

Witnessing the bits, in Scala

[ demo ]

How to represent integers?

• The integer 754 can be represented in various ways:
• Roman numerals: “DCCLIV”
• Decimal numeral system: “754”
• Binary numbers: “1011110010”

• …in the Zorp-planet the following representation is

used:

■■■■■■■■■■

[more value types in the online material,
 including rational and floating point numbers]

Binary numeral system

• Computing is easy in the binary numeral system (= base 2

numeral system)

• compare it to
• multiplying two Roman numerals, or
• memorising the multiplication and summation tables in

base10 numeral system


• Computers use binary numeral system

• Binary numbers (and base 16 hexadecimal numbers) are part of

general knowledge for a programmer

Hexadecimal digits

One hexadecimal digit represents 4 bits: Example (25257):

Bit positions in a word

bit positions

least significant most significant

1029: c4 e2 7d 19 02 vbroadcastsd (%rdx),%ymm0 102e: c4 e2 7d 19 0c 0a vbroadcastsd (%rdx,%rcx,1),%ymm1 1034: c4 e2 7d 19 14 4a vbroadcastsd (%rdx,%rcx,2),%ymm2 103a: 48 83 c2 08 add $0x8,%rdx 103e: c5 fd 28 18 vmovapd (%rax),%ymm3 1042: c4 e2 fd b8 e3 vfmadd231pd %ymm3,%ymm0,%ymm4 1047: c4 e2 f5 b8 eb vfmadd231pd %ymm3,%ymm1,%ymm5 104c: c4 e2 ed b8 f3 vfmadd231pd %ymm3,%ymm2,%ymm6 1051: c5 fd 28 58 20 vmovapd 0x20(%rax),%ymm3 1056: c4 e2 fd b8 fb vfmadd231pd %ymm3,%ymm0,%ymm7 105b: c4 62 f5 b8 c3 vfmadd231pd %ymm3,%ymm1,%ymm8 1060: c4 62 ed b8 cb vfmadd231pd %ymm3,%ymm2,%ymm9 1065: c5 fd 28 58 40 vmovapd 0x40(%rax),%ymm3 106a: c4 62 fd b8 d3 vfmadd231pd %ymm3,%ymm0,%ymm10 106f: c4 62 f5 b8 db vfmadd231pd %ymm3,%ymm1,%ymm11 1074: c4 62 ed b8 e3 vfmadd231pd %ymm3,%ymm2,%ymm12 1079: c5 fd 28 58 60 vmovapd 0x60(%rax),%ymm3 107e: c4 62 fd b8 eb vfmadd231pd %ymm3,%ymm0,%ymm13 1083: c4 62 f5 b8 f3 vfmadd231pd %ymm3,%ymm1,%ymm14 1088: c4 62 ed b8 fb vfmadd231pd %ymm3,%ymm2,%ymm15 108d: 48 01 c8 add %rcx,%rax 1090: 48 ff cb dec %rbx 1093: 75 94 jne 1029

https://github.com/pkaski/cluster-play/blob/master/haswell-mm-test/libmynative.c

?

Intel Skylake – machine code example (***)

Example: The innermost loop of a matrix multiplication subroutine implemented with Intel X86-64 machine code with AVX2 and FMA extensions supported by the Skylake architecture

From decimal to binary

20 = 1 21 = 2 22 = 4 23 = 8 24 = 16 25 = 32 26 = 64 27 = 128 28 = 256 29 = 512 210 = 1024 . . .

• Input: base 10 number
• Initialise all bits in the result off (to value

0)

• Repeat until
• Subtract from the largest possible

number of form , and

• Set bit on (to value 1)

∈ Z≥0 = 0

• 2j

− 2j ≥ 0 j

From binary to decimal

• Input: base 2 number
• Initialise the result to value 0
• Repeat for every bit position
• If bit is on (has value 1),


add to the result

∈ Z≥0 j 2j j

20 = 1 21 = 2 22 = 4 23 = 8 24 = 16 25 = 32 26 = 64 27 = 128 28 = 256 29 = 512 210 = 1024 . . .

In the exercises

• Conversions between bases 2 and 10
• Operating on bits in a word
• value of a bit?
• set bit on (to value 1)
• set bit off (to value 0, “reset”, “clear”, “mask”)
• toggle a bit (to the opposite value 0 to1, 1 to 0)
• shifting a word left (“left shift”)
• shifting a word right (“right shift”)
• right arithmetic shift of a word
• Basics of floating point numbers

Boolean operations on bits

Exclusive

• r

(minimum) (maximum) (toggle) (sum, parity)

Boolean operations on words

Bit operations (by convention) work independently on each bit position of a word

Forcing and toggling the value of a bit

Forcing and toggling the value of a word

Shifting bits in a word

Example: Hexadecimal input in Scala

scala> val y = 0x3E7 y: Int = 999

Example: Hexadecimal output in Scala

def toHexString(v: Int): String = "0x%X".format(v)

scala> toHexString(754) res: String = 0x2F2 scala> toHexString(999) res: String = 0x3E7 scala> toHexString(0x3E7) res: String = 0x3E7

Example: value at bit position

scala> val x = 0x3A // 111010 in binary x: Int = 58 scala> (x >> 0)&1 // value of bit at position 0 res: Int = 0 scala> (x >> 1)&1 // value of bit at position 1 res: Int = 1 scala> (x >> 2)&1 // value of bit at position 2 res: Int = 0 scala> (x >> 3)&1 // ... res: Int = 1 scala> (x >> 4)&1 res: Int = 1 scala> (x >> 5)&1 res: Int = 1 scala> (x >> 6)&1 res: Int = 0

Example: word with bit at j in value 1

1 << j 1L << j

(32 bit word; Int) (64 bit word; Long)

What about floating point numbers?

scala> 0.3 - 0.1 res: Double = 0.19999999999999998

Why is the result not 0.2 ?

Floating point standard

http://dx.doi.org/10.1109%2FIEEESTD.2008.4610935

IEEE Std 754–2008 Float = IEEE 754-2008 “binary32” Double = IEEE 754-2008 “binary64”

IEEE 754-2008 “binary64” floating point number nearest to 0.3

scala> tellAboutDouble(0.3) I am a 64-bit word that Scala views as having format 'Double' ... ... my bits are, in binary, (0011111111010011001100110011001100110011001100110011001100110011)_2 ... or equivalently, in hexadecimal, 0x3FD3333333333333 ... Scala prints me out as 0.3 ... indeed, my bits acquire meaning as specified in the binary interchange format 'binary64' in the IEEE Std 754-2008 IEEE Standard for Floating-Point Arithmetic http://dx.doi.org/10.1109%2FIEEESTD.2008.4610935 ... this format is a bit-packed format with three components: (0011111111010011001100110011001100110011001100110011001100110011)_2 a) the sign (bit 63): 0 b) the biased exponent (bits 62 to 52): 01111111101 c) the trailing significand (bits 51 to 0): 0011001100110011001100110011001100110011001100110011 ... my biased exponent 1021 indicates that I am a _normalized_ number with a leading 1-digit in my significand and an unbiased exponent -2 = 1021 - 1023 ... that is, in _binary radix point notation_, I am exactly (1.0011001100110011001100110011001100110011001100110011)_2 * 2^{-2} ... or what is the same in _decimal radix point notation_, I am exactly 0.299999999999999988897769753748434595763683319091796875

scala> tellAboutDouble(0.1) I am a 64-bit word that Scala views as having format 'Double' ... ... my bits are, in binary, (0011111110111001100110011001100110011001100110011001100110011010)_2 ... or equivalently, in hexadecimal, 0x3FB999999999999A ... Scala prints me out as 0.1 ... indeed, my bits acquire meaning as specified in the binary interchange format 'binary64' in the IEEE Std 754-2008 IEEE Standard for Floating-Point Arithmetic http://dx.doi.org/10.1109%2FIEEESTD.2008.4610935 ... this format is a bit-packed format with three components: (0011111110111001100110011001100110011001100110011001100110011010)_2 a) the sign (bit 63): 0 b) the biased exponent (bits 62 to 52): 01111111011 c) the trailing significand (bits 51 to 0): 1001100110011001100110011001100110011001100110011010 ... my biased exponent 1019 indicates that I am a _normalized_ number with a leading 1-digit in my significand and an unbiased exponent -4 = 1019 - 1023 ... that is, in _binary radix point notation_, I am exactly (1.1001100110011001100110011001100110011001100110011010)_2 * 2^{-4} ... or what is the same in _decimal radix point notation_, I am exactly 0.1000000000000000055511151231257827021181583404541015625

IEEE 754-2008 “binary64” floating point number nearest to 0.1

0.3 – 0.1 computed with “binary64” floating point numbers

The closest “binary64” floating point number

scala> tellAboutDouble(0.3-0.1) I am a 64-bit word that Scala views as having format 'Double' ... ... my bits are, in binary, (0011111111001001100110011001100110011001100110011001100110011001)_2 ... or equivalently, in hexadecimal, 0x3FC9999999999999 ... Scala prints me out as 0.19999999999999998 ... indeed, my bits acquire meaning as specified in the binary interchange format 'binary64' in the IEEE Std 754-2008 IEEE Standard for Floating-Point Arithmetic http://dx.doi.org/10.1109%2FIEEESTD.2008.4610935 ... this format is a bit-packed format with three components: (0011111111001001100110011001100110011001100110011001100110011001)_2 a) the sign (bit 63): 0 b) the biased exponent (bits 62 to 52): 01111111100 c) the trailing significand (bits 51 to 0): 1001100110011001100110011001100110011001100110011001 ... my biased exponent 1020 indicates that I am a _normalized_ number with a leading 1-digit in my significand and an unbiased exponent -3 = 1020 - 1023 ... that is, in _binary radix point notation_, I am exactly (1.1001100110011001100110011001100110011001100110011001)_2 * 2^{-3} ... or what is the same in _decimal radix point notation_, I am exactly 0.1999999999999999833466546306226518936455249786376953125

Floating point numbers

Float = IEEE 754-2008 “binary32” Double = IEEE 754-2008 “binary64”

http://dx.doi.org/10.1109%2FIEEESTD.2008.4610935

IEEE Std 754–2008

Floating point numbers are rational numbers, which have significand

• f fixed precision and a base 2 exponent in a fixed range in the

radix representation [more on this in the reading material] It is good to be aware of the properties of floating point numbers when computing professionally with them –– make yourself familiar with the standard and remember that you are making computations with a fixed set of 232 (264) rational numbers, which become more sparse the larger their absolute values are

Exercises

• bitsQuiz


–– convert between base 2 and base 10

• wordOps


–– bit operations for words (how many bits with value 1 in a word etc.)

• parity


–– computing and checking the parity bit

• float


–– getting familiar with floating point numbers and their limitations

• base64


–– encoding binary to text in base 64

• errorCorrect (challenge problem)


–– one bit error correction, for example with Hamming code

• rationalDecompose (challenge problem)


–– radix-point decomposition of a rational number