A Concurrent Specification for POSIX File Systems Gian Ntzik, Pedro - - PowerPoint PPT Presentation

▶
a concurrent specification for posix file systems
SMART_READER_LITE
LIVE PREVIEW

A Concurrent Specification for POSIX File Systems Gian Ntzik, Pedro - - PowerPoint PPT Presentation

Apr 04, 2024 270 likes •564 views

A Concurrent Specification for POSIX File Systems Gian Ntzik, Pedro da Rocha Pinto and Philippa Gardner Imperial College London gian.ntzik08@imperial.ac.uk pedro.da-rocha-pinto09@imperial.ac.uk p.gardner@imperial.ac.uk Dagstuhl 2016 1/22

slide-1
SLIDE 1

A Concurrent Specification for POSIX File Systems

Gian Ntzik, Pedro da Rocha Pinto and Philippa Gardner

Imperial College London gian.ntzik08@imperial.ac.uk pedro.da-rocha-pinto09@imperial.ac.uk p.gardner@imperial.ac.uk

Dagstuhl 2016

1/22

slide-2
SLIDE 2

2/22

POSIX: Portable Operating System Interface

◮ An operating system standard ◮ A large part is devoted to the file system

>20 base definitions, >50 system interfaces

◮ It is written in English

slide-3
SLIDE 3

3/22

Formal Sequential Specifications

◮ Based on first-order logic

◮ Z Notation: specification language with refinement to

implementation

◮ Forest: Haskell DSL for type-safe file-system interactions

These approaches lack modularity for client reasoning

◮ Based on separation logics

◮ SSL [ESOP’14], Fusion Logic [OOPSLA’15]

These approaches give modular client reasoning. The second has been used to find bugs in implementations of e.g. rm − r.

slide-4
SLIDE 4

4/22

Concurrency in the POSIX File System

◮ Understanding the intended POSIX concurrent behaviour is

difficult

◮ It contains ambiguities

Example: I/O atomicity

I/O is intended to be atomic to ordinary files and pipes and FIFOs. This volume of POSIX.1-2008 does not specify behavior of concurrent writes to a file from multiple

  • processes. Applications should use some form of

concurrency control.

slide-5
SLIDE 5

5/22

Concurrency in the POSIX File System

◮ Understanding the intended POSIX concurrent behaviour is

difficult

◮ It contains ambiguities ◮ Information is fragmented throughout the whole standard

Example: unlink(p) atomically removes the file at path p. But resolving the path is not atomic.

Any part of the path of a file could be changed in parallel to a call to unlink(), resulting in unspecified behavior.

slide-6
SLIDE 6

6/22

Example: unlink(/usr/bin/git)

The operation takes a sequence of atomic actions

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

0101111011... 8 0110011011... 9

git

File-system notation: FS[2 → S] and FS(2) = S where S = {(share, 3), (bin, 4), (local, 5)}

slide-7
SLIDE 7

7/22

Sequence of Atomic Actions

FS1 FS1 FS2 FS2 FS3 FS4 Environment: mupltiple atomic updates Thread: single atomic read in path traversal Thread: single atomic update

usr bin git

unlink(/usr/bin/git)

slide-8
SLIDE 8

8/22

Atomic Specification: atomic Hoare triple

Atomic Hoare triples introduced in TaDA, encoded in Iris:

P Q

ℂ

A x ∈ X. P(x) C Q(x)

slide-9
SLIDE 9

9/22

Atomic Specification: refinement

P Q

ℂ

C ⊑ A x ∈ X.

  • P(x) , Q(x)
slide-10
SLIDE 10

10/22

Sequence of Atomic Actions: refinement

P1 P2 P3 P4 Pn−1 Pn

...

ℂ

C ⊑ A x1 ∈ X1.

  • P1(x1) , P2(x1)
  • ;

A x2 ∈ X2.

  • P3(x2) , P4(x2)
  • ; . . . ;

A xn−1 ∈ Xn−1.

  • Pn−1(xn−1) , Pn(xn−1)
slide-11
SLIDE 11

11/22

Concurrent unlink Specification

unlink(p/a, ret) ⊑ let r = resolve(ι0, p) in if ¬iserr(r) then ret = remlink(r, a) else ret = r fi

slide-12
SLIDE 12

11/22

Concurrent unlink Specification

unlink(p/a, ret) ⊑ ∃r. resolve(ι0, p, r); if ¬iserr(r) then ret = remlink(r, a) else ret = r fi

slide-13
SLIDE 13

11/22

Concurrent unlink Specification

unlink(p/a, ret) ⊑ ∃r. resolve(ι0, p, r); [¬iserr(r)] ; remlink(r, a, ret) ⊔ [iserr(r)] ; [ret = r]

slide-14
SLIDE 14

11/22

Concurrent unlink Specification

unlink(p/a, ret) ⊑ ∃r. resolve(ι0, p, r); {true, ¬iserr(r)} ; remlink(r, a, ret) ⊔ {true, iserr(r)} ; {true, ret = r}

slide-15
SLIDE 15

11/22

Concurrent unlink Specification

unlink(p/a, ret) ⊑ ∃r. resolve(ι0, p, r); {true, ¬iserr(r)} ; remlink(r, a, ret) ⊔ {true, iserr(r)} ; {true, ret = r} remlink(r, a, ret) . A FS ∈ FS. fs (FS) ∧ r ∈ dom(FS) , ∃S. fs (FS[r → S]) ∧ FS(r) = S ∪ {(a, −)} = ⇒ ret = 0

  • ⊓

A FS ∈ FS. fs (FS) ∧ r ∈ dom(FS) , fs (FS) ∧ {(a, −)} ∈ FS(r) = ⇒ ret = −1 ∗ errno = ENOENT

slide-16
SLIDE 16

12/22

Concurrent unlink Specification

letrec resolve(ι, p, ret) ∃b, a′, p′. if p = a′/p′ then ∃ι′. A FS ∈ FS. fs (FS) ∧ ι ∈ dom(FS) , fs (FS) ∧ ((a′, ι′) ∈ FS(ι) ∧ b = 0) ∨ ((a′, −) ∈ FS(ι) ∧ b = −1)

  • ;

if b = 0 then resolve(ι′, p′, ret) else . . . fi else . . .

slide-17
SLIDE 17

13/22

Concurrent rename Specification

rename(p/a, p′/b, ret) ⊑ ∃r1, r2. resolve(ιroot, p, r1) resolve(ιroot, p′, r2); if ¬iserr(r1) ∧ ¬iserr(r2) then // success cases: rename file(r1, r2, a, b) ⊓ rename dir(r1, r2, a, b) // error cases: ⊓ err file and dir(r1, r2, a, b) ⊓ err target notempty dir(r1, r2, b) ⊓ err source ancestor(r1, r2) ⊓ err dir and file(r1, r2, a, b) else if iserr(r1) ∧ ¬iserr(r2) then // error in resolve of source [ret = −1 ∗ errno = r1] else if ¬iserr(r1) ∧ iserr(r2) then // error in resolve of target [ret = −1 ∗ errno = r2] else if iserr(r1) ∧ iserr(r2) then // error in both resolves; either error code is set [ret = −1 ∗ errno ∈ {r1, r2}] fi

slide-18
SLIDE 18

14/22

Concurrent rename Specification

// rename file, if target exists it must be a file and it is replaced let rename file(r1, r2, a, b) ∃ιa, ιb. A FS ∈ FS. fs(FS) ∧ r1, r2 ∈ dom(FS) , fs(FS[r1 → S][r2 → (FS(r2) \ {(b, −)}) ∪ {(b, ιa)}]) ∧ FS(r1) = S ∪ {(a, ιa} ∧ isfile(FS, ιa) ∧ (∃ιb. (b, ιb) ∈ FS(r2) = ⇒ isfile(FS, ιb)) = ⇒ ret = 0

slide-19
SLIDE 19

14/22

Concurrent rename Specification

// rename file, if target exists it must be a file and it is replaced let rename file(r1, r2, a, b) ∃ιa, ιb. A FS ∈ FS. fs(FS) ∧ r1, r2 ∈ dom(FS) , fs(FS[r1 → S][r2 → (FS(r2) \ {(b, −)}) ∪ {(b, ιa)}]) ∧ FS(r1) = S ∪ {(a, ιa} ∧ isfile(FS, ιa) ∧ (∃ιb. (b, ιb) ∈ FS(r2) = ⇒ isfile(FS, ιb)) = ⇒ ret = 0

  • // rename directory, if target exists it must be an empty directory and it is replaced

let rename dir(r1, r2, a, b) ∃S, ιa, ιb. A FS ∈ FS. fs(FS) ∧ r1, r2 ∈ dom(FS) , fs(FS[r1 → S][r2 → (FS(r2) \ {(b, −)}) ∪ {(b, ιa)}]) ∧ FS(r1) = S ∪ {a, ιa} ∧ isdir(FS, ιa) ∧ r2 ∈ descendants(FS, r1) ∧ (∃ιb. (b, ιb) ∈ FS(r2) = ⇒ isempdir(FS, ιb)) = ⇒ ret = 0

slide-20
SLIDE 20

14/22

Concurrent rename Specification

// rename file, if target exists it must be a file and it is replaced let rename file(r1, r2, a, b) ∃ιa, ιb. A FS ∈ FS. fs(FS) ∧ r1, r2 ∈ dom(FS) , fs(FS[r1 → S][r2 → (FS(r2) \ {(b, −)}) ∪ {(b, ιa)}]) ∧ FS(r1) = S ∪ {(a, ιa} ∧ isfile(FS, ιa) ∧ (∃ιb. (b, ιb) ∈ FS(r2) = ⇒ isfile(FS, ιb)) = ⇒ ret = 0

  • // rename directory, if target exists it must be an empty directory and it is replaced

let rename dir(r1, r2, a, b) ∃S, ιa, ιb. A FS ∈ FS. fs(FS) ∧ r1, r2 ∈ dom(FS) , fs(FS[r1 → S][r2 → (FS(r2) \ {(b, −)}) ∪ {(b, ιa)}]) ∧ FS(r1) = S ∪ {a, ιa} ∧ isdir(FS, ιa) ∧ r2 ∈ descendants(FS, r1) ∧ (∃ιb. (b, ιb) ∈ FS(r2) = ⇒ isempdir(FS, ιb)) = ⇒ ret = 0

  • // error:

source is file, target is an existing directory let err file and dir(r1, r2, a, b) ∃ιa, ιb. A FS ∈ FS. fs(FS) ∧ r1, r2 ∈ dom(FS) , fs(FS) ∧ (a, ιa) ∈ FS(r1) ∧ isfile(FS, ιa) ∧ (b, ιb) ∈ FS(r2) ∧ isdir(FS, ιb) = ⇒ ret = −1 ∗ errno = EISDIR

slide-21
SLIDE 21

15/22

Client Reasoning

◮ Lock Files ◮ Named Pipes ◮ Advisory record locks (on-going)

slide-22
SLIDE 22

16/22

Example: lock-file /tmp/ .X0-lock is locked

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git

slide-23
SLIDE 23

17/22

Lock Files

◮ lock(path): atomically create a non-existing lock file at path ◮ unlock(path): remove the lock file identified by path ◮ Implemented similarly to spin locks

◮ open(path, O CREAT|O EXCL) to try to lock ◮ unlink to unlock

slide-24
SLIDE 24

18/22

Lock-file Implementation

letrec lock(path) let fd = open(path, O CREAT|O EXCL); if fd = −1 then lock(path) else close(fd) fi let unlock(path) unlink(path)

slide-25
SLIDE 25

19/22

Lock-file Specification: a first try

◮ We want clients to observe lock and unlock operations as single atomic

actions lock(path) ⊑ A v ∈ {0, 1}.

  • Lock(path, v) , Lock(path, 1) ∗ v = 0
  • unlock(path) ⊑
  • Lock(path, 1) , Lock(path, 0)
  • where Lock(path, v) is an abstract predicate.

◮ This specification cannot be guaranteed only by the module ◮ The file system is globally shared ◮ The module cannot enforce ownership of the path ◮ The module and the environment must agree a priori on a protocol for the use

  • f path
slide-26
SLIDE 26

20/22

Lock-file Protocol Agreement

We want the module and the environment to agree on a boundary

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git

LF

slide-27
SLIDE 27

21/22

Lock-file Specification: a right way

LF(/p/a) ⊢ lock(/p/a) ⊑ A v ∈ {0, 1}.

  • Lock(/p/a, v) , Lock(/p/a, 1) ∗ v = 0
  • LF(/p/a) ⊢ unlock(/p/a) ⊑
  • Lock(/p/a, 1) , Lock(/p/a, 0)
  • ◮ LF(/p/a) denotes what the implementation and the environment agree on:

◮ The path /p is not changed ◮ Only the module creates /p/a (locks the lock) ◮ Only the module removes /p/a (unlocks the lock)

slide-28
SLIDE 28

22/22

Conclusions and Future

◮ Refinement-style specifications for multiple atomic actions ◮ Combination of the refinement calculus and TaDA ◮ Concurrent specification of the POSIX file system (fragment) ◮ Client reasoning: lock files, named pipes, advisory record locks ◮ How do we justify the concurrent specification? Refine to a

correct and tested reference implementation

◮ How do we generate tests from our concurrent specification?