Part III Part III Storage Management Storage Management Chapter - - PowerPoint PPT Presentation

Part III Part III Storage Management

Chapter 10: File System Interface

Storage Management

Chapter 10: File-System Interface

1

Fall 2010

Files Files

A file is a named collection of related information that is recorded on secondary storage. The operating systems maps this logical storage g y g g unit to the physical view of information storage. A file may have the following characteristics File Attributes File Operations File Operations File Types File Structures File Structures Internal Files

2

File Attributes File Attributes

File Name: The symbolic name is perhaps the only h d bl fil ib human readable file attribute. Identifier: A unique number assigned to each file f id ifi i for identification purpose. File Type: Some systems recognize various file Wi d i d l

• types. Windows is a good example.

File Location: A pointer to a device to find a file. File Size: The current size of a file, or the maximum allowed size. File Protection: This is for access-control. File Date, Time, Owner, etc.

3

File O File Operations: erations: 1/2 1/2 p

A file can be considered as an abstract data type h h d d i i that has data and accompanying operations. Creating a file Writing a file Reading a file g Repositioning within a file Deleting a file Deleting a file Truncating a file Other operations (e g appending a file renaming a Other operations (e.g., appending a file, renaming a file)

4

File File Operations: Operations: 2/2 2/2 File File Operations: Operations: 2/2 2/2

disk system-wide

• pen-file table

file

process

• pen-file table

file index file pointer file open count disk location

• ne file

access right

5

File File Structure Structure File File Structure Structure

S t t ifi fil t th t Some systems support specific file types that have special file structures. For example, files that contain binary executables. An operating system becomes more complex when more file types (i.e., file structures) are yp ( ) supported. In general, the number of supported file types In general, the number of supported file types is kept to minimum.

6

File File Access Access Methods ethods File File Access Access Methods Methods

Access method: how a file be used Access method: how a file be used. There are three popular ones: Sequential access method for sequential files Direct access method for direct files Indexed access method for indexed files.

7

Sequential Sequential Access ccess Method Method Sequential Sequential Access Access Method Method

With the sequential access method, a file is With the sequential access method, a file is processed in order, one record after the other. If p is the file pointer, the next record to be p p , accessed is either p+1 (forward) or p-1 (i.e., backward).

end of file current record beginning end of file next record

8

rewind

read/write

Direct Direct Access Access Method Method Direct Direct Access Access Method Method

A file is made up of fixed length logical records A file is made up of fixed-length logical records. The direct access method uses a record number to id tif h d F l d i identify each record. For example, read rec 0, write rec 100, seek rec 75, etc. Some systems may use a key field to access a record (e.g., read rec “Age=24” or write rec “Name=Dow”). This is usually achieved with hashing. Since records can be accessed in random order, direct access is also referred to as random access. Direct access method can simulate sequential access.

9

Direct access method can simulate sequential access.

Indexed Indexed Access Access Method ethod Indexed Indexed Access Access Method Method

With the indexed access method a file is sorted in With the indexed access method, a file is sorted in ascending order based on a number of keys. E h di k bl k t i b f fi d Each disk block may contain a number of fixed- length logical records. An index table stores the keys of the first block in each block. We can search the index table to locate the block that contains the desired record. Then, search the block to find the desired record. This is exactly a one-level B-, B+ or B* tree.

10

This is exactly a one level B , B or B tree. Multi-level index access method is also possible.

data file

last name logical rec #

index table

Adams Arthur Ashcroft, … Asher, … Atkins g Ashcroft Smith, …. Sweeny, … Swell, … Smith

index tables are stored in physical memory

11

in physical memory when file is open

Directory Directory Structure: Structure: 1/2 1/2 Directory Directory Structure: Structure: 1/2 1/2

A large volume disk may be partitioned into A large volume disk may be partitioned into partitions, or mini disks, or volumes. Each partition contains information about files Each partition contains information about files within it. This information is stored in entries of a device directory or volume table of content (VTOC) device directory or volume table of content (VTOC). The device directory, or directory for short, stores the name location size type access method etc of the name, location, size, type, access method, etc of each file. Operations perform on directory: search for a file Operations perform on directory: search for a file, create a file, delete a file, rename a file, traverse the file system, etc.

12

y ,

Directory Directory Structure: Structure: 2/2 2/2 Directory Directory Structure: Structure: 2/2 2/2

Th fi l d di t There are five commonly used directory structures: Single-Level Directory Two-Level Directory Tree-Structure Directories Acyclic Graph Directories Acyclic-Graph Directories General Graph Directories

13

Single-Level Directory Single-Level Directory

All files are contained in the same directory. It is difficult to maintain file name uniqueness. CP/M-80 and early version of MS-DOS use y this directory structure.

14

T L l L l Di t 1/2 1/2 Tw o- w o-Leve evel Di Direc rectory:

• ry: 1/2

1/2

This is an extension of the single-level directory This is an extension of the single-level directory for multi-user system. Each user has his/her user file directory The Each user has his/her user file directory. The system’s master file directory is searched for the user directory when a user job starts. y j Early CP/M-80 multi-user systems use this structure.

15

T L l L l Di t 2/2 2/2 Tw o- w o-Leve evel Di Direc rectory:

• ry: 2/2

2/2

To locate a file path name is used For To locate a file, path name is used. For example, /user2/a is the file a of user 2. Different systems use different path names Different systems use different path names. For example, under MS-DOS it is C:\user2\a. \ \ The directory of a special user, say user 0, may contain all system files. y y

16

Tree Tree Structured Structured Directory Directory Tree Tree-Structured Structured Directory Directory

Each directory or subdirectory contains files Each directory or subdirectory contains files and subdirectories, and forms a tree. Directories are special files Directories are special files.

17

/bin/mail/prog/spell

A l A li G h G h Di t 1/2 1/2 Acyc cycli lic- c-Grap raph Di Direc rectory:

• ry: 1/2

1/2

Thi f di i This type of directories allows a file/directory to be shared by multiple be shared by multiple directories. This is different from two This is different from two copies of the same file or directory. directory. An acyclic-graph directory is more flexible directory is more flexible than a simple tree

• structure. However, it is

file count is shared by directories dict and spell

18

, more complex.

Ac Acyclic-Graph Director clic-Graph Directory: : 2/2 2/2 y y

Since a file have multiple absolute path names, how do we calculate file system statistics or do backup? we calculate file system statistics or do backup? Would the same file be counted multiple times? How do we delete a file? How do we delete a file? If sharing is implemented with symbolic links, we

• nly delete the link if we have a list of links to the
• y de ete t e

we ave a st o s to t e

• file. The file is removed when the list is empty.

Or, we remove the file and keep the links. When the O , we e

• ve

e e d eep e s W e e file is accessed again, a message is given and the link is removed. Or, we can maintain a reference count for each shared file. The file is removed when the count is

19

zero.

General General Graph raph Directory: Directory: 1/2 1/2 General General Graph Graph Directory: Directory: 1/2 1/2

It is easy to traverse the It is easy to traverse the directories of a tree or an acyclic directory system. acyclic directory system. However, if links are added arbitrarily the added arbitrarily, the directory graph becomes arbitrary and may arbitrary and may contain cycles. H d h f

a cycle

How do we search for a file?

20

General General Graph raph Directory: Directory: 2/2 2/2 General General Graph Graph Directory: Directory: 2/2 2/2

How do we delete a file? We can use reference count! How do we delete a file? We can use reference count! In a cycle, due to self-reference, the reference count may be non-zero even when it is no longer count may be non-zero even when it is no longer possible to refer to a file or directory. Thus garbage collection may needed A garbage Thus, garbage collection may needed. A garbage collector traverses the directory and marks files and directories that can be accessed. A second round removes those inaccessible items. To avoid this time-consuming task a system can To avoid this time-consuming task, a system can check if a cycle may occur when a link is made. How? You should know!

21

File Sharing File Sharing

When a file is shared by multiple users, how i i ? can we ensure its consistency? If multiple users are writing to the file, should all of the writers be allowed to write? Or, should the operating system protect the , p g y p user actions from each other? This is the file consistency semantics. This is the file consistency semantics.

22

File File Consistency Consistency Semantics Semantics File File Consistency Consistency Semantics Semantics

Consistency semantics is a characterization of Consistency semantics is a characterization of the system that specifies the semantics of multiple users accessing a shared file i l l simultaneously. Consistency semantics is an important criterion for evaluating any file system that supports file for evaluating any file system that supports file sharing. There are three commonly used semantics There are three commonly used semantics Unix semantics Session Semantics Session Semantics Immutable-Shared-Files Semantics A file session consists all file accesses between

23

A file session consists all file accesses between

• pen() and close().

Unix Unix Semantics Semantics Unix Unix Semantics Semantics

W it t fil b i ibl Writes to an open file by a user are visible immediately to other users who have the file t th ti

• pen at the same time.

All users share the file pointer. Thus, advancing the file pointer by one user affects all sharing users. A file has a single image that interleaves all accesses, regardless of their origin. g g File access contention may cause delays.

24

Session Semantics Session Semantics

Writes to an open file by a user are not visible immediately to other users who have the same file immediately to other users who have the same file

• pen simultaneously.

Once a file is closed the changes made to it are visible Once a file is closed, the changes made to it are visible

• nly in sessions started later.

Already open instances of the file are not affected by Already-open instances of the file are not affected by these changes. A file may be associated temporarily with several A file may be associated temporarily with several and possible different images at the same time. Multiple users are allowed to perform both read Multiple users are allowed to perform both read and write concurrently on their image of the file without delay

25

without delay. The Andrew File System (AFS) uses this semantics.

Immutable Immutable Shared Shared Files Files Semantics emantics Immutable Immutable-Shared Shared-Files Files Semantics Semantics

Once a file is declared as shared by its creator it Once a file is declared as shared by its creator, it cannot be modified. An immutable file has two important properties: An immutable file has two important properties: Its name may not be used Its content may not be altered Its content may not be altered Thus, the name of an immutable file indicates that the contents of the file is fixed a constant rather the contents of the file is fixed – a constant rather than a variable. The implementation of these semantics in a The implementation of these semantics in a distributed system is simple, since sharing is disciplined (i e read-only)

26

disciplined (i.e., read only).

File File Protection Protection File File Protection Protection

We can keep files safe from physical damage (i.e., reliability) and improper access (i.e., protection). Reliability is generally provided by backup. The need for file protection is a direct result of the The need for file protection is a direct result of the ability to access files. Access control may be a complete protection by Access control may be a complete protection by denying access. Or, the access may be controlled.

27

File File Protection: Protection: Types Types of

• f Access

Access File File Protection: Protection: Types Types of

• f Access

Access

Access control may be implemented by limiting Access control may be implemented by limiting the types of file access that can be made. The types of access may be The types of access may be Read: read from the file W i i i h fil Write: write or rewrite the file Execute: load the file into memory and i execute it Append: write new info at the end of a file Delete: delete a file List: list the name and attributes of the file

28

File File Protection: Protection: Access Access Control:

• ntrol: 1/4

1/4 File File Protection: Protection: Access Access Control: Control: 1/4 1/4

The most commonly used approach is to make The most commonly used approach is to make the access dependent on the identity of the user. Each file and directory is associated with an Each file and directory is associated with an access matrix specifying the user name and the types of permitted access types of permitted access. When a user makes a request to access a file or di t hi /h id tit i d a directory, his/her identity is compared against the information stored in the access t i matrix.

29

File File Protection: Protection: Access Access Control:

• ntrol: 2/4

2/4 File File Protection: Protection: Access Access Control: Control: 2/4 2/4

A M t i

File 1 File 2 File 3 File 4

A t 1 A t 2

Access Matrix

Own R Own R Inquiry C dit

File 1 File 2 File 3 File 4 Account 1 Account 2 U A

R W R W Credit Own Inquiry Inquiry

User A

R Own R W W R Inquiry debit Inquiry Credit

User B

R R Own R Inquiry debit

User C

30

W W

File File Protection: Protection: Access Access Control:

• ntrol: 3/4

3/4 File File Protection: Protection: Access Access Control: Control: 3/4 3/4

A O B C

File 1

Own R W R R W

File 1

Access-control Lists

In practice, the access matrix is sparse.

B Own R W C R

File 2

The matrix can be decomposed into

W A Own B

File 3

p columns (files), yielding access-control lists (ACL)

Own R W W

( ) However, this list can be very long!

B R C Own R

File 4

31

very long!

R R W

File File Protection: Protection: Access Access Control:

• ntrol: 4/4

4/4 File File Protection: Protection: Access Access Control: Control: 4/4 4/4

Decomposition by

File 1 File 3

p y rows (users) yields capability tickets. E h h

File 1 Own R W File 3 Own R W

User A

Each user has a number of tickets for file/directory access.

File 1 R File 2 Own R

User B

File 3 File 4 R

y These tickets may be authorized to loan or b i t th

R R W W R

be given to other users. All tickets may be held and managed by the

File 1 R File 2 R

User C

File 4 Own R

and managed by the OS for better protection.

W W

Capability Lists

32

Capab ty sts

Th E d The End

33