The Windows File Management 18

 

 

 

 

 

Microsoft SMB Protocol and CIFS Protocol Overview

 

The Server Message Block (SMB) Protocol is a network file sharing protocol, and as implemented in Microsoft Windows is known as Microsoft SMB Protocol. The set of message packets that defines a particular version of the protocol is called a dialect. The Common Internet File System (CIFS) Protocol is a dialect of SMB. Both SMB and CIFS are also available on VMS, several versions of Unix, and other operating systems. The technical reference to CIFS is available from Microsoft Corporation at here. Information on licensing CIFS can be found at here. Information on licensing Microsoft SMB Protocol can be found at here. Although its main purpose is file sharing, additional Microsoft SMB Protocol functionality includes the following:

 

  1. Dialect negotiation
  2. Determining other Microsoft SMB Protocol servers on the network, or network browsing
  3. Printing over a network
  4. File, directory, and share access authentication
  5. File and record locking
  6. File and directory change notification
  7. Extended file attribute handling
  8. Unicode support
  9. Opportunistic locks

 

In the OSI networking model, Microsoft SMB Protocol is most often used as an Application layer or a Presentation layer protocol, and it relies on lower-level protocols for transport. The transport layer protocol that Microsoft SMB Protocol is most often used with is NetBIOS over TCP/IP (NBT). However, Microsoft SMB Protocol can also be used without a separate transport protocol, the Microsoft SMB Protocol/NBT combination is generally used for backward compatibility. The Microsoft SMB Protocol is a client-server implementation and consists of a set of data packets, each containing a request sent by the client or a response sent by the server. These packets can be broadly classified as follows:

 

1.      Session control packets - Establishes and discontinues a connection to shared server resources.

2.      File access packets - Accesses and manipulates files and directories on the remote server.

3.      General message packets - Sends data to print queues, mailslots, and named pipes, and provides data about the status of print queues.

 

Some message packets may be grouped and sent in one transmission to reduce response latency and increase network bandwidth. This is called "batching.".

 

Opportunistic Locks

 

An opportunistic lock (also called an oplock) is a lock placed by a client on a file residing on a server. In most cases, a client requests an opportunistic lock so it can cache data locally, thus reducing network traffic and improving apparent response time. Opportunistic locks are used by network redirectors on clients with remote servers, as well as by client applications on local servers. Opportunistic locks coordinate data caching and coherency between clients and servers and among multiple clients. Data that is coherent is data that is the same across the network. In other words, if data is coherent, data on the server and all the clients is synchronized. Opportunistic locks are not commands by the client to the server. They are requests from the client to the server. From the point of view of the client, they are opportunistic. In other words, the server grants such locks whenever other factors make the locks possible.

When a local application requests access to a remote file, the implementation of opportunistic locks is transparent to the application. The network redirector and the server involved open and close the opportunistic locks automatically. However, opportunistic locks can also be used when a local application requests access to a local file and access by other applications and processes must be delegated to prevent corruption of the file. In this case, the local application directly requests an opportunistic lock from the local file system and caches the file locally. When used in this way, the opportunistic lock is effectively a semaphore managed by the local server, and is mainly used for the purposes of data coherency in the file and file access notification.

Before using opportunistic locks in your application, you should be familiar with the file access and sharing modes. The maximum number of concurrent opportunistic locks that you can create is limited only by the amount of available memory. For example, the maximum number created in test conditions with Windows 2000 and the NTFS file system is 359,000 locks. Local applications should not attempt to request opportunistic locks from remote servers. An error will be returned by DeviceIoControl() if an attempt is made to do this. Opportunistic locks are of very limited use for applications. The only practical use is to test a network redirector or a server opportunistic lock handler. Typically, file systems implement support for opportunistic locks. Applications generally leave opportunistic lock management to the file system drivers. Anyone implementing a file system should use the Installable File System (IFS) Kit. Anyone developing a device driver other than an installable file system should use the Windows Driver Kit (WDK). Opportunistic locks and the associated operations are a superset of the opportunistic lock portion of the Common Internet File System (CIFS) protocol, an Internet Draft. The CIFS protocol is an enhanced version of the Server Message Block (SMB) protocol. The CIFS Internet Draft explicitly identifies that a CIFS implementation may implement opportunistic locks by refusing to grant them.

 

Sparse Files

 

A file in which much of the data is zeros is said to contain a sparse data set. Files like these are typically very large, for example, a file containing image data to be processed or a matrix within a high-speed database. The problem with files containing sparse data sets is that the majority of the file does not contain useful data and, because of this, they are an inefficient use of disk space. The file compression in the NTFS file system is a partial solution to the problem. All data in the file that is not explicitly written is explicitly set to zero. File compression compacts these ranges of zeros. However, a drawback of file compression is that access time may increase due to data compression and decompression.

Support for sparse files is introduced in the NTFS file system as another way to make disk space usage more efficient. When sparse file functionality is enabled, the system does not allocate hard drive space to a file except in regions where it contains nonzero data. When a write operation is attempted where a large amount of the data in the buffer is zeros, the zeros are not written to the file. Instead, the file system creates an internal list containing the locations of the zeros in the file, and this list is consulted during all read operations. When a read operation is performed in areas of the file where zeros were located, the file system returns the appropriate number of zeros in the buffer allocated for the read operation. In this way, maintenance of the sparse file is transparent to all processes that access it, and is more efficient than compression for this particular scenario.

The default data value of a sparse file is zero; however, it can be set to other values.

 

Sparse File Operations

 

To determine whether a file system supports sparse files, call the GetVolumeInformation() function and examine the FILE_SUPPORTS_SPARSE_FILES bit flag. Most applications are not aware of sparse files and will not create sparse files. The fact that an application is reading a sparse file is transparent to the application. An application that is aware of sparse-files should determine whether its data set is suitable to be kept in a sparse file. After that determination is made, the application must explicitly declare a file as sparse, using the FSCTL_SET_SPARSE control code. After an application has set a file to be sparse, the application can use the FSCTL_SET_ZERO_DATA control code to set a region of the file to zero. In addition, the application can use the FSCTL_QUERY_ALLOCATED_RANGES control code to speed searches for nonzero data in the sparse file. When you perform a write operation (with a function or operation other than FSCTL_SET_ZERO_DATA) whose data consists of nothing but zeros, zeros will be written to the disk for the entire length of the write. To zero out a range of the file and maintain sparseness, use FSCTL_SET_ZERO_DATA. A sparseness-aware application may also set an existing file to be sparse. If an application sets an existing file to be sparse, it should then scan the file for regions which contain zeros, and use FSCTL_SET_ZERO_DATA to reset those regions, thereby possibly deallocating some physical disk storage. An application upgraded to sparse file awareness should perform this conversion.

When you perform a read operation from a zeroed-out portion of a sparse file, the operating system may not read from the hard drive. Instead, the system recognizes that the portion of the file to be read contains zeros, and it returns a buffer full of zeros without actually reading from the disk. As with any other file, the system can write data to or read data from any position in a sparse file. Nonzero data being written to a previously zeroed portion of the file may result in allocation of disk space. Zeros being written over nonzero data (only with FSCTL_SET_ZERO_DATA) may result in a deallocation of disk space. Note that it is up to the application to maintain sparseness by writing zeros with FSCTL_SET_ZERO_DATA. Defragmentation tools that handle compressed files on NTFS file systems will correctly handle sparse files on NTFS file system volumes. Large and highly fragmented sparse files can exceed the NTFS limitation on disk extents before available space is used.

 

Obtaining the Size of a Sparse File

 

Use the GetCompressedFileSize() function to obtain the actual size allocated on disk for a sparse file. This total does not include the size of the regions which were deallocated because they were filled with zeroes. Use the GetFileSize() function to determine the total size of a file, including the size of the sparse regions that were deallocated.

 

Sparse Files and Disk Quotas

 

A sparse file affects user quotas by the nominal size of the file, not the actual allocated amount of disk space. That is, creating a 50-MB file with all zero bytes consumes 50 MB of that user's quota. This means that as the user writes data to the sparse file, he need not worry about exceeding his hard quota limit, because he has already been charged for the space. System administrators should not count on the size of a sparse file remaining small. Therefore they should not grant their users hard quota limits that exceed the physical space available, despite the use of sparse files.

 

Symbolic Links

 

A symbolic link is a file-system object that points to another file system object. The object being pointed to is called the target. Symbolic links are transparent to users; the links appear as normal files or directories, and can be acted upon by the user or application in exactly the same manner. Symbolic links are designed to aid in migration and application compatibility with UNIX operating systems. Microsoft has implemented its symbolic links to function just like UNIX links. Symbolic links are available in NTFS starting with Windows Vista.

 

Symbolic Link Effects on File Systems Functions

 

Several standard file functions are affected by the use of symbolic links. The following Table lists those functions and describes the changes in behavior. In the descriptions column, the following terms are used:

  1. Source file - The original file that is to be copied.
  2. Destination file - The newly created copy of the file.
  3. Target - The entity that a symbolic link points to.

 

Functions

Description

CopyFile()

If the source file is a symbolic link, the actual file copied is the target of the symbolic link. If the destination file already exists and is a symbolic link, the symbolic link is overwritten by the source file.

CopyFileEx()

The flag COPY_FILE_COPY_SYMLINK is now available. If COPY_FILE_COPY_SYMLINK is specified and:

  1. If the source file is a symbolic link, the symbolic link is copied, not the target file.
  2. If the source file is not a symbolic link, there is no change in behavior.
  3. If the destination file is an existing symbolic link, the symbolic link is overwritten, not the target file.
  4. If COPY_FILE_FAIL_IF_EXISTS is also specified, and the destination file is an existing symbolic link, the operation fails in all cases.

 

If COPY_FILE_COPY_SYMLINK is not specified and:

  1. If COPY_FILE_FAIL_IF_EXISTS is also specified, and the destination file is an existing symbolic link, the operation fails only if the target of the symbolic link exists.
  2. If COPY_FILE_FAIL_IF_EXISTS is not specified, there is no change in behavior.

CreateFile()

If the call to this function creates a new file, there is no change in behavior. If FILE_FLAG_OPEN_REPARSE_POINT is specified and:

  1. If an existing file is opened and it is a symbolic link, the handle returned is a handle to the symbolic link.
  2. If CREATE_ALWAYS, TRUNCATE_EXISTING, or FILE_FLAG_DELETE_ON_CLOSE are specified, the file affected is a symbolic link.

 

If FILE_FLAG_OPEN_REPARSE_POINT is not specified and:

  1. If an existing file is opened and it is a symbolic link, the handle returned is a handle to the target.
  2. If CREATE_ALWAYS, TRUNCATE_EXISTING, or FILE_FLAG_DELETE_ON_CLOSE are specified, the file affected is the target.

CreateHardLink()

If the path points to a symbolic link, the function creates a hard link to the target.

DeleteFile()

If the path points to a symbolic link, the symbolic link is deleted, not the target. To delete a target, you must call CreateFile() and specify FILE_FLAG_DELETE_ON_CLOSE.

FindFirstChangeNotification()

If the path points to a symbolic link, the notification handle is created for the target. If an application has registered to receive change notifications for a directory that contains symbolic links, the application is only notified when the symbolic links have been changed, not the target files.

FindFirstFile()

If the path points to a symbolic link, the WIN32_FIND_DATA buffer contains information about the symbolic link, not the target.

FindFirstFileEx()

If the path points to a symbolic link, the WIN32_FIND_DATA buffer contains information about the symbolic link, not the target.

FindNextFile()

If the path points to a symbolic link, the WIN32_FIND_DATA buffer contains information about the symbolic link, not the target.

GetBinaryType()

If the path points to a symbolic link, the target file is used.

GetCompressedFileSize()

If the path points to a symbolic link, the function returns the file size of the target.

GetDiskFreeSpace()

If the path points to a symbolic link, the operation is performed on the target.

GetDiskFreeSpaceEx()

If the path points to a symbolic link, the operation is performed on the target.

GetFileAttributes()

If the path points to a symbolic link, the function returns attributes for the symbolic link.

GetFileAttributesEx()

If the path points to a symbolic link, the function returns attributes for the symbolic link.

GetFileSecurity()

If the path points to a symbolic link, the function returns attributes for the symbolic link.

GetTempPath()

If the path points to a symbolic link, the temp path name maintains any symbolic links.

GetVolumeInformation()

If the path points to a symbolic link, the function returns volume information for the target.

SetFileAttributes()

If the path points to a symbolic link, the function returns attributes for the symbolic link.

SetFileSecurity()

If the path points to a symbolic link, the function returns attributes for the symbolic link.

SetVolumeName()

If the path points to a symbolic link, the function acts on the target.

 

 

 

Programming Considerations

 

Keep the following programming considerations in mind when working with symbolic links:

  1. Symbolic links can point to a non-existent target.
  2. When creating a symbolic link, the operating system does not check to see if the target exists.
  3. If an application tries to open a non-existent target, ERROR_FILE_NOT_FOUND is returned.
  4. Symbolic links are reparse points.
  5. There is a maximum of 31 reparse points (and therefore symbolic links) allowed in a particular path.

 

Creating Symbolic Links

 

The function CreateSymbolicLink() allows you to create symbolic links using either an absolute or relative path. Symbolic links can either be absolute or relative links. Absolute links are links that specify each portion of the path name; relative links are determined relative to where relative–link specifiers are in a specified path. Relative links are specified using the following conventions:

 

  1. Dot (. and ..) conventions - for example, "..\" resolves the path relative to the parent directory.
  2. Names with no slashes (\) - for example, "tmp" resolves the path relative to the current directory.
  3. Root relative, for example, "\windows\system32" resolves to the <current drive>:\windows\system32. directory
  4. Current working directory-relative - for example, if the current working directory is c:\windows\system32, "c:file.txt" resolves to c:\windows\system32\file.txt. Note:   If you specify a current working directory–relative link, it is created as an absolute link, due to the way the current working directory is processed based on the user and the thread.

 

A symbolic link can also contain both junction points and mounted folders as a part of the path name. Symbolic links can point directly to a remote file or directory using the UNC path. Relative symbolic links are restricted to a single volume.

 

Example of an Absolute Symbolic Link

 

In this example, the original path contains a component, 'x', which is an absolute symbolic link. When 'x' is encountered, the fragment of the original path up to and including 'x' is completely replaced by the path that is pointed to by 'x'. The remainder of the path after 'x' is appended to this new path. This now becomes the modified path.

X: C:\alpha\beta\absLink\gamma\file

Link: absLink maps to \\machineB\share

Modified Path: \\machineB\share\gamma\file

 

Example of a Relative Symbolic Links

 

In this example, the original path contains a component 'x', which is a relative symbolic link. When 'x' is encountered, 'x' is completely replaced by the new fragment pointed to by 'x'. The remainder of the path after 'x', is appended to the new path. Any dots (..) in this new path replace components that appear before the dots (..). Each set of dots replace the component preceding. If the number of dots (..) exceed the number of components, an error is returned. Otherwise, when all component replacement has finished, the final, modified path remains.

X: C:\alpha\beta\link\gamma\file

Link: link maps to ..\..\theta

Modified Path: C:\alpha\beta\..\..\theta\gamma\file

Final Path: C:\theta\gamma\file

 

 

 

< Windows Files 17 | Win32 Programming | Win32 File Index | Windows Files 19 >