Jump to content

NTFS

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by JoaquinMiller (talk | contribs) at 00:16, 21 November 2007 (O! My. Sorry. I should preview more carefully.). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

NTFS
Developer(s)Microsoft
Full nameNew Technology File System
IntroducedJuly 1993 with Windows NT 3.1
Partition IDs0x07 (MBR)
EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (GPT)
Structures
Directory contentsB+ tree
File allocationBitmap/Extents
Bad blocksBitmap/Extents
Limits
Max volume size256 TiB minus 64 KiB [1]
Max file size2 TiB[2]
Max no. of files4,294,967,295 (232-1)
Max filename length255 UTF-16 characters[3]
Allowed filename
characters		any character except '\0' (NULL) and '/' (slash) [4] The Win32 Subsystem also excludes the use of \ (backslash) : (colon) * (asterisk)  ? (Question mark) " (quote) < (less than) > (greater than) and | (pipe)
Features
Dates recordedCreation, modification, POSIX change, access
Date range1 January 1601 – 28 May 60056 (File times are 64-bit numbers counting 100-nanosecond intervals (ten million per second) since 1601, which is 58,000+ years)
Date resolution100ns
ForksYes
AttributesRead-only, hidden, system, archive
File system
permissions		ACLs
Transparent
compression		Per-file, LZ77 (Windows NT 3.51 onward)
Transparent
encryption		Per-file,
DESX (Windows 2000 onward),
Triple DES (Windows XP onward),
AES (Windows XP Service Pack 1, Windows Server 2003 onward)
Other
Supported
operating systems		Windows NT family (Windows NT 3.1 to Windows NT 4.0, Windows 2000, Windows XP, Windows Server 2003, Windows Vista), DOSBox, and the Windows NT VDM

NTFS is the standard file system of Windows NT, including its later versions Windows 2000, Windows XP, Windows Server 2003, Windows Server 2008, and Windows Vista.[5] NTFS stands for NT File System.[6] The NT (which might (or might not) stand for New Technology) is from Windows NT.

NTFS replaced Microsoft's previous FAT file system, used in MS-DOS and early versions of Windows. NTFS has several improvements over FAT and HPFS (High Performance File System) such as improved support for metadata and the use of advanced data structures to improve performance, reliability, and disk space utilization plus additional extensions such as security access control lists and file system journaling. The exact specification is a trade secret, although (since NTFS v3.00) it can be licensed commercially from Microsoft through their Intellectual Property Licensing program.

Versioning

NTFS has five versions:

  • v1.0
  • v1.1
  • v1.2 found in NT 3.51 and NT 4
  • v3.0 found in Windows 2000
  • v3.1 found in Windows XP, Windows Server 2003, and Windows Vista

These final three versions are sometimes referred to as v5.0, v5.1, and v6.0, after the version of Windows NT with which they ship. Each newer version added extra features, for example Windows 2000 introduced quotas while Windows Vista introduced Transactional NTFS, NTFS symbolic links, and self-healing functionality.[7]

Internals

In NTFS, everything that has anything to do with a file (file name, creation date, access permissions and even contents) is stored as metadata. This abstract approach allowed easy addition of filesystem features during the course of Windows NT's development — an interesting example is the addition of fields for indexing used by the Active Directory software

NTFS allows any sequence of short (16-bit) values for name encoding (file names, stream names, index names, etc.). This means UTF-16 codepoints are supported, but the filesystem does not check whether the sequence is valid UTF-16 (it allows any sequence of short values, not restricted to those in the Unicode standard).

Internally, NTFS uses B+ trees to index file system data. Although complex to implement, this allows faster file open times in most cases. A file system journal is used in order to guarantee the integrity of the file system itself (but not of each individual file). Systems using NTFS are known to have improved reliability compared to FAT file systems.[8]

The Master File Table (MFT) essentially contains metadata about every file and directory on an NTFS file system. It includes parameters such as location, size, and permissions. It is used to aid in minimizing disk fragmentation.

Interoperability

Details on the implementation's internals are closed, so third-party vendors have a difficult time providing tools to handle NTFS.

Linux

A number of different software implementations have been developed to allow NTFS volumes to be read, and in some cases modified, under Linux:

  • Linux kernel 2.2: NTFS partitions can be read by the kernel since version 2.2.0.
  • Linux kernel 2.6: contains a driver written by Anton Altaparmakov (Cambridge University) and Richard Russon. It supports file read, overwrite and resize, in some cases.
  • NTFSMount: A userspace driver with limited file and directory read/write support is available using ntfsmount[9]
  • NTFS-3G: A userspace driver with full read/write support based on ntfsmount and included in many Linux distributions
  • NTFS for Linux: A commercial driver with full read/write support available from Paragon.
  • Captive NTFS: A 'wrapping' driver which uses Windows's own driver, ntfs.sys. This method is slower than the native drivers.

Note that all 3 userspace drivers, namely NTFSMount, NTFS-3G and Captive NTFS, are built on FUSE, a Linux kernel module tasked with bridging userspace and kernel code to save and retrieve data.

Almost all drivers listed above (except Paragon NTFS for Linux) are open source (GPL). Due to the complexity of the internal NTFS structures, both the built-in 2.6.14 kernel driver and the FUSE drivers will deny changes to the volume when they are considered to be unsafe, thus avoiding corruption.

Windows

While the different NTFS versions have a great degree of both forward and backward compatibility, there are technical considerations for mounting newer NTFS volumes in older versions of Windows. This affects dual-booting, and external portable hard drives.

For example, "Previous Versions" (a.k.a. Volume Shadow Copy) are lost because the older OS doesn't understand how to keep the new features' data updated.[10]

Others

FreeBSD, eComStation and Mac OS X versions 10.3 and later offer read-only NTFS support (there is a beta NTFS driver that allows write/delete for eComStation, but is generally considered unsafe). A free third-party tool for BeOS, which was based on NTFS-3G, allows full NTFS read and write. The read/write NTFS-3G driver has been also ported to FreeBSD, Mac OS X, NetBSD, Haiku and FreeDOS.

A commercial read/write driver for DOS called "NTFS4DOS" also exists.[1]

Compatibility with FAT

Microsoft currently provides a tool (convert.exe) to convert HPFS (only on Windows NT 3), FAT16 and, on Windows 2000 and higher, FAT32 to NTFS, but not the other way around.[11] Various third-party tools are all capable of safely resizing NTFS partitions. Microsoft added the ability to shrink or expand a partition with Windows Vista.

For historical reasons, the versions of Windows that do not support NTFS all keep time internally as local zone time, and therefore so do all file systems other than NTFS that are supported by current versions of Windows. However, Windows NT and its descendants keep internal timestamps as UTC and make the appropriate conversions for display purposes. Therefore, NTFS timestamps are in UTC. This means that when files are copied or moved between NTFS and non-NTFS partitions, the OS needs to convert timestamps on the fly. But if some files are moved when daylight saving time (DST) is in effect, and other files are moved when standard time is in effect, there can be some ambiguities in the conversions. As a result, especially shortly after one of the days on which local zone time changes, users may observe that some files have timestamps that are incorrect by one hour. Due to the differences in implementation of DST between the northern and southern hemispheres, this can result in a potential timestamp error of up to 4 hours in any given 12 months.[12]

Features

NTFS v3.0, the third version of NTFS to be introduced, includes several new features over its predecessors: disk usage quotas, sparse file support, reparse points, distributed link tracking and file-level encryption, also known as the Encrypting File System (EFS).

Alternate data streams (ADS)
Alternate data streams allows files to be associated with more than one data stream. For example, a file such as text.txt can have an ADS with the name of text.txt:secret.txt (of form filename:ads) that can only be accessed by knowing the ADS name or by specialized directory browsing programs. Alternate streams are not detectable in the original file's size but are lost when the original file (i.e. text.txt) is deleted with a RemoveFile or RemoveFileTransacted call (or a call that uses those calls), or when the file is copied or moved to a partition that doesn't support ADS (e.g. a FAT partition, a floppy disk, or a network share). While ADS is a useful feature, it can also easily eat up hard disk space if unknown either through being forgotten or not being detected.
Quotas
Disk quotas were introduced in NTFS v3. They allow the administrator of a computer that runs a version of Windows that supports NTFS to set a threshold of disk space that users may utilise. It also allows administrators to keep track of how much disk space each user is using. An administrator may specify a certain level of disk space that a user may use before they receive a warning, and then deny access to the user once they hit their upper limit of space. Disk quotas do not take into account NTFS's transparent file-compression, should this be enabled. Applications that query the amount of free space will also see the amount of free space left to the user who has a quota applied to them.
Sparse files
Sparse files are files which contain sparse data sets, data mostly filled with zeroes. Many scientific applications can generate very large sparse data sets. Because of this, Microsoft has implemented support for sparse files by allowing an application to specify regions of empty (zero) data. An application that reads a sparse file reads it in the normal manner with the file system calculating what data should be returned based upon the file offset. As with compressed files, the actual size of sparse files are not taken into account when determining quota limits.[13]
Reparse points
This feature was introduced in NTFS v3. These are used by associating a reparse tag in the user space attribute of a file or directory. When the object manager (see Windows NT line executive) parses a file system name lookup and encounters a reparse attribute, it knows to reparse the name lookup, passing the user controlled reparse data to every file system filter driver that is loaded into Windows 2000. Each filter driver examines the reparse data to see if it is associated with that reparse point, and if that filter driver determines a match then it intercepts the file system call and executes its special functionality. Reparse points are used to implement Volume Mount Points, Directory Junctions, Hierarchical Storage Management, Native Structured Storage and Single Instance Storage:
Volume mount points
Similar to Unix mount points, where the root of another file system is attached to a directory. In NTFS, this allows additional file systems to be mounted without requiring a separate drive letter (like C: or D:) for each.
Directory Junctions
Similar to Volume Mount Points, however directory junctions reference other directories in the file system instead of other volumes. For instance, the directory C:\exampledir with a directory junction attribute that contains a link to D:\linkeddir will automatically refer to the directory D:\linkeddir when it is accessed by a user-mode application. [14] This function is conceptually similar to symbolic links to directories in Unix except that the target in NTFS must always be another directory. (Typical Unix filesystems allow the target of a symbolic link to be any type of file.)
Hard links
Originally included to support the POSIX subsystem in Windows NT, hard links are similar to directory junctions, but used for files instead of directories. Hard links can only be applied to files on the same volume since an additional filename record is added to the file's MFT record. Short (8.3) filenames are also implemented as additional filename records that don't have separate directory entries.
Hierarchical Storage Management (HSM)
Hierarchical Storage Management is a means of transferring files that are not used for some period of time to less expensive storage media. When the file is next accessed the reparse point on that file determines that it is needed and retrieves it from storage.
Native Structured Storage (NSS)
NSS was an ActiveX document storage technology that has since been discontinued by Microsoft. It allowed ActiveX documents to be stored in the same multi-stream format that ActiveX uses internally. An NSS file system filter was loaded and used to process the multiple streams transparently to the application, and when the file was transferred to a non-NTFS formatted disk volume it would also transfer the multiple streams into a single stream.[15]
Volume Shadow Copy
The Volume Shadow Copy (VSC) service keeps historical versions of files and folders on NTFS volumes by copying old, newly-overwritten data to shadow copy (copy-on-write). The old file data is overlaid on the new when the user requests a revert to an earlier version. This also allows data backup programs to archive files currently in use by the file system. On heavily loaded systems, Microsoft recommends setting up a shadow copy volume on separate disk to reduce the I/O load on the main volume.
File compression
NTFS can compress files using a variant of the LZ77 algorithm (also used in the popular ZIP file format).[16] Although read-write access to compressed files is transparent, Microsoft recommends avoiding compression on server systems and/or network shares holding roaming profiles because it puts a considerable load on the processor.[17]
Single-user systems with limited hard disk space will probably use NTFS compression successfully. The slowest link in a notebook is not the CPU but the speed of the hard drive, so NTFS compression allows the limited, slow storage space to be better used, in terms of both space and (often) speed. NTFS compression can also serve as a replacement for sparse files when a program (e.g. a download manager) is not able to create files without content as sparse files.
Single Instance Storage (SIS)
When there are several directories that have different, but similar, files, some of these files may have identical content. Single instance storage allows identical files to be merged to one file and create references to that merged file. SIS consists of a file system filter that manages copies, modification and merges to files; and a user space service (or groveler) that searches for files that are identical and need merging. SIS was mainly designed for remote installation servers as these may have multiple installation images that contain many identical files; SIS allows these to be consolidated but, unlike for example hard links, each file remains distinct; changes to one copy of a file will leave others unaltered. This is similar to copy-on-write, which is a technique by which memory copying is not really done until one copy is modified.[18]
Encrypting File System (EFS)
EFS provides strong and user-transparent encryption of any file or folder on an NTFS volume. EFS works in conjunction with the EFS service, Microsoft's CryptoAPI and the EFS File System Run-Time Library (FSRTL).
EFS works by encrypting a file with a bulk symmetric key (also known as the File Encryption Key, or FEK), which is used because it takes a relatively smaller amount of time to encrypt and decrypt large amounts of data than if an asymmetric key cipher is used. The symmetric key that is used to encrypt the file is then encrypted with a public key that is associated with the user who encrypted the file, and this encrypted data is stored in an alternate data stream of the encrypted file. To decrypt the file, the file system uses the private key of the user to decrypt the symmetric key that is stored in the file header. It then uses the symmetric key to decrypt the file. Because this is done at the file system level, it is transparent to the user.[19] Also, in case of a user losing access to their key, support for recovery agents that can unencrypt files has been built in to the EFS system.
Symbolic links
Symbolic links were introduced in Windows Vista.[20] Symbolic links (or Soft links) are resolved on the client side. So when a symbolic link is shared, the target is subject to the access restrictions on the client, and not the server.
Transactional NTFS
As of Windows Vista, applications can use Transactional NTFS to group changes to files together into a transaction. The transaction will guarantee that all changes happen, or none of them do, and it will guarantee that applications outside the transaction will not see the changes until the precise instant they're committed.[21]

Limitations

The following are a few limitations of NTFS:

Reserved File Names
Though the file system supports paths up to about 32,000 Unicode characters with each path component (directory or filename) up to 255 characters long, certain names are unusable, since NTFS stores its metadata in regular (albeit hidden and for the most part inaccessible) files; accordingly, user files cannot use these names. These files are all in the root directory of a volume (and are reserved only for that directory). The names are: $MFT, $MFTMirr, $LogFile, $Volume, $AttrDef, . (dot), $Bitmap, $Boot, $BadClus, $Secure, $Upcase, and $Extend;[22] . (dot) and $Extend are both directories; the others are files.
Maximum Volume Size
In theory, the maximum NTFS volume size is 264-1 clusters. However, the maximum NTFS volume size as implemented in Windows XP Professional is 232-1 clusters. For example, using 64 KiB clusters, the maximum NTFS volume size is 256 TiB minus 64 KiB. Using the default cluster size of 4 KiB, the maximum NTFS volume size is 16 TiB minus 4 KiB. Because partition tables on master boot record (MBR) disks only support partition sizes up to 2 TiB, dynamic or GPT volumes must be used to create bootable NTFS volumes over 2 TiB.
Maximum File Size
Theoretical: 16 EiB minus 1 KiB ( bytes). Implementation: 16 TiB minus 64 KiB ( bytes)
Alternate Data Streams
Care must be exercised when copying or moving files from NTFS to other filesystem types. Windows system calls and programs can have varying behavior with regard to alternate data streams and might silently strip those which could not be stored on the destination filesystem. A safe way of copying or moving files is to use the BackupRead and BackupWrite system calls, which allow to enumerate streams, to verify whether each stream could be written to the destination volume and to knowingly skip offending streams.
Maximum date
NTFS was built to only recognize dates up to May 28 60056. After this date everything would reverse back to the date of January 1, 1601, in an event similar to the Year 2000 problem or the Year 2038 problem (although obviously much more remote).

Developers

NTFS developers include:

References

  1. ^ Microsoft Corporation. "Determining Maximum Volume Size". Retrieved 2007-08-21.
  2. ^ "NTFS Data Solutions Inc". Retrieved 2007-07-07.
  3. ^ Richard Russon and Yuval Fledel. "NTFS Documentation". Retrieved 2007-07-01.
  4. ^ UTF-16 codepoints accepted, but not validated
  5. ^ Custer, Helen (1994). Inside the Windows NT File System. Microsoft Press. ISBN 978-1-55615-660-1.
  6. ^ Nancy Winnick Cluts (January 1995). "Making Room for Long Filenames". Microsoft Developer Network. Retrieved 2007-10-23.
  7. ^ Loveall, John (2006). "Storage improvements in Windows Vista and Windows Server 2008" (PowerPoint). Microsoft Corporation. pp. 14–20. Retrieved 2007-09-04.
  8. ^ "Microsoft TechNet Resource Kit"
  9. ^ "ntfsmount wiki page on linux-ntfs.org"
  10. ^ cfsbloggers (July 14, 2006). "How restore points and other recovery features in Windows Vista are affected when dual-booting with Windows XP". The Filing Cabinet. Retrieved 2007-03-21.
  11. ^ "How to Convert FAT Disks to NTFS". Microsoft Corporation. 2001-10-25. Retrieved 2007-08-27. {{cite web}}: Check date values in: |date= (help)
  12. ^ "Beating the Daylight Savings Time bug and getting correct file modification times" The Code Project
  13. ^ "Sparse Files", MSDN Platform SDK: File Systems. Retrieved May 22, 2005.
  14. ^ Mark Russinovich, "Inside Win2K NTFS, Part 1"
  15. ^ John Saville, "What is Native Structured Storage?"
  16. ^ "File Compression and Decompression". MSDN Platform SDK: File Systems. Retrieved on 2005-08-18.
  17. ^ "Best practices for NTFS compression in Windows." Microsoft Knowledge Base. Retrieved on 2005-08-18.
  18. ^ "Single Instance Storage in Windows 2000" (PDF). Microsoft Research and Balder Technology Group.
  19. ^ How EFS Works, Microsoft Windows 2000 Resource Kit
  20. ^ "Symbolic Links". MSDN. Retrieved 2007-01-05.
  21. ^ "Transactional NTFS". MSDN. Retrieved 2007-02-02.
  22. ^ "How NTFS Works" Windows Server 2003 Technical Reference

See also

  • Linux-NTFS – an open source project to add NTFS support to the Linux kernel (write support is limited, but can be used for simple tasks), and write POSIX-compatible utilities for accessing and manipulating NTFS (ntfsprogs; includes ntfsls, ntfsresize, ntfsclone, etc). Linux NTFS FAQ and howto
  • NTFS-3G for Mac OS X
  • Captive NTFS – a shim which used the Windows NTFS driver to access NTFS filesystems under Linux
  • NTFS.com – documentation and resources for NTFS
  • Microsoft NTFS Technical Reference
Retrieved from "https://en.wikipedia.org/enwiki/w/index.php?title=NTFS&oldid=172810569"