Personal Storage Table

In computing, a Personal Storage Table (.pst) is an open proprietary file format used to store copies of messages, calendar events, and other items within Microsoft software such as Microsoft Exchange Client, Windows Messaging, and Microsoft Outlook. The open format is controlled by Microsoft who provide free specifications and free irrevocable technology licensing.

The file format may also be known as a Personal Folders (File) or Post Office File. When functioning in its capacity as a cache for Outlook's Cached Exchange Mode feature, it may be called an Off-line Storage Table (.ost) or an Off-line Folders (File).

Overview

In Microsoft Exchange Server, the messages, the calendar, and other data items are delivered to and stored on the server. Microsoft Outlook stores these items in a personal-storage-table (.pst) or off-line-storage-table (.ost) files that are located on the local computer. Most commonly, the .pst files are used to store archived items and the .ost files to maintain off-line availability of the items. This is an essential feature of Microsoft Outlook.[1]

The size of these files no longer counts against the size of the mailbox used; by moving files from a server mailbox to .pst files, users can free storage space on their mailservers. To use the .pst files from another location the user needs to be able to access the files directly over a network from their mail client. While it is possible to open and use a .pst file from over a network, this is unsupported, and Microsoft advises against it, as .pst files are prone to corruption when used in this manner.

Both the .pst and .ost files use a fixed-block-based allocation scheme; the file is enlarged by a fixed amount of bytes, and the file internally maintains information about the allocated and non-allocated blocks. So, when data files like email messages are added to a .pst file, its file size is automatically adjusted by the mail client (if necessary). When mail is deleted from a .pst file, the size of the .pst file will stay the same, marking the space as unallocated so that it will hold future data items. Recently removed data items can actually be recovered from .pst and .ost files.

To reduce the size of .pst files, the user needs to compact them.

Data access

Password protection can be used to protect the content of the .pst files. However, Microsoft admits that the password adds very little protection, due to the existence of commonly available tools which can remove or simply bypass the password protection. The password to access the table is stored without the first and last XOR CRC-32 integer representation of itself in the .pst file. Outlook checks to make sure that it matches the user-specified password and refuses to operate if there is no match. The data is readable by the libpst project code.

Microsoft (MS) offers three values for the encryption setting: none, compressible, and high.

• None the .pst data is stored as plain text.
• Compressible the .pst data is encrypted with a byte-substitution cipher with a fixed substitution table.
• High (sometimes called "better") encryption is similar to a WWII German Enigma cipher with three fixed rotors.

Note that neither of the two encryption modes uses the user-specified password as any part of the key for the encryption.

Support

The .pst file format is supported by several Microsoft client applications, including Microsoft Exchange Client, Windows Messaging, and Microsoft Outlook; in the previous two, it was often used with a Personal Address Book (.pab) file, which stores address book. The .pst file format is an open format for which Microsoft provides free specifications and irrevocable free patent licensing through the Open Specification Promise.

The libpst project includes tools to convert .pst files into open formats such as mbox and LDAP Data Interchange Format. libpst is licensed under the GPL and is now included in Fedora 10. MVCOM is a commercially licensed COM Component that provides access to .pst files without MAPI.

There are tools to convert .pst to other formats or to upload to other online e-mails like Gmail, for example.

Formats and size

The file is structured as a B-tree with 512 byte nodes and leaves. All PST files begin with the four-byte magic string "!BDN", a four-byte CRC number, and a two-byte magic string of "SM".

Outlook 2002 and earlier use ANSI (extended ASCII with a codepage) encoding for their .pst and .ost files. This format has a maximum size of 2 GiB (2³¹ bytes) and does not support unicode. A file exceeding this size is likely to give error messages, such as ".pst has reached maximum size limit," and could become corrupted. Although superseded, this format is supported by all Outlook versions, by Internet Message Access Protocol Version 4rev1 (IMAP4) accounts and by HTTP accounts.

From Outlook 2003 and onward, the new standard format for .pst and .ost files is Unicode (UTF-16 little-endian), with 64-bit pointers instead of 32-bit to allow larger than 2 GiB sizes. The limit became 20 GB for Outlook 2003-2007, and increased to 50 GB from Outlook 2010. A file that is created in the personal-folders format in Outlook 2003 or later is not compatible and cannot be opened by earlier versions.

As with any file, .pst files can become corrupted. Growth over the limit has been a consistent problem; ANSI .pst that grew beyond 2 GiB and Unicode .pst that grew beyond 20 or 50 GB would become unusable. The scanpst.exe tool has been included with all versions of Outlook to detect and repair .pst database corruption, and Microsoft at one point offered a special-purpose PST2GB that would simply truncate the file to 2 GiB to allow Outlook to repair it.

Entourage and Outlook for Mac

Microsoft Entourage was Microsoft's email and personal information program for Mac OS X (replaced by Outlook for Macintosh in Microsoft Office for Mac 2011). While superficially similar to Outlook, it was an entirely different application, and used a unique database format which cannot be imported or exported, though user data can be imported and exported to and from another unique format called .rge (a bundle consisting of many individual files plus metadata). Entourage 2008, the current version as of May 2010, has no support for .pst files, though there exists Microsoft's .pst import tool for Entourage 2004; however, the tool could only import .pst files from Outlook for Mac 2001, and not any Windows versions. Entourage's replacement, Outlook for Office 2011 for Intel Macs, was able to import Outlook .pst files from Windows; however, data will be stored as many individual files, rather than in a single database such as .pst or the Entourage database.

Outlook for Mac 2001, which runs under Mac OS 9 or the Mac OS X Classic Environment, connects exclusively to Exchange servers, and to this day is closer to its Windows counterpart than Entourage is. It works directly with 'Outlook 97-2002' .pst files, and can freely interchange those files with Outlook for Windows, as recent versions are still compatible with the older .pst format. The limit for Outlook 2011 is 10GB.

External links

• Microsoft has released some of the format specification and reference implementation.