Resource fork
The resource fork is a construct of the Mac OS operating system used to store structured data in a file, alongside unstructured data stored within the data fork. A resource fork stores information in a specific form, such as icons, the shapes of windows, definitions of menus and their contents, and application code (machine code). For example, a word processing file might store its text in the data fork, while storing any embedded images in the same file's resource fork. The resource fork is used mostly by executables, but every file is able to have a resource fork.
The Macintosh file system
Originally conceived and implemented by programmer Bruce Horn, the resource fork was used for three purposes with Macintosh file system. First, it was used to store all graphical data on disk until it was needed, then retrieved, drawn on the screen, and thrown away. This software variant of virtual memory helped Apple to reduce the memory requirements of the Apple Lisa from 1 MB to 128 KB in the Macintosh. Second, because all the pictures and text were stored separately in a resource fork, it could be used to allow a non-programmer to translate an application for a foreign market, a process called internationalization and localization. And finally, it could be used to distribute nearly all of the components of an application in a single file, reducing clutter and simplifying application installation and removal.
The resource fork is implemented in all of the file systems used for system drives on the Macintosh (MFS, HFS and HFS Plus). The presence of a resource fork makes it easy to store a variety of additional information, such as allowing the system to display the correct icon for a file and open it without the need for a file extension in the file name. While access to the data fork works like file access on any other operating system — pick a file, pick a byte offset, read some data — access to the resource fork works more like extracting structured records from a database. Microsoft Windows also has a concept of “resources”, but these are completely different from resources in Mac OS.
The resource fork is sometimes used to store the metadata of a file, although it can also be used for storing the actual data, as was the case with font files in the classic Mac operating systems. Note that the Macintosh file systems also have a separate area for metadata distinct from either the data or resource fork. Being part of the catalogue entry for the file, it is much faster to access this. However, the amount of data stored here is minimal, being just the creation and modification timestamps, the file type and creator codes, fork lengths, and the file name. Some files have only a resource fork. Classic 68k applications are one example, where even the executable code is contained in resources of type ‘CODE’. Later PowerPC binaries store the executable code in the data fork.
As resource forks are supported only on the file systems HFS and HFS Plus, they cannot be used on operating systems which use other file systems. At present, HFS is supported only by the Macintosh operating system, which means that only machines running Mac OS can use resource forks. Even in a Mac OS system, resource forks cannot be used if the Unix File System has been installed. In the HFS Plus file system, which is currently the system most commonly used under Mac OS, settings can be made to allow other forks in addition to the data and resource forks, to create a “multi-fork” application. However, as forks can make it difficult to exchange files with other operating systems, this feature is not in common use. Even in Mac OS X, resource forks are seldom used anymore.
Currently, Mac OS X does support resource forks on Windows SMB shares by creating a hidden file in the same directory with the data fork file, with the characters “._” at the beginning of the file name. However, this may be annoying for some users, especially because some Windows power users always keep hidden files visible, which is sensible for everyday Windows use. Besides, Windows does not treat those files correctly as the file itself is moved or removed. A few resource fork files created by Mac OS X on an SMB share can be disabled by an Apple supported action.
Resource identifiers
Each resource has an OSType identifier (a four byte value) and an ID (a signed 16-bit word), as well as an optional name. There are standardised resource types for dialog boxes (‘DITL’), images (‘PICT’), sounds (‘snd ’) — and even for executable binaries (‘CODE’) which, until the advent of the PowerPC processor, were without exception stored in the resource fork. Subroutines for rendering windows are stored in their own type of resources (‘WDEF’), subroutines for rendering menus in theirs (‘MDEF’), and if there is a type of data you think does not fit any of the standardised categories, you can just as well use a type of your own (e.g. ‘John’) — actually any four characters or 32-bit value can serve as a resource type. This arrangement enabled users to easily customise not only individual applications but also the operating system itself, using tools such as ResEdit to modify the resources of an application file or any of the system files.
Within an application or other code, resources can be loaded simply using a combination of their type, ID or name, without regard to how and where they are stored in the resource fork. The client is returned a Handle to the loaded resource which can then be accessed like any other heap-based data. The OS component that facilitates this is the Resource Manager. In addition to abstracting the details of the data storage from the data itself, the Resource Manager also arranges sets of open resource forks into a stack, with the most recently opened file on top. When trying to load a resource, it will look in the top of the stack first, (perhaps the current document's resource fork), then the next one down (the application's resource fork), then the next one (system resource forks). This arrangement is very powerful — it permits local resources to override more global ones lower down — so an application can provide its own icons or fonts in place of the standard system ones, for example. It also allows an application to load resources from the system using the same API as any other resource, without regard to where or how that resource is stored — to the application, all resources are equally available and easy to use. The system reserves resource IDs in a certain range to help avoid resource conflicts arising from this. Resource Manager APIs allow the programmer to manipulate the stack and modify the search behaviour.
Editing resource forks
As the resource fork can be edited with a resource editor such as ResEdit, it can be used to localize and customize software. In addition, most resource editors allow visual editing of data. In Mac OS X, it is possible to use resources when developing an application. However, if the application may need to be used in UFS, it is also possible to configure it so that the entire resource fork is moved to the data fork, using the Raw Resource File setting. The integrated development environments distributed for free by Apple Inc., which include MPW and Apple Developer's Tools, include a compiler called Rez. This uses a dedicated language, also called Rez, which can be used to create a resource fork by compiling source code. A decompiler, DeRez, which can be used to change a resource fork back into Rez code is also included.
In the structure of the resource fork, there is a piece of data called a “resource map” which stores the positions of resource data items. This can be used to allow random access to resource data based on the defined IDs and names. The resource fork can be thought of as consisting of essentially two objects, the resource map and the resource data itself, but in fact each data type is a hierarchical structure which stores multiple items of data. The format in which the information in the resource data is stored is defined based on the types of information, which are known as “resource types.” Resource data often makes references to other types of data.
To view the resource fork in the Terminal application. Append “/..namedfork/rsrc” to your command. e.g., take the command “ls -aol IMG_0593.jpg” then append the resource fork viewing suffix “ls -aol IMG_0593.jpg/..namedfork/rsrc” to view the ls -aol command information of the resource fork of file “IMG_0593.jpg”
How a resource fork is accessed
A resource fork is accessed via an API called the Resource Manager.
- When a resource fork is accessed, data including the start position and length of the resource data and resource map is read in from the header.
- If a resource type to read in has been specified, a check is performed to make sure that type is present in the resource list, and the number of items of data containing that type and their offsets in the resource reference list from the start position of the resource map is found.
- The resource ID, the offset of the resource name, the resource properties, and the offset of the data from the start position of the resource data is found.
- If resource data with the specified ID or name is present in the resource data, the offset obtained above is accessed, the data length is found, and all the data stored there is read in, and returned as the return value.
There are also File Manager APIs such as PBOpenRF() that allow access to the raw resource fork; however, they should be used only for applications such as copying a file – Apple strongly warns against using the resource fork as a “second data fork.”
From the POSIX interface, the resource fork can be accessed as filename/..namedfork/rsrc or as filename/rsrc; the shorter form has been deprecated on Mac OS X 10.4. [citation needed]
Data types in a resource fork
The smallest elements making up a resource fork are called data types. There are several data types. After a resource fork is accessed, its contents can be found by reading it in as appropriate for the data types defined in advance. Placing definitions inside the program stating how data is to be treated makes it possible to store resources called TMPL resources as well. Using this method increases the visibility of the data when viewed with a program such as ResEdit, making later editing simpler. As the Macintosh platform originated with Motorola-based processors (68k and PPC), the data is serialized to disk in big endian format.
The following is a list of the major data types, in alphabetical order.
| Data type (actual name) | Description |
|---|---|
| BBIT (binary bit) | Represents a single boolean bit (true or false). Normally the number of BBITs must be a multiple of 8. |
| BOOL (boolean) | Represents a boolean value. It consists of 2 bytes; 256 is true, and 0 is false. |
| CHAR (character) | Represents a one-byte character. |
| CSTR (C string) | Represents a string of the form used in the C programming language, i.e. a series of bytes terminated by 0x00. |
| DLNG (decimal long word integer) | A decimal long word (4 byte) integer. Represents values between approximately −2.1 billion and 2.1 billion. |
| HEXD (hex dump) | Indicates that the data from this position to the end is hexadecimal. This is used to represent code resources or compressed data. |
| HLNG (long word hexadecimal) | This data is treated as a 4 byte hexadecimal value. It is used, among other things, to represent integers greater than 2.1 billion, such as unsigned long values in C. |
| PSTR (Pascal string) | Represents a Pascal string, with the first byte giving the length of the string. |
| TNAM (type name) | A string representing a value such as a creator code, which is always 4 bytes long. |
| RECT (rectangle) | Represents the coordinates of the corners of a rectangle. Always 8 bytes long. |
Major resource types
Note that types must be 4 bytes long, so types like snd and STR actually have a space (0x20) at the end.
| Name of resource type (actual name) | Description |
|---|---|
| alis (alias) | Stores an alias to another file, in a resource fork of a file whose "alias" attribute bit is set |
| ALRT (alert) | Defines the shape of an application alert box |
| APPL (application) | Stores application information |
| BNDL (bundle) | Defines data such as a file type icon used in an application |
| cicn (color icon) | Defines a color icon used in data |
| clut (color look-up table) | Defines a color palette used in data |
| CNTL (control) | Defines the details of a component positioned in a window |
| CODE (code resource) | Stores the machine code for the program |
| CURS (cursor) | Defines the shape of a monochrome cursor |
| DITL (dialog item list) | Defines a component of a window |
| DLOG (dialog) | Defines the shape of a dialog box for an application |
| FREF (file reference) | Defines a file type handled by an application |
| hfdr (icon balloon help) | Defines the contents and shape of the balloon help displayed when the cursor hovers over the file in the Finder |
| icl8 (8 bit icon list) | Defines an icon displayed in the Finder |
| icns (32 bit icon list) | Defines an icon displayed in the Finder |
| ICON (icon) | Defines a monochrome item used in data |
| kind (file description) | Defines a description of a file type |
| MBAR (menu bar) | Defines a menu and menu bar for an application |
| MDEF (menu definition) | Defines a menu for an application. Can also be used to define menus with complex shapes such as color palettes. |
| MENU (menu) | Defines the menu items in an application |
| MooV (movie) | Stores a QuickTime movie |
| open (open) | Defines a file type which the application can open |
| PICT (picture) | Stores a PICT image contained in the file |
| PREF (preference) | Stores the environment settings for an application |
| snd (sound) | Stores a sound used in the file |
| STR (string) | Stores a string or hexadecimal data used in the file |
| STR# (string list) | Stores multiple strings used in the file |
| styl (style) | Defines style information, such as the font, color and size of text |
| TEXT (text) | Stores text |
| TMPL (template) | Defines the format for the resource data |
| vers (version) | Defines the version or region of use of the file |
| WDEF (window definition) | Defines a window for the application. Windows of an unspecified shape can also be defined. |
| WIND (window) | Defines the shape of an application window |
Major resource editors
- ResEdit
- Distributed free of charge by Apple. Can be used for visual editing of resource data. If the structure of data is known, it can display a range of different types of data in a visual format.
- Resorcerer
- Expensive, but popular, as it can be used for visual editing of many more types of data than ResEdit.
- HexEdit
- A binary editor, which in fact is normally used more for editing the data fork rather than the resource fork.
- ResKnife
- Open‐source editor for Mac OS X
- Rezycle
- A Mac OS X tool that extracts resources from a resource fork into separate binary files while converting many types into formats suitable for modern development.
Compatibility problems
The complexity of programming with resource forks has led to compatibility problems when accessing other file systems via file sharing protocols such as AFP, SMB, NFS and FTP, when storing to non-HFS volumes, or when transmitting files to other systems in other ways (such as via email). The AFP protocol natively supports Resource Forks, and so resource forks are typically transmitted to these volumes as-is, and stored by the server transparently to clients. The SMB protocol supports a file metadata system similar to Macintosh forks known as Alternate Data Streams. Mac OS X did not support storing resource forks in ADSes on SMB volumes by default until OS X 10.6. In previous versions of the OS, including upgraded versions of 10.6, this feature can be enabled with a param change or by creating a special file.[1]
Networked file sharing protocols such as NFSv3 and FTP do not have a concept of file metadata, and so there is no way to natively store resource forks. This is also true when writing to certain types of local file systems, including UFS, and on SMB volumes where Alternate Data Stream support is not enabled. In those cases, OS X stores metadata and resource forks using a technique called AppleDouble, in which the data fork is written as one file, and the resource fork and metadata are written as an entirely separate file preceded by a “._” naming convention. For example: ExampleFile.psd would contain the data fork, and ._ExampleFile.psd would contain the resource fork and metadata.
Compatibility problems can arise because Mac OS X will handle storage of resource forks differently, depending on OS X version, settings, and file system type. For example, on an SMB network with a mixture of 10.5 and 10.6 clients. A freshly installed 10.6 client will look for and store resource forks on an SMB volume in ADSes, but the 10.5 client will (by default) ignore ADSes and use AppleDouble format to handle forks. If a fileserver supports both AFP and NFS, then clients using NFS will store files in AppleDouble format, whereas AFP users will use AppleSingle format (resource forks are stored natively) [dubious – discuss]. In those cases, compatibility can sometimes be maintained by forcing clients to use, or not use, AppleDouble format.
Many fileservers providing AFP support do not natively support resource forks on their local file systems. In those cases the forks may be stored in special ways, such as specially named files, special directories, or even Alternate Data Streams.
Another challenge is preserving resource forks when transmitting files using non-resource fork-aware applications or with certain transfer methods, including email and FTP. A number of file formats, such as MacBinary and BinHex, have been created to handle this. Command-line system tools SplitForks and FixupResourceForks allow manual flattening and merging of resource forks. In addition, a file server seeking to present file systems to Macintosh clients must accommodate the resource fork as well as the data fork of files; UNIX servers providing AFP support usually implement this with hidden directories.
Older applications written with the Carbon API have a potential issue when being ported to the current Intel Macs. While the Resource Manager and operating system know how to deserialize data correctly for common resources like “snd ” or “moov”, resources created using TMPL resources have to be byte swapped manually to ensure file interoperability between PPC and Intel-based versions of an application. (While the resource map and other implementation details are big endian, the Resource Manager by itself doesn't have any knowledge of the contents of a generic resource, and so cannot perform the byte swapping automatically.)
Until the advent of Mac OS X v10.4, the standard UNIX command line utilities in Mac OS X (such as cp and mv) did not respect resource forks. To copy files with resource forks, one had to use ditto or CpMac and MvMac.
Other operating systems
The concept of a resource manager for graphics objects, to save memory, originated in the OOZE package on the Alto in Smalltalk-76.[2] The concept is now largely universal in all modern operating systems. However, the concept of the resource fork remains peculiar to the Macintosh. Most operating systems used a binary file containing resources, which is then “tacked onto” the end of an existing program file. This solution is used on Microsoft Windows for instance, and similar solutions are used with the X Window System, although the resources are often left as a separate file.
Although the Windows NT NTFS can support forks (and so can be a file server for Mac files), the native feature providing that support, called an alternate data stream, (introduced for this very reason) has never been used extensively — certainly not as a true resource fork. However, Windows operating system features (such as the standard Summary tab in the Properties page for non-Office files) and Windows applications are using them more often now, and Microsoft was developing a next-generation file system that has this sort of feature as basis.
Early versions of the BeOS implemented a database within the file system, which could be used in a manner analogous to a resource fork. Performance issues led to a change in later releases to a system of complex file system attributes. Under this system resources were handled in a fashion somewhat more analogous to the Mac.
AmigaOS does not use forked files. Its executable files are internally divided into a modular structure of large pieces (hunk) capable of storing code, data, and additional information. Similarly, data and project files have a chunk structure codified in the IFF standard. Other file types are stored similarly to other operating systems. Though not strictly a resource fork, AmigaOS stores meta data in files known as .info files. .info files can be identified by the .info extension; for example, if you save a project to a disk, two files will be saved, MyProject and MyProject.info. MyProject would be the actual project data and MyProject.info would contain the project icon, information regarding which program is needed to open the project (since there is no application binding in AmigaOS), special project options and any user comments. .info files are invisible on the Amiga's desktop (Workbench). The icon on the desktop, taken from the .info itself, is the interface metaphor through which the user interacts both with the project itself and its associated .info file. A dialog box accessible by right-clicking the icon allows the user to see and modify the metadata present in the .info file. .info files can be seen as individual files in the Command line interface or a File manager. Modern AmigaOS clones (AROS, MorphOS and AOS4) inherit the structure (complete with metadata) of the .info files of older AmigaOS versions, and can also accept standard PNG graphic files as icon bitmaps in their .info files.
NeXT operating systems NeXTSTEP and OPENSTEP, and its successor, Mac OS X, and other systems like RISC OS implemented another solution. Under these systems the resources are left in an original format, for instance, pictures are included as complete TIFF files instead of being encoded into some sort of container. These resources are then placed in a directory along with the executable code and “raw data”. The directory (called a “bundle” or “application directory”) is then presented to the user as the application itself. This solution provides all of the same functionality as the resource fork, but allows the resources to be easily manipulated by any application – a “resource editor” (like ResEdit) is not needed. From the command line interface, the bundle appears to be a normal directory. This approach was not an option on the original Mac OS, since the file system (MFS) did not support folders/directories. Mac OS X does retain the classic Resource Manager API as part of its Carbon libraries for backward compatibility. However, the resources themselves can now be stored in separate data files within the file system — the Resource Manager now hides this implementation change from the client code.
References
- ^ "Mac OS X v10.5, v10.6: About named streams on SMB-mounted NAS, Mac OS X, and Windows servers". Retrieved 2010-04-19.
- ^ "The Early History of Smalltalk". Retrieved 2008-07-24.
External links
- Description of the Resource File Format
- Apple Developer Resource Library: Resource Manager Reference
- Apple Developer Resource Library: Resource Management, Bundles
- The Grand Unified Model – History of the resource fork, from folklore.org
- Rezycle – Resource extraction tool
- What’s up with Mac OS X Resource forks, Extended Attributes, NTFS Streams and Dot-Underscore files?
- When I save a file using SMB protocol, what information is saved in the “dot-underscore” (._) files? How is this information stored on an NTFS file system?