S-expression
The term S-expression or sexp (S stands for symbolic) refers to a convention for representing semi-structured data in human-readable textual form. S-expressions are probably best known for their use in the Lisp and Scheme programming languages. Other uses of S-expressions are in Lisp-derived languages such as DSSSL, and as mark-up in communications protocols like IMAP and John McCarthy's CBCL. The details of the syntax and supported data types vary in the different languages, but the most common feature among these languages is the use of S-expressions as parenthesized prefix notation (sometimes known as Cambridge Polish notation).
S-expressions are used for both code and data in Lisp (see McCarthy Recursive Functions of Symbolic Expressions [1]). S-expressions were originally intended only as machine representations of human-readable M-expressions, but Lisp programmers soon started using S-expressions as the default notation.
S-expressions can either be single objects such as numbers, LISP atoms including the special atoms nil and t, or cons pairs, written as (x . y). Longer lists are made up of nested cons pairs, for example
(1 . (2 . (3 . nil))) which can also be written more intelligibly as (1 2 3).
Program code can be written in S-expressions, using prefix notation. An extra piece of syntactic sugar for writing Lisp programs is that the common expression (quote x) can be written with the abbreviation 'x.
Example in Common Lisp:
(defun factorial (x)
(if (zerop x) 1
(* x (factorial (- x 1)))))
Example in Scheme:
(define (factorial x)
(if (zero? x) 1
(* x (factorial (- x 1)))))
Standardization
In May of 1997, Ron Rivest submitted an Internet-Draft [2] to be considered for publication as an RFC. The draft defined a syntax based on Lisp S-expressions but intended for general-purpose data storage and exchange (similar to XML) rather than specifically for programming. It was never approved as an RFC, but it has since been cited and used by other RFCs [3] and several other publications [4]. It was originally intended for use in SPKI.
Rivest's format defines an S-expression as being either an octet-string (a series of bytes) or a finite list of other S-Expressions. It describes three interchange formats for expressing this structure. One is the "advanced transport", which is very flexible in terms of formatting, and is syntactically similar to Lisp-style expressions, but they are not identical. The advanced transport, for example, allows octet-strings to be represented verbatim (the string's length followed by a colon and the entire raw string), a quoted form allowing escape characters, hexadecimal, Base64, or placed directly as a "token" if it meets certain conditions. (Rivest's tokens differ from Lisp tokens in that the former are just for convenience and aesthetics, and treated exactly like other strings, while the latter have specific syntactical meaning.) Another interchange format, intended to be more compact, easier to parse, and unique for any abstract S-expression, is the "canonical representation" which only allows verbatim strings, and prohibits whitespace as formatting outside strings. Finally there is the "basic transport representation", which is either the canonical form or the same encoded as Base64 and surrounded by braces, the latter intended to safely transport a canonically-encoded S-expression in a system which might change spacing (e.g. an email system which has 80-character-wide lines and wraps anything longer than that).
This format has not been widely adapted for use outside of SPKI. Rivest's S-expressions web page provides C source code for a parser and generator, which could theoretically be adapted and embedded into other programs, though licensing on these programs is unclear. However, there are no restrictions on independently implementing the format.