Motion JPEG

In multimedia, Motion JPEG (M-JPEG) is an informal name for a class of video formats where each video frame or interlaced field of a digital video sequence is separately compressed as a JPEG image. Originally developed for multimedia PC applications, where more advanced formats have displaced it, M-JPEG is now used by many portable devices with video-capture capability, such as digital cameras.

Encoding

Motion JPEG uses a lossy form of intraframe compression based on the discrete cosine transform (DCT). This mathematical operation converts each frame/field of the video source from the time domain into the frequency domain (aka transform domain.) A perceptual model based loosely on the human psychovisual system discards high-frequency information, i.e. sharp transitions in intensity, and color hue. In the transform domain, the process of reducing information is called quantization. In laymen's terms, quantization is a method for optimally reducing a large numberscale (with different occurrences of each number) into a smaller one, and the transform-domain is a convenient representation of the image because the high-frequency coefficients, which contribute less to the over picture than other coefficents, are characteristically small-values with high compressibility. The quantized coefficients are then sequenced and losslessly packed into the output bitstream. Nearly all software implementations of M-JPEG permit user control over the compression-ratio (as well as other optional parameters), allowing the user to tradeoff picture-quality for smaller filesize. In embedded applications (such as miniDV, which uses a similar DCT-compression scheme), the parameters are pre-selected and fixed for the application.

M-JPEG can be considered an alternate to MPEG-1 and MPEG-2, which are also intraframe-only compression schemes. (Neither of these compression methods use the more computationally intensive technique of interframe prediction. See also: I-frame) Whereas modern interframe video-codecs, such as H.264/MPEG-4 AVC, achieve real-world compression-ratios of 1:50 or better, M-JPEG's lack of interframe prediction limits its efficiency to 1:20 or lower, depending on the tolerance to spatial artifacting in the compressed output. Nevertheless, because frames are compressed independently of one another, and the codec's relatively low computational complexity, M-JPEG achieves a lower group-delay than is possible for any interframe video-format, whose nature necessitates the buffering of (sometimes large) groups of frames to operate on.

As a purely intraframe compression scheme, the image-quality of M-JPEG is directly a function of each video frame's static (spatial) complexity. Frames with large smooth-transitions or monotone surfaces compress well, and are more likely to hold their original detail with few visible compression artifacts. On the other hand, frames exhibiting complex textures, fine curves and lines (such as writing on a newspaper) are prone to exhibit the hallmark DCT-artifacts of ringing, smudging, and macroblocking. By extension, M-JPEG compressed-video is insensitive to motion-complexity, i.e. variation over time. It is neither hindered by highly random motion (such as the surface-water turbulence in a large waterfall), nor helped by the absence of motion (such as static landscape shot by tripod), which are two opposite extremes commonly used to test intraframe video-formats.

For QuickTime formats, Apple has defined two types of coding: MJPEG-A and MJPEG-B. MJPEG-B no longer retains valid JPEG Interchange Files within it, hence it is not possible to take a frame into a JPEG file without slightly modifying the headers.

Applications

Although the bitrate of Motion JPEG is substantially less than completely uncompressed video, it is substantially higher than that of video codecs which use inter-frame motion compensation such as MPEG-1. The large library of legacy software, low computational requirement, ease of editing, and widely available JPEG libraries, ensures M-JPEG content will be playable well into the future, even if the applications/equipment which created the content no longer exist.

Video capture and editing

M-JPEG is frequently used in non-linear video editing systems. Modern desktop CPUs are powerful enough to work with high-definition video, across the wide variation in graphics and operating-systems in use. Because it is a mature codec, needs no special hardware on modern PCs, and natively offers random-access to any frame, M-JPEG support is widespread in video-capture and editing equipment.

Game consoles

The PlayStation game console has integrated M-JPEG decompression hardware in order to play in-game FMV sequences. The Playstation Portable handheld game console can play M-JPEG from the Memory Stick Pro Duo under the .avi extension with a resolution of 480×272. It can also record clips in M-JPEG via its Go!Cam camera. Nintendo's Wii game console can play M-JPEG-encoded videos on SD card using its Photo Channel. The SanDisk Sansa e200 and the Zen V digital audio players play short M-JPEG videos.

Digital cameras

Prior to the recent rise in MPEG-4 encoding in consumer devices, a progressive scan form of M-JPEG also saw widespread use in the “movie” modes of Digital Still Cameras, allowing video encoding and playback through the integrated JPEG compression hardware with only a software modification. Again, the resultant quality is inferior compared to a similar sized MPEG, particularly as sound (when included) was often uncompressed PCM at a low sample rate or low-compression, low processor-demand ADPCM.

To compensate and keep file sizes and transfer rates under control, frame sizes and rates, along with sound sampling rates, were kept relatively low with very high levels of compression for each individual frame. Resolutions of 160×120 or 320×240 are common sizes, typically at 10, 12 or 15 frames/second, with picture quality equivalent to a JPEG setting of “50” with mono ADPCM sound sampled at ~8 kHz.

This results in a very basic, but still servicable video output at a similar storage cost to MPEG (~120 kB/s video rate, ~8 kB/s audio – or approx 1 Mbit/s at 320×240 resolution), but with minimal processing overheads. This video is typically stored in the popular Windows AVI or Apple QuickTime MOV container files, generally viewable natively (or after installation of a simple Codec) in most versions of the applicable operating system.

The AMV video format, common on cheap "MP4" players, is a modified version of M-JPEG.

In addition to portable players (which are mainly "consumers" of the video), many video-enabled digital cameras use M-JPEG for video-capture. For instance:

• in August 2008, Nikon announced the D90, the first D-SLR to record video. The format used is M-JPEG. The D90 uses three different motion JPEG formats: 320×216 pixels, 640×424 pixels and 1280×720 pixels.

• in June 2009, Pentax announced that the upcoming K7 camera will use M-JPEG in resolutions 640×416, 1280×720, and 1536×1024. The data rate for the M-JPEG files created can be up to 74 Mbit/s.

HDTV Media Players

In spite of the historical widespread use of M-JPEG for the “movie” modes of Digital Still Cameras, as of December 2009 there are no HDTV media players that support the codec.

IP Cameras

Many network-enabled cameras provide M-JPEG streams that network clients can connect to. Mozilla-based browsers have native support for viewing these M-JPEG streams.

Some network-enabled cameras provide their own M-JPEG interfaces as part of the normal feature set. For cameras that don't provide this feature natively, a server can be used to transcode the camera pictures into an M-JPEG stream and then provide that stream to other network clients.

M-JPEG over HTTP

HTTP streaming separates each image into individual HTTP replies on a specified marker. RTP streaming creates packets of a sequence of JPEG images that can be received by clients such as QuickTime or VLC.

The server software mentioned above streams the sequence of JPEGs over HTTP. A special mime-type content type multipart/x-mixed-replace;boundary= informs the browser to expect several parts as answer separated by a special boundary. This boundary is defined within the MIME-type. For M-JPEG streams the JPEG data is sent to the client with a correct HTTP-header. The TCP connection is not closed as long as the client wants to receive new frames and the server wants to provide new frames. Two basic implementations of such a server are test-server "cambozola" and webcam server "MJPG-Streamer".

Client software

Mozilla based browsers, such as Netscape and Firefox, have support for viewing M-JPEG streams. Google_Chrome and Opera stream M-JPEG natively. Other browsers, such as Internet Explorer can display M-JPEG streams with the help of external plugins. Cambozola is an applet that can show M-JPEG streams in Java-enabled browsers.

Digital videos

Digital Video (DV) adopts a similar method by compressing video frames individually.

Criticisms

• Unlike the video formats specified in international standards such as MPEG-2 and the format specified in the JPEG still-picture coding standard, there is no document that defines a single exact format that is universally recognized as a complete specification of “Motion JPEG” for use in all contexts. This raises compatibility concerns about file outputs from different manufacturers.
• JPEG is inefficient, using more bits to deliver equal quality, compared to more modern formats (such as JPEG 2000). Since the development of the original JPEG standard in the early 1990s, technology improvements have made intraframe compression possible. Technology improvements can be found in the designs of H.263v2 Annex I and MPEG-4 Part 2, which use frequency-domain prediction of transform coefficient values, and in H.264/MPEG-4 AVC, which uses spatial prediction and adaptive transform block size techniques and more sophisticated entropy coding than what was practical when the first JPEG design was developed. These new developments make M-JPEG appear outdated and inefficient

See also

• Motion JPEG 2000

External links

• MJPEG tools for recording from MJPEG capture cards and manipulating the data
• Apple QuickTime Format, including specification for MJPEG-A & MJPEG-B pp96
• RFC 2435 RTP Payload Format for JPEG-compressed Video
• Cambozola
• MJPG-Streamer