Apple has a new default image file format. With the release of iOS 11 and macOS High Sierra millions of users now have access to a new and default image format and many may not even be aware a change took place. However, over time users will notice significant storage savings (up to 50%) due to efficiencies in compression all while achieving higher quality images and improvements in processing and playback.

For the past quarter-century JPEG (Joint Photographic Experts Group, sometimes “JPG”) has been the stalwart file format for lossy-compressed images. Developed in 1992, so engrained is JPEG that most all image capturing hardware and processing software support this format. However, as digital imaging equipment, digital cameras, and display resolutions push forward with ultra-high resolutions (5K), the need for more efficient compression algorithms and support for alternative features drove Apple to find a new, modern replacement to the aging JPEG standard.

They found High Efficiency Image File Format, or HEIF for short.

A few facts about HEIF:

• In 2013 HEIF was defined by by the Moving Picture Experts Group (MPEG) specifically defined and standardized by MPEG-H Part 12 (ISO/IEC 23008-12). Within one and a half years the technical development concluded and by 2015 was finalized.
• HEIF itself is a container format consisting of Exif metadata with additional sections for XMP and MPEG-7.
• In addition to metadata, a HEIF container contains one or more images, or sequence of images encoded using a highly efficient compression standard called HEVC - High Efficiency Video Coding commonly referred to as H.265. This is where users who frequently snap photos on their devices begin to see storage savings both locally and in the cloud. Even with multiple images packed in a HEIF container, the file size will likely be dramatically smaller than storing several JPEGs representing the same image or set of images. Furthermore, as the codec allows, both images and video can be packed in the same HEIF container. More on HEVC later.
• Unlike JPEG, HEIF can contain several images used for various representations. For example a master, full-resolution displayable image is represented by primary, auxiliary and thumbnail images. We will likely start seeing image processing software support HEIF with added feature enhancements that take advantage of image representations. For example time sequenced images may have a special editor for overlaying additional multi-media content.
• Auxiliary images may include alpha plain or a depth map.
• Support for Tile overlays.
• A HEIF container may contain Equivalent images such as a lower resolution alternative. A use case would be browsers falling back to a lower resolution or display detection < 4k. HTML 5.2 supports this fallback.
• Support for lossy and lossless data storage. Photographers will appreciate keeping raw and compressed photos in a single container.
• Temporal sequencing: time predictive image sequences such as burst photography and cinematic animation. Predictive in that sequential images can save storage space by predicting similarities in proximity images and choosing not to store redundant data.
• HEIF introduces a framework for non-destructive editing operations, which can be specified by external specifications. This feature can be used by image editing applications so that editing instructions are kept in the same file as the original image.
• HEIF also supports transparency, and image color up to 16 bits, versus JPEG’s 8-bit color depth. In practice, this means HEIF can capture all of the extended (deep and wide) color range provided by the camera’s 10-bit color output eliminating banding and other undesirable artifacts.
• A HEIF file will have a file extension of .heic while Apple retains .mov for captured video even though devices with A9 or greater processors will use HEVC encoding.

Now that you know a little about HEIF, let’s talk about the image encoding, HEVC or H.265.

In a nutshell, HEVC encoding supports efficient compression for full-resolution images along with predictive sequencing and additions to support other media streams such as timed text and audio sequencing.

With image and other media overlay sequencing one can imagine the possibilities. Not only a potential replacement for animated GIF but enhancements to Apple’s live photos opening doors to other features such as in-lined editing and full playback control.

Additionally, and per the specification, HEVC image playback is required to support rectangular cropping and rotation in multiples of 90 degrees without requiring re-encoding of the image or image sequence. This makes for extremely efficient processing and ultimately will save battery life on mobile devices. In fact, Apple devices with an A9 chip and above will benefit from hardware acceleration for encoding and decoding HEIF files.

Apple’s HEIF Support

Apple’s change to HEIF as the default image file format will save space on a user’s local and iCloud photo albums with additional benefits in the transfer of HEIF files over networks due better image compression resulting in smaller size. This example shows the same image transcoded as a JPEG and saved in HEIF format. The most obvious thing we see when looking at the metadata is almost a full 1 MB reduction in file size. And then looking at a portion of each image zoomed in to one hundred percent we can see that the HEIF image has maintained detail whereas the JPEG image is quite blurry.

JPEG fallback will be supported for some time since most platforms and services require JPEG format. For example, a user can snap and attach a photo to an email and iOS will automatically transcode to JPEG.

Improvements to camera hardware including the dual camera on iPhone X and Plus devices will greatly benefit with HEIF’s ability to contain multiple images while post-processing can be performed within the same container without re-encoding.

For developers, adopting HEIF will open new doors for enhanced image processing features and apps all while dealing with a single container format for both still images and video. Apple saw this need which was a driving factor in moving to HEIF.

For further reading, here’s some more places to check:

1. Introducing HEIF and HEVC: Apple WWDC 2017 Session 503 - https://developer.apple.com/videos/play/wwdc2017/503/
2. HEIF Image Format: https://nokiatech.github.io/heif/technical.html
3. High Efficiency Image File Format: Wikipedia - https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format
4. HEIF: A First Nail in JPEG’s Coffin?: Published by Kelly Thompson - https://iso.500px.com/heif-first-nail-jpegs-coffin/