Picture, If You Will: Augmented Reality and Print

For the past several years, there have been attempts to “make print interactive” via what has come to be known as “Augmented Reality (AR).” The basic mechanism is some kind of code printed on or embedded in a printed image. That can either be a Quick Response (QR) code, some kind of invisible watermark, or advanced image recognition. The print is then scanned with a mobile device running an app that can read the code and thus access a website, launch a video, bring up 3D imagery or animation, access social media, or perform some other electronic action.

The past five years have seen some compelling examples that hint at the breadth of what is possible with Augmented Reality:

• Real estate signage features a QR code that, when scanned, takes you on a virtual tour of a home, condo, or apartment.
• The Graph Expo Show Daily (produced by your Printing News writers and editors) that features QR codes that, when scanned, launch a web page with video interviews.
• An elaborate hardcover graphic novel called Anomaly that has embedded AR triggers that launch additional content. (Anomaly 2 is coming in 2016.)
• The famous IKEA catalog that lets you use your mobile phone and the IKEA app to place furniture items virtually in your home.
• VMA’s Storyboard magazine, a print publication that, when the images accompanying a story are scanned with an app, launch a video related to the story.

Many other examples abound. AR is applied to print either to add additional content like so-called rich media (audio, video, animation) or even to glean data for marketing analytics purposes. Marketers found the advantage of email marketing to be that, when a potential customer clicked a web link in an email, the sender knew exactly where that customer was coming from, and could easily gauge the effectiveness—or the ROI—of a given email campaign. For years, the gripe was that you couldn’t do that with print. With AR, now you can.

Ultimately, it’s about getting print “networked.”

When you print something, said Ed Lee, Group Director of InfoTrends' Worldwide Consumer and Professional Imaging Services, “you take the digital file out of the ecosystem. What could you do with a piece of paper? Is there a way for that piece of paper to get back into the electronic world?”

A lot of AR development has been on the consumer side. Photobooks are one area where InfoTrends has been seeing a lot of activity, such as adding the ability to embed watermarks in images in printed photo albums that launch related video content. Think of the applications for wedding albums, for example. AR can also be used for educational purposes.

“Shutterfly does offer AR in their photobooks for audio captioning,” said Alan Bullock, Associate Director of InfoTrends’ Connected Imaging Trends (CIT). “They have a special app that supports that.” Called Photo Story for Classrooms, it allows students to create self-published books that have 30-second audio clips on each page.

Little Triggers

Augmented Reality, regardless of its specific implementation, requires three elements: the trigger, which is the mechanism on the printed page that contains the AR data or destination; the actual destination, rich media, or other content that the trigger triggers; and a reader, which is a mobile app that can scan the trigger.

Take the QR code. QR codes have proliferated because they are easy and essentially free to create. There are many free online QR generation tools, at least for QR-ing URLs, such as www.qr-code-generator.com. You simply paste in the destination website and it will generate a QR code. (Some now have for-pay subscription plans that offer print-resolution QR codes and other advanced features.) The QR code is then downloaded, and placed in a page layout like any other image. When it’s printed, the end user can read the code with any of a wide variety of—usually—free mobile apps, such as QRReader.

QR codes are more effective than you would think; industry sage Heidi Tolliver-Walker recently published a report that found that QR codes are accessed by around 30 percent of mobile users. A third of users is not a majority, but it’s still nothing to sneeze at.

The problem with QR codes is...well, just look at them. It’s in the eye of the beholder, but the general feeling, particularly among designers, is they’re not aesthetically pleasing and take up valuable design real estate.

“There are more subtle ways [than QR codes] of making a link,” said Bullock, “such as image recognition, where you train the system to look the image up, or with a digital watermark where an image is transformed subtly.”

“Nobody wants to put a QR code on the design because it disrupts the design experience,” said David Murphy, Worldwide Director of Marketing & Business Development, PageWide Web Press Division, at HP. HP has developed a digital watermarking technology for AR which it calls Link, which is designed to make print part of “the Internet of things.” (We’ll return to Link shortly.)

Regardless of the actual mechanism used for the AR trigger, there needs to be some kind of symbol or design element that indicates that the AR content exists.

Who AR You?

What are the solutions for transforming images and making them AR-able?

Metaio—and its corresponding reading app called Junaio—had been pioneers in the AR space. However, last year Metaio was bought by Apple, and no word has come out since, much to the consternation of Metaio users.

(If Apple does end up including some kind of AR solution in the iOS—and at press time this is purely speculative—it could solve one of AR’s big problems: the proliferation of mutually incompatible platforms and formats. QR codes have caught on because they’re a standard: any code reader can read any QR code. But with AR, you often have to juggle a bunch of apps. “We need standardization,” said InfoTrends’ Lee. “It’s the Wild West today.”)

Another pioneering name in the AR space is Layar, which was recently acquired by Blippar. (Read that last sentence aloud on Talk Like a Pirate Day.) The Layar website says, in bold capital letters, “Easily create your own interactive augmented reality experiences.” I took that as a challenge and tried it out. And you know what? They’re right! (See “Layar Cake” sidebar.)

HP has also been active in Augmented Reality. HP’s Aurasma platform has carved out a niche in the AR space, but in production print, the company is pushing its Link platform, which it will be showing at drupa. Link is part of HP’s strategy of, said Murphy, “merging physical and electronic content into a single medium. As all things in the world become part of the Internet of things, we believe printed materials should, as well.” Users embed invisible watermarks in images via Link Creation Studio which then can be read by the LinkReader mobile app. HP is targeting Link not just to content developers and end users, but also print providers, who can use it as a workflow tool to track jobs in production, much like they use barcodes today.

But the real action is in end-use products.

“We think Link can be applied to printed books, direct mail, corrugated packaging, transactional and transpromo invoices, and the like,” said Murphy. In packaging applications alone, he added, “it can increase user engagement after the purchase.” Think sweepstakes, giveaways, and other perks.

There is also one other advantage of Link. “This technology is as close to free as possible,” said Murphy. “We want to make this ubiquitous wherever there is print. We have this crazy fantasy—that we think will become a reality—that everything that is printed will be connected to the Internet.”

AR You Experienced?

AR—regardless of the specific implementation—is typically (to the extent that it is typical) part of the content creation process. “In the production print world, agencies are usually involved in AR development using whatever technology they decide to trigger the AR content,” said InfoTrends’ Bullock.

And the challenge isn’t necessarily the mechanics of adding AR but generating the actual rich media content that the AR triggers. If you want to launch a video, you have to shoot and edit the video. If you want to launch 3D animation, you need to create the 3D animation. Developing the rich media can be costly even if the AR development software is inexpensive or free. (Indeed, Storyboard magazine has discontinued using AR for cost reasons.)

It’s likely that AR is gong to be one of those “it takes a crowd to draw a crowd” technologies, and as more and more examples turn up—Lego and Hallmark have been experimenting with AR, the former using shape recognition to have Lego spaceships virtually “take off,” the latter creating interactive greeting cards—more designers and content creators will be interested in taking advantage of it.

Understanding how AR works, how to develop it, and whom to partner with, may be one of the best value-added services a print provider can offer.

Layar Cake

Inspired by a recent blogpost by Heidi Tolliver-Walker, I experimented with Layar’s Augmented Reality platform to test how easy or difficult it can be to add AR to print. To my surprise, it was pretty easy to use. More importantly, it worked!

Once you set up an account with Layar, you create a campaign through Layar Creator. This campaign can be a magazine, newspaper, print ad, direct mail, packaging, poster—pretty much the gamut of print applications. Once you select your campaign (I chose “print advertisement”), you upload an image and Layar analyzes it and stores what is essentially a map of that image. When the mobile app scans a printed or other electronic version of the image and looks it up in the Layar database, it’s recognized and triggers whatever action or content is associated with it. (If any image lacks enough detail—say, it’s an empty sky or a file that just contains text—it will be rejected.) For my experiment, I uploaded a PDF of the cover of my most recent book, and Layar Creator was happy.

Once the image is analyzed, you then “edit” it, which means you add what it is you want the AR to trigger. It can be a website, a video, a “like on Facebook” button, an e-mail, or a “Buy” button. I chose “website” since my AR trigger was going to send the user to the book’s Amazon page. In Edit mode, you essentially format the icon that the end viewer is going to see when they scan the page with the Layar app, as well as add the URL to which to send the end user.

Once that’s done, you test the AR by aiming your Layar app-enabled mobile device at the screen and scan the image. The app finds the AR trigger, and, when it’s tapped, it goes to the proper URL. If the test works properly (mine did), click the Publish button, and the campaign—i.e., the scanned image—is stored on Layar’s site.

Once the “image map” is scanned and stored, any time the Layar app comes across that image, it will trigger the AR. So I went and scanned the physical book cover with the app—success! Then I scanned the image on the Amazon page—success (if kind of redundant)! I created a quick InDesign one-page flyer for the book, imported an image of the cover, then printed it on my ancient desktop printer and scanned it—success!

As long as the image is not altered after Layar scans it, it can be added to any print or electronic application and anyone using the Layar app can trigger the AR.

How long did all the foregoing take? About 10 minutes. How much did it cost? Nothing, although I was limited to one campaign that is only active for 30 days. There are for-pay plans that offer more functionality and longer active time. —Richard Romano