Augmented Reality Eyewear

Augmented reality overlays the real world with a virtual world from the media, the World Wide Web, television and virtual worlds…. in real time. The overlay physically occurs on the eyewear lens, yet the images and data appear to sit about three metres away from your face within the perspective of your environment.

Augmented reality was first conceived as far back as the early ’60s when a motorbike simulator used a hint of the technology. Boeing picked it up in the ’80s foreseeing the benefits it would offer in terms of aeronautical maintenance, then; the concept went to labs to be developed for the military.

In the ’90s it started to take off. Contraptions the size of storm trooper helmets were developed to deliver the experience of augmented reality. The bulky size, weight and cost (approx. AUD$200,000) hardly made this a realistic option for the consumer.

..it will take decades before it’s finished, but the eventual impact of augmented reality will be at least as big and probably bigger than the impact of the net

From that point, the race was on to develop affordable, functional, stylish augmented reality eyewear.

Along the way, significant inroads have been made using smart phone technology which superimposes information on top of your view of the world through the device’s screen. For example, a residential real estate application enables smart phone users to stand on a street and, via their phone, receive advice and directions on properties for sale in the immediate area. For a real estate agent, it’s the perfect way to communicate immediate, relevant information to prospective buyers.

While smart phone applications are good fun, even useful, they have to be carried and the information is viewed in front of you on a screen, as you would a book or a file on a laptop. The prospect of Augmented Reality Eyewear presents a more immersive experience – like looking through a window with the images projected in real time through the optical lens.

Futurist Dr. Ian Pearson believes augmented reality visors will be one of the fastest growing products once they hit the market. “Augmented reality taps into a lot of basic human needs and wants. Not having it is really a bottleneck in development, so when it arrives in this form it will be very popular, at the right price.”

While Dr. Ian Pearson doesn’t not believe the market is ready to embrace augmented reality eyewear yet, he anticipates that in 18 months the time will be right.

“It will take decades before the technology completed, but the eventual impact of augmented reality will be at least as big and probably bigger than the impact of the net,” he said.

Video vs See Through Technology

There are two different modes of augmented reality glasses that require different technologies and deliver different levels of opaqueness on the overlayed image.

Video augmented reality glasses offer the experience of looking through a camcorder at a 60 inch television screen about three metres away from you. Surrounding light is blocked out and the contrasting colours are intense. It’s completely immersive and perfect if you’re watching a movie. This mode of augmented reality eyewear is already available on the market, however the frames are bulky and purpose built for gaming or movie viewing.

See-through augmented reality glasses provide a more ‘natural’ experience – information floats on top of the world as you see it through the glasses. The frames are similar to traditional frames and carry the technology in the temple zone. This version of the technology is intended for indoor or outdoor use which makes styling and weight more integral to development.

Explore Engage

In Australia, Explore Engage, is at the forefront of augmented reality eyewear development. The company has developed world leading technology that will make see-through augmented reality eyewear a realistic commodity for industry and consumers in the very near future.

Explore Engage is a privately owned business established just 12 months ago and located in Sydney. Already they’ve partnered with companies including Opera Australia, Clipsal, McDonalds, HSBC and Yahoo to create unique augmented reality applications for events and promotions.

Explore Engage plans to launch its augmented reality glasses to the mass market at a price point below $1000 within the next two years and expects that within three to five years, use of the technology will be widespread.

The glasses will have a stereo camera system built into the frame which locks on to visual patterns or clues and also enables them to approximate depth. These cameras are programmed to recognise these clues and to activate data or graphic images in response. These images are projected onto the lenses of the glasses and appear to float in real time, around the wearer. In addition ear plugs connected to the frames provide accompanying audio effects.

Distracting?

One of the most obvious questions is about how distracting images would be on your lenses as you’re walking around.

Surely, having data floating in front of your eyes and sounds ringing in your ears as you’re walking down the street, driving a car, or watching a sporting event will be incredibly distracting?

Paul Kouppas, Chief Technical Officer at Explore Engage, says “absolutely not”. He says augmented reality software is programmed to understand its surroundings which means it recognises when you’re driving and what digital information (if any at all) should be visible to the wearer. “In addition, as it can be activated by hand signals and gestures, it can be switched on or off as required with a simple wave of the hand,” he said

“Augmented reality is all about being able to understand the environment – it’s in real time and you can opt in opt out as you want,” he said.

Scott O’Brien, Chief Marketing Officer of Explore Engage says there is no other company in Australia that is at the same stage of development and only a few in the world.

“We’ve focussed on see-through augmented reality eyewear because we believe this is where there will be more consumer take up. However, see-through lenses are more technologically demanding in terms of operating parts and therefore more expensive,” said Mr. O’Brien.

“I do see the future as being the see-through model – there are far more applications – to be able to augment your surroundings – in fact almost any app designed for a smart phone can be developed for use with augmented reality (AR) eyewear.”

Dr. Pearson says “there are already some simple augmented reality apps on iPhones and other portables, so the very early stuff is already here but its peanuts compared to what can be done with visors. Expect to see semi-transparent AR visors from several suppliers entering the market in 18 to 24 months.”

How it Works

Augmented Reality Eyewear picks up signals as you move around – cameras in the frames or on the lenses recognise locations through GPS or lock on to visual clues, implement an immediate search for data then project it in real time via the lenses. The information can be delivered as text, images, graphics or even video.

Accelerometres, gyro sensors, and magnetometers follow the direction in which the wearer is looking and track objects the wearer observes, thus enabling the software to process and download relevant information then overlay the virtual data or imagery accurately on the real world. “The marriage of some, or all, of these technologies with what is visibly tracked by the camera dramatically improves the accuracy of the augmented experience,” said Mr. Kouppas. “The challenge is to not overload the computing processor, so that the software’s reaction time is compromised, thereby increasing latency between the overlaid digital image and what can visibly be seen of the ‘real’ environment.”

The difficulty is to achieve the greatest field of vision from the smallest amount of technology as it’s the technology relevant to field of vision that impacts the size or bulk of the eyewear and therefore affects the overall style.

“Micro displays built into the arms of the frames take the image or data and project it onto the lens. The image is projected somewhat horizontally and staggered into the wearer’s field of view. Alternatively, we can put the technology on the front lower part of the lens and, using mirrors and lenses, bounce it up and onto the lens. The latter option can achieve a larger 50 – 60 degree field of view. However this process makes the frames bulky,” said Mr. Kouppas.

See through augmented reality eyewear holds the technology in the frames at the temple. Explore Engage is now working to achieve the greatest field of view with the smallest frames and has approached the Sydney based eyewear designer Catherine Federici at EyeCandy Optics to contribute her expertise on frame styling.

While their current model, which fits the technology into a slightly larger than standard frame, only achieves 30 degrees, they are certain this field of view can be expanded to near 50 to 60 degrees.

Another challenge is how you make the data or images appear right in front of people without them getting a headache or cross focus. To overcome this, “we bend and adjust the light using optics and electronics so the perception is that the vision is way out there rather than being right in front of your eyes,” said Mr. Kouppas.

Data Intensive

While the potential of augmented reality glasses is infinite, currently there is just one limitation – the experience can be data intensive.

Mr. O’Brien says to use the glasses, you currently need to be plugged into a smart phone or a more capable processing device that can access information from relevant sites such as Wikipedia, Google, etc.

“Applications already exist that allow you to download relevant information to where you may be, or what you are seeing, such as point of interest applications. However, very soon we will see many more applications hit the market that allow for digital content to appear in your surroundings,” he said.

“I can see a time in the near future when augmented reality glasses will be completely wireless. In fact, in Australia, the National Information and Communications Technology Centre for Excellence, (NICTA) and CSIRO are developing Gigabit Wireless (GiFi) which will allow much greater wireless data transmission. In addition, new systems are being developed that will reduce the current error rates of GPS technology from a metre down to centimetres – whether you are indoors or outside,” he said.

The award winning world-first GiFi chip delivers short-range multi-gigabit data transfer indoors. This complete wireless system on a single integrated circuit provides around 10 times the bandwidth at one-tenth the cost of existing technologies.

Coupled with the speed of GiFi is the arrival of much more accurate satellite technology. The New Block IIF satellite technology launched by Boeing last year, in conjunction with the U.S. Air Force, is the first in a series of new satellites designed to overhaul the existing network that has been providing GPS data for nearly two decades.

The satellite has a host of enhanced features and functionality that will improve GPS signal strength, quality and accuracy. According to reports, the new satellites will have “two times greater predicted signal accuracy” predicted to improve overall accuracy from rough estimates of six metres to a tight radius of about half a metre.

The improvements will make a significant difference to location applications and therefore bring mass take-up of augmented reality eyewear much closer. By boosting the signal, the possibility of having GPS function in large indoor facilities, like shopping centres or convention centres is significantly increased. The ability for applications to provide more accurate information based on the location of the wearer will be finessed, and with enhanced visual recognition technology, AR could quickly evolve to provide the greatest detail about our surrounds – even in the most remote locations on the planet.

The Applications

The applications for augmented reality eyewear are infinite and span all aspects of work and life – from medicine and border security through to gaming, entertainment, tourism and, importantly, retail.

Augmented reality technology is already used in conjunction with haptic devices for medical training. Surgeons and nurses wear the glasses while they simulate a surgical procedure. Digital images of a body or organ are overlayed with data. The trainee uses their hands to simulate the operation, which causes the haptic device to give kinaesthetic feedback. The trainee experiences the physical sensations of cutting through tissue or working on organs.

Mr. O’Brien says that while the military is an obvious market for augmented reality, Explore Engage believes the biggest potential is consumers. “The military have the budget to commit to developing augmented reality eyewear but we’re more interested in taking this product to the mass market. In doing so, we can quickly create awareness, uptake and demonstrate the many enterprise solutions it offers,” he said.

Gaming is an industry Mr. O’Brien believes will be quick to take up the technology. “You can incorporate objects from games into your own environment – helicopters flying around the room, planes landing beside you or hologram football players running on a field the size of your lounge room floor.”

Tourism is another enormous opportunity. The augmented reality glasses will overlay whatever the tourist sees and provide information in the tourist’s own language, as and when they want it.

Theatre and sporting events are two of the most likely areas that Explore Engage will move into. “Augmented Reality Eyewear can provide audience members with information and special effects that both enhance their experience and build on their knowledge,” said Mr. O’Brien.

“So you might be watching a game of cricket and you’ll be able to track the players, find out about their past performance and so on. Or, in a massive production such as ‘Ben Hur’ that played at the ANZ Stadium in Sydney last year, the entire arena in front of you might appear to fill with special effects, you could see virtual images hurtling across the stage and around you. You really become part of the production.

“For years we have enjoyed movies with these 3D special effects and now with augmented reality used in live events, special effects can be enjoyed in real time. These effects overcome the problems of expensive props and occupational health and safety, and they create a sense of large scale, amazing productions. With augmented reality, you can open a whole new world.”

As for the visually impaired, according to Explore Engage, there could be an augmented reality product on the way sooner rather than later. Mr. O’Brien says that while the technology is not yet available for people who suffer visual impairment, it is a very real possibility. “Our lenses will be tailored to recognise the individual wearer’s requirements and programmed with prescriptive calibrations on the fly so they can experience all the special effects and information that every other wearer obtains,” he said.

Retail Applications

Explore Engage has already developed an application that enables consumers to use augmented reality to virtually try on jewellery. The customer uses hand signals to indicate the necklace or earrings they want to try on and instantly view themselves with the piece in place. They swipe their hand again to see the next piece.

In late 2008, Ray Ban created an Augmented Reality Virtual Mirror application in which users could download an application, install, and then see themselves augmented with a selection of sunnies using a webcam. Explore Engage aims to expand on this application with recent advancements in the company’s technology that will allow a customer’s facial structure to be analysed. The software will then suggest appropriate frames which can be augmented onto their face in real-time. The entire application could be ported to mobile Smartphone devices, and delivered to consumers via app stores.

“Optical stores could even have a kiosk at the window so that after hours, people walking past the store can use a hand gesture to swipe through the frames and try on glasses,” said Mr. Kouppas.

Mr. O’Brien says the virtual glasses they try on could also be overlaid with information about the brand, price, and optical qualities. “It could provide customers with product and pricing information as they try on glasses.”

Dr. Ian Pearson says the technology opens up enormous potential for retailers to use the information collected through loyalty programs. “The shopper will have a lot of information technology that knows all about them, and of course the store knows something about them too if they can recognise them from a loyalty card or face recognition. Putting all that information to good use, the shop… (can) guide the shopper to things the shopper wants and is most likely to buy. There is an obvious synergy. The shopper wants to spend and the shop wants them to. So they will both use the data in conjunction to the best effect.”

Dr. Pearson predicts that as shoppers walk through stores, they will be shown products, features, prices and services that are relevant to them. Special 3D effects will entertain and delight as they make their way through the store, making the experience totally immersive.

Importantly, he says, this new technology will help shop assistants to understand what their customers are looking for and when they want attention. In doing so, assistants will be able to offer a greater, more personalised level of service than ever before. It’s a ‘win win’ for everyone.

In the Future

Navigation: You’ll be able to follow the arrows overlaid on the street in the future as you self navigate your way to meet friends at a new cafe. Speeches: Stand in front of an audience and make a speech as the words appear like a ‘cue card’ right in front of your eyes on the inside of your lenses. Cooking: Follow virtual recipes and watch demonstrations as you cook in your own kitchen without turning away from the job in hand.

Vuzix Augmented Reality Goggles

The New York based company Vuzix launched their augmented reality goggles at last years Silmo in Paris. Small cameras are centred on the outside of each lens feeds continuous video through a smart phone or mobile computer to an LCD screen mounted inside each lens. At around USD$2,000, these glasses, which use the camera based augmented reality technology, are affordable compared to others that have come and gone over the years. However, in terms of being a fashion statement, more design is required!