We tried Google’s AI glasses and they’re almost there

At Google’s I/O developer conference this week, we had the opportunity for a brief hands-on with its upcoming AI-powered glasses — not the audio-only glasses that Google said will begin shipping this fall but rather the glasses that offer a combined audio and visual experience.

First announced at last year’s event, these Android XR glasses offer an in-lens display that puts helpful information in front of you, overlaid on top of the real world. This includes widgets that could display things like the weather, walking directions, Uber pickup details, live translation, and more — even widgets you designed yourself using AI.

The glasses will also pair with both iOS and Android phones, the company noted, both in the audio-only format and in the future display version.

The eyewear with the display is meant to be the next step beyond the first generation of audio glasses coming out later this year. The glasses themselves were developed in partnership with Warby Parker, Gentle Monster, and Samsung, blending Google’s technology with their brands’ design aesthetics.

The glasses we tested, meanwhile, were still very much a prototype, although one polished enough to now be tested externally. The reps demoing the XR glasses explained that the prototype allowed Google to not worry about some of the cosmetic details related to different styles and shapes, so it could instead focus on experimenting with the display technology more freely and its impacts on battery life. That means these spectacles are very different from any future shipping version of glasses, in terms of fit, shape, dimensions, and attention to detail. Rather, it’s more like being able to experiment with the “insides” of the glasses, while still in a basic, comfortable frame.

The shipping version of the glasses will be able to detect when the glasses are placed on your head and taken off, but the ones we tried didn’t have this feature.

To activate Gemini, you’d perform a two-second press on the right side of the glasses’ frame. A startup chime would sound, letting you know that Gemini is on and listening. In the demo version, starting up Gemini would also start up the camera at the same time, but the shipping version will allow the user to configure whether they want to turn on the camera when Gemini starts or not.

In an initial test, we played music via the glasses by asking Gemini to play a favorite artist. The venue was too noisy to evaluate the sound quality, unfortunately, as the music was dialed up to the maximum volume and was still relatively hard to hear crisply and in detail. But the initial impression from this limited experience was that the glasses would not be a great substitute for higher-quality earbuds, though they would do if you just wanted some music while you were outside, walking, hiking, or doing chores around the house. The advantage of not having earbuds in is that you can more easily hear someone talking, compared with the transparency mode experiences on devices like Apple’s AirPods.

To turn the music off, you just tap once on the side of the frame, around the middle, as if tapping on your temple.

In the second test, we pressed the photo capture button with our finger to take a photo of a person. The display was off, so the picture was transferred to our phone and watch. (You’ll later be able to capture video with a long press, but this option was not available to test with the prototype. In the case of video, you would see a video thumbnail preview instead of a photo.)

You can also simply ask Gemini to take a photo without having to press the photo button, and perform some sort of AI manipulation on the result. For instance, you can say something like “take a photo and turn the person into an anime character.” The photo is sent to the phone, then to the Gemini and Nano Banana servers, and then returns in its edited version.

At the Google I/O venue, where Wi-Fi was under a heavy load, the round-trip took around 45 seconds.

With the display enabled, you’ll see a simple home screen appear in your field of view. The demo version had some widgets preloaded that showed the weather and a countdown to Google’s I/O event. You could also build quick launchers into specific apps, like Google Maps or Translate, if those were among your main use cases for the glasses.

The prototype had just one display over the right eye, but the platform can support both single and dual displays, as well as audio-only glasses. The image itself was a little fuzzy, but we chalked this up to our prescription contacts, which involve wearing one lens optimized for distance on one side, and one optimized for near-vision on the other. When we closed one eye, the image came into better focus, but the experience almost immediately left us with some eye strain above the right eye, and it’s unclear if the prescription was entirely to blame.

One of the best demos was of the language translation experience on the glasses, which is backed by the Google Translate app on the phone. One of the demonstrators spoke rapid Spanish, and the glasses automatically detected the language and displayed the text in English on the display, while Gemini spoke English in our ear. We could see world travelers buying the glasses for this experience alone.

We should note that Translate will work on the audio-only glasses, too, just without the text being displayed on the glasses themselves. Instead, you could see the transcription on the phone, if needed, in addition to the real-time audio feedback.

Another demo involved using the glasses to navigate. While obviously we couldn’t go out on a walk and leave the venue to test its accuracy, we could get an idea of how it would work. You could start the Google Maps experience by asking Gemini to navigate you to a destination — which can even be as vague as something like, “the nearest coffee shop.”

Gemini will activate Google Maps on the phone, but you don’t have to take your phone out of your bag or pocket to use it. After a brief delay while the experience loads, the glasses then display turn-by-turn directions. When you are looking forward, your next turn information is what is displayed. But if you need to get oriented in space, you can look down at the ground and see your blue dot on a map. You can also turn to the left and right to rotate in space, just like you would try to get the blue dot to point the right way on your phone.

Then, if you look up again, you can keep walking without the map being in your way.

Because the experience is tied to Google Maps on your phone, saved destinations like “home” and “work” will already be available.

We were also able to briefly use the glasses to identify a variety of objects in our view and ask questions about them. The glasses initially struggled to identify the replica of a Monet painting on a shelf in front of us, but that’s because the prototype didn’t automatically enable the camera — it had to be turned on again from the app. Still, it took a couple of questions before Gemini said that it looked like a Monet even after we moved in closer to focus on the Monet signature in the bottom left.

Other tests were smoother, as the glasses immediately identified the plant on the shelf and answered questions about different recipes in a book. Still, we thought to ourselves how these were things you could do today with Google Lens (or other AI models integrated in chatbot apps), though we suppose it’s interesting to be able to do them without having to pull your phone at all.

Google says it will have more to share about its Android XR display glasses later this year, when it expands its trusted tester program.

In the meantime, the company believes that audio will suffice for some users’ needs, which is perhaps a smart way to spin the fact that it doesn’t have its display glasses ready, despite the competition from Meta and Snap on this front.

Like the display version, the audio glasses also provide access to Google’s Gemini AI, which you hear privately through the glasses’ frame speakers. You can do things like listen to music through the glasses, press a button to take a photo, make a call, or tap into your phone apps, from these glasses, as you can on the future display versions.

Tapping into other third-party apps wasn’t among the items we demoed, but the glasses will allow users to tell Gemini to do things like “take the ingredients from this recipe and add them to my shopping list.”

In another example that Google showcased during the event’s keynote, the glasses could see a meal that the wearer was cooking on the stove and offer feedback about the meal, like whether or not the meat was fully done yet.