How to Use Siri's New Visual Intelligence to Identify Anything on Your Screen or in the Real World

Siri is getting significantly smarter thanks to Apple Intelligence. The virtual assistant can now understand your personal context and see exactly what is on your screen. You can instantly pull up old flight details or identify a movie right from an Instagram reel. If you are running the public beta, there are dozens of new tricks to test out. Here is a step-by-step guide to using the most powerful new features.

Step 1: Identify what's on your screen

Siri can now "see" the media you are looking at. You might come across a video clip on social media and not know the title. You no longer have to dig through the comments to find out. Just ask Siri.

1. Open a video clip or photo on your device.
2. Ask Siri, "What TV show is this from?" or "What movie is this?"
3. Siri will identify the media and show you exactly where to stream it.

This works for real-world places too. You might see a restaurant recommended in an Instagram reel. Just say, "Get me directions to this restaurant." Siri will read the context clues, find the location, and open Apple Maps for you.

Step 2: Extract recipes and plan trips

You can stop manually copying and pasting text. Siri can digest on-screen information and format it perfectly into other apps. This is incredibly useful for saving recipes or organizing trip details.

1. Open a web page or social media post featuring a recipe.
2. Say, "Add these ingredients and steps for the recipe to an Apple note."
3. Siri creates the note instantly with all the text formatted for you.

You can even follow up by asking Siri to add those specific ingredients to your grocery list in Reminders.

If a recipe is too long to fit on your screen, you can take a couple of screenshots. Open the new Siri app and tap the + icon. Attach your screenshots and ask Siri to extract the text directly from those files.

Step 3: Tap into your personal context

Apple promised deep personal context for Siri. They finally delivered. Siri can dig through your photos, files, and calendar to answer incredibly specific questions about your life.

1. Ask Siri a vague question about a past event. Try saying, "What was that restaurant in Miami my family and I ate at back in March?"
2. Siri will search your data and provide the exact answer. It will even pull up photos from that specific meal.

You can easily ask follow-up questions. Say, "Did we see a movie there too?" Siri will scan your local files to find the answer. It can locate the exact PDF file of your Project Hail Mary 70mm IMAX tickets and bring it up on screen.

Step 4: Explore the real world with Visual Intelligence

Visual Intelligence lets Siri analyze the physical world through your camera. This is the perfect tool for traveling or shopping.

1. Open the Camera app.
2. Tap the Siri button at the bottom of your screen.
3. Point your camera at a landmark, a product, or a book.
4. Take a picture.

Siri will instantly tell you what you are looking at. You can use this to identify a product like an Anker Laptop Power Bank and get direct Amazon shopping links.

You can also use this feature for live translation. Point your camera at a foreign menu and tap the Translate button. Siri will overlay the English text right on top of the original words.

Step 5: Automate your schedule and smart home

Siri is much better at handling complex daily tasks. You can manage your calendar, update reminders, and control your smart home using natural language.

1. Ask Siri to check your day by saying, "What do I have going on today?"
2. Follow up by adjusting a task. Say, "Move the Primary Tech Daily reminder to 4:30 PM."
3. Siri will update the due date in the background immediately.

You can also ask Siri to create multi-stop routes. Ask for directions to Disney World, but tell Siri to make a stop at Starbucks first. Apple Maps will build the route with both stops included.

You can even create complex home automations without ever opening the Home app. Just say, "Create a home automation to lock all the doors at midnight."

Siri will automatically group your smart locks and schedule the routine. Your doors will secure themselves perfectly when the clock strikes twelve.

You're all set

Siri is finally becoming the true personal assistant we always wanted. These new AI features make your phone infinitely more useful. Try out the on-screen awareness next time you browse social media. Go test out Visual Intelligence on your next neighborhood walk. Your iPhone is capable of a lot more today.