Ten wild ways people are using ChatGPT’s new Vision feature

The internet is raving about ChatGPT’s new Vision feature.

The latest updates to the OpenAI tool were rolled out earlier this week. They allow ChatGPT to “see” when users upload images, which they can then discuss with the chatbot. Additional “Listen” and “Speak” features allow users to have conversations with ChatGPT.

The new artificial intelligence (AI) capabilities leverage GPT-3.5 and GPT-4, which “apply their linguistic reasoning capabilities to a wide range of images, such as photos, screenshots, and documents that contain both text and images.” , it’s called a Monday OpenAI blog post.

People around the world have started trying out the new features and sharing their experiences on social media. Below are 10 creative ways ChatGPT users are taking advantage of this new Vision feature.

Identify movie scenes

On X, formerly Twitter, some users alerted their followers that they could upload a screenshot of a movie and have ChatGPT identify the movie. In an example posted by @skalskip92ChatGPT identified pulp Fiction from a screenshot showing actors John Travolta and Samuel Jackson. ChatGPT also shared information about the film’s historical context and, upon request, its rating on IMDB.

writer Peter Yang also tested this feature with a screenshot from the 2000 Ridley Scott film gladiator.

Do homework for children

AI developer McKay Wrigley posted a video on X showing how ChatGPT can now explain scientific diagrams to students. In Wrigley’s example, he posted a diagram showing the inside of a human cell and asked for help understanding each component. ChatGPT came back with short descriptions for each cell part.

Yang also tested this tutor-like feature by sending ChatGPT an image of an addition worksheet for a math class. ChatGPT provided answers to all math problems contained on the sheet.

“Kids will never do homework again,” Yang tweeted.

Offer coaching tips

Co-founder of Create Labs Abran Maldonado tweeted that he provided ChatGPT with two photos taken during a football game “in honor of football season.” ChatGPT then explained what appeared to be happening in each photo and offered six quarterback coaching tips. Maldonado predicted that the new Vision feature will “change coaching and sports analysis forever.”

Write code

Users have also discovered that ChatGPT can write code based on uploaded images, charts, and graphs. In one example, Wrigley on X shared a photo of diagrams drawn on a whiteboard, which ChatGPT then converted into code.

Several X users shared another video showing how ChatGPT created a website with a design that resembles a sketch drawn on paper. A photo of it was then uploaded for the chatbot to assess.

Fit a bike seat

ChatGPT can guide users through instructions for various random activities, including adjusting a bicycle seat. In an example shared by OpenAI, a user having difficulty lowering the seat of their bike could take a photo of the bike and follow step-by-step instructions that walk them through the necessary adjustments. According to OpenAI’s video, users can ask follow-up questions and send additional images to walk through specific steps if they encounter problems. The vision feature can be used to repair other random items around the house, OpenAI’s blog post said.

Take better photos

Ethan Mollick, a professor who studies the impact of AI on education, said on X that ChatGPT’s vision feature can help users create better photos. Mollick uploaded a photo to ChatGPT and asked for specific instructions on how to improve the image. The response he received showed that ChatGPT gave tips on framing, lighting, perspective, and more.

Pietro Schiranowhose X-bio says he works in AI, posted on ChatGPT describing the space’s design elements and explaining the historical context for his name suggestion.

Avoid traffic tickets

Yang tweeted: “I will never get a traffic ticket again“Now ChatGPT’s vision feature is available. Yang posted a photo of a sign that had several specific parking instructions posted on it, each giving different instructions about when people could and could not park in that area. Yang told ChatGPT a specific time and day of the week, asking if parking was safe.

Analyze works of art

ChatGPT tapped Art analysis when Schirano asked about the meaning of a four-panel cartoon. ChatGPT’s analysis broke the cartoon down by group and ultimately provided an overall rating of its importance.

Decipher handwritten notes

ChatGPT can be used to read messy or flowery handwriting styles. In an example shared by Mollick on X, a photo of part of a handwritten manuscript was uploaded to ChatGPT for decryption. According to Mollick, the chatbot performed quite well.

“Will likely be a big deal for a number of academic fields, particularly as AI can ‘reason’ about text.” he tweeted.

Find Waldo

Perhaps most importantly, ChatGPT can help children (and adults) around the world find Waldo.

“I found him!” ChatGPT replied to a Where’s Waldo? Page uploaded by Schirano with instructions on where to look.