As if walking a robot dog wasn’t strange enough, Boston Dynamics’ spot can now take visitors on a walk as it takes on the role of an English tour guide.
In a new video, the engineering firm demonstrated Spot’s new ability to answer questions and crack jokes using a range of accents as well as several different personalities.
The robot, decorated with little hats and googly eyes, takes guests to different locations and describes what it sees.
As he opens and closes his gripper to mimic a mouth and turns to look at people, Spot’s performance comes impressively close to that of a real leader.
Powered by ChatGPT, Spot’s creators say they were surprised by some of the unexpected responses the robot dog produced.
Spot the robot dog now has the ability to play tour guide, answer questions and describe the world around him using ChatGPT
What is ChatGPT?
ChatGPT is an AI chatbot developed by Open AI in November 2022.
When it was released, it was the fastest app in history to reach 100 million users.
It uses a specially trained Large Language Model to predict what should follow in text strings.
This allows the AI to respond to natural language prompts and help with a variety of tasks.
ChatGPT has been used with varying degrees of success for everything from coding to legal brief writing
The footage shows the $75,000 (£61,857) robot taking on the personality of a “chic butler” and says: “My linguistics have been carefully crafted to provide an authentic British experience.”
When senior software developer Matt Klingensmith asks if he enjoys his work, the robot replies, “Ah, Mr. Matt, my job as a tour guide is a great satisfaction.”
“I find the dissemination of knowledge quite rewarding, don’t you agree?”
However, this is far from the only character that the robot dog plays, as it takes on the voices and characteristics of a “precious metal cowgirl”, an “excited tour guide” or a “nature documentary”.
The robot also demonstrates an impressive ability to recognize and respond to objects in the world.
While taking on the role of a “1920s archaeologist,” Spot refers to the camera crew, calling them “a fellow explorer with a camera.”
One of the most surprising characters is “Josh,” a sarcastic and moody robot handler who Mr. Klingensmith says was “an experience I’ve never had with a robot in my entire life.”
When asked for a haiku about the space, the Josh personality replies, “The generator hums quietly in a space devoid of joy, much like my soul, Matt.”
However, all of these different features and personalities are the product of simple adaptations of the same code.
Using a variety of prompts, the large language model behind Spot is able to create a full spectrum of personalities from “Fancy Butler” to “Josh.”
Boston Dynamics provides the AI with this map of the offices, including some labeled locations and some short descriptions
The team provided ChatGPT with a carefully designed prompt that converts visual and audio information into speech and commands for the robot.
Prompts include information such as “Use the Tour Guide API to guide guests through a building using a robot.” Tell guests about what you see and make up interesting stories about it.”
They add context like: “Personality: “You are a snarky, sarcastic robot who is unhelpful””.
The model then receives a plan of the building with some labeled rooms and short descriptions.
However, the AI is given just enough information to get started.
Its creators say: “The LLM can be thought of as an improv actor – we provide a rough script and the LLM fills in the gaps on the fly.”
To convert visual information into text that ChatGPT can use, Spot uses a program called the Visual Question Answering model that labels images from the robot’s cameras.
With all this information, ChatGPT 4 can command the robot to move around its environment, respond to questions, and make comments about its surroundings.
Some of the behaviors Spot was able to generate with ChatGPT at its core surprised even its developers.
“Josh,” the sarcastic tour guide, was one of the most surprising characters and a very different experience than most previous robot-human interactions
This diagram shows how information in the real world is converted into prompts that the AI can use to control the robot’s speech and movement
In a blog post, Mr. Klingensmith and his team point out that large language models (LLMs) like ChatGPT tend to produce “emergent behavior” that is not predicted by the code.
In one example, the team asked Spot who his parents were, and the robot led them to the display of older Spot models, saying that these were his “eldest.”
Likewise to the question “Who is Marc Raibert?” The robot replied: “I don’t know.” Let’s go to the IT helpdesk and ask.”
No parental concepts or a command to ask for help were programmed in, although the team insists this is not evidence the robot is thinking.
“To be clear, these anecdotes do not suggest that the LLM is conscious or even intelligent in the human sense – they simply demonstrate the power of statistical association,” the team writes.
“But the noise and mirrors the LLM puts up to appear intelligent can be quite convincing,” they added.
Boston Dynamics added (1) a navigation system, (2) a microphone, (3) a speaker, and (4) a gripper arm and camera
This is not the first role that Boston Dynamics has proposed for Spot, as the company has outfitted the robot for various purposes.
Spot engineer Zack Jackowski previously said that the next big application of the technology would be the development of robotic guards that can patrol industrial facilities and factories.
Mr. Jackowski suggests that Spot could autonomously walk through locations and collect data with its sensors to detect problems such as open doors or fire hazards.
Spot has been used to inspect nuclear power plants, oil rigs, construction sites and even has an eye on the ruins of Pompeii.
Boston Dynamics’ other robots have also set the bar for bipedal mobility, with videos showing Atlas jumping and performing parkour with ease.
A recent video showed how Atlas could help on construction sites by transporting bags of tools and lumber up and down scaffolding.
WHAT IS THE BOSTON DYNAMICS SPOT MINI ROBO-DOG?
Boston Dynamics first introduced Spot, the most advanced robot dog ever, in a video released in November 2017.
The company, best known for its 1.7 meter tall humanoid robot Atlas, has unveiled a new “lightweight” version of its Spot robot.
The robot dog was shown trotting around a yard with a promise that more information would be coming “soon” from the notoriously secretive company.
“Spot is a small four-legged robot that fits comfortably in an office or home,” the company says on its website.
It weighs 25 kg (55 lb), or 30 kg (66 lb) if you include the robot arm.
Spot is all-electric and can run for about 90 minutes on a charge, depending on what it’s doing, the company says, boasting, “Spot is the quietest robot we’ve built.”
Spot was first introduced in 2016 and an earlier version of the mini version of Spot with a strange extendable neck was already shown as a household helper.
In the company’s previous video, the robot can be seen leaving the company’s headquarters and seemingly entering a home.
There he helps load a dishwasher and carries a trash can to the trash can.
At some point it also encounters a fallen banana peel and falls dramatically, but uses its extendable neck to push itself back up.
“Spot is one of the quietest robots we have ever built due to its electric motors, the company says.”
“It has a variety of sensors, including depth cameras, a solid state gyro (IMU), and proprioception sensors in the limbs.”
“These sensors help with navigation and mobile manipulation.”
“Spot performs some tasks autonomously, but often uses a human for overall leadership.”