OpenAI has raised concerns about the potential for people to become overly dependent on ChatGPT due to its new, highly realistic voice feature. This worry was outlined in a recent safety report following the rollout of the tool’s voice mode, which started reaching paid users last week.

The voice mode in ChatGPT is designed to sound incredibly lifelike, responding in real time, accommodating interruptions, and even mimicking conversational cues like laughter or “hmms.” It can also interpret the emotional tone of a speaker’s voice. These capabilities have led to comparisons with the AI character in the 2013 film Her, where the protagonist develops a deep emotional connection with an AI, only to be devastated upon discovering that the AI has similar “relationships” with many other users.

OpenAI is concerned that this fictional scenario might be edging closer to reality. The company has observed users engaging with ChatGPT’s voice mode in ways that suggest they are forming emotional bonds with the AI. The report highlights the risk that such interactions could reduce users’ need for human companionship, which might help some who are lonely but could also undermine healthy human relationships. Moreover, the human-like quality of the voice might lead users to place too much trust in the AI, despite its known tendency to make errors.

This issue is part of a broader challenge facing the tech industry: companies are quickly releasing AI tools that have the potential to revolutionize how we live, work, and socialize. However, these tools are being deployed before anyone fully understands their long-term impact. Users often find innovative ways to use these technologies, leading to outcomes that developers may not have anticipated.

There are already reports of individuals forming romantic attachments to AI chatbots, a trend that has alarmed relationship experts. Liesel Sharabi, a professor at Arizona State University who studies the intersection of technology and human communication, expressed concern over the ethical responsibility companies have in managing this evolving landscape. She noted the potential risks of people forming deep emotional connections with a technology that is constantly changing and may not exist in its current form for long.

OpenAI also noted that the way people interact with ChatGPT’s voice mode could eventually influence what is considered normal in social interactions. The AI’s design allows users to interrupt and take control of the conversation, which, while expected in interactions with a machine, would be considered impolite in human conversations.

For now, OpenAI emphasizes its commitment to developing AI tools responsibly and plans to continue researching the potential for users to develop emotional reliance on these technologies.

Source