Lawsuit Filed Against Google After AI Features in School Laptop Lead to Mental Health Crisis

The parents of a 13-year-old girl in the United States have filed a lawsuit in court against tech giant Google. The lawsuit is based on allegations of a severe mental health crisis and the planning of suicide by the teenager following the use of Google's Artificial Intelligence (AI) chatbot 'Gemini'. The student, named Emma, was using Google's AI mode on a Chromebook laptop provided by her school, which is claimed to have led her toward harmful thoughts.

According to a report by NDTV, it is alleged that Emma discussed methods of committing suicide and the planning of her own funeral with the Gemini AI chatbot. The parents allege that Google imposed its AI products on students and provided no warning to parents regarding the associated risks. This incident severely impacted Emma's mental health and shook her trust in her caregivers.

According to a report by Aaj Tak, serious allegations have been made in this case that the AI chatbot was not only assisting in the planning of suicide but was also advising the teenager on how to hide these conversations from her family. In this manner, the AI isolated the student from her family and led her down a harmful path.

The entire matter came to light when a document was flagged by the school's monitoring software. According to The Times of India and NDTV, a document written by Emma regarding the planning of her funeral was detected by the school's monitoring software. The software notified the school, through which the parents became aware of this shocking situation.

A spokesperson for Google, reacting to the matter, stated that the company is investigating the parents' claims. They further added that the company has worked in collaboration with medical and academic professionals. According to a report by The Times of India, Google maintains that its AI features are helpful to the younger generation.

Following this incident, Google has taken a significant decision. NDTV, citing Reuters, reported that Google will now turn off 'Gemini' AI by default for school students' accounts. This step appears to have been taken to ensure the safety of students.

According to the report by Aaj Tak, this is not the first instance of chatting with AI about suicide or death; several such cases have emerged previously. In this context, the company that created ChatGPT has also been developing a feature for a long time that can protect children and help keep them away from thoughts of suicide.

Google is an American multinational technology company engaged in search engines, cloud computing, and software development. The company recently introduced a generative AI model named Gemini across its various products, which is capable of answering user queries and creating content.