Several students at Westfield High School are under investigation after being accused of using AI to generate fake pornographic material depicting fellow classmates. The scandal demonstrates some of the potential dangers of AI becoming too advanced.
The Westfield High School principal recently sent an email to parents, cautioning them about the risks of artificial intelligence and informing them that an investigation had been initiated in response to students’ complaints.
The email read as follows:
“I am writing to make you aware of a situation that resulted in widespread misinformation and resulted in significant worry and concern amongst the student body of Westfield High School. Earlier today, students brought to our attention that some of our students had used Artificial Intelligence to create pornographic images from original photos. There was a great deal of concern about who had images created of them and if they were shared. At this time, we believe that any created images have been deleted and are not being circulated. This is a very serious incident. We are continuing to investigate and will inform individuals and families of students involved once the investigation is complete. This will happen before the weekend. We made counseling available for all affected students and encouraged them to return to class when they felt able to do so. Additionally, our School Resource Officer and the Westfield PD have been made aware of our investigation. If a parent/guardian thinks their child is a victim of a criminal act in relation to this incident please report the matter to Westfield Police.
I wanted to make you aware of the situation, as, in addition to harming the students involved and disrupting the school day, it is critically important to talk with your children about their use of technology and what they are posting, saving and sharing on social media. New technologies have made it possible to falsify images and students need to know the impact and damage those actions can cause to others.
We will continue to educate your children on the importance of responsible use of technology and hope you reinforce these messages at home.”
According to some Westfield parents, their kids were targeted by students weaponizing generative AI to create what’s known as deep fakes—or fake images that look remarkably real. The fakes often use real pictures of people’s faces that can be found on social media and combine it with generative pornographic images created by a simple AI prompt.
As AI becomes more advanced and easier to use for the average person, the risk of it being used for nefarious purposes increases. Deep fakes of celebrities were a popular scandal last year, and as the technology makes its way into schools, even more sinister possibilities are revealed.
Congress is considering introducing a bill that would make the non-consensual sharing of AI-generated deep fakes illegal. Representative Joe Morelle—a Democrat from New York—is authoring the bill.
As AI continues to improve, its capabilities prove to be very useful. However, the technology does not come without its downsides and this AI high school scandal is just one example of many such negatives.
