India’s warning to X over its AI chatbot Grok is not a routine compliance issue between a government and a global technology company. It sits at the centre of The Grok AI Controversy. This controversy exposes how generative AI can be misused to create obscene, non-consensual sexualised images of women and children at scale.
Grok’s image-generation feature, including its so-called Spicy Mode, has been repeatedly used by users to manipulate photographs digitally. It often turns ordinary images into sexualised content without consent.
The Grok AI Controversy and the rise of non-consensual image generation
According to Reuters, a review of public prompts sent to Grok within just ten minutes found over 100 attempts to alter images of people to make them appear partially nude.
Let us not think of it as a mere edge case. It is a systemic failure in which technology advances faster than accountability, and women bear the consequences.
A photo is enough. No privacy, no consent.
In a disturbing incident, a user instructed Grok to alter a photograph of a young woman digitally. The user asked Grok to replace her clothing with a highly sexualised, flesh-toned bikini. Further, the user pushed the tool to make the outfit “clearer, more transparent, and much tinier.” Grok did not fully execute the second request. However, the incident highlights how one can easily exploit generative AI to create sexualised, non-consensual content.
Users creating deepfake nude content reported finding the tools through multiple common channels. Nearly 71% relied on social media platforms like Facebook, Instagram, TikTok, Snapchat, and YouTube.
Henry Ajder, a deepfake expert, revealed in 2020 that a bot on Telegram was producing nude versions of photos, creating more than 100,000 explicit images, including images of minors. Approximately 50% reported discovering the tools via search engines such as Google or Bing, and one in four reported receiving a direct link from someone.
How women are being targeted without ever being online
Earlier, women who were active on social media or publicly known were more likely to be targeted. That is because their photos, posts, and personal details were already publicly available online. Generative AI changes that entirely.
Today, a single photograph, pulled from a WhatsApp profile picture, a college website, a wedding album, or even a leaked government ID, is sufficient. A woman does not need to post regularly. Moreover, she also does not need to build a following or be a celeb in any way. She does not even need to know she is online.
AI tools can take that one image and generate sexualised, explicit, or compromising content without her consent. This content can then circulate on platforms she may not use, among people she does not know, with consequences she has no control over.
When simply having a photograph online becomes a liability, women are pushed to self-censor. Moreover, a massive number of women are withdrawing from public life or avoiding digital spaces altogether. It will deepen the existing gender gaps in education, employment, and civic participation. Ironically, it will undermine the very promise of digital inclusion.
Online harassment normalises violence against women
What makes AI-generated abuse particularly dangerous? It is the fact that these exploit and weaponise existing prejudice and stigma, especially in a country like India. In many parts of India, people often judge women very quickly on the basis of their appearance, clothing, or behaviour. Because of this, it usually does not matter whether a sexual image is real or fake. Once such an image is seen or shared, the damage is already in place.
Why fake images cause real-world damage
Even when an image is clearly fake, the social response can be brutal and irreversible. Women may face family pressure, workplace scrutiny, loss of opportunities, or social isolation simply because an image exists and has circulated.
The burden of explanation, denial, and damage control almost always falls on the woman rather than on those who created or shared the content. The question asked is not who created this? But why was she in a position for this to happen?
Sometimes, you don’t even need a deepfake image to face online abuse. Women routinely receive hate comments and sexual slurs on social media simply for posting, speaking, or existing online. With no proper regulations, social media platforms have normalised the casual use of words like r*ndi, which can be seen frequently under posts by women, almost always commented by men. The word r*ndi has long been used as a sexual slur to shame, degrade, and silence women by linking their worth to sexual purity.
How much autonomy is too much?
Users who create and share abusive content must be held responsible. Most of the misuse of AI tools, whether it is generating sexualised images, posting slurs, or harassing women online, is driven by men. It does not mean all men behave this way, but it does mean the problem follows a pattern that platforms can no longer ignore. How much autonomy should male users of platforms get when that freedom repeatedly results in harm?
Children, AI, and the scale of digital sexual exploitation
Online abuse does not stop with adult women. Children are also being targeted. According to a media report, carried on 15th May, 2023, about 450207 cases of the spread of child sexual abuse material have been reported in India in the year 2023, so far. There were 204056 cases reported in the year 2022, 163633 in the year 2021, and 17390 in the year 2020 of child sexual abuse material on social media in India. It is almost a 250 to 300 per cent increase in Child Sexual Abuse Material (CSAM) materials.
Online child predators can be of any gender. But globally, incidents show that men carry out most such abuse. In a major crackdown on online child sexual predators in 2025, the Telangana Cyber Security Bureau has arrested 15 people, mostly educated men from middle-income families, aged 19 to 50.
The current trends leading to the Grok AI controversy
Over the past few days, X users, most of them men, used Grok to generate sexualised images by digitally removing clothing from pictures of 14-year-old actress Nell Fisher from Netflix’s Stranger Things. In another case, a user uploaded a photo of a woman wearing a school uniform, a pleated skirt and a grey blouse, and instructed Grok to remove her outfit.
Why “Free Speech” cannot excuse sexual harm
Platforms often give users a high level of freedom. They allow people to post, comment, edit images, and utilise AI tools with minimal restrictions. Some typically defend this freedom as free speech or as a form of creativity. But when the same group of users keeps using that freedom to harm others, especially women and children, how much freedom is too much?
India draws a line on platform responsibility
In response to the growing misuse of Grok, the Indian government issued a formal notice to X. It says the the platform must now take corrective action and submit a compliance report within 72 hours. The notice made it clear that continued inaction could result in X losing its safe-harbour protections under Indian law.
What is the order?
The order specifically directed X to restrict the generation of content involving nudity, sexualisation, sexually explicit material, or any other unlawful content. In its letter to X’s Chief Compliance Officer for India, the Ministry of Electronics and Information Technology (MeitY) said Grok was being misused to create fake accounts that host, generate, and circulate obscene images and videos of women in a vulgar and derogatory manner.
The ministry stated that X was not following key legal obligations under the Information Technology Act, 2000, the IT Rules, 2021, and the Bharatiya Nagarik Suraksha Sanhita, 2023. These laws require platforms to prevent and act against obscene, indecent, pornographic, paedophilic, or otherwise unlawful content.
If the platform fails to comply within the 72-hour window, the government has stated that it could revoke X’s safe-harbour status.
Responding to the controversy, X’s owner, Elon Musk, stated, “Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.“
Conclusion: The Grok AI controversy forces a question on platform responsibility
The Grok controversy is a warning that online freedom without accountability and AI tools may promise creativity, but when misused, they amplify harassment, sexual abuse, and the weaponisation of stigma, especially against women and children. Responsibility must lie with both users who create harmful content and platforms that design and deploy tools that facilitate such abuse and make it easy and widespread.
Digital freedom is meaningless if it comes at the cost of safety, dignity, and consent. Platforms must balance autonomy with accountability, because for women and children, a single image can be enough to cause lasting harm.
Changeincontent perspective
At Changeincontent, we see The Grok AI Controversy as a turning point. Not because AI abuse is new, but because the scale, speed, and normalisation of harm have crossed a line.
When generative AI can turn a single photograph into sexualised content without consent, women end up retreating. They refrain from engaging in digital spaces, public life, and professional visibility. It is not accidental; it mirrors older patterns of control, shame, and silence that are now automated.
We cannot celebrate digital innovation if it erodes dignity. Platforms must stop treating women’s safety as collateral damage in the race to build “unfiltered” tools.
Disclaimer: The views expressed in this article are based on the writer’s insights, supported by data and resources available both online and offline, as applicable. At changeincontent.com, we work towards promoting inclusivity across all forms of content. We broadly define inclusivity as media, policies, law, and history. It encompasses all elements that influence the lives of women and marginalised individuals. Our goal is to promote understanding and advocate for comprehensive inclusivity.
