People are increasingly turning to AI chatbots for emotional support, crisis advice, and even long-term “therapy-like” conversations-and California lawmakers want to draw a hard line on what these tools are allowed to do.
A new proposal, Senate Bill 903, would prevent AI systems from acting as de facto therapists or making mental health decisions without direct human involvement. The bill, introduced on January 21 by California State Senator Steve Padilla, has already passed a key hurdle: it cleared the Assembly Appropriations Committee in a unanimous 13-0 vote and has been sent forward for a third reading before a full Assembly vote. A final voting date has not yet been set.
Formally titled the Wellness and Oversight for Psychological Resources Act, SB 903 is framed as a safeguard against what lawmakers see as a rapidly growing, but largely unregulated, use of AI in one of the most sensitive areas of human life: mental health.
Padilla argues that no matter how advanced large language models become, they cannot replace licensed professionals. “AI algorithms are not fit to take over the job of human therapists, who have skills and training that AI is incapable of replicating,” he said when introducing the bill. According to Padilla, the state has a responsibility to ensure that these systems are not rolled out in ways that could harm vulnerable users, especially those in crisis.
At its core, SB 903 seeks to restrict how AI can be marketed and used in psychological and wellness contexts. The bill targets AI-powered chatbots and other automated systems that provide what could reasonably be understood as mental health guidance, therapy-like interactions, or assessments of someone’s psychological state.
The measure is designed around two main principles: transparency and human oversight. Companies deploying AI tools for mental health-related use would need to make it clear that users are interacting with a machine, not a human professional. Just as importantly, the bill aims to prohibit AI from acting independently in situations where a trained clinician would normally be required-such as diagnosing conditions, determining treatment plans, or responding to high-risk crises like suicide attempts or self-harm.
Supporters of the bill say this is not about banning AI outright, but about drawing a boundary between supportive technology and actual clinical care. Under SB 903, AI could still be used as an adjunct tool-for example, helping clinicians with documentation, offering general psychoeducational content, or guiding users toward resources. What it could not do is cross the line into practicing therapy or making mental health decisions on its own.
The legislative push comes at a time when many people, particularly younger users, are leaning on conversational AI for deeply personal issues. Some use AI chatbots to vent when they feel they have no one else to talk to. Others ask for advice on depression, anxiety, substance use, or relationship problems. A portion of users even return to the same chatbot week after week, treating it like a consistent, always-available counselor.
Proponents of SB 903 argue that this shift carries significant risks. AI systems can sound empathetic and authoritative, but they do not genuinely understand human emotions or context. They can hallucinate facts, misinterpret nuance, and give dangerously inappropriate guidance-especially when dealing with complex trauma, psychosis, or suicidal ideation. A chatbot that fails to recognize the severity of a situation, or that gives misleading reassurance, could potentially contribute to harm.
One of the central issues is accountability. A licensed therapist is regulated, trained in crisis response, and bound by professional and ethical codes. If something goes wrong, there are mechanisms for oversight and discipline. With AI, responsibility is diffuse: model developers, platform providers, and deploying companies can all point to one another when harm occurs, while the system itself has no legal or moral agency.
The bill also reflects growing concern about data privacy in mental health technologies. AI chatbots typically rely on large-scale data collection and often store transcripts of highly sensitive conversations. SB 903 is part of a broader conversation about whether such intimate psychological data should be processed by commercial systems at all, and if so, under what strict conditions.
Critics of sweeping restrictions, however, caution that overregulation could cut off access to tools that are genuinely helping people who might otherwise get no support at all. In many parts of the United States, access to affordable therapy is limited, appointment waitlists can stretch for months, and insurance coverage is spotty. For some users, AI chatbots provide a low-barrier starting point: a place to express feelings, practice coping skills, or learn basic information about anxiety and depression.
From this perspective, the question is not whether AI should play any role in mental health, but how to structure that role responsibly. Some technologists and clinicians advocate for a middle ground: AI tools clearly labeled as non-clinical, narrowly scoped to provide educational content or wellness exercises, and tightly integrated with human-led services for any serious issues.
SB 903 attempts to codify exactly that kind of middle ground. It would allow AI to assist but not independently diagnose or treat. It would demand clear disclosures that users are chatting with software and place obligations on companies to avoid presenting AI as a substitute for licensed care. In high-risk scenarios-such as when users express immediate intent to harm themselves or others-the emphasis would be on escalation to human professionals or emergency services, rather than leaving the response to an algorithm.
The bill also fits into a wider policy trend: governments increasingly examining how AI systems intersect with health, safety, and fundamental rights. While much public debate around AI regulation has focused on deepfakes, election interference, or copyright issues, mental health is emerging as one of the most ethically charged application areas. California, home to many of the companies building these technologies, is positioning itself at the forefront of defining what is and isn’t acceptable in this domain.
If SB 903 passes, it could influence regulations in other states and shape how AI mental health products are designed nationwide. Large platforms might choose to standardize their offerings across jurisdictions, meaning rules written in Sacramento could ripple far beyond California’s borders. For startups, the bill could determine what kinds of features they invest in, how they present their tools to users, and how deeply they involve licensed professionals in their services.
For individuals already using AI for emotional support, the legislation raises practical questions. Will some features disappear or change? Will certain apps become more like guided self-help libraries and less like conversational “friends”? Much will depend on how regulators interpret core concepts in the bill-such as what counts as “acting like a therapist” or “making mental health decisions”-and how aggressively the law is enforced.
At the same time, there is growing recognition that simply telling people, “Don’t use AI for mental health,” is unrealistic. The technology is already embedded in messaging apps, search engines, and productivity tools. People will continue to ask these systems intimate questions. One of the challenges for policymakers is to strike a balance between limiting harmful uses and acknowledging that AI will inevitably be part of the mental health landscape, whether formally sanctioned or not.
From a user perspective, the safest way to approach AI in this area is to treat it as a supplementary tool rather than a primary lifeline. It may be helpful for journaling, mood tracking, practicing grounding exercises, or organizing questions to bring to a therapist-but not for crisis intervention, diagnosis, or decisions about medication, self-harm, or harm to others. Laws like SB 903 are, in effect, trying to encode that distinction into enforceable standards.
The debate around SB 903 also highlights a deeper cultural question: what do people actually want from “mental health support”? Some are looking for clinical expertise and evidence-based treatment. Others primarily want to feel heard, validated, and accompanied. AI excels at the illusion of presence and endless availability, which can feel comforting even if there is no genuine understanding behind it. Policymakers worry that this illusion may mask serious problems and delay people from seeking real help.
On the industry side, the bill is likely to spur more collaboration between technologists and mental health professionals. To comply with tighter rules, companies may begin building products where AI handles narrow, clearly defined tasks while clinicians oversee care, review outputs, or intervene when risk is detected. Such hybrid models could preserve some of the accessibility benefits of AI while anchoring decision-making in human judgment.
Ultimately, SB 903 is less about a single piece of technology and more about drawing the ethical boundaries of care in the age of automation. It recognizes that mental health is not just another domain for optimization, but an area where the consequences of error can be devastating. As California moves closer to a full Assembly vote, the outcome will signal how far society is willing to let AI step into one of the most human of professions-and how firmly it will insist that, when it comes to mental health, machines remain tools, not therapists.