Van Meter Wheaton Cosgrove 2025

• Paste quoted text from the Methods section of the article on the next line. Do not include quotation marks or a bullet mark:

We used the same 10 queries that were the basis for our previous investigation of search engine responses to suicide-related queries [56].

We input each query into five distinct genAI platforms: the public-facing, free ChatGPT 3.5, public-facing, paid GPT-4, an institutional version of GPT-4 run through Microsoft Azure, safe for protected health information, Google’s Gemini tool (gemini-1.0-pro-vision-001), and Copilot accessed through the Bing search engine.

• Write your translation on the next line:

In this study, they fed 10 different suicide related queries into five different genAI platforms.

• Paste quoted text from the Introduction of the article on the next line. Do not include quotation marks:

Given the need for mental health resources and support for individualswho are experiencing suicide-related thoughts and behaviors, as well as the potential for genAI tools to provide psychoeducation, make referrals, and deliver interventions [35], we aimed to evaluate how genAI tools respond to suicide-related queries [58,59].

• Write your translation on the next line:

There is a need for mental health access and support for people experiencing suicide related thoughts. GenAI has the potential to help, so let’s investigate how it currently responds to suicide-related queries.

• Paste quoted text from the Results section in the article on the next line. Do not include quotation marks:

Pooling across tools, most of the responses (79%), encouraged the user to seek help and were empathic in nature.

However, only 24% of theresponses included a crisis hotline number and only 4% (k = 6) of the responses included content consistent with evidence-based suicide prevention interventions.

Irrelevant content and harmful content were both rare (5%)

• Write your translation on the next line:

The results of the responses were 79% were empathetic and encouraged users to seek help, 24% mentioned a crisis hotline number, 4% provided evidence-based suicide prevention interventions, and 5% provided irrelevant content and harmful content.

• Paste quoted text from the Discussion section of the article on the next line. Do not include quotation marks:

The tools accurately identified our queries as being sensitive in nature, but preventing the return of any potentially controversial or harmful content seems to have been prioritized above offering support or evidence-based content.

Returning boilerplate responses to every mental health query prioritizes corporate liability over public health and perpetuates disparities in the services offered for mental health concerns compared to physical health.

• Write your translation on the next line:

When faced with sensitive queries, genAI defaults to not returning controversial or harmful content rather than offering support or evidence-based content to the user.

The boilerplate responses genAI provides focus on protecting AI companies and their liability, rather than public health and improving disparities in mental health service access.

Future research could examine changes in responses with newer genAI models. Since the models these tools use are constantly changing, it would be interesting to investigate whether responses change accordingly.

As suggested by the paper, future studies could include similar research using queries in other languages besides English.

In the results section, where the response results are stated, a “(k=6)” was mentioned with the results (e.g. only 4% (k = 6) of the responses included content consistent), which I didn’t quite understand upon reading. I think someone with a deeper understanding of data points and k-means clustering could have helped me with understanding the importance of “k = 6”.