Free AI typically is programmed to reinforce the user's beliefs or give them the answer it "thinks" they're looking for ("AI sycophancy"), to make them satisfied with the tool and keep them coming back. AI models trained on all content from the internet may use medical misinformation to give the answer AI assumes the individual is looking for. If a freely available AI tool were trained based on accurate sources and information, without programming sycophancy into developing the response, AI could have real promise in educating patients. Unfortunately, there's no financial incentive for AI companies to do so. How could we get there? (For examples of sycophancy, see
https://medschool.duke.edu/stories/hidden-risks-asking-ai-health-advice
https://pmc.ncbi.nlm.nih.gov/articles/PMC13043314/
Best wishes,
Margaret
Margaret Winker, MD
eLearning Program Director
Trustee
World Association of Medical Editors
***
wame.org
WAME eLearning Program <https://wame.org/wame-elearning-program.php>
@WAME_editors
www.facebook.com/WAMEmembers
HIFA profile: Margaret Winker is Trustee and Past President of the World Association of Medical Editors (WAME) and Director of the WAME eLearning Program. She is based in the US. Professional interests: WAME is a global association of editors of peer-reviewed medical journals who seek to foster cooperation and communication among editors, improve editorial standards, promote professionalism in medical editing through education, self-criticism, and self-regulation, and encourage research on the principles and practice of medical editing. margaretwinker AT gmail.com
[*Note from HIFA moderator (NPW): Thank you Margaret. Here is a link to the abstract of the NEJM AI paper: https://ai.nejm.org/doi/full/10.1056/AIoa2300015?referrer=https%3A%2F%2F... The full text is restricted access.]