In Utah, an AI chatbot is being introduced to autonomously handle prescription refills after an initial review period by real doctors. The AI is programmed to prioritize safety and refer uncertain cases to human professionals, aiming to balance innovation and consumer protection. However, concerns have been raised about the lack of oversight and the potential risks of AI taking on roles traditionally filled by human clinicians. The FDA's role in regulating such AI applications remains uncertain, as prescription renewals are typically governed by state law, yet the FDA has authority over medical devices. This matters because it highlights the tension between technological advancement and the need for regulatory frameworks to ensure patient safety in healthcare.
