Google's AI health advice is under scrutiny not primarily for accuracy, but due to its failure to leave an evidentiary trail. This lack of evidence prevents the reconstruction and inspection of AI-generated outputs, which is crucial in regulated domains where mistakes need to be traceable and correctable. The inability to produce contemporaneous evidence artifacts at the moment of generation poses significant governance challenges, suggesting that AI systems should be treated as audit-relevant entities. This issue raises questions about whether regulators will enforce mandatory reconstruction requirements for AI health information or if platforms will continue to rely on disclaimers and quality assurances. This matters because without the ability to trace and verify AI-generated health advice, accountability and safety in healthcare are compromised.
