American Association of Veterinary Radiologists guidelines, submission and review process for Veterinary Radiology Artificial Intelligence (AI) AAVR GMLP SaMD Product Certification
The following US Food and Drug Administration’s (FDA’s) Artificial Intelligence (AI) Good Machine Learning Practice (GMLP) guidelines and submission process have been adopted by the AAVR AI GMLP committee. Any company developing a veterinary radiology AI Software as a Medical Device (SaMD) product that wishes to receive AAVR AI GMLP committee certification must adhere to these guidelines and review process.
It is this committee’s recommendation that while implementing each aspect of the US FDA’s Good Machine Learning Practices (GMLP) is important, it is more important that each aspect of GMLP be addressed in a statement so there is transparency to the veterinary community (see figure below).
Veterinary radiology AI is a new area. Those who lead this area have the ethical responsibility to provide product knowledge to the veterinary community under the principles of transparency, honesty and integrity. Conversely, the veterinary professional community must understand that this is an ever-evolving field and available products will be in different development stages. A complete GMLP statement and implementation should be viewed as a commitment to GMLP principles but does not guarantee an end product’s capabilities.
For a veterinary radiology AI software product to earn the AI GMLP certification seal the company must submit a complete GMLP statement to the AAVR AI GMLP review committee. Submitted statements will first be evaluated by the AAVR AI GMLP committee for completeness. AI products with complete GMLP
statements, accepted for posting on the AAVR AI GMLP web page by the AAVR AI GMLP committee and while in accordance with the AAVR GMLP guidelines will have the right to display the AAVR AI GMLP certified seal on their web page, if they choose to. Complete GMLP statements will be posted on the AAVR AI GMLP guidelines web page with the company name, product name and product statement. Each AI product submission will require a separate GMLP statement with a clear association between the statement and the AI product. The AAVR AI GMLP certified seal must be associated with a specific AI product on a company’s web page if the company displaying the seal has more than one veterinary radiology AI product on that web page or on their website. Any company displaying the AI GMLP certified seal on their web page or website that is not listed on the AAVR AI GMLP web page AND/OR is not clearly associating the AI GMLP certified seal with the appropriate AI SaMD product; is in direct violation of the seal’s terms of use; DOES NOT have the right to display the AAVR AI GMLP certified seal; and offending party or parties will be prosecuted to the full extent of the law for false advertising and trademark infringement. IT CANNOT BE OVERSTATED that this certification seal is considered a trusted mark by the veterinary community and any inappropriate or unauthorized use of the AAVR AI GMLP certification seal will be
prosecuted to the full extent allowed by the law.
GMLP statements submitted to the AAVR AI GMLP committee will not be considered complete or evaluated by the AAVR AI GMLP committee until ALL required GMLP fields are included in the product statement submission. GMLP statements will not be posted until all required information is included and the submitted GMLP process has been approved by the AAVR AI GMLP committee. The AAVR AI GMLP committee recognizes the important responsibility of setting minimum standards for AI product development and therefore has absolute and final authority on this certification process to ensure the certification seal remains the trusted mark of excellence by the veterinary community.
Good Machine Learning Practices (GMLP):
Figure 1 source:US FDA Artificial Intelligence and Machine Learning Discussion Paper Figure 2
https://www.fda.gov/media/122535/download
Each GMLP statement must address how the product specifically satisfies subheadings A-D below. Each statement should be easily understood by a professional in the veterinary community who has limited to no veterinary radiology AI knowledge or experience. Each statement should state the method used to satisfy the subheading and when the method was implemented or a timeline for implementation. If the method is to be implemented in the future, an additional statement can be added stating how the product currently satisfies that subheading. If a current solution is not implemented for a subheading, “No solution at present.” must appear immediately after the applicable subheading title and should be in at least 10 point Arial bold font. No other words must appear between the subheading and this bolded sentence.
Required written statement subheadings:
A. Explanation of AI model development
B. Explanation of AI model pre-release assurance of safety and effectiveness
C. Explanation of AI production model deployment and on-going monitoring
D. Explanation of AI device ongoing re-training, modifications and versioning
Each submitted statement must also include the accompanying completed table.
Meet Our Team
Andrew Fox DVM DACVR: Dr. Andrew Fox completed his residency in radiology/diagnostic imaging at the University of Georgia in 2015. He is originally from NY, where has attended the Cornell College of Agriculture and Life Sciences, before moving on to veterinary school at Ross University. His journey continued with clinical experience at Texas A&M university, which preceded a one-year rotating internship in small animal medicine and surgery at the Louisiana State University veterinary teaching hospital. Dr. Fox joined the team at VICSD after years of high-volume, specialty referral practice in Southern California.
Jayashree Kalpathy-Cramer, PhD: Jayashree Kalpathy-Cramer has a PhD in Electrical Engineering. She is co-director of Harvard’s Center for Machine Learning, Athinoula A. Martinos Center for Biomedical Imaging, Associate Professor of Radiology at Harvard Medical School and Assistant in Neuroscience at Massachusetts General Hospital. She has been working in human medical artificial intelligence for over 5 years and been involved in some of the earliest ground-breaking work in human ophthalmology AI. In addition to her responsibilities at Harvard University, Jayashree is an invited speaker around the world lecturing on radiology AI development.
Tara Retson, MD, PhD: Dr Retson completed her dual degrees from Thomas Jefferson University with a PhD in Neuroscience, and is currently in the research radiology residency track at University of California San Diego. Self-taught in algorithm development, she has spent the last two years working with leaders in deep learning on both the development of algorithms and real-world validation of clinical applications.
Ty Vachon, MD: Dr. Vachon brings applied clinical machine learning experience and informatics expertise to the AI Standards committee. He was commissioned as a Naval Officer in 2002 and after he earned his MD at the Uniformed Services University, he completed his internship and radiology residency at the Naval Medical Center San Diego. During his 16 year Navy career he achieved nationwide accolades in informatics and patient safety. His academic and scientific background includes multiple peer reviewed publications in orthopedics and radiology. It was a natural progression to explore machine learning and Dr. Vachon spent years of self education learning about algorithm development, testing, validation as well as essential data science skills and data curation. After leaving active duty in 2018, he now advises several small and large startups as well as established leaders in the field of clinical medicine, machine learning, informatics and process improvement. He has been recognized by the American Medical Association Physician Innovation Network as a leader and he has served as an expert panelist on artificial intelligence in medicine. His most recent print publication “A Radiologist’s Introduction to AI and Machine Learning’ has been delivered to over 1000 practicing radiologists with the goal of educating his peers and advancing our field, one person at a time.
Seth Wallack DVM DACVR: Dr. Wallack received his veterinary degree and completed a four-year radiology residency at the University of California, Davis. He is board certified in veterinary radiology and has been practicing continuously in the San Diego veterinary community since 2002. Dr. Wallack has special interests in teleradiology, computer programming and teleradiology platforms, informatics and artificial intelligence. He has created two worldwide veterinary teleradiology platforms, holds a US patent on real-time ultrasound and is passionate about innovating for the veterinary profession. His passion for innovation led him to the world of veterinary radiology AI in 2016. He has also published scientific articles specific for MRI and authored “The Handbook of Veterinary Contrast Radiography.” He has lectured on various topics, including radiography, in both regional and national meetings.
