Abstract
Recent breakthroughs in AI technology have resulted in sophisticated tools designed to aid doctors by aiding them in making diagnoses, documenting cases, and formulating decisions. The initiative incorporates an AI-driven healthcare advisory tool for Radiology using models like OpenAI's Whisper and Google's Gemini APIs alongside NLP methods for automatically creating organized diagnostic summaries based on verbal inputs. Using an integrated framework comprising React for frontend development, Fast API as its backend engine, and Firebase for cloud-based data management, this system transmutes unstructured medical recordings of Radiology into structured diagnostic reports effortlessly. Speech recognition software by Whisper achieves precise text conversion through an adapted method utilizing about 500 recordings collected in medical speech transcriptions and intents datasets obtained from Kaggle, augmented with synthesized voice material produced by Google's TTS technology. Subsequently, the audio transcript undergoes processing through the Gemini API, converting it into organized medical records comprising elements like Initial Symptoms, Observations, and Diagnosis. The front-end provides users like physicians and radiologists with easy-to-navigate tools allowing them to input medical information quickly, watch live transcripts in progress, and get instant feedback on their work via artificial intelligence-assisted summaries. The FastAPI component oversees interactions among the front end, the Whisper AI engine, and the Gemini service, guaranteeing smooth audio processing and output creation. Firebase manages data retention and updates, ensuring both safe cloud based file management and instantaneous accessibility of healthcare records in real time. This comprehensive model markedly decreases laborious tasks requiring human effort, improves precision in diagnosis.