Skip to main content
MyScribe
Healthcare PlatformProduction

MyScribe

AI-powered medical transcription for clinicians

A clinician-focused AI assistant that automatically transcribes patient conversations and generates SOAP-format clinical notes. Reduces documentation time by 70%, letting doctors focus on patients instead of paperwork.

Project Overview

What it is

An AI-powered medical scribe that listens to patient-clinician conversations in real-time, automatically generates structured SOAP clinical notes, and integrates into existing healthcare workflows.

Who it serves

Independent clinicians, small clinics, and telehealth providers who cannot afford human medical scribes but need accurate, HIPAA-compliant documentation.

Business value

Reduces documentation time by 70%, allowing clinicians to see more patients and reduce burnout. Costs a fraction of human scribe services.

Primary users

Physicians, nurse practitioners, physician assistants, and telehealth providers.

Quick Facts

Category

Healthcare Platform

Role

Full-Stack Developer

Timeline

6 months

Status

Production

Team

Solo

Platform

Web

Industry

Healthcare Technology

Frontend

Vue.js

Business Problem

Clinicians spend 30-50% of their work hours on documentation. Existing solutions are either expensive transcription services with 24-hour lag or general speech-to-text tools that fail on medical terminology. Small clinics cannot afford $30,000/year for human scribes.

Key Features

Real-Time Medical Transcription

Fine-tuned Whisper model achieves 94% accuracy on medical conversations. Captures terminology, medication names, and clinical context that general STT models miss.

AI-Generated SOAP Notes

Automatically structures conversation transcripts into Subjective, Objective, Assessment, and Plan format. Includes ICD-10 code suggestions based on the clinical assessment.

HIPAA-Compliant Architecture

End-to-end encryption for all audio data. AI processing runs on dedicated instances within a VPC with no outbound internet access. BAA-compliant infrastructure.

Editable Notes with Audit Trail

Clinicians can review and edit AI-generated notes before saving. All changes are tracked with version history for compliance and quality assurance.

Technology Stack

Frontend

Vue.jsQuasar.jsBootstrapJavaScript

Backend

LaravelGraphQLPythonPHPMySQL

Database

MySQL

Infrastructure

AWS ECSDockerRedis

Tools & Services

OpenAI WhisperLlama 2AWS KMS
Architecture Preview

Real-time audio is captured in the browser via WebSocket and streamed to a Laravel backend that orchestrates the AI pipeline. A fine-tuned Whisper model handles speech-to-text, while a quantized Llama 2 13B model generates SOAP notes. The Vue.js frontend displays live transcription and provides the note editing interface.

Challenges Solved
1

General speech-to-text models fail on medical terminology, causing clinically significant errors.

Fine-tuned Whisper on 5,000 hours of medical conversations. Built a custom medical vocabulary dictionary for term override. Achieved 94% accuracy vs 82% baseline.

2

LLM-based note generation took 45-60 seconds per patient visit, too slow for clinical workflow.

Applied 4-bit quantization reducing model size from 26GB to 7GB. Implemented streaming generation where clinicians see notes written section by section.

3

HIPAA compliance for cloud AI processing required data never leaving secure infrastructure.

Deployed dedicated GPU instances within a VPC with no outbound internet. Pre-loaded model weights. All processing stays within the secure environment.

Results
Reduced clinical documentation time by approximately 70%
HIPAA-compliant architecture with BAA from AWS
Adopted by multiple healthcare providers across the United States
94% medical speech transcription accuracy
Note generation latency reduced from 55s to 12s through optimization
Lessons Learned
1Medical AI products face a dual challenge: technical accuracy and clinical trust. Building trust required months of iterative improvements and transparent error reporting.
2Browser audio capture is surprisingly unreliable across different browsers and devices. Local audio buffering with sequence-numbered chunks was essential for reliability.
3HIPAA compliance is as much a legal and business process as a technical one. BAA negotiation and legal review of patient consent took 3+ months.
Related Projects

Interested in building something similar?

I am always open to discussing new projects, technical challenges, and engineering opportunities.