Building a Scalable AI-Powered Transcription System for Healthcare

Project Year

2022

Industry

Software Development

 Qavi’s Approach

Execution Overview

A UAE-based software house aimed to develop an AI-powered healthcare solution for multiple hospitals. The product’s core functionality was to transcribe patient-doctor sessions, generate structured summaries, and store the data securely for future reference. The backend was developed using NestJS, while the frontend was built with Next.js.

The system leveraged Azure Transcription to process audio recordings and incorporated a form for doctors to input physical examination details. The combined data was then processed using ChatGPT to generate summaries and insights. The project evolved to support multi-tenancy, feature flagging, real-time monitoring, and HIPAA compliance.

Challenges

Real-Time Transcription Complexity

The initial approach of processing audio chunks in real time led to memory leaks and high latency.

Multi-Tenancy Implementation

 The system needed to support multiple hospitals while maintaining data segregation.

User Management Migration

 The client introduced an external system for user management, requiring a seamless migration without data loss

Compliance with HIPAA Regulations

Patient data security had to meet strict healthcare standards.

Performance & Monitoring

 Ensuring smooth system performance and quickly identifying issues was critical.

Solution Approach

Optimized Real-Time Transcription Process

Initially, audio chunks were sent from the frontend to the backend, which then forwarded them to the transcription service over WebSockets. However, this created performance bottlenecks. The solution:

01

Removed Backend from the Real-Time Pipeline

Audio chunks were sent directly to the transcription service.

02

Efficient Data Flow:

The transcription service processed audio in real time and stored the full audio in AWS S3.

03

Latency Reduction:

Once the session ended, the complete transcription and audio URL were sent to the backend, improving efficiency.

Multi-Tenancy & User Management Migration

To support multiple hospitals, multi-tenancy was introduced:

External System Integration

User authentication was migrated to the software house’s external system.

User Data Mapping

The user table in our database retained only the external system’s user ID, mapping it to existing records.

Organization-Based Access Control

Only admins could create organizations, invite doctors, and manage permissions.

White-Labeling Support

Custom redirect URLs were configured for each hospital.

Feature Flagging with Unleash

To customize services for different hospitals:

Unleash (Self-Hosted)

Unleash (Self-Hosted) was integrated for feature flag management.

Organizations as Feature Groups

Enabled or disabled services based on hospital requirements.

Backend & Frontend Integration

Allowed seamless toggling of features without redeploying the system.

Datadog Integration

Datadog Integration: Monitored API requests, flagged errors, and provided real-time alerts for system failures.

Transaction Tracking: Logged server-side transactions to diagnose and resolve bottlenecks

SonarQube for Code Quality: Integrated into CI/CD pipelines to enforce high code quality and run unit tests.

HIPAA Compliance & Secure Data Handling

FHIR Standard for Patient Data: Patient information was stored securely using healthcare industry best practices.

PHI Data Encryption: Any Personally Identifiable Health Information (PHI) required by HIPAA was encrypted before storage.

Access Controls: Ensured that only authorized personnel could access sensitive patient records.

AI-Assisted Development & Code Generation

To enhance development efficiency and maintain high code quality, we leveraged GitHub Copilot along with Claude Sonnet 3.5 for AI-powered code generation. These tools helped:

Speed Up Development: Assisted in writing boilerplate code and optimizing functions.

Improve Code Consistency: Suggested best practices and efficient patterns.

Enhance Debugging & Refactoring: Provided intelligent recommendations for performance improvements.

Agile Development & Collaboration

Daily Scrum Meetings:

 Developers shared updates and discussed blockers.

Weekly Backlog Grooming:

Tasks were refined based on evolving requirements.

Sprint Planning & Reviews:

Fortnightly sprint planning with effort estimation.

Strict Code Review Process:

  1. Feature branches were named after JIRA tickets.
  2. PRs required approval from at least two developers before merging

Key Features

Benefits Provided

Future Enhancements

AI-Based Medical Imaging Analysis

Automating X-ray and CT scan analysis using AI models.

Real-Time Transcription Improvements

Exploring next-gen models to further reduce latency

Advanced Insights & Analytics

Enhancing ChatGPT-generated reports with predictive healthcare trends.

Conclusion

This case study demonstrates how AI-driven automation, secure data handling, and scalable architecture can transform healthcare transcription and record-keeping, improving efficiency and compliance in medical institutions.