Shush

Overview:

Shush is an application that showcases the deployment of a WhisperV3 model with Flash Attention v2 on Modal and interacts with it through a NextJS app. The main purpose of this app is to offer a comprehensive demonstration for individuals interested in utilizing high-performance models and reliable APIs on demand with auto-scaling capabilities. It is a demo application constructed with Next.js for the frontend and Modal for the backend.

Features:

• WhisperV3 Model Deployment: Demonstrates the deployment of a WhisperV3 model with Flash Attention v2 on Modal.
• NextJS Integration: Utilizes a NextJS app for interacting with the deployed model.
• Auto-Scaling Capabilities: Showcases the ability to run high-performance models and APIs on demand with auto-scaling.

Set Up:

1. Visit modal.com and create a free account.
2. Install the Modal python package and authenticate in your CLI by following the instructions on the website.

Deploy Backend:

We will be using Modal to deploy and serve the WhisperV3 model. Execute the following commands in your terminal:

# Commands to deploy the WhisperV3 model using Modal