A highly scalable and efficient system for transforming video files into HLS format using a microservices architecture.

Overview

This project is designed to transform videos into HLS format using a microservices architecture. The system is written in TypeScript and uses asynchronous processing to ensure scalability and efficiency. Key components include an Express.js server for handling HTTP requests, MongoDB for data storage, RabbitMQ for message queuing, and FFmpeg for video processing.

Project Structure

• src/: Contains the main application code.
  • index.ts: Entry point of the Express.js server.
  • db.ts: MongoDB connection setup.
  • routes/: Defines the API routes for video operations.
  • utils/: Utility functions, including RabbitMQ setup.
• models/: Mongoose models for MongoDB.
• video-processing-service/: Microservice for processing videos.
  • src/: Contains the video processing code.
  • index.ts: Entry point of the video processing service.
  • db.ts: MongoDB connection setup for the service.
  • videoProcessor.ts: Contains the logic for converting videos to HLS format.
• types/: TypeScript type definitions.
• .env: Environment variables.
• Dockerfile: Docker configuration for containerization.
• charts/hls-microservice-backend-chart/: Helm chart for Kubernetes deployment.

Features

• Upload videos via a REST API.
• Store video metadata in MongoDB.
• Queue video processing tasks using RabbitMQ.
• Convert videos to HLS format asynchronously.
• Fetch processed videos with pagination support.

Getting started

Requirements

• Node.js (v14 or later)
• MongoDB
• RabbitMQ
• Docker
• Kubernetes (optional, for Helm chart)

Quickstart

1. Clone the repository

   git clone https://github.com/ShivamB25/hls-microservice-backend.git
   cd hls-microservice-backend

2. Install dependencies

   npm install
   cd video-processing-service
   npm install
   cd ..

3. Set up environment variables
   • Copy .env.example to .env and fill in the necessary details.
4. Run the services
   • Start MongoDB and RabbitMQ.
   • Run the main service:

     npm start

   • Run the video processing service:

     cd video-processing-service
     npm start

5. Docker and Kubernetes (optional)
   • Build Docker images:

     docker build -t hls-microservice-backend .
     cd video-processing-service
     docker build -t video-processing-service .

   • Deploy using Helm:

     helm install microservice-example_ ./helm

Documentation

For more detailed information, refer to the following documentation files:

1. Kubernetes Deployment Guide

   • An in-depth explanation of how Kubernetes is used in the project, including setting up a Kubernetes cluster, using kubectl, and the purpose of each YAML file in the Helm chart.

2. Code Architecture

   • An overview of the project's architecture, explaining the roles of the Express server, video-processing service, MongoDB, and RabbitMQ, along with detailed descriptions of key files and directories.

3. RabbitMQ Usage

   • An explanation of the purpose of RabbitMQ in the project, including how it is used for message queuing, how it enables asynchronous processing, and how it is configured within the project. Examples of publishing and consuming messages are also provided.

License

This project is licensed under the MIT License. See the LICENSE file for more details.