Initial commit

2026-06-11 10:46:09 +02:00
commit 81ad7e4ea9
223 changed files with 39530 additions and 0 deletions
@@ -0,0 +1,339 @@
+# HLS Transcoder Service
+
+A scalable HLS (HTTP Live Streaming) transcoding service for converting MP4 videos into adaptive bitrate HLS streams using FFmpeg.
+
+## Features
+
+- ✅ Automatic video detection and transcoding
+- ✅ Multiple quality variants (1080p, 720p, 480p)
+- ✅ Master playlist generation for adaptive bitrate streaming
+- ✅ Concurrent job processing
+- ✅ Database integration with Prisma
+- ✅ Comprehensive error handling and logging
+- ✅ Docker support with multi-stage builds
+- ✅ Health checks and graceful shutdown
+
+## Prerequisites
+
+### For Local Development
+- Node.js 18+
+- FFmpeg
+- PostgreSQL
+- TypeScript
+
+### For Docker
+- Docker Desktop (with Windows Support)
+- Docker Compose 2.0+
+
+## Quick Start
+
+### Development Mode
+
+```bash
+# Install dependencies
+cd transcoder
+npm install
+
+# Set environment variables
+export DATABASE_URL="postgresql://cms_user:changeme@localhost:5432/cms_db"
+export UPLOADS_DIR="/path/to/uploads"
+export NODE_ENV="development"
+
+# Run in development mode
+npm run dev
+
+# Or build and run
+npm run build
+npm start
+```
+
+### Docker Compose
+
+```bash
+# From project root
+docker-compose up -d
+
+# View logs
+docker-compose logs -f transcoder
+
+# Stop services
+docker-compose down
+```
+
+## Configuration
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `DATABASE_URL` | PostgreSQL connection string | (required) |
+| `UPLOADS_DIR` | Path to uploads directory | `/uploads` |
+| `POLL_INTERVAL` | Check for new videos every N ms | `5000` |
+| `CONCURRENT_JOBS` | Videos to transcode simultaneously | `2` |
+| `NODE_ENV` | Environment (development/production) | `production` |
+
+### Example Configuration
+
+```bash
+DATABASE_URL=postgresql://cms_user:secure_password@db.example.com:5432/cms_db
+UPLOADS_DIR=/mnt/storage/uploads
+POLL_INTERVAL=10000
+CONCURRENT_JOBS=4
+```
+
+## Output Format
+
+### Directory Structure
+```
+/uploads/hls/
+├── {video_id_1}/
+│   ├── master.m3u8      # Master playlist
+│   ├── 1080p.m3u8       # 1080p variant
+│   ├── 1080p_000.ts     # TS segments
+│   ├── 1080p_001.ts
+│   ├── 720p.m3u8        # 720p variant
+│   ├── 720p_000.ts
+│   └── 480p.m3u8        # 480p variant
+└── {video_id_2}/
+    └── ...
+```
+
+### Master Playlist Format
+
+```m3u8
+#EXTM3U
+#EXT-X-VERSION:3
+#EXT-X-STREAM-INF:BANDWIDTH=3500000,RESOLUTION=1920x1080
+1080p.m3u8
+#EXT-X-STREAM-INF:BANDWIDTH=1800000,RESOLUTION=1280x720
+720p.m3u8
+#EXT-X-STREAM-INF:BANDWIDTH=900000,RESOLUTION=854x480
+480p.m3u8
+```
+
+## API Reference
+
+### Database Schema
+
+```prisma
+model Video {
+  id                  String   @id
+  transcodingStatus   String   @default("uploaded")
+  // ... other fields
+  
+  @@index([transcodingStatus])
+}
+```
+
+### Status Lifecycle
+
+```
+uploaded → transcoded  (success)
+uploaded → failed      (error)
+```
+
+## Building & Deploying
+
+### Build Docker Image
+
+```bash
+# Build locally
+docker build -f Dockerfile.transcoder -t college-transcoder:latest .
+
+# Build with custom tag
+docker build -f Dockerfile.transcoder -t my-registry.com/transcoder:v1.0 .
+
+# Push to registry
+docker push my-registry.com/transcoder:v1.0
+```
+
+### Docker Compose Deployment
+
+Update `docker-compose.yml`:
+
+```yaml
+transcoder:
+  image: my-registry.com/transcoder:v1.0
+  environment:
+    DATABASE_URL: postgresql://...
+    UPLOADS_DIR: /uploads
+  volumes:
+    - /mnt/storage/uploads:/uploads
+```
+
+### Production Checklist
+
+- [ ] Use strong PostgreSQL password
+- [ ] Set appropriate `CONCURRENT_JOBS` for your hardware
+- [ ] Monitor disk space requirements
+- [ ] Set up log rotation for transcoder logs
+- [ ] Configure backup for HLS output directory
+- [ ] Monitor transcoder health via container health checks
+- [ ] Set resource limits in Docker (CPU, memory)
+
+## Monitoring & Debugging
+
+### View Logs
+
+```bash
+# Docker Compose
+docker-compose logs -f transcoder
+
+# Single container
+docker logs -f college-transcoder
+
+# Filter logs by level
+docker logs college-transcoder | grep "\[Error\]"
+```
+
+### Check Transcoding Status
+
+```bash
+# PostgreSQL query
+psql $DATABASE_URL -c \
+  "SELECT id, title, transcodingStatus FROM \"Video\" 
+   ORDER BY createdAt DESC LIMIT 10;"
+```
+
+### Verify HLS Files
+
+```bash
+# Check master playlist
+cat /uploads/hls/{video_id}/master.m3u8
+
+# Verify variant playlists
+for variant in 1080p 720p 480p; do
+  echo "=== $variant.m3u8 ===="
+  head -5 /uploads/hls/{video_id}/$variant.m3u8
+done
+
+# Check segment files
+ls -lh /uploads/hls/{video_id}/*.ts | head -5
+```
+
+### Test with FFprobe
+
+```bash
+# Check segment file
+ffprobe /uploads/hls/{video_id}/1080p_000.ts
+
+# Check master playlist validity
+ffprobe /uploads/hls/{video_id}/master.m3u8
+```
+
+## Performance Tuning
+
+### Adjust Batch Delay
+
+```bash
+# Faster processing (shorter delay between batches)
+export SLEEP_BETWEEN_BATCHES=10
+
+# Slower processing (longer delay, less CPU usage)
+export SLEEP_BETWEEN_BATCHES=60
+```
+
+### FFmpeg Presets
+
+Edit `transcoder/index.ts` to adjust quality presets:
+
+```typescript
+const HLS_PRESETS = [
+  { name: "1080p", width: 1920, height: 1080, bitrate: "3500k", maxrate: "4000k" },
+  { name: "720p", width: 1280, height: 720, bitrate: "1800k", maxrate: "2000k" },
+  { name: "480p", width: 854, height: 480, bitrate: "900k", maxrate: "1000k" },
+];
+```
+
+## Troubleshooting
+
+### Transcoder Won't Start
+
+**Check logs:**
+```bash
+docker logs college-transcoder 2>&1 | head -50
+```
+
+**Common issues:**
+- Database not accessible: Verify `DATABASE_URL`
+- FFmpeg not found: Check Docker image build
+- Uploads directory missing: Create `/uploads/videos` and `/uploads/hls`
+
+### No Videos Being Transcoded
+
+**Check video status:**
+```bash
+psql $DATABASE_URL -c "SELECT id, transcodingStatus FROM \"Video\";"
+```
+
+**Check file existence:**
+```bash
+ls -la /uploads/videos/
+```
+
+**Check permissions:**
+```bash
+stat /uploads/videos/ | grep Access
+chmod 755 /uploads/videos/
+chmod 644 /uploads/videos/*.mp4
+```
+
+### Transcoding Fails
+
+**Check FFmpeg:**
+```bash
+docker exec college-transcoder ffmpeg -version
+```
+
+**Check disk space:**
+```bash
+df -h /uploads
+# Need: ~3x original file size for temp files
+```
+
+**Check database:**
+```bash
+psql $DATABASE_URL
+SELECT * FROM "Video" WHERE transcodingStatus = 'failed';
+```
+
+## Development
+
+### Project Structure
+
+```
+transcoder/
+├── index.ts           # Main transcoder logic
+├── package.json       # Dependencies
+├── tsconfig.json      # TypeScript config
+└── dist/              # Compiled output (generated)
+```
+
+### Build for Development
+
+```bash
+cd transcoder
+npm install
+npm run build
+```
+
+### Type Checking
+
+```bash
+cd transcoder
+npx tsc --noEmit
+```
+
+## Contributing
+
+To improve the transcoder:
+
+1. Add new quality presets to `HLS_PRESETS`
+2. Adjust FFmpeg encoding parameters
+3. Add metrics/monitoring
+4. Implement retry logic
+5. Support additional input formats
+
+## License
+
+Part of the OWI CMS project.