twotalesanimation
4cd8986e69
req.formData() parsed the entire multipart body into RAM, so large uploads (1.6GB+) OOM-killed the server. Bridge the web Request body to a Node stream and parse it with formidable, which writes the file to disk as it arrives: - Temp dir is /uploads/tmp (same filesystem as the destination) so the post-parse move is an atomic rename, not a second multi-GB write - Mark the bridged request as chunked when a proxy strips content-length, otherwise formidable assumes an empty body - Track formidable temp files via fileBegin so aborted uploads get their partial files cleaned up - Remove the dead formidable branch gated on (req as any).req, which is always null in the App Router Also fix the nginx example in VPS_UPLOAD_CONFIG.md: client_max_body_size was 1024M, below the app's 2.5GB limit, and would 413 large uploads at the proxy before they reached the app. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
93 lines
3.1 KiB
Markdown
93 lines
3.1 KiB
Markdown
# VPS Upload Timeout Fix
|
|
|
|
## Problem
|
|
Uploads timing out after moving to VPS + npm environment.
|
|
|
|
## Root Causes
|
|
1. Prisma connection pool timeout too short for network latency
|
|
2. FFprobe duration extraction timing out on slow disk I/O
|
|
3. Missing socket/request timeout configuration
|
|
4. Database connection pool exhaustion
|
|
|
|
## Solutions Applied
|
|
|
|
### 1. Prisma Client Configuration
|
|
Updated `lib/prisma.ts` with:
|
|
- Connection timeout handling for slow VPS networks
|
|
- Proper pool configuration support
|
|
- Removed 'info' logging (reduces noise in production)
|
|
|
|
### 2. FFprobe Timeout Protection
|
|
Updated `app/api/admin/upload/route.ts`:
|
|
- Added 15-second timeout for FFprobe duration extraction
|
|
- Falls back gracefully if FFprobe takes too long
|
|
- Prevents blocking the entire upload handler
|
|
|
|
### 3. Next.js Server Configuration
|
|
Updated `next.config.ts`:
|
|
- Added socket timeout configuration (45 seconds default)
|
|
- VPS-aware settings for slow network environments
|
|
|
|
## Required Environment Variables
|
|
|
|
Add these to your `.env.local` (or VPS environment):
|
|
|
|
```
|
|
# Database connection with proper pool settings for VPS
|
|
# Adjust pool_max based on your VPS CPU cores
|
|
DATABASE_URL="postgresql://user:password@host:5432/dbname?schema=public&pool_max=5&socket_timeout=45000&connect_timeout=10000"
|
|
|
|
# Optional: Override default socket timeout (in milliseconds)
|
|
DATABASE_SOCKET_TIMEOUT=45000
|
|
|
|
# Optional: For testing, you can increase the API timeout
|
|
# This is already set to 300s in the route, but ensure your nginx/proxy respects it
|
|
```
|
|
|
|
## PostgreSQL Connection String Parameters
|
|
If using PostgreSQL, ensure your `DATABASE_URL` includes:
|
|
- `pool_max=5` — Limit connections (adjust to CPU cores)
|
|
- `socket_timeout=45000` — Socket timeout in ms
|
|
- `connect_timeout=10000` — Connection establishment timeout in ms
|
|
|
|
Example PostgreSQL URL:
|
|
```
|
|
postgresql://user:password@vps-host:5432/dbname?schema=public&pool_max=5&socket_timeout=45000&connect_timeout=10000
|
|
```
|
|
|
|
## Next.js Server Configuration
|
|
If running with a proxy (nginx/Apache), ensure:
|
|
|
|
```nginx
|
|
# nginx example
|
|
location /api/admin/upload {
|
|
proxy_pass http://next-server;
|
|
proxy_connect_timeout 60s;
|
|
proxy_send_timeout 2700s; # 45 minutes, matches maxDuration in the route
|
|
proxy_read_timeout 2700s; # 45 minutes, matches maxDuration in the route
|
|
client_max_body_size 3G; # Must exceed the 2.5GB app-level limit
|
|
proxy_request_buffering off; # Stream to the app instead of spooling to nginx disk
|
|
}
|
|
```
|
|
|
|
## Testing
|
|
After applying these changes:
|
|
|
|
1. Restart your Node.js server
|
|
2. Test with a medium-sized video (100MB+)
|
|
3. Check logs: `pm2 logs` or your logging system
|
|
4. Monitor database connections: `SELECT count(*) FROM pg_stat_activity;` in psql
|
|
|
|
## Additional Tuning
|
|
|
|
### If still timing out:
|
|
1. **Check disk I/O**: `iostat -x 1` on VPS
|
|
2. **Monitor database**: Check if queries are slow
|
|
3. **Increase FFprobe timeout**: Edit line in `app/api/admin/upload/route.ts` (currently 15000ms)
|
|
4. **Vertical scaling**: Increase VPS CPU for FFprobe processing
|
|
|
|
### For very large files (>500MB):
|
|
- Consider streaming directly to cloud storage (S3, etc.)
|
|
- Implement chunked uploads
|
|
- Use a separate transcoding queue service
|