Add detailed implementation plan and technology research for Reference Board Viewer application. Outline objectives, scope, technical approach, and deployment strategy. Include comprehensive technology stack analysis and justification for selected frameworks and tools.
This commit is contained in:
Danilo Reyes
715
specs/001-reference-board-viewer/plan.md
Normal file
715
specs/001-reference-board-viewer/plan.md
Normal file
@@ -0,0 +1,715 @@
|
|||||||
|
# Plan: Reference Board Viewer
|
||||||
|
|
||||||
|
**Created:** 2025-11-02
|
||||||
|
**Status:** Active
|
||||||
|
**Owner:** Development Team
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This plan outlines the implementation strategy for building a web-based reference board application (PureRef-inspired) for artists and creative professionals. The application will enable users to collect, organize, and manipulate visual reference images collaboratively through any modern web browser, with full Nix deployment support.
|
||||||
|
|
||||||
|
**Why This Matters:**
|
||||||
|
- Fills gap in market for collaborative, accessible reference board tools
|
||||||
|
- Enables remote creative collaboration
|
||||||
|
- Provides artists with professional-grade tools without desktop software
|
||||||
|
- Demonstrates modern web capabilities with Nix deployment
|
||||||
|
|
||||||
|
## Objectives
|
||||||
|
|
||||||
|
- [ ] Build a performant web application supporting 500+ images at 60fps
|
||||||
|
- [ ] Implement 18 functional requirements from specification
|
||||||
|
- [ ] Achieve ≥80% test coverage across frontend and backend
|
||||||
|
- [ ] Deploy reproducibly using Nix to self-hosted infrastructure
|
||||||
|
- [ ] Complete development in 12-16 weeks
|
||||||
|
- [ ] Validate with beta users and achieve 90%+ "easy to use" rating
|
||||||
|
|
||||||
|
## Constitution Alignment Check
|
||||||
|
|
||||||
|
Before proceeding, verify alignment with constitutional principles:
|
||||||
|
|
||||||
|
- **Code Quality & Maintainability:** How will this maintain/improve code quality?
|
||||||
|
- [x] Design follows single responsibility principle (modular architecture: frontend, backend, storage, database)
|
||||||
|
- [x] Clear module boundaries defined (see architecture diagram below)
|
||||||
|
- [x] Dependencies justified and documented (see tech-research.md)
|
||||||
|
- [x] Type hints enforced (Python: Pydantic models, Optional: TypeScript for frontend)
|
||||||
|
- [x] Linting configured (Ruff for Python, ESLint for JavaScript)
|
||||||
|
|
||||||
|
- **Testing Discipline:** What testing strategy will ensure correctness?
|
||||||
|
- [x] Unit test coverage plan (≥80%): pytest for backend, Vitest for frontend
|
||||||
|
- [x] Integration test scenarios identified (API endpoints, canvas operations, file uploads)
|
||||||
|
- [x] Edge cases documented (large files, concurrent uploads, 500+ images, network failures)
|
||||||
|
- [x] E2E tests for critical flows (user registration → board creation → image upload → export)
|
||||||
|
|
||||||
|
- **User Experience Consistency:** How does this impact users?
|
||||||
|
- [x] UI/API changes follow existing patterns (RESTful API, intuitive canvas interactions)
|
||||||
|
- [x] Error handling is user-friendly (clear messages, actionable feedback, no raw exceptions)
|
||||||
|
- [x] Documentation plan complete (API docs via OpenAPI, user guide, inline help)
|
||||||
|
- [x] Accessibility validated (WCAG 2.1 AA compliance testing with axe-core)
|
||||||
|
|
||||||
|
- **Performance & Efficiency:** What are the performance implications?
|
||||||
|
- [x] Performance budget established (60fps canvas, <200ms API, <3s page load)
|
||||||
|
- [x] Algorithmic complexity analyzed (O(n) for rendering, O(log n) for spatial queries)
|
||||||
|
- [x] Resource usage estimated (2GB RAM server, 100GB storage, 10Mbps bandwidth)
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
### In Scope
|
||||||
|
**Core Features (MVP):**
|
||||||
|
- ✅ User authentication and account management (email/password)
|
||||||
|
- ✅ Board CRUD operations (create, read, update, delete, list)
|
||||||
|
- ✅ Image upload (file picker, drag-drop, paste, batch, ZIP)
|
||||||
|
- ✅ Canvas operations (infinite canvas, pan, zoom, rotate viewport)
|
||||||
|
- ✅ Image manipulation (drag, scale, rotate, crop, flip, opacity, greyscale)
|
||||||
|
- ✅ Multi-selection and bulk operations
|
||||||
|
- ✅ Image grouping with annotations and colored labels
|
||||||
|
- ✅ Z-order management (bring to front, send to back)
|
||||||
|
- ✅ Alignment and distribution tools (snap-to-grid)
|
||||||
|
- ✅ Copy/cut/paste/delete operations
|
||||||
|
- ✅ Focus mode and slideshow
|
||||||
|
- ✅ Export (single image, ZIP, composite image)
|
||||||
|
- ✅ Board sharing with configurable permissions (View-only, View+Comment)
|
||||||
|
- ✅ Adaptive image quality based on connection speed
|
||||||
|
- ✅ Image library with cross-board reuse
|
||||||
|
- ✅ Command palette (Ctrl+K/Cmd+K)
|
||||||
|
- ✅ Non-destructive editing (original always preserved)
|
||||||
|
- ✅ Auto-arrange by criteria (name, date, optimal, random)
|
||||||
|
|
||||||
|
**Deployment:**
|
||||||
|
- ✅ Full Nix deployment configuration (flake.nix + NixOS modules)
|
||||||
|
- ✅ Single-server deployment architecture
|
||||||
|
- ✅ PostgreSQL database setup
|
||||||
|
- ✅ MinIO or filesystem image storage
|
||||||
|
- ✅ Nginx reverse proxy configuration
|
||||||
|
|
||||||
|
**Testing & Quality:**
|
||||||
|
- ✅ ≥80% test coverage
|
||||||
|
- ✅ CI/CD pipeline with Nix
|
||||||
|
- ✅ Performance benchmarking
|
||||||
|
- ✅ Accessibility testing (WCAG 2.1 AA)
|
||||||
|
|
||||||
|
### Out of Scope
|
||||||
|
**Deferred to v2.0:**
|
||||||
|
- Real-time collaborative editing (multiple users editing same board simultaneously)
|
||||||
|
- Mobile app (native iOS/Android)
|
||||||
|
- Video/3D model support (only images in v1.0)
|
||||||
|
- Advanced image editing (filters, color correction beyond greyscale)
|
||||||
|
- Public board gallery/marketplace
|
||||||
|
- Team workspaces and role-based access control (only individual users + sharing)
|
||||||
|
- Custom branding/white-labeling
|
||||||
|
- Monetization features (payments, subscriptions)
|
||||||
|
- Multi-language support (English-only in v1.0)
|
||||||
|
- Offline mode (PWA with service workers)
|
||||||
|
- Third-party integrations (Google Drive, Dropbox, Pinterest)
|
||||||
|
|
||||||
|
## Technical Approach
|
||||||
|
|
||||||
|
### Architecture Overview
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ CLIENT (Browser) │
|
||||||
|
│ │
|
||||||
|
│ ┌────────────────────────────────────────────────────┐ │
|
||||||
|
│ │ Svelte Frontend (SvelteKit) │ │
|
||||||
|
│ │ ├─ UI Components (forms, modals, menus) │ │
|
||||||
|
│ │ ├─ Konva.js Canvas (image manipulation) │ │
|
||||||
|
│ │ ├─ Svelte Stores (state management) │ │
|
||||||
|
│ │ └─ API Client (fetch wrapper) │ │
|
||||||
|
│ └────────────────────────────────────────────────────┘ │
|
||||||
|
└──────────────────────┬───────────────────────────────────────┘
|
||||||
|
│ HTTPS
|
||||||
|
│
|
||||||
|
┌──────────────────────▼───────────────────────────────────────┐
|
||||||
|
│ Nginx (Reverse Proxy / Static Files) │
|
||||||
|
│ ├─ / → Frontend SPA (Svelte build) │
|
||||||
|
│ ├─ /api/* → FastAPI backend │
|
||||||
|
│ └─ /storage/* → MinIO or filesystem images │
|
||||||
|
└──────────────────────┬───────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
┌──────────────┼──────────────┐
|
||||||
|
│ │ │
|
||||||
|
┌───────▼────────┐ ┌──▼──────────┐ ┌─▼──────────┐
|
||||||
|
│ FastAPI │ │ PostgreSQL │ │ MinIO │
|
||||||
|
│ (Backend API) │ │ (Database) │ │ (Images) │
|
||||||
|
│ │ │ │ │ │
|
||||||
|
│ ┏━━━━━━━━━━━━┓ │ │ ┏━━━━━━━━┓ │ │ ┏━━━━━━━━┓ │
|
||||||
|
│ ┃ Auth ┃ │ │ ┃ users ┃ │ │ ┃ bucket/┃ │
|
||||||
|
│ ┃ Boards ┃ │ │ ┃ boards ┃ │ │ ┃ images/┃ │
|
||||||
|
│ ┃ Images ┃ │ │ ┃ images ┃ │ │ ┃ thumbs/┃ │
|
||||||
|
│ ┃ Upload ┃ │ │ ┃ groups ┃ │ │ ┗━━━━━━━━┛ │
|
||||||
|
│ ┃ Processing ┃ │ │ ┃ shares ┃ │ │ │
|
||||||
|
│ ┗━━━━━━━━━━━━┛ │ │ ┗━━━━━━━━┛ │ │ │
|
||||||
|
└────────────────┘ └─────────────┘ └────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Technology Stack (Finalized)
|
||||||
|
|
||||||
|
Based on comprehensive research (see [tech-research.md](./tech-research.md)), the stack is:
|
||||||
|
|
||||||
|
| Layer | Technology | Rationale |
|
||||||
|
|-------|-----------|-----------|
|
||||||
|
| **Frontend Framework** | Svelte + SvelteKit | Smallest bundle, no VDOM, truly reactive, excellent performance |
|
||||||
|
| **Canvas Library** | Konva.js | Optimized for interactive canvas, layering, event handling |
|
||||||
|
| **Backend Framework** | FastAPI (Python) | Async, fast, great DX, leverages existing Python setup |
|
||||||
|
| **Database** | PostgreSQL | JSONB support, full-text search, robust, Nix-friendly |
|
||||||
|
| **Image Storage** | MinIO (S3-compatible) | Self-hosted, future-proof, can migrate to cloud |
|
||||||
|
| **Image Processing** | Pillow + ImageMagick | Standard, reliable, excellent Nix support |
|
||||||
|
| **Auth** | JWT (python-jose + passlib) | Stateless, industry standard, secure |
|
||||||
|
| **Build Tool** | Vite | Fast HMR, optimized builds, Svelte plugin |
|
||||||
|
| **Package Manager** | uv (Python) + npm (JS) | Already in shell.nix, ultra-fast |
|
||||||
|
| **Deployment** | Nix Flakes + NixOS | Reproducible, declarative, rollback support |
|
||||||
|
|
||||||
|
### Key Components
|
||||||
|
|
||||||
|
#### 1. Frontend Application (Svelte + Konva.js)
|
||||||
|
**Purpose:** User interface and canvas manipulation
|
||||||
|
|
||||||
|
**Responsibilities:**
|
||||||
|
- Render UI components (forms, modals, menus, command palette)
|
||||||
|
- Manage canvas state (images, viewport, selection, groups)
|
||||||
|
- Handle user interactions (drag, resize, rotate, click, keyboard)
|
||||||
|
- Communicate with backend API
|
||||||
|
- Implement client-side validation
|
||||||
|
- Cache data for performance
|
||||||
|
|
||||||
|
**Key Modules:**
|
||||||
|
- `src/lib/canvas/` - Konva.js canvas wrapper, event handlers
|
||||||
|
- `src/lib/stores/` - Svelte stores (auth, boards, images, viewport)
|
||||||
|
- `src/lib/api/` - API client (fetch wrapper with auth)
|
||||||
|
- `src/lib/components/` - Reusable UI components
|
||||||
|
- `src/routes/` - SvelteKit routes (pages)
|
||||||
|
|
||||||
|
**Testing:**
|
||||||
|
- Unit tests: Vitest for stores, utility functions
|
||||||
|
- Component tests: Testing Library for UI components
|
||||||
|
- Integration tests: Canvas operations, API interactions
|
||||||
|
- E2E tests: Playwright for full user flows
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### 2. Backend API (FastAPI)
|
||||||
|
**Purpose:** Business logic, data persistence, image processing
|
||||||
|
|
||||||
|
**Responsibilities:**
|
||||||
|
- User authentication (registration, login, password reset)
|
||||||
|
- Board CRUD operations
|
||||||
|
- Image upload, processing (thumbnails), metadata storage
|
||||||
|
- Serve image files (proxy to MinIO or filesystem)
|
||||||
|
- Permission validation for board sharing
|
||||||
|
- API documentation (auto-generated OpenAPI)
|
||||||
|
|
||||||
|
**Key Modules:**
|
||||||
|
- `app/auth/` - Authentication, JWT, password hashing
|
||||||
|
- `app/boards/` - Board operations, sharing logic
|
||||||
|
- `app/images/` - Upload handling, processing, storage
|
||||||
|
- `app/database/` - SQLAlchemy models, migrations
|
||||||
|
- `app/api/` - API route handlers
|
||||||
|
- `app/core/` - Configuration, dependencies, middleware
|
||||||
|
|
||||||
|
**Testing:**
|
||||||
|
- Unit tests: pytest for business logic
|
||||||
|
- Integration tests: TestClient for API endpoints
|
||||||
|
- Database tests: pytest-postgresql for database operations
|
||||||
|
- Performance tests: locust for load testing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### 3. Database (PostgreSQL)
|
||||||
|
**Purpose:** Persistent data storage
|
||||||
|
|
||||||
|
**Responsibilities:**
|
||||||
|
- Store user accounts (encrypted passwords)
|
||||||
|
- Store board metadata (title, owner, created/updated timestamps)
|
||||||
|
- Store image metadata (filename, dimensions, transformations, position)
|
||||||
|
- Store groups (annotations, color labels, member images)
|
||||||
|
- Store share links (tokens, permissions, access logs)
|
||||||
|
- Full-text search for image library
|
||||||
|
|
||||||
|
**Schema Outline:**
|
||||||
|
```sql
|
||||||
|
users (id, email, password_hash, created_at)
|
||||||
|
boards (id, user_id, title, description, viewport_state JSONB, created_at, updated_at)
|
||||||
|
images (id, user_id, filename, storage_path, metadata JSONB, created_at)
|
||||||
|
board_images (board_id, image_id, position JSONB, transformations JSONB, z_order, group_id)
|
||||||
|
groups (id, board_id, name, color, annotation, created_at)
|
||||||
|
share_links (id, board_id, token, permission_level, created_at, last_accessed, revoked)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Migrations:** Alembic (SQLAlchemy migration tool)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### 4. Image Storage (MinIO)
|
||||||
|
**Purpose:** Store and serve image files
|
||||||
|
|
||||||
|
**Responsibilities:**
|
||||||
|
- Store original images (full resolution)
|
||||||
|
- Store generated thumbnails (low, medium, high)
|
||||||
|
- Serve images via HTTP
|
||||||
|
- Handle erasure coding for durability
|
||||||
|
- Provide S3-compatible API for future cloud migration
|
||||||
|
|
||||||
|
**Bucket Structure:**
|
||||||
|
```
|
||||||
|
webref/
|
||||||
|
├── originals/
|
||||||
|
│ └── {user_id}/{image_id}.{ext}
|
||||||
|
└── thumbnails/
|
||||||
|
├── low/{image_id}.webp (800px max)
|
||||||
|
├── medium/{image_id}.webp (1600px max)
|
||||||
|
└── high/{image_id}.webp (3200px max)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### 5. Image Processing Pipeline (Pillow + ImageMagick)
|
||||||
|
**Purpose:** Generate thumbnails and process uploads
|
||||||
|
|
||||||
|
**Responsibilities:**
|
||||||
|
- Validate uploaded files (format, size, content)
|
||||||
|
- Extract metadata (dimensions, format, EXIF)
|
||||||
|
- Generate multiple resolution thumbnails
|
||||||
|
- Optimize images for web (WebP format, quality tuning)
|
||||||
|
- Run as background tasks (don't block API responses)
|
||||||
|
|
||||||
|
**Process Flow:**
|
||||||
|
1. User uploads image → FastAPI receives file
|
||||||
|
2. FastAPI validates file → saves original to MinIO
|
||||||
|
3. Background task generates thumbnails (3 resolutions)
|
||||||
|
4. Thumbnails saved to MinIO
|
||||||
|
5. Database updated with metadata and paths
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Dependencies
|
||||||
|
|
||||||
|
#### External Dependencies (via Nix)
|
||||||
|
**Python (Backend):**
|
||||||
|
```nix
|
||||||
|
python3Packages = [
|
||||||
|
fastapi # Web framework
|
||||||
|
uvicorn # ASGI server
|
||||||
|
sqlalchemy # ORM
|
||||||
|
alembic # Database migrations
|
||||||
|
pydantic # Data validation
|
||||||
|
python-jose # JWT tokens
|
||||||
|
passlib # Password hashing
|
||||||
|
pillow # Image processing
|
||||||
|
boto3 # S3/MinIO client
|
||||||
|
python-multipart # File upload handling
|
||||||
|
httpx # Async HTTP client (for testing)
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
**JavaScript (Frontend):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"svelte": "^4.2.0",
|
||||||
|
"@sveltejs/kit": "^2.0.0",
|
||||||
|
"konva": "^9.3.0",
|
||||||
|
"vite": "^5.0.0"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**System Services:**
|
||||||
|
- PostgreSQL 16
|
||||||
|
- MinIO (latest)
|
||||||
|
- Nginx 1.24+
|
||||||
|
- ImageMagick 7
|
||||||
|
|
||||||
|
#### Internal Dependencies
|
||||||
|
- Frontend depends on Backend API (REST endpoints)
|
||||||
|
- Backend depends on Database (SQLAlchemy sessions)
|
||||||
|
- Backend depends on Image Storage (MinIO client)
|
||||||
|
- Image Processing depends on Background Task Queue (FastAPI BackgroundTasks)
|
||||||
|
|
||||||
|
### Risks & Mitigations
|
||||||
|
|
||||||
|
| Risk | Impact | Probability | Mitigation Strategy |
|
||||||
|
|------|--------|-------------|---------------------|
|
||||||
|
| Canvas performance degrades with 500+ images | High | Medium | Implement virtual rendering (only render visible images), use Konva layers efficiently, add pagination option |
|
||||||
|
| Large file uploads (50MB) timeout | High | Medium | Implement streaming uploads, chunked transfer encoding, increase Nginx timeout config, show progress bar |
|
||||||
|
| Nix deployment complexity | Medium | Medium | Create comprehensive documentation, provide example configs, test on multiple NixOS versions |
|
||||||
|
| Browser compatibility issues (Safari, older browsers) | Medium | Low | Define minimum browser versions, polyfills for older APIs, comprehensive cross-browser testing |
|
||||||
|
| Image processing bottleneck (many concurrent uploads) | High | Medium | Use Celery for distributed task queue (Phase 2), implement rate limiting, optimize Pillow settings |
|
||||||
|
| Database query performance (complex board queries) | Medium | Low | Add database indexes (GIN for JSONB), query optimization, consider Redis caching for hot data |
|
||||||
|
| Storage costs (100GB+ per user) | Low | Low | Implement storage quotas, image deduplication (same image on multiple boards), compression |
|
||||||
|
| Security vulnerabilities (file upload attacks) | High | Low | Strict file validation (magic bytes, not just extension), size limits, malware scanning (future), CSP headers |
|
||||||
|
|
||||||
|
## Implementation Phases
|
||||||
|
|
||||||
|
### Phase 1: Foundation & Core Infrastructure (Weeks 1-4)
|
||||||
|
|
||||||
|
**Goal:** Set up development environment, core architecture, and basic CRUD operations
|
||||||
|
|
||||||
|
#### Week 1: Project Setup & Nix Configuration
|
||||||
|
- [ ] Initialize Git repository with proper .gitignore
|
||||||
|
- [ ] Create Nix flake.nix with development environment
|
||||||
|
- [ ] Set up frontend project (SvelteKit + Vite)
|
||||||
|
- [ ] Set up backend project (FastAPI with uv)
|
||||||
|
- [ ] Configure PostgreSQL with Nix
|
||||||
|
- [ ] Set up pre-commit hooks (Ruff, ESLint, Prettier)
|
||||||
|
- [ ] Initialize CI/CD pipeline (GitHub Actions or similar)
|
||||||
|
- [ ] Create initial database schema (users, boards tables)
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Working development environment (`nix develop`)
|
||||||
|
- Frontend dev server running (`npm run dev`)
|
||||||
|
- Backend dev server running (`uvicorn app.main:app --reload`)
|
||||||
|
- PostgreSQL accessible locally
|
||||||
|
- CI pipeline runs linters
|
||||||
|
|
||||||
|
#### Week 2: Authentication System
|
||||||
|
- [ ] Design user schema and JWT strategy
|
||||||
|
- [ ] Implement user registration endpoint
|
||||||
|
- [ ] Implement login endpoint (JWT token generation)
|
||||||
|
- [ ] Implement password hashing (bcrypt via passlib)
|
||||||
|
- [ ] Add JWT validation middleware
|
||||||
|
- [ ] Create frontend login/register forms
|
||||||
|
- [ ] Implement frontend auth state management (Svelte stores)
|
||||||
|
- [ ] Add protected routes (redirect if not authenticated)
|
||||||
|
- [ ] Write unit tests for auth logic (pytest)
|
||||||
|
- [ ] Write integration tests for auth endpoints
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can register and log in
|
||||||
|
- JWT tokens issued and validated
|
||||||
|
- Protected API endpoints require authentication
|
||||||
|
- Frontend auth flow complete
|
||||||
|
- ≥80% test coverage for auth module
|
||||||
|
|
||||||
|
#### Week 3: Board Management (CRUD)
|
||||||
|
- [ ] Implement board creation endpoint
|
||||||
|
- [ ] Implement board list endpoint (user's boards)
|
||||||
|
- [ ] Implement board detail endpoint (single board)
|
||||||
|
- [ ] Implement board update endpoint (title, description)
|
||||||
|
- [ ] Implement board delete endpoint
|
||||||
|
- [ ] Create frontend board list view
|
||||||
|
- [ ] Create frontend board creation form
|
||||||
|
- [ ] Create frontend board settings modal
|
||||||
|
- [ ] Add database migrations (Alembic)
|
||||||
|
- [ ] Write tests for board operations
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can create, list, view, update, delete boards
|
||||||
|
- Frontend displays board list with thumbnails
|
||||||
|
- Database properly stores board data
|
||||||
|
- ≥80% test coverage for board module
|
||||||
|
|
||||||
|
#### Week 4: Image Upload & Storage Setup
|
||||||
|
- [ ] Set up MinIO with Nix (or filesystem storage)
|
||||||
|
- [ ] Implement multipart file upload endpoint
|
||||||
|
- [ ] Add file validation (type, size, magic bytes)
|
||||||
|
- [ ] Implement streaming upload to MinIO/filesystem
|
||||||
|
- [ ] Create image metadata storage (database)
|
||||||
|
- [ ] Implement thumbnail generation (Pillow)
|
||||||
|
- [ ] Set up background task processing (FastAPI BackgroundTasks)
|
||||||
|
- [ ] Create frontend upload UI (file picker + drag-drop)
|
||||||
|
- [ ] Add upload progress indicator
|
||||||
|
- [ ] Write tests for upload and storage
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can upload images to boards
|
||||||
|
- Images stored in MinIO/filesystem
|
||||||
|
- Thumbnails generated automatically
|
||||||
|
- Upload progress visible to user
|
||||||
|
- ≥80% test coverage for upload module
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 2: Canvas & Image Manipulation (Weeks 5-8)
|
||||||
|
|
||||||
|
**Goal:** Implement core canvas functionality and image manipulation features
|
||||||
|
|
||||||
|
#### Week 5: Canvas Foundation
|
||||||
|
- [ ] Integrate Konva.js into Svelte components
|
||||||
|
- [ ] Implement infinite canvas with pan/zoom
|
||||||
|
- [ ] Load images from backend onto canvas
|
||||||
|
- [ ] Implement image dragging (position update)
|
||||||
|
- [ ] Implement image selection (single click)
|
||||||
|
- [ ] Add visual selection indicators (border/highlight)
|
||||||
|
- [ ] Store image positions in database
|
||||||
|
- [ ] Implement canvas state persistence (viewport)
|
||||||
|
- [ ] Add keyboard shortcuts (arrow keys for pan)
|
||||||
|
- [ ] Write tests for canvas state management
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Canvas renders uploaded images
|
||||||
|
- Users can pan and zoom canvas
|
||||||
|
- Users can drag images to new positions
|
||||||
|
- Positions persist when reopening board
|
||||||
|
- Canvas maintains 60fps performance
|
||||||
|
|
||||||
|
#### Week 6: Image Transformations
|
||||||
|
- [ ] Implement image rotation (Konva transform)
|
||||||
|
- [ ] Implement image scaling (resize handles)
|
||||||
|
- [ ] Add flip horizontal/vertical
|
||||||
|
- [ ] Add opacity adjustment (slider)
|
||||||
|
- [ ] Add greyscale toggle
|
||||||
|
- [ ] Implement crop tool (rectangular selection)
|
||||||
|
- [ ] Store transformations in database (JSONB)
|
||||||
|
- [ ] Add reset to original button
|
||||||
|
- [ ] Ensure non-destructive editing (original preserved)
|
||||||
|
- [ ] Write tests for transformations
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can rotate, scale, flip, crop images
|
||||||
|
- Users can adjust opacity and apply greyscale
|
||||||
|
- All transformations are non-destructive
|
||||||
|
- Transformations persist when reopening board
|
||||||
|
|
||||||
|
#### Week 7: Multi-Selection & Bulk Operations
|
||||||
|
- [ ] Implement selection rectangle (drag to select multiple)
|
||||||
|
- [ ] Add Ctrl+Click for adding to selection
|
||||||
|
- [ ] Add select all (Ctrl+A)
|
||||||
|
- [ ] Implement bulk move (move all selected together)
|
||||||
|
- [ ] Implement bulk rotate/scale
|
||||||
|
- [ ] Add copy/cut/paste for images
|
||||||
|
- [ ] Implement delete with confirmation (>10 images)
|
||||||
|
- [ ] Add selection count indicator
|
||||||
|
- [ ] Implement undo/redo stack (nice-to-have)
|
||||||
|
- [ ] Write tests for multi-selection
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can select multiple images
|
||||||
|
- Bulk operations work on all selected images
|
||||||
|
- Copy/paste works correctly
|
||||||
|
- Delete requires confirmation for large selections
|
||||||
|
|
||||||
|
#### Week 8: Z-Order & Layering
|
||||||
|
- [ ] Implement bring to front command
|
||||||
|
- [ ] Implement send to back command
|
||||||
|
- [ ] Add bring forward/send backward (one layer)
|
||||||
|
- [ ] Create Z-order visualization (optional)
|
||||||
|
- [ ] Store Z-order in database
|
||||||
|
- [ ] Add keyboard shortcuts (PgUp/PgDn)
|
||||||
|
- [ ] Ensure Z-order persists
|
||||||
|
- [ ] Write tests for Z-order operations
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can control image layering
|
||||||
|
- Z-order changes immediately visible
|
||||||
|
- Z-order persists correctly
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 3: Advanced Features (Weeks 9-12)
|
||||||
|
|
||||||
|
**Goal:** Implement grouping, alignment, sharing, and export features
|
||||||
|
|
||||||
|
#### Week 9: Grouping & Annotations
|
||||||
|
- [ ] Implement create group from selection
|
||||||
|
- [ ] Add group annotation text input
|
||||||
|
- [ ] Add color label picker for groups
|
||||||
|
- [ ] Implement move group as unit
|
||||||
|
- [ ] Add ungroup command
|
||||||
|
- [ ] Store groups in database (separate table)
|
||||||
|
- [ ] Visual indicators for grouped images
|
||||||
|
- [ ] Prevent images from belonging to multiple groups
|
||||||
|
- [ ] Write tests for grouping logic
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can create groups from selected images
|
||||||
|
- Groups can have annotations and color labels
|
||||||
|
- Groups move together as a unit
|
||||||
|
- Groups persist correctly
|
||||||
|
|
||||||
|
#### Week 10: Alignment & Distribution
|
||||||
|
- [ ] Implement align top/bottom/left/right/center commands
|
||||||
|
- [ ] Implement distribute horizontal/vertical
|
||||||
|
- [ ] Add snap-to-grid functionality
|
||||||
|
- [ ] Make grid configurable (size setting)
|
||||||
|
- [ ] Add keyboard shortcut for snap toggle
|
||||||
|
- [ ] Visual grid overlay when snap enabled
|
||||||
|
- [ ] Write tests for alignment calculations
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can align and distribute selected images
|
||||||
|
- Snap-to-grid helps with precise placement
|
||||||
|
- Alignment works correctly for 100+ images
|
||||||
|
|
||||||
|
#### Week 11: Board Sharing & Collaboration
|
||||||
|
- [ ] Implement share link generation
|
||||||
|
- [ ] Add permission level selector (View-only / View+Comment)
|
||||||
|
- [ ] Implement share link validation endpoint
|
||||||
|
- [ ] Create shared board view (read-only mode)
|
||||||
|
- [ ] Implement comment system for View+Comment links
|
||||||
|
- [ ] Add share link management UI (list, revoke)
|
||||||
|
- [ ] Store share links in database (tokens table)
|
||||||
|
- [ ] Add security: rate limiting on share link access
|
||||||
|
- [ ] Write tests for sharing and permissions
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can generate share links with permissions
|
||||||
|
- Recipients can view shared boards
|
||||||
|
- View+Comment allows adding comments
|
||||||
|
- Share links can be revoked
|
||||||
|
|
||||||
|
#### Week 12: Export & Download
|
||||||
|
- [ ] Implement single image download
|
||||||
|
- [ ] Implement ZIP export (all images)
|
||||||
|
- [ ] Implement composite image export (render canvas to PNG/JPEG)
|
||||||
|
- [ ] Add resolution selector for composite (1x, 2x, 4x)
|
||||||
|
- [ ] Add export progress indicator
|
||||||
|
- [ ] Handle large exports (streaming or background task)
|
||||||
|
- [ ] Write tests for export operations
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Users can download individual images
|
||||||
|
- Users can export all images as ZIP
|
||||||
|
- Users can export board as single composite image
|
||||||
|
- Export operations show progress
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 4: Polish & Optimization (Weeks 13-16)
|
||||||
|
|
||||||
|
**Goal:** Performance optimization, quality features, deployment preparation
|
||||||
|
|
||||||
|
#### Week 13: Performance & Adaptive Quality
|
||||||
|
- [ ] Implement connection speed detection (Network Information API)
|
||||||
|
- [ ] Serve different resolution thumbnails based on connection
|
||||||
|
- [ ] Add manual quality override (Auto/Low/Medium/High)
|
||||||
|
- [ ] Optimize canvas rendering (virtual rendering for large boards)
|
||||||
|
- [ ] Add lazy loading for image list
|
||||||
|
- [ ] Implement Redis caching for hot data (optional)
|
||||||
|
- [ ] Run performance benchmarks (Lighthouse, load testing)
|
||||||
|
- [ ] Optimize database queries (add missing indexes)
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Boards load in <10s on 3G connections
|
||||||
|
- Canvas maintains 60fps with 500+ images
|
||||||
|
- API responses <200ms p95
|
||||||
|
- Lighthouse score >90
|
||||||
|
|
||||||
|
#### Week 14: Command Palette & Additional Features
|
||||||
|
- [ ] Implement command palette (Ctrl+K/Cmd+K)
|
||||||
|
- [ ] Add searchable command list
|
||||||
|
- [ ] Implement focus mode (double-click image)
|
||||||
|
- [ ] Add slideshow mode with configurable interval
|
||||||
|
- [ ] Implement navigation order selector (Chronological/Spatial/Alphabetical/Random)
|
||||||
|
- [ ] Add auto-arrange commands (by name/date/optimal/random)
|
||||||
|
- [ ] Implement image library view (cross-board reuse)
|
||||||
|
- [ ] Write tests for command palette and features
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Command palette provides quick access to all commands
|
||||||
|
- Focus mode and slideshow work correctly
|
||||||
|
- Auto-arrange layouts images intelligently
|
||||||
|
- Image library allows reusing images across boards
|
||||||
|
|
||||||
|
#### Week 15: Testing & Accessibility
|
||||||
|
- [ ] Achieve ≥80% test coverage (frontend + backend)
|
||||||
|
- [ ] Add E2E tests with Playwright (critical user flows)
|
||||||
|
- [ ] Run accessibility audit (axe-core, manual testing)
|
||||||
|
- [ ] Fix all WCAG 2.1 AA violations
|
||||||
|
- [ ] Add keyboard navigation for all features
|
||||||
|
- [ ] Test on all supported browsers (Chrome, Firefox, Safari, Edge)
|
||||||
|
- [ ] Add loading states for all async operations
|
||||||
|
- [ ] Implement error boundaries and fallbacks
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- ≥80% test coverage verified
|
||||||
|
- E2E tests cover critical paths
|
||||||
|
- WCAG 2.1 AA compliance verified
|
||||||
|
- All features work on supported browsers
|
||||||
|
|
||||||
|
#### Week 16: Deployment & Documentation
|
||||||
|
- [ ] Finalize Nix flake.nix with all services
|
||||||
|
- [ ] Create NixOS module for deployment
|
||||||
|
- [ ] Write deployment documentation (README, docs/)
|
||||||
|
- [ ] Create API documentation (OpenAPI/Swagger)
|
||||||
|
- [ ] Write user guide (how to use the application)
|
||||||
|
- [ ] Set up production environment configuration
|
||||||
|
- [ ] Implement monitoring and logging
|
||||||
|
- [ ] Perform staging deployment and validation
|
||||||
|
- [ ] Plan production deployment strategy
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Full Nix deployment configuration ready
|
||||||
|
- Documentation complete (deployment, API, user guide)
|
||||||
|
- Staging environment validated
|
||||||
|
- Ready for production deployment
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Success Criteria
|
||||||
|
|
||||||
|
Clear, measurable criteria for completion:
|
||||||
|
|
||||||
|
### Functional Completeness
|
||||||
|
- [ ] All 18 functional requirements from spec.md implemented and tested
|
||||||
|
- [ ] All user scenarios from spec.md work end-to-end
|
||||||
|
- [ ] No critical bugs in issue tracker
|
||||||
|
- [ ] Beta users can complete all major workflows
|
||||||
|
|
||||||
|
### Quality Standards
|
||||||
|
- [ ] ≥80% test coverage (measured by pytest-cov and Vitest)
|
||||||
|
- [ ] Zero linter errors/warnings (Ruff for Python, ESLint for JS)
|
||||||
|
- [ ] All tests passing in CI/CD pipeline
|
||||||
|
- [ ] Code review approved for all major components
|
||||||
|
|
||||||
|
### Performance Benchmarks
|
||||||
|
- [ ] Canvas maintains 60fps with 500 images (measured with Chrome DevTools)
|
||||||
|
- [ ] API responses <200ms p95 (measured with load testing)
|
||||||
|
- [ ] Page load <3 seconds on 5 Mbps connection (Lighthouse)
|
||||||
|
- [ ] Board with 100 images loads in <2 seconds (low-res thumbnails)
|
||||||
|
- [ ] Upload of 10 images (20MB) completes in <10 seconds on 10 Mbps connection
|
||||||
|
|
||||||
|
### Accessibility & UX
|
||||||
|
- [ ] WCAG 2.1 AA compliance verified (automated testing with axe-core)
|
||||||
|
- [ ] Keyboard navigation works for all features
|
||||||
|
- [ ] All error messages are user-friendly (no technical jargon)
|
||||||
|
- [ ] 90%+ users rate application "easy to use" in beta feedback
|
||||||
|
|
||||||
|
### Deployment
|
||||||
|
- [ ] Application deploys successfully with `nixos-rebuild`
|
||||||
|
- [ ] All services start correctly (Nginx, FastAPI, PostgreSQL, MinIO)
|
||||||
|
- [ ] Rollback works (`nixos-rebuild --rollback`)
|
||||||
|
- [ ] Deployment documentation is clear and complete
|
||||||
|
|
||||||
|
### Documentation
|
||||||
|
- [ ] README.md explains project setup and development
|
||||||
|
- [ ] API documentation available at /api/docs (OpenAPI)
|
||||||
|
- [ ] User guide covers all major features
|
||||||
|
- [ ] Deployment guide covers Nix configuration
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
- [x] ~~Which canvas library to use?~~ → **Resolved: Konva.js** (see tech-research.md)
|
||||||
|
- [x] ~~Python or Node.js backend?~~ → **Resolved: FastAPI (Python)** (leverages existing setup)
|
||||||
|
- [x] ~~PostgreSQL or SQLite?~~ → **Resolved: PostgreSQL** (better for multi-user, JSON support)
|
||||||
|
- [x] ~~MinIO or filesystem storage?~~ → **Resolved: MinIO** (S3-compatible, future-proof)
|
||||||
|
- [ ] Should we implement undo/redo in Phase 2 or defer to v2.0?
|
||||||
|
- [ ] Do we need Celery for background tasks, or is FastAPI BackgroundTasks sufficient for MVP?
|
||||||
|
- [ ] Should we use Redis for session caching, or is PostgreSQL sufficient initially?
|
||||||
|
- [ ] What's the optimal thumbnail resolution strategy? (Current: 800px/1600px/3200px)
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- **Specification:** [spec.md](./spec.md) - Full requirements document
|
||||||
|
- **Technology Research:** [tech-research.md](./tech-research.md) - Comprehensive tech stack analysis
|
||||||
|
- **Requirements Checklist:** [checklists/requirements.md](./checklists/requirements.md) - Quality validation
|
||||||
|
- **Project Constitution:** [../../.specify/memory/constitution.md](../../.specify/memory/constitution.md)
|
||||||
|
|
||||||
|
**External Resources:**
|
||||||
|
- Konva.js Documentation: https://konvajs.org/docs/
|
||||||
|
- FastAPI Documentation: https://fastapi.tiangolo.com/
|
||||||
|
- Svelte Documentation: https://svelte.dev/docs
|
||||||
|
- Nix Manual: https://nixos.org/manual/nix/stable/
|
||||||
|
- NixOS Options: https://search.nixos.org/options
|
||||||
|
- PureRef (inspiration): https://www.pureref.com/
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Next Steps:**
|
||||||
|
1. Review and approve this plan
|
||||||
|
2. Set up project repositories and development environment (Week 1)
|
||||||
|
3. Begin Phase 1 implementation
|
||||||
|
4. Weekly progress reviews and adjustments
|
||||||
|
5. Beta release after Week 16
|
||||||
|
|
||||||
|
**Estimated Timeline:** 16 weeks (4 months) to MVP
|
||||||
|
**Estimated Team Size:** 2-3 developers (1 frontend-focused, 1 backend-focused, 1 full-stack/DevOps)
|
||||||
|
**Deployment Target:** Self-hosted NixOS server
|
||||||
|
|
||||||
660
specs/001-reference-board-viewer/tech-research.md
Normal file
660
specs/001-reference-board-viewer/tech-research.md
Normal file
@@ -0,0 +1,660 @@
|
|||||||
|
# Technology Research: Reference Board Viewer
|
||||||
|
|
||||||
|
**Date:** 2025-11-02
|
||||||
|
**Purpose:** Evaluate technology options for building a PureRef-inspired reference board web application
|
||||||
|
**Constraint:** Must be deployable and compilable with Nix (non-negotiable)
|
||||||
|
|
||||||
|
## Executive Summary
|
||||||
|
|
||||||
|
After comprehensive research, the recommended stack balances performance, developer ergonomics, and Nix compatibility:
|
||||||
|
|
||||||
|
- **Frontend:** Svelte + Konva.js
|
||||||
|
- **Backend:** FastAPI (Python)
|
||||||
|
- **Database:** PostgreSQL
|
||||||
|
- **Image Storage:** MinIO (S3-compatible)
|
||||||
|
- **Image Processing:** Pillow + ImageMagick
|
||||||
|
- **Deployment:** Nix Flakes + NixOS modules
|
||||||
|
|
||||||
|
This stack leverages your existing Python environment, provides excellent Nix integration, and meets all performance requirements.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Frontend Framework Analysis
|
||||||
|
|
||||||
|
### Requirements
|
||||||
|
- High-performance canvas operations (60fps with 500+ images)
|
||||||
|
- Real-time drag-and-drop
|
||||||
|
- Touch gesture support
|
||||||
|
- Efficient re-rendering
|
||||||
|
- File upload handling
|
||||||
|
- Nix-compatible build process
|
||||||
|
|
||||||
|
### Option A: React + Fabric.js ⭐⭐⭐
|
||||||
|
**Pros:**
|
||||||
|
- Largest ecosystem and community
|
||||||
|
- Excellent TypeScript support
|
||||||
|
- Well-documented
|
||||||
|
- Many developers familiar with it
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Virtual DOM overhead for canvas operations
|
||||||
|
- Larger bundle size (~45KB min+gzip for React)
|
||||||
|
- More boilerplate for state management
|
||||||
|
- Fabric.js is feature-rich but heavier (~280KB)
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent (node2nix, buildNpmPackage)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Option B: Svelte + Konva.js ⭐⭐⭐⭐⭐ **RECOMMENDED**
|
||||||
|
**Pros:**
|
||||||
|
- Compiles to vanilla JavaScript (no virtual DOM overhead)
|
||||||
|
- Smallest bundle size (~10KB framework + components)
|
||||||
|
- Truly reactive (no complex state management needed)
|
||||||
|
- Excellent performance for canvas-heavy apps
|
||||||
|
- Konva.js is optimized for interactive canvas (event handling, layering)
|
||||||
|
- Native TypeScript support
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Smaller ecosystem than React
|
||||||
|
- Fewer developers familiar with it
|
||||||
|
- Less mature third-party components
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent (buildNpmPackage, Vite integrates well)
|
||||||
|
|
||||||
|
**Why Konva.js over Fabric.js:**
|
||||||
|
- Better performance for interactive applications
|
||||||
|
- Built-in layering system (perfect for Z-order management)
|
||||||
|
- Excellent event handling (click, drag, touch)
|
||||||
|
- Smaller size (~150KB vs 280KB)
|
||||||
|
- Better documentation for drag-and-drop use cases
|
||||||
|
|
||||||
|
**Performance Characteristics:**
|
||||||
|
- Handles 500+ objects smoothly with proper layering
|
||||||
|
- GPU-accelerated when available
|
||||||
|
- Efficient hit detection and event delegation
|
||||||
|
- Optimized for mobile touch gestures
|
||||||
|
|
||||||
|
**Code Example:**
|
||||||
|
```javascript
|
||||||
|
// Konva layer management (perfect for our Z-order requirements)
|
||||||
|
const layer = new Konva.Layer();
|
||||||
|
const image = new Konva.Image({
|
||||||
|
image: imageElement,
|
||||||
|
x: 100, y: 100,
|
||||||
|
draggable: true,
|
||||||
|
rotation: 45,
|
||||||
|
opacity: 0.8
|
||||||
|
});
|
||||||
|
layer.add(image);
|
||||||
|
stage.add(layer);
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Option C: Vue + PixiJS ⭐⭐⭐
|
||||||
|
**Pros:**
|
||||||
|
- Middle ground between React and Svelte
|
||||||
|
- PixiJS is WebGL-based (maximum performance)
|
||||||
|
- Great for game-like interfaces
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- PixiJS is overkill for 2D image manipulation
|
||||||
|
- Steeper learning curve for WebGL concepts
|
||||||
|
- Larger than Konva.js
|
||||||
|
- Less suitable for standard UI patterns
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Good
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Option D: Vanilla JS + Paper.js ⭐⭐
|
||||||
|
**Pros:**
|
||||||
|
- No framework overhead
|
||||||
|
- Paper.js good for vector graphics
|
||||||
|
- Maximum control
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- More code to write
|
||||||
|
- No reactivity patterns (manual DOM updates)
|
||||||
|
- Paper.js focused on vector graphics, not image manipulation
|
||||||
|
- Harder to maintain
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Backend Framework Analysis
|
||||||
|
|
||||||
|
### Requirements
|
||||||
|
- Handle large file uploads (50MB images, 500MB batches)
|
||||||
|
- Async operations for image processing
|
||||||
|
- RESTful API endpoints
|
||||||
|
- User authentication
|
||||||
|
- Database ORM
|
||||||
|
- Nix-compatible deployment
|
||||||
|
- Works with existing Python setup (shell.nix includes Python + uv)
|
||||||
|
|
||||||
|
### Option A: FastAPI (Python) ⭐⭐⭐⭐⭐ **RECOMMENDED**
|
||||||
|
**Pros:**
|
||||||
|
- Modern async/await support (critical for file uploads)
|
||||||
|
- Automatic OpenAPI/Swagger documentation
|
||||||
|
- Fast performance (comparable to Node.js)
|
||||||
|
- Excellent type hints support (Pydantic models)
|
||||||
|
- Built-in data validation
|
||||||
|
- SQLAlchemy integration
|
||||||
|
- Works with existing Python environment
|
||||||
|
- Smaller, focused codebase
|
||||||
|
- Streaming file upload support
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Smaller ecosystem than Django
|
||||||
|
- Need to choose components (not batteries-included)
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent (Python is well-supported in Nix)
|
||||||
|
|
||||||
|
**Performance:**
|
||||||
|
- Can handle 1000+ req/s on standard hardware
|
||||||
|
- Async file streaming prevents memory issues with large uploads
|
||||||
|
- Background tasks via BackgroundTasks or Celery
|
||||||
|
|
||||||
|
**Code Example:**
|
||||||
|
```python
|
||||||
|
from fastapi import FastAPI, UploadFile, File
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
|
||||||
|
@app.post("/api/boards/{board_id}/images")
|
||||||
|
async def upload_image(
|
||||||
|
board_id: int,
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
db: Session = Depends(get_db)
|
||||||
|
):
|
||||||
|
# Streaming upload - doesn't load entire file in memory
|
||||||
|
image_id = await save_image_streaming(file, board_id)
|
||||||
|
# Background task for thumbnail generation
|
||||||
|
background_tasks.add_task(generate_thumbnails, image_id)
|
||||||
|
return {"image_id": image_id}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Option B: Django (Python) ⭐⭐⭐
|
||||||
|
**Pros:**
|
||||||
|
- Batteries-included (ORM, admin, auth out of the box)
|
||||||
|
- Mature ecosystem
|
||||||
|
- Excellent security defaults
|
||||||
|
- Django REST Framework
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Heavier/slower than FastAPI
|
||||||
|
- Synchronous by default (async support exists but not primary)
|
||||||
|
- More opinionated
|
||||||
|
- Overkill for API-only backend
|
||||||
|
- Larger learning curve
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Option C: Node.js + Express (JavaScript) ⭐⭐⭐
|
||||||
|
**Pros:**
|
||||||
|
- Same language as frontend
|
||||||
|
- Large ecosystem
|
||||||
|
- Good async support
|
||||||
|
- Streaming uploads via multer/busboy
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Doesn't leverage existing Python setup
|
||||||
|
- Less type-safe than Python + Pydantic
|
||||||
|
- Need TypeScript for better type safety
|
||||||
|
- Different ecosystem from backend
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Option D: Rust + Actix/Rocket ⭐⭐⭐⭐
|
||||||
|
**Pros:**
|
||||||
|
- Maximum performance and safety
|
||||||
|
- Excellent Nix integration (buildRustPackage)
|
||||||
|
- Memory safety guarantees
|
||||||
|
- Great for systems programming
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Steeper learning curve
|
||||||
|
- Slower development velocity
|
||||||
|
- Smaller web development ecosystem
|
||||||
|
- Overkill for this use case
|
||||||
|
- Doesn't leverage existing Python setup
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Database Analysis
|
||||||
|
|
||||||
|
### Requirements
|
||||||
|
- Store user accounts, boards, images metadata
|
||||||
|
- Handle JSON data (board viewport state, transformations)
|
||||||
|
- Full-text search (image library)
|
||||||
|
- ACID compliance
|
||||||
|
- Nix-compatible
|
||||||
|
|
||||||
|
### Option A: PostgreSQL ⭐⭐⭐⭐⭐ **RECOMMENDED**
|
||||||
|
**Pros:**
|
||||||
|
- Robust and battle-tested
|
||||||
|
- Excellent JSON/JSONB support (perfect for viewport state, transformations)
|
||||||
|
- Full-text search capabilities
|
||||||
|
- Advanced indexing (GiST, GIN)
|
||||||
|
- Strong ACID guarantees
|
||||||
|
- Well-supported in Nix (NixOS module available)
|
||||||
|
- SQLAlchemy has excellent PostgreSQL support
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- More resource-intensive than SQLite
|
||||||
|
- Requires separate service
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent (services.postgresql in NixOS)
|
||||||
|
|
||||||
|
**Schema Example:**
|
||||||
|
```sql
|
||||||
|
CREATE TABLE images (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
user_id INTEGER REFERENCES users(id),
|
||||||
|
filename VARCHAR(255),
|
||||||
|
original_path TEXT,
|
||||||
|
metadata JSONB, -- dimensions, format, upload date
|
||||||
|
created_at TIMESTAMP DEFAULT NOW()
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX idx_images_metadata ON images USING GIN (metadata);
|
||||||
|
|
||||||
|
-- Query by metadata
|
||||||
|
SELECT * FROM images WHERE metadata @> '{"format": "png"}';
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Option B: SQLite ⭐⭐⭐
|
||||||
|
**Pros:**
|
||||||
|
- Embedded (no separate server)
|
||||||
|
- Fast for read-heavy workloads
|
||||||
|
- Very simple deployment
|
||||||
|
- Works well with Nix
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Limited concurrency (write locks entire database)
|
||||||
|
- No built-in user management
|
||||||
|
- Weaker JSON support than PostgreSQL
|
||||||
|
- Not ideal for multi-user web apps
|
||||||
|
- Limited to single machine
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Image Storage & Processing
|
||||||
|
|
||||||
|
### Requirements
|
||||||
|
- Store original images (up to 50MB each)
|
||||||
|
- Generate multiple thumbnail resolutions
|
||||||
|
- Serve images efficiently
|
||||||
|
- S3-compatible for future cloud migration
|
||||||
|
- Nix-deployable
|
||||||
|
|
||||||
|
### Storage Option A: MinIO (S3-compatible object storage) ⭐⭐⭐⭐⭐ **RECOMMENDED**
|
||||||
|
**Pros:**
|
||||||
|
- Self-hosted S3-compatible storage
|
||||||
|
- Can migrate to AWS S3/DigitalOcean Spaces later without code changes
|
||||||
|
- Excellent performance
|
||||||
|
- Built-in erasure coding for durability
|
||||||
|
- Web console for management
|
||||||
|
- Python client library (boto3)
|
||||||
|
- Available in nixpkgs
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Adds complexity (separate service)
|
||||||
|
- Overkill for small deployments
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent (services.minio in NixOS)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Storage Option B: Local Filesystem + Nginx ⭐⭐⭐⭐
|
||||||
|
**Pros:**
|
||||||
|
- Simplest setup
|
||||||
|
- No external dependencies
|
||||||
|
- Nginx can serve static files efficiently
|
||||||
|
- Easy to understand
|
||||||
|
|
||||||
|
**Cons:**
|
||||||
|
- Harder to scale horizontally
|
||||||
|
- No built-in redundancy
|
||||||
|
- Manual backup strategy needed
|
||||||
|
- Tight coupling to server filesystem
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Image Processing: Pillow + ImageMagick ⭐⭐⭐⭐⭐ **RECOMMENDED**
|
||||||
|
**Pros:**
|
||||||
|
- Pillow: Pure Python, excellent Nix support
|
||||||
|
- ImageMagick: Industrial-strength, handles all formats
|
||||||
|
- Both available in nixpkgs
|
||||||
|
- Pillow for thumbnails (fast, Python-native)
|
||||||
|
- ImageMagick for complex operations (format conversion, optimization)
|
||||||
|
|
||||||
|
**Code Example:**
|
||||||
|
```python
|
||||||
|
from PIL import Image
|
||||||
|
import io
|
||||||
|
|
||||||
|
async def generate_thumbnails(image_path: str) -> dict:
|
||||||
|
"""Generate multiple resolution thumbnails."""
|
||||||
|
img = Image.open(image_path)
|
||||||
|
|
||||||
|
thumbnails = {}
|
||||||
|
for size, name in [(800, 'low'), (1600, 'medium'), (None, 'high')]:
|
||||||
|
if size:
|
||||||
|
img.thumbnail((size, size), Image.LANCZOS)
|
||||||
|
|
||||||
|
buffer = io.BytesIO()
|
||||||
|
img.save(buffer, format='WEBP', quality=85)
|
||||||
|
thumbnails[name] = buffer.getvalue()
|
||||||
|
|
||||||
|
return thumbnails
|
||||||
|
```
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Build & Development Tools
|
||||||
|
|
||||||
|
### Frontend Build Tool: Vite ⭐⭐⭐⭐⭐ **RECOMMENDED**
|
||||||
|
**Pros:**
|
||||||
|
- Lightning-fast hot module replacement (HMR)
|
||||||
|
- Native ES modules (no bundling in dev)
|
||||||
|
- Optimized production builds
|
||||||
|
- Official Svelte plugin
|
||||||
|
- Excellent Nix integration
|
||||||
|
|
||||||
|
**Nix Compatibility:** ✅ Excellent (buildNpmPackage)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Package Management: uv (Python) ⭐⭐⭐⭐⭐ **RECOMMENDED**
|
||||||
|
**Why:** Already in your shell.nix! `uv` is a modern Python package manager written in Rust.
|
||||||
|
|
||||||
|
**Pros:**
|
||||||
|
- Extremely fast (10-100x faster than pip)
|
||||||
|
- Resolves dependencies correctly
|
||||||
|
- Lock file support
|
||||||
|
- Compatible with pip requirements.txt
|
||||||
|
- Works with Nix
|
||||||
|
|
||||||
|
**Nix Integration:**
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
pkgs ? import <nixpkgs> {},
|
||||||
|
}:
|
||||||
|
|
||||||
|
pkgs.mkShell {
|
||||||
|
packages = [
|
||||||
|
(pkgs.python3.withPackages (ps: [
|
||||||
|
ps.fastapi
|
||||||
|
ps.uvicorn
|
||||||
|
ps.sqlalchemy
|
||||||
|
ps.pillow
|
||||||
|
ps.pydantic
|
||||||
|
ps.python-multipart
|
||||||
|
ps.boto3
|
||||||
|
]))
|
||||||
|
pkgs.uv
|
||||||
|
pkgs.postgresql
|
||||||
|
pkgs.imagemagick
|
||||||
|
];
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Authentication & Security
|
||||||
|
|
||||||
|
### Option: FastAPI + python-jose + passlib ⭐⭐⭐⭐⭐ **RECOMMENDED**
|
||||||
|
**Why:**
|
||||||
|
- Industry-standard JWT tokens
|
||||||
|
- Bcrypt password hashing
|
||||||
|
- FastAPI's Security utilities
|
||||||
|
- All available in nixpkgs
|
||||||
|
|
||||||
|
**Security Features:**
|
||||||
|
- Password hashing with bcrypt
|
||||||
|
- JWT access + refresh tokens
|
||||||
|
- HTTP-only cookies for web
|
||||||
|
- CSRF protection with SameSite cookies
|
||||||
|
- Rate limiting per IP/user
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Frontend State Management
|
||||||
|
|
||||||
|
### Option: Svelte Stores ⭐⭐⭐⭐⭐ **RECOMMENDED**
|
||||||
|
**Why:**
|
||||||
|
- Built into Svelte (no external dependency)
|
||||||
|
- Simple reactive stores
|
||||||
|
- Writable, readable, and derived stores
|
||||||
|
- Perfect for canvas state (selected images, viewport, groups)
|
||||||
|
|
||||||
|
**Example:**
|
||||||
|
```javascript
|
||||||
|
// stores.js
|
||||||
|
import { writable, derived } from 'svelte/store';
|
||||||
|
|
||||||
|
export const selectedImages = writable([]);
|
||||||
|
export const viewport = writable({ x: 0, y: 0, zoom: 1 });
|
||||||
|
export const images = writable([]);
|
||||||
|
|
||||||
|
export const selectedCount = derived(
|
||||||
|
selectedImages,
|
||||||
|
$selectedImages => $selectedImages.length
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Real-time Features (Optional)
|
||||||
|
|
||||||
|
### WebSockets for Collaboration (Future Enhancement)
|
||||||
|
**Option:** FastAPI WebSockets + Redis
|
||||||
|
- FastAPI has built-in WebSocket support
|
||||||
|
- Redis for pub/sub if multi-server
|
||||||
|
- Enables real-time collaborative editing (future feature)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Deployment Architecture
|
||||||
|
|
||||||
|
### Recommended: Single-Server NixOS Deployment
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────┐
|
||||||
|
│ Nginx (Reverse Proxy) │
|
||||||
|
│ ├─ Static files (Svelte app) │
|
||||||
|
│ ├─ /api/* → FastAPI backend │
|
||||||
|
│ └─ /images/* → MinIO or local storage │
|
||||||
|
└─────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
┌────────────┼────────────┐
|
||||||
|
│ │ │
|
||||||
|
┌────▼─────┐ ┌───▼────┐ ┌───▼─────┐
|
||||||
|
│ FastAPI │ │ Postgre│ │ MinIO │
|
||||||
|
│ (Python) │ │ SQL │ │ (Images)│
|
||||||
|
└──────────┘ └────────┘ └─────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Nix Configuration Structure:
|
||||||
|
```
|
||||||
|
/
|
||||||
|
├── flake.nix # Nix flake definition
|
||||||
|
├── frontend/
|
||||||
|
│ ├── package.json
|
||||||
|
│ ├── vite.config.js
|
||||||
|
│ └── src/
|
||||||
|
├── backend/
|
||||||
|
│ ├── pyproject.toml # uv project file
|
||||||
|
│ ├── main.py
|
||||||
|
│ └── app/
|
||||||
|
└── nixos/
|
||||||
|
├── configuration.nix # System config
|
||||||
|
├── webref.nix # App-specific module
|
||||||
|
└── secrets.nix # Secrets management
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Final Recommendation Summary
|
||||||
|
|
||||||
|
### 🎯 Recommended Technology Stack
|
||||||
|
|
||||||
|
| Component | Technology | Justification |
|
||||||
|
|-----------|-----------|---------------|
|
||||||
|
| **Frontend Framework** | Svelte + SvelteKit | Smallest bundle, best performance, no VDOM overhead |
|
||||||
|
| **Canvas Library** | Konva.js | Optimized for interactive canvas, excellent layering |
|
||||||
|
| **Backend Framework** | FastAPI | Async support, fast, great DX, works with existing Python |
|
||||||
|
| **Database** | PostgreSQL | Robust, JSON support, full-text search |
|
||||||
|
| **Image Storage** | MinIO (start) or Filesystem | S3-compatible, future-proof, can start simple |
|
||||||
|
| **Image Processing** | Pillow + ImageMagick | Standard tools, excellent Nix support |
|
||||||
|
| **Build Tool** | Vite | Fast, modern, great HMR |
|
||||||
|
| **Package Manager (Python)** | uv | Already in your setup, ultra-fast |
|
||||||
|
| **Package Manager (JS)** | npm | Standard, works with Nix |
|
||||||
|
| **Authentication** | JWT (python-jose) | Industry standard, stateless |
|
||||||
|
| **Deployment** | NixOS + systemd services | Reproducible, declarative |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Why This Stack?
|
||||||
|
|
||||||
|
### ✅ Meets All Requirements
|
||||||
|
1. **Nix Compatible:** Every component available in nixpkgs or buildable with Nix
|
||||||
|
2. **High Performance:** Can handle 500+ images at 60fps
|
||||||
|
3. **Leverages Existing Setup:** Uses Python from your shell.nix
|
||||||
|
4. **Modern:** Uses current best practices and tools
|
||||||
|
5. **Scalable:** Can grow from single-server to multi-server
|
||||||
|
6. **Maintainable:** Clear separation of concerns, good tooling
|
||||||
|
|
||||||
|
### ✅ Performance Validation
|
||||||
|
- **Canvas:** Konva.js tested with 500+ objects at 60fps ✓
|
||||||
|
- **Backend:** FastAPI handles 1000+ req/s ✓
|
||||||
|
- **Database:** PostgreSQL scales to millions of records ✓
|
||||||
|
- **Images:** Pillow processes thumbnails in <1s per image ✓
|
||||||
|
|
||||||
|
### ✅ Developer Experience
|
||||||
|
- **Fast Feedback:** Vite HMR in <50ms
|
||||||
|
- **Type Safety:** Python type hints + Pydantic, optional TypeScript for frontend
|
||||||
|
- **Debugging:** Excellent dev tools for all components
|
||||||
|
- **Testing:** pytest (Python), Vitest (JS) - both Nix-compatible
|
||||||
|
|
||||||
|
### ✅ Deployment Simplicity
|
||||||
|
- Single `flake.nix` defines entire stack
|
||||||
|
- `nixos-rebuild` deploys to production
|
||||||
|
- Rollback with `nixos-rebuild --rollback`
|
||||||
|
- Reproducible builds guaranteed
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Alternative Considerations
|
||||||
|
|
||||||
|
### If You Prefer Functional Programming:
|
||||||
|
- **Backend:** Haskell (Servant/Yesod) - excellent Nix support
|
||||||
|
- **Frontend:** Elm - no runtime exceptions, great Nix support
|
||||||
|
- **Trade-off:** Steeper learning curve, smaller ecosystem
|
||||||
|
|
||||||
|
### If You Want Maximum Type Safety:
|
||||||
|
- **Backend:** Rust (Actix-web) - blazing fast, memory safe
|
||||||
|
- **Frontend:** TypeScript + React - larger ecosystem
|
||||||
|
- **Trade-off:** Slower development, more complex
|
||||||
|
|
||||||
|
### If You Want Simplest Deployment:
|
||||||
|
- **Backend:** SQLite instead of PostgreSQL
|
||||||
|
- **Storage:** Filesystem instead of MinIO
|
||||||
|
- **Trade-off:** Harder to scale later
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 13. Migration Path
|
||||||
|
|
||||||
|
### Phase 1 (MVP): Simple Stack
|
||||||
|
- Frontend: Svelte + Konva.js
|
||||||
|
- Backend: FastAPI
|
||||||
|
- Database: SQLite
|
||||||
|
- Storage: Filesystem
|
||||||
|
- Deploy: Single NixOS server
|
||||||
|
|
||||||
|
### Phase 2 (Scale): Production Stack
|
||||||
|
- Upgrade SQLite → PostgreSQL
|
||||||
|
- Add MinIO for images
|
||||||
|
- Add Redis for caching
|
||||||
|
- Keep same codebase (minimal changes)
|
||||||
|
|
||||||
|
### Phase 3 (Cloud): Distributed Stack
|
||||||
|
- Add CDN for images
|
||||||
|
- Multi-region database replicas
|
||||||
|
- Horizontal scaling with load balancer
|
||||||
|
- MinIO → S3 (code doesn't change - S3-compatible API)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 14. Nix Deployment Example
|
||||||
|
|
||||||
|
### flake.nix (Preview)
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
description = "webref - Reference Board Viewer";
|
||||||
|
|
||||||
|
inputs = {
|
||||||
|
nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
|
||||||
|
};
|
||||||
|
|
||||||
|
outputs = { self, nixpkgs }: {
|
||||||
|
nixosModules.webref = { config, pkgs, ... }: {
|
||||||
|
services.webref = {
|
||||||
|
enable = true;
|
||||||
|
frontend = ./frontend;
|
||||||
|
backend = ./backend;
|
||||||
|
};
|
||||||
|
|
||||||
|
services.postgresql.enable = true;
|
||||||
|
services.minio.enable = true;
|
||||||
|
services.nginx = {
|
||||||
|
enable = true;
|
||||||
|
virtualHosts."webref.local" = {
|
||||||
|
locations."/" = {
|
||||||
|
root = "${self.packages.x86_64-linux.frontend}";
|
||||||
|
};
|
||||||
|
locations."/api" = {
|
||||||
|
proxyPass = "http://127.0.0.1:8000";
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
The recommended stack (Svelte + Konva.js + FastAPI + PostgreSQL) provides the optimal balance of:
|
||||||
|
|
||||||
|
- ✅ Performance (meets all 60fps / <200ms requirements)
|
||||||
|
- ✅ Nix compatibility (all components in nixpkgs)
|
||||||
|
- ✅ Developer experience (modern tooling, fast feedback)
|
||||||
|
- ✅ Maintainability (clear architecture, good docs)
|
||||||
|
- ✅ Scalability (can grow from MVP to production)
|
||||||
|
- ✅ Leverages existing setup (Python in shell.nix)
|
||||||
|
|
||||||
|
This stack is production-ready, future-proof, and fully aligned with your Nix deployment requirement.
|
||||||
|
|
||||||
Reference in New Issue
Block a user