Frog-Lover-Antony
144 lines
8.9 KiB
Markdown
144 lines
8.9 KiB
Markdown
This is a design draft. Lots of things are missing. Fill the holes during planning. Ask user if things are unclear.
|
|
|
|
# Rules
|
|
|
|
- Project uses copyparty (github.com/9001/copyparty) for both application, content, user data and interactions trough it's web-dav like interface. Check github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md for http api reference.
|
|
- Project must be tested with a set of curl/nodejs sanity checks and an automated browser in docker, using gitea actions for releases and local docker for development loops.
|
|
- Project's repository lives at gitea.antonene.space/KORMsoftware/Copyparty-content-tag.
|
|
- Plan development and track progress in PLAN.md.
|
|
- Make tests for each feature (unless bulletproof or not observable from tests e.g. css styles on web-pages, that will require human interaction).
|
|
- All scripts must be served from copyparty. If you add an external library, download a local copy.
|
|
|
|
# Functionality
|
|
|
|
- The tagging system is a fully-static html+js application running on copyparty as a backend. It is supposed to make image/video/animation uplolading, tagging and sorting easy.
|
|
- Meets users with a "directory" input field if "current-cwd-header" is not set (basically a login screen). It is a custom url text field that allows setting the working directory. Below is a "check" button and an "enter" button. Check button gets user's permissions for the chosen directory and tells the user whether it is accessible and whether the user has write access.
|
|
- After picking a working directory the user must be dropped into the main ui.
|
|
- App loads database for all files and their tags in tsv format stored under `{cwd}/db/latest.tsv`.
|
|
- App lists new content in `{cwd}/content/unmanaged`, checks if files are not already in the loaded database: if not - moves them to `{cwd}/content/managed` under their new db-related ids, processes hashes and adds them to the database.
|
|
- App computes similarity hashes (phash.js) and main colors for images (1-4) (only images for now).
|
|
|
|
# Copyparty API wrapper (verified against localhost:8086)
|
|
|
|
All operations use `PW: <password>` header for auth. Anonymous sees only read-only `/public/`. Authenticated user sees both read (`/public/`, no write) and admin (`/private/user`, full access). Server returns 403 on wrong password.
|
|
|
|
## API endpoints used by the app
|
|
|
|
### Directory resolution ("Check" button)
|
|
- `GET /{path}/?ls` → JSON with `dirs[]`, `files[]`, `perms[]` array. If `"write"` ∈ `perms`, directory is writable.
|
|
- The "Enter" button stores `{path}` as CWD in localStorage and redirects to main UI.
|
|
|
|
### TSV database read/write
|
|
- **Read**: `GET /{cwd}/db/latest.tsv?txt&pw=...` → raw text body. File may not exist on first run (404) — app creates fresh empty DB.
|
|
- **Write full**: `PUT` to `/private/user/db/latest.tsv?j` with `Replace: 1` header + binary body → JSON `{filesz, fileurl, sha512}` on success.
|
|
- **Append row**: `PUT` to same URL with `?apnd&j` (no Replace needed) appends bytes to existing file.
|
|
|
|
**IMPORTANT**: The `?replace` URL parameter does NOT work for PUT uploads — only the `Replace: 1` HTTP header overwrites. Without it, copyparty deduplicates by appending a timestamp suffix.
|
|
|
|
### File management
|
|
- **List dir**: `GET /{path}/?ls&dots` → JSON listing (includes dotfiles).
|
|
- **Move file**: `POST /{src}?move=/{dst}` → returns "k" on success. Source gone, dest appears with new name.
|
|
- **Copy file**: `POST /{src}?copy=/{dst}` → same pattern as move.
|
|
- **Delete single**: `POST /{path}to?delete&j` → JSON confirm.
|
|
- **Delete batch**: `jPOST /?delete&j` body `["/abs/path1", "/abs/path2"]` at root level.
|
|
- **Mkdir**: `mPOST /{parent}/?replace` with `act=mkdir&name=<dir>` → HTML redirect + "aight" confirm. Auto-creates missing parents.
|
|
|
|
### File upload
|
|
- **PUT (binary)**: `PUT /{path}/?j` body=file bytes, `Replace: 1` header to overwrite.
|
|
- **Multipart**: `mPOST /{dir}/?replace&j` with `f=@file;filename=name` on same URL. Note: the raw curl syntax for multipart requires correct `-F` formatting — malformed field names cause "expected field" errors.
|
|
|
|
### Search
|
|
- `jPOST /?ls` body `{"q":"raw_field=value"}` → plain-text result lines. Syntax from copyparty's search tab (`name=`, `ext=`, etc). Returns raw field matches, not a filtered JSON listing. For app use, prefer full-dir `?ls` then client-side filtering over server search.
|
|
|
|
## Permission model (confirmed)
|
|
- Root `/?ls`: unauthenticated → `acct: "*", perms: []`. Authenticated → `acct: "user"`, both routes visible with route-specific permissions.
|
|
- `/private/user/` → `perms: ["read","write","move","delete","dot","get","admin"]`
|
|
- `/public/` → `perms: ["read","get"]`
|
|
- File-level tags include `w:` (copyparty's W-hash), `up_by`, `up_ip`, `.up_at`.
|
|
|
|
# Data model
|
|
|
|
All content metadata lives in a single TSV at `{cwd}/db/latest.tsv`. The app reads this on startup into memory, performs all tag operations in-memory, then writes back the full TSV after each batch of changes.
|
|
|
|
## DB file ID strategy
|
|
|
|
File IDs use copyparty's W-hash (`w` tag from `?ls` JSON response). Every file listing entry includes a `tags.w` field — a base32/sha512-derived unique identifier for that file on that volume. This is more stable than filename or path, surviving renames and moves within the same mount.
|
|
|
|
**Alternative considered**: Generate UUIDv4 on first scan. **Rejected** — adds state not present in copyparty itself; W-hash already exists server-side and survives across sessions. For new files scanned from unmanaged, we fetch `?ls` to get each file's W-tag before importing.
|
|
|
|
## TSV format specification
|
|
|
|
Tab-separated values. Each row = one managed content item. Columns:
|
|
|
|
| Col | Name | Type | Description | Example |
|
|
|---|---|---|---|---|
|
|
| 0 | `file_id` | string | copyparty W-hash of the file | `72EJcK7mxEGM2ygG` |
|
|
| 1 | `original_name` | string | Filename as uploaded to unmanaged (for display) | `IMG_1234.jpg` |
|
|
| 2 | `content_type` | enum | `image`, `video`, `animation` | `image` |
|
|
| 3 | `similarity_hash` | hex-string | phash output for fast nearest-neighbor search | `0a1b2c3d...` (64-char hex) |
|
|
| 4 | `base_colors` | string | Up to 4 hex colors, pipe-separated, each with percentage | `#f5a623_45,#234_15\|...` → TBD exact format |
|
|
| 5 | `album_uuid` | uuid or empty | v4 UUID string if assigned to album, empty otherwise | `550e8400-e29b-41d4-a716-446655440000` |
|
|
| 6 | `album_position` | int or empty | Zero-based position within the album group | `3` |
|
|
| 7 | `user_tags` | string | Comma-separated user-assigned tags, normalized to lowercase | `sunset,beach,golden hour` |
|
|
| 8 | `modified_at` | unix-timestamp | Epoch seconds of last metadata modification (not file creation) | `1784243166` |
|
|
|
|
File rows are appended on new-file scan. Full-file rewrite on bulk tag changes. The app maintains an in-memory index by `file_id` for O(1) lookup and avoids writing unchanged rows.
|
|
|
|
## Color format specification
|
|
|
|
Base colors extracted via quantization of the downsampled image (k-means or median-cut, TBD). Stored as pipe-separated entries: `{hex}_{percentage}`. Example:
|
|
|
|
```
|
|
#e85d04_34,#f48c06_28,#faed00_18,#fffef2_12,#ef7d00_1,#fcffb8_1,#ff004d_1
|
|
```
|
|
|
|
This permits up to N colors (implementation will decide) while keeping the CSV row parseable. Percentage sums ≈ 100%.
|
|
|
|
# UI Architecture
|
|
|
|
Single-page application. No build step — all JS/CSS served from `/public/` on copyparty. Three views:
|
|
|
|
1. **Login** (CWD selector) — directory input + Check/Enter buttons
|
|
2. **Tagging** (main work view) — untagged queue + tag editor
|
|
3. **Gallery** — browse filtered/sorted managed content
|
|
|
|
The app uses browser localStorage for persistence of CWD, password, and UI preferences. All content state lives server-side (copyparty filesystem). The TSV is the source of truth; the browser never "owns" data permanently.
|
|
|
|
```
|
|
### HEADER (top) ###
|
|
[last part of current working directory (centered)] [change working directory button (aligned right)]
|
|
|
|
### MAIN TAGGING UI ###
|
|
[open gallery button]
|
|
|
|
[grid of untagged images (images load into viewer when clicked)]
|
|
[previous item arrow] [image viewer] [next item arrow]
|
|
[album selector field (select from last albums, type to search, create new)]
|
|
[album position field (disabled unless an album is set)]
|
|
[tag input field (suggests existing, splits tags on comma)]
|
|
|
|
[similarity hash]
|
|
[base colors display (small)]
|
|
[delete file button]
|
|
```
|
|
|
|
```
|
|
### GALLERY ###
|
|
### HEADER (top) ###
|
|
[back button (to tagging ui)]
|
|
[tag filter picker (user tags/albums)]
|
|
[order-by picker (creation time/modification time/similarity/color similarity)]
|
|
[image grid (grouped by album)] [tag editor (sidebar) - allow for album/album position/user tag modifications when an image is clicked]
|
|
[context menu (right-click) - find similar by similarity hash/color, trash file]
|
|
```
|
|
|
|
# Tag types
|
|
|
|
0. DB modification time - timestamp of last tag modification time
|
|
1. Content type - image/video/animation
|
|
2. Similarity hash
|
|
3. Base colors
|
|
4. Album UUID - if set, content belongs to an ordered group of content items
|
|
5. № in album - position of the item inside the group
|
|
6. User tags - a comma-separated list of entries
|