Copyparty-content-tag/PLAN.md

# Copyparty Content Tag — Project Plan

## Phase 0: Infrastructure & Discovery (IN PROGRESS)

### ✅ Copyparty API Capability Tests (2026-07-16)

Test server: `localhost:8086`, user `user:12345`, config `copyparty-test-config.yaml`

#### Authentication
| Test | Endpoint | Result | Notes |
|------|----------|--------|-------|
| Anonymous listing | `GET /?ls` | ✅ PASS | Returns `acct: "*"`, read-only `/public/` only |
| PW header auth | `GET /?ls` + `PW: 12345` header | ✅ PASS | Returns `acct: "user"`, both routes visible |
| URL param auth | `GET /?ls&pw=12345` | ✅ PASS | Identical to header auth |
| Wrong password | `GET /private/?ls` + wrong PW | ✅ PASS → 403 | Forbidden on write volumes |

#### Permission model (confirmed)
- `/public/` → `perms: ["read","get"]` — serves app files, read-only
- `/private/user/` → `perms: ["read","write","move","delete","dot","get","admin"]` — full workspace

#### Directory operations
| Test | Endpoint | Result | Notes |
|------|----------|--------|-------|
| Mkdir (auto-parent) | `mPOST /content/?replace` + `act=mkdir&name=unmanaged` | ✅ PASS | Creates intermediate `/content/` automatically |
| JSON listing | `GET /path/?ls` | ✅ PASS | Full metadata: size, timestamp, w-hash, permissions |
| Dotfile listing | `GET /path/?ls&dots` | ✅ PASS | Includes hidden files/dirs |
| Plaintext listing | `GET /path/?ls=t` | ✅ PASS | Human-readable terminal format |
| Tree listing | `GET /path/?tree=.` | ✅ PASS | Returns nested dir structure as JSON |

#### File CRUD
| Test | Endpoint | Result | Notes |
|------|----------|--------|-------|
| PUT upload (new file) | `PUT /path/file.txt?j` + body | ✅ PASS | Returns `{filesz, fileurl, sha512}` |
| PUT overwrite | `PUT /path/file.txt?j` + `Replace: 1` header | ✅ PASS | **Must use header, NOT `?replace` param** |
| PUT dedup (no replace) | `PUT /path/file.txt?j` without Replace header | ✅ PASS | Creates timestamped copy — auto-dedup behavior |
| Append to file | `PUT /path/file.tsv?apnd&j` + body bytes | ✅ PASS | Appends raw bytes, returns new sha512 |
| Multipart upload | `mPOST /dir/?replace&j` + `-F f=@file.txt` | ✅ PASS | Returns `{files: [{fn, path, sha512, sz}]}` |
| Read as plaintext | `GET /path/file.tsv?txt` | ✅ PASS | Full file content in response body |
| Move file | `POST /src?move=/dst` | ✅ PASS | Source gone, dest appears at new path |
| Copy file | `POST /src?copy=/dst` | ✅ PASS | Both source and copy exist |
| Delete single | `POST /path/file?delete&j` | ✅ PASS | Returns textual confirm: "deleted 1 files" |
| Batch delete | `jPOST /?delete&j` + JSON array of abs paths | ✅ PASS | Deletes multiple paths at root level |

#### Search
| Test | Endpoint | Result | Notes |
|------|----------|--------|-------|
| Server search | `jPOST /?ls` + `{"q":"name=latest.tsv"}` | ⚠⚠ PARTIAL | Returns raw text lines, not filtered JSON listing. Useful for verification but NOT for app data fetching — use full `?ls` + client-side filter instead. |

#### File metadata (copyparty-provided)
Every file in `?ls` response includes:
- `href` — relative filename/path
- `sz` — size in bytes
- `ts` — unix timestamp
- `ext` — file extension
- `tags.w` — **W-hash** (base32 sha512-derived, server-computed file ID) ← **our primary key**
- `tags.up_by`, `tags.up_ip`, `tags._up_at` — upload provenance

#### Upload tracking
- `GET /?ups` — lists recent uploads by IP with human-readable format
- `GET /?ru&j` — server-wide recent uploads (JSON) when authed

### Key findings for implementation

1. **W-hash is our stable file ID** — no need to generate external identifiers. Survives moves within same volume.
2. **PUT overwrite requires `Replace: 1` HTTP header**, not URL parameter. Without it, copyparty auto-deduplicates (creates timestamped copy).
3. **TSV DB strategy**: Read with `?txt`, write with PUT + `Replace: 1` for full rewrite, or `?apnd` for append-only rows.
4. **Server search is unusable** for structured data retrieval — returns raw text lines. App must list dirs then filter client-side.
5. **Batch delete at root**—requires absolute paths and root-level jPOST call.

---

### ❌ Test cleanup completed
All test artifacts removed from `/private/user/`. Workspace clean except pre-existing `tag-data/` directory (27 files, 127MB — left untouched).

---

## Phase 1: Core app scaffold (COMPLETE)

- [x] HTML scaffold — `src/index.html` complete (Login, Tagging, Gallery views + context menu)
- [x] CSS — `src/style.css` complete (dark theme, all views styled)
- [x] Copyparty API wrapper — `.part1_config_auth.js` complete (Config, CP_CP, Auth)
- [x] TSV DB operations — `.part2_tsv_db.js` complete (parse, serialize, CRUD, UUID)
- [x] Scan + import pipeline — `.part3_scan_import.js` complete (content detect, dir init, import)
- [x] Bug fixes (2026-07-17):
  - Fixed `TAB.join(f)` → `f.join(TAB)` in part2 (was producing literal "Symbol" string)
  - Fixed `dbMap[fileId]` → `dbMap.has(fileId)` / `dbMap.set()` (part2 uses Map, part3 accessed as plain object)
  - Fixed missing `cwd` arg in `TsvDb.appendRow()` call in part3
  - Fixed `CP.createDir` / `CP.moveFile` → `CP_CP.createDir` / `CP_CP.moveFile` in part3
- [x] Smoke-test bug fixes (2026-07-17):
  - Fixed `_url()` double-slash when cwd starts with `/` — now normalizes base and path before joining
  - Fixed `createDir()` 405 crash on existing directory — now idempotent, accepts 405 as success
  - Fixed `checkPerms()` calling `CP.ls()` before `CP` assigned — changed to `this.ls()`
  - Fixed `deleteFile()` missing `&j` in query params — now matches verified endpoint
  - Fixed `writeFile()` ignoring `replace` param — now actually sets `Replace: 1` header when true
- [x] `app.js` assembled — Login view (Check/Enter), view switching, Tagging view (grid, viewer+nav, album/tag editors, delete), Gallery integration

## Phase 2: Content processing (COMPLETE)

- [x] Perceptual hash (phash) via canvas — 16x16 grayscale, median threshold → 64-char hex
- [x] Median-cut color extraction — up to 4 dominant colors with percentages

## Phase 3: Tagging UI (COMPLETE)

- [x] Image viewer + prev/next navigation (keyboard arrows too)
- [x] Album assigner with UUID generation
- [x] Tag input with autocomplete from existing tags
- [x] Delete button wired to copyparty delete API
- [x] Debounced save (800ms) on tag/album changes

## Phase 4: Gallery + filters (COMPLETE)

- [x] Gallery grid grouped by album (ungrouped section for no-album images)
- [x] Tag filter picker (real-time, debounced 250ms)
- [x] Similarity/color-based sorting
- [x] Context menu (find similar by hash/color, trash file)
- [x] Sidebar tag editor

## Phase 5: Test infrastructure (COMPLETE)

- [x] Curl sanity test suite (`tests/api-tests.sh`) — 12 tests covering auth, CRUD, mkdir
- [x] Node.js TSV unit tests (`tests/tsv-test.js`) — **37 tests, all passing**
- [x] Gitea Actions CI workflow (`.gitea-ci.yaml`)

## Remaining / Next Steps

- [ ] Deploy to copyparty and smoke-test in browser (requires running copyparty instance)
- [ ] Automated browser tests in Docker (Playwright/Puppeteer container)
- [ ] Album naming: currently uses raw UUIDs in album selector — consider a human-readable album name → UUID mapping
- [ ] `.part5_gallery.js` added to `index.html` script list (currently loaded dynamically via fetch in app.js; both approaches work)

## File Inventory

| File | Purpose | Lines |
|------|---------|-------|
| `src/index.html` | All 3 views markup + context menu | ~186 |
| `src/style.css` | Dark theme, layout for all views | ~170 |
| `src/.part1_config_auth.js` | Config, CP_CP API wrapper, Auth | ~220 |
| `src/.part2_tsv_db.js` | TSV parser/serializer, DB CRUD | ~240 |
| `src/.part3_scan_import.js` | Content detect, dir init, import | ~150 |
| `src/.part4_image_processing.js` | Phash + median-cut color extraction | ~295 |
| `src/.part5_gallery.js` | Gallery view logic | ~new |
| `src/app.js` | Main app: Login, Tagging, view switching | ~500 |
| `tests/api-tests.sh` | Curl-based API tests | ~new |
| `tests/tsv-test.js` | Node.js TSV unit tests (37 passing) | ~new |
| `.gitea-ci.yaml` | CI workflow | ~new |