RUWIO/dev_docs/electron-rendering-integration-plan.md

Electron Rendering Integration Plan

Date: 2026-06-05

Verdict

Yes, the MP4 rendering part can be included in the Electron app.

The current render server is already made from desktop-compatible pieces:

• Node orchestration in server/server.js
• a browser render surface in server/render.html
• frame capture through Chromium
• MP4 encoding through ffmpeg
• progress, cancellation, temp files, validation, and cleanup around the job

The main caveat is that the server cannot be bundled as-is and expected to work in a packaged desktop app. It currently assumes:

• puppeteer-core is installed in the server package
• Chromium exists at PUPPETEER_EXECUTABLE_PATH or /usr/bin/chromium
• ffmpeg exists on PATH
• the Vue app talks to http://localhost:6000 with VITE_RENDER_SERVER
• Electron Builder packages only dist/, dist-electron/, and root package.json

So the feature is feasible, but the right implementation is to move the render job into the Electron main process behind IPC, not to expose an HTTP render server from the desktop app.

Recommended Architecture

Use Electron as the local render host:

• Vue renderer keeps owning project state and export UI.
• Preload exposes a narrow desktop render API on window.ruwioDesktop.
• Electron main validates render requests, starts/cancels jobs, captures frames, runs ffmpeg, and returns a saved MP4 path or file bytes.
• The standalone server/ package remains available for web or self-hosted server deployments.

Preferred render surface:

• Use a hidden/offscreen Electron BrowserWindow to load a packaged render page.
• Call webContents.executeJavaScript or webContents.send to initialize the scene and seek each frame.
• Use webContents.capturePage() for PNG frame capture.
• Pipe PNG frames into ffmpeg the same way server/server.js does today.

This avoids bundling a second Chromium through Puppeteer. Electron already ships Chromium, so the desktop build only needs to solve ffmpeg packaging.

Fallback/spike option:

• Start the existing Express render server as a child process bound to 127.0.0.1 with a random token.
• Point the renderer at that local URL.
• This is useful for a quick feasibility spike, but it keeps the localhost server, token plumbing, Puppeteer, and external Chromium dependency in the desktop app. It should not be the final architecture.

Dependency Strategy

Chromium

Do not add a separate Chromium binary for the first native implementation. Use Electron's bundled Chromium through a hidden render window.

Only keep Puppeteer for the standalone server unless a spike proves Electron capture cannot match render quality or performance.

ffmpeg

ffmpeg is the real packaged-app dependency.

Implementation options:

1. Bundle per-platform ffmpeg binaries as extraResources.
2. Use a maintained npm package that supplies platform-specific binaries.
3. Support a user-configured/system ffmpeg path as a fallback.

Recommended first production path:

• Add an internal resolver such as electron/ffmpegPath.cts.
• In development, prefer process.env.FFMPEG_PATH, then ffmpeg on PATH.
• In packaged builds, prefer process.resourcesPath/bin/<platform>/ffmpeg.
• Show a friendly export error if no usable binary is found.

Implementation Plan

Phase 1: Extract Shared Render Validation

Goal: keep server and desktop limits identical.

Tasks:

1. Move request validation and playback range helpers out of server/server.js into a shared module.
2. Keep the module Node-compatible and browser-safe where practical.
3. Export:
   • validateRenderRequest(scene, options)
   • playbackRange(scene)
   • render limits and allowed ffmpeg presets
4. Update server/server.js to use the shared validator.
5. Add focused tests for invalid dimensions, fps, duration, frame count, layers, keyframes, crf, and preset.

Acceptance:

• Existing server behavior is unchanged.
• Desktop rendering can reuse the same validation before launching any render surface or child process.

Phase 2: Add Desktop Render IPC Contract

Goal: let the Vue app request MP4 rendering without knowing whether the backend is HTTP or Electron.

Tasks:

1. Extend src/types/desktop.d.ts with a render API:

renderMp4(options: {
  scene: Scene
  fps?: number
  crf?: number
  preset?: string
  suggestedName?: string
}): Promise<{ path: string } | null>
onRenderProgress(callback: (event: {
  jobId: string
  progress: number
  done: boolean
  error?: string
}) => void): () => void
cancelRender(jobId: string): Promise<void>

2. Add preload wrappers that call ipcRenderer.invoke and subscribe to progress events.
3. Add main-process handlers:
   • ruwio:render-mp4
   • ruwio:render-cancel
4. Validate every IPC payload in main before starting a job.
5. Keep job IDs and abort controllers in main, not in the Vue renderer.

Acceptance:

• A renderer call can start a fake/stub render and receive progress.
• Cancel reaches the main process.
• Invalid payloads fail before any child process or hidden window is created.

Phase 3: Build The Electron Render Worker

Goal: replace Puppeteer with Electron's own Chromium for desktop exports.

Tasks:

1. Create an Electron-only render worker module, for example electron/renderWorker.cts.
2. Package a render page:
   • first pass: reuse server/render.html as an extra resource
   • later pass: consider generating a render page from the shared browser renderer to reduce drift
3. Create a hidden BrowserWindow per job or a serialized reusable worker window:
   • show: false
   • webPreferences.sandbox: true
   • no Node integration
   • restricted navigation
4. Load the render page from a local file URL.
5. Initialize the scene.
6. For each frame:
   • seek to the target time
   • wait for the render function to settle
   • capture the page or canvas bounds
   • write PNG bytes to ffmpeg.stdin
   • emit progress over IPC
7. On cancel or timeout:
   • abort the frame loop
   • kill ffmpeg
   • close the worker window
   • remove temp files

Acceptance:

• A tiny scene renders to MP4 in npm run desktop:dev.
• Progress updates reach ExportDialog.
• Cancel stops capture and ffmpeg.
• Temp directories are removed on success, failure, and cancel.

Phase 4: Package ffmpeg

Goal: make MP4 export work in packaged desktop builds.

Tasks:

1. Choose the binary source:
   • committed/release-managed binaries under a vendor/ or bin/ folder, or
   • npm package binaries copied into Electron Builder resources
2. Add Electron Builder extraResources entries for:
   • Linux ffmpeg
   • Windows ffmpeg.exe
   • macOS ffmpeg
   • render worker HTML/assets
3. Add ffmpeg path resolution for dev and packaged builds.
4. Verify executable permissions on macOS/Linux packaged output.
5. Document licensing obligations for the selected ffmpeg build.

Acceptance:

• npm run desktop:pack includes the render page and ffmpeg.
• The unpacked app exports MP4 without a separately installed render server.
• Missing ffmpeg produces a clear user-facing error.

Phase 5: Switch ExportDialog To Desktop Rendering

Goal: use native rendering in Electron and keep the current server path for the web app.

Tasks:

1. Add src/lib/desktopRenderExport.ts.
2. In ExportDialog.vue, route MP4 export as:
   • if window.ruwioDesktop?.renderMp4 exists: use desktop IPC render
   • else: use current downloadFromServer
3. Reuse current progress UI and cancellation behavior.
4. Save the finished MP4 through a native save dialog from main.
5. Keep VITE_RENDER_SERVER support for browser builds and remote/self-hosted workflows.

Acceptance:

• Desktop users do not need cd server && npm run start.
• Browser users can still use the external render server.
• The MP4 button reports a helpful error when no server is configured in browser mode.

Phase 6: Keep Standalone Server Working

Goal: avoid breaking existing server deployments.

Tasks:

1. Keep server/server.js as an Express wrapper around shared validation/render helpers where useful.
2. Keep server/render.html or replace it only after desktop and server render outputs are compared.
3. Preserve /health, /render, /events, /file, and DELETE behavior.
4. Add a smoke test or manual check for server MP4 export after shared refactors.

Acceptance:

• cd server && npm run start still starts the local render server.
• The Vue app can still render through VITE_RENDER_SERVER and VITE_RENDER_TOKEN.

Phase 7: Verification Matrix

Run these before calling the integration complete:

• npm run build
• npm run desktop:pack
• npm run test:render-server
• npm run test:render-server:e2e
• desktop dev MP4 export of a small scene
• desktop packaged MP4 export of a small scene
• cancel an MP4 export after progress starts
• export a scene with image assets
• export a scene with text/fonts
• export a scene at non-default fps
• export a scene with hidden layers skipped
• server-mode MP4 export still works

Manual platform checks:

• Linux AppImage
• Windows NSIS build
• macOS DMG build

Risks And Mitigations

• ffmpeg packaging varies by platform: solve with explicit binary resolution and an unpacked-app smoke check before installer work.
• render drift between server/render.html and src/lib/render.ts: keep parity tests, then consider generating the desktop/server render page from shared code.
• large scenes can freeze the desktop app: keep render work in main-controlled worker windows, serialize jobs initially, enforce the same render limits as the server.
• IPC can move large payloads: scenes are JSON-sized today, but avoid sending frame buffers over IPC; frame capture and ffmpeg piping should stay in main.
• untrusted project assets reach Chromium: keep the hidden render window sandboxed, restrict navigation, block external network requests unless explicitly allowed, and keep existing project validation work moving.

Suggested First Pull Request

Keep the first PR small and proof-oriented:

1. Extract shared render validation from server/server.js.
2. Add a stubbed desktop render IPC contract.
3. Add desktopRenderExport.ts and route the MP4 button to desktop IPC when available.
4. Return a fake progress/done result in development or behind a temporary flag.

That proves the app integration and cancellation shape before dealing with frame capture and ffmpeg packaging.

Suggested Second Pull Request

Implement the real Electron render worker:

1. Hidden render window.
2. Frame capture loop.
3. ffmpeg stdin piping.
4. Temp directory cleanup.
5. Dev-only ffmpeg path resolution.

Suggested Third Pull Request

Make it shippable:

1. Package ffmpeg and render worker assets.
2. Verify desktop:pack.
3. Add packaged-app manual test notes.
4. Document ffmpeg licensing and update README desktop export notes.