Feature Description: A centralized Prompt-Based Video Studio dialog for Pitivi (accessible via the aivfx / Win+F9 action) that offers two distinct modes: one for creative generation and one for precise, prompt-based editing.
1. Add AI VFX (The Creative Engine)
-
Prompt-to-Overlay: Users input natural language prompts (e.g., “cinematic rain streaks” or “retro CRT glitches”) to generate unique assets.
-
Seamless Integration: The system uses AI matting to automatically remove backgrounds from generated content, placing it as a transparent layer over the current timeline selection.
-
Smart Duration: The generated effect automatically matches the duration of the selected clip on the timeline.
2. General Effects / AI Assistant (The Precision Engine)
-
Semantic Editing: This mode translates text commands into GES (GStreamer Editing Services) API calls.
-
Example: A user types “Make this clip grayscale and cut the last 2 seconds.”
-
Execution: The AI doesn’t “hallucinate” the video; it programmatically applies the
videobalancefilter and adjusts the clip’sdurationproperty.
-
-
Frame-Perfect Results: Since this uses Pitivi’s underlying engine, it guarantees the 100% precision that professional editors require.
-
Safety Stack: Every action taken by the AI Assistant is recorded in Pitivi’s standard Undo/Redo history, allowing users to revert any automated change instantly.
Technical Implementation:
-
UI: A dedicated GTK4 dialog (as seen in my current MR prototype) to handle input and parameter tweaking.
-
Backend: Leveraging GStreamer 1.28’s inference elements for local model execution to ensure user privacy and offline functionality.
-
Bridge: A Python-based mapping layer that converts NLP (Natural Language Processing) intent into Pitivi-specific method calls.
Motivation: It helps beginners perform complex tasks quickly while providing advanced users with a “shortcut” for tedious manual edits.
Happy to discuss feasibility, help prototype, or implement a proof-of-concept. What do maintainers think about adding AI/ML features?