LyGen is a lyric video website that reimagines the lyric video creation process by balancing control and creative freedom.
Modern lyric video making tools are either too complicated (e.g., After Effects), where users spend the most time making non-creative decisions, or too simplistic but with extremely limited creative controls. LyGen automates tedious processes like lyric syncing, allowing users to focus on expression. Meanwhile, with innovative features such as the “Importance Curve” and MIDI integration, users can dynamically set lyric emphasis in sync with music, either through MIDI playback or live recording.
LyGen took shape during Cornell's BURE CS Summer Research Program, after which it was my student research project advised by Prof. Abe Davis.
Key Features
• Automate Tedious Work
LyGen streamlines the lyric video creation process by automating repetitive tasks, such as syncing lyrics with audio. This frees users from spending excessive time on non-creative details, allowing them to focus on designing a visually engaging experience.
• Importance Curve
The Importance Curve allows users to adjust the emphasis of individual words or lines, dynamically enhancing the visual flow of the video. Each word’s importance can be set through an intuitive curve interface, giving users precise control over the expression and impact of the lyrics.
• Lyrical Instrument
The Lyrical Instrument is LyGen’s defining feature, letting users customize the meaning of “importance.” Through this tool, users can link various effects—such as text size, color changes, or animations—to different importance levels, enabling them to create visuals that capture the song’s emotional tone. Users can also save custom instruments for future projects.
• MIDI Integration (WIP)
The MIDI feature will bring a new level of interactivity to LyGen, allowing users to control timing and emphasis directly through a MIDI keyboard. By pressing keys in sync with the song, users can assign importance to lyrics based on keystroke velocity, making it easy to convey musical dynamics and rhythm visually.
Frameworks
Frontend:
General Framework: React + Next.js
UI: Material UI
Canvas: Fabric.js
Animation: Anime.js
Backend: Flask + Musixmatch API