● Work in progress
Controllable Music Upmixing
Source & reference · 5s excerpt · for research illustration only
reference original stereo
Prompt-driven upmixes
Same mono input, different text prompts — the model re-spatializes per instrument from the prompt. Headphones recommended.
wide pads, tight rhythm
upmix result
drum room
upmix result
intimate vocal, vast rest
upmix result
make it surround sound — vocals circling
upmix result
● Work in progress
Dominant-Note–Conditioned Generation & Editing
Early experiments on music generation and editing using a simple harmonic control signal: the smoothed dominant note estimated from a chromagram.
How it works
- Generation: input a chromagram → compute a temporally smoothed dominant-note trajectory → generate 44.1 kHz stereo audio conditioned on that trajectory.
- Editing: take any source audio → extract its chromagram → set / modify the dominant-note condition → perform inversion-based editing that keeps part of the original character while steering harmonic content toward the target notes. Intended as a musician-facing editing tool that can operate on full mixed songs (not just isolated stems), where traditional DAW workflows struggle to “surgically” decouple and rewrite harmonic content.
Disclaimer. The model is trained on a licensed dataset for
research in generative music. The editing demos include short excerpts from songs I personally enjoy; these are
used only for my own testing and are not used for training.
⚠️ Playback note: if the videos don’t play in Microsoft Edge, please open
this page in Chrome or another browser.
✓ Published ·
IEEE Big Data 2024
Graph Neural Network Guided Music Mashup Generation
Original 1 provides the vocals and instrumental; Original 2 provides an instrumental. A graph neural network rearranges and fuses the two instrumentals into the mashup backing track.
Model overview.
Example 1
Original 1vocals + instr.
Original 2instrumental
MashupGNN output
Example 2
Original 1vocals + instr.
Original 2instrumental
MashupGNN output
Example 3
Original 1vocals + instr.
Original 2instrumental
MashupGNN output
Example 4
Original 1vocals + instr.
Original 2instrumental
MashupGNN output
Example 5
Original 1vocals + instr.
Original 2instrumental
MashupGNN output
✓ Published ·
IEEE Big Data 2024
Diffusion Models for Automatic Music Mixing
A diffusion model takes a multi-track recording with uneven track levels and rebalances it into a coherent mix. Compare the imbalanced input against the model’s output.
Example 1
Imbalanced input
Diffusion mix
Example 2
Imbalanced input
Diffusion mix
Example 3
Imbalanced input
Diffusion mix
Selected publications
- An Automated Pop Song Mashup System Using Drum Swapping — Xinyang Wu, Andrew Horner. Proc. Meetings on Acoustics (ASA/POMA), 2023.
- Graph Neural Network Guided Music Mashup Generation — Xinyang Wu, Andrew Horner. IEEE Big Data, 2024.
- Diffusion Models for Automatic Music Mixing — Xinyang Wu, Andrew Horner. IEEE Big Data, 2024.
- Learning to Rank Music Mashups — Xinyang Wu, Andrew Horner. Proc. Meetings on Acoustics (ASA/POMA), 2025.