You Cannot Make the Beat First
Testing whether a local music model can fix a backing track and add melody afterwards. An identical supplied score reproduces the composition exactly and the performance not at all — correlation 0.133 against a 0.95 same-performance reference.
TL;DR — You cannot build a beat in YuE2 and then sing over it. Supplying the identical score — verified byte-identical — produces a backing track that correlates with the original at 0.133, where 0.95 is what the same performance actually looks like.
Every producer works the same way: lay down drums and bass, get the groove right, then write the melody on top of it. I wanted to know whether YuE2 could work that way too. The short answer is no, and the useful part is exactly where it breaks.
Attempt one: ask for an instrumental
Every guide to this model says the same thing — leave the lyrics field empty and you get an instrumental. I did that, then split the result into a vocal stem and a backing stem and measured each.
The vocal stem came back at an RMS of 0.0934, against 0.0872 for the same song generated with lyrics. Thirty-eight percent of the energy is voice, versus forty-one percent for the version that had words to sing.
An empty lyrics field does not mean "play the backing." It means "sing without words." The score still had 371 notes written into its vocal part; the model just had nothing to put on them, so it hummed. The advice is not wrong in general — it comes from the browser demo, which behaves differently — but it does not hold in the Python runtime.
Attempt two: hand the second render the first one's score
If I cannot get a backing track out, maybe I can get the same backing track twice. YuE2 writes a readable score before it renders anything, and you can hand that score back to it. So: generate once, take the score, generate again with the same score and the lyrics added.
The score transferred perfectly.
| Render | BPM | Score length | Vocal notes | Chords | Score SHA-256 |
|---|---|---|---|---|---|
| A — lyrics empty | 116 | 175.9 s | 371 | 85 | 0f5984530f9d |
| B — A's score plus lyrics | 116 | 175.9 s | 371 | 85 | 0f5984530f9d |
| C — control, no score supplied | 116 | 136.6 s | 254 | 64 | 8b7d4a491867 |
Same hash. The project's own comparison tool reports match: true with zero differences. Whatever else is going on, the composition is identical.
Then I split both renders and correlated the backing stems.
| Backing stems compared | Waveform correlation |
|---|---|
| A vs B — identical score | +0.133 |
| A vs C — different scores | +0.027 |
| B vs C — different scores | +0.036 |
| reference: same latents, different decoder | +0.954 to +0.991 |
0.133. Two renders of a byte-identical composition share almost nothing at the waveform level. The bottom row is the calibration: when I decoded the same cached audio through two different decoders in an earlier test, correlation ran from 0.954 to 0.991. That is what one performance heard twice looks like. This is not that.
The frequency balance shifts too. Sub-bass energy goes from 23.9% to 18.4% and the 60-120 Hz band from 37.9% to 45.8% — the weight distribution between the kick and the bass is simply different.
What a score actually buys you
The documentation is explicit, and I had read it before running any of this: "Editing style, lyrics or ABC requires new semantic generation and synthesis. Cached latents are reusable when changing only the decoder, not when changing the music."
Supplying a score fixes what is played — every pitch, every duration, every chord symbol, the meter, the tempo, the section boundaries. It does not fix how it sounds. Those are two different layers, and only one of them is under your control.
The player-piano comparison is almost exact. A piano roll determines every note and every duration, and tells you nothing about the instrument it will be fed into.
The honest capability list
| What a producer would want | Available in YuE2 |
|---|---|
| Render a beat, keep it, sing over it | No — there is no audio input of any kind |
| Render backing only | No — an empty lyrics field produces humming, not an instrumental |
| Reuse a backing track across takes | No — identical scores give a backing correlation of 0.133 |
| Get stems out | No — one mixed file per render, nothing in the runtime emits parts |
| Fix the melody and harmony exactly | Yes — the score round-trips to the byte |
| Change one passage and leave the rest | Yes — measured surgical in an earlier test |
The last two rows are worth holding onto. In an earlier test I edited thirty-two notes in the final chorus of a finished song and re-rendered: the edited passage moved up 1.8 semitones and two control windows measured identical to the hertz. Score-level control is real and precise. It is just control over the composition, not over the recording.
So what do you do instead
Two options, neither of them what you wanted.
Post-production. Render twice, separate both, and paste one render's backing under another render's vocal. This works, but it is editing rather than generating, and you pay for it in whatever the source separation smears.
A different model. Something that takes audio as a conditioning input, or emits stems. YuE2 does neither, and the runtime has no code path that would.
YuE2 is a composition tool, not an arrangement tool. It will write you a song and perform it. It will not hand you the parts.
Appendix: run it yourself
The three calls and the check that they did what I claim. Full numbers and the fixed/not-fixed breakdown are in the Axortex version.
Reproducing the test
# A - ask for an instrumental the way every guide says to
pipe(style=STYLE, lyrics="", cot="full", seed=871001).save_artifacts("out/A")
# B - inherit A's score exactly, add the words
abc = Path("out/A/score.abc").read_text(encoding="utf-8")
pipe(style=STYLE, lyrics=LYRICS, cot="full", seed=871001, abc=abc).save_artifacts("out/B")
# C - control: same style, same lyrics, same seed, no score supplied
pipe(style=STYLE, lyrics=LYRICS, cot="full", seed=871001).save_artifacts("out/C")
# confirm B really inherited A
python skills/yue2-music/scripts/abc_tools.py compare out/A/score.abc out/B/score.abc --voices Vocal
# -> {"match": true, "differences": []}
# split each render and correlate the backing stems
python -m demucs --two-stems=vocals -n htdemucs -o sep out/A/audio.flac out/B/audio.flac
Sources: YuE2 repository, generation and covers reference, generation docs.
Image: William Braid White, sectional illustration of player piano interior action, 1909, Public domain, via Wikimedia Commons.
← Back to all posts