Page 1 of 1

Create new stems for Unmix Stems

Posted: Mon Aug 03, 2020 5:48 pm
by edwardkort
Is there any capability to create different stem types (like guitar, flute, etc.) for use in the Unmix Stems procedure? If not now, any plans for the future?

Thanks,
Edward Kort

Re: Create new stems for Unmix Stems

Posted: Mon Aug 03, 2020 6:18 pm
by Robin Lobel
These unmixing process are driven by deep learning AI models, which requires thousands of examples for each instrument and days of computing to train proper separation model. Experiments were made to train it on flute and guitar, but unfortunately it does not work as well as flute is pretty close to vocals in term of spectral signature, and guitar spectral signature is so wide that it's hard to properly capture it and discriminate it from say piano for instance. That being said, it may evolve in the future.

Re: Create new stems for Unmix Stems

Posted: Mon Aug 03, 2020 7:17 pm
by edwardkort
Thank you for the quick reply.

Cheers,
Edward Kort

Re: Create new stems for Unmix Stems

Posted: Mon Aug 03, 2020 9:08 pm
by nkf
Robin Lobel wrote:
Mon Aug 03, 2020 6:18 pm
These unmixing process are driven by deep learning AI models, which requires thousands of examples for each instrument and days of computing to train proper separation model. Experiments were made to train it on flute and guitar, but unfortunately it does not work as well as flute is pretty close to vocals in term of spectral signature, and guitar spectral signature is so wide that it's hard to properly capture it and discriminate it from say piano for instance. That being said, it may evolve in the future.
I just want to say, that I appreciate these kind of answers from a developer - no nonsensical promises, but straight to the facts and limitations of the technology. Well done.

Re: Create new stems for Unmix Stems

Posted: Mon Aug 03, 2020 10:18 pm
by HowlingUlf
nkf wrote:
Mon Aug 03, 2020 9:08 pm
Robin Lobel wrote:
Mon Aug 03, 2020 6:18 pm
These unmixing process are driven by deep learning AI models, which requires thousands of examples for each instrument and days of computing to train proper separation model. Experiments were made to train it on flute and guitar, but unfortunately it does not work as well as flute is pretty close to vocals in term of spectral signature, and guitar spectral signature is so wide that it's hard to properly capture it and discriminate it from say piano for instance. That being said, it may evolve in the future.
I just want to say, that I appreciate these kind of answers from a developer - no nonsensical promises, but straight to the facts and limitations of the technology. Well done.
Yepp! Now we know that! 8-)

Re: Create new stems for Unmix Stems

Posted: Tue Aug 04, 2020 5:41 pm
by JesusSheep00a
Robin Seems to have Mic Bleed .Instruments are getting left in Vocal Layer.

Re: Create new stems for Unmix Stems

Posted: Tue Aug 04, 2020 9:08 pm
by Robin Lobel
A future update could add sensitivity parameters for each instrument.

Re: Create new stems for Unmix Stems

Posted: Wed Aug 05, 2020 3:18 am
by JesusSheep00a
Robin Check out this reaper video and download : https://youtu.be/L3iZSIC5XrE:)

Re: Create new stems for Unmix Stems

Posted: Wed Aug 05, 2020 3:21 am
by JesusSheep00a
Also add add more instruments In AI.Like Keyboards ,Synths..
You could add a way that instruments could be added to the AIs collection of instruments.

Re: Create new stems for Unmix Stems

Posted: Wed Aug 05, 2020 10:47 am
by Howl
Robin Lobel wrote:
Tue Aug 04, 2020 9:08 pm
A future update could add sensitivity parameters for each instrument.
that is great. perhaps, spectralayers has the ability, by hand, but unmix components, you can in way transpose to unmix stems, or layers of sounds. by amplitude, transients, etc. enough under the hood, to extract frequencies, like factorsynth 2 (M4L device, i own it, so perhaps..).
by hand is to most "creative" way, but dissecting a waveform in spectral components, can deliver new ideas. this is more for sounddesign of course.
and in a way all the tools are already there, i must work with spectralayers pro 7 more, if it would be real addition.
perhaps a niche question and yes, the tools available, i repeat are perhaps enough, it is more for another workflow, another perspective.
perhaps it is only me...

Re: Create new stems for Unmix Stems

Posted: Wed Aug 05, 2020 4:17 pm
by JesusSheep00a
Have SpectraLayers Be able to add addional instruments to is AI .It is now calling Synth Other.Will it be able to Put all pianos in the piano stem Same with Drums ect?

Re: Create new stems for Unmix Stems

Posted: Wed Aug 12, 2020 5:01 pm
by Robbie D
Robin Lobel wrote:
Mon Aug 03, 2020 6:18 pm
These unmixing process are driven by deep learning AI models, which requires thousands of examples for each instrument and days of computing to train proper separation model. Experiments were made to train it on flute and guitar, but unfortunately it does not work as well as flute is pretty close to vocals in term of spectral signature, and guitar spectral signature is so wide that it's hard to properly capture it and discriminate it from say piano for instance. That being said, it may evolve in the future.
Hi Robin,

Often, when I want to do this kind of work, I may have a small sample of the same instrument I want to isolate/extract/unmix (maybe from a intro, break, or different take).

Is it possible you could add the option to “feed” the AI a clean snippet of the instrument you want to unmix to give a more accurate result?

Re: Create new stems for Unmix Stems

Posted: Sat Nov 21, 2020 4:32 am
by Geppastro
Robin Lobel wrote:
Mon Aug 03, 2020 6:18 pm
These unmixing process are driven by deep learning AI models, which requires thousands of examples for each instrument and days of computing to train proper separation model. Experiments were made to train it on flute and guitar, but unfortunately it does not work as well as flute is pretty close to vocals in term of spectral signature, and guitar spectral signature is so wide that it's hard to properly capture it and discriminate it from say piano for instance. That being said, it may evolve in the future.
About guitars: I think of electric and acoustic ones as sonically different instruments. Maybe concentrating (initially at least) on just acoustics would be a little easier? Acoustics I would think have a less wide spectral signature. At the same time, they are present in a lot of recordings, maybe steel strings ones more so. Wouldn' t a gradual approach (i.e. starting for example with just acoustic steel strings) be more feasible?