From 87f463a75e9a1780e12e850826e3442dfb844534 Mon Sep 17 00:00:00 2001 From: Ted Moore Date: Tue, 7 Jun 2022 15:58:48 +0100 Subject: [PATCH 1/5] =?UTF-8?q?=E2=9A=A0=EF=B8=8F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- doc/BufNNDSVD.rst | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/doc/BufNNDSVD.rst b/doc/BufNNDSVD.rst index bb95b5e..4585a74 100644 --- a/doc/BufNNDSVD.rst +++ b/doc/BufNNDSVD.rst @@ -3,10 +3,12 @@ :sc-categories: FluidManipulation :sc-related: Classes/FluidBufNMF :see-also: -:description: - Find Initial Bases and Activations for FluidBufNMF via Non-Negative Double Singular Value Decomposition . +:description: Find Initial Bases and Activations for BufNMF +:discussion: - See http://nimfa.biolab.si/nimfa.methods.seeding.nndsvd.html + BufNNDSVD can help + + via Non-Negative Double Singular Value Decomposition .See http://nimfa.biolab.si/nimfa.methods.seeding.nndsvd.html @@ -37,6 +39,20 @@ :control method: The method used for the decomposition. Options are: + + :enum: + + :0: + NMF-SVD + + :1: + NNDSVDar + + :2: + NNDSVDa + + :3: + NNDSVD :control windowSize: @@ -49,4 +65,3 @@ :control fftSize: The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision. The -1 default value will use the next power of 2 equal or above the highest of windowSize and (bandwidth - 1) * 2. - From 0c0ff5421ee1f568114c92ea14a9e0f06c9f3248 Mon Sep 17 00:00:00 2001 From: Ted Moore Date: Tue, 7 Jun 2022 19:39:42 +0100 Subject: [PATCH 2/5] BufNNDSVD RST & SC examples --- doc/BufNNDSVD.rst | 28 +++++++++++++--------------- example-code/sc/BufNNDSVD.scd | 29 ++++++++++++++++++++--------- 2 files changed, 33 insertions(+), 24 deletions(-) diff --git a/doc/BufNNDSVD.rst b/doc/BufNNDSVD.rst index 4585a74..d07f792 100644 --- a/doc/BufNNDSVD.rst +++ b/doc/BufNNDSVD.rst @@ -6,31 +6,29 @@ :description: Find Initial Bases and Activations for BufNMF :discussion: - BufNNDSVD can help + BufNNDSVD uses Nonnegative Double Singular Value Decomposition which can help decide how to initialise BufNMF, by suggesting how many components to request (and what bases to seed) in order to account for a certain percentage of the variance in a buffer. - via Non-Negative Double Singular Value Decomposition .See http://nimfa.biolab.si/nimfa.methods.seeding.nndsvd.html - - + See http://nimfa.biolab.si/nimfa.methods.seeding.nndsvd.html and https://www.sciencedirect.com/science/article/abs/pii/S0031320307004359 for more info. :control source: - The index of the buffer to use as the source material to be decomposed through the NMF process. The different channels of multichannel buffers will be processing sequentially. + The index of the buffer to analyse and suggest a number of components for. :control bases: - The index of the buffer where the different bases will be written to and/or read from: the behaviour is set in the following argument. + The index of the buffer where the bases will be written to. These will be the bases and are suggested seed for a BufNMF processes. The number of bases (i.e., channels) in this buffer when the process is complete is the number of components needed to cover the percentage of variance in the buffer. :control activations: - The index of the buffer where the different activations will be written to and/or read from: the behaviour is set in the following argument. + Must be set as an empty buffer (of any size), however no information is returned in this buffer. :control minComponents: - Minimum number of estimated components + Minimum number of estimated components to return (minimum number of channels in ``bases`` buffer when the process is complete) :control maxComponents: - Maximum number of estimated components + Maximum number of estimated components to return (maximum number of channels in ``bases`` buffer when the process is complete) :control coverage: @@ -43,24 +41,24 @@ :enum: :0: - NMF-SVD + **NMF-SVD:** Nonnegative Double Singular Value Decomposition :1: - NNDSVDar + **NNDSVDar:** Nonnegative Double Singular Value Decomposition where any elements that are zero are filled with a random value between 0 and the average * 0.01. :2: - NNDSVDa + **NNDSVDa:** Nonnegative Double Singular Value Decomposition where any elements that are zero are filled with the average value. :3: - NNDSVD + **NNDSVD:** Nonnegative Double Singular Value Decomposition :control windowSize: - The window size. As spectral differencing relies on spectral frames, we need to decide what precision we give it spectrally and temporally, in line with Gabor Uncertainty principles. http://www.subsurfwiki.org/wiki/Gabor_uncertainty + The window size. We need to decide what precision we give it spectrally and temporally, in line with Gabor Uncertainty principles. http://www.subsurfwiki.org/wiki/Gabor_uncertainty :control hopSize: - The window hop size. As sinusoidal estimation relies on spectral frames, we need to move the window forward. It can be any size but low overlap will create audible artefacts. The -1 default value will default to half of windowSize (overlap of 2). + The window hop size. It can be any size but low overlap will create audible artefacts. The -1 default value will default to half of windowSize (overlap of 2). :control fftSize: diff --git a/example-code/sc/BufNNDSVD.scd b/example-code/sc/BufNNDSVD.scd index c892ab1..1cd3ac0 100644 --- a/example-code/sc/BufNNDSVD.scd +++ b/example-code/sc/BufNNDSVD.scd @@ -1,31 +1,42 @@ code:: + ( -b = Buffer.read(s,FluidFilesPath("Nicol-LoopE-M.wav")); +~src = Buffer.read(s,FluidFilesPath("Nicol-LoopE-M.wav")); ~bases = Buffer.new(s); ~activations = Buffer.new(s); ~resynth = Buffer.new(s); ) -//how many bases do I need to decompose the buffer with 90% accuracy +//how many bases do I need to decompose the buffer while accounting for 90% of the variance? ( Routine{ - FluidBufNNDSVD.process(s, b, ~bases, ~activations, coverage: 0.9, method: 1).wait; + FluidBufNNDSVD.process(s, ~src, ~bases, ~activations, coverage: 0.9, method: 1).wait; "% bases".format(~bases.numChannels).postln; }.play; ) -//check how many bases we are returned: - -//try the same process with less accuracy +//try the same process with less of the variance preserved ( Routine{ - FluidBufNNDSVD.process(s, b, ~bases, ~activations, coverage: 0.5).wait; + FluidBufNNDSVD.process(s, ~src, ~bases, ~activations, coverage: 0.5).wait; "% bases".format(~bases.numChannels).postln; }.play ) +// peek at the bases +~bases.plot; + //use the bases to run NMF on -FluidBufNMF.process(s, b, resynth: ~resynth, bases: ~bases, activations: ~activations,actMode: 2, components: ~bases.numChannels, action: {\done.postln;}) -{PlayBuf.ar(~resynth.numChannels, ~resynth)[2]}.play +FluidBufNMF.process(s, ~src, resynth: ~resynth, resynthMode:1, bases: ~bases, basesMode:1, activations: ~activations, components: ~bases.numChannels, action: {"done".postln;}) + +// peek at the components +FluidWaveform(~resynth,bounds:Rect(0,0,1000,1000)); + +// listen to component index 2: +( +{ + PlayBuf.ar(~resynth.numChannels, ~resynth)[2] +}.play +) :: From 6e3a45e682228d973b9dbccaf8ee7aa1c6cdb4f4 Mon Sep 17 00:00:00 2001 From: Ted Moore Date: Wed, 8 Jun 2022 10:15:40 +0100 Subject: [PATCH 3/5] NNDSVD RST with descriptions of method --- doc/BufNNDSVD.rst | 18 +++++++++--------- example-code/sc/BufNNDSVD.scd | 3 +++ 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/doc/BufNNDSVD.rst b/doc/BufNNDSVD.rst index d07f792..cb5b124 100644 --- a/doc/BufNNDSVD.rst +++ b/doc/BufNNDSVD.rst @@ -6,21 +6,21 @@ :description: Find Initial Bases and Activations for BufNMF :discussion: - BufNNDSVD uses Nonnegative Double Singular Value Decomposition which can help decide how to initialise BufNMF, by suggesting how many components to request (and what bases to seed) in order to account for a certain percentage of the variance in a buffer. + BufNNDSVD uses Nonnegative Double Singular Value Decomposition which can help decide how to initialise BufNMF, by suggesting how many components to request (and what bases and activations to seed) in order to account for a certain percentage of the variance in a buffer. In general, using this process to seed a BufNMF decomposition should substantially increase the speed with which BufNMF converges and avoid especially poor local minima. See http://nimfa.biolab.si/nimfa.methods.seeding.nndsvd.html and https://www.sciencedirect.com/science/article/abs/pii/S0031320307004359 for more info. :control source: - The index of the buffer to analyse and suggest a number of components for. + The buffer to analyse and suggest a number of components for. :control bases: - The index of the buffer where the bases will be written to. These will be the bases and are suggested seed for a BufNMF processes. The number of bases (i.e., channels) in this buffer when the process is complete is the number of components needed to cover the percentage of variance in the buffer. + The buffer where the bases will be written to. These are suggested seed for a BufNMF process. The number of bases (i.e., channels) in this buffer when the process is complete is the number of components needed to cover the requested percentage of variance in the buffer. :control activations: - Must be set as an empty buffer (of any size), however no information is returned in this buffer. + The buffer where the activations will be written to. These are suggested seed for a BufNMF process. The number of bases (i.e., channels) in this buffer when the process is complete is the number of components needed to cover the requested percentage of variance in the buffer. :control minComponents: @@ -36,21 +36,21 @@ :control method: - The method used for the decomposition. Options are: + The method used to account for certain values before processing. Zeros in the matrix will remain zero when "updated" because the updates are being multiplied by a scalar, therefore it may be useful to change any zeros to something else before the process. Options are: :enum: :0: - **NMF-SVD:** Nonnegative Double Singular Value Decomposition + **NMF-SVD** Nonnegative Double Singular Value Decomposition where any negative values are first converted to their absolute value. This is likely to be quicker than the other options, but less rigorous. :1: - **NNDSVDar:** Nonnegative Double Singular Value Decomposition where any elements that are zero are filled with a random value between 0 and the average * 0.01. + **NNDSVDar** Nonnegative Double Singular Value Decomposition where any elements that are zero are first filled with a random value between 0 and the ``average * 0.01`` (essentially small random values). This may be slightly faster but slightly less accurate than other options. :2: - **NNDSVDa:** Nonnegative Double Singular Value Decomposition where any elements that are zero are filled with the average value. + **NNDSVDa** Nonnegative Double Singular Value Decomposition where any elements that are zero are first filled with the average value. :3: - **NNDSVD:** Nonnegative Double Singular Value Decomposition + **NNDSVD** Nonnegative Double Singular Value Decomposition where values are not changed according to any criteria. This promotes sparse results from the subsequent NMF (because, with multiplicative updates, zeros remain zeros). This may or may not be desirable (not least because sparsity implies the need for more components, but also the specific domain might imply that reasonable decomposition just isn't going to be sparse). :control windowSize: diff --git a/example-code/sc/BufNNDSVD.scd b/example-code/sc/BufNNDSVD.scd index 1cd3ac0..0dab4b0 100644 --- a/example-code/sc/BufNNDSVD.scd +++ b/example-code/sc/BufNNDSVD.scd @@ -27,6 +27,9 @@ Routine{ // peek at the bases ~bases.plot; +// peek at the activations +~activations.plot; + //use the bases to run NMF on FluidBufNMF.process(s, ~src, resynth: ~resynth, resynthMode:1, bases: ~bases, basesMode:1, activations: ~activations, components: ~bases.numChannels, action: {"done".postln;}) From 28dc8a3a65fceb1e116dd9a8c774da4a65fabbaf Mon Sep 17 00:00:00 2001 From: James Bradbury Date: Wed, 8 Jun 2022 10:21:26 +0100 Subject: [PATCH 4/5] add token for |buffer| --- doc/BufNNDSVD.rst | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/doc/BufNNDSVD.rst b/doc/BufNNDSVD.rst index cb5b124..5d1a369 100644 --- a/doc/BufNNDSVD.rst +++ b/doc/BufNNDSVD.rst @@ -12,23 +12,23 @@ :control source: - The buffer to analyse and suggest a number of components for. + The |buffer| to analyse and suggest a number of components for. :control bases: - The buffer where the bases will be written to. These are suggested seed for a BufNMF process. The number of bases (i.e., channels) in this buffer when the process is complete is the number of components needed to cover the requested percentage of variance in the buffer. + The |buffer| where the bases will be written to. These are suggested seed for a BufNMF process. The number of bases (i.e., channels) in this buffer when the process is complete is the number of components needed to cover the requested percentage of variance in the buffer. :control activations: - The buffer where the activations will be written to. These are suggested seed for a BufNMF process. The number of bases (i.e., channels) in this buffer when the process is complete is the number of components needed to cover the requested percentage of variance in the buffer. + The |buffer| where the activations will be written to. These are suggested seed for a BufNMF process. The number of bases (i.e., channels) in this buffer when the process is complete is the number of components needed to cover the requested percentage of variance in the buffer. :control minComponents: - Minimum number of estimated components to return (minimum number of channels in ``bases`` buffer when the process is complete) + Minimum number of estimated components to return (minimum number of channels in ``bases`` |buffer| when the process is complete) :control maxComponents: - Maximum number of estimated components to return (maximum number of channels in ``bases`` buffer when the process is complete) + Maximum number of estimated components to return (maximum number of channels in ``bases`` |buffer| when the process is complete) :control coverage: @@ -61,5 +61,3 @@ The window hop size. It can be any size but low overlap will create audible artefacts. The -1 default value will default to half of windowSize (overlap of 2). :control fftSize: - - The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision. The -1 default value will use the next power of 2 equal or above the highest of windowSize and (bandwidth - 1) * 2. From 801e04182c352d0051df18f59b4f0785b26453ce Mon Sep 17 00:00:00 2001 From: Ted Moore Date: Wed, 8 Jun 2022 15:39:15 +0100 Subject: [PATCH 5/5] feedback --- doc/BufNNDSVD.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/BufNNDSVD.rst b/doc/BufNNDSVD.rst index 5d1a369..46a451c 100644 --- a/doc/BufNNDSVD.rst +++ b/doc/BufNNDSVD.rst @@ -61,3 +61,5 @@ The window hop size. It can be any size but low overlap will create audible artefacts. The -1 default value will default to half of windowSize (overlap of 2). :control fftSize: + + The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision. The -1 default value will use the next power of 2 equal or above the highest of windowSize and (bandwidth - 1) * 2.