(continued from previous post)
Anyway, onto the main topic of this post, which is what I just noticed about the figure in Bonada's thesis (which was in turn copied from "Improvements to a Sample-Concatenation Based Singing Voice Synthesizer"), picrel. At the top, we see the original spectrum and its harmonic envelope. In the middle plot, we see an intermediate visualization of peaks after being moved, but before equalization. Notably, there now exists one discontinuity due to one peak being used twice. In the bottom plot, we see the rendered spectrum after equalization. Notably, there is still only one discontinuity in the spectrum. The proposed technique eliminates gaps in the spectrum, but not discontinuities, because the SPP algorithm, as described, regardless of the rendering variant, scales all the bins in each region, which would produce a discontinuity at the boundary between regions.
Instead, in the bottom figure, the amount of scaling appears to vary non-linearly, such that it is continuous. This is especially apparent since the figure shows a dotted line representing the spectrum before equalization. You could say that this is just how the figure was drawn, however it is highly consistent across the whole figure; furthermore, it is consistent with other papers and the descriptions in some of the patents seem to suggest that the amplitude discontinuities may be solved. Based on this, I have a hypothesis on how the non-uniform amplitude scaling may have been implemented. The process would be to create a first envelope of the harmonic peaks before transformation; create a second envelope of the harmonic peaks after frequency shifting, but before equalization; and finally scale each bin based on the ration of the first envelope to the second envelope. Although I have not tested it, looking at the figure visually, the proposed procedure would seem to match very closely, assuming a smoothing interpolation. Furthermore, it may also explain why the intermediate stage that shows the spectrum after shifting, but before equalization, is explicitly shown. Such a thing is not actually created by the SPP algorithm as described, but it does make sense to show for the proposed non-uniform amplitude scaling. Finally, in theory, this procedure would not produce any additional amplitude discontinuities, further consistent with the figure.
I will implement this idea as a variant in my existing SPP algorithm, which has already been implemented such that multiple variants (and combinations thereof) can exist in the same procedure (using preprocessor flags). Furthermore, I will consider the possibility that the same procedure is also used for the unwrapped phase envelope.
