NRPN ability

[klerg]

Hello

Can Fluidsynth add Non Registered Parameters (NRPNs) for setting the Low Pass Filter ?

Thanks a lot

[coderofsalvation]

according to the soundfont spec yes.
not sure whether fluidsynth implemented it..

[derselbst]

It would be extremely helpful if somebody provided an example MIDI file and concisely describe what the expected behavior of the respective low pass filter destination generators would be.

[coderofsalvation]

I could do that, I have an electribe SX of which the midi knobs send out CC (for example #74) but also NRPN.
AFAIK most soundfont editors are able to define modulators (with CC-source):

• CC (D0 40 for CC7 val 64) ==> lowpass filter

So the question is also perhaps: which editors allow NRPN modulators as per the spec (Vienna?):

• NRPN for (D C 3F 7F for NRPN number (MSB+LSB) and value ===> lowpass filter

the midi-file alone is not enough otherwise

[klerg]

Sure but not many follow the soundfont spec
I have no clue but it is possible fluidsynth did this

That's fine by me I just attached the full Midi and sample MP3 of how it sounds on SB AWE32 up to Audigy 2 sound cards. As you can tell, the low pass filter has a sweep that is hardly, if at all, heard in Fluidsynth, how can it sound close to SB hardware ?

Nice, but I just did this. Well, I know the ESX-1 is not the only synth which is able to do that also. Yes, but modulators are something else and came a bit after NRPN's and most soundfont synths support modulators too.

I'm at a lost here on why a soundfont editor needs to allow NRPN's but I hope it is not only Vienna

I'll take your word for it but what else do you need ?

MIDI.zip

[coderofsalvation]

initially I thought the issue was about fluidsynth being able to support NRPN for modulator control (instead of only via CCs, like mentioned in the EMU soundfont spec.).

After reading the spec again, it seems that NRPN directly maps to the Defined Generator Enumerators (page 30 http://www.synthfont.com/sfspec24.pdf).
(This makes it not interesting/flexible enough for me anymore, which is why I will detach from this issue).
Btw. for whatever its worth, the 2.4 spec mentions:

So it specifically mentions breaking with the traditional AWE32 mappings (the attached midifile?), so technically the midi-file is using either an AWE32-filtercutoffsweep-via-NRPN or v2.4-filtercutoffsweep-via-NRPN.

[klerg]

Well, I do not know of any soundfont synth that is able to use NRPN's for modulator control, but have only seen ones that map to CC's like you stated and go by the Emu soundfont spec.

All I can tell from the MIDI is that it has NRPN messages, but I'm not sure what they are. And if it maps with anything in the soundfont 2.1
spec, why does it sound wrong in Fluidsynth ?
(That is not good to hear I feel the topic is still interesting enough and your input was of great help as well)
Yes, it may not be worth much, what the 2.4 spec says.

The problem with what it states here is it may be wrong. As the audio file attached was played on an SB Live! from 1999 to 2000, so it did not support the 2.4 spec but no later than 2.1 spec yet it sounds identical to an AWE32 or 2.0 spec device

[derselbst]

I found some time looking into this. First off all cudos to @coderofsalvation for already pointing to the correct section of the spec that explains why the provided MIDI works fine with EMUs hardware synth's (which are SoundFont 1.0 based), but not with a SoundFont 2 compliant synth, like fluidsynth. Let's take a deeper technical look:

All I can tell from the MIDI is that it has NRPN messages, but I'm not sure what they are.

A NRPN message is a series of regular MIDI control change (CC) messages, which are order-dependent. Provided, to mess around with even more proprietary low-level stuff of certain MIDI hardware. In the SoundFont world, they allow real-time manipulation of soundfont generators.

MIDI CCs 99 and 98 select the most significant and least significant byte (MSB and LSB resp.) of the NRPN.
These are followed by a data value composed of CC 38 and CC 6 for MSB and LSB resp.

All those CCs are forbidden to modulate as per section 8.2.1! Since their purpose is to manipulate on a lower level.

The SoundFont 2.0 spec demands to only process those NRPN messages, which have CC99 set to value 120. The provided MIDI file uses a value of 127 for this controller. Presumably a leftover of the legacy SoundFont 1.0 hardware. Anyway, that's the reason why these messages are completely ignored by fluidsynth.

Suppose one would hack the MIDI and change all the CC99 controllers to a value of 120. The next thing that follows as per SF2 spec would be one or more CC98 messages for selecting the destination generator to manipulate.

Your MIDI file recurringly selects generators 7, 21, 22, and 24. In the SF2 world these are

• modulation envelope to pitch (7)
• modulation LFO delay (21)
• modulation LFO frequency (22)
• vibrato LFO frequency (24)

Listening closely to your provided rendering, I cannot really hear any tremolo or vibrato. Therefore I believe the selected generators were different ones back in the SoundFont 1.0 world.

From listening, I would have expected that the MIDI file manipulates generators

• 8 (low pass filter frequency cutoff),
• 9 (low pass filter frequency resonance),
• 10 (modulation LFO to filter frequency cutoff), and
• 11 (modulation envelope to frequency cutoff).

One could speculate whether the SoundFont 1.0 generator 7 selected by the MIDI file actually corresponds to generator 8 in the SF2 world. To be sure, one would need to do some time-consuming reverse-engineering with real hardware. And only then one could think about converting the MIDI NRPNs of your MIDI file to become compatible with the SF2 world and yield the desired low-pass filter effect. Theoretically, given this logic is understood by having a few more examples of MIDI files like those at hand, it might even be possible to convert these one the fly in fluidsynth. Just keep in mind there is a reason why SoundFont 1.0 never really happened: There is no publicly known SF1 spec that defines the sound articulation. Rather it seems, the hardware-chip design defines the sound articulation.

Lastly, the NRPN message is completed using CCs 38 and 6 to compose a 14-bit value meant to manipulate the selected generator somehow. For details, pls. refer to SF2 section 9.6.3. No idea how this would work in SoundFont 1.0.

[klerg]

That is a good thing to hear for sure. Yes but this section of the spec does not explain why the example MIDI sounds fine on an AWE64 (which is a SoundFont 2.0 device) and a SB Live! (which is a SoundFont 2.01/2.1 device) All we know on SoundFont 1.0 is that it used SBK files and worked on the AWE32 card only. Sure no way to avoid the technical here.

I see your definition of NRPN follows closely what is stated in the spec and is almost identical as well. Yes,
NRPN and RPN messages were around when MIDI was first made back in the early 80s or so. That's right, and my goal here is to control the soundfont generators in real-time, and NRPN allows this in some way.

To be frank, I never fully understood how an NRPN message works but what you state here must be correct. And it is not as simple as a CC message but it needs to be more complex to manipulate the generators in real-time.

Sure I will take your word for it as section 8.2.1 hardly makes any sense for me. I thought CCs were meant to control the sound on a more higher, and less detailed, level.

Again, I'm going to agree with what you state here on how SoundFont 2.0 spec deals with NRPN messages, and the value of CC99 set to 120 as well. Yes I can see that it is set to 127 in a MIDI editor. True, this must be some thing that is meant for SoundFont 1.0. But how is the AWE64, SB Live!, and even up to Audigy 2 ZS able to recognize it as they are SF 2.0-2.4 ?

Sure it is easy to edit this MIDI file in an editor and set all of the CC99 controllers to 120. Yes a lot of this is
not much different from how NRPN and RPN messages work on most other MIDI devices that allow for real-time control of the generators.

A look at this example MIDI file in any MIDI editor gives the same result that you listed below

Yes, Mod Env to pitch, Mod LFO delay, Mod LFO frequency, and Vibrato LFO frequency all have to with pitch, and only result in a tremelo or vibrato, but that is not how this MIDI sounds on an AWE64 or SB Live! so SF1 is different.

Yes, from listening you can hear a filter sweep and not any pitch effect so the generators do not match. LPF Freq cutoff, LPF freq res, Mod LFO to Filter Cutoff, and Mod Env to Freq cutoff all control the filter.

I also think that generators 7, 21, 22, and 24 in SoundFont 1.0 correspond to generators 8, 9, 10, and 11 in SoundFont 2.x devices, as nothing else really makes sense. Well, I'm not sure what reverse-engineering actually means other than it is time-consuming as you put it and needs the real hardware. I'm willing to try this out then but do not know which hardware I need, will a SB Live! work and what sort of tests are going to help get the low-pass filter effect from this MIDI file then ? I may able to find a few more SF1 MIDI files but it is likely they will use the same NRPN messages that this MIDI file does. Yes, the only way to support a SF1 MIDI is to convert it on the fly in Fluidsynth to SF2. That is true, there is no SF1 spec and it may always remain a 'closed' format. If I understand it right, then the EMU8000 chip is the one that has a rather unknown method it uses to defines the sound articulation, just as you say it is. But it is possible to guess how SF1 NRPN's work.

I'd need to take a look at the event list of this MIDI file to see anything that even resembles a '14-bit' value that is supposed to modify a selected generator in some way. Sure, but it may work differently to SF1. When I post the full audio of this MIDI you will get a much better idea of how it works in SF1 too.

[derselbst]

Yes but this section of the spec does not explain why the example MIDI sounds fine on an AWE64 (which is a SoundFont 2.0 device) and a SB Live! (which is a SoundFont 2.01/2.1 device) [...] But how is the AWE64, SB Live!, and even up to Audigy 2 ZS able to recognize it as they are SF 2.0-2.4 ?

All those devices can still have some kind of SoundFont 1.0 software-compatibility layer programmed in their ROM, to retain backward compatibility for purposes like NRPN. Keep in mind that the introduction of the SF2 spec also mentions some kind of "SoundFont 1.5", which never officially was, but perhaps that's the term EMU refers to their state of development back in those days.

I'm willing to try this out then but do not know which hardware I need, will a SB Live! work and what sort of tests are going to help get the low-pass filter effect from this MIDI file then ?

I'd recommend you start by understanding how your SBLive interprets the well-known SF2-like NRPN messages. Particularly for "generator 8 - low pass filter frequency cutoff" as this should be pretty straight forward to recognize audibly. I.e. you would send

CC99 = 120
CC98 = 8

and then try to find out how the 14-bit "Data"-value influences the generator. I.e. what do big / small values mean, is there an obvious mapping to Hertz, centibels, whatever.

Then try to reproduce the results by using the presumed SF1-like NRPN messages, by sending CC99=127, an easily recognizable (SF2-)data value and try to find the corresponding CC98 number. Ideally, one would find that the SF1 data values have the same mapping, ranges and audible influence as the SF2 data values. In this case one could try sending the SF2 NRPN messages to fluidsynth and hear how this behaves.

[klerg]

Yes I hope this means is any SoundFont 2.x software synth can have its own compatibility layer to be
backward compatible with SoundFont 1.0 NRPN messages. The issue is to find out how SF1 NRPN's work as
no spec exists. Based on what the SF2 specs mentions, there was a SoundFont 1.5 at some point, but that
was never released, and it can also refer to a term that EMU uses for a state of development, as you say.

Sure will try this soon but I do not see why you call the NRPN messages as 'SF2-like' instead of just 'SF2'
on the SBLive. Yes, "generator 8 - low pass filter frequency cutoff" is the most obvious to hear. But also
'generators 9, 10, and 11' are going to be pretty audible as well.

I hope when you mention the '14-bit Data value' that means CCs 38 plus 6 and how setting them to 0 all the way up to 127 affects the overall sound and if that maps to Hertz, cents, freq or something else.

Here I can tell why you refer to the SF1 NRPN message as 'SF1-like' as we are not sure if that is right here. Yes, CC99 must be 127, what do you mean by 'easily recognizable (SF2-) data value' ? The SF1 CC98 number is 7 if 8 in SF2. It would make sense if the SF1 data values match the same as SF2 ones or closely at the very least or else the sound is going to be different. As Fluidsynth fully supports SF2 2.4 spec then SF2 NRPN messages must work without an issue. I will post the full audio of this SF1 MIDI next week.

[derselbst]

why you call the NRPN messages as 'SF2-like' instead of just 'SF2'

Sry for the confusion, just scratch out the "-like".

I hope when you mention the '14-bit Data value' that means CCs 38 plus 6

Correct.

what do you mean by 'easily recognizable (SF2-) data value' ?

I mean CCs 38 and 6 that you derived from your listening tests with the SF2 NRPN messages.

I think in general you've understood my idea. Looking forward to your findings.

[klerg]

Sure that's fine I will forget about the "-like" then

Sounds good

Got it but what am I supposed to do with the CCs 38 and 6 that come from the listening tests with SF2 NRPN messages ?

I really hope so as it is important to move forward here. Yes and the same for me as well.

[derselbst]

Got it but what am I supposed to do with the CCs 38 and 6 that come from the listening tests with SF2 NRPN messages ?

Try to understand their meaning and influence on the sound. I.e. in the end it would be good to know if that 14-bit value composed of those two CCs behave just as simple additive logic to a generator as demanded by the SF2 spec and implemented here:

fluidsynth/src/synth/fluid_voice.c

Line 438 in db86c36

return (fluid_real_t)(voice->gen[num].val + voice->gen[num].mod + voice->gen[num].nrpn);

I.e.: Generator Value = Initial generator value defined in the SoundFont + current modulation influence by any modulators + 14-bit value of the latest NRPN message

[klerg]

Well, the meaning is real-time control of generators but the influence on sound varies. Will you be able to tell from
the full audio of this SF1 MIDI if the 14-bit value does more than 'simple additive to a generator' ?

[derselbst]

Might be difficult without any modulators. In case there is something similar like an initial generator value in SF1, one might at least be able to testify whether the NRPN messages overrides that initial value or is applied additive.

[klerg]

I know SoundFont 1.0 has no support for modulators. I'm pretty sure SF1 has an initial generator value or it had another name. But I do not see why it matters if the NRPN messages override anything or it is applied additive as you put it here ?

[derselbst]

It matters, because someone, probably me, at some time would need to know how to implement this as accurately as possible to real hardware. The behavior of the NRPN messages in SF1 could be completely different to SF2. I've just provided two simple examples.

[klerg]

It may be that the Soundfont 1.0 software compatibility layer programmed in the SB Live! and up to Audigy 2 ZS does not 100% match the AWE32. Yes we know that the NRPN messages in SF1 differ than SF2, but not what is the same. How will I be able to tell if an NRPN overrides or adds anything ?

[derselbst]

As mentioned, you'll need to play around with the initial value of a generator. I.e. when you know the min - max range, set the initial value and NRPN value to the middle. If it adds up, generator value will be at max. It it overrides, it will stay middle.

[klerg]

Well, I just came home tonight so will post the full audio of this MIDI soon. But by initial value of a generator, do you mean the lowest value up to the highest ? And is the middle NRPN value going to be 64 ? Is it better if it overrides or adds up generator ?

[derselbst]

But by initial value of a generator, do you mean the lowest value up to the highest ?

Not sure if I understand your question. The initial value that value, the generator has initially, specified in the SF2 / SBK file.

And is the middle NRPN value going to be 64 ?

That's what you need to find out during your reverse-engineering tests.

Is it better if it overrides or adds up generator ?

Neither is "better". In terms of implementation and compatibility to SF2, additive would be simpler. Apart from that, it's just different, not "better".

[klerg]

To me it looks like you did get the question. Sure so the initial value of the generator is going to be the default, how will NRPN message change that ?

I see and expect it will not take very long to find the middle NRPN value then.

That is good to know. Well then, let's hope it turns out to be additive as the simple way is always the best. But if it ends up to override is that fine too ?

[derselbst]

Here's the audio zip @klerg sent me:

MIDI Full.zip

Could you share some details about it?

[klerg]

Glad the audio can be uploaded on here

Sure what would you like to know, and does it give a clue on how "CCs 38 and 6...manipulate the selected generator" in SF1 ?

[derselbst]

I will not commit to any timeline. I'm pretty busy these days and as you can see that there has been quite little activity lately.

More example MIDIs would be greatly appreciated.

[klerg]

I see what is it you can commit to then ? Well I hope you can find some time for this and that is true the activity is a little not a lot.

Why do you need more MIDIs how will that help at all ?

[derselbst]

Why do you need more MIDIs how will that help at all ?

Well, we either attempt to implement all AWE32 NRPNs or none at all. And to verify it's working as it should, I'd need more example MIDI + rendering of how it's supposed to sound on the AWE32 or SBLive. It would also serve as a reference for others.

And maybe - who knows - if I find a tune I like, I might prioritize this a bit...

[klerg]

Yes, and the docs give info on how to implement every AWE32 NRPN as well. The MIDI that I already sent uses the most NRPN's at four (7, 21, 22, & 24) whereas the other MIDI's use only two (21, 22). I know what you mean but all of the rest have copyrights too.

Sure, here is the link to the MIDI's but only the following have NRPN's: ALTITUDE.MID, AMERISS.MID, frenetic.mid, ISEULT.MID, Marathon.mid, MYSTRAVE.MID, TEK.MID, trancpar.mid, TRUTH.MID and Uplift.mid. Do you have a SB Live or up to Audigy 2 ZS to hear how they sound ?

[derselbst]

The MIDI that I already sent uses the most NRPN's at four (7, 21, 22, & 24) whereas the other MIDI's use only two (21, 22).

MIDIs using less NRPNs are also handy for testing purposes, thanks for the link, that should do.

Do you have a SB Live or up to Audigy 2 ZS to hear how they sound ?

Nope, I don't. Pls. kindly render them for me.

[klerg]

Well, all of the MIDIs use much less NRPNs then the first one I sent already. Sure that is fine, sounds good

I see that is too bad, which ones do you need as an audio ?

[derselbst]

Pls. render the 10 files you've mentioned. Ideally, use a freely available SoundFont2 for all of them so it allows me to reproduce the MIDIs with fluidsynth.

This is now the third time that I'm asking for this. The more counter questions you're asking the more I believe you're not serious. Keep in mind that you are asking me to invest a serious amount of my spare time to implement this feature for you. So I guess I can expect you to invest a few minutes to render them so that I have a solid reference base.

[klerg]

I'm not sure how long that is going to take. Fine, I may need to mix and match and use more than one SoundFont2 for some presets and tracks to get the best sound.

I count no more than twice, the first time you only asked for MIDIs. Well, I was not sure that you were serious to commit so had to ask. My hope is that this will not take up too much of your time, and that it is not just something for myself but anyone. To be fair it is going to be longer than a few minutes but closer to twenty or even thirty, is this the last thing you will need from me ?

[derselbst]

I have created #1342 to condense the information that we have talked about over the past three months.

[klerg]

Well, it looks like I've done my part, and I'm sure this will be put into FluidSynth as soon as that is possible.

[klerg]

Here is one more document that might help with NRPN support, but it may repeat some info from the AWE32 FAQ
SoundFont_NRPNs.PDF