I don't know ffmpeg so this is not a full answer, but "where to go for the answer"
The function you are looking for is not Normalisation.
What you need are Expansion, Compression & Limiting, or a more comprehensive auto-gain or envelope follower.
Many people get confused by that one. Normalisation is done by lifting all the sound track as one unit to a new maximum peak. It doesn't affect any individual parts of that sound. You use it, if at all, after all other processing. It is not a dynamic effect, but a static one. It's simply like an overall volume knob, nothing else.
Expansion lifts the volume when it falls below a set threshold.
Compression lowers the volume when it gets above a set threshold. It has a curve to set how hard it does this.
Limiting is like very hard compression, which 'brick walls' any sounds from going above a set threshold.
These all act on every tiny part of the sound [adjustable] so used to extremes can seriously affect the end result, making some unwanted background noises too prominent.
Another option would be an Envelope Follower, or an auto-gain, which would attempt to do this without compression, by automatically 'chasing' the volume to keep it level, like a much better version of the old cassette recorder auto record level, or Windows' auto mic gain.
You may have to employ some or all of the above if the volume changes are dramatic.
All of these processors are available for high-end DAWs [Digital Audio Workstations] & some may be available as freeware, though I don't know any specific sources. Audacity is a commonly-used freeware audio editor, which has a selection of plugins which may be suitable.
As mentioned in comments, VLC has a built-in compressor which may go some way towards correcting this, & also a normaliser you could use as make-up gain afterwards.
I think for anything else, look to see if ffmpeg can do any of those processes, or lift out the audio & transfer it to a dedicated audio package.
