11 years after the bug was filed, Mozilla finally got to work on this popular feature request, and it seems to be working okay now (tested in Firefox 66.0.3/Ubuntu).
(Thanks to @PerJohansson for pointing out that they've made the setting much more difficult to find since FF 59.)
I've just updated this answer with some easier steps, tested with Firefox 75.
NOTE that if you'd like to disable shortcuts for all sites by default, navigate to about:config, and set the value of permissions.default.shortcuts to 2. Thanks to @inetknght for the tip, which is also documented here: https://support.mozilla.org/en-US/questions/1241294#answer-1175070
For disabling shortcuts on a per-site basis, you can find screenshots for each step below.
- Press Ctrl-I (if you can) to show the Page Info window; or else right-click on an empty portion on the web page, and select
View Page Info from the context menu.
- Select the
Permissions tab
- Adjust the
Override Keyboard Shortcuts setting.
Here are some recent (April 2020) screen shots for each step.
- If you use the mouse to open the Page Info window:

- Select the
Permissions tab:

- Adjust the permissions appropriately:

Mozilla has more information on the Page Info window here:
https://support.mozilla.org/en-US/kb/firefox-page-info-window
And if you're interested in the history of this fix, here are the related Mozilla tickets:
https://bugzilla.mozilla.org/show_bug.cgi?id=380637 and
https://bugzilla.mozilla.org/show_bug.cgi?id=1445942