I tried to follow the instructions in https://superuser.com/a/1285784/936664 but could not find the file LayoutModification.xml. The only file in the location for me is called DefaultLayouts.xml and this did not have anything relating to pinning or taskbars.
Bonus points for a solution that does not require admin rights. I found this while trying to google for a solution https://www.cnet.com/how-to/heres-how-to-get-rid-of-internet-explorer/ but I do not have the required administrative rights to open "Turn Windows features on or off". Additionally, I do not mind if IE stays in the Start menu.