3

My desktop is a MacPro6,1 Desktop (Black Can). Here'my Grub command line arguments for Pie 

intel_iommu=on pci=hpbussize=10,hpmemsize=2M,nocrs,realloc

Even though I force a realloc I see the following BAR errors:

[    0.710600] pci 0000:ad:01.0: BAR 13: no space for [io  size 0x1000]
[    0.710601] pci 0000:ad:01.0: BAR 13: failed to assign [io  size 0x1000]
[    0.710602] pci 0000:ad:00.0: BAR 14: no space for [mem size 0x00100000]
[    0.710603] pci 0000:ad:00.0: BAR 14: failed to assign [mem size 0x00100000]
[    0.710604] pci 0000:ad:01.0: BAR 14: no space for [mem size 0x00100000]
[    0.710604] pci 0000:ad:01.0: BAR 14: failed to assign [mem size 0x00100000]
[    0.710605] pci 0000:ad:01.0: BAR 13: no space for [io  size 0x1000]
[    0.710606] pci 0000:ad:01.0: BAR 13: failed to assign [io  size 0x1000]
[    0.710607] pci 0000:ae:00.0: BAR 0: no space for [mem size 0x00008000 64bit]
[    0.710608] pci 0000:ae:00.0: BAR 0: failed to assign [mem size 0x00008000 64bit]
[    0.710609] pci 0000:ae:00.0: BAR 0: no space for [mem size 0x00008000 64bit]
[    0.710609] pci 0000:ae:00.0: BAR 0: failed to assign [mem size 0x00008000 64bit]
[    0.710642] pci 0000:af:00.0: BAR 5: no space for [mem size 0x00000200]

After realloc some BAR memory and IO allocations are successfull,

[    0.710910] pci 0000:cb:00.0: BAR 4: [io  0xb000-0xb01f] conflicts with 0000:a9:00.0 [io  0xb000-0xb01f]
[    0.710910] pci 0000:cb:00.0: BAR 4: failed to assign [io  size 0x0020]
[    0.710911] pci 0000:cb:00.0: BAR 0: no space for [io  size 0x0008]
[    0.711081] pci 0000:02:00.0: BAR 4: assigned [io  0x2000-0x20ff]
[    0.711092] pci 0000:06:00.0: BAR 4: assigned [io  0x3000-0x30ff]

I know what I need to do.. but I do not see an option on how to implement this. I need to enable PCI addressing above 4GB. Most of the forums post to enable this in the BIOS.. but this is a UEFI Mac.

Any help would be appreciated. I am phasing out MACOS. Thanks.

BTW my eGPU works with MacOS. It's connected via thunderbolt. Everything works on Linux, except allocating the resources for my thunderbolt attached GPU. My thunderbolt attached jbods are all accessible.
The Nvidia driver even installs. Just not enough memory and IO resources for the to function properly. nvidia-smi fails to detect the device.

Thomas
  • 6,143
  • 12
  • 30
  • 36
Jerry Normandin
  • 31
  • 1
  • 3

1 Answers1

1

The secret is to make hpmemsize huge - but the issue is when it becomes too big (in my experience, >128MB), it wipes out the 32-bit memory from the root window, leaving a range with higher start than end. It turns out that the memory resources have to be inherited from the parent bridge (not allocated completely at random) so the root port, for example, that hosts the Thunderbolt, should have a lot of memory under them - in my case with JHL6540 on Dell 9370, it is 737MB (32-bit) and 1184M (64-bit prefetchable). Use "sudo lspci -vvvv" or similar command.

My external graphics refuses if the main BAR0 (256MB) cannot be allocated - but I cannot make the hpmemsize big enough without the 32-bit resources disappearing, unless I patch the kernel. Under certain conditions, the amdgpu driver fails to check that the BAR has failed, and requests a resource from the kernel of base address 0x0 and size 0x0 - hence triggering a bug detection and basically requiring a reboot to make the system fully usable again.

If you are still having this issue now, then try Linux 4.20-rc3 (or at least 4.19 kernel), and try increasing the hpmemsize and observing what happens in lspci. Anyways, soon I am free from semester and I am going to be trying to get my kernel patch submitted, which sorts this mess out.

PS: pci=hpbussize=N has no effect unless you also have pci=assign-busses - and that breaks Thunderbolt unless you have the new Thunderbolt 3 Titan Ridge (2018) controllers.

I am not even entirely certain that the others do much without pci=assign-busses - although I have never tested it, so don't take my word for it.

Also, remember pci=hpiosize exists (defaults to 256 bytes). The pci=hpbussize defaults to 1 and pci=hpmemsize defaults to 2MB.

Karatekid430
  • 141
  • 2