ASRock X370 Pro - Can't POST with 8 GPUs
Printed From: ASRock.com
Category: Technical Support
Forum Name: AMD Motherboards
Forum Description: Question about ASRock AMD motherboards
URL: https://forum.asrock.com/forum_posts.asp?TID=7873
Printed Date: 08 Feb 2025 at 1:10pm
Software Version: Web Wiz Forums 12.04 - http://www.webwizforums.com
Topic: ASRock X370 Pro - Can't POST with 8 GPUs
Posted By: ASRickroll
Subject: ASRock X370 Pro - Can't POST with 8 GPUs
Date Posted: 28 Feb 2018 at 5:08pm
|
I am currently transforming my gaming PC into a mining rig by adding PCIe slots and GPUs, hence the X370 (unusual choice for mining) The GPUs are 8xASUS RX 580 Dual 8GB I get "0d" error on Dr. Debug and cannot even POST (blank screen, cannot enter BIOS) when I install 8 GPUs Everything works fine with 7 GPUs (I can remove any one GPU) everything works fine. However when adding 8th GPU, it can't POST and always displays "0d" I have tried replacing 1-2 GPUs with 2 spare GTX 1050Ti, neither combination works. Mix and matching my 10 GPUs works fine if there are only 7 GPUs total. Adding the 8th one, whichever it is, always results in no-POST. I am using some kind of PCIe PLX card (not sure what to call it) to turn a single 16x PCIe to eight 1x PCIe (see pics below) I have tried using combination of the PLX card and other PCIe slots to no avail (e.g.: 6 GPUs on PLX card, 2 GPUs on mobo's PCIe 1x slots) I even have a second PLX card and have tried swapping them or even using them together (e.g.: 4 GPUs on the first PLX card, and the rest on the second PLX card) Neither works if I install more than 7 GPUs Again, everything works OK if I only use 7 GPUs max, regardless of which PLX card or PCIe slot I use. I have tried replacing the PSU (I have two SilverStone 1500W and 1300W Ti), replacing PCIe riser (I have 20 of them), replacing the PCIe cables (total 16 PCIe 8pin cables from both PSUs), replacing PLX card (I have 2 of them), all to no avail. I have started to suspect that I have either hit the processor or the mobo limit (Ryzen mining mobo from Biostar only supports 6 GPUs) Is there a maximum GPU limitation on AMD Ryzen itself? Or is this limitation on ASRock mobo? Could someone experienced enlighten me? Any help is appreciated! My specs and pictures are below Mobo: ASRock X370 Pro (BIOS v4.40) CPU: Ryzen 1700 GPUs: 8xASUS RX 580 Dual 8GB, 2xGTX 1050Ti 4GB RAM: 2x8GB Corsair Vengeance RGB DDR4 PC3000 SSD: WD Blue 1TB (M2 SATA) PSU: SilverStone Strider 1500W Ti and 1300W Ti (using only one of them) OS: Windows 10 Pro (should not be an issue since the system cannot even POST) PLX 1->8 PCIE slot extender (see pics below) bought from Amazon No brand PCIe 1x risers bought from Amazon Pic of whole setup:  Pic of Dr. Debug error code "0d" when using 8 GPUs  The PLX card that I'm using to create extra PCIe x1 slots:  P.S.: After failing to POST with 8 GPUs, and I remove 1 of the GPU, the mobo seems to "remember" and I cannot POST unless I clear the CMOS Clearing CMOS has no effect when 8 GPUs are installed Will provide more information if requested |
Replies:
Posted By: JohnM
Date Posted: 28 Feb 2018 at 8:41pm
|
" rel="nofollow - I admit I know very little about what you're trying to do but one thought struck me. How much power does one of those GPUs take? Is 1.5 kW enough for your set up?
Edit: A quick Google suggests around 150 W each. So for eight of them, 1.2 kW. That leaves about 300 W for everything else. Personally, I'd allow a bit more headroom. Can you power four of your cards with your second PSU? ------------- ASRock Fatal1ty AB350 ITX P4.90, AMD Ryzen 5 2400G, 2x8GB Corsair CMK16GX4M2A2666C16, 250GB Samsung 960EVO, 500GB Samsung 850EVO, 4TB WD Blue, Windows 10 Pro 64, Corsair SF450, Cooler Master Elite 110 |
Posted By: JohnM
Date Posted: 28 Feb 2018 at 9:02pm
|
Then again, I suppose they don't draw full power at start up. Their power requirements at idle are probably quite low. ------------- ASRock Fatal1ty AB350 ITX P4.90, AMD Ryzen 5 2400G, 2x8GB Corsair CMK16GX4M2A2666C16, 250GB Samsung 960EVO, 500GB Samsung 850EVO, 4TB WD Blue, Windows 10 Pro 64, Corsair SF450, Cooler Master Elite 110 |
Posted By: ASRickroll
Date Posted: 28 Feb 2018 at 11:12pm
Hi JohnM  thanks for your reply.Yes, you are right. However I've tried replacing 2 of RX 580 (150W each, requires 8pin PCIE cable each) with 2 GTX 1050Ti (75W each, no PCIE cable required). Which supposedly should bring down the overall power requirement by 150W, but no luck. I currently don't have dual PSU adapter so I cannot test dual PSU config.
|
Posted By: MisterJ
Date Posted: 01 Mar 2018 at 12:18am
ASRickroll, perhaps you have just exceeded the limit of your processor/board. Please open an AMD Support Ticket and ask them. My post code list, which wardog ask me not to post a link to, gives this for the 0x0D code:
------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: ASRickroll
Date Posted: 01 Mar 2018 at 4:22am
Thanks MisterJ. I did what you suggested but I got an error when creating AMD Support ticket. Tried twice. Getting the same error. Seems like a bug in their ticketing system.  |
MisterJ wrote:Posted By: MisterJ
Date Posted: 01 Mar 2018 at 4:30am
|
" rel="nofollow - ASRickroll, I just submitted a ticket and it worked. It took a long time, maybe more than a minute. Please give it another go. Are you in the US as I am? Enjoy, John.
------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: kerberos_20
Date Posted: 01 Mar 2018 at 1:38pm
|
try to free some pci-e devices like usb, sata, soundcard, ... whatever u dont need and is connected o chipset but dunno if it helps, my x370 has 7 hardware switches (4 free atm) on chipset and 2 useable from cpu which is max 9 devices ------------- http://valid.x86.fr/diq4l4" rel="nofollow">  http://www.userbenchmark.com/UserRun/47132492" rel="nofollow - userbenchmark |
Posted By: JohnM
Date Posted: 01 Mar 2018 at 8:11pm
|
" rel="nofollow - Have you tried putting the 8th GPU in the other motherboard slot? I'm slightly puzzled by the spec: AMD Ryzen series CPUs - 2 x PCI Express 3.0 x16 Slots (single at x16 (PCIE2); dual at x16 (PCIE2) / x4 (PCIE4))* Where does it get the four PCIe 3.0 lanes from for the PCIE4 slot? *Supports NVMe SSD as boot disks If M2_1 is occupied, PCIE4 will be disabled. So it shares with the M.2 socket? I guess a mining rig doesn't need an NVMe boot SSD. Other X370 boards I've seen allow the primary set of 16 PCIe 3.0 lanes to be shared 8+8 with the secondary GPU slot, while B350 boards keep all 16 for the primary GPU slot and allocate four PCIe 2.0 lanes from the B350. ------------- ASRock Fatal1ty AB350 ITX P4.90, AMD Ryzen 5 2400G, 2x8GB Corsair CMK16GX4M2A2666C16, 250GB Samsung 960EVO, 500GB Samsung 850EVO, 4TB WD Blue, Windows 10 Pro 64, Corsair SF450, Cooler Master Elite 110 |
Posted By: ASRickroll
Date Posted: 01 Mar 2018 at 8:49pm
Thanks for the suggestion. I have tried disabling LAN, sound card, etc. but no luck.
That will be the last 3 things I want to try before I give up (I'll go back to it tomorrow): 1. Install the 8th GPU directly into mobo PCIe slot 2. Set up a dual PSU (1500W + 1300W) to see if it's power issue 3. Use M2-to-PCIe adapter on that M2_1 slot and insert the 8th GPU into this adapter or directly insert the last GPU into PCIe4 (4x) That's true, basically there are 2 M2 slots: one is M2 SATA (I'm using it for the WD Blue SSD) and the other one is M2 PCIe which, like the manual says, is sharing 4x PCIe lanes with PCIe4 slot. About the 8+8, that's also true for this mobo. I can set it in the BIOS (1x16 or 2x8 mode) but I think this is not necessary since the mobo automatically switches to 2x8 if you insert 2 devices into the SLI/CrossFire slots. I have tried manually setting the mode to 2x8 and just connect everything into the PLX card. No luck though
 |
Posted By: JohnM
Date Posted: 01 Mar 2018 at 9:29pm
|
Good luck, you have a few things to try so one of them might work! Let us know how you get on. ------------- ASRock Fatal1ty AB350 ITX P4.90, AMD Ryzen 5 2400G, 2x8GB Corsair CMK16GX4M2A2666C16, 250GB Samsung 960EVO, 500GB Samsung 850EVO, 4TB WD Blue, Windows 10 Pro 64, Corsair SF450, Cooler Master Elite 110 |
Posted By: MisterJ
Date Posted: 02 Mar 2018 at 2:39am
|
ASRickroll, please try again to open an AMD Support Ticket. I only reported your ticket problem. Enjoy, John. EDIT: AMD response to my Ticket was to try a different browser. ------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: ASRickroll
Date Posted: 04 Mar 2018 at 10:24am
I used different browser (was Chrome, now Firefox) and still getting the same error submitting ticket. I tried again without attaching any files and it was submitted, fingers crossed. By the way I tried inserting the last GPU into PCIe slot directly and also tried using the M.2 socket (was using M.2 -> PCIe adapter) without any luck. I'm now convinced that the problem is not because I hit PCIe lanes limit. Someone in another forum mentioned that it might be because the motherboard does not have enough power to support that many PCIe devices (even though all my PCIe risers are self-powered directly from PSU).
|
Posted By: MisterJ
Date Posted: 04 Mar 2018 at 10:51am
|
" rel="nofollow - ASRickroll, did you try attaching a Word Doc, PDF or JPG? Thanks for informing us. Please let us know what they have to say. Thanks and enjoy, John.
------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: benb
Date Posted: 09 Mar 2018 at 1:03pm
|
" rel="nofollow - Did you get a custom bios that enables "Above 4G Decoding" from Asrock? They can build it for you, wish they would just let it in all their bios. They gave me a custom bios a while ago that enabled the flag, but it is 2.12 ish . I am going to ask them for an updated bios because the newer agesa is suppose to address some of the linux/ryzen issues for early series chips. "Above 4G Decoding" works as long as CSM is disabled or if enabled only with UEFI OpRoms(There are three, PXE,Storage and Something else) You might want to disable CSM first , ie remove some cards boot , disable CSM and only allow UEFI to load. Hopefully if we bother Asrock enough they will just roll the "Above 4G Decoding" out to their bios with a comment "For mining/advance users", something along those lines. Biostar and MSI have rolled out the option to their bioses and I think gigabyte also. Only ASUS and Asrock for some reason aren't rolling it out. Example MSI rolled "Above 4G Decoding" out to all their x370 motherboards. http://www.nag.co.za/2017/07/14/msi-revamps-their-bios-to-support-crypto-mining-farms-better/ |
Posted By: pbfarmer
Date Posted: 09 Mar 2018 at 6:48pm
Was wondering why this option wasn't available - was thinking it just wasn't applicable for AMD for some reason, until I just tried installing my 8th card. Same issue as OP - can use 7 GPUs in any config (including 5 PCIE + 2 M.2 - all risered,) but adding the 8th won't post - get 0d error. Hardware isn't faulty, plenty of PCIE lanes, power shouldn't be an issue, so now I'm assuming the 4G decoding issue is the problem. I've tried disabling CSM but it doesn't change anything. I unfortunately installed Win10 using an MBR/FAT32 image, so I seem to be running in legacy mode regardless of that setting (it gets ignored or auto-reset?) I suppose I can reinstall Windows and try disabling CSM again, before reaching out to ASRock for a custom rom. Are you running 8+ GPUs w/ the custom uefi? |
Posted By: MisterJ
Date Posted: 10 Mar 2018 at 12:37am
|
pbfarmer, you can use diskpart to convert your disk to GPT. All data will be lost, so do it during W10 install. Hit Shift-F10 to open an Administrator Command Prompt. Type diskpart. Do a List disk and find your disk. Do a select disk #, where # is the number just found. Then do a convert GPT. Diskpart is very unforgiving. It says nothing and ask nothing, just does what told. Make double sure you have selected the correct disk. Please post for all of us what "Above 4G Decoding" means. Thanks and enjoy, John. ------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: TooQik
Date Posted: 10 Mar 2018 at 12:07pm
Here's a couple of articles that explain the boot process and the hardware address mapping that occurs for the PCI and PCIe buses: http://resources.infosecinstitute.com/system-address-map-initialization-in-x86x64-architecture-part-1-pci-based-systems/#gref" rel="nofollow - http://resources.infosecinstitute.com/system-address-map-initialization-in-x86x64-architecture-part-1-pci-based-systems/#gref http://resources.infosecinstitute.com/system-address-map-initialization-x86x64-architecture-part-2-pci-express-based-systems/#gref" rel="nofollow - http://resources.infosecinstitute.com/system-address-map-initialization-x86x64-architecture-part-2-pci-express-based-systems/#gref My understanding may not be 100%, but what I gather from reading the two articles is the current motherboard BIOS doesn't allow the system address mapping to exceed 4GB, which is a legacy amount to support 32 bit systems. When the eighth GPU is being added it's unable to map the PCIe device to the system memory map as it exceeds the 4GB of system memory. Motherboard manufacturers need to include code to allow this limitation to be overcome ie. Above 4G decoding. |
Posted By: MisterJ
Date Posted: 11 Mar 2018 at 12:35am
|
Thanks, TooQik. It looked like a great article, but the site is really bad. It kept displaying stuff I did not want and then gave a full screen display about "phishing". I closed the site and will avoid it, but learned a lot before I surrendered. Booting and allocating RAM is very complicated and I see why a switch to get rid of the 4GB limit is required. I hope all of you needing it are pestering ASRock to implement it permanently. I also wonder if AMD could put the switch in their area so all boards get it. Perhaps some need to petition AMD for the function. Thanks and enjoy, John.
------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: JohnM
Date Posted: 11 Mar 2018 at 1:02am
You can do it in situ with Windows 10 using mbr2gpt.exe. I made the same mistake myself and this worked perfectly. https://www.intel.co.uk/content/www/uk/en/support/articles/000024558/memory-and-storage/intel-optane-memory.html" rel="nofollow - https://www.intel.co.uk/content/www/uk/en/support/articles/000024558/memory-and-storage/intel-optane-memory.html Don't worry about the title of the URL - it isn't about Optane at all. It's just that you need UEFI boot in order to use Optane. ------------- ASRock Fatal1ty AB350 ITX P4.90, AMD Ryzen 5 2400G, 2x8GB Corsair CMK16GX4M2A2666C16, 250GB Samsung 960EVO, 500GB Samsung 850EVO, 4TB WD Blue, Windows 10 Pro 64, Corsair SF450, Cooler Master Elite 110 |
Posted By: MisterJ
Date Posted: 11 Mar 2018 at 3:50am
|
Cool, thanks, JohnM! Enjoy, John. ------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: ASRickroll
Date Posted: 11 Mar 2018 at 3:37pm
|
Thanks everyone The original BIOS that shipped with my board (v3.10) did not have "Above 4G Decoding" option. However I updated the BIOS to v4.40 prior and the option now shows up. What I did try was to boot with 7 GPUs, enter BIOS and enabled the option, save, restart, turn off, insert the eight card. It didn't help though, but I haven't tried it with CSM disabled and UEFI boot disk (is this a necessary step). I am too tired now and actually resorted to putting the 8th GPU in another PC. I will give it a try again later next week. Appreciate the help guys I hope ASRock engineers hear us
Do you have contact details of ASRock BIOS engineers team? I want to bother them as well
|
Posted By: MisterJ
Date Posted: 12 Mar 2018 at 12:00am
|
ASRickroll, seems like I read that UEFI is required. If so, Disable CSM then you must have W10 installed in UEFI mode which requires a GPT Style system disk. You may need to reinstall W10. Pleas tell us where the
"Above 4G Decoding" option is in your BIOS/UEFI. Thanks and enjoy, John. ------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: benb
Date Posted: 12 Mar 2018 at 6:02am
|
" rel="nofollow -
First lets see if the board can initialize properly with all the video cards. 0) Clear your CMOS , want to clear out anything the bios has learned about how to initialize the PCIE bus. 1) Disconnect your boot drive. 1a) We just want to make sure the board can initialize the PCI-E bus properly with all those cards, which has nothing to do with the OS. 2)Boot to the bios configuration , go to Boot-> the CSM -> Change each one to UEFI only ,(Launch PXE, Launch Storage, Launch Video) 3) Change the Boot Option 1 to be UEFI: Built-in EFI Shell 4) Enable "Above 4G Decoding" 5) Save the configuration and reboot. The machine might reset a few times as it tries to learn how to initialize the PCIE bus. When the machine stabilizes to a constant Dr Debug code, look at the code. Match the code up with one from here https://www.asrock.com/support/faq.asp?id=334 . A0-A7 just means it is waiting for sata disk with UEFI load. If you get 92-99 that is PCI-E error, so try starting with say 3 video cards first and seeing if the PCI-E bus initializes correctly.
|
Posted By: JohnM
Date Posted: 12 Mar 2018 at 11:28am
|
If you do need to change to UEFI boot you don't need to reinstall Windows 10. You can run the mbr2gpt.exe command that's included in the OS and it converts an existing MBR partitioned drive to GPT. I used it myself and it works. Details are here: https://www.intel.co.uk/content/www/uk/en/support/articles/000024558/memory-and-storage/intel-optane-memory.html" rel="nofollow - https://www.intel.co.uk/content/www/uk/en/support/articles/000024558/memory-and-storage/intel-optane-memory.html ------------- ASRock Fatal1ty AB350 ITX P4.90, AMD Ryzen 5 2400G, 2x8GB Corsair CMK16GX4M2A2666C16, 250GB Samsung 960EVO, 500GB Samsung 850EVO, 4TB WD Blue, Windows 10 Pro 64, Corsair SF450, Cooler Master Elite 110 |
Posted By: ASRickroll
Date Posted: 12 Mar 2018 at 11:52am
I just checked again. The name of the option is actually "Above 4G MMIO Enable" which is disabled by default. I wonder if this is the same thing?
Thanks benb, I will try that. The new option that shows in recent BIOS versions is "Above 4G MMIO Enable" (disabled by default) - is this the same thing as "Above 4G Decoding"?
|
Posted By: benb
Date Posted: 13 Mar 2018 at 4:50am
|
Above 4G MMIO is the same as above Above 4G decoding. The 4G MMIO Limit , needs to be less than 42 bits for NVIDIA cards. 40 Bits the default should be fine. Only thing I see that might be an issue is usually if you have the MMIO Limit bits you also have a another option for the Base Address. Basically this sets up a range of addresses the gpu cards are mapped to. Here is an example of what I am talking about on a supermicro motherboard. https://www.servethehome.com/nvidia-smi-issues-get-nvidia-cuda-working-grid-tesla-gpus/" rel="nofollow - https://www.servethehome.com/nvidia-smi-issues-get-nvidia-cuda-working-grid-tesla-gpus/ Anyways I had to roll back to my custom 2.10 version because I couldn't get it to boot with the Xeon Phi card , I have a feeling it is MMIO Limit bits need to be correct or there is some issue with the bios. I get a bios error 07 , which is memory error. Basically the Xeon Phi gets memory mapped, and I am pretty sure that this memory mapping is incorrect. BIOS might be fine for Nvidia gpus with above 4g decoding, Xeon Phi is very picky. Bad thing is the Xeon Phi keeps the machine from posting at all.
|
Posted By: MisterJ
Date Posted: 13 Mar 2018 at 5:15am
|
Thanks, ASRickroll. I suspect my board does not have the option but would like to ask under what major section is
"Above 4G MMIO Enable". Something like Advanced-AMD PBS. I am still running UEFI 2.00 since ASRock has not released anything for X399 since 11-28-2017. Thanks and enjoy, John. ------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: pbfarmer
Date Posted: 13 Mar 2018 at 5:36pm
Sooo.... After days of tinkering, I've hit a dead end. Figured out my CSM setting was getting auto-reset because my GPUs were using custom roms with invalid UEFI signatures. Fixed all the cards, updated my Win10 install to GPT/UEFI, and finally got the CSM setting to stick, still no 8th card. Then discovered (between this thread and a support email) that the latest UEFI rom for my motherboard (Fatal1ty X370 Gaming K4) contains the 4G MMIO setting, so I installed the latest rom and played around w/ install order of cards for a while. Actually got the board to POST w/ 8 cards plugged in, but the system never actually presented the 8th card - seems resources were never assigned. I think this minor win was more due to the order I was plugging in the cards than the 4G setting, but it doesn't really matter, since the 8th card was still non-functional. Digging into the Device Manager and System Info in Windows, as well as the UEFI shell, I've confirmed that the UEFI is never assigning resources above 4G, regardless of the Above 4G MMIO setting. Other settings including MMIO bits, CSM and PCIe lane speed have no effect on this either. It appears @benb is correct - there needs to be an additional setting in the UEFI for this Above 4G setting to be useful. Either an explicit control to define a >4G base address, or a control to restrict the <4G available space, forcing assignments above 4G. There's mention in other forums of a TOLUD (top of lower usable dram) setting on some ASRock Intel boards, which sounds like it performs exactly this function. I guess I need to go back to support and ask about this additional setting now. @benb - can you confirm (screenshot maybe) that your GPU(s) are indeed using >4G addresses w/ your custom rom? Also, what board are you using, and could you possibly post your rom somewhere? Even if I can't use it, i'd like to see if I can pick out the change they made using a hex editor. Thanks
|
Posted By: MisterJ
Date Posted: 13 Mar 2018 at 11:09pm
|
" rel="nofollow - I'm wondering if any of you have run GPU-Z ( https://www.techpowerup.com/gpuz/ ) to see if it tells you anything of interest. Worth a try? Enjoy, John.
------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: pbfarmer
Date Posted: 14 Mar 2018 at 6:10am
Unfortunately, in general the machine doesn't even post w/ the 8th card installed. In the rare case that it does, it's only because the card has somehow gone undetected by the UEFI, hence Windows doesn't know it exists.
|
Posted By: MisterJ
Date Posted: 14 Mar 2018 at 7:46am
|
" rel="nofollow - pbfarmer, I assume you have tried various sets of 8 cards. Are all these the same make/model? If 8 will not boot, can you remove any one and boots OK? Have you every looked in the device manager (when you can boot - maybe only 7 cards)? Use the View-'Resources by type'. If it is not too huge, post a screenshot or upload it so we can take a look. I am really curious what it looks like with so many cards. Please also post a screenshot of GPU-Z. Please do not forget me on the request of where the "Above 4G Decoding" option is located. Thanks and enjoy, John. ------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: pbfarmer
Date Posted: 14 Mar 2018 at 9:10am
All 8 cards are Sapphire Nitro+ RX 580 8GB models (2 are the SE version, which is just a different shroud + LEDs.) Any combination of 7 cards on any combination of 7 slots works perfectly. Simply adding the 8th card, regardless of which one, or which slot, causes the UEFI to stop loading w/ a '0d' error. After removing the 8th card, and clearing the CMOS (to clear the sticky error state,) the PC resumes normal operation. Here's the resource (memmap) list - I've highlighted the biggest segments for each of the 7 cards. As you can see, 1.) they're each using over 256MB a piece, and 2.) nothing is mapped above 4GB. I'm assuming the issue is that the device mapping space spills over 2GB once the 8th card is installed.  Not sure what you want to see from GPU-Z - there's no way to really show details of 7 cards from that app. As for the Above 4G MMIO option, here's a screenshot:  Thanks! |
Posted By: MisterJ
Date Posted: 14 Mar 2018 at 9:49am
|
Thanks much, pbfarme. Your Device Manager does not look that different from mine which obviously stops at 4GB (Also RX 580). I really don't need to see the GPU-Z, but do wonder if there is any information there of interest. I am going to open my UEFI and compare to your screenshot - thanks. I want to do some studying of what you posted and think after seeing my UEFI. If I have any thoughts, I will post. Enjoy, John.
------------- Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD |
Posted By: ASRickroll
Date Posted: 17 Mar 2018 at 1:27pm
|
Thanks guys great community AMD came back to me and reassured that it's not Ryzen limitation, here's their reply (my name omitted)
By the way, I tried something crazy (TRY IT AT YOUR OWN RISK). I installed all 8 GPUs but left the 8th GPU's USB cable unplugged, booted the system, it POSTed OK, then I plugged the 8th GPU's USB cable during Windows boot and I could find the 8th GPU in the device manager but it had this error "This device cannot find enough free resources that it can use. (Code 12)
|
Posted By: kerberos_20
Date Posted: 17 Mar 2018 at 4:46pm
|
amd pbs features doesnt seems to be working on ryzen 1, not sure on ryzen 2, but for sure all those options on mine 1600x doesnt do anything at all ------------- http://valid.x86.fr/diq4l4" rel="nofollow"> http://www.userbenchmark.com/UserRun/47132492" rel="nofollow - userbenchmark |
Posted By: Proper
Date Posted: 10 Jun 2018 at 11:40am
|
I am not sure if any info on this would be relevant but I will post it. I had issues using large number of GPUs with Threadripper platfrom X399, after fairly long exploration of the issue I found that other devices on the board use up address allocation space in limited buffer and leave no space for more than 7 PCIe devices. I was able to free up this space by disabling devices like sound controller, individual USB ports. This allowed me to run total of 12 GPUs. I maybe able to do more but above 4g decoding is not supported on my board at the moment, I am waiting for that. I wonder if doing same thing on Ryzen board may solve the issue.
|
Posted By: kerberos_20
Date Posted: 12 Jun 2018 at 7:10pm
|
bios with 1.0.0.2 agesa has 64bit adressing (above 4g) option ------------- http://valid.x86.fr/diq4l4" rel="nofollow"> http://www.userbenchmark.com/UserRun/47132492" rel="nofollow - userbenchmark |
Posted By: pbfarmer
Date Posted: 18 Jul 2018 at 5:50pm
|
It worked!!! I updated to the latest UEFI, which added another 'Above 4G' option. But that one doesn't seem to work properly either, as it allowed me to boot w/ 8 GPUs attached, but still didn't assign resources to the 8th one, and no devices were actually assigned addresses above 4G. Next, tho I had already previously disabled any audio, which didn't help, I've now also disabled some USB and ethernet controllers, and viola - 8th GPU assigned and available in Windows! Thanks for the tip!
|