|
5th Titan Z = 'bF" mobo startup error |
Post Reply 
|
Page 123> |
| Author | |
GPUnk 
Newbie 
Joined: 03 Jun 2015 Location: CT Status: Offline Points: 19 |
 Post Options
 Thanks(0)
 Quote Reply
 Topic: 5th Titan Z = 'bF" mobo startup errorPosted: 03 Dec 2015 at 5:57am |
|
Xaltar, how have you been. A little late on this, but I was eventually able to get the 11th Core working, having a Titan X as primary, and the full 5 Titan Zs for rendering as external GPU cards. I then proceeded to test a 12th GPU (my old 660Ti) and it worked, so I got another Z and successfully booted into Win 7 64 with 13 GPU. What made this all work was disabling the LSI SATA ports in my BIOS and using strictly the 6 default SATA ports. It was funny, just by trial I discovered it. I would find after messing with BIOS some drives weren't recognized but all GPU worked. I then put 2 and 2 together and realized that the drives all had something in common, connection to the extra LSI SATA ports (on the X79 Extreme11). So I moved them to my reg SATA interface on the board, shut off LSI in BIOS, and now I have it all working.
I have since went on to buy water-blocks for all 6 Titan Z cards, and I have them water-cooled. That was crazy, a lot of trial and error getting that set up right, but now have it all running.
|
|
|
X79 Extreme 11 / Intl Core i7 3930K/ 32gb DDR3 1600 Corsair Vengnce/ Win 7 64bit / Thermaltake 1500 W PSU / GTX Titan X
GPU Rendering - 4 Titan Z (external USB 3.0 PCIE riser)/ EVGA 1600 W PSU (2) |
|
 |
|
|
GPUnk
Newbie
Joined: 03 Jun 2015 Location: CT Status: Offline Points: 19 |
Post Options
Thanks(0)
Quote Reply
Posted: 15 Jul 2015 at 3:15am |
|
So, was waiting for an answer on the ability of the user (via a specific parameter, or modification of settings) to modify address space allocation through the BIOS. ASRock R&D replied:
"Because the address randomly changed may cause some unexpected symptom or error on the system, our R&D team need to get the whole system including all devices, and then do the further check." I guess I am now past the 'try this (BIOS)' stage, and if I wanted to get any more insights it would require sending in my system for an eval. They don't have any ready answers or suggestions, and it appears they think testing would involve random and unexpected phenomenon, which leads me to believe I'm at the resource limit. I'm not going to send in for testing, maybe I'm past the point of where ASRock's own ready knowledge goes. But I'll keep my eye out for any new info that comes my way. By the way, I tried adding the 660Ti as a 12th connected GPU, and I hit the "bF" error again, even with the 4G decoding enabled. I also tried turning off any peripheral I could find in BIOS (USB 3.0, Audio, Lan), but to no avail. Anyone interested, my hardware is in my signature. Xaltar, you actually mentioned music recording in one of your earlier posts, and what is odd is that I never had any issue with removing and reinstalling my Creative Soundblaster X-Fi audio card. Given that it was going into a PCI lane, I thought it might get muffed up with all the tinkering. But nope, always works. Guess my PC likes music production...!
|
|
|
X79 Extreme 11 / Intl Core i7 3930K/ 32gb DDR3 1600 Corsair Vengnce/ Win 7 64bit / Thermaltake 1500 W PSU / GTX Titan X
GPU Rendering - 4 Titan Z (external USB 3.0 PCIE riser)/ EVGA 1600 W PSU (2) |
|
|
|
|
Xaltar
Moderator Group 
Joined: 16 May 2015 Location: Europe Status: Offline Points: 25073 |
Post Options
Thanks(0)
Quote Reply
Posted: 10 Jul 2015 at 12:21pm |
Hehe, glad to be of help :) Xfast RAM is an app
 I was suggesting you remove it if you have it installed. It improves performance but also gives access to the "memory hole", removing it may allow additional resources for your GPUs, I doubt it but its worth a shot, you can always install it again if it doesn't help  |
|
|
|
|
odiebugs
Newbie
Joined: 07 Jul 2015 Status: Offline Points: 193 |
Post Options
Thanks(0)
Quote Reply
Posted: 10 Jul 2015 at 9:43am |
|
Do you have a BIOS or a UEFI ?
So someone can help you, can you please post all the (model numbers ) of your hardware.
|
|
|
asrocking
|
|
|
|
|
GPUnk
Newbie
Joined: 03 Jun 2015 Location: CT Status: Offline Points: 19 |
Post Options
Thanks(0)
Quote Reply
Posted: 10 Jul 2015 at 9:18am |
|
I did not see the XRam listed as a parameter in the BIOS, but when I did look it up, if I am not mistaken, it actually looked like a utility you could download and then tweak your system with.
Just so I am getting you, you were imagining me seeing an already existing BIOS modifiable that I would be able to disable right? Or, were you actually thinking of me downloading the external utility and then tweaking something? I understand now that the OS offers a user a viewing lens to the BIOS. I guess a desirable BIOS will let everything load if you 'ask' it to properly, and a 'good' OS will be able to descriptively tell you what the issue specifically is, if there exists one, and give you possible options to rectify it. It is also more clear to me that there is not ever really a 'GPU-limit', but rather a 'resource limit'. In my case a resource limit may effectively be limiting the # of GPUs, without actually having had an intent to do so, per-say. It is because of this my search continues in looking to see what that resource is, and if I can negotiate with some part of the BIOS and OS to yield that additional resource. Kind of imagine it like the Matrix, trying to negotiate with the process to free the GPU core. Xaltar, you thus far have been both a Morpheus and an Oracle. Thanks!...
|
|
|
X79 Extreme 11 / Intl Core i7 3930K/ 32gb DDR3 1600 Corsair Vengnce/ Win 7 64bit / Thermaltake 1500 W PSU / GTX Titan X
GPU Rendering - 4 Titan Z (external USB 3.0 PCIE riser)/ EVGA 1600 W PSU (2) |
|
|
|
|
Xaltar
Moderator Group
Joined: 16 May 2015 Location: Europe Status: Offline Points: 25073 |
Post Options
Thanks(0)
Quote Reply
Posted: 08 Jul 2015 at 12:55pm |
|
The hardware addresses in modern systems are assigned dynamically. Its been a while but if memory serves each piece of hardware needs an IRQ (interrupt request), IO ranges and memory ranges. Things like GPUs need more IO and memory range allocation than other more simple hardware. If you look under device manager -> device properties -> Resources you will see what windows assigns to the device. So far as I am aware resource allocation is taken care of by PnP (plug and play) and allocated by windows. I know there used to be a way to force resource allocation but I have long since forgotten how to do it. This was back in the days of windows 98se trying to get 4 different sound cards to play nice together for a studio recording rig hehe.
Looking under resources in windows 8.1 I see there is a check box to allow changing the allocation manually but it is grayed out. The resource tab will also show you conflicts, if you open up the properties of the GPU that is flagged "!" it may give you more info as to where the conflict is coming from. Back in the days of windows 95/98 these problems were a lot more common place and building systems back then often meant spending hours allocating resources to hardware that was trying to share with incompatible devices, especially with purpose built systems like studio/sound setups. I suspect that the checkbox to allow manual allocation may become available in the advent of a conflict. Another thing that may be useful to try is uninstalling Xfast RAM. Xfast RAM makes memory in the "memory hole" available to the system. Memory holes in the past were often used as additional resource allocation on certain devices so removing Xfast RAM may give your more resources to play with. I know there used to be an option on old socket 3-7 (486-Pentium) boards to enable/disable the 15-16mb hole for ISA expansion cards but I have not read up on the technicalities in modern boards for quite some time. May be worth a look. Sorry about all the clarifications in brackets, I was trying to make sure anyone else reading could follow the conversation. Your particular case is a very interesting one
 Edited by Xaltar - 08 Jul 2015 at 12:56pm |
|
|
|
|
GPUnk
Newbie
Joined: 03 Jun 2015 Location: CT Status: Offline Points: 19 |
Post Options
Thanks(0)
Quote Reply
Posted: 08 Jul 2015 at 4:01am |
|
I actually tried disabling LSI Connector (extra non-boot SATA drive connections on mobo, I think), and USB 3.0 in BIOS, and tried installing the last rogue core...
Guess what happened...BSOD. Oh well.... Just a piece of info to share, 2 of my Z's (4 cores) are by default overclocked. I actually tried overclocking all of my active 10 GPU cores simultaneously for added render power insanity. With my active 10 GPU cards overclocked (core clock) by 100 MHz on MSI Afterburner, my 15 Amp breaker for rig trips. In fact, during an extended render w/o overclock (15 minutes), the power draw also tripped the breaker. Temps were all > 85 C across last I saw before I tripped. This makes me realize I am facing a more innate limit of electrical power if cards are maxed. So, I have to limit usage (ie lower temp, less active cores) on longer render projects, ie videos involving a lot of simulated lighting. (Lighting, and hair rendering, is a killer...) I would like that last core active for those shorter burst image renders. Some people who bitcoin attach their rigs to 20 amp outlets on their breaker to avoid tripping. The gaming, bitcoin, and rendering communities often come across the same issues, because of a desire to throttle GPUs. Many discussions surround manipulating the BIOS and motherboard (jumpers, pcie lanes) to achieve this. But thanks for the suggestion. I am wondering in my case where device manager and GPU-Z 'see' the 11th core, does this mean that I am actually beyond a need to manipulate the BIOS any further.
|
|
|
X79 Extreme 11 / Intl Core i7 3930K/ 32gb DDR3 1600 Corsair Vengnce/ Win 7 64bit / Thermaltake 1500 W PSU / GTX Titan X
GPU Rendering - 4 Titan Z (external USB 3.0 PCIE riser)/ EVGA 1600 W PSU (2) |
|
|
|
|
Xaltar
Moderator Group
Joined: 16 May 2015 Location: Europe Status: Offline Points: 25073 |
Post Options
Thanks(1)
Quote Reply
Posted: 08 Jul 2015 at 1:20am |
|
One thing I can think of that may help is disabling all unnecessary devices in BIOS, that may free up some address space for the additional GPU. I doubt it will help but its worth a shot while you wait
|
|
|
|
|
GPUnk
Newbie
Joined: 03 Jun 2015 Location: CT Status: Offline Points: 19 |
Post Options
Thanks(0)
Quote Reply
Posted: 08 Jul 2015 at 12:43am |
|
I received a response from ASRock R&D basically saying there is no restriction on the # of GPU's per say placed in a BIOS or motherboard. However, if address space is not allocated to a GPU, it may not function properly...
I don't know if they were referring to BIOS or OS as far as the address space goes, but I am going to assume they are speaking of BIOS. So, I am now asking them if there is a setting that can be modified in BIOS which will allow the user allocate address space to a GPU. I have no idea if that term 'address space' could mean more power, memory, or what....but I will see how they answer my question. You know, there seems to be a lot of people saying things like: "Windows 7 can only support 4 GPU (then 6, then 8 GPU in other posts)....", or "upgrade to Windows 8, and you will get one more GPU recognized by the OS", or "motherboards only typically support 7-8 GPU." But, how do people know this, it half feels like vague new territory sometimes. Also FYI, not only does device manager show 11 GPU (with one having a yellow triangle), but GPU-Z shows 11 GPU, just that one has an unknown BIOS. I tried less than 10 GPU will all the cards, and the 1 will always install itself if it was the one previously unrecognized. But as soon as I go above 10, then a new 11th takes its place, tries to install and cannot. That feels like a 'resource issue' with Windows 7 address space, but I have no idea what that resource space consists of, or how to stretch that space further. Anyway, I will update!
 |
|
|
X79 Extreme 11 / Intl Core i7 3930K/ 32gb DDR3 1600 Corsair Vengnce/ Win 7 64bit / Thermaltake 1500 W PSU / GTX Titan X
GPU Rendering - 4 Titan Z (external USB 3.0 PCIE riser)/ EVGA 1600 W PSU (2) |
|
|
|
|
Xaltar
Moderator Group
Joined: 16 May 2015 Location: Europe Status: Offline Points: 25073 |
Post Options
Thanks(0)
Quote Reply
Posted: 04 Jul 2015 at 5:49pm |
|
Nice to see you are making progress. I applaud your patience as well as the constant updates
Good luck getting that beast of a machine purring like a kitten
Edited by Xaltar - 04 Jul 2015 at 6:14pm |
|
|
|
|
Post Reply
|
Page 123> |
Tweet
|
| Forum Jump | Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum |
Topic Options
GPUnk wrote: