Intel's Atom C2000 chips are bricking products
Printed From: ASRock.com
Category: Technical Support
Forum Name: Intel Motherboards
Forum Description: Question about ASRock Intel Motherboards
URL: https://forum.asrock.com/forum_posts.asp?TID=4376
Printed Date: 26 Dec 2024 at 6:36pm
Software Version: Web Wiz Forums 12.04 - http://www.webwizforums.com
Topic: Intel's Atom C2000 chips are bricking products
Posted By: Piru
Subject: Intel's Atom C2000 chips are bricking products
Date Posted: 07 Feb 2017 at 9:52pm
|
Intel Atom C2000 chips have a flaw that will lead to total bricking of the system. The issue has been covered by for example The Register: http://www.theregister.co.uk/2017/02/06/cisco_intel_decline_to_link_product_warning_to_faulty_chip/" rel="nofollow - http://www.theregister.co.uk/2017/02/06/cisco_intel_decline_to_link_product_warning_to_faulty_chip/ The problem is confirmed by this Intel Errata document: https://www-ssl.intel.com/content/dam/www/public/us/en/documents/specification-updates/atom-c2000-family-spec-update.pdf" rel="nofollow - https://www-ssl.intel.com/content/dam/www/public/us/en/documents/specification-updates/atom-c2000-family-spec-update.pdf As far as I can tell ASRock has 3 models that are potentially affected: http://www.asrockrack.com/general/productdetail.asp%3FModel=C2550D4I#Specifications%20%20%20%20%20" rel="nofollow - http://www.asrockrack.com/general/productdetail.asp?Model=C2550D4I#Specifications http://www.asrockrack.com/general/productdetail.asp%3FModel=C2750D4I#Specifications%20%20%20%20%20" rel="nofollow - http://www.asrockrack.com/general/productdetail.asp?Model=C2750D4I#Specifications http://www.asrockrack.com/general/productdetail.asp%3FModel=1U12LW-C2750#Specifications%20" rel="nofollow - http://www.asrockrack.com/general/productdetail.asp?Model=1U12LW-C2750#Specifications Some questions to ASRock: - Are ASRock boards affected by this issue? - If ASRock boards are affected, what will ASRock do to remedy this situation? Will the affected boards be RMAd? |
Replies:
Posted By: wantastic
Date Posted: 09 Feb 2017 at 6:10am
|
" rel="nofollow - I'd second this - I've interestingly already had one of the C2750D4I's fail just a few months ago just over 2 years into it's life span, which I had thankfully replaced under warranty, however if these products are possibly affected by this issue, I would imagine that the replacement I received may also still have this same problem. If ASRock products are indeed affected, what happens if one of these chips fail after the warranty period? Would these still be covered if the issue is classed as a design flaw by Intel? Would there be any opportunity to replace the potentially faulty chips prior to them breaking?
|
Posted By: Xaltar
Date Posted: 09 Feb 2017 at 6:15am
|
I would take this up with Tech Support directly. Please do post back your findings here though for anyone else interested. http://event.asrock.com/tsd.asp" rel="nofollow - http://event.asrock.com/tsd.asp ------------- 
|
Posted By: LucidWaker
Date Posted: 18 Feb 2017 at 9:20pm
|
https://www.servethehome.com/intel-atom-c2000-series-bug-quiet/" rel="nofollow - https://www.servethehome.com/intel-atom-c2000-series-bug-quiet/ ASrocks respons here: "ASRock Rack ??Not seeing major issues from customers yet. Working with a vendor on this issue. Current course of action is to replace boards under three-year warranty." I am using an ASRock C2550D4I and have send a support request through the ASRock Rack pages. I will give an update here on their response. As far as I am aware, the B0 stepping for Intel Atoms C2550 is affected by the fault and no other steppings exist (Please correct me if I'm wrong). That would mean that a replacement board of the same type would have the same issue present, so a direct replacement is not a structural solution here, unfortunately. ------------- ASRock C2550D4I User |
Posted By: wardog
Date Posted: 18 Feb 2017 at 9:36pm
Have you seen note, in plain simple wording, of just how heavy a workload over what period of time Intel is talking of when they refer to this "degradation' and subsequent failure? Cisco's switches I can see a relationship to workload and a time to detonation.  |
LucidWaker wrote:Posted By: LucidWaker
Date Posted: 18 Feb 2017 at 10:11pm
" rel="nofollow -
In the updated product specification, the bug we're dealing with is stated as follows: AVR54: System May Experience Inability to Boot or May Cease Operation Problem: The SoC LPC_CLKOUT0 and/or LPC_CLKOUT1 signals (Low Pin Count bus clock
outputs) may stop functioning. Implication: If the LPC clock(s) stop functioning the system will no longer be able to boot. Workaround: A platform level change has been identified and may be implemented as a workaround
for this erratum. I have no real knowledge on the workings of CPU's on this level so I don't know if this is some basic functionality where other functionalities revolve around (this seems plausible as it apparently plays a role in booting the system) or that this is related to load driven operations. As far as I understand it, it's supposed to be related to degradation of the actual semiconductor structure in the chip. Degradation rates of semiconductor materials will increase with heavier loads, but server boards are usually powered on for longer periods, so it will keep degrading over time regardless of load. So, I think load will play a part in how fast fatal degradation sets in, but it will happen over time regardless. I read in one of the related articles that a fix could possibly be to reduce voltages in the related chip structures to decrease degradation rates but I don't know if ASRock themselves are able to achieve this on existing motherboards. ------------- ASRock C2550D4I User |
Posted By: wardog
Date Posted: 19 Feb 2017 at 1:42am
" rel="nofollow -
Thanks LucidWalker. Platform level, meaning other than the CPU itself. As you imply in the below. Wonderful,.. Intel. What is this now? The third major Erratum is as many generation's coming out of Intel?
|
Posted By: wardog
Date Posted: 24 Feb 2017 at 6:39pm
| http://fudzilla.com/news/processors/42942-intel-c3000-atom-chips-out" rel="nofollow - http://fudzilla.com/news/processors/42942-intel-c3000-atom-chips-out |
Posted By: LucidWaker
Date Posted: 26 Feb 2017 at 12:52am
|
" rel="nofollow - Heard nothing yet from ASRock customer service. In the meantime weighing my options to replace my C2550D4I with another mITX board with CPU, might as well grab the opportunity to upgrade the CPU to fully support a minimal of two FullHD stream through Plex... Any idea what ASRock is planning to do with the Atom C3000 series? ------------- ASRock C2550D4I User |
Posted By: C2000Brick
Date Posted: 26 Feb 2017 at 2:51pm
| I really hope ASRock will not try to sweep that issue under the carpet. I bought a C2750D4I in October 2015 an have a 3 year warranty. I expect a clear statement from ASRock that also later failures caused by this issue are covered. |
Posted By: wardog
Date Posted: 26 Feb 2017 at 5:37pm
|
" rel="nofollow - OK. While I do not work for ASRock in any way, shape, or form, this talk is getting out of hand. C2000Brick, please provide links to where on Intels forum you have been previously lambasting them for a defective product? None? I didn't think so .......... I do understand everyone is nervous and upset over this Intel blunder. I would be too. Yet enough is enough here. So, let's all take a deep breath and cool off until ASRock, and more importantly the gorilla in the room, Intel, do decide a course of action. By all means carry on discussing it. But let's cease with the nuclear tactics toward both ASRock and Intel. First and last warning all. ... wardog, Moderator |
Posted By: MisterE
Date Posted: 26 Feb 2017 at 11:10pm
|
" rel="nofollow - This problem possible bricked my hardware: https://forums.freenas.org/index.php?posts/351822/ I am still investigating. |
Posted By: Xaltar
Date Posted: 27 Feb 2017 at 12:10am
|
I have to agree with Wardog here. This should be an informative thread that ASRock tech support can look at and benefit from so they can take whatever steps they can to correct the issue/s outlined here. Please bare in mind that ASRock can only provide such fixes as they have been provided by intel, we are not talking about a manufacturing error here, it is a CPU level issue that is the sole responsibility of intel to correct. Once intel have created a fix (if that is even possible) they will then likely distribute firmware/BIOS updates to board partners to tune to their product lines and release. Just like Wardog, I do not speak for ASRock, this is just common sense and a little speculation on my part. -------------
|
Posted By: parsec
Date Posted: 27 Feb 2017 at 7:25pm
|
A vaguely similar situation involving Intel was the P67 chipset issue, where some of the SATA ports could deteriorate over time, and fail. A major difference between that issue and the Atom C2000 processor situation is Intel itself announced the problem shortly after the P67 chipset was released and in use in mother boards. So far, Intel has not seemed to acknowledge the situation with the Atom C2000 processor in the same way. In the case of the P67 chipset, Intel released an updated version of the P67 chipset, labeling it "Generation 3", and provided it to mother board manufactures. The manufactures included it in their P67 products, using the "Gen 3" label to differentiate them in the marketplace. As I recall, mother boards with the defective P67 chipset could be returned to retailers without any questions, and the owner could then get a board with the updated, P67 Gen 3 chipset. Intel absorbed all the costs of this recall. Assuming the C2000 processor has this problem (not saying it doesn't) then it is up to Intel to make the first move and work with mother board manufactures to deal with it. We will need to wait and see what Intel's reaction to this is, and I'm sure the mother board manufactures are also dependent upon what Intel does. ------------- http://valid.x86.fr/48rujh" rel="nofollow"> 
|
Posted By: LucidWaker
Date Posted: 28 Feb 2017 at 5:23pm
|
RIP(?) ASRock C2550D4I 28-11-2015 - 27-02-2017 Yesterday, a family member sent me a message saying my server was unreachable. Had a chance to check it out this morning and it wasn't showing FreeNAS on the standard IP, Plex was not working and no image on the monitor that I have hooked up. Hard reboot showed no booting on the screen. Unfortunately, I'm no expoert on this area, was glad to get it up and running back when I built it... Don't know any other way of troubleshooting this, anyone any ideas?  Was already looking into the possibility to do a preventative hardware swap but time was not on my side... ------------- ASRock C2550D4I User |
Posted By: Xaltar
Date Posted: 28 Feb 2017 at 5:26pm
|
Contact Tech Support directly. There is not much that can be done to troubleshoot SoC based systems aside from swap in another PSU and RAM. http://event.asrock.com/tsd.asp" rel="nofollow - http://event.asrock.com/tsd.asp -------------
|
Posted By: LucidWaker
Date Posted: 01 Mar 2017 at 7:56pm
|
Turned off and on again, worked for a few hours, but then it froze again. Monitor hooked up showed lines like: "101139 D - 0xfffff80025b0d700 (zio_read_intr_6_0)" Turning off and on again after sleeping on it and now everything is blank. Tried IPMI to further troubleshoot but I never actively set it up so was hoping I could deduce some default settings from the ASRock IPMI manual. 192.168.3.1 appeared to be the default address, it was pingable, but none of the IPMI tools I tried worked (tools wouldn't load or couldn't get them compiled correctly). Through Putty the connection got refused. Looks like it's about RMA time. Unfortunately the Dutch ASRock support just tells you to go the your reseller, but I'm going to try to do it directly through them anyway as I don't have the best experience with resellers handling my RMA's... "Lost" serialnumber stickers causing loss of warranty, those kind of things... Bonus (not my thread but just to illustrate the C2000 issue): https://www.reddit.com/r/freenas/comments/4x1kh1/asrock_c2550d4i_sudden_death/" rel="nofollow - https://www.reddit.com/r/freenas/comments/4x1kh1/asrock_c2550d4i_sudden_death/ ------------- ASRock C2550D4I User |
Posted By: bitlisz
Date Posted: 11 Mar 2017 at 4:18am
|
So this is why my C2550D4I sent to RMA 2 times last year, as no boot, just IMPI. 1st board failed after 20 months use in last July (SN: 49M0XD028***), They happily replaced after one month wait. 2nd board survived only 5 days (SN: 66M0XD9F0***), 3rd board got after 2 weeks, and still Rocks from last Sept (SN: 64M0XD9F0***). I hope they can have a clear statement which Serial Number range affected.
|
Posted By: stewmohr
Date Posted: 10 Apr 2017 at 6:04pm
|
I believe I'm seeing similar symptoms. System was running perfectly for 11 months, over last 24 hours, the system locked up once (strange), rebooted, it ran for a few hours, locked up, reset again and now will not post, but can access IPMI fine. When power is applied, fans spin for 2-3 seconds and then stop. IPMI reports server is "powered off" and trying to power on via IPMI is not successful, tries 3 times and then fails. Will commence RMA process, bought from the US into Australia, hope it goes ok. Really disappointed. |
Posted By: FredDB
Date Posted: 17 Apr 2017 at 9:10pm
|
" rel="nofollow - Hi, I'm the owner of a ASRock C2750D4I mother board. I've bought this MB at the end of march 2017. At that time, I was not aware about this problem with the intel Atom C2000 series. I've contacted the shop where i've bought this MB and they told me that the problem was fixed beginning of february 2017. Someone knows if this is true? If this is true, how can i identify if my cpu is part of the fixed batch (new revision number or other ?). Thank you! Fred |
Posted By: jabesse
Date Posted: 19 Apr 2017 at 7:43pm
|
Hi, I've probably seen this problem on my C2750D4I twice already. The 1st MB went dead after about a year. The 2nd MB went dead in a couple of days. Just received my 3rd MB. Hope the issue is fixed on this one... I'll look up the serial numbers when I get home. Best regards, Jeroen
|
Posted By: FredDB
Date Posted: 21 Apr 2017 at 2:20am
|
Hi Jeroen, I'm very curious about the serial of your mother board. Mine is 69S0XW000... (... = 3 digits). I bought it end of march 2017. Fred |
Posted By: jabesse
Date Posted: 21 Apr 2017 at 5:43am
|
Hi, 1st: 4BM0XD016... 2nd: 66M0XD9F0... 3dr: 73S0XW002... Hope it helps...
|
Posted By: stewmohr
Date Posted: 13 May 2017 at 9:51am
|
It if helps, my replacement board that support confirmed has the fix has serial starting 73S0XW003... |
Posted By: wilkie
Date Posted: 07 Jun 2017 at 8:43am
| My replacement board has a serial starting with 73S0XW003... and it stopped working after 2 days of use. :/ :/ |
Posted By: wanstronian
Date Posted: 16 Oct 2017 at 8:57pm
|
" rel="nofollow - My 2015-bought C2550 has just bricked. ASRock's RMA page suggests that warranty is only 1 year - can anybody comment? I'm in the UK so any RMA process is going to take weeks - no access to my data in all that time. This is really unacceptable. I can either wait for ages in the hope of getting a new board, or change to a non-C2000 board which will mean changing my case, buying SATA controllers etc. and spending a whole lot more. Really wish I'd just bought an off-the-shelf Netgear ReadyNAS - I'm sure it would still be working right now, and I'd have spent a third of the cost :-(
|
Posted By: mellhen
Date Posted: 25 Oct 2017 at 10:03pm
|
My C2250D4I just broke. Two days ago the System froze without any apparent reason (no temperature or voltage warnings). A restart brought it back for a couple of ours. It froze again. Then, when manually restarted it did not show any POST and after a few second it turned it self off. I suspected the PSU to be broken and tried three different units. With all three it showed the same symptoms. The IPMI is still accessible, but did/does not indicate any problems. EDIT 1: I contacted my dealer for support, awaiting the reply. Can someone confirm, that ASRock extends the warranty for c2000 bug affected products, as stated here: "ASRock Rack: Not seeing major issues from customers yet. Working with a vendor on this issue. Current course of action is to replace boards under three-year warranty." https://servethehome.com/intel-atom-c2000-series-bug-quiet/" rel="nofollow - https://servethehome.com/intel-atom-c2000-series-bug-quiet/ S/N: 59M0XD001814 Firmware: 0.19.0 EDIT 2: I got a replacement board. Apparently it was the CPU. ASRockRack Europe replaced the board with a brand new one (including SATA cables, manual, packaging). S/N: 75S0XW003087 Revision: 1.03 (With B0 stepping of Atom CPU) BMC F/W: 0.30.0 Because I own another potentially affected board with C2750 CPU, the very friendly customer support of ASRockRack Europe told me: "We now provide 4-year manufacture warranty our C2750D4I and c2550D4I boards ; if you ever experienced a sudden No POST with your board, please contact us. We can run a simple test to find the telltale sign. If the issue is confirmed, we will help you replace the board under our (extended) warranty policy."
Hope that helps. |
Posted By: wantastic
Date Posted: 11 Feb 2018 at 7:17am
|
I just wanted to share my update from about a year ago - my second C2750D4I appears to have been hit with the same problem as my first with the ability to no longer POST, no video, powers off after seconds. I've just fired an e-mail over to the ASRock support guys after reading about the extended 4 year warranty - I really wasn't aware of this, so many thanks for recording this in the forum! I'm about 4 months from the end of my extended warranty in that case. Based on what I've read, the newer replacement boards are fixed and no longer exhibit the problems which appear to cause these issues (please correct me if I'm wrong there!). My plan, assuming the issue is indeed confirmed with ASRock, and a replacement board sent out to me, I'll ensure I update you on this forum to let you know what happens, or if I ever run into any other problems. |
Posted By: wantastic
Date Posted: 17 Feb 2018 at 4:24am
|
So it turns out from contacting ASRock, they've been very helpful and I've ran through some troubleshooting steps with them - it turns out that a bit of a giveaway seems to be the +3V, +5V, and +12V rails as viewed from the BMC interface - on both of my boards that have failed, these all show as "Not Available". The CMOS battery voltage also never changes - even if you take the battery out - it's never refreshed. My other symptoms include the system powering up after being disconnected from the mains for a short while, but never POSTing. Then, after subsequent power cycles, the board routinely powers itself off after a couple of seconds. Although they've not gone into any further detail, the above was confirmed by them to be a defective board (after swapping PSU and testing that this wasn't at fault), and I'm happy to say that ASRock have been very quick at confirming the issue and generating an RMA for me with no hassles. Despite the two issues I've now had with this particular board and me now looking to be sent a third, I cannot fault the support ASRock have provided, and as I believe this is largely not their fault and is predominately all down to the issue with the Intel chip, the fact that ASRock have provided an additional year's extended warranty and been helpful all the way, this alone means I'll buy from them again. I can't say what would happen if anyone happens to be outside of the now 4 year warranty, I suspect you won't be so lucky, but kudos to ASRock for extending it by a year in the first place! |
Posted By: wantastic
Date Posted: 28 Feb 2018 at 8:50pm
|
Hi all, Just thought I'd drop a quick update, ASRock have sent me out a replacement board which I'm yet to test but I'm hopeful this should be the last time it's needed on this. Just for anybody else's interest, my replacement system serial number starts differently to the others mentioned - 7BS0X..... |
Posted By: asriznet
Date Posted: 16 May 2018 at 7:06am
| " rel="nofollow - Any updates on the replaced board? |
Posted By: wantastic
Date Posted: 18 May 2018 at 6:25am
|
Hi @asriznet, mine's still working, however I've had to power it off for the last month due to me working on the house and having to constantly turn power off for sockets. In my previous experience, my boards have lasted 1-2 years before showing signs of problems, so I suspect, and this is only assuming if there's still a fault in the current batch of boards being sent out, that it won't be until 2019 until I see any more issues. One interesting piece of info I have found, which I'm not sure if I need to worry about or not though, is that the CPU itself still seems to report a B0 stepping. I run Linux on my system, and running "cpuid | head -n15" outputs info from the CPU itself. On my system is reports an Avoton B0 stepping, whereas the errata that Intel released was addressed in the C0 stepping. According to the motherboard box though, it does say it's C0 stepping, so I'm not sure if it's a reporting fault in cpuid, or if it is indeed something sinister. Sadly, the only way for me to tell for certain is to pry the heatsink off the chip on the motherboard and check that's actually written on the chip itself, which with the house work I've been doing lately, hasn't lent me the time to do! I'm not sure how many people still track this thread, but if anyone else who's had replacement boards is able to confirm if their system reports a B0 or C0 stepping, that would be helpful - if some C0's are being reported, that would help suggest that fixed CPUs are indeed in the wild, and it might just be my tool, being a tool. |
Posted By: asriznet
Date Posted: 18 May 2018 at 6:56am
|
" rel="nofollow - Just curious does the failed board show any anomalies in CPU temp? my sensor readings are showing excellent readings, it's even not hot enough that I can touch the heatsink with my bare hands. my issue is intermittent but more frequent these days that the board will auto shuts off and when it attempts to boot again will not POST. when it's encountering the no POST issue, IPMI sensor readings are not showing much of the data, only CMOS battery and 5V,3V standby readings are populated. The others shows as Not Available.
|
Posted By: wantastic
Date Posted: 18 May 2018 at 7:21am
|
I can't say I spotted any temp anomalies, although I have heard of others having that issue. Normally, I run mine in an open case with a fan nearby circulating some air, but not directly onto the heatsink, and it's never felt or appeared to get too warm to the touch. When my board did fail, ASRock did ask about my power sensor readings though, and on both occasions I had the same issue with ATX+5VSB, +3VSB, and BAT showing a reading - everything else was Not Available as you say. Also, if you check carefully, you might spot that some of the values never change, which I consider to be quite unusual. Just to help clarify my events if they relate to yours as well, I'd find my system would crash and not POST. If I removed all power from the system by turning off the PSU, left it for a while, and turn back on, the board would power up and fans etc.. would spin, but would never POST. I could leave it like that for 24 hours, with no change, but it'd still be powered up. I'd see more sensor readings at this stage: * ATX+5VSB * +3VSB * Vcore * VCCM * +1.1 * +1.0 * BAT * ...and my fan RPMs. At this point, if I powered the system off from the power button on the front of the system, and power it back on at any point after this, the system would begin to start up again, fans spinning etc.. for about 2 seconds before it auto shut off. At that point, my sensors just read: * ATX+5VSB * 3VSB * BAT Note as well, that *all* of my voltages always reported a value that never changed apart from ATX+5VSB which did fluctate slightly, which I'd expect. But my +3VSB for example, always read 3.48 volts, even after swapping PSUs, and the BAT signal always reported 3.159 volts even with the battery removed! The point at which ASRock decided to issue me an RMA was when they noticed that the +3V, +5V, and +12V signals were always showing Not Available, which apparently did "not look good". Hope that helps somewhat..! |
Posted By: AlexKey
Date Posted: 27 Jul 2020 at 8:00am
|
Very disappointed, after 6 month of support mails regarding my dead C2750D4I unit I got response that motherboard are now out of warranty, asking 50 usd for diagnostics and dead silence afterwards. Anyone can please make detailed photo of restored from avr bug motherboard or tell what was made to make it working? If Cisco and synology can be fixed by simple resistor, may be this one can too. I found nothing on ix's forums, main source of bricked asrock rack boards. I want try to restore it before throwing out and never buy anything from asrock afterwards. |
Posted By: MsNIKITA
Date Posted: 16 Dec 2023 at 11:39am
I know this is really late, I doubt any of yous still have your board knocking around but I had issues with one of my c2750d4i and I came across this thread purely by accident.. I tried it and it's magically revived my board for the time being. https://www.truenas.com/community/threads/how-to-fix-asrock-c2750d4i-with-c2000-bug.90451/" rel="nofollow - How to fix Asrock c2750d4i with C2000 bug Good Luck |