X399 Taichi freeze with legacy virtual machine

Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic

   This is going to be an issue I doubt many will ever encounter, but as the result is quite catastrophic, I wanted to share. Yes, I'm aware Threadripper is a workstation, not a server CPU - but the honest fact is that AMD and Intel workstation systems are widely used in test/development server workloads, even if not officially supported.
Scenario:
- Multiple systems with X399 Taichi, Threadripper 1920X, 64 GB memory, single Samsung 960 EVO SSD, Intel X520 network card and various basic console grade display controllers
- BIOS 1.80, all defaults except IOMMU disabled
- Windows Server 2016 with all latest updates, running a failover cluster with Hyper-V workload, storage via iSCSI

This setup has worked perfectly so far and Threadripper is by far my favorite current hardware option for these environments. However, the problem occurs when running legacy workloads.

Add to this setup a single virtual machine running Debian 3.1 (from 2005). It's been running on a very recent Xeon system now for months without problems. When moved to Threadripper, it starts up just fine, but after two-three hours, it freezes the system. Being in a failover cluster, the workload moves to the next node, then that freezes as well, until every node has frozen completely and we have a major cluster...issue. It's not a reboot or a blue screen, simply a total system freeze that only recovers with the reset button. Nothing meaningful is logged into the event log.

Now, obviously, a 2005 OS and kernel 2.6.8 have no idea about Threadripper. It would make sense that it would not even boot. However, I find it dangerous that it completely freezes the system. Running systems this old is a bad idea, obviously, but there are cases when legacy workloads need to be simulated. Being able to crash the hardware with old software is, simply put, a bad thing.

For obvious reasons, I prefer not to test this setup too many times. It may be just a test/development environment, but I still dislike total cluster crashes. I've relocated the old workload to Intel hardware for now. Most of all, I wanted to raise this to the attention of anyone curious as this MAY be sign of a more serious flaw.

Author	Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic
Kyuu Members Profile Send Private Message Find Members Posts Add to Buddy List Newbie Joined: 28 Sep 2017 Location: Tampere, FI Status: Offline Points: 9	Post Options Post Reply Quote Kyuu Report Post Thanks(0) Quote Reply Topic: X399 Taichi freeze with legacy virtual machine Posted: 20 Nov 2017 at 11:21pm
	This is going to be an issue I doubt many will ever encounter, but as the result is quite catastrophic, I wanted to share. Yes, I'm aware Threadripper is a workstation, not a server CPU - but the honest fact is that AMD and Intel workstation systems are widely used in test/development server workloads, even if not officially supported. Scenario: - Multiple systems with X399 Taichi, Threadripper 1920X, 64 GB memory, single Samsung 960 EVO SSD, Intel X520 network card and various basic console grade display controllers - BIOS 1.80, all defaults except IOMMU disabled - Windows Server 2016 with all latest updates, running a failover cluster with Hyper-V workload, storage via iSCSI This setup has worked perfectly so far and Threadripper is by far my favorite current hardware option for these environments. However, the problem occurs when running legacy workloads. Add to this setup a single virtual machine running Debian 3.1 (from 2005). It's been running on a very recent Xeon system now for months without problems. When moved to Threadripper, it starts up just fine, but after two-three hours, it freezes the system. Being in a failover cluster, the workload moves to the next node, then that freezes as well, until every node has frozen completely and we have a major cluster...issue. It's not a reboot or a blue screen, simply a total system freeze that only recovers with the reset button. Nothing meaningful is logged into the event log. Now, obviously, a 2005 OS and kernel 2.6.8 have no idea about Threadripper. It would make sense that it would not even boot. However, I find it dangerous that it completely freezes the system. Running systems this old is a bad idea, obviously, but there are cases when legacy workloads need to be simulated. Being able to crash the hardware with old software is, simply put, a bad thing. For obvious reasons, I prefer not to test this setup too many times. It may be just a test/development environment, but I still dislike total cluster crashes. I've relocated the old workload to Intel hardware for now. Most of all, I wanted to raise this to the attention of anyone curious as this MAY be sign of a more serious flaw.
	Evaluating X399 + Threadripper systems for light server use.

MisterJ Members Profile Send Private Message Find Members Posts Add to Buddy List Senior Member Joined: 19 Apr 2017 Status: Offline Points: 1097	Post Options Post Reply Quote MisterJ Report Post Thanks(0) Quote Reply Posted: 21 Nov 2017 at 3:19am
	Kyuu, I suspect you are aware that this is a user forum and almost no employees for ASRock come here. We do have occasional Linux and Server posters but at least I know little about either. I do remember that there have been special Linux Kernels released for TR and certain versions of Linux. I also know that AMD has made some processor updates for Linux. I certainly see you have an unacceptable situation. I recommend you open a ticket with AMD and be sure to give details for your 1920X like Stepping and Revision. They may be able to offer help, but at least advice. You may find some useful information searching for Threadkiller and Linux. Good luck and enjoy, John.
	Fat1 X399 Pro Gaming, TR 1950X, RAID0 3xSamsung SSD 960 EVO, G.SKILL FlareX F4-3200C14Q-32GFX, Win 10 x64 Pro, Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.00, 2xHDDs WD

Kyuu Members Profile Send Private Message Find Members Posts Add to Buddy List Newbie Joined: 28 Sep 2017 Location: Tampere, FI Status: Offline Points: 9	Post Options Post Reply Quote Kyuu Report Post Thanks(0) Quote Reply Posted: 21 Nov 2017 at 2:22pm
	Thank you MisterJ, I will try to document this more technically so AMD would listen to it. I thought to start here as there has been good discussion here. Full Threadripper support for Linux is a new thing but can be readily found for any current distribution. I'm aware of these options, however they do not apply to legacy workloads. I will admit that piecing this information together is somewhat problematic and AMD hasn't done as good a job with documentation as they have with hardware. I would, however appreciate someone directing me to the right way to contact AMD about this.
	Evaluating X399 + Threadripper systems for light server use.