NVMe Windows Boot Issue (H470M Pro4)

Hello,

I have been building a new PC with the ASRock H470M Pro4 motherboard and intel i3 10100 CPU. I am having issues with booting my machine. I am using a Samsung 970 EVO M.2 NVMe as the boot drive installed in the Ultra M.2 slot on the motherboard.

I am loading windows using an installation media USB drive, everything goes smoothly and windows loads up.

I then attach a SATA HDD and the ASRock logo gets stuck on the screen (the one that comes after the initial one where we can enter the BIOS by hitting F2). I have tried different SATA HDDs on different ports, with the same result.

Here is the most frustrating thing though, the screen still gets stuck even after removing all the SATA HDDs. The only option I am left with at that point is to reinstall windows, which works. However, the same stuff happens again when I connect a SATA HDD.

Also when the screen gets stuck the VGA and BOOT leds on the motherboard light up. I don't know what that corresponds to on the POST Status Checker (PSC).

Here's the stuff I have tried and my configuration:
1) Disabled CSM in the BIOS
2) SATA Mode Selection = AHCI
3) Disconnected the RAM sticks, checked them one at a time.
4) Running this setup off of the HDMI port of the motherboard (I don't have a graphics card as this is a NAS build).
5) Installed the NVMe drive on the PCIe slot using an adapter (running into the same issue).
6) Flashed BIOS to the latest version.
7) Running all this using a 650W Seasonic PSU.

Any feedback or suggestions would be greatly appreciated. Is this a boot issue or a faulty motherboard?
Thanks in advance,
Alex