This article provides current support information for Spectre and Meltdown on the Steadfast Public Cloud.

Some brief contextual information:

• Spectre is also known as CVE-2017-5715 and CVE-2017-5753, and referred to in the Xen advisory by the names SP1 and SP2
• Meltdown is also known as CVE-2017-5754 and referred to in the Xen advisory as SP3
• The information contained in this article makes assumptions specific to Steadfast's Cloud environment and they are not necessarily applicable to other clouds

Current System Status

The current public cloud platform is vulnerable to both variants of Spectre, but has been patched against Meltdown attacks.  The platform is based on CentOS 7 and Xen 4.8.  Prior to February 11th, the platform was based on CentOS 6 and Xen 4.4.  Xen 4.4 will not receive updates to fix either Meltdown or Spectre. On January 17th, the first patches that mostly mitigate Meltdown were made available for Xen 4.8, but no patches are yet available to mitigate Spectre.  Our migration to Xen 4.8 was the culmination of several months of testing and planning, which we started prior to disclosure of the security issues. We did not modify our plans as a result, other than to increase the urgency of our roll-out.

On February 23rd, the first available Xen patches which mitigate the SP2 variant of Spectre became available, however the solution required a compiler update that was not yet available on CentOS. It is expected that these patches will be applied to a release version of Xen for CentOS soon and we will begin testing it as soon as it is available.

Customers with dedicated hypervisors have not yet all been upgraded to patched Xen 4.8.  We prioritized public cloud hypervisors due to the higher risk of information exposure in the mixed-user environment.  We are currently working on a plan to bring dedicated hypervisors in line with the public cloud with individual customers.

Current Risks to VMs

Due to the way Xen is designed, Linux VMs do not need Meltdown kernel patches inside the VM.  They already cannot exploit the Meltdown issue using the normal Linux method.  However, Xen must be fixed to prevent users on Linux VMs from exploiting Xen directly to read the memory of other VMs on the same physical server.

Windows VMs cannot exploit the Xen variant of the Meltdown vulnerability, but Meltdown can be exploited inside a Windows VM to read its own memory from an unprivileged account unless the correct Windows Update package has been installed in the VM.  Linux VMs may be able to exploit the Xen issue to read the memory of unsuspecting Windows and Linux VMs even if they are patched.

Both types of VMs are vulnerable to Spectre, however the amount of exposure caused by Spectre is more limited and thus less of a danger overall than Meltdown. Some of the system updates available may mitigate part of Spectre, but the availability of patches varies wildly by operating system. Xen patches along with CPU microcode are required to provide complete mitigation, but these are not yet available.

Customers with dedicated hypervisors are protected from other customers, but the risks within VMs still apply as described above.  If you trust all the VMs on a single physical server, along with all the users and applications hosted on them, the risks associated with Spectre and Meltdown may be relatively low and you may not need to urgently upgrade.  If you have concerns about Meltdown on your private hypervisors, please contact our support team and we will plan out an upgrade to the patched Xen 4.8 version right away.  Otherwise, we will be contacting customers with dedicated hypervisors soon to work on upgrade planning soon.

Current Mitigation Options

The most complete mitigation available is running on patched hypervisors.  All public cloud VMs are currently on these hypervisors since February 11th.

If you are running Linux, you should also apply kernel updates that provide fixes to Spectre.  Updates fixing Meltdown are not required. Please note that working kernels are now available for all CentOS VMs:

• It is now safe to upgrade the kernel for CentOS 7 VMs to kernel-plus version 3.10.0-693.21.1.el7.centos.plus or newer.
• It is now safe to upgrade the kernel for CentOS 6 VMs to kernel version 2.6.32-696.20.1.el6 or newer.

If you are running Windows, you should also apply Windows updates:

• Windows Server 2016: Build 14393.2007 (KB4056890)
• Windows Server 2012 R2: KB4056898
• Windows Server 2008 R2: KB4056897 or KB4056894

If you trust your VM users, you may also want to consider a dedicated hypervisor, which guarantees all the VMs on the physical server are under your control.  This option does not itself prevent Meltdown or Spectre from being exploited, but limits the number of people who may have access to the server memory.  Keep in mind that if you operate applications that face untrusted users, an exploitable vulnerability in such an application could allow a user to run their own code and subsequently exploit Meltdown or Spectre.  If you'd like to explore the option of switching to private hypervisors, please contact our sales team for further information.

Meltdown Patch Information

The Meltdown patches that are currently available for Xen are known as "XPTI stage 1."  These patches do the following:

• Mostly resolve the Meltdown vulnerability in Xen, preventing most memory from being exposed between VMs
• Reduce overall performance of Linux VMs similar to the impact of the Meltdown patches for Linux

These patches do not do the following:

• Fix any issues related to Spectre inside VMs or in Xen
• Fix the CentOS issue with patched kernels being unable to boot
• Fix the Meltdown issue inside Windows VMs without additional updates
• Fix the Meltdown issue completely in Xen -- there are some possible exposure issues that remain, which are expected to be fixed in future "stages"

References

• Steadfast Meltdown and Spectre Advisory
• Xen Security Advisory XSA-254

If you need any help or have any questions regarding any of this information, please contact our support team.

Updates

March 13, 2018 @ 3:00 PM

• Updated to include CentOS 7 safe kernel-plus package from the official distribution instead of Steadfast-released version.
• Added a note about the newly available Xen SP2 Spectre patch.

February 12, 2018 @ 10:45 AM

• Updated to indicate that public cloud VMs are now running on Meltdown-patched hypervisors.

February 7, 2018 @ 6:35 PM

• Updated to indicate that both CentOS 7 and CentOS 6 users can upgrade to working kernels

January 26, 2018 @ 3:35 PM

• Updated to indicate that only CentOS 7 users need to avoid kernel updates now that CentOS 6 kernel 2.6.32-696.20.1.el6 is available

January 22, 2018 @ 1:45 PM

• Updated references to testing of Meltdown patch to indicate testing is complete.

January 18, 2018 @ 6:30 PM

• Rewrote the "Current Risks to VMs" section to be clearer and less redundant
• Clarified that applications that face untrusted users should be considered during risk analysis
• Added some clarifications around dedicated hypervisors
• Added a clarification about the optional status of the patched hypervisors and when the migration will become mandatory'