Penguin Computing™ Statement on Speculative Execution Side-Channel Attacks

BIOS Updates to Penguin Computing-based Infrastructure – 2/14/2018

Penguin Computing has been working with Intel and other technology partners to help our customers address the issues related to Penguin-based systems.

We support the efforts by the Linux kernel developers and compiler writers to come up with work-arounds for this issue in the Linux kernel and compilers.

Where a complete fix requires CPU microcode updates, we have always recommended that these NOT be delivered through a BIOS update but rather through the “microcode_ctl” program that is part of most Linux distributions. Similar capabilities exist in other operating systems. Firmware updates delivered this way are easily rolled back if issues are discovered. And microcode updates delivered this way can be more easily automated at dramatically lower risk of failure. As we saw with the issues with the initial microcode, those that had deployed BIOS updates could not easily roll back that change.

We have been informed that Intel has “root caused” the issues with some microcode and is in the process of more thorough testing of the revised code. When testing is completed, Penguin will be able to provide instructions on how to update microcode_ctl without waiting for Red Hat or other Linux distributions to repackage the update.

We do not expect to recommend that customers update their BIOS until a longer period of testing has been completed and even then, we don’t recommend updating the BIOS if the only reason is microcode. BIOS updates come with many more changes that must all be evaluated for their impact on system function.

Penguin Computing On Demand (POD) – 1/17/2018

POD’s compute environment is a single-tenant bare-metal service, protecting customers from being affected by Spectre and Meltdown attacks while running in our compute environment. Due to the protected environment of our computing environment, Penguin Computing is prioritizing patches to our Login Node and Scyld Cloud Workstation instances. To ensure cloud security and system stability after updates are installed, there is an active effort underway to test the relevant security patches as they are provided by our vendors and partners.

Updating the production environment will start in January 2018, though final resolution of Spectre and Meltdown vulnerability patching is dependent on future releases of updated software from upstream vendors.

Penguin Computing will patch POD as quickly as possible and with minimal impact to our customers. If your environment on POD requires downtime to upgrade, our team will reach out to coordinate a downtime window with you.

This announcement will be updated as our patching process progresses.

Scyld ClusterWare – 1/11/2018

On January 11th 2018, Penguin Computing released Scyld ClusterWare 6.9.8 and 7.4.3, which include RedHat’s kernel patches outlined in the RHSA-2018:0008 advisory.

• Scyld ClusterWare 6.9.8 for RHEL/CentOS v6Kernel: 2.6.32-696.18.7.el6.698g0000
• Scyld ClusterWare 7.4.3 for RHEL/CentOS v7Kernel: 3.10.0-693.11.6.el7.743g0000

Links with Details:

• https://access.redhat.com/security/vulnerabilities/speculativeexecution
• https://access.redhat.com/errata/RHSA-2018:0008

Before installing or updating Scyld ClusterWare, if your cluster uses any 3rd-party drivers (e.g., Ethernet, InfiniBand, GPU, Omni-Path, parallel storage such as Panasas), verify that those 3rd-party drivers can be rebuilt or relinked to the new kernel. If an install or update involves upgrading to a new RHEL/CentOS base distribution, then verify that your cluster’s 3rd-party applications are all supported by that new base distribution.

If your cluster uses Panasas storage, then you must ensure that a Panasas kernel module is available that matches the Scyld ClusterWare kernel you are about to install (see: http://www.my.panasas.com).

Penguin strongly encourages customers to use the install-scyld -u (or update-scyld) command to continuously update both Scyld ClusterWare and the base OS to apply all security patches the base operating system distribution has made available.

Important details in Scyld ClusterWare’s Release Notes should be read before conducting any upgrade: https://www.penguinsolutions.com/support/documentation/

Scyld Cloud Manager – 1/17/2018

Customers running Penguin Computing’s Scyld Cloud Manager should coordinate with your Managed Services representative to schedule upgrades.

Scyld Cloud Workstation – 1/17/2018

Scyld Cloud Workstation is not directly impacted by the Spectre and Meltdown vulnerabilities, but the underlying OS might be. Please check with your operating system distribution and system vendors for any relevant security patches.