This form logs you into your management portal account. To access your help desk account, click here and use the form to the right of the news.
This form logs you into your management portal account. To access your help desk account, click here and use the form to the right of the news.
|
Jun 12 |
Switch as313-2.chi18 OS Upgrade
Posted by Marc Schulz on 12 June 2018 01:28 PM |
|
We have scheduled a maintenance to upgrade the OS on this switch to apply bugfixes. Date: Tuesday, June 19th, 2018 Maintenance Scope: as313-2.chi18 Switch Only Customer Impact: Customers on this switch will experience a short 10-15 minute outage while the switch reboots and applies the new OS version. If you have any questions regarding this maintenance please feel free to contact us via our helpdesk or by email at support@steadfast.net Update: This maintenance is complete. Read more » | |
|
Apr 25 |
Network Maintenance - 350 E. Cermak Road (Chi01/02)
Posted by Marc Schulz on 25 April 2018 03:43 PM |
|
We have scheduled an emergency maintenance to restore a redundant link on an aggregation device that was previously causing performance issues. The optics for this link have been replaced and link stability tested to ensure stability. Date: Friday, April 27th, 2018 Maintenance Scope: chi01/02 Network Resources Customer Impact: Customers may experience a short 30 second loss of connectivity while spanning-tree reconverges. Once reconvergence is complete the link will be monitored for stability. Any additional instability will lead to us taking this link back out of service. If you have any questions regarding this maintenance please feel free to contact us via our helpdesk or by email. Update 1:20am CDT: Maintenance Complete. No Issues were seen with the link post reintroduction. Read more » | |
|
Mar 14 |
R1Soft Server Backup Upgrade
Posted by Kevin Stange on 14 March 2018 03:11 PM |
|
We have scheduled an upgrade of our R1Soft Server Backup (formerly Idera Server Backup or R1Soft CDP) platform software on cdp01.steadfast.net to version 6.6.1. This upgrade fixes a number of bugs. This upgrade is being performed during the daytime, mid-week, because it is the lowest utilization period. Date: Tuesday, March 20th, 2018 Customer Impact: During the maintenance period, backups will not be performed and restoration services will be unavailable. This maintenance will not impact customer equipment or services other than backup tasks and does not affect private backup servers or customers using Veeam. It is safe to upgrade the agent to version 6.6.1 before the manager has been upgraded. The full official release notes may be viewed here. If you have any questions regarding this maintenance, or for assistance in upgrading the agent on your server(s), please feel free to contact us via our helpdesk or by email. Read more » | |
|
Feb 28 |
Hybrid Performance SAN Maintenance (Chicago, IL) March 11th, 2018
Posted by Marc Schulz on 28 February 2018 03:25 PM |
|
We have scheduled a software upgrade of our SAN infrastructure in Chicago, IL. This is purely a backend upgrade and does not change interactions with client environments. Date: Sunday, March 11th, 2018 Maintenance Scope: Hybrid Performance SAN (350 E. Cermak Chicago) Customer Impact: There is no expected impact during this maintenance as our SAN infrastructure is fully redundant. While we don’t expect any issues we recommend taking a backup prior to the start of this window. This maintenance does not impact our NetApp SAN (Public Cloud Infrastructure) If you have any questions or concerns please feel free to reach out to our support team at support@steadfast.net. Update: 2:15am CST (3:15am CDT) This maintenance has completed successfully. Read more » | |
|
Jan 3 |
Security Advisory: Meltdown and Spectre CPU Defects
Posted by Kevin Stange on 03 January 2018 06:04 PM |
|
Note: On May 21st, 2018 an additional vulnerability identified as variant 4, called Speculative Store Bypass, was disclosed as CVE-2018-3639. Mitigations for variant 4 are not discussed in this advisory. We are aware of a new series of issues affecting all modern CPU models which can allow for applications to potentially access information and execute code that should not be allowed. Some reports of these problems began to surface in the media starting around January 1st, but the reports were incomplete. This class of vulnerabilities is reported to affect all operating systems and CPU types, including servers, home computers, and likely mobile and embedded devices. This situation is serious and affects everyone, so we are making it a top priority to gather and communicate information as soon as we have it, and we are committed to ensuring mitigations are in place as soon as possible. The vulnerabilities have been labeled CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754 in the Common Vulnerabilities and Exposures database. They have also been given the nicknames Meltdown and Spectre. Due to mistakes in the responsible disclosure process, some information about the issues was unintentionally disclosed early. Consequently the reporters decided to move up the formal disclosure date to January 3rd from the following week. As a result of the shortened preparation window by major vendors, most vendors did not have mitigation patches available by the disclosure date. PatchingThe following information describes patching processes for various operating systems. Please find the section below matching your operating system to learn how to apply the relevant fixes for it. Note that in addition to the fixes provided in your operating system, hardware changes may also be needed. At this time, very little information is available as to which hardware will be fixed. See the hardware fixes section for information about progress. Important! Due to the nature of the issue, some of the patches may negatively impact the performance of your server. This happens because the vulnerable behavior is part of a performance optimization feature. Removing the optimization helps mitigate the problem but also eliminates the performance benefit of the optimization. If you experience significant performance issues with your applications after applying updates, please check with your application vendor for advice or contact our support team so we can work with you to look for ways to counter the performance loss. Windows ServerMicrosoft has produced updates for the following supported Windows server versions effective January 3rd:
Windows Server 2008 and 2012 (non-R2 variants) do not currently have available updates. Windows Server 2003 is End of Life and will not be updated. To protect computer systems running Windows desktop or server operating systems, open the Windows Update application for your version of Windows. Select the option "Check for Updates" and make sure that an update matching the appropriate KB number above is included in the list of available updates. Make sure the matching updates are selected, then choose to install the updates. After installing updates you must reboot your computer. Due to the performance impact of Windows updates, Microsoft has not fully mitigated the vulnerabilities by default. Additional actions may be required depending upon the types of services running on your server. Please see the guidance article from Microsoft Support for further steps to protect your server. In order to fully patch Spectre variant 2, you also need to apply microcode updates. These are available for some systems inside of Windows Server 2016, but the hotfix is not available via Windows update. Please see this article for information on how to obtain these hotfixes, please see this article. CentOSPackage updates for CentOS 7 and CentOS 6 are available as of January 4th. CentOS 5 and older are End of Life and no fixes will be provided for these versions. You should keep applying newer kernel updates as they become available to ensure you have the latest versions of Meltdown and Spectre fixes. Currently, two packages need to be updated. You can check the version numbers in this list to verify they are up to date. The highlighted numbers are the ones that will have changed from previous package versions. Effective in May 2018, Red Hat published updated microcode_ctl packages which include Intel processor microcode updates which should be applied to enhance mitigation of Spectre variant 2.
To check the current versions of the packages run the command: rpm -q kernel kernel-plus microcode_ctl To apply all available updates to your server (recommended), run the following commands: yum clean metadata If you would rather limit the update to the specific packages relevant to this issue, run these commands instead: yum clean metadata If you are running CentOS 7 in a VM, make sure an update for the "kernel-plus" package is available and matches the safe version number indicated above. if it does not, please run the following command first, then repeat the two commands above: yum -y install http://mirror.steadfast.net/centos-steadfast/7/x86_64/steadfast-release-1-2.noarch.rpm Make sure that the version numbers of the packages to be installed are greater than or equal to the numbers listed above. After the update process completes, you must reboot your server by running the If you are running KVM-based virtual machines on your server, you should also update the libvirt and QEMU packages. The updated versions are as follows:
To update these packages, run the following commands, or simply run a full system update as noted above: yum clean metadata It isn't necessary to reboot the server after applying these particular updates, but you must restart your virtual machines for the new protections to be effective. Don't forget to apply the full set of updates inside your virtual machines as well. If you need any assistance with any part of this process, please contact support and we'll be happy to assist you. CloudLinuxCloudLinux 7 and 6 patches are now available for production use. CloudLinux servers are based upon CentOS, so to apply the updates, follow the same process as with CentOS indicated above, but note that the following kernel versions should be included rather than the CentOS versions:
After applying updates, you must reboot for the changes to become effective. DebianDebian has released an update which only addresses CVE-2017-5754 (Meltdown). Debian releases prior to 7 (Wheezy) are End of Life and will not be fixed. You can check the version numbers in this list to verify they are up to date. The highlighted numbers are the ones that will have changed from previous package versions.
To check the current versions of the packages run the command: dpkg -s linux-image-`uname -r` | grep Version To apply all available updates to your server (recommended), run the following commands: apt-get update If you would rather limit the update to the specific packages relevant to this issue, run these commands instead: apt-get update Make sure an updated package for linux-image is included among the updates. After the update process completes, you must reboot your server by running the If the portion of the version number of your linux-image package before the - symbol is substantially different than the version above, you may first need to make sure your are using the latest kernel series for your version of Debian. For this case, first run these commands, filling in the number from the list below in place of the word "version", then reboot your server:
apt-get update
If you need any assistance with any part of this process, please contact support and we'll be happy to assist you. KernelCareKernelCare is currently providing patches for CentOS 6 and 7 kernels that fix part of Spectre (CVE-2017-5753) and fix Meltdown (CVE-2017-5754). If you have automatic updates enabled (the default) you should receive the fixes automatically. If you observe any stability or behavioral issues with severs that have been patched, you may want to apply security fixes using the normal process for your operating system noted above instead. You must reboot after applying package updates. VMwareUpdated VMware ESXi releases are available for currently supported products as of January 3rd:
The recommended procedure for updating VMware is using the vSphere Update Manager. If you need assistance with applying software updates for VMware products licensed through Steadfast, please contact our support team. Don't forget to apply the relevant security updates to your virtual machine operating systems as well. Steadfast Public CloudThe latest information regarding our Public Cloud platform is now located in a separate article. Please check here for further information. Hardware Patches (BIOS and Microcode Updates)Intel provided microcode updates for all CPUs that Steadfast currently supports. This does not include Pentium 4, Pentium D, Celeron, Core 2 Quad, Core 2 Duo, or any Xeon lines prior to the E55xx series, which are considered EOL by Steadfast. If you have a server with a CPU that is not supported, please contact our sales team to arrange an upgrade. Steadfast uses exclusively Supermicro motherboards for servers. Supermicro has disclosed a list of products for which provide CPU code fixes. See the Supermicro advisory for status information. Currently motherboards with code updates available include the X11, X10, and some X8 series. X9 motherboard BIOS updates have not yet been made available. Steadfast provides newly ordered servers running the latest available BIOS code, which will include the relevant microcode patches if they are available. For customers running CentOS or Windows Server 2016, it is not necessary to apply the BIOS update, provided you take OS-level action instead. OS-provided microcode updates include the latest set of Intel code for CPUs that have already been fixed, and this code will be applied each time you reboot your server after the package has been installed. A more permanent BIOS fix is optional in this situation. See the above sections for CentOS and Windows for information on how to apply the current microcode updates to your servers. Microcode updates are not applied by other supported operating systems. Due to the logistics and risks of updating BIOS code on all servers operating in the data centers and because most of our customers run versions of Windows or Linux with OS-level microcode patching support, we have decided not to proactively update BIOS versions on any hardware that is already provisioned. If you are running a system which cannot apply OS-level microcode updates or would like extra peace of mind from having a BIOS update that contains the microcode, please reach out to support to request a version check and update. Please note that BIOS updates require downtime and may be risky, so we advise avoiding the update unless it is necessary. ReferencesThe following references may be useful reading:
If you have any questions, please contact our support team and we'll do our best to help you. Document RevisionsWe will update this article with further information on how to obtain and apply patches to affected systems as soon as it is available. The following changes have been made since the original posting. May 25, 2018 @ 5:35 PM CDT:
Mar 13, 2018 @ 4:50 PM CDT:
Feb 7, 2018 @ 6:30 PM CST:
Jan 26, 2018 @ 3:30 PM CST:
Jan 18, 2018 @ 3:30 PM CST:
Jan 16, 2018 @ 4:45 PM CST:
Jan 10, 2018 @ 2:15 PM CST:
Jan 5, 2018 @ 2:45 PM CST:
Jan 5, 2018 @ 2:15 PM CST:
Jan 4, 2018 @ 6:30 PM CST:
Jan 4, 2018 @ 5:30 PM CST:
Jan 4, 2018 @ 5:00 PM CST:
Jan 4, 2018 @ 11:30 AM CST:
Read more » | |