This document outlines how to utilize DEBs to grant the Steadfast Support Staff access to your server.

This package will work for Debian 9, Ubuntu 18.04, and later versions. Other comparable dpkg-based distributions may work but have not been tested. 

Adding the Steadfast Debian repository

You can install a DEB file on your system to automatically add the repository data to the apt configuration.

Install the steadfast-release DEB. 

wget https://mirror.steadfast.net/debian-steadfast/steadfast-release.deb
dpkg -i steadfast-release.deb

Installing the steadfast-keys DEB

After installing the steadfast-release DEB as above, you can use apt to install the steadfast-keys DEB.

Note: If you already have a /root/.ssh/authorized_keys2 file in place, this action will overwrite it. The existing file will be saved to /root/.ssh/authorized_keys2.orig.

# apt install steadfast-keys
 <snip>

 Complete!

Note that on the initial install you are prompted to import the GPG key. Verify that the key ID matches the information below:
Key ID: 47c9d9af
and answer "yes". This warning will only occur once.

At this point, the Steadfast Staff Public Keys have been installed to your system at /root/.ssh/authorized_keys2. To update the keys with the most recent list, run "apt update" on your system.

Removing access

If you've been supplied a system and you would like to revoke access for the Steadfast Support Staff to your system, simply remove the steadfast-keys RPM:

# apt remove steadfast-keys
 <snip>

 Complete!

You can verify that the deinstall took place by checking if the /root/.ssh/authorized_keys2 file still exists.

This document outlines how to utilize RPMs to grant access to the Steadfast Staff access to your server.

Adding the Steadfast YUM repository

You can install an RPM to your system to automatically add the repository data to the yum configuration.

Install the steadfast-release RPM.  The repository package file is identical for all versions of CentOS.

# rpm -ivh http://mirror.steadfast.net/centos-steadfast/steadfast-release.rpm
 Retrieving http://mirror.steadfast.net/centos-steadfast/steadfast-release.rpm
 Preparing... ########################################### [100%]
 1:steadfast-release ########################################### [100%]
 

Installing the steadfast-keys RPM

After installing the steadfast-release RPM as above, you can use yum to install the steadfast-keys RPM.

Note: If you already have a /root/.ssh/authorized_keys2 file in place, this action will overwrite it. The existing file will be saved to /root/.ssh/authorized_keys2.rpmorig.

# yum install steadfast-keys
 <snip>
 Transaction Summary
 ================================================================================
 Install 1 Package(s)
 
 Total download size: 15 k
Installed size: 10 k
Is this ok [y/d/N]: y
Downloading Packages:
warning: /var/cache/yum/x86_64/7/steadfast/packages/steadfast-keys-20210712-1603.noarch.rpm: Header V4 RSA/SHA256 Signature, key ID 47c9d9af: NOKEY
Public key for steadfast-keys-20210712-1603.noarch.rpm is not installed
steadfast-keys-20210712-1603.noarch.rpm                                                          |  15 kB  00:00:00
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-steadfast-2021
Importing GPG key 0x47C9D9AF:
 Userid     : "Steadfast Networks (Package Signing) <support@steadfast.net>"
 Fingerprint: c686 fee8 1733 0bf2 71cd d566 f94c 75d8 47c9 d9af
 Package    : steadfast-release-2-3.noarch (@steadfast)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-steadfast-2021
Is this ok [y/N]: y
 <snip>
 Running Transaction
 Installing : steadfast-keys 1/1 

 Installed:
 steadfast-keys.noarch 0:20210712-1603
 
 Complete!

Note that on the initial install you are prompted to import the GPG key. Verify that the key ID matches the information below:
Key ID: 47c9d9af
and answer "yes". This warning will only occur once.

At this point, the Steadfast Staff Public Keys have been installed to your system at /root/.ssh/authorized_keys2. To update the keys with the most recent list, run "yum update" on your system.

Removing access

If you've been supplied a system and you would like to revoke access for the Steadfast Support Staff to your system, simply remove the steadfast-keys RPM:

# yum remove steadfast-keys
 <snip>
 Removing:
 steadfast-keys noarch 0:20210712-1603 installed 15 k
 
 Transaction Summary
 ================================================================================
 Remove 1 Package(s)
 
Installed size: 15 k
 Is this ok [y/N]: y
 <snip>
 Running Transaction
 Erasing : steadfast-keys 1/1 
 
 Removed:
 steadfast-keys.noarch 0:20210712-1603
 
 Complete!

You can verify that the deinstall took place by checking if the /root/.ssh/authorized_keys2 file still exists.

There are no automated backups taken for Virtual Machines, so you will need to ensure your own data is backed up using the system provided within the Control Panel.

Creating a Backup

1. Go to your OnApp/VM Control Panel's Virtual Machines menu.

2. Click the label of the machine you want to back up.

3. Mouse over the "Storage" menu and click the "Disks" item. You'll see a list of the disks allocated to that virtual machine.

4. Under the “Actions” column, click on the gear icon  () for the disk. Then click on “Backup.”  You'll see a list of all the backups taken and pending for that virtual machine, along with tools to restore backups, delete them, and convert them to templates.

   • To make a backup, click the "Take Backup" button at the end of the list.

   • To restore a backup, click the "Restore" icon () next to the backup you want to revert to.

Scheduling a Backup

We no longer allow automated backups inside OnApp due to the inefficiency of the built-in automatic backup systems.  Please contact us for more information about our Backup solutions.

*****Warning: CentOS 6 is now EOL. As CentOS 6 will no longer receive security and other important updates, it is highly recommended that you upgrade to an actively supported operating system*****

Starting with CentOS 6.8, a newly introduced update to NSS causes certain applications to be unable to connect via TLS using GCM ciphers on virtual machines. This article describes the technical problem and how to apply the solution.

Symptoms and Detection

This issue affects virtual machines in very specific cases.  It can be reproduced with a very simple connection test:

# curl https://google.com --ciphers ecdhe_rsa_aes_128_gcm_sha_256
Illegal instruction (core dumped)

This will cause other applications to crash with similar error messages when they attempt to connect to a TLS server or serve a TLS client using any GCM cipher. You can verify that the issue is caused by misdetected hardware capabilities, by repeating the same command with NSS_DISABLE_HW_GCM=1 set:

# NSS_DISABLE_HW_GCM=1 curl https://google.com --ciphers ecdhe_rsa_aes_128_gcm_sha_256
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="https://www.google.com/">here</A>.
</BODY></HTML>

If you use another cipher, there's no problem:

# curl https://google.com --ciphers ecdhe_rsa_aes_128_cbc_sha_256
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="https://www.google.com/">here</A>.
</BODY></HTML>

Solution

Red Hat and CentOS fixed this issue in NSS Softokn version 3.14.3-23.3.el6_8.  To apply this fix on a system that is experiencing the bug, try the following command:

NSS_DISABLE_HW_GCM=1 yum -y update nss-softokn nss-softokn-freebl

References

This issue has been reported and discussed in a number of places.

• Original Chromium Report
• Mozilla Bug Report Containing Patch
• Red Hat Bug Report
• CentOS Bug Report
• Red Hat Errata Release

This article explains noteworthy differences and improvements between CentOS 6 and CentOS 7 to help in deciding which version to use.

What is CentOS 7?

CentOS 7 is the most recent major release of the CentOS Linux distribution.  It is based on Red Hat Enterprise Linux 7, which is derived from Fedora Linux version 19.  It was first released on July 7th, 2014 and will be supported for 10 years, up through June 30th, 2024.

What's New in CentOS 7?

CentOS 7 includes several major changes that pertain to booting up and managing the system.  It is the first to introduce "systemd" which controls how services are started up, as well as many system settings.  It also includes "firewalld" as a new method of managing the server's firewall.

CentOS 7 also provides updates most common hosting software.  The following table lists the differing versions of commonly-used components between CentOS 5, 6, and 7.

Compatibility

The system has been designed to be backward-compatible with old methods of managing hardware and services, so older software will usually continue to function correctly, even though it cannot take advantage of any new system features.

MySQL has been replaced with MariaDB.  The two database servers are compatible and function the same way.  If your application works with MySQL 5.5, it should have no trouble working with MariaDB 5.5 and shouldn't notice the difference.

In most cases, newer software versions are backward compatible with old applications.  If in doubt, you should check with your application vendor to ensure there are no compatibility issues with CentOS 7 or the newer software it provides.

Should I Choose CentOS 7?

We recommend choosing CentOS 7 for new deployments unless there's a specific reason to avoid it.  Using CentOS 7 has the following advantages:

• As the newest release of CentOS, it will be supported for the longest period of time. Security and major bug fixes will be provided until 2024.
• All software that Steadfast supports with CentOS 6 is known to work well with CentOS 7.
• It includes new features and optimizations that will never be included in previous CentOS releases.
• It boots faster and runs better on most newer hardware than previous CentOS releases.
• Applications will soon begin to discontinue support for older releases in order to take advantage of new technology.

CentOS 6 still remains the better choice in the short term in some specific cases:

• You have applications or scripts that you know are not compatible with systemd, or with one of the new software versions listed in the table above.
• You have an existing cluster of servers and would like to maintain the same software configurations across the entire environment.
• You are using very old legacy hardware which CentOS 7 doesn't support.  If this is the case, we encourage you to consider upgrading to a newer server.

In mid-2017, CentOS 6 will stop being revised for new types of hardware, after which it will become increasing likely that some Steadfast products will no longer be compatible.  If you build an environment based on version CentOS 6 now, you may have trouble expanding it in the future without having to mix different versions of CentOS.

Getting Advice

If you aren't sure which version of CentOS to select or you need any other help finding the right solution for your environment, contact us, and we'll be happy to help!

This article explains some considerations when choosing an operating system for your dedicated server or cloud VM. If you have any questions or need advice, feel free to contact our sales or support teams.

Narrowing Your List

The first thing to consider is what software you intend to run. For example, if you absolutely need to run Windows software, like game servers, .NET applications, desktop applications, you should choose a version of Windows. You should review the system requirements for your software carefully to find out what supported operating systems you can choose from. If you do not have any specific needs for software, we will always recommend using Linux because it offers the greatest flexibility in terms of supported software and ease of later migration.

Selecting a Linux Distribution

Steadfast Networks supports the following Linux distributions:

• CentOS
• Debian GNU/Linux

CentOS is our preferred Linux option because it has a very long life cycle and the most available software supporting it. More detail on CentOS can be found here. If you are already familiar with desktop Linux, CentOS is most similar to Fedora in how it is maintained and how files are organized. If you are familiar with Ubuntu, Debian GNU/Linux will be most similar to what you have used before.

If you have no preference, we recommend selecting the latest CentOS release available. If you specifically like the Debian-style system layout, Debian is a suitable alternative, though its support life cycle is shorter and less clearly defined. In many cases, upgrading from version to version is easier to do on Debian systems, so once a Debian release goes out of maintenance, you can use a well-documented in-place upgrade procedure to move to the new release. CentOS can be upgraded in this manner, but the procedure is not as well tested or supported. Please note that, in general, Steadfast recommends clean installations when upgrading Linux to a new major version to reduce the risk of complications. In-place upgrades are not guaranteed or officially supported by Steadfast technical support.

Why Debian instead of Ubuntu?

Steadfast prefers Debian over Ubuntu because of the additional software flexibility and stability it offers. Debian provides one of the largest software repositories of any distribution and operates on the widest variety of platforms, but also operates with one of the smallest footprints in terms of storage and memory. Debian maintains three release branches, marked stable, testing, and unstable. Stable is always the current release, which retains older but very well-tested software versions of packages with bug fixes and security patches as needed. This maintains a highly stable system which can be selectively upgraded using "backports" to access newer versions of commonly requested software if needed. The testing branch of software may also be used selectively at the risk of reducing stability of the system. Debian's support lifecycle is unfortunately somewhat ambiguous, though it has typically been 3 to 4 years. The years between releases are spent extensively testing updates and improvements nominated for the next release. Releases are not delivered until they meet specific criteria, rather than based on a routine timeline, which reduces the chances problems due to rushing. It also provides one of the most robust and reliable upgrade processes from one release to another, which mitigates a lot of the disruptions, even on production systems.

Ubuntu builds on top of Debian releases to produce frequently updated, desktop-tuned releases. We have found that Ubuntu suffers similar issues to Fedora. A new release is provided once every 6 months and only the last two are supported. After support ends, a distribution upgrade is required to continue receiving security and bug fix updates, which can be very disruptive to a production server environment. Debian supports releases for much longer and continues to hold back packages to older versions to reduce the risk of introducing instability and volatility to production environments.

Ubuntu answers this criticism with a secondary release cycle called "Long Term Support." LTS releases are made stable every fourth release of Ubuntu and receive security and bug fix updates for 5 years. This is shorter than the 10 years for Red Hat Enterprise distributions, but longer than a typical Debian release cycle. Unfortunately, our experiences with Ubuntu have indicated that package quality is often lower in Ubuntu than equivalent Debian releases and the installation process is not tuned well for server environments, which makes fast deployment and flexible installation options difficult to offer. Ubuntu has also proven more difficult to support and maintain for our customers, so we have opted to support only Debian.

Why CentOS instead of Red Hat Enterprise Linux or Fedora?

Due to the software license for Red Hat's distribution, they are required to release the source code to its components. CentOS rebuilds these packages, tests them to ensure compatibility with Red Hat's builds, and releases equivalent releases. This means that effectively CentOS is the same software release with a different name and different branding. CentOS does not include a support contract, however Steadfast Networks is intimately familiar with maintaining CentOS systems and certifies the releases on our own hardware. Since we take care of the support for you, the separate support contract isn't needed. For this reason, we don't offer Red Hat Enterprise Linux and offer CentOS as a completely compatible replacement for it. If you need RHEL for compliance of some kind, we can install it if you can provide the media and the license.

Fedora is a community-driven project intended primarily to support desktop users and help to test new software nominated for inclusion in future versions of Red Hat Enterprise Linux. Fedora releases are made once per six months, and only supported for approximately one year. After the year ends, any update to software, including for security reasons, requires you to upgrade your system to a new version of Fedora. These properties make Fedora a cutting-edge Linux distribution, but they also make the platform volatile and unstable. Upgrading the entire distribution, every six to twelve months can be very disruptive to a production server environment, so we have decided not to support Fedora on servers.

A lot of the software from Fedora considered most useful to server environments is available in a free repository called Extra Packages for Enterprise Linux (EPEL) which we can add to your server if needed.

Why not another Linux distribution?

Steadfast made determinations about which Linux distributions to support by identifying those which had the highest demand, widest support for hosting software such as control panels, best hardware support, and longest life cycle. CentOS meets these needs and satisfies most situations which require a distribution laid out like Red Hat Enterprise Linux using software packaged in RPM files. Debian meets these needs and satisfies most situations where a Debian-style system layout and software packaged in DEB format is needed.

Distributions that are very different from Debian or CentOS are rarely requested and often best suited to experienced users with specialized needs. We can attempt to install unsupported Linux distributions on servers at your request, but we can't guarantee that they'll work on our hardware or that we'll be able to fix problems with them that come up. Our modern dedicated server offerings feature remote console access (IPMI), which means you can try to install any operating system you like, if you don't mind dealing with any issues on your own. If in doubt, choosing CentOS or Debian helps ensure you server will be as reliable as possible and that we'll be able to help when something goes wrong.

Selecting a Windows Release

Steadfast supports the following Windows releases:

• Windows Server 2016
• Windows Server 2012 R2
• Windows Server 2008 R2

Steadfast currently recommends Windows Server 2012 R2.  Windows Server does not support 32-bit hardware.

Server 2008 R2 will remain supported until at least 2018 and Server 2012 will remain supported until at least 2023.

For information about Windows Server 2008 R2 editions, see this page. For information about Windows Server 2012 editions, see this page.

Why Windows Server instead of Windows 7 or 8?

Windows 7 and 8 are not considered a server-grade release and are not designed for server hardware or data center licensing. As a result, we discourage their use and do not offer licensing for them. Windows Server 2008 R2 is based on the same core as Windows 7, and Windows Server 2012 is based on the same core as Windows 8. We've found almost everything that can run on 64-bit desktop releases of Windows will run on the equivalent server release.

Certain CPU-intensive applications (trading applications in particular) will cause "clock drift" on Windows systems. Severe enough clock drift will cause Windows to re-sync with the system's hardware clock (also know as RTC, or Real Time Clock). This can cause Windows to change the clock to UTC or GMT in virtualized environments.

"Clock Drift" in this context is defined as the clock going out of sync. This is caused by Windows using SNTP (Simplified Network Time Protocol) rather than a full NTP service; as well as Windows having a too-infrequent clock update cycle by default. There are two ways to alleviate this issue.

Correcting clock drift by installing a third-party NTP service

The most reliable manner to correct this issue is to use a third-party implementation of the NTP service to update the system's clock. We have been succesful at using the Meinberg NTP daemon port for Windows, which includes an easy-to-use installer. You can download it at the following link:

http://www.meinberg.de/english/sw/ntp.htm

1. After downloading and running the installer, step through the default for each option, until you reach the "Configuration File Settings" screen. Here, choose the following:
   • Location of configuration file: Leave at default.
   • Create an initial configuration file with the following settings: Leave checked
   • Want to use predefined public NTP servers (see www.pool.ntp.org)? Choose Choose "United States of America"
   • You can specify up to 9 NTP servers (comma separated) you want to use: Leave blank.
   • Use fast initial sync mode (iburst) Leave checked.
   • Add local clock as a last resort reference Leave unchecked.
2. When prompted to review settings, click "No".
3. At the "Setting up NTP Service" screen, click "Next >"
4. At the "Enter the User ID and password used for running the service" screen, enter a secure password for an NTP account, and click "Next >"
5. Click Finish

This will replace the Windows W32Time service with the Meinberg NTP daemon. You can get up-to-date time statsitics by clicking Start > All Programs > Meinberg > Network Time Protocol > Quick NTP Status.

This will also automatically set the clock to update on a more frequent, and more accurate, basis.

Correcting clock drift by altering the W32Time service parameters in the Windows Registry

This will possibly help, but is not a recommended solution. Microsoft acknowledges that the Windows W32Time service is insufficient for any high-accuracy applications:

"We do not guarantee and we do not support the accuracy of the W32Time service between nodes on a network. The W32Time service is not a full-featured NTP solution that meets time-sensitive application needs. The W32Time service is primarily designed to do the following:

• Make the Kerberos version 5 authentication protocol work.
• Provide loose sync time for client computers.

The W32Time service cannot reliably maintain sync time to the range of 1 to 2 seconds. Such tolerances are outside the design specification of the W32Time service."

-- Source: Microsoft Support Article ID 939322: "Support boundary to configure the Windows Time service for high accuracy environments"

If you would like to make adjustments to the Windows Time Service regardless, follow the below steps:

1. Click "Start", then "Run...", and enter "gpedit.msc"
2. Navigate through the Local Group Policy Editor tree as follows: Local Computer Policy > Computer Configuration > Administrative Templates > System > Windows Time Service
3. Double-click on "Global Configuration Settings"
4. In the Global Configuration Settings window, click "Enabled" to enable the options pane.
5. Adjust the MinPollInterval and MaxPollInterval parameters to suit your needs. Note that this parameter is defined in log base-2; meaning that it will update according to the following formula: 2 ^ MinPollInterval. By default, this is 6, or 2 ^ 6 = 64 seconds. By default, Windows will update the clock somewhere between 64 and 1024 seconds.
6. Click Apply, then OK, and close the Local Group Policy Editor window.

If you have any questions, or experience further issues with clock drift on Windows systems, please contact support

Virtual machines are created from templates. We provide a number of supported configurations, similar to those available for dedicated servers, including CentOS, Debian, and Windows, along with various Control Panels. To create a virtual machine:

 On your Control Panel's dashboard, click the "Build" and then "Create New Virtual Machine" links in the top right corner of your control panel (https://vm.steadfast.net) or go to "Virtual Machines" in the menu on the left and click the "+" button on the top right of your virtual machine listing.

1. Select the template of what OS and pre-installed software you want:

   • Template

     • For Linux, we recommend CentOS 7 x64 or AlmaLinux 8 x64. For Windows, we recommend Windows 2016 Standard x64. If you want a control panel, please select a template with Interworx or cPanel.  We recommend choosing InterWorx.

     • Operating System: Click the box for the Operating System / Distribution group you wish to expand.

     • Template: Choose the base template or a template with specific pre-installed software from the list.  All templates are free except for supported Windows templates which are billed at $0.01 per hour.

     • If you have created custom templates, they will appear under the “Your templates” group.

2. Once you have highlighted the template you want, at the bottom of the web page click on Next.

3. Fill in the Virtual Machine properties form:

   • Label: The label is used to identify your system in the control panel.  it is suggested to use a friendly name so you can easily identify it.

   • Hostname: The hostname is used to identify your system on the Internet and via command line.

   • Password: Give your virtual machine a secure Password. If left blank, a password will be generated randomly.

4. Once you have filled in the Virtual Machine properties, click Next.

5. Select the VM Resources:

   • Resources

     • Set up your VM with at least 256 MB of RAM for a Linux Virtual Machine or 512 MB for a Windows Virtual Machine. You can always upgrade or downgrade your memory once you have everything installed and running.

     • Set the resources needed for this VM: RAM, CPU cores, CPU priority.

     • The amount of CPU resource a VM is given is the CPU priority (you can think of this as its share percentage) multiplied by the number of cores allocated to that VM. This is a minimum guaranteed number, so clients can burst over it, up to 100% multiplied by the number of cores. For example, on our systems:

       • 100% x 1 core = ~2.5 GHz (not burstable)

       • 10% x 1 core = ~250 MHz (burstable to ~2.5 GHz)

       • 10% x 4 cores = ~1000 MHz (burstable to ~10 GHz)

       • Note: You will get a better value getting 10% on each of two cores than getting 10% on one core.

       • Note: On newer plans, only full cores are offered for allocation and are always burstable to 100%, so the priority option may not appear for you.

       • On average, you'll get ~50 MHz usable (guaranteed ~25 Mhz) per 1% of the CPU, up to 100% load per core which is around 2.5 GHz on average.

       • It is recommended to select at least 5% CPU and for standard server operations (multi-threaded applications or multiple applications running) we recommend increasing the number of cores before increasing the percentage to give yourself access to more resources. 

       • Most VMs are fine with 5% of the CPU. If you notice things are running slowly you can always instantly increase your CPU resources at a later time.

   • Primary disk

     • Set the primary disk size.
     • You can set up secondary disks after the initial setup.

   • Swap disk

     • We recommend using SAS storage for swap space. Generally 1 GB is more than enough swap space for most uses, we do not generally recommend using more than 2 GB.

     • Set the swap disk size.

   • Network configuration

     • Unless you have specific plans for using the internal network, set up your initial interface using the public network. There is no fee for selecting higher ports speeds, so you will likely just want to select the maximum 100 Mbps port speed.  The port speed can be set lower if you wish to limit your potential bandwidth fees.

     • Choose a network zone and set the port speed for this VM.

     • An IP address will be assigned automatically based on the network zone selected.  You can add additional IPs after provisioning.

     • The public network gives you access to the public Internet and outbound usage over your included amount is billed per GB. Inbound usage is not charged.

     • The internal network gives you access to other VMs and our internal network and is to be used to connect to the VPN, backup servers, and/or other servers on our network.

6. Once you have selected the initial VM resources, at the bottom of the web page click on Next.

7. We recommend leaving all the default settings as-is on the "Automation Settings" page unless you've been instructed to by our staff or other documentation.

8. Review the confirmation page and then click the Create Virtual Machine button to start the creation process.

We also have directions for setting up VMs using specific common control panels such as InterWorx, cPanel.

If you experience any issues creating a virtual machine, please contact support for assistance.

This article describes an issue where Debian 8.x VMs and Ubuntu 14.04 VMs running 3.x kernels will crash after running for exactly 5 minutes when running on Xen 4.8-based hypervisors.

Issue Description

This issue is caused by a bug in some 3.x kernel series, including 3.13 and 3.16 when handling how Xen 4.8 presents virtual CPUs to the VM.  This problem causes the kernel to produce a "machine check exception" and shut off.  This problem goes away if you upgrade to a newer version of Debian or Ubuntu that includes a 4.x series kernel such as 4.4 or 4.9, and does not appear to affect older kernels including version 3.2 and 2.6 series from previous versions of Debian and Ubuntu.

Resolution

To prevent this problem, the boot option "nomce" should be added to the kernel parameters.  This can be done by running the following commands from inside the VM:

sed -i 's/^# kopt=.*/\0 nomce/' /boot/grub/menu.lst*
update-grub

Once you have applied this configuration, reboot the VM to prevent further crashes.

You can verify that your server is running the correct command line options by running the following command:

cat /proc/cmdline

The output will resemble the following including the "nomce" option:

root=/dev/xvda1 ro nomce

In response to this issue we have corrected the latest version of the Debian 8 template available for provisioning new VMs and have discontinued offering the Ubuntu 14.04 template. Ubuntu 18.04 is expected to be available soon, so this change follows closely to the typical schedule for template removal.

References

• Debian Bug Report #860236

DNS resolvers are used to look up IP addresses, mail servers, and other information related to a domain name.  Resolvers are required to use the Internet effectively.  This article describes the resolver configuration for servers hosted with Steadfast.  A new dedicated server or cloud VM will automatically be configured to use these settings by default.

We provide the following DNS resolver addresses for all customers on our network:

IPv4:

• 216.86.146.8
• 216.86.146.9

IPv6:

• 2607:f128:1::2
• 2607:f128:1::3

If your server connects only to our Internal Network and has no public Internet Connection:

• 10.1.251.10

You can enter the appropriate IP addresses under "Resolver IPs" in your Windows Interface Configuration, in WHM, or by inserting the following lines into /etc/resolv.conf on a Linux or FreeBSD system:

The pairs of IPs above are each hosted on physically separate servers.  You should always use at least two separate servers, if possible.  Feel free to use these IPs as backup addresses even if you run your own DNS resolver.

Please note that if you are considering running your own resolver, you should make sure it is not publicly available unless absolutely necessary.  Please see our article about DNS Amplification Attacks for further information.

Basic Setup

It's now possible for all customers to add and remove domains without administrator assistance. To do this, go to "Domains" and select "Add domain" then specify the new domain and route you wish to add. Once this is done, you can begin using the filtering system by changing the MX records for your domains as follows:

• <domain> should be the domain name. Some DNS systems expect the domain to be left blank.
• <TTL> can be left to your DNS system's default or any value of your preference.
• The value of 10 is the priority and can be changed at your discretion as well
• Both records should be left at the same priority.

Per-domain and per-mailbox user creation

It's now possible to create per-domain and per-mailbox user accounts. By selecting "Permissions" under the "Users" heading, it is possible to create restrictions on which features the user accounts may access.

Per-mailbox users are created by selecting "Webinterface users" under the "Users" heading. These users may:

• Access mailbox-level email logs and quarantine.
• Configure mailbox-level scheduled protection reports.
• Report spam that was missed by the filter.
• Mark the mailbox as whitelisted from filtering.

Per-domain users are created by selecting "Manage domain users" under the "Users" heading. These users may:

• Manage domain filtering and delivery policies.
• Manage whitelists and blacklists.
• Access domain-level email logs and quarantine.
• Configure domain-level and mailbox-level scheduled protection reports.
• Report spam that was missed by the filter.
• Manage per-mailbox "webinterface users"

Quarantine Access and Spam Reporting

There are three ways a user can access his quarantine and report spam:

• By logging into the web interface.
• Setting up scheduled protection reports
• By setting up the IMAP quarantine system.

Any user with a domain or webinterface (per-mailbox) account can access any method.

Using the Web Interface

When logged into the web interface, a domain or webinterface user can use the "Report Spam" or "Spam Quarantine" options to manage the quarantine and report misclassified messages.

Setting up Protection Reports

Domain-wide protection reports can be set up by domain users by selecting "Manage Settings" under the "Protection Report" heading. Per-mailbox protection reports can be set up by either domain or webinterface users by selecting "Manage recipients" under "Protection Report." These reports are emailed to the user on the schedule indicated and individual messages can be released via a link next to each listed message.

Using the IMAP System

To set up the IMAP system, a user may simply add an IMAP account to their favorite email client with the following details:

• Server: spamfilter01.steadfast.net
• Username: domain name or webinterface user name (email address)
• Password: user's password

Three mailboxes will be visible:

• Caught
• Release
• Spam

"Caught" is a browseable folder that shows all quarantined messages. "Release" and "Spam" cannot be browsed. To release any caught messages that is not spam, a user can simply drag the message from "Caught" to "Release" and it will be delivered normally. To mark any uncaught message as spam, simply drag it from a usual mailbox to the "Spam" folder.

Either action will report the message as misclassified so that future messages may be filtered more correctly.

Further Reading

Extensive documentation for the SpamExperts appliance is available via the official Wiki. Please be aware that not all features listed are available on our filtering system. If you have any questions or need assistance, please contact support and we'll be happy to assist you.

Important: Do not set a regular gateway for the internal network in /etc/sysconfig/network-scripts/ifcfg-eth1, /etc/network/interfaces or Windows Network configuration. These will cause the external network connection to work improperly.

Once the internal network is set up, you should be able to successfully ping int.shell01.backup.steadfast.net

Linux

CentOS or Fedora

1. Edit the file /etc/sysconfig/network-scripts/route-eth1 (replace "eth1" with your actual interface name) to establish the default route for the subnet: 10.0.0.0/9 via 10.3.128.1
2. Also remember to set the Internal NIC to boot on startup. In /etc/sysconfig/network-scripts/ifcfg-eth1 (replace "eth1" with your actual interface name) set: /> IPADDR=10.x.x.x
NETMASK=255.255.128.0 Do not put a GATEWAY line in this file, or you may experience public network connectivity issues.
3. Once this is done, reload the network configuration: /etc/init.d/network restart

Debian or Ubuntu

1. Make sure to include this interface definition below the one for eth0 in /etc/network/interfaces. Replace ''10.x.x.x'' with your internal network IP and all three instances of "eth1" with your actual interface name. Make sure to maintain the indentation as listed below:

   auto eth1
   iface eth1 inet static
       address ''10.x.x.x''
       netmask 255.255.128.0
       post-up ip route add 10.0.0.0/9 via 10.3.128.1 dev eth1
       pre-down ip route del 10.0.0.0/9

   Do not put a gateway line (as in eth0) in the eth1 section of this file, or you may experience public network connectivity issues.
2. Once this is done, reload the network configuration: /etc/init.d/networking restart

Windows

Windows 2008

1. Access your internal network interface properties and configure the correct IP address and subnet mask (255.255.128.0) are entered, but do not enter anything in the gateway field.
2. Start a command prompt and run the following: route -p add 10.0.0.0 mask 255.128.0.0 10.3.128.1 This avoids the need for installation of Routing and Remote Access which is not supported by Windows Web Server 2008. Installing RRAS is an alternate method for other editions of Windows Server 2008, and the process for Windows 2003 can be used in that case.

The -p flag stores the route in the registry (so it can be restored at reboot) under the following key:

If you modify the entries directly in the registry, you must restart Windows for the changes to take effect.

1. Access your internal network interface properties and configure the correct IP address and subnet mask (255.255.128.0) are entered, but do not enter anything in the gateway field.
2. Under Control Panel -> Administrative Tools -> Routing and Remote Access, right click on the name of the system and select Configure and Enable Routing and Remote Access.
3. Select Custom configuration, check LAN routing, select Finish and choose Yes for starting the service.
4. When prompted to wait, it is recommended you wait. Continue to '''wait a few seconds''' after the wait dialog box has gone away. Additional items will eventually show up under the computer name in the Routing and Remote Access window.
5. Click on '''IP Routing''' if it does not expand automatically
6. Right click on Static Routes and select New Static Route.
7. Select the correct Interface that is connected to the internal network.
8. Destination should be set to 10.0.0.0, netmask of 255.128.0.0 and gateway of 10.x.x.x where the x.x.x is set accordingly for the servers own internal IP address. Metric should be left at 1.
9. Click Ok and now you have created that route. Hooray!

If you experience trouble with Internal Network routing, please feel free to contact support.

This document describes how to connect to the Steadfast Networks VPN server for accessing the Internal Network and IPMI remote server management.

Requesting Access

Access to the VPN is freely available to all customers with a service that is provides IPMI or Internal Network access.  To request access, your name must be listed as an authorized contact in our management portal, or the request for your access must come from someone in that list.

Each computer and person that will connect to the VPN should have its own set of credentials.

Access requests should include the following information:

• Client ID
• Authorized User or Device Name
• Authorized User's Email Address

An email address and client ID are required.  You must also provide either a user or device name to identify the credentials.

Please send the request via email or our helpdesk to the Tech Support department from a person on the authorized contact list for your account in our management portal.  Once the request is received, it will be reviewed within 1 business day, and the credentials will be made available to you if approved.

Revoking Access

In case VPN credentials are compromised or you need to revoke access from a person who is no longer authorized, please contact our Tech Support department via email or our helpdesk.  Please indicate if the situation is an emergency and our team will escalate the request to be handled as quickly as possible.

If you terminate your services with Steadfast, all VPN credentials will be revoked as part of the account closure process.  We also may revoke VPN credentials of any user found to be abusing the service, or if we suspect a compromise.  We will make every attempt to notify affected users if we revoke credentials proactively.

About the VPN Service

The VPN server is running software provided by the OpenVPN project. More information about this project is available at: http://openvpn.net/

The VPN provides access to servers and services that are accessible on the Internal Network and to IPMI services for Dedicated Servers.  It is supported on Windows 7 and later, Mac OS X 10.7 and later, Linux, and mobile devices running iOS 9.0 or Android 4.1 and later.

Please note that though the Internal Network and IPMI are both accessible using the same VPN service, it is not possible to communicate between IPMI devices and the Internal Network directly.  The VPN has the ability to reach both networks, but the networks are not otherwise connected together.

Internal Network

The Internal Network is a service included with Dedicated Servers and Cloud accounts, and available to Colocation accounts upon request.  This is a separate network that links servers and services within Steadfast to one another, but is not accessible from the Internet.  You can communicate with your servers' Internal Network IP addresses and transfer data for no additional charge.  Please note that bandwidth on the VPN is limited and performance will be lower than it would be when accessing your server over the Internet directly.

IPMI

IPMI is a service provided with all Steadfast dedicated servers.  It allows you to remotely access your server's keyboard, video, mouse (KVM), and power controls.  It works with most modern web browsers running on Windows, Mac OS X, and Linux.  Remote KVM requires Java to be installed on your computer.

Your server's IPMI address can be found in the management portal Devices list or your server's welcome email.  It can be accessed only when your are connected to the VPN.

Connecting to the VPN

Your VPN credentials will be delivered in a ZIP file.  The software required to use them is not included.  It must be downloaded separately as indicated in the following sections.  Please locate the section for your computer or device and follow those instructions.  If you need assistance, please feel free to contact us for help.

Windows

Windows devices need to use the OpenVPN community client. Make sure you are running Windows 7 or later. If your computer is running Windows Vista or XP, you should not use the computer to access the Internet.

• Go to the following link to download the software: https://openvpn.net/community-downloads/
• Click the button that says "See Details" next to the top version in the list.
• Click the button next to the "Windows Installer" entry in the list which starts with the name "openvpn-install."
• Run the downloaded installer program to install the VPN client, selecting all the default options.
• Once the VPN client is installed, copy the "steadfast.ovpn" file from the ZIP file into c:\Users\<Your Username>\OpenVPN\config. Create this folder if it doesn't exist.

The VPN client will start automatically when you start up or log into your computer. The icon will appear in the system notification area (tray) when it is running:

To connect to the VPN, right click the icon and click "Connect." The first time you connect, you may be prompted to allow OpenVPN to modify the firewall and add your user account to the OpenVPN Administrators group. Allow both of these actions or the VPN will not work properly.

While connecting, a popup window will appear with information about the connection attempt. It will disappear when the connection is finished. The icon will change to indicate the connection is active with a closed lock and green screen:

Mac OS X (Tunnelblick)

MacOS X devices should use the Tunnelblick OpenVPN software.

• Go to the following link to download the software: https://tunnelblick.net/downloads.html
• Click on the "Stable" download and install it on your computer.
• Once installed, start Tunnelblick
• To install the VPN configuration, drag the "steadfast.ovpn" file from the ZIP file to the Tunnelblick icon on the menubar:
• Make sure you choose OpenVPN version 2.4.0 or later to avoid various known issues with VPN connections

To connect to the VPN, click the Tunnelblick icon on the menu bar and select the "Connect" option that matches your VPN profile. Once you have connected the icon will change from gray to black:

Android Devices (4.1 or later)

Android 4.1 and later devices should use the OpenVPN Connect app from Play Store.

• Go to the following link to install the app from your Android device or a computer logged into the same Google account as your phone: https://play.google.com/store/apps/details?id=net.openvpn.openvpn
• Open the ZIP file and copy the "steadfast.ovpn" file to your computer
• Copy the "steadfast.ovpn" file to your device via your preferred transfer method
• Once the app is installed and the "steadfast.ovpn" is on your device, open the OpenVPN Connect app
• Tap the green plus button in the lower right corner

  

• Browse to the location on your phone where you saved the VPN configuration file
• Tap the "steadfast.ovpn" file in the list so it has a check mark to its right, then tap "IMPORT" in the upper right corner

  

• Enter a name for the profile or accept the default and tap "ADD" in the upper right corner

  

To connect to the VPN, open the OpenVPN Connect app and tap the toggle switch for the VPN profile:

Once you have connected, the status indicator will show "CONNECTED" and the toggle switch will move to the right:

iOS Devices (9.0 or later)

Apple iOS 9.0 or later devices, including iPhones, iPads, and iPods should use the OpenVPN Connect app from the App Store.

• Go to the following link to install the app from your iOS device or a computer logged into your Apple account: https://itunes.apple.com/us/app/openvpn-connect/id590379981
• Open the ZIP file and copy the "steadfast.ovpn" file to your computer
• Using iTunes, select the File Sharing feature, click on "OpenVPN" and add the "steadfast.ovpn" file under "OpenVPN Documents"
• Once the "steadfast.ovpn" file is on your device, open the OpenVPN app
• You should see a message that there is 1 new OpenVPN profile available for import; tap the "ADD" button.

  

• Enter a name for the profile or accept the default and tap "ADD" in the upper right corner

  

• You will see a notice that "OpenVPN" would like to add VPN Configurations; tap "Allow"

  

To connect to the VPN, open the OpenVPN Connect app and tap the toggle switch for the VPN profile:

Once you have connected, the status indicator will show "CONNECTED" and the toggle switch will move to the right:

Linux

On some GNU/Linux distributions, NetworkManager-openvpn provides an easy method to configure and run OpenVPN from within GNOME. Otherwise, try running the following command as root:

openvpn --config steadfast.ovpn

The openvpn package is standard on all Linux distributions, but you will need version 2.4.0 or later for full compatibility with our VPN service.

If using NetworkManager, you will need to use the individual certificate and key files contained in the ZIP file for connecting as NetworkManager does not yet support the "ovpn" configuration file format.

Contents of The ZIP File

• steadfast.ovpn - This is the primary OpenVPN configuration file. It includes the content of the files below. The other files are provided in case the application you are using does not support reading the certificate information directly from this configuration file.
• steadfastca.crt - This is a public certificate file used to verify the VPN and your personal certificate. It is used for the "ca" configuration option with OpenVPN.
• client1.crt - This is your personal public certificate file. It is used for the "cert" configuration option with OpenVPN.
• client1.key - This is your personal private key file. Authentication with the VPN is accomplished using this file. It is important that you keep this file secret. The file is used for the "key" configuration option with OpenVPN.

Testing the Connection

Once connected, you should be able to run:

ping 10.2.255.10

To run ping from Windows, run a Command Prompt (normally found under Accessories) or run cmd.exe  Then select the resulting black box and type"ping 10.2.255.10" and press enter.

To run ping from Mac OS X, go to Applications, then Utilities and then select Terminal.  Select the resulting terminal window and type"ping 10.2.255.10" and press enter.

If the ping is successful, you should see lines that begin with something similar to "64 bytes from 10.2.255.10" that are produced about once per second.

Custom templates can be used to quickly deploy custom systems and configurations. If you have a set configuration for your web servers, you only need to set them up once, then save the template, and you'll be able to deploy future web servers almost instantly.

Create a Custom Template

You can create custom templates by making a backup of an existing virtual machine and saving it as a template for future use. To create a custom template:

1. Create a new virtual machine and configure it as you would like for your template.

2. From the selected VM, mouse over the “Storage” menu and then click the "Backups" item.

3. In the list of backups, click "Convert to Template" icon next to the backup you want to convert.

   • If a backup does not exist, you will need to create one. See Backup and Restore Virtual Machines for more information.

4. On the next screen, enter a label for your template/backup, then click the "Convert Backup" button.

5. The backup will be scheduled for creation. When conversion is complete, it will be then listed on the "Templates" tab, from where you can edit it.

Edit and Delete Custom Templates

You can edit or remove your custom templates at any time. To do so:

1. Go to your Control Panel’s "Templates" menu. Your custom templates will be listed there.

2. In the “Actions” column, click the gear icon next to the template.  

   1. Click the "Edit Template" if you wish to edit, make your changes on the screen that follows, and click Save Template.

   2. Click the "Delete Template" if you want to delete it.

If you need assistance with this process, please feel free to contact support.

You can adjust CPU and RAM resources for all VMs. Some VMs can have their CPU and RAM adjusted without needing to be powered off using the “resize without reboot” option.

Windows 2008 and Windows 7 VMs can be adjusted without rebooting; Windows 2003 requires a reboot. With Linux it depends on kernel. CentOS 5 with kernel 2.6.18 can be resized; CentOS 6 and Ubuntu cannot.

To adjust VM CPU & RAM resources:

1. Go to your Control Panel’s Virtual Machines menu.

2. Click the label of the machine you want to resize, to show its details screen.

3. On the right side of the details, click on the green “Tools” menu.

4. Under VM Options, click on “Edit Virtual machine”

5. Change CPU core, priority and RAM values and click the "Save" button.

If the VM template allows resize without reboot, the resize should be completed automatically and you will be returned to the VM details screen with a message indicating the resize was successful. If the template does not allow this, you will be asked to confirm that the VM will be rebooted so that the resize can take place.

How to order control panel licenses for your VM

Prerequisite: You must have created a Virtual Machine (VM) before you can license a control panel for it. Please see Creating a Virtual Machine for more information.

1) Go to https://manage.steadfast.net and login to your account.
2) Click on "Billing and Services" and select "View Services".
3) Select your Cloud Platform service.
4) On the right side of the service page is a section called "Order Control Panel Licenses"
5) Select your VM from the VM drop down.
6) Select the license type that you want to issue.
7) Click "Purchase"

For cPanel
cPanel will automatically be licensed on the main IP of the selected server.

For Interworx
You will be returned to the screen for your cloud service. There will be a link to your control panel order in the "Order control panel licenses" box. Click on this link to go to the licensing section and follow the instructions in the "Interworx Licensing" box.

If you have any issues please contact our support department at support@steadfast.net.

In March 2013, the Open DNS Resolver Project identified many IP addresses on our network that were of moderate to severe risk of participating in a DNS amplification attack.  This attack queries name servers for large results using a fake source address.  This request causes the response to go back to the faked address, resulting in a large amount of data being sent to a computer that did not request it.  This effect, when used with thousands of DNS servers, directs a very large amount of traffic to a single IP to form an efficient distributed attack.  The anti-spam organization Spamhaus was recently the victim of an attack that may have been as large as 300 Gbps using this technique.

This attack can be performed easily using a server you control without compromising its security, and could result in heavy outbound bandwidth usage.  This may impact performance of your services and cause unexpected bandwidth overage bills.

The most common environment we have found with a problem is a CentOS 5.x server running BIND with default settings.  The default BIND version for CentOS 5.x causes a moderate risk.  A severe risk occurs if any DNS server is configured to act as a public DNS resolver.  A public resolver is a server that allows anyone to query it for the DNS records of a domain it does not directly host.

You can confirm your server is affected by querying the server from a Linux or Mac command line on a separate computer:

dig steadfast.net @<server ip>

A version of the "dig" command for Windows can be downloaded from here.

If the resulting status is NOERROR, your server allows queries that it should not.  If the information contains AUTHORITY results but does not containANSWER results, it is a moderate risk.  If it contains any ANSWER results, then the risk is severe.  Other statuses, such as REFUSED, NXDOMAIN, SERVFAIL, or a timeout error message, do not indicate an issue.

To mitigate a moderate risk, the best option is to install the CentOS 5.x bind97 package.  In a cPanel/WHM environment, upgrading to bind97 can be accomplished with the following commands:

cp -Rf /var/named/ /var/named.bak
/scripts/update_local_rpm_versions --edit target_settings.named uninstalled
/scripts/update_local_rpm_versions --edit target_settings.bind uninstalled
yum -y remove bind bind-utils bind-devel bind-libs caching-nameserver
yum -y install bind97 bind97-libs bind97-utils bind97-devel
/usr/local/cpanel/scripts/rebuilddnsconfig

In a non-cPanel environment, you can perform similar steps, but you will likely need to rebuild the /etc/named.conf file from the /etc/named.conf.rpmsave.

Another option that can be used to mitigate the moderate risk level is to upgrade your server from CentOS 5.x to CentOS 6.x.  This upgrade enables you to access other new and improved software and may improve server performance.  However, doing this usually requires reinstalling your operating system and restoring data from backups.

To mitigate a severe risk, you must reconfigure your name server manually.  Recursive resolver behavior is not the default, which means that a configuration change was made to enable recursion on your server.

For advice on how to adjust a server to prevent public recursion or limit the IP ranges that can use recursion, or any other questions about the topics discussed in this article, please visit our Help Desk or email us.  BIND consulting is covered under managed services.

In 2018 we saw a significant increase in reports of amplification attacks that take advantage of the LDAP protocol over UDP (CLDAP).  This attack queries LDAP servers for large results using a fake source address. This request causes the response to go back to the faked address, resulting in a large amount of data being sent to a computer that did not request it. This effect, when used with thousands of LDAP servers, directs a very large amount of traffic to a single IP to form an efficient distributed attack.

Most LDAP servers and clients use the TCP protocol, which prevents amplification because of a connection handshake that verifies the source and destination can communicate with one another.  UDP does not perform this verification, so the LDAP server can be convinced to send traffic to a destination that is unverified.

The easiest way to solve this issue is to enable a firewall on your server that blocks the LDAP port 389 from being accessed via UDP.  LDAP is most commonly used on Windows servers running Active Directory services.  If you have a program that is using LDAP via UDP from another server, you should add a firewall exception to allow that application to continue to work, or change that application to use LDAP over TCP.  LDAP may also be running with encryption (LDAPS) on port 636, but this protocol only supports TCP.

To disable access to LDAP over UDP if you do not have any servers that access it, follow these steps:

1. Right click on Start, then click Run and type "wf.msc" click "OK"
2. Click on the "Inbound Rules" option on the left side of the window.
3. Locate the rule called "Active Directory Domain Controller - LDAP (UDP-In)"
4. Right click on the rule and select "Disable Rule"

If you need to allow access to LDAP from other servers, follow these steps:

1. Right click on Start, then click Run and type "wf.msc" click "OK"
2. Click on the "Inbound Rules" option on the left side of the window.
3. Locate the rule called "Active Directory Domain Controller - LDAP (UDP-In)"
4. Right click on the rule and select "Properties"
5. Click on the "Scope" tab
6. Under the "Remote IP address" section, select the option "These IP addresses:"
7. For each IP address or range that should have access, click "Add..." and enter the correct ranges.
8. Once you have entered all the ranges that should have access, click "OK" to save the rule.

If you wish to restrict the LDAP over TCP or the Secure LDAP service for security reasons, you may also wish to modify these rules using the same steps above:

• Active Directory Domain Controller - LDAP (TCP-In)
• Active Directory Domain Controller - Secure LDAP (TCP-In)

If you are running an LDAP server on Linux, you should modify your LDAP server configuration in accordance with its documentation to disable or restrict LDAP over UDP, or configure your system firewall accordingly.  Steadfast does not currently support any standalone LDAP servers or any products with an exposed LDAP server.

For advice on how to adjust a server to prevent LDAP amplification or limit the IP ranges that can make LDAP queries, or any other questions about the topics discussed in this article, please visit our Help Desk or email us. LDAP configuration on Windows servers is covered under managed services.

In 2018 we have seen a large number of DDoS attacks making use of unsecured memcached services running on the internet.  On some Linux distributions memcached servers default to listening on all network interfaces, including those facing the internet.  Exposing the service puts servers at risk of participating in an amplification attack and may expose some sensitive information stored by the application using memcached. This attack queries memcached servers for large results using a fake source address. This request causes the response to go back to the faked address, resulting in a large amount of data being sent to a computer that did not request it. This effect, when used with thousands of memcached servers, directs a very large amount of traffic to a single IP to form an efficient distributed attack.

If you are using memcached only with an application running on the same server, you should configure the service to listen only on the local interface so that it can never be exposed on the internet.  To do this:

On CentOS:

1. Edit the file /etc/sysconfig/memcached
2. Find the line that begins with OPTIONS= and add -l 127.0.0.1 between the quotation marks.
3. If there is no such line, add one that says OPTIONS="-l 127.0.0.1"
4. Restart the service by running the command

   service memcached restart

On Debian or Ubuntu:

1. Edit the file /etc/memcached.conf
2. Find the line that begins with -l and make sure it reads -l 127.0.0.1 
3. If there is no such line, add one at the end of the file that says -l 127.0.0.1
4. Restart the service by running the command

   service memcached restart

If you are running an application on another server that needs to connect to memcached, you should configure the server firewall to only accept connections on port 11211 from IP address ranges of application servers that need to connect to this server.

If you aren't using memcached, you should remove or disable the software. To remove it:

On CentOS:

yum remove memcached

On Debian or Ubuntu:

apt-get remove memcached

For advice on how to adjust a server to prevent memcached amplification, or any other questions about the topics discussed in this article, please visit our Help Desk or email us.  memcached is not supported software, but our support team can assist with firewall and package management to disable or restrict access to it.

Recently, a large number of DDoS attacks have begun to make use of unsecured SNMP services running on the Internet.  SNMP services have a default community (authentication name) called "public" which can be used to return some read-only monitoring statistics about a server.  Exposing the "public" community puts servers at a moderate to severe risk of participating in an SNMP amplification attack and may expose some information that makes them easier to exploit, such as the version of Linux being used and the network configuration. This attack queries SNMP servers for large results using a fake source address. This request causes the response to go back to the faked address, resulting in a large amount of data being sent to a computer that did not request it. This effect, when used with thousands of SNMP servers, directs a very large amount of traffic to a single IP to form an efficient distributed attack.

You can confirm your server is affected by querying the server from the local command line, or from a separate computer's Linux or Mac command line (with the net-snmp package installed):

snmpwalk -v 2c -c public <server IP>

If the result of these commands is not “Timeout: No response from <server IP>” then your server allows queries that it should not.

If you have full management, our monitoring system needs SNMP but does not need the "public" community.

If you aren't using SNMP (and you don't have full management) on your server, we recommend you remove it:

CentOS:

yum remove net-snmp

Debian, Ubuntu or Jumpbox:

apt-get remove snmpd

If you have full management, you should instead edit the file /etc/snmp/snmpd.conf and remove or comment (place a # before) the line that reads:

com2sec notConfigUser default public

Then run:

service snmpd restart

If you are using SNMP for some purpose, please change the /etc/snmp/snmpd.conf to not expose the community "public" to the Internet. Use a random string that's at least 16 characters long. You can replace the community in the config file on the line described above by changing the word "public" to something else. If this is a switch or other network device, rather than a server, please disable the default "public" community setting for your switch and change the community name if you need to monitor the switch.

Please also note that this is not SMTP (an email service), it is SNMP (a monitoring service). Your server was not hacked by this activity, so unless you see further reports, the problem should be resolved with one of the actions indicated above.

For advice on how to adjust a server to prevent SNMP amplification, or any other questions about the topics discussed in this article, please visit our Help Desk or email us. SNMP consulting is covered under managed services.

Spamhaus provides a set of managed block lists to assist with identifying and blocking IP addresses and domain names that are likely to send out spam or cause malware infections.  These lists are available via the spamhaus.org web site as well as via the DNS-based Block List (DNSBL) standard.  To limit the load on their infrastructure, Spamhaus only permits users to query the service for non-commercial purposes and sets a cap on the number of daily queries allowed.

As of August 2016, due to the fact that Steadfast is a commercial business and has a high volume of DNS traffic, Spamhaus has requested that we reject all queries to the DNS block lists via our public resolvers.  This means that instead of fetching a usual DNSBL response code, the resolver will return NXDOMAIN, which indicates no result is available.  This response should not cause any issues for mail servers using Spamhaus services, except that they will no longer be able to use the block lists for filtering email.

We cannot grant exceptions to the query restrictions on our public DNS resolvers.  It is not possible for Steadfast to meet the usage terms of the free DNS feed and we cannot reasonably meter the usage of the paid feed to provide a bundled version to our customers.

If you need the data from Spamhaus as part of an anti-spam effort or product, you have two options.  You may either run a DNS resolver locally on your server to query the DNS block lists directly if you meet the free feed criteria, or you may contact an authorized Spamhaus reseller to gain access to the paid version of the data feed intended for commercial use and high-volume consumption.

For more information on the data feed and its restrictions, please see the following web site:

• https://www.spamhaus.org/organization/dnsblusage/

If you have questions about how to run a local DNS resolver, please feel free to contact us.

If your server or VM is frequently running out of memory this article should be of great assistance to you and will guide you through diagnosing what the issue is as well as listing some possible fixes.

First of all, to see how much memory you are currently using, run free -m. It will give you output such as:

             total       used       free     shared    buffers     cached
Mem:           363        354          9          0         46        137
-/+ buffers/cache:        170        193
Swap:         1023         53        970

The "used" value (354) will almost always be close to the total memory (363) in the system. This is because Linux uses spare memory to cache data in order to reduce the reliance on the hard drive. Here 137MB are being used for cache. You can read more about this in the "Why is so much memory in use on my server when nothing is running?" article.

The main thing you're going to want to look at is the "-/+ buffers/cache:" used value (170). That is the actual amount of memory your applications are currently using. For best performance, this number should be less than your total (363) memory and in order to prevent out of memory errors, it needs to be less than the total memory (363) and swap space (1023).

ps

In order to see where all your memory is going, just run ps aux. That will show the percentage of memory each process is using (in the %MEM column) and you can use it to identify the top memory users (usually Apache or MySQL). Remember to add together all instances of the service to calculate how much memory the service is using as a whole.

Resolving: High Apache Memory Usage

Apache is often one of the biggest memory users. Apache is run as a number of 'servers' and incoming requests are shared among them. The memory used by each server grows, especially when the web page being returned by that server includes PHP or Perl that needs to load in new libraries, and it is no uncommon for each server process to use as much as 10% of a server's memory.

In order to reduce the memory usage you can reduce the number of servers by editing your httpd.conf file. There are three settings you are going to want to look at: StartServers, MinSpareServers, and MaxSpareServers. Each can be reduced to a value of 1 or 2 and your server should still respond promptly. After changing the settings in the httpd.conf file be sure to restart Apache by running "service httpd restart" to assure the new settings are in effect.

Resolving: High MySQL Memory Usage

MySQL is relatively memory efficient on install as most memory intensive features are not enabled by default, but you can add the following lines to the /etc/my.cnf file, under the mysqld section, to free up some additional memory:

Again, make sure you restart the service for those settings to take affect. This can be done by running "service mysql restart"

Resolving: High SpamAssassin Memory Usage

SpamAssassin can also be a major memory user as it can create multiple threads/processes and each of those threads can use a good amount of memory, but SpamAssassin normally works very well with just one thread. You can reduce the 'children' setting and reclaim some memory on your server for other apps to run with.

To do this change the SPAMDOPTIONS line in the /etc/init.d/spamassassin file to:

Final Solution: Add Memory

A simple solution to resolving most out of memory problems is to add more memory. If you'd like to increase the memory on your VM, just modify your VM directly from your control panel at https://vm.steadfast.net/ and if you have a dedicated server simply email us at sales@steadfast.net in order to get an upgrade.

There are a few common ways to restrict SSH access to your server but still allow our technicians to access your server.

Changed SSH Port or Requirement to Use sudo or su

In this case, please do the following:

1. Log into the management interface.
2. Click on "Device Manager"
3. Click on the server that is affected by your changes.
4. Click on "Edit" link on the "Device Metadata" section.
5. Input the changes in port, username, password, and any additional login directions that might be needed.

Firewalled or Restricted SSH connections to certain IP ranges

In this case, please be sure to allow the IP ranges:

67.202.100.0/23 (Chicago Offices)
10.252.0.0/24   (Edison, NJ Office)
10.254.4.0/24   (Corporate & Engineering Office)
2607:f128::/48  (For IPv6)

SSH Public Key Authentication Only

For CentOS Systems: We now have an RPM that can be installed to handle this automatically. See the following knowledge base article: Adding Support Staff SSH Keys using RPM.  If you use this method, the keys will update automatically when we publish a new version.

If you are not using CentOS or wish to maintain the list of keys manually, you can find the current key file here.

To use it, download and place the file at /root/.ssh/authorized_keys2 on your server.

Note: As our staff changes, we will update this list of keys.  We recommend that you check the file at the link above periodically for a new version.  The modification date is always listed in the comment at the top of the file.

1. Set up a cPanel Virtual Machine by selecting the appropriate cPanel x64 option, listed under Template Sets, then CentOS 6 or CentOS 5.
2. Get the license as directed below, then log into your server via SSH and run the following command to update your server's licensing information: /usr/local/cpanel/cpkeyclt

If you prefer to install cPanel from scratch, you may, by using the following directions:

1. Setup a cPanel compatible VM by selecting a base CentOS image, listed under Template Sets, then CentOS 6 or CentOS 5.
2. Run the following commands as root on your VM: cd /home
wget -N http://httpupdate.cpanel.net/latest
sh latest
3. Allow 30 minutes to 1 hour for installation. Once the process completes, you should be able to access your control panel.
4. Get the license as directed below, then log into your server via SSH and run the following command to update your server's licensing information: /usr/local/cpanel/cpkeyclt

Once installed and licensed, you can access WHM at https://YOURIP:2087/ using the username "root" and your root password. Once you have created user accounts, cPanel can be accessed for those accounts at https://YOURIP:2083/.

WHM User Guide: http://docs.cpanel.net/twiki/bin/view/AllDocumentation/WHMDocs/WebHome
cPanel User Guide: http://docs.cpanel.net/twiki/bin/view/AllDocumentation/CpanelDocs/WebHome

Getting a License

cPanel includes a 45 day trial license by default. To add or remove licenses you must open a ticket with our sales department or email sales@steadfast.net. cPanel licenses are $10 a month each. We are hope to have automated cPanel license ordering available in the next several months.

1. Set up an InterWorx Virtual Machine by selecting the appropriate InterWorx x64 option, listed under Template Sets, then CentOS 6 or CentOS 5.
2. Install your license. Get the license as directed below, then access https://YOURIP:2443/nodeworx/ and enter your license key along with your desired administrator username and password.

Once installed and licensed, you can access InterWorx at http://YOURIP:2443/ using the username and password you selected.

InterWorx Documentation: hhttp://www.interworx.com/support/docs

Getting a License

InterWorx does not include a license by default. To add or remove licenses you must open a ticket with our sales department or email sales@steadfast.net. InterWorx licenses are $8 per month each. We hope to have automated InterWorx license ordering available in the next several months.

The installation instructions are fairly detailed, so it is most efficient for us to point you to the official documentation. Our support staff can assist you with installation of LiteSpeed in most common configurations.

You will access the Litespeed administrative interface at http://YOURIP:7080/

Litespeed Documentation: http://www.litespeedtech.com/docs/

Getting a Litespeed License

Litespeed does not include a license by default. To add or remove licenses you must open a ticket with our sales department or email sales@steadfast.net. Litespeed VPS licenses are $12 a month each. We hope to have automated Litespeed license ordering available in the next several months.

If you don't want to purchase a license the Standard version is free, but has limited features. There is a 15 day free trial (up to 2 CPUs) here for the Enterprise version if you would like to test it out first: http://www.litespeedtech.com/trial/license

Differences between Standard and Enterprise: http://www.litespeedtech.com/litespeed-web-server-editions.html

Windows has some particular issues and qualities that don't always mix well with our virtual environment, so we have noted some specific directions relating to Windows based virtual machines here.

Install Times

Linux VMs can be deployed almost instantly, but Windows requires various install scripts to be run forcing numerous system reboots. This means Windows installs can take up to two hours. If your VM says "Pending," it is still installing. Do not try to unlock it or alter it during this stage. Once it goes to the "Off" state, simply start it and login to the console.

Console/Remote Dekstop

Windows VMs are setup with Remote Desktop by default and you can access it using the Administrator account/password using any Remote Desktop client. You can also login to your VM by using the web based console by going to Overview -> Console when the VM is on. If the Console has trouble connecting you may need to shutdown your VM and start it again (a simple reboot may not work). If the console shows a garbled screen, simply disconnect from the console and reconnect.

Windows 2008 R2 and 2012 Licensing

Licensing is done automatically, but if you are having licensing issues verify that you can ping "kms.steadfast.net" (if not verify network settings) and be sure time and time zone are correct. To activate, go Start -> Run -> "cmd" and run the following commands in the console:

cscript c:\windows\system32\slmgr.vbs /skms kms.steadfast.net:1688
cscript c:\windows\system32\slmgr.vbs /ato

Windows 2003 Licensing

For Windows 2003 and Windows 2003 R2 the license should already be installed, but Windows may ask you to, "Select the licensing mode you want to use." If it does ask, select to use the "Per Device or Per User" method. Then, it should finish configuring your Windows install and you should be good to go.

If you experience any issues with activation or installation, please feel free to contact support.

This article provides current support information for Spectre, Meltdown, and related side channel vulnerabilities on the Steadfast Public Cloud.

Here is a summary of the currently known issues:

The information contained in this article makes assumptions specific to Steadfast's Cloud environment and they are not necessarily applicable to other clouds

Current System Status

The following table describes the current issues and the fix status for each:

The status information indicated above may vary for customers with dedicated hypervisors based on maintenance arrangements.  We are continuously working to keep dedicated hypervisors in line with the public cloud.

Platform History

The current public cloud platform has been patched against Meltdown and Spectre variant 2 attacks.  The platform is based on CentOS 7 and Xen 4.8.  Prior to February 11, 2018, the platform was based on CentOS 6 and Xen 4.4.  Xen 4.4 will not receive updates to fix either Meltdown or Spectre. On January 17th, the first patches that mostly mitigate Meltdown were made available for Xen 4.8, but no patches are yet available to mitigate Spectre.  Our migration to Xen 4.8 was the culmination of several months of testing and planning, which we started prior to disclosure of the security issues. We did not modify our plans as a result, other than to increase the urgency of our roll-out.

On February 23, 2018, the first available Xen patches which mitigate variant 2 of Spectre became available, however the solution required a compiler update and microcode updates that were not yet available on CentOS.  We deployed the update with initial mitigation once the compiler update was available in April.

In early June 2018, we deployed CPU microcode updates once they were made available by Intel to mitigate Spectre variant 2 using CPU assistance.

In early August 2018, we deployed a version of Xen containing the Lazy FP fix as well as performance improvements related to the Meltdown fix.  The fix for Speculative Store Bypass and Spectre v3a was included, but not effective at that time.  We deployed the required microcode update to support the SSB fix in late September 2018.

Available information indicates no fix is required in Xen for Spectre variants 1, 1.1, and 1.2.

Available information indicates that SpectreRSB and NetSpectre are proof of concept attacks for which the Spectre v2 fixes provided mitigation.

There are two different fixes for L1TF depending on the VM operating mode.  A complete fix for Windows VMs required a microcode update (the same as the SSB microcode update) and to disable hyperthreading features of the CPUs.  A complete fix for Linux VMs requires only a Xen software update.  The necessary code updates for L1TF were deployed in late September.  Hyperthreading was turned off completely in the environment in early 2019 to fully mitigate risks to Windows VMs.

MDS mitigation requires a new CPU capability which requires microcode updates for all affected processors, hyperthreading must be disabled, and Xen and guest kernels must be updated to take advantage of the new capability.  We obtained the required microcode on June 19, 2019 and all hypervisors were fully patched by July 2, 2019.

SWAPGS mitigation requires no action on the hypervisor because Xen does not use the vulnerable CPU feature.

TAA mitigation requires no action on the hypervisor because the CPU does not support the vulnerable feature.

SRBDS does not affect the hypervisor because the CPU does not support the vulnerable feature.

Important: To fully take advantage of all available mitigations it is necessary to upgrade your VM to the latest available kernel or Windows update version, then fully shut down and boot up the VM.  This will ensure that the CPU capabilities provided by the new microcode are seen by the OS and used optimally.

Current Risks to VMs

Due to the way Xen is designed, Linux VMs do not need Meltdown kernel patches inside the VM.  They already cannot exploit the Meltdown issue using the normal Linux method.  However, Xen must be fixed to prevent users on Linux VMs from exploiting Xen directly to read the memory of other VMs on the same physical server.

Windows VMs cannot exploit the Xen variant of the Meltdown vulnerability, but Meltdown can be exploited inside a Windows VM to read its own memory from an unprivileged account unless the correct Windows Update package has been installed in the VM.  Linux VMs may be able to exploit the Xen issue to read the memory of unsuspecting Windows and Linux VMs even if they are patched.

Both types of VMs are vulnerable to Spectre, however the amount of exposure caused by Spectre is more limited and thus less of a danger overall than Meltdown. Some of the system updates available may mitigate part of Spectre, but the availability of patches varies wildly by operating system. Xen patches along with CPU microcode are required to provide complete mitigation for Spectre and other side channel issues.

Current Mitigation Options

The most complete mitigation available is running on patched hypervisors.  All public cloud VMs are currently on these hypervisors since February 11th and are being kept up to date with patches are soon as they are ready.

If you are running Linux, you should also apply kernel updates that provide fixes to Spectre and other Side Channel issues.  Updates fixing Meltdown and TAA are not required. Please note that working kernels are now available for all CentOS VMs that fix all variants up to and including MDS:

• It is safe to upgrade the kernel for CentOS 7 VMs to kernel-plus version 3.10.0-957.12.2.el7.centos.plus or newer.
• It is safe to upgrade the kernel for CentOS 6 VMs to kernel version 2.6.32-696.20.1.el6 which includes limited Spectre fixes.
• It is NOT safe to upgrade CentOS 6 VMs to kernel version 2.6.32-754.14.2.el6 or newer unless you add "eagerfpu=off" to the kernel command line.
  • This issue is caused by a bug with the Lazy FP fix inside the CentOS 6 kernel, so this fix must be turned off.
  • If you set "eagerfpu=off" it is safe to upgrade to 2.6.32-754.14.2.el6, which includes all other fixes through MDS.
  • If you feel that Eager FPU is an important fix, we suggest planning to upgrade to CentOS 7, or switching to kernel 4.4 provided by ELRepo.
• Kernel updates are not needed to address SWAPGS when running under Xen.

If you are running Windows, you should apply Windows updates to patch issues up to and including SWAPGS:

• Windows Server 2019: Build 17763.615 (KB4507469)
• Windows Server 2016: Build 14393.3085 (KB4507460)
• Windows Server 2012 R2: KB4507457 (Security only) or KB4507448 (Full update)
• Windows Server 2008 R2: KB4507456 (Security only) or KB4507449 (Full update)

Important: After applying updates you should fully shut down your VM, then start it back up from the control panel.  This ensures the VM boots up with awareness of microcode features that enable all fixes to be effective.

If you trust your VM users, you may also want to consider a dedicated hypervisor, which guarantees all the VMs on the physical server are under your control.  This option does not itself prevent side channel vulnerabilities from being exploited, but limits the number of people who may have access to the server memory.  Keep in mind that if you operate applications that face untrusted users, an exploitable vulnerability in such an application could allow a user to run their own code and subsequently exploit Meltdown, Spectre, or other side channel vulnerabilities.  If you'd like to explore the option of switching to private hypervisors, please contact our sales team for further information.

References

• Steadfast Meltdown and Spectre Advisory
• Xen Security Advisory XSA-254
• Xen Security Advisory XSA-263
• Xen Security Advisory XSA-267
• Xen Security Advisory XSA-273
• Xen Security Advisory XSA-289
• Xen Security Advisory XSA-297
• Xen Security Advisory XSA-305
• Xen Security Advisory XSA-320

If you need any help or have any questions regarding any of this information, please contact our support team.

Updates

June 9, 2020 @ 5:45 PM

• Updated SRBDS information to indicate no vulnerability exists on the environment

June 9, 2020 @ 12:50 PM

• Added initial information for Special Register Buffer Data Sampling

November 11, 2019 @ 3:15 PM

• Added initial information for TSX Asynchronous Abort

August 7, 2019 @ 12:30 PM

• Added initial information for SWAPGS

July 2, 2019 @ 5:30 PM

• Updated MDS information to indicate mitigation is fully deployed to the public cloud environment

June 25, 2019 @ 11:40 AM

• Updated MDS information to indicate that microcode is now being deployed
• Add notes to indicate VMs should be shut down and then started up manually to ensure microcode features are detected by VMs

June 19, 2019 @ 4:30 PM

• Updated MDS information to indicate that microcode is now in testing
• Removed outdated information related to private hypervisors

May 20, 2019 @ 3:05 PM

• Added initial information for MDS (Microarchitectural Data Sampling)
• Updated Spectre v1 information to clarify there are no known hypervisor vulnerabilities
• Added Spectre v3a information
• Added clarifications of dates now that more than a year has passed
• Added additional details related to Intel Advisories
• Minor wording corrections and clarifications

September 24, 2018 @ 5:15 PM

• Converted the information to tables and cleaned up the details
• Updated the status to indicate that L1TF and SSB mitigation is now available on public cloud hypervisors

September 14, 2018 @ 2:30 PM

• Corrected the status of SSB mitigation to clarify that the hypervisor is not vulnerable, but mitigation still is needed for VMs

August 30, 2018 @ 3:55 PM

• Corrected status of L1TF and SSB to indicate full mitigation including CPU microcode is now in testing

August 30, 2018 @ 11:30 AM

• Added information about L1 Terminal Fault
• Updated mitigation status to indicate that Lazy FP is now mitigated and that Spectre 1.1 does not affect the environment
• Added notes regarding Meltdown performance improvements
• Updated patching version information for Windows and Linux VMs

August 1, 2018 @ 11:05 AM

• Added information about NetSpectre and SpectreRSB

July 17, 2018 @ 5:15 PM

• Added information related to Speculative Store Bypass, Lazy FPU, and Spectre variants 1.1 and 1.2
• Updated status information for public cloud hypervisors to indicate that Spectre variant 2 is mitigated
• Removed the meltdown patch information that is outdated
• Added a warning about the Lazy FPU bug preventing new CentOS 6 kernels from booting

March 13, 2018 @ 3:00 PM

• Updated to include CentOS 7 safe kernel-plus package from the official distribution instead of Steadfast-released version.
• Added a note about the newly available Xen SP2 Spectre patch.

February 12, 2018 @ 10:45 AM

• Updated to indicate that public cloud VMs are now running on Meltdown-patched hypervisors.

February 7, 2018 @ 6:35 PM

• Updated to indicate that both CentOS 7 and CentOS 6 users can upgrade to working kernels

January 26, 2018 @ 3:35 PM

• Updated to indicate that only CentOS 7 users need to avoid kernel updates now that CentOS 6 kernel 2.6.32-696.20.1.el6 is available

January 22, 2018 @ 1:45 PM

• Updated references to testing of Meltdown patch to indicate testing is complete.

January 18, 2018 @ 6:30 PM

• Rewrote the "Current Risks to VMs" section to be clearer and less redundant
• Clarified that applications that face untrusted users should be considered during risk analysis
• Added some clarifications around dedicated hypervisors
• Added a clarification about the optional status of the patched hypervisors and when the migration will become mandatory'

Installing/Using the CloudFlare cPanel Module

1. Sign-up for a CloudFlare Account: https://www.cloudflare.com/sign-up.html
2. Follow the up-to-date install instructions here: https://www.cloudflare.com/wiki/Cpanel
3. Once it is installed, you can simply use Cloudflare using the Cloudflare icon in the Advanced section of cPanel.

For additional help with CloudFlare you can access their documentation and helpdesk here: http://www.cloudflare.com/help.html

What is CloudFlare?

CloudFlare is a system that acts as a proxy between your visitors and our server. By acting as a proxy, CloudFlare caches static content for your site, which lowers the number of requests to our servers, but still allows visitors to access your site. This automated installer for CloudFlare allows you to setup basic cloudflare protection. The installer is still in beta. There is a risk that it will cause a redirect loop or negatively impact your site. We recommend preforming the installation during low traffic periods.

Advantages of the CloudFlare system:

• Site Performance Improvement: CloudFlare has proxy servers located throughout the world. Proxy servers are located closer to your visitors, which means they will likely see page load speed improvements as the cached content is delivered from the closest caching box instead of directly off our server. There is a lot of research that shows that a faster a site, the longer a visitor stays.
• Bot and Threat Protection: CloudFlare uses data from Project Honey Pot and other third party sources, as well as the data from its community, to identify malicious threats online and stop the attacks before they even get to your site. You can see which threats are being stopped through your CloudFlare dashboard here.
• Spam Comments Protection: CloudFlare leverages data from third party resources to reduce the number of spam comments on your site
• Alerting Visitors of Infected Computers: CloudFlare alerts human visitors that have an infected computer that they need to take action to clean up the malware or virus on their machine. The visitor can enter a CAPTCHA to gain access to your site.
• Offline Browsing Mode: In the event that our server is unavailable, visitors should still be able to access your site since CloudFlare serves the visitor a page from its cache.
• Lower CPU Usage: As fewer requests hit our server, this lowers the overall CPU usage of your account.
• New Site Stats: You have good tools to evaluate human traffic coming to your site, but no insight into search engine crawlers and threats. With CloudFlare, now you do.

There are some limitations of the CloudFlare system:

• Currently, requests must be directed to www.yourdomain.tld instead of domain.tld (which means you may need to make some configuration changes: WordPress installations are automatically adjusted).
• CloudFlare may affect internal statistic programs that read directly from Apache logs. (CloudFlare will not affect web-based analytic programs that use JavaScript like Google Analytics.)
• While your logs will reflect fewer requests to your server and therefore lower load, the experience to your visitors should be unaffected.
• CloudFlare caches static content from your site. While this reduces the load on your server, it means that if you make a change to an existing static file, like an image, there may be a delay before the change appears. While you are updating your site, you can put CloudFlare in .Development Mode. so changes appear immediately.
• CloudFlare's basic mode cannot handle SSL certificates. If you need to use an SSL certificate, that part of your site needs to be on a subdomain that is not protected.

With our Windows Dedicated Servers most clients manage things over remote desktop protocol (RDP) and we're often asked about using multiple monitors with RDP.

Default settings for connecting to a remote server are typically fine for most users, but those who require multiple monitors for their sessions, such as traders or system administrators, may need to configure RDP to use multiple monitors in their remote sessions.

Reconfiguring remote desktop protocol (RDP) for this is simple and can be done in one of two ways.

1. The first method is directly through the RDP interface. Open the Remote Desktop and click the "Options" button on the bottom left-hand corner of the window. Click on the "Display" tab and tick the checkbox that reads "Use all my monitors for the remote session" Once this is selected, you can then click "Connect" and proceed with connecting to the server as normal. If you would like this to be the default behavior for RDP, click on the "General" tab and click "Save" before connecting to your remote server.
2. Alternatively, you can launch RDP from the command line and specify the multimon flag:
   
   mstsc.exe -multimon
   
   Launching RDP in this manner will auto-check the "Use all my monitors for the remote session" box and allow you to bypass the previous steps.

Support for multiple monitors is available when connecting from any Windows 7/8.1/10 computer, however, there are restrictions when connecting to a computer using multi-monitor mode. When connecting to Windows 7 computers, only computers that are running Windows 7 Enterprise or Ultimate can be connected to in multi-monitor mode. When connecting to Windows 8.1, only computers that are running Windows 8.1 Professional or Enterprise can be connected to in multi-monitor mode. Both Standard and Datacenter editions of Windows Server 2008, Windows Server 2012, & Windows Server 2016 support multi-monitor mode.

Multi-monitor mode supports up to 16 monitors, with a maximum resolution of 4096 x 2048 per monitor.

Newly set up Windows servers can be connected using Remote Desktop Connection. This transmits your desktop across the Internet, allowing connection from a remote location.

Connecting from Windows XP

If you are not running the most recent version (7.0) of the Remote Desktop Client, you can download it from the following link: http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=20609 Note that using an out-of-date version of the Remote Desktop Client will cause issues when connecting to a Windows Server 2008 R2 or 2012 R2 system.

If you get the following message:

"The remote computer requires Network Level Authentication, which your computer does not support. For assistance, contact your system administrator or technical support."

you will need to both upgrade to the newest release of the Remote Desktop Client, and either:

• follow the steps outlined in the following knowledge base article: http://support.microsoft.com/kb/951608/
• Turn off Network Level Authentication in the Remote Desktop settings on your server. (If you need assistance with this, please contact Support)

To connect:

1. Click Start, then All Programs, then Accessories, and finally Remote Desktop Connection.
2. In the Remote Desktop Connection dialog box, enter your server's Main IP address in the Computer: field
3. Click Connect
4. When prompted, enter your Username ("Administrator" by default) and password.

You are now connected to the remote server.

To disconnect:

1. Click Start, and then Log Off

Connecting from Windows Vista or Windows 7

Both Windows Vista and Windows 7 should already have an up-to-date Remote Desktop client installed that can connect to Windows 2008 R2 or 2012 R2 systems.

To connect:

1. Click the Windows logo in the lower left corner of your screen, then All Programs, then Accessories, and finally Remote Desktop Connection.
2. In the Remote Desktop Connection dialog box, enter your server's Main IP address in the Computer: field
3. Click Connect
4. When prompted, enter your Username ("Administrator" by default) and password.
5. If you get a warning that says "The identity of the remote computer cannot be verified", check the "Don't ask me again for this computer" box, and click Yes

You are now connected to the remote server.

To disconnect:

1. Click Start, and then Log Off

Connecting from Windows 8.1

Both Windows 8.1 should already have an up-to-date Remote Desktop client installed that can connect to Windows 2008 R2 or 2012 R2 systems.

To connect:

1. Click the Windows logo in the lower left corner of your screen, type in Remote and click on Remote Desktop Connection. You can also right-click on the icon and pin it to either your start menu or taskbar for easier access moving forward
2. In the Remote Desktop Connection dialog box, enter your server's Main IP address in the Computer: field
3. Click Connect
4. When prompted, enter your Username ("Administrator" by default) and password.
5. If you get a warning that says "The identity of the remote computer cannot be verified", check the "Don't ask me again for this computer" box, and click Yes

You are now connected to the remote server.

To disconnect:

1. Click Start, and then Log Off

Connecting from Mac OS X

Install the Remote Desktop Client for OS X

To connect:

1. Click the Finder icon in the dock, open Applications, and then double-click on Remote Desktop Connection
2. In the Remote Desktop Connection dialog box, enter your server's Main IP address in the Computer: field
3. Click Connect
4. When prompted, enter your Username ("Administrator" by default) and password.

You are now connected to the remote server.

To disconnect:

1. Click Start, and then Log Off

With the Steadfast Cloud Platform you can set firewall rules for the network interfaces of virtual machines. There are two types of firewall rules:

• ACCEPT defines the types of packets which will be accepted by a firewall

• DROP specifies the packets which will be rejected by a firewall

Important: In some cases, rebooting a VM from the console instead of the control panel may cause inconsistent firewall behavior.  To workaround this issue, if you experience it, we recommend one of the following actions:

• Configure a software firewall on the inside of your VM at the OS level instead of using the control panel firewall settings

• Always reboot your VM using the control panel, rather than rebooting it from the console.

Configuring a Rule

1. Go to your Control Panel’s Virtual Machines menu.

2. Click the label of the machine to which you want to configure a firewall rule.

3. Mouse over the “Networking” menu and click on the “Firewall” item.

4. On the page that appears set the following:

   • Choose the network interface.

   • Specify if the rule should be commanded to such that requests are  accepted or dropped.

   • Set the source IP address this rule will apply to.

   • Set the destination port this rule will apply to.

   • Choose the protocol (TCP or UDP).

5. Click the “Add Rule” button to save the rule.

Example:

• The "Int1 ACCEPT 122.158.111.21 22 TCP" firewall rule means that the Int1 network interface will accept all the requests and packets addressed from a 122.158.111.21 IP using TCP protocol on a 22 port.

• The "Int2 DROP 122.158.111.21 22 UDP" firewall rule means that the Int2 network interface will reject all the requests and packets running from a 122.158.111.21 IP using TCP protocol on a 22 port.

Setting Rule Priorities

Since some rules can override each other, it is important to set the order in which they are processed. Use the up/down arrows next to a rule to set priority.

Editing or Deleting an Existing Rule

1. Go to your Control Panel’s Virtual Machines menu.

2. Click the label of the machine to which you want to configure a firewall rule.

3. Mouse over the “Networking” menu and click on the “Firewall” item.

4. On the page that appears you’ll see the list of all the rules:

   • Click the Edit icon next to a rule to edit its parameters.

   • To delete a rule from a VM, click the Delete icon next to a rule and confirm.

Setting Up Default Rules

OnApp allows setting default firewall rules which will be applied to all IP addresses with all ports for all protocols. To set default firewall rules:

1. Go to your Control Panel’s Virtual Machines menu.

2. Click the label of the machine to which you want to configure a firewall rule.

3. Mouse over the “Networking” menu and click on the “Firewall” item.

4. On the page that appears scroll down to the Default Firewall Rules section and set the following:

   • Choose the interface for which you want to define a rule.

   • Define a rule (Accept or Drop).

For example, if you choose the network interface Int1 with the ACCEPT rule, it means that firewall set to Int1 network interface will accept all the requests and packet types from all IPs. If you define the Int2 network interface with the DROP rule, the firewall of the Int2 will block all the requests and packets addressed to it.

Administrators in OnApp have full control over the lifecycle of Virtual machines. Virtual Machines can be in the following states:

• Created: A machine is created when you successfully "Add New Virtual Machine" from the Virtual Machines menu, having selected its template and set its properties, resources and network requirements.
• Build: A Virtual Machine must be built after it is created. Building is the process of actually allocating the physical resources specified during its creation. This can be done manually, or automatically if you check the "Build virtual machine automatically" box during the creation process.
• Powered on: A power on starts the virtual machine, its operating system and processes. You have to manually start a virtual machine after it has been created and built. To do this, click the "Power On" button for that machine on the Virtual Machines menu, or the details screen for that machine.
• Powered off: A power off will attempt to gracefully shut down a virtual machine and its operating system, which typically involves terminating all running applications. If the operating system cannot be stopped, it will be forcefully stopped. To power off a machine, click its "Power On" button on the Virtual Machines menu, or the details screen for that machine.
• Shut down: The VM has been forcefully stopped.
• Rebooted: Rebooted means a virtual machine has been powered off, and then powered on again.
• Deleted: When a virtual machine is deleted its contents and any backups stored are permanently removed unless you choose to convert those backups to templates first. By default, users can delete their own virtual machines. Administrators can delete any virtual machine. You can delete a machine using the "Delete Virtual Machine" link on its details screen.
• Re-built: To rebuild a virtual machine means to reinstall the template and reconfigure the resources and network. All data will be lost.
• Failed: A failed virtual machine is one that is down, for example because of hardware or network problems. You will have to start the machine manually when those problems have been solved.

This article is a reproduction of a standard Steadfast Networks welcome email sent to new cloud platform customers. If you have lost your welcome email, you can use this information as a reference for service features and add-on services you are eligible to request.

NOTE: Please read this email message completely, as it contains important information about your new service with Steadfast Networks. This email, along with a significant amount of useful documentation is located in the Knowledgebase at http://steadfast.net/support/kb/c15

== Virtual Machine Management Details ==

Your account has been set up and is now accessible with the following information:

URL: https://vm.steadfast.net
Username: [Client ID]
Password: Must be set via https://manage.steadfast.net -> Billing & Services -> View Services -> [Service ID] -> Cloud Management

Note: We apologize for any inconvenience in not setting the password, but for security reasons we do not send usernames and passwords in the same email.

Your account is connected to our internal network, which allows you to communicate with other servers that have this feature privately and unmetered. If you have a backup account with us, you can connect to the backup server using the IP 10.2.255.10 or hostname int.shell01.backup.steadfast.net over the internal network to avoid being charged for bandwidth used to make backups.

Note: You will need to follow the directions at http://steadfast.net/support/kb/67 when setting up your internal network.

If you would like access to Internal Network via VPN, please contact support@steadfast.net to request a VPN key.  The VPN is supported on Windows Vista and later, MacOS X 10.4 and later, Linux, and FreeBSD.  VPN access is also available on devices powered by iOS 6 and later or Android 4.0 and later.

When setting up your first Virtual machine please use this Knowledgebase article as a guide: http://steadfast.net/support/kb/74

== Account Details ==

You can access our billing and account management interface at the following URL (or you can log in from the login box at the top of any page of our main site) using the information previously sent to you.  Your client ID is your login.

URL: https://manage.steadfast.net
Client ID: [Client ID]
Password: Use "retrieve password" link at https://manage.steadfast.net to retrieve

== Additional IP Addresses ==

Your account is allowed up to 64 IPs without pre-approval.  Be careful in the assignment of these IPs as you cannot remove an IP from your account without contacting ips@steadfast.net.  Additional IPs may be requested by sending an email to ips@steadfast.net and must be properly justified according to ARIN rules.

If you would like us change the reverse DNS entries for your IPs, please send an email to dns@steadfast.net.

== Backup Service ==

It is possible to take manual on-demand backups using our cloud control panel.  For automatic backups, please contact us for more information.   To request backup space, please contact sales@steadfast.net.  For details on pricing and plans, please see https://www.steadfast.net/backup-disaster-recovery/disaster-recovery-it-infrastructure

== Enterprise Spam Filtering ==

We offer access to an Enterprise Spam Filtering appliance to any customers that wish to use it.  This service is $5 per month per domain.  If you would like to have this service activated for your account, please send an email to sales@steadfast.net listing the domains you would like to set up.

== Announcements and Maintenance ==

As many of our customers have expressed that they do not want to receive service notices to the primary contacts within their accounts, we do not notify customers of most routine maintenance and service changes via email.  If you would like to be notified when we post an announcement, please visit the following link and enter your email address in the "Subscribe" box on the right side of the page:

https://support.steadfast.net/News/List

You can also click on the "XML" link in the "Subscribe" box to access an RSS feed of announcements which you can subscribe to in your favorite RSS reader.  We also provide links to recent announcements and company blog posts at the bottom of our front page if you prefer to check manually.

== Requesting Technical Support ==

For someone to request support on one of your systems, the email address, name, and phone number MUST be listed as one of the contacts on your account. Additional authorized contacts can be added by accessing our management system and clicking "Client Profile," then "View Profile," and then clicking edit at the top of the "Authorized Contacts" section. This is required for security purposes.

If you have any further support questions, please visit our help desk at https://support.steadfast.net, or email support@steadfast.net.  Please be sure to list your server's primary IP address and as much detail as possible about your problem when requesting support to ensure faster service.

== Updating Account Information ==

The name and email address used in https://vm.steadfast.net will always match the information used in https://manage.steadfast.net, so you do not need to manually update this in both locations.  This information can only be edited at https://manage.steadfast.net.  Please keep this information up to date.

== Virtual Machine Management Password Reset ==

If you ever forget the password for your Virtual Machine Management account (https://vm.steadfast.net) you can reset it via https://manage.steadfast.net by going to Billing & Services -> View Services -> [Service ID] -> Cloud Management.  Any users on your account with view or edit permissions to "Services" can reset the password.

Thank you for your business!