Configure NTP on a VM

Many software systems that depend on careful sequencing of events rely on a stable, consistent system clock. These systems often use system logs with timestamps to ensure time synchronization and debug issues as they occur. To help keep system clocks in sync, Compute Engine instances are preconfigured to use network time protocol (NTP), a bundled solution of time synchronization hardware and software.

NTP is also helpful in the rare case of a leap second, a one-second adjustment made to UTC time to account for changes in the Earth's rotation. Leap seconds don't happen at routine intervals, because the Earth's rotation speed varies irregularly in response to climatic and geological events. Previous leap seconds have noticeably impacted a variety of services and applications on the web. NTP servers help ensure that all servers report the same time during the event of a leap second.

This document describes how to configure NTP servers on your compute instances, including proper handling of leap seconds.

Google NTP servers and leap smearing

Leap seconds for a Unix operating system are commonly implemented by repeating the last second of the day. This can cause problems with software that expects timestamps to only ever increase. To get around this problem, the time servers at Google Cloud "smear" the extra second over 24 hours—12 before and 12 after the leap second event—so that computers don't see the extra second all at once as a repeated timestamp. This reduces risk in systems that depend on a consistent timestamp. We recommend that you configure all of your compute instances to use the internal Google NTP services.

Configure NTP for your instances

Google Cloud can't predict how external NTP services, such as pool.ntp.org, handle leap seconds. We strongly recommend that you don't use external NTP sources with your compute instances. If you use both Google's NTP service and an external service, you can cause unpredictable changes in the system time. Using only a single external NTP source is preferable to using a mix, but external NTP services, such as pool.ntp.org, often use stepping to handle leap seconds. As a result, your instances might see repeated timestamps.

The safest approach is to configure your compute instances to use a single NTP server—the internal NTP server provided by Google. For more information about the internal NTP server provided by Google, see the Google Public NTP FAQ.

To ensure that your compute instances are correctly configured, follow these instructions.

Linux (chrony)

By default, the majority of new Linux releases use chrony to manage their NTP settings and time synchronization. To make sure that chrony uses only the internal NTP service, check the chrony configuration and remove external NTP servers.

  1. Use ssh to connect to your compute instance.

    Console

    To use the console to connect to the compute instance using SSH, follow these steps:

    1. Go to the VM instances page in the Google Cloud console.

      Go to VM instances

    2. Click the SSH button for the instance that you want to configure.

      SSH button.

    gcloud

    To use the Google Cloud CLI to connect to the compute instance using SSH, run the following command:

     gcloud compute instances ssh INSTANCE_NAME 

    Replace INSTANCE_NAME with the name of the instance that you're connecting to.

  2. On your instance, run chronyc sources to check the current state of your NTP configuration:

     $ chronyc sources 

    The output looks similar to the following:

     210 Number of sources = 2 MS Name/IP address Stratum Poll Reach LastRx Last sample =============================================================================== ^* metadata.google.internal 2 6 377 4 -14us[ -28us] +/- 257us ^- 38.229.53.9 2 6 37 4 -283us[ -297us] +/- 28ms 

    If you see a single record pointing at metadata.google or metadata.google.internal, you don't need to make any changes. If you see multiple sources, mixed between metadata.google and a public source such as pool.ntp.org, update your sources to remove any external NTP servers.

    In the example output, there are two records, one pointing at metadata.google.internal and another pointing to an external address. Because there are multiple sources, you would update your NTP servers to remove the 38.229.53.9 address, as described in the next step.

  3. Configure your NTP servers to remove external NTP servers.

    To remove the additional NTP server from the list, edit the /etc/chrony/chrony.conf file using your favorite text editor. Find all the lines that start with server external_source_ip_or_name and remove them.

    After editing your /etc/chrony/chrony.conf file, restart the chrony service. The command to restart might vary depending on the Linux distribution, as shown in the following examples:

     sudo service chrony restart 
     sudo systemctl restart chrony 
  4. Verify your configuration by running the chronyc sources command again:

     $ chronyc sources 

    The output should look similar to the following:

     210 Number of sources = 1 MS Name/IP address Stratum Poll Reach LastRx Last sample =============================================================================== ^* metadata.google.internal 2 7 377 98 -1343ns[-1588ns] +/- 396us 

Linux (ntpd)

Most older Linux distributions use ntpd to manage their NTP settings and time synchronization. To make sure that ntpd uses only the internal NTP service, check the ntpd configuration and remove external NTP servers.

  1. Use ssh to connect to your compute instance.

    Console

    To use the console to connect to the compute instance using SSH, follow these steps:

    1. Go to the VM instances page in the Google Cloud console.

      Go to VM instances

    2. Click the SSH button for the instance that you want to configure.

      SSH button.

    gcloud

    To use the Google Cloud CLI to connect to the compute instance using SSH, run the following command:

     gcloud compute instances ssh INSTANCE_NAME 

    Replace INSTANCE_NAME with the name of the instance that you're connecting to.

  2. On your instance, run ntpq -p to check the current state of your NTP configuration:

     $ ntpq -p 

    The output looks similar to following:

     remote refid st t when poll reach delay offset jitter ============================================================================== *metadata.google 255.28.23.83 2 u 27 64 1 0.634 -2.537 2.285 *217.162.232.173 130.149.17.8 2 u 191 1024 176 79.245 3.589 27.454 

    If you see a single record pointing at metadata.google or metadata.google.internal, you don't need to make any changes. If you see multiple sources, mixed between metadata.google and a public source such as pool.ntp.org, you must update your sources to remove any external NTP servers.

    In the example output, there are two records, one pointing at metadata.google and another pointing to an external address. Because there are multiple sources, you would need to update your NTP servers to remove the *217.162.232.173 address, as described in the next step.

  3. Configure your NTP servers to remove external sources.

    To configure your NTP servers, edit the /etc/ntp.conf file using your favorite text editor. Find the servers section of the configuration, and remove all non-Google NTP sources, for example:

    vim /etc/ntp.conf 
    # You do need to talk to an NTP server or two (or three). #server ntp.your-provider.example ... server metadata.google.internal iburst 

    After editing your /etc/ntp.conf file, restart the NTP service. The command to restart might vary based on the Linux distribution:

    sudo service ntp reload 
  4. Verify your configuration by running the ntpq -p command again:

    ntpq -p 
     remote refid st t when poll reach delay offset jitter ============================================================================== *metadata.google 255.28.23.83 2 u 27 64 1 0.634 -2.537 2.285 

Windows

  1. Go to the VM instances page in the Google Cloud console.

    Go to VM instances

  2. Click the RDP button next to the Windows instance that you want to connect to.

    SSH button.

  3. After you're logged in, right-click on the PowerShell icon and select Run as administrator.

    The PowerShell icon.

  4. When the command-prompt loads, run the following command to see the current NTP configuration:

    w32tm /query /configuration 
     [Configuration] ... Type: NTP (Local) NtpServer: metadata.google.internal, ... 

    If you see a single record pointing at metadata.google or metadata.google.internal, you don't need to make any changes. If you see multiple sources, mixed between metadata.google and a public source, you need to remove the external server. Follow the Windows' guide for configuring your NTP server.

  5. To ensure the most widespread software compatibility on Windows VMs, Google recommends that you use the gVNIC driver to ensure sub-millisecond NTP accuracy with metadata.google.

    If you must use VirtIO with your Windows VM, to get sub-millisecond accuracy with NTP servers, Google recommends that you don't use the Windows Time Service (stop and unregister w32tm).

    1. Stop the Windows Time Service:

      net stop w32time 
    2. Remove the Windows Time Service from the registry:

      w32tm /unregister 
    3. After the Windows Time Service has been stopped and removed from the registry, install the Meinberg NTP client.

      Follow the configuration instructions as provided in the Meinberg documentation.

    4. Configure the NTP server for the Meinberg NTP client as metadata.google.internal.

      After you finish configuring NTP, wait between five and 15 minutes for the system clock in the VM to stabilize with the NTP server.

      For information about why we don't recommend using w32tm, see the Known Issues documentation.

Use leap smearing with systems external to Google Cloud

The leap smearing feature of Google's NTP servers is a convenient way to manage the risk involved with replaying a second on time sensitive systems. Other NTP services may provide an acceptable workaround for most software systems. However, it's important that you don't mix Google leap smearing NTP services with public NTP stepping services.

To synchronize devices outside Google Cloud to smeared time, you can use Google Public NTP for those devices. The Google Public NTP uses the same leap second smearing that's provided to Compute Engine VMs.

What's next