$Id: README 1.40 2023/12/19 10:23:33 martin.burnicki REL_M $ This is the README file for mbgtools-lx-4.2.26 ---------------------------------------------- Please send comments and required modifications to Meinberg support Contents -------- 1. Notes and Description 1.1 Supported Devices 1.2 Supported Linux Versions 1.3 Known Limitations 2. Driver Files and Programs 3. Installation 3.1 Unpacking the Sources 3.2 Checking the Kernel Build Environment 3.3 Compiling the Driver 3.4 Installing the Driver 3.5 Loading the Kernel Module 3.6 If the Kernel Module Fails to be Loaded ... 3.7 Compiling for a Different Target Kernel 3.8 Compiling for a Completely Different Kernel Source Tree 3.9 Staged Builds 4. Using the Driver With ntpd 4.1 Configuration With ntpd's Shared Memory Driver 4.2 Configuration With ntpd's Parse Driver 5. Using the Driver With chrony 6. Notes for SuSE/openSUSE Linux 6.1 Old SuSE Naming Conventions 6.2 NTP and "Operation not permitted" Under SuSE Linux 10.1 and Newer 6.3 Error "module is unsupported" 7. "Permission denied" With SELinux (e.g. RedHat/Fedora/CentOS) 1. Notes and Description ------------------------ This driver package allows PCI and ISA cards as well as USB bus-level devices manufactured by Meinberg to be operated under the Linux operating system. Beside the possibility to configure or monitor the devices using the included user space programs, the driver package also enables the devices to be used as reference time source for an existing NTP daemon (ntpd or chronyd). In addition, some shared object libraries provide API functions that third-party applications can use to access the devices directly. Several of the supported devices can be used in parallel, and up to 4 devices can be used at the same time as reference time sources for the NTP daemon (this is a limitation of the NTP daemon). If a USB device is to be used as a reference time source for the NTP daemon, the device must already be connected when ntpd starts. Otherwise the daemon will not be able to detect the device. 1.1 Supported Devices --------------------- The driver package supports all PCI and ISA cards as well as all Meinberg USB devices that were introduced prior to the release of the package. Meinberg thoroughly takes care that API calls are kept compatible across old and new devices, as well as across old and new versions of the driver packages. Meinberg takes great care to ensure that API functions on old and new devices as well as old and new versions of the driver software remain compatible. 1.2 Supported Linux Versions ---------------------------- Whether this driver can be used on a given Linux distribution and version depends mainly on the kernel version which comes with a given Linux distribution, not on the specific type of distribution, e.g. SuSE/openSUSE, Debian/Ubuntu, RedHat/Fedora/CentOS, etc. This version of the driver package should compile fine with Linux kernels 2.6.x, 3.x, 4.x, 5.x and 6.x up to at least 6.6 - on standard PCs (i386 architecture) - on Intel/AMD 64 Bit systems (x86_64 architecture) - on SPARC64 architecture - partially on IA64 (Itanium) architecture - partially on ARM architecture Linux kernels 2.4.x and older are not supported anymore. 1.3 Known Limitations --------------------- Loading the kernel module may "taint" the kernel. This is because the kernel driver isn't shipped with and hasn't been signed by the Linux distribution. This is reported in the syslog messages, but doesn't cause any limitations. When the command "make install" is executed, some Linux distributions may display an error message like: sign-file: certs/signing_key.pem: No such file or directory There is no way to avoid this because obviously a self-compiled kernel module can not be signed with the certificate of the maintainers of the Linux distribution. 2. Driver Files and Programs ---------------------------- "mbgtools for Linux" is based on Meinberg's common driver library "mbglib" and implements a couple of user space programs, shared object libraries that can be used by 3rd programs, and a loadable kernel driver module that actually accesses the hardware. The source code of each programs can be found in a subdirectory with an associated name. All user space programs can be called with parameter -? or -h to show some usage information. mbgstatus This program prints some status information for a device. The kind of information to be printed depends on the specific type of the card. One or more parameters '-v' increase verbosity. mbgctrl This program can be used to do some basic configuration on a device. mbgirigcfg This program can be used to check and configure IRIG settings of devices that provide an IRIG timecode input or output. mbgsetsystime This program reads the time from a device and sets the system time once. If NTP is used on a system which has an unreliable clock, this program can be used to set the system time initially, before the NTP daemon starts. The program should not be run if an NTP daemon is already active and adjusts the system time. Doing so would simply mess up the time synchronization done by the NTP daemon. mbgshowsignal This program displays the modulation signal (i.e. the second marks) received by DCF77 longwave receivers, if the device supports this. mbggpscap This example program reads time capture events from the time capture FIFO buffer of a device. This works only with cards that provide time capture inputs, and the DIP switches on those cards must have been set up properly to enable time capturing. mbghrtime This example program reads high-resolution time stamps including status information from a device, if the device supports this. mbgfasttstamp This examle program demonstrates how to read high resolution time stamps from a device in a very much faster way than mbghrtime does. However, this works only with devices that support memory mapped I/O. Current PCI Express cards support this feature, but older PCI cards may not, and USB devices don't, either. mbgxhrtime This example program also shows how to get time stamps faster than shown in mbghrtime. This is not as easy as mbgfasttstamp but can be used with every card which can be used with mbghrtime, even if the card does not support memory mapped I/O. The program starts a polling thread which reads a high resolution time stamp from the device, plus an associated cycles count from the OS. This is done once per second, and the current cyles count is then used to extrapolate the time between calls. A limitation of this approach is that the cycles count is fetched from the time stamp counter of the CPU. This may not be very reliable ond older CPU types, because older CPUs may return different cycles count on different CPU cores. So the program takes care that it is always executed always on the same CPU core. Also, power saving mechanisms (Intel Speedstep, or AMD Cool'n'Quiet) should be disabled for older CPU types to prevent the extrapolated time stamps from being messed up due to CPU clock changes. mbgcmptime This program compares the times provided by different devices, and shows the computed difference. The cycles counter is used to compensate the latency that is introduced because the 2 devices can only be accessed one after the other. mbgtcrcal This program can be used with Time Code Reader (TCR) devices to configure a compensation value for the propagation delay of the incoming time code signal. mbgdevio, mbgutil Shared object libraries providing API functions that can also be used by 3rd party programs to directly access a device. For details, see: https://kb.meinbergglobal.com/kb/driver_software/meinberg_sdks/start mbgsvcd The Meinberg Service Daemon. It needs to be started to feed the time stamps from the device(s) to the shared memory driver of the NTP daemon. This yields better accuracy than the older, interrupt driven approach. See chapter 4.1. for details. mbgclock.ko A kernel module which implements the device driver. 3. Installation --------------- The driver package can be cloned or updated from Meinberg's read-only git repository at https://git.meinbergglobal.com/drivers/mbgtools-lx.git/ Alternatively, specific versions can be downloaded as archive from the git repo above, and the current the current release version is available via the Meinberg software download page at https://www.meinbergglobal.com/english/sw/#linux A tar.gz archive is also available on the USB thumb drive shipped with the devices. 3.1 Unpacking the Sources ------------------------- If the source code is available as .tar.gz archive, unpack the archive file using the command tar xvzf mbgtools-lx*.tar.gz After the archive file has been unpacked, 'cd' to the base directory that has been created corresponding to the name and version of the package. To clone the driver package from the Meinberg readonly git repo, run the command git clone https://git.meinbergglobal.com/drivers/mbgtools-lx.git which creates a directory mbgtools-lx, to which you should 'cd'. Afterwards, you can run git pull to upgrade the package to the latest version. Also you can checkout other branches or specific versions as usual with git. 3.2 Checking the Kernel Build Environment ----------------------------------------- Because the driver module is linked into the kernel, it is VERY important that the module is compiled using configuration and version information that matches the target kernel, which is usually the running kernel. This is the reason why the kernel sources and/or headers must have been installed. Under Debian/Ubuntu, the kernel headers can be installed using the following commands: sudo apt-get install linux-headers-$(uname -r) sudo apt-get build-dep linux-headers-$(uname -r) -or-, if the commands above are not supported: sudo apt-get install linux-headers-generic sudo apt-get build-dep linux-headers-generic Under Fedora/Redhat/CentOS, the RPM packages "kernel-devel" and perhaps "kernel-headers" need to be installed, and of course the compiler and make utility: sudo dnf install kernel-devel-$(uname -r) gcc make -or-, if the dnf command is not available: sudo yum install kernel-devel-$(uname -r) gcc make Under SuSE/openSUSE, the RPM package(s) to be installed are called "kernel-devel" or e.g. "kernel-default-devel" on SLES11 SP2. As a dependency, the associated kernel source package is automatically installed, i.e. "kernel-source" or "kernel-default-source". In addition the gcc compiler package and make utility need to be installed: sudo zypper in kernel-devel gcc make On *very* old SuSE systems the "zypper" command is not yet available. For example in SuSE 9.1 the packages "kernel-source", "kernel-syms" as well as the compiler "gcc" and the "make" utility are required and can be installed using the standard system configuration tool "yast". On arch Linux, the kernel headers can be installed with the following command: sudo pacman -S linux-headers ATTENTION: It is very important that the version of the installed kernel headers matches the version of the running kernel. Build errors due to version mismatches may occur for example - if a newer kernel version is already available as an online update, but has not yet been installed - if a newer kernel has already been installed but the system has not yet been rebooted In such case, an installation of the kernel headers as explained above may install the headers for a different kernel than the one that is currently running. Use the package manager of the distribution to see if there's a version of the kernel headers package that matches the output of the command "uname -r". Unlike with earlier driver versions or versions of the Linux kernel, the kernel sources *don't* need to be configured anymore because the configurations of the different kernel flavours are usually shipped with the development packages provided by the distribution. Loading kernel drivers that have been built for a different kernel into a running kernel may result in system crashes, if not even the module refuses to be loaded. To avoid this problem, a strong version checking can be done by the build environment which requires symbol version information to be available in the kernel source package. It is also possible to install the driver package for Linux on a Raspberry Pi running Raspbian. However, it is a little bit tricky to properly set up the build environment on these systems. For details, please see the specific knowledge base page at https://kb.meinbergglobal.com/kb/driver_software/driver_software_for_linux/linux_driver_package_on_raspberry_pi or contact Meinberg support (techsupport@meinberg.de) for further information. 3.3 Compiling the Driver ------------------------ Make sure your working directory is the driver base directory. If the driver package has already been built before then make clean should be run first to remove old object files and binaries. Then simply type make to compile the utility programs first, then the kernel module. On particular versions of specific Linux distributions some build errors may occur because the distro is shipped with an older kernel, but the distro maintainers have backported some kernel API changes from a newer kernel to the old one. Usually it's hard to detect such requirements automatically, but this driver package contains workarounds for a few such cases. For example, on RHEL/CentOS 5.5 there may be an errors like error: redefinition of typedef 'bool' error: redeclaration of enumerator 'false' error: redeclaration of enumerator 'true' This can easily be fixed if "make" is called with specific options, e.g.: make KERNEL_HAS_BOOL=1 KERNEL_HAS_TRUE_FALSE=1 instead of a simple "make" command. If any other errors or warnings are displayed then please report to Meinberg. On success, a message similar to the one below is shown: Build completed successfully. Now type make install to install the executable files. 3.4 Installing the Driver ------------------------- Installation has to be done with administrator privileges. If the current user has no root privileges, you will be asked for the required password. Just type make install to copy the compiled binaries to their target directories. Programs that only read from a device to display some information can be run by any user, so the files are copied to /usr/local/bin. Programs that may change the device configuration, or set the system time, require extended permissions when executed, and thus are copied to /usr/local/sbin. See also chaper 1.3, "Known Limitations" for error messages that may be displayed during installation. If you want to remove the binaries you can type make uninstall 3.5 Loading the Kernel Module ----------------------------- After the binaries have been installed, the module can be loaded for the first time. The command syntax is: modprobe mbgclock [io=] [irq=] The command line parameters are *only* required for very old ISA cards: io= The port base address of an ISA card which defaults to io=0x300 for unmodified switch/jumper settings. irq= The IRQ number of an ISA card which must be selected via a switch or jumper on the board. The default switch setting for ISA clocks is to use no interrupt. This parameter is required only for an ISA clock which shall be used with NTP. For PCI cards these settings are determined automatically, and for USB devices, they are not required at all. When the module is loaded, it generates some startup messages which can be checked by running the command "dmesg". In addition, these messages are added to the system logging facility, which should be checked for errors or warnings. If the module was loaded successfully, it should be listed if you type lsmod You may try to display the device status by entering mbgstatus The output shows some information on the device status, depending on the type of device. After the kernel module has been loaded manually for the first time as described above, the Linux system "knows" which devices are supported by the mbgclock module, and loads the module automatically whenever a supported device is installed or connected. The udev rules installed with the driver package create a device node for each device, and also a refclock link for use with NTP. The location of the rules file is: /etc/udev/rules.d/55-mbgclock.rules However, on systems with a *very* old udev version installed, the mbgclock module may not be loaded automatically at startup. In this case you may want to add the command modprobe mbgclock to one of the startup files, e.g. /etc/init.d/boot.local for SuSE/openSUSE systems. This may also be required if an old ISA card is to be be used. Those cards can not be detected automatically, and the resource assignments (I/O ports and IRQ) can not be determined automatically, either. So the module has to be loaded manually with the appropriate parameters. See above. Once the module has been loaded and the ISA card has been found, the device nodes should be created automatically, similar to PCI or USB devices. 3.6 If the Kernel Module Fails to be Loaded ... ----------------------------------------------- If the kernel module fails to be loaded, in most cases this is because the build environment has not been set up properly. For example, if after the original system installation the kernel has been updated, but the kernel sources have not. On RPM based systems this may happen if the kernel has been updated via an online update, but the kernel sources are installed from the original installation media, e.g. CDROM or DVD. Under Debian/Ubuntu there have been cases where the build environment looked properly at the first glance, but the command "modprobe mbgclock" just prints: FATAL: Module mbgclock not found. even though "make install" had been run before without errors. In these cases the kernel module had been compiled and installed for a different version of the kernel than currently running. This can be detected by running the following command: # find /lib/modules/ -name mbgclock.ko; uname -r /lib/modules/2.6.27-7-generic/extra/mbgclock.ko 2.6.27-7-generic If the version info in the two lines of output is not exactly the same, the kernel headers do not match the running kernel. Check your package management software, e.g. synaptic, and compare available and installed versions of the kernel package (e.g. linux-image-generic) and the kernel headers (e.g. linux-headers-generic). Synaptic reports these as so-called meta packages which link to the correct versions. You have to uninstall the kernel headers, then you can optionally install any available online updates, and finally re-install the headers corresponding to the current kernel version. 3.7 Compiling for a Different Target Kernel ------------------------------------------- By default, the build environment is accessed via the directories /lib/modules/$(TARGET_KERNEL)/source /lib/modules/$(TARGET_KERNEL)/build as usual, and $(TARGET_KERNEL) is replaced by the exact string that is printed by the command "uname -r". In order to build the driver for a different target system with a dfferent kernel version, it is possible to override the target kernel version on the command line, e.g.: TARGET_KERNEL=2.6.17 make Of course the target version identifier has to match match the output of the "uname -r" command on the specified target system, and of course the kernel sources for that target system must have been installed on the build host and must be configured correctly for the target kernel. The compiled kernel module has to be copied manually to the target system. 3.8 Compiling for A Completely Different Kernel Source Tree ----------------------------------------------------------- If the kernel driver is to be build for an completely different kernel source tree that is not accessible via the default paths mentioned in the previous chapter, the full path to the kernel tree can be specified on the command line, e.g.: BUILD_DIR=/usr/src/linux-3.16.7-53 make Please note that a valid kernel configuration needs to be available under the specified path, and the compiled kernel module must only be copied to and run on a system where the kernel is running that has been build under the specified source tree. 3.9 Staged Builds ----------------- A staged build is commonly used to build a package for a remote target. Instead of being installed on the build system, the compiled binaries are usually just installed into a local output directory, from where they are later copied to the target system. Therefore, no root permissions are required for a staged build, and instead of some properties of the build system, it's more important to correctly specify the properties of the final target system. Environment variables useful for staged builds: STAGED_BUILD: If defined, some system checks are skipped, and no root permissions are requested for install, so it's only possible to install to local subdirs that need to be defined, too. INSTALL_MOD_PATH: The base directory to which the kernel driver binary is installed. DESTDIR: The base directory to which the user space stuff is installed. prefix: Prefix for the 'bin' directory, e.g. '/usr' instead of '/usr/local'. DONT_BUILD_USERSPACE: If defined, only build/install the kernel driver. DONT_BUILD_DRIVER: If defined, only built/install the user space stuff. BUILD_DIR: should point to the configured kernel source tree, as explained in the previous chapter. The former makefile variable 'CALLED_FROM_SPEC' is now called 'STAGED_BUILD', but a 'CALLED_FROM_SPEC' alias is provided for backward compatibility. 4. Using the Driver With ntpd ----------------------------- This driver package supports two different ways to provide the NTP daemon with the reference time from a Meinberg PCI card or USB device. In any way the ntp.conf files needs to be edited to tell the NTP daemon which way it should expect the time from the card and driver. Basic information on how to use Meinberg radio clocks with ntpd can be found at http://www.meinberg.de/english/info/ntp.htm The latest NTP sources are available at http://support.ntp.org You may probably also find the sources included in your Linux distribution. 4.1 Configuration With ntpd's Shared Memory Driver -------------------------------------------------- This approach yields the highest accuracy and thus should be used preferably. The driver package contains an additional service called mbgsvcd (Meinberg Service Daemon) which periodically sends a call to the kernel driver that lets the kernel driver read both the PCI card's on-board time and the associated system time with highest resolution, and as close as possible after each other. The returned time stamp pairs are fed into the NTP daemon via the shared memory driver. This eliminates any interrupt latencies and processing times required by the earlier approach using the type 8 driver (see next chapter). If there is already an existing configuration for the parse driver (type 8) in ntp.conf, e.g. server 127.127.8. mode 2 ... then this line and all associated lines referring to the same address 127.127.8. should be removed from the ntp.conf file. The parse driver supports up to 4 devices in parallel, so there can possibly be several entries with ranging from 0 through 3, which should all be removed. NTP's shared memory driver supports up to 4 devices with an index in the range 0 through 3. So for every card the following lines should be added to the ntp.conf file to tell the ntpd to expect the reference time via its shared memory driver (type 28) with index : server 127.127.28. minpoll 4 maxpoll 4 iburst fudge 127.127.28. refid GPSs Please note the refid is only informational and can be set to any approriate string up to 4 characters. In the example above, "GPSs" has been chosen to indicate a GPS time source (GPS) made avialable via shared memory (s). Possible choices could be GPS card refid GPSs DCF77 card refid DCFs TCR card refid TCRs PTP card refid PTPs After the configuration has been finished, the NTP daemon needs to be (re-)started. Also the mbgsvcd daemon needs to be started to feed the reference time to the shared memory. Simply type "mbgsvcd" to start this daemon in case it hasn't been started already during installation. Be sure *both* daemons ntpd and mbgsvcd have been configured to be started automatically when the system reboots. If the card is synchronized to its time source, and both ntpd and mbgsvcd have been running for some time, "ntpq -p" should finally display something like this: # ntpq -p remote refid st t when poll reach delay offset jitter ======================================================================= *SHM(0) .GPSs. 0 l 3 16 377 0.000 0.002 0.003 Please note the resulting accuracy (offset and jitter) also depends strongly on the PC hardware and the Linux kernel version. 4.2 Configuration With ntpd's Parse Driver ------------------------------------------ The approach to use the parse driver (type 8) to provide the NTP daemon with the reference time from a Meinberg PCI card or USB device has been supported by all earlier versions of this driver package, and is still supported by this version. However, this approach can suffer from interrupt latencies which can reduce the resulting accuracy of the system time, so the shared memory approach described in the previous chapter should be used preferably. If ntpd's parse driver (type 8) is used then the NTP daemon accesses Meinberg devices via device names /dev/refclock- with index in the range 0 through 3. On recent systems, symbolic links are created automatically by the udev system for up to 4 devices. Be sure an entry for refclock- is included in the ntp.conf file which is usually located in the /etc directory. The lines should look like server 127.127.8. mode 2 # mode 2 for all Meinberg PCI cards fudge 127.127.8. time1 0.0 # no systematic delay fudge 127.127.8. refid GPSi # informational, depending on card type fudge 127.127.8. flag1 1 time2 7200 # optionally, set trust time with matching the index number used for the symbolic link and "mode 2" telling the NTP daemon to use the data format of the Meinberg standard time string. The "fudge" lines setup some NTP parameters for this clock. The time1 parameter is a build-in compensation of a constant time delay which should be set to 0. The refid parameter is a string of maximum 4 characters which is displayed for example in the output of the ntpq command. We suggest to set refid depending on the card type, for example: GPS card refid GPSi DCF77 card refid DCFi TCR card refid TCRi PTP card refid PTPi The fudge command "flag1 1 time2 7200" can be used to set the so called trust time interval for the card. The trust time is a time interval for which the card is still accepted as reference time source if it has been synchronized but then starts freewheeling, e.g. because the antenna has been disconnected. Normally, the oscillator on the card is much better than the cheap crystal on the mainboard of the PC, so if the oscillator has been disciplined before, it makes sense to keep on using the card as time source for a while even if it starts freewheeling, instead of discarding the time source with the good oscillator immediately and relying on undisciplined system time. If the trust time interval is not explicitely configured using the fudge command, the default trust time of 30 minutes is used. In the example above, the trust time is set to 2 hours (7200 seconds). If the NTP daemon is already running, restart it and check the syslog file for messages generated by the daemon. An NTP daemon shipped with a Linux distribution may have been compiled without support for Meinberg clocks using the parse driver. If this is the case, you will find a message in the syslog which says that the daemon is unable to handle 127.127.8., so you may have to recompile the NTP package with support for Meinberg clocks, or you may try to use the Shared Memory driver as described in the previous chapter. 5. Using the Driver With chrony ------------------------------- Chrony is an alternative implementation of an NTP daemon, which provides the same shared memory (SHM) interface as ntpd, so mbgsvcd can also be used to write the current time from the device to the SHM segment. To tell chrony to read the time from the SHM segment, the following line has to be put into the chrony.conf file: refclock SHM 0 poll 3 refid MBG ### Note: This has not yet been tested by Meinberg, and eventually the following line has to be added, too: server 127.127.1.1 iburst Please note that only the SHM support can be used with chrony. There is no PARSE driver as with ntpd. 6. Notes for SuSE/openSUSE Linux -------------------------------- 6.1 Old SuSE Naming Conventions ------------------------------- The current (v4.x) NTP daemons are called ntpd while older (v3.x) daemons were called xntpd. Up to SuSE Linux 9.3 the run control script used to start or stop the NTP daemon was still called rcxntpd, even though the NTP v4 binary was called ntpd. Starting with SuSE Linux 10.0 this script has been renamed to rcntp. 6.2 NTP and "Operation not permitted" Under SuSE Linux 10.1 and Newer --------------------------------------------------------------------- SuSE Linux 10.1 and newer openSUSE versions are installed by default with AppArmor running. AppArmor controls which applications are allowed to access certain resources and files. If the NTP daemon writes an error message saying "Operation not permitted" to the syslog whenever it starts and tries to open the device /dev/refclock-0 or the shared memory segment, in most cases AppArmor is running but has not been configured to let ntpd access this device or resource. In order to allow that access, a text editor can be used to edit the file /etc/apparmor.d/usr.sbin.ntpd and add the entry /dev/mbg* rwl, at the end of the existing configuration block. Alternatively, the associated Yast module can be used instead the editor program to add that entry. If the NTP daemon has been configured to generate log or statistics files, appropriate AppArmor configuration entries must be added for those files. Alternatively, AppArmor can be disabled, either temporarily by running rcapparmor stop or permanently using the yast module for AppArmor. 6.3 Error "module is unsupported" --------------------------------- If "modprobe mbgclock" is run e.g. under SLES11 SP2, an error is displayed: FATAL: module '/lib/modules/3.0.13-0.27-default/extra/mbgclock.ko' is unsupported Use --allow-unsupported or set allow_unsupported_modules to 1 in /etc/modprobe.d/unsupported-modules so as a temporary workaround the command modprobe --allow-unsupported mbgclock loads the module. To be able to load the module automatically at system startup the file /etc/modprobe.d/unsupported-modules has to be modified as given in the error message quoted above. 7. "Permission denied" With SELinux (e.g. RedHat/Fedora/CentOS) --------------------------------------------------------------- RedHat Linux and other distributions based on RedHat (e.g. Fedora or CentOS) often use SELinux to control which processes are allowed to access certain files or devices. If a new device is installed, access to this device is by default denied to the NTP daemon. Please see the README file in the SELinux subdirectory for instructions how to grant the required permissions to the NTP daemon.