Systemd Mount Point Management
- May
- 17
Today we use systemd to manage our boot process and do nice things like reducing boot time by parallelizing operations and managing service dependencies. One area that still needs some work is /etc/fstab. We're all familiar with making entries there to mount local disks, remote filesystems and perform bind mounts through fusefs. The fstab is still a supported and convenient way to manage mounts but it isn't used directly anymore. Instead, systemd uses a component to parse the file and build standard unit files that can be handled in the normal systemd way. The parsing process is supposed to pay attention to things like file system hierarchies so that it can produce the correct dependencies so that everything gets mounted as intended during the boot process. That is of course if it worked properly. I found out the hard way that it doesn't; at least not in the combination of mounts I had in my /etc/fstab.
/dev/mapper/ingest--digitalarchives--vg-root/ ext4 errors=remount-ro 0 1
/dev/mapper/ingest--digitalarchives--vg-swap_1 none swap sw 0 0
10.20.40.13:/mnt/tank/unix/home/home nfs defaults 0 0
10.20.40.13:/mnt/tank/sftp /sftp nfs defaults 0 0
/sftp/sftp_user/reports /var/www/drupal7/sites/default/files/private/reports none bind 0 0
The fstab had the normal filesystem entries to get the machine up and running including the root slice, swap, etc. There are a few NFS mounts in there and a single bind mount. The bind mount was the one causing the issue; local filesystems and NFS mounts seem to come up fine but the bind mount depended on the NFS mount as it was a subdirectory. The bind mount would execute first and then the NFS mount that overlapped it would hang and force systemd to emergency mode.
Looking at the systemd documentation and it's handling of fstab it seemed that the dependency could manually be stated there as an option. Following examples I tried the following without success.
/sftp/sftp_user/reports /var/www/drupal7/sites/default/files/private/reports none bind,x-systemd.requires=/sftp 0 0
In the end I left the NFS mounts in the fstab and created my own unit file in /etc/systemd/system named var-www-drupal7-sites-default-files-private-reports.mount that looks like this:
[Unit]
Description=Mount for reports
After=remote-fs.target
[Mount]
What=/sftp/sftp_hslc/reports
Where=/var/www/drupal7/sites/default/files/private/reports
Type=none
Options=bind
[Install]
WantedBy=multi-user.target
This makes sure all the NFS file mounting is finished before it tries the bind mount. Now things mount in the proper order and the machine actually boots.
I suspect the problem has to do with the fact that I am using a bind mount. Perhaps the bind mounts are done before remote filesystems without regard to options specified in fstab. I could call it a bug since systemd does not interpret fstab and produce the correct intended result.
Using networksetup to add static route(s) to a Mac OS VPN
- February
- 4
These commands were executed in a user context, no sudo needed. The routes added to your VPN connection should survive a reboot.
First I list out my network connections to make sure I have the name correct:
$ networksetup -listallnetworkservices
An asterisk (*) denotes that a network service is disabled.
Ethernet
Wi-Fi
home.mydomain.net
*INT
Bluetooth PAN
Thunderbolt Bridge
COGENT-HSLC
GUEST
COGENT-ACCESSPA
Verizon-DSL
DMZ
MGMT
STORAGE
55 with Default
55
52
52 with Default
HSLC
I want to add a route to home.mydomain.net so I do:
$ networksetup -setadditionalroutes home.mydomain.net 192.168.2.0 255.255.255.0 192.168.1.71
I happen to know that the IP address on the other side of the gateway is 192.168.1.71. If you don't know, connect to your vpn and look in the routing table with:
$ netstat -nr -f inet
If you want more than one route you have to combine them all into one command. You can't add them one at a time as the current route will override the previous. So to add two routes I would do:
$ networksetup -setadditionalroutes home.mydomain.net 192.168.2.0 255.255.255.0 192.168.1.71 192.168.3.0 255.255.255.0 192.168.1.71
If your VPN name has a space in it, put quotes around the name like:
$ networksetup -setadditionalroutes "home vpn" 192.168.2.0 255.255.255.0 192.168.1.71
Finally, to delete all the additional routes just leave the route blank.
$ networksetup -setadditionalroutes "home vpn"
To check your work:
$ networksetup -getadditionalroutes "home vpn"
Moving a CentOS 7 VM from VMWare to Xenserver
- December
- 14
After searching and finding all the little pieces of this process I thought I would document it here for future referral.
The main issue with moving the VM from one hypervisor platform to the other is that the initial RAM disk doesn't have the Xen network and block device drivers. So even if you can successfully convert the disk through export/import or a third party (QEMU perhaps) application, the machine won't come up.
What's needed is to rebuild the RAM disk with the right drivers and then tell grub to use it.
First, let's build the new RAM disk image. Log into the running VM under VMWare and do the following:
# cd /boot
# ls initramfs*
initramfs-0-rescue-898e9089de1c4bcfac2d751bbd9e7f10.img
initramfs-3.10.0-693.21.1.el7.x86_64.img
initramfs-3.10.0-693.21.1.el7.x86_64kdump.img
initramfs-3.10.0-693.el7.x86_64.img
initramfs-3.10.0-693.el7.x86_64kdump.img
initramfs-3.10.0-862.el7.x86_64.img
initramfs-3.10.0-862.el7.x86_64kdump.img
I'm going to use the newest kernel to build my RAM disk. If you have a different kernel (likely), adjust the name as needed.
# mkinitrd --omit-scsi-modules --with=xennet --with=xenblk --preload=xenblk /boot/initramfs-3.10.0-862.el7.x86_64-XEN.img 3.10.0-862.el7.x86_64
ls -ld initramfs*
-rw-------. 1 root root 51307405 Mar 14 2018 initramfs-0-rescue-898e9089de1c4bcfac2d751bbd9e7f10.img
-rw------- 1 root root 21008864 May 10 2018 initramfs-3.10.0-693.21.1.el7.x86_64.img
-rw------- 1 root root 13336203 May 10 2018 initramfs-3.10.0-693.21.1.el7.x86_64kdump.img
-rw-------. 1 root root 20349498 Mar 14 2018 initramfs-3.10.0-693.el7.x86_64.img
-rw-------. 1 root root 18730293 Mar 14 2018 initramfs-3.10.0-693.el7.x86_64kdump.img
-rw------- 1 root root 21049754 May 10 2018 initramfs-3.10.0-862.el7.x86_64.img
-rw------- 1 root root 12976861 Jul 13 07:33 initramfs-3.10.0-862.el7.x86_64kdump.img
-rw------- 1 root root 21075430 Dec 14 10:29 initramfs-3.10.0-862.el7.x86_64-XEN.img
Okay, you can see the new image and it's about the same size as the other one too.
Next, we need to copy the text from a boot entry that grub uses. Find it in /boot/grub2/grub.cfg. Don't edit this file, just find the section that looks like this:
menuentry 'CentOS Linux (3.10.0-862.el7.x86_64) 7 (Core)' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-693.el7.x86_64-advanced-0a5149be-1fe5-40bf-8597-c0c588af88c8' {
load_video
set gfxpayload=keep
insmod gzio
insmod part_msdos
insmod xfs
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1' efa02a93-827e-4734-8a44-7eb1ef93119a
else
search --no-floppy --fs-uuid --set=root efa02a93-827e-4734-8a44-7eb1ef93119a
fi
linux16 /vmlinuz-3.10.0-862.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8
initrd16 /initramfs-3.10.0-862.el7.x86_64.img
}
Take that text and add it to the end of the file in /etc/grub.d/40_custom. You then want to change two things. First change the name of the entry so you 'll be able to see it when you boot. Then change the name of the RAM disk image being used. My finished file looks like this:
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.
menuentry 'CentOS Linux (3.10.0-862.el7.x86_64) 7 (Core) with xen' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-693.el7.x86_64-advanced-0a5149be-1fe5-40bf-8597-c0c588af88c8' {
load_video
set gfxpayload=keep
insmod gzio
insmod part_msdos
insmod xfs
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1' efa02a93-827e-4734-8a44-7eb
1ef93119a
else
search --no-floppy --fs-uuid --set=root efa02a93-827e-4734-8a44-7eb1ef93119a
fi
linux16 /vmlinuz-3.10.0-862.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8
initrd16 /initramfs-3.10.0-862.el7.x86_64-XEN.img
}
Lastly, we need to rebuild the grub menu.
# grub2-mkconfig --output=/boot/grub2/grub.cfg
That should do it. Now you can export your machine and when it boots under Xenserver make sure you choose the right boot option when it starts up. You can make that the default option in grub if you want to.
I did not have any issue using the RAM disk containing the xen drivers under VMWare either so I'll bet you could just copy the file over the other RAM disk and it would boot under either OS. May try that just to make sure it works okay.
CentOS 7 and VMWare ESXi Networking via CLI
- August
- 30
Easiest way to get your network up and running correctly and quickly:
1. Make sure you configure your network with static IP address (if needed) when installing. Enter the IP address, network, gateway, etc.
2. Let VMWare assign the MAC. Much easier than trying to manage it through udev or the config files.
Your network configuration file is in /etc/sysconfig/network-scripts/ directory. You only need to touch the file for the specific adapter that is created (e.g. ifcfg-ens192).
If you add another adapter to the machine in VMWare use the following steps:
1. I usually reboot to make sure it is properly detected by the OS.
2. Do an ifconfig -a to see the new adapter name.
3. Copy the ifcfg-ensnnn file to the new name, then edit the contents. Go over each item carefully. Make sure you change the DEVICE line to the new name. Change the UUID line. To get the UUID you can:
nmcli -t --fields uuid,device con show
4. Reboot to make sure it all works, your adapter should be up and have the right IP address, etc. Use ifconfig -a to check it.
To add a static route:
Create a file in /etc/sysconfig/network-scripts called route-ens192 (or whatever the adapter is).
Put the route parameters in there like this:
10.0.0.0/8 via 10.182.3.1 dev ens192
Reboot, then check netstat -nr to make sure you see the proper kernel route.
Zwift Group Rides
- January
- 27
With the cold and snow here in Philly I've been doing plenty of riding on Zwift lately. When I started out using Zwift I was very content to ride solo especially during the beta phase when it was more difficult to match the pace of other riders and drafting didn't work very well. These days it's not uncommon to see several hundred people riding around Watopia at any time of day or night. You can easily find someone who is putting out about the same power as you are and usually other riders welcome the company.
I've also tried a few "group" rides on Zwift. Basically you show up at the start/finish banner, don the recommended jersey (not all riders do this), add some letters to your name to identify yourself as a member of the group (again, not all do it), and wait for the leader to start sending group texts so everyone starts at the same time and maintains the correct pace.
While it's kind of fun to see myself riding along in a big pack of riders reminiscent of past season road racing, there are some challenges with group riding on Zwift. I'm sure these challenges have not gone unnoticed by Zwift development staffers and I'm hopeful that they will give them the excellent attention that they have also devoted to other features within the app. Meanwhile, here is a view from my perspective:
Identification - It's somewhat difficult to discern who you are supposed to be riding with. As a workaround ride leaders have requested donning a particular jersey or adding certain letters to your display name as a way to indicate that you are on that group ride. Not an ideal solution. Some people just don't do it and can't say I blame them much because at the very least it's inconvenient to have to go clicking through screens in the app setup. The jersey idea is a good one if it could be quicker and easier to do. If not a specific jersey for a ride, perhaps a bib instead? That would be really cool.
As far as adding letters to your name, I think that is a bit of a hack. A more sophisticated way to present the names in the right-hand pane would be to use a unique background color for riders in your group or something along those lines. I'd also like to see the ability to do some filtering in the user pane so that only your group riders would show up perhaps. Watching the names on the rider list flip around while waiting for riders to catch up or a ride to start is bound to give me A.D.D. eventually. And it would also be nice to have a keystroke that would return my view to myself easily instead of having to find myself in the list and click.
Communication - Although I consider myself to be a descent multi-tasker, I find it difficult to type and bike even with my new Wahoo Fitness Desk. I have heard some folks comment on speech-to-text apps, haven't tried those yet but at least that would keep your hands on the handlebars more of the time and less sweat dripping down into the keyboard. TeamSpeak seems like a good approach and is certainly more natural. Only issue is that it is a third-party app so you have to start it up seperately. Thinking out of the box a bit, if voice could be integrated into the Zwift app it would be really cool to relate the volume you hear to how close the rider is. That way you could yell out to people nearby just like you would on the road. Maybe the leader could override so everyone could hear him/her. That would do away with some of the complexities of the TeamSpeak app like private channels and such.
Exclusivity - You can jump into a group ride or a race without any type of invite. Is this a feature or a challenge? Depends on your point of view. For a group ride I could care less. But if there are people off the front in a race and someone decides to pull them for a few minutes that's not good etiquette. Along the thoughts of eventual scalability of the app maybe races could have there own race course without any other riders on it. Not sure developers thoughts on this but most other online games have followed this model as the number of users grows past the capabilities or asthetics of a single environment.
Graphics - First, I have to comment that the graphics work on Zwift has been excellent! It does seem odd though that you can sprint right up into a big pack of riders and end up on the front. It's like there is no yellow line rule. Not exactly a real racing scenerio. Maybe the algorithm could be tuned a bit so you could actually get "boxed in" and such. That would be more realistic for a race. For a normal group ride probably doesn't matter too much.
Trainer Differences - I can imagine how hard it must be to try to keep everything fair for everyone. So many variables that are beyond the control of developers especially when using old-style trainers without power control. I seem to notice that things are a bit better than they used to be; assuming things are being tuned up within the software to flatten things out. But I still see the occasional person pushing incredible wattages or beating my sprint in half the time. I would also think that it may be difficult to stay with a group if you are on an old-style trainer as I have seen wider fluctuations in power from riders using that equipment.
The good thing out of all this is that group riding is possible whenever you have the urge to jump on the bike and ride. You can meet people from all over the world and then drop them. I've been pretty impressed with the demeanor and courtesy of Zwifters; hopefully this is a reflection of those qualities in the cycling community at large. I'm hoping Zwift continues to be successful and doesn't attract the types of individuals that like to bully, cheat or deny service to others; some of the issues that have arisen in other gaming communties.
Happy Zwifting!