• View
• Edit
• History
• Print

RemoteBackup

2013-11-19

• While working with an HP t5540 thin client with 1GB of RAM, booting PuppyLinux from one of my PuppyPlayer USB thumb drives will copy the system into RAM, rather than running from the USB drive, and this prevents the system init from mounting the USB drive device on /initrd/mnt/dev_ro2. My PuppyPlayer setup depends on that mount being in place. To prevent this problem, when using my PuppyPlayer USB drives, I found that I can edit the USB drives /syslinux.cfg file, changing 'pfix=copy' to 'pfix=nocopy' instead. Now, no matter how much RAM a system has the system will always boot as expected.

Remote System at XOL (defunct, relocated to AR, see below 2017-10-24)

2014-06-26

For a remotely located system I'm using an HP t5540 name "eye" running Puppy Linux Lucid 5.2.8 with a 16GB USB thumb drive as boot and storage media. This USB drive is installed in one of the two 'secure' USB ports inside the top cover.

The system will be running hands-off and headless. In order to make this system useful and secure, there are three areas that required specific attention and configuration adjustments to the 'standard' Lucid 5.2.8 setup: 1) system environment, 2) network configuration, and 3) secure filesystem. A fourth area turned up after some experience with the system: remote rebooting.

System Environment

As a headless system, there is no need to run X-Windows. Too, I have discovered that while the system boots through to X and the JWM desktop when a monitor is attached, the system will start the Xorg Wizard on the console terminal if a monitor is NOT attached. This wizard requires local keyboard and screen access to complete, which is not possible as a headless system. I have discovered that I can get rid of the wizard by logging in remotely and killing the 'dialog' process, three times. This then leaves the system without X running nor is it configured.

The best approach then seems to simply eliminate the start of X from the boot sequence and this is done by editing the file 'syslinux.cfg' on the root the USB drive (or in /initrd/mnt/dev_ro2/ once LP is running) and adjusting the 'pfix' element of the kernel loader command to include 'nox', e.g. 'puppy pfix=copy,nox'. This change results in the system booting up to just the ordinary textual console terminal.

One other element of the general configuration is the installation of the SSH-server pet, which makes it possible to connect to the system remotely. Here, the key requirement is to move the incoming host's public SSH key into /root/.ssh/authorized_keys. This allows the the logging-in client to do so without a password prompt. so, with my chained setup, I have put the public key from root@humbaba into jblaser@minithor.testxolights.com:/home/jblaser/.ssh/authorized_keys, and the public key from jblaser@minithor into root@eye:/root/.ssh/authorized_keys.

Also, the root password has been changed to something more secure than the default.

Network Configuration

It is best to let the system boot using DHCP to configure the network, then set the assigned 'static' IP. This has the advantage that the system will automatically use changes to the default router and nameserver settings, at least upon reboot. To make this work I had to add the following to the end of /etc/rc.d/rc.local:

## wait for dhcp to get started and set our network
echo "waiting for dhcp setup..." > /dev/console
while ! ps aux | grep dhcpcd | grep -v grep ; do sleep 5 ; done

## make sure we have DNS configured
while ! cat /etc/resolv.conf | grep nameserver > /dev/null ; do sleep 5 ; done

## get gateway set by dhcp
while ! route -n | grep -v Gateway | grep G > /dev/null ; do sleep 5 ; done
gip=`route -n | grep -v Gateway | grep G | tr -s ' ' | cut -d ' ' -f 2`

## kill dhcpcd deamon
echo "  killing dhcpcd, retaining resolv.conf" > /dev/console
killall -9 dhcpcd			## use TERMINATE to retain resolv.conf

## now set a static IP address
echo "  setting statip IP address to 192.168.1.29" > /dev/console
ifconfig eth0 192.168.1.29

## now re-set the gateway after setting IP
echo "  re-setting default gateway to $gip" > /dev/console
route add default gateway $gip		## changing eth0 requires re-set

Secure Filesystem

Since this is a remotely located system, the system must have an encrypted filesystem, in case it falls into the wrong hands. I've learned how to do set up an encrypted ext2 filesystem, which requires the manual entry of a passphrase on each mount. This will only be necessary when the system is rebooted and that should be a rare event. Using the 16GB USB drive, I have partitioned it into two paritions. The first is 1GB FAT32, which holds the MBR, bootstrap, Lucid system files, and the layered 'user save' filesystem. The second is 15GB EXT2, which holds all the archive/backup files and is encrypted.

I have learned that the following can be used with a file or a device. In my case I used the 15GB partition /dev/sdb2.

1. load the kernel crypto modules

 * modprobe cryptoloop
 * modprobe aes|blowfish|des|etc.

1. attach a loopback device using encryption

 * losetup-FULL -e aes /dev/loop2 /dev/sdb2
 * enter a secure passphrase for the new filesystem

1. (one-time) create the filesystem

 * mkfs -t ext2 /dev/loop2

1. mount the encrypted filesystem

 * mount /dev/loop2 /mnt/XXXX

1. unmount the encrypted filesystem (auto-detaches the loopback device, in the process)

 * umount /mnt/XXXX

1. if necessary, before a mount and umount, the loopback can be manually detached

 * losetup-FULL -d /dev/loop2

I wrote a script to help me mount and unmount:

#!/bin/bash

###################
##
## archmount.sh
##
## 2014-06-27 :: JRB
##
## Helper script to mount and unmount encrypted USB partition
##
###################

eusb='/dev/sdb2'
emnt='/mnt/arch'

if [[ $1 == '' || $1 == 'm' ]] ; then

	## mount the encrypted partition

	## make sure we're not already mounted
	test=`mount | grep "$emnt" | tr -s ' ' | cut -d ' ' -f 1`
	if [[ $test != '' ]] ; then
		echo "$test ($eusb) already mounted on $emnt"
		exit
	fi

	## get next available loopback device
	loopdev=`losetup -f`

	## create mount point, if needed
	if [[ ! -d $emnt ]] ; then
		mkdir $emnt
	fi

	## load kernel modules, if needed
	if ! lsmod | grep cryptoloop > /dev/null ; then
		modprobe cryptoloop
	fi
	if ! lsmod | grep aes_generic > /dev/null ; then
		modprobe aes
	fi

	## establish encrypted loopback (will prompt for passphrase)
	if losetup-FULL -e aes $loopdev $eusb ; then
		## mount the encrypted device
		mount $loopdev $emnt
		echo "$loopdev ($eusb) mounted on $emnt"
	else
		## loopback failed, release loopback device
		## making sure $loopdev is not blank
		if [[ $loopdev != '' ]] ; then
			losetup -d $loopdev
		fi
		echo "mount of $eusb ($loopdev) failed"
	fi
elif [[ $1 == 'u' ]] ; then

	## un-mount ecrypted partition

	## determine loopback device in use
	loopdev=`mount | grep "$emnt" | tr -s ' ' | cut -d ' ' -f 1`

	## perform un-mount
	if umount $emnt ; then
		## un-mount succeeded, release loopback device
		if [[ $loopdev != '' ]] ; then
			if losetup $loopdev > /dev/null 2>&1 ; then
				losetup -d $loopdev
			fi
		fi
		echo "$loopdev ($eusb) unmounted from $emnt"
	else
		echo "un-mount of $emnt failed"
	fi

elif [[ $1 == '-h' || $1 == '--help' ]] ; then

	## show usage
	bin=`echo $0 | sed 's/.*\///'`
	echo
	echo "Mounts and un-mounts encrypted partition $eusb"
	echo "Default behavior without CL parameter is 'mount'"
	echo
	echo "USAGE: $bin [-h|--help|m|u]"
	echo
	echo "WHERE: -h|--help  Produces this message and exits"
	echo "       m  Mounts encrypted partition $eusb on $emnt"
	echo "       u  Un-mounts encrypted partition"
	echo
	exit
fi

Remote Rebooting

Another issue has arisen related to remote rebooting. It turns out that if I execute 'reboot' while in a remote SSH session, the reboot script kills the SSH daemon, which kills the SSH connection, which kills the current shell, which kills execution of the script before all of the script commands are executed. This results in the system staying up, but with the SSH daemon shutdown, leaving me no way to further remotely control the system. Only an on-site reboot will work at that point.

I have found that the solution is to run the reboot command in a detached sub-shell by enclosing it in parentheses. Since I probably wouldn't remember to do that when the moment came to enter the command, I have created a small script that replaces the default reboot command, /sbin/reboot', now renamed '/sbin/reboot.orig'. It starts a detached sub-shell which then calls the renamed default command:

#!/bin/bash

(reboot.orig) &
exit

It just occurs to me that another approach would be to create a watchdog of sorts that would restart the ssh daemon if it is not found running. This would resurrect my ability to try again to use the correct commands to shutdown. But with my little replacement script now in place there is no need.

Remote backup at AR - 2017-10-24

XOLogic has closed its SSH capability from the outside, thus preventing future offsite backups to those premises. Instead, I have removed the equipment (thin client EYE) from XOLogic and relocated it in the gazebo, which does have a hardwire connection to the local network. Copying HUMBABA's public ssh key to EYE:/root/.ssh/authorized_keys, and couple of small adjustments to the initialization section of HUMBABA:/root/offsitebackup/offsitebackup.sh were all the actions required to re-enable the backups.