Enabling OpenVPN configuration / autostart on Ubuntu

This assumes that you’ve already made sure that your configuration is valid and is able to connect (you can do this by calling openvpn --config /etc/openvpn/FILENAME.conf directly. It won’t be daemonized, but it will give you any errors on the console directly).

There’s a few details you’ll have to get right before the openvpn daemon starts your configuration automagically under Ubuntu:

  1. Your configuration has to be under /etc/openvpn/FILENAME.conf. The .conf part is important. If it ends with .ovpn or anything else, it won't be loaded.
  2. Ubuntu isn't set to start all configurations by default. You can change this by editing /etc/default/openvpn. Change the AUTOSTART variable to the configurations you want to start when the daemon starts. The example in the file says "all", which means that all defined configurations will start. This is OK if you want to keep openvpn up at all times.
  3. You have to tell systemd that you've changed the default file. If you don't do this, nothing will have appeared to change for openvpn - unless you restart the OS. And you don't want to restart your server just to make a setting visible. Do systemctl daemon-reload to make systemd reload the settings (this is also in the comments in the file, but hey, you don't have time to read those, so now you're searching Google instead).
  4. Restart openvpn: service openvpn restart
  5. Confirm that everything went OK by looking in /var/log/syslog

Building Gearman: cannot find the flags to link with boost thread

While running ./configure things may bail out with the message “cannot find the flags to link with boost thread”. This is cause by configure not being able to find the development version of the libboost-thread libraries on the current library path. If you’ve not built libboost from source, you probably need to install the -dev version of the libboost-thread package for your distribution.

For ubuntu this package is named libboost-thread-dev (you’ll find specific versions under names such as “libboost-thread1.46-dev”. Do apt-cache search libboost-thread to see which packages are available).

For Gentoo the package is named dev-libs/boost, so just emerge that one (thanks to J/#gearman).

THIS. CHANGES. EVERYTHING. – Useful Bash/*nix Tricks I Never Stumbled Across in the Last 15 Years

A thread at /r/linux sought out to reveal all the magic ways of increasing productivity under Linux (or other *nix based OS-es), and as most people I thought that there wouldn’t be much news here.

But I was wrong. So very, very wrong.

  1. disown – a way to disown a process, making it continue running in the background if you have to log out or close a long running session over ssh because you’re going somewhere, but want to keep the currently running process still running. If you’ve ever thought “why the fsck didn’t I run this under screen?”, then this trick is for you. This is a new future, and I’m proud to be a part of it.
  2. CTRL+r in bash – allows you to search your bash history buffer. I’ve known about this, I’ve just never picked up the habit of actually using it. Will do that now.
  3. ssh-copy-id – Appends your public key to the authorized_keys file at the destination computer.
  4. man ascii – the manual page entry for ascii contains an ascii table, right there in your terminal.
  5. xargs ‐‐max-procs and parallel – allows you to duplicate the functionality of xargs, but in parallel. Starts up all the processes at the same time, instead of starting them one by one.

Head over to the thread for other goodies such as a sudo alias for writing files when you’ve opened them without the correct permissions directly in vim.

Evolution & Exchange: Unable to retrieve message

Some time after upgrading to Ubuntu 11.10 I ended up with the dreaded “Unable to retrieve message” in Evolution (which I use for Exchange connectivity). This has usually corrected itself by simply restarting Evolution, but this time nothing would help. I stumbled across a thread that provided a few ways to possibly solve the issue, but the .evolution directory didn’t contain any live installation in Ubuntu.

Turns out the directory is:

.local/share/evolution

As both my mailstore and address book lives on the Exchange server, I decided to just move the evolution directory to a new name and recreate the evolution directory from scratch. This takes a bit of time while Evolution indexes everything, but after a while everything were back to normal.

rsyslogd stuck at eating 100% (or more) CPU after upgrading to Ubuntu Natty Narwhal

This might also happen after upgrading to maverick, so don’t ignore the explanation even if you’re a version or two behind (.. or reading this at a much later time and we’ve all switched to implants).

Apparently the reason for rsyslogd getting stuck is a mismatch between how the kernel provides access to rsyslogd and what rsyslogd expects. If rsyslogd fails to get access to elements in the proc file system (/proc/kmsg was suggested in a bug thread), it locks up and spews out error messages at a great rate.

From /var/log/syslog

Apr 29 08:04:08 ubuntu kernel: Cannot read proc file system: 1 - Operation not permitted.
Apr 29 08:05:08 ubuntu kernel: last message repeated 13208405 times
Apr 29 08:06:08 ubuntu kernel: last message repeated 13297682 times
Apr 29 08:07:08 ubuntu kernel: last message repeated 14241325 times
Apr 29 08:08:09 ubuntu kernel: last message repeated 14397034 times
Apr 29 08:08:43 ubuntu kernel: last message repeated 7302035 times

Yes, that’s about 62 million error messages in less than 5 minutes. This demands quite a bit of CPU.

The reason for this is that the kernel API changed somewhere between the current Ubuntu version (2.6.38 in Natty) (and possibly the one in Maverick) and the one I was running (2.6.31). When rsyslogd runs under the latter, everything goes haywire. The solution is to make sure your kernel is upgrade to the most recent version – and that you’re actually running it.

First, stop rsyslogd to make your system a bit more responsive again:

sudo service rsyslogd stop

Updating Ubuntu should already have installed the newest kernel versions, but you might have told Ubuntu to use the existing configuration file instead of overwriting it when you updated (I almost do that automagically, which left me a couple of kernel versions behind). You can re-run this process and get grub to use an updated kernel version:

sudo update-grub

This might ask you again about whether you want to overwrite the current configuration file, and will also allow you to inspect the differences between the currently installed file and the one that update-grub wants to install. See if there are any significant changes (pay attention to information such as which partitions to use for booting), and if looks OK – allow the file to be replaced.

update-grub will then update your boot sequence with the new configuration file, and after rebooting (press ESC if you need to see the grub menu to make any changes), your new kernel should be running smoothly and rsyslogd should hopefully behave properly again.

up2date: The package .. is signed, but with an unknown GPG key.

After attempting to install libevent-devel on one of our cluster nodes, up2date started complaining about missing GPG signatures. We use the Fedora EPEL repository for fairly new items, and apparently the key were missing / not updated.

The error messages:

warning: rpmts_HdrFromFdno: V3 DSA signature: NOKEY, key ID 217521f6
The package .. is signed, but with an unknown GPG key. Aborting...
Package .. has a unknown GPG signature. Aborting...

The fix for us was to download the keys from Fedora’s key page (we just needed the EPEL-key), and add them with rpm:

wget https://fedoraproject.org/static/217521F6.txt
rpm --import 217521F6.txt 

.. and things worked as it should yet again – and we still get our packages verified. You can also confirm that the key id from the error message (217521F6) is the same as the key that you’ve downloaded from Fedora (or if you’re using another repository they’ll probably provide their keys as well).

A Quick Introduction to chmod and Octal Numbers

Someone asked what the difference between doing a chmod 777 and chmod 755 is today, and hopefully this short, informal post will provide you with the answer (if you want to jump straight through to the conclusion, man chmod).

Octal Numbers

The number you provide as an argument to chmod is an octal number telling chmod what access you want to provide to a file (or a directory, device, etc – an entry on the file system). The number are in fact three discreet values, 7, 5 and 5. Each of the values correspond to a set of three bits, either one being zero or one. Three bits makes up a value from 0 – 7, hence an octal number (a decimal number has the digits 0 – 9 for each digit, an octal number has 0 – 7, a binary number has 0 – 1, a hexadecimal number has 0 – F (15)).

If you tried to count from 0 to 10 (decimal) in octal, it’d be: 0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 12. 12 in octal is the same value as 10 in decimal. The big difference is that both octal and decimal maps very neatly on top of binary numbers, being exactly three or four bits.

The usual way to write an octal number in a programming language is by appending a zero in front of it, such as 0755. This tells the compiler that the number is written in octal notation, and the value is then parsed as such. chmod parses all numbers as octal, and does actually handle four digits. Since missing digits are considered to be zero, the first digit is usually not included (or simply as a zero – which will look the same as the representation used in certain programming languages). The first, usually unused digit, have a special meaning, setting the “set user id” (suid), “set group id” (guid) or the “restricted deletion” or “sticky” attributes (you can read more about these options in the manual page).

File permissions

Now that we know what an octal number is, it’s time to look at how the file permissions work. Each file has three sets of permissions, one set for the user owning the file, one set for the group owning the file and one set for anyone else. If you want to take a look at these values on a unix based system, simply type ls -l to list files in a verbose way. Your result will look something like:

-rw-r--r--  1 mats mats        35 2008-08-23 20:24 IMPORTANTFILE

The permissions are listed in the first column, containng “-rw-r–r–“. The first character “-” indicates if the file is a directory (d), if the suid or guid bits are set etc.

This leaves us with “rw-r–r–” – the three sets of permissions. “rw-” is for the user owning the file, “r–” is for the group owning the file and the last “r–” are for anyone else (or ‘other’ as it’s called). The “r” means read, the “w” means write and the currently missing letter is “x”, which means execute (for files) or search (for directories). The “execute” setting is used to let bash (or another shell) attempt to run the file as a script, attempting to parse the first line as a path to the interpreter for the file (i.e. #!/usr/bin/python).

We have three flags (read, write, execute) that can be either on or off. This should remind us of three bits, either being 0 (not set) or 1 (set). And an octal digit is exactly three bits. This means that an octal digit maps exactly to the bit sequence needed to set permissions for a file. A 7 is “111”, a 5 is “101”, a 4 is “100” and so on. Mapping this to permissions:

7 = 111 = rwx
6 = 110 = rw-
5 = 101 = r-x
4 = 100 = r--
3 = 011 = -wx
2 = 010 = -w-
1 = 001 = --x
0 = 000 = ---

When calling chmod 755 on a directory we’re telling chmod to “set the read, write and search bits for me, the read and search bits for the group and the read and search bits for other users” (‘search’ for directories, ‘execute’ for files).

Another example is 644 that maps to 110 100 100, which again maps to “rw-r–r–” which usually is the standard access mode for files (and 755 for directories).

Handling Permissions With Symbols

I’m now going to eliminate the need for remembering everything I’ve written so far in the post, but at least you’ll know what people are talking about when they’re telling you to chmod something this-or-that.

You can also use the symbols directly with chmod, either adding, removing or setting the permissions for one of the three groups.

Examples:

To remove all access for other users (but leaving group and user intact)
chmod o-rwx file

To give everyone read access
chmod a+r file

To give everyone read – and search – access
chmod a+rx directory

To set particular user modes for each group
chmod u=rw,g=w,o=w file (a file that the user can read, but anyone can write to)

And with that I chmod this post a+r.