A Quick Introduction to chmod and Octal Numbers

Someone asked what the difference between doing a chmod 777 and chmod 755 is today, and hopefully this short, informal post will provide you with the answer (if you want to jump straight through to the conclusion, man chmod).

Octal Numbers

The number you provide as an argument to chmod is an octal number telling chmod what access you want to provide to a file (or a directory, device, etc – an entry on the file system). The number are in fact three discreet values, 7, 5 and 5. Each of the values correspond to a set of three bits, either one being zero or one. Three bits makes up a value from 0 – 7, hence an octal number (a decimal number has the digits 0 – 9 for each digit, an octal number has 0 – 7, a binary number has 0 – 1, a hexadecimal number has 0 – F (15)).

If you tried to count from 0 to 10 (decimal) in octal, it’d be: 0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 12. 12 in octal is the same value as 10 in decimal. The big difference is that both octal and decimal maps very neatly on top of binary numbers, being exactly three or four bits.

The usual way to write an octal number in a programming language is by appending a zero in front of it, such as 0755. This tells the compiler that the number is written in octal notation, and the value is then parsed as such. chmod parses all numbers as octal, and does actually handle four digits. Since missing digits are considered to be zero, the first digit is usually not included (or simply as a zero – which will look the same as the representation used in certain programming languages). The first, usually unused digit, have a special meaning, setting the “set user id” (suid), “set group id” (guid) or the “restricted deletion” or “sticky” attributes (you can read more about these options in the manual page).

File permissions

Now that we know what an octal number is, it’s time to look at how the file permissions work. Each file has three sets of permissions, one set for the user owning the file, one set for the group owning the file and one set for anyone else. If you want to take a look at these values on a unix based system, simply type ls -l to list files in a verbose way. Your result will look something like:

-rw-r--r--  1 mats mats        35 2008-08-23 20:24 IMPORTANTFILE

The permissions are listed in the first column, containng “-rw-r–r–“. The first character “-” indicates if the file is a directory (d), if the suid or guid bits are set etc.

This leaves us with “rw-r–r–” – the three sets of permissions. “rw-” is for the user owning the file, “r–” is for the group owning the file and the last “r–” are for anyone else (or ‘other’ as it’s called). The “r” means read, the “w” means write and the currently missing letter is “x”, which means execute (for files) or search (for directories). The “execute” setting is used to let bash (or another shell) attempt to run the file as a script, attempting to parse the first line as a path to the interpreter for the file (i.e. #!/usr/bin/python).

We have three flags (read, write, execute) that can be either on or off. This should remind us of three bits, either being 0 (not set) or 1 (set). And an octal digit is exactly three bits. This means that an octal digit maps exactly to the bit sequence needed to set permissions for a file. A 7 is “111”, a 5 is “101”, a 4 is “100” and so on. Mapping this to permissions:

7 = 111 = rwx
6 = 110 = rw-
5 = 101 = r-x
4 = 100 = r--
3 = 011 = -wx
2 = 010 = -w-
1 = 001 = --x
0 = 000 = ---

When calling chmod 755 on a directory we’re telling chmod to “set the read, write and search bits for me, the read and search bits for the group and the read and search bits for other users” (‘search’ for directories, ‘execute’ for files).

Another example is 644 that maps to 110 100 100, which again maps to “rw-r–r–” which usually is the standard access mode for files (and 755 for directories).

Handling Permissions With Symbols

I’m now going to eliminate the need for remembering everything I’ve written so far in the post, but at least you’ll know what people are talking about when they’re telling you to chmod something this-or-that.

You can also use the symbols directly with chmod, either adding, removing or setting the permissions for one of the three groups.

Examples:

To remove all access for other users (but leaving group and user intact)
chmod o-rwx file

To give everyone read access
chmod a+r file

To give everyone read – and search – access
chmod a+rx directory

To set particular user modes for each group
chmod u=rw,g=w,o=w file (a file that the user can read, but anyone can write to)

And with that I chmod this post a+r.

jQuery, .getJSON and the Same-Origin Policy

When creating a simple mash-up with data from external sources, you usually want to read the data in a suitable format – such as JSON. The tool for the job tends to be javascript, running in your favourite browser. The only problem is that requests made with XHR (XMLHttpRequest) has to follow the same origin policy, meaning that the request cannot be made for a resource living on another host than the host serving the original request.

To get around this clients usually use JSONP – or a simple modification of the usual JSON output. The data is still JSON, but the output also includes a simple callback at the end of the request, triggering a javascript in the local browser. This way the creator of the data actually tells the browser (in so many hacky ways) that it’s OK, I’ve actually thought this through. Help yourself.

In jQuery you can trigger the usual handling of events by using “?” as the name of your callback function. jQuery will handle this transparently and then trigger the function you provided to .getJSON in the first place.

Example

url = "http://feeds.delicious.com/v2/json/recent?callback=?";

$.getJSON(url, function(data) { alert(data); });

There’s an article up at IBM’s developerWorks giving quite a few more examples and information about the issue.

Escaping Characters in a Solr Query / Solr URL

We’re using our own Solr library at Derdubor at the moment, but we’ve only been using it for indexing content. The query part was never standardized in our common library as we usually used an alternative output format, but during the last days that has changed. We now have a parser for the default XML outputter and we’re also supporting facets and field queries (or constraints as they’re abstracted as in our library).

This means that we’re feeding content into the query that may contain foreign characters, in particular those who have special meaning in a Solr query. You can find the complete list of characters that need to be escaped in a SOLR or Lucene query in the Lucene manual.

To escape the characters we use this very simple and stupid PHP method:

    static public function escapeSolrValue($string)
    {
        $match = array('\\', '+', '-', '&', '|', '!', '(', ')', '{', '}', '[', ']', '^', '~', '*', '?', ':', '"', ';', ' ');
        $replace = array('\\\\', '\\+', '\\-', '\\&', '\\|', '\\!', '\\(', '\\)', '\\{', '\\}', '\\[', '\\]', '\\^', '\\~', '\\*', '\\?', '\\:', '\\"', '\\;', '\\ ');
        $string = str_replace($match, $replace, $string);

        return $string;
    }

We used a regular expression first, but the sheer amount of backslashes made it a regular .. hell … to read. So to make it easier for the persons maintaining this in the future, we went the easy to read / easy to maintain road for this one.

Table 100% Width and Margins

While hacking aboooooot today I found the need for making a table behave like a regular block element. I have a section floated to the right of a table, and the table should occupy the rest of the available spot. Making the table’s width 100% would make it adjust it size according to its parent instead of the available place (.. with margin-right set to the width of the other element + whitespace).

The best solution I’ve found so far is to create a wrapper div around the table, and then setting the table width to 100%. This makes the table adjust its size according to the parent – which now is a block element.




...

PHP: Fatal error: Can’t use method return value in write context

Just a quick post to help anyone struggling with this error message, as this issue gets raised from time to time on support forums.

The reason for the error is usually that you’re attempting to use empty or isset on a function instead of a variable. While it may be obvious that this doesn’t make sense for isset(), the same cannot be said for empty(). You simply meant to check if the value returned from the function was an empty value; why shouldn’t you be able to do just that?

The reason is that empty($foo) is more or less syntactic sugar for isset($foo) && $foo. When written this way you can see that the isset() part of the statement doesn’t make sense for functions. This leaves us with simply the $foo part. The solution is to actually just drop the empty() part:

Instead of:

if (empty($obj->method()))
{
}

Simply drop the empty construct:

if ($obj->method())
{
}

Fixing Stuttering Audio With ffmpeg and Quicktime

We’re using ffmpeg to encode videos for flash on several of the sites I’m involved with, and this usually works flawless. From time to time there’s however certain video files that makes something go wrong, usually small issues like stuttering at random places. Today I decided to go bug hunting and try to find out exactly what triggered this behaviour in one of our recent videos.

After quite a bit of debugging and re-encoding the offending video segment (which limits the rate of debug attempts, as you have to wait a couple of minutes ++ for each encode), I decided to try to simply use -acodec copy for the audio codec. The Quicktime file already used AAC as its codec, and the file we were encoding also used AAC. The stuttering disappeared! This indicates that the sound encoding of the process were to blame for the stuttering, so if you’re having a sound related problem, try to copy the input codec to the output source.

As libfaac and libfaad were the two only involved libraries of the encoding and decoding process, the first thing to try were to check if there were any new versions of these libraries available. And lo’ and behold, both libfaac and libfaad had been updated since the versions included in our ubuntu version (no real shocker there, as things in the audio and video codec world moves with a rather high velocity).

I removed the packages (sudo apt-get remove libfaac-dev libfaad-dev) from Ubuntu, downloaded the new libfaac and libfaad versions, compiled (./configure && make) and installed them (sudo make install), and then recompiled ffmpeg with the new libraries in place. ffmpeg then complained about libfaac.so.2 missing, but a quick run of ldconfig (sudo ldconfig) fixed that issue.

Re-encoding the file yet again – and wooho! The offending file now works as it should. This probably also solve the issue we’ve been seeing for several other files too. The new versions of libfaac and libfaad solved the issues we were having.

BTW: There’s a small fix needed to make libfaac compile with gcc4.

Making Spotify, Wine and PulseAudio Play Together

In what has now become a series on this marvellous blog, we’re diving into how to make PulseAudio play nice with Spotify running under Wine. After yesterday’s article about getting any sound through PulseAudio at all, the next issue that surfaced was that the output from Spotify running under Wine were just garbled noise. This worked with the previous ALSA or OSS setup, but with PulseAudio everything went haywire!

Luckily the massive wisdom of teh intarwebs came to the rescue again, this time through an almost three year old post by Paul Betts: “Make Wine and PulseAudio Get Along“.

The solution is to first use padsp winecfg to configure padsp for a specific process, then use padsp to redirect any access to /dev/dsp while the provided application runs.

First run:

padsp winecfg

Then add padsp to your spotify startup script (or if you do this manually, just .. remember to type it. OK?):

padsp wine "C:\program files\spotify\spotify.exe"

Restart Spotify and enjoy the massive collection of great music!

Missing Devices or Sound Card in Pulse Audio

After getting the missing sound control panel in Ubuntu back yesterday, the next problem that turned up was that no actual applications would play any sound – other than the annoying beep! beep! beep!

The reason is that after the pulseaudio configuration has been installed, pulseaudio it self may be missing all it device drivers and other packages.

The solution for this issue can be found at the page for PulseAudio at the Ubuntu Wiki.

Install the missing PulseAudio packages through APT:

sudo apt-get install libasound2-plugins "pulseaudio-*" paman padevchooser paprefs pavucontrol pavumeter

Restart the application that needs sound support and everything should work. If you’re still not getting any sound, check the PulseAudio device configuration (by default installed under Applications, Sound and Video, PulseAudio Device Chooser) and that your device shows up there (mine didn’t before installing the above packages).

When the Ubuntu Sound Control Panel Doesn’t Show Up

After upgrading from Ubuntu 9.0 to 9.10 the other day, I suddenly got a very annoying beep every time I saved a file in NetBeans. Horrible stuff! Disabling it in NetBeans seems impossible (and Firefox makes the same sound when it encounters an error), so removing it in the Ubuntu Sound Configuration Panel sounded (!) like the best option.

One problem: The Sound Control Panel (when selecting System -> Preferences -> Sound) didn’t open. It attempts to load, but then disappears without a trace. I couldn’t find a notice in any of the logfiles either, so my only hope of a remedy was the global debugging power of teh intarwebs.

This was actually harder than I imagined, as I had to wade through large amounts of documentation of how to open the control panel at all. I had found the menu item, but it didn’t work.

Luckily someone had documented several issues at a page named “Sound Solutions for Ubuntu 9.04“, and while I didn’t have the exact problems listed there, they mentioned the required packages for pulseaudio and the GUIs.

Installing the padevchooser package should pull in all the dependencies:

root@ubuntu:~# apt-get install padevchooser

(Use sudo apt-get install padevchooser if you’re not root)

At the same split second the installer finished, the sound configuration dialog opened and I were able to switch to the NO SOUND scheme in Ubuntu. No beeps!

dpkg –configure Error for mysql-server-5.1

After starting the year by upgrading my Ubuntu desktop at work to 5.1 (and spending a couple of hours after borking my new kernel), my mysql-server had become lost in the transition.

Reinstalling the mysql-server package gave a cryptic error message:

Setting up mysql-server-5.1 (5.1.37-1ubuntu5) ...
Installing new version of config file /etc/init.d/mysql ...
 * Stopping MySQL database server mysqld                                 [ OK ] 
Warning: found usr.sbin.mysqld in /etc/apparmor.d/force-complain, forcing complain mode
 * Starting MySQL database server mysqld                                 [fail] 
invoke-rc.d: initscript mysql, action "start" failed.
dpkg: error processing mysql-server-5.1 (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of mysql-server:
 mysql-server depends on mysql-server-5.1; however:
  Package mysql-server-5.1 is not configured yet.
dpkg: error processing mysql-server (--configure):
 dependency problems - leaving unconfigured
Processing triggers for menu ...
No apport report written because the error message indicates its a followup error from a previous failure.
                          Errors were encountered while processing:
 mysql-server-5.1
 mysql-server
E: Sub-process /usr/bin/dpkg returned an error code (1)

Quite weird, yes. Trying to find anything in the mysql-logs didn’t yield anything either (both were empty), but luckily the syslog provided an answer this time. Among loads of other messages regarding mysql the answer hid from general view:

Jan  7 14:30:48 ubuntu mysqld_safe[20249]: 100107 14:30:48 [ERROR] /usr/sbin/mysqld: unknown option '--skip-bdb'
Jan  7 14:30:48 ubuntu mysqld_safe[20249]: 100107 14:30:48 [ERROR] Aborting

Taking a look at the mysql config file in /etc/mysql/my.cnf confirmed this:

#
# * BerkeleyDB
#
# Using BerkeleyDB is now discouraged as its support will cease in 5.1.12.
skip-bdb

Apparently not just the support ceased, but also the skip-bdb parameter. That seems a bit odd – I’d suggest at least keeping the parameter around for another release cycle, but anyhow:

#
# * BerkeleyDB
#
# Using BerkeleyDB is now discouraged as its support will cease in 5.1.12.
# skip-bdb

And suddenly everything went in the right general direction!