Patching Zend_OpenId To Maintain Error Messages

To help anyone else in their quest to implement Zend_OpenId in their code, I’ve created a small patch against trunk which adds error internal error messages in most places where there’s a return false;. This should help when debugging your code, both if you’re implementing an authentication service and use Zend_OpenId as a consumer or if you’re implementing just the consumer part of the equation.

I’ve uploaded the diff: Consumer.php.patch (Consumer.php.patch.txt).

The only thing that seemed a bit weird was that one of the error messages that were present in 1.6.0 had been removed in trunk:

if ($status != 200) {
    this->_setError("'Dumb' signature verification HTTP request failed");
    return false;

I’ve readded this message in my patch, but I feel that the error message should be something like:

$this->_setError("The server responded with HTTP status code: " . $status);


Oh well. Hopefully someone will find this useful.

Adventures in OpenID and Zend Framework

I’ve been toying around with OpenID and the Zend Framework for a night or two now, and I’ve made a few experiences I thought I should share with the intarwebs (now, that’s probably the point where you should make the decision to stop reading for most blog posts). Quite some time has passed since I last had anything to do with OpenID, so just getting up and running was a challenge.

An OpenID identifier is usually represented by an URL (such as<login>), which the OpenID consumer then contacts to get information about how to communicate with the OpenID identity provider (Yahoo! in this case). The consumer contacts the provider, gets an URL to redirect the client to, and receives notice after the client has authenticated with the provider.

First I’d like to say that OpenID seems to be too hard to use for any other than those who have a particular interest in it. I have a Yahoo! account and a Google Account, which both can be used for OpenID authentication. I have no idea how I use my Google Account for this, without having to provide endpoints manually. Ugly.

I did at least get the Yahoo! authentication working, but I’m still undecided on wether I’m going to implement OpenID support in any current project. Possibly. We’ll see.

Anyways, my implementation in Zend Framework is mostly a copy of the tutorial in the ZF manual, but there is one important point that they do not mention: In the standard installation, you have to use Zend_Session to handle your sessions. That means calling Zend_Session::start() instead of session_start(), as Zend_Session cannot be used after a session has been started. This dependency kind of killed my enthusiasm, as we just pull the parts of Zend Framework that we need into our project as thing progresses. Changing how we use sessions is a bit too much to ask. Luckily you can still use $_SESSION as usual after starting Zend_Session, but sitll. Not too fond of that. I hope that it will be decoupled some time in the future..

Testing code:

require_once 'Zend/OpenId/Consumer.php';

$consumer = new Zend_OpenId_Consumer();

if (!empty($_POST['openid_identity']))
        die("OpenID login failed.");
        print('We logged in!');

if (isset($_GET['openid_mode']))
        case 'id_res':
            $consumer = new Zend_OpenId_Consumer();
            $id = false;
            if ($consumer->verify($_GET, $id))
                $status = "VALID " . htmlspecialchars($id);
                $status = "INVALID " . htmlspecialchars($id);


        case 'cancel':
            print("someone pressed cancel!");

Switch out $_POST[‘openid_identity’] with your OpenID identifier (the whole URL), and you should be all set.

If you keep getting failed logins without a redirect, check that you have https support in PHP through openssl (the module is named php_openssl). Zend Framework provides no hint that this can be a problem, but after stepping through the source (I’m test driving NetBeans 6.5) the solution became apparent.

PHP and Annotations

After hacking together some code to solve an issue that came up on an IRC channel I’m up today about how to provide a URL mapping for individual methods — and keeping the responsibility in the method itself, I stumbled across addendum. Addendum implements annotations parsing for PHP and works by using the reflection API in PHP 5.1+. This allows you to add annotations which indicate to your framework which methods should be exposed to the web and which should be kept private. There are loads of other ways to do this (both dynamically and statically), but this is one way that may appeal to someone.

PHP Namespaces and the What The Fuck problem

As many people has discovered during the last days, some of the developers behind PHP has finally reached a decision regarding how to do introduce namespaces into PHP. This is an issue which has been discussed on and off for the last three years (or even longer, I can’t really keep track), with probably hundreds of threads and thousand of mail list posts (read through the php.internals mail list archive if you’re up for it). The current decision is to go with \ as the namespace separator. An acceptable decision by itself, and while I personally favored a [namespace] notation, I have no reason to believe that this is not a good solution.

There does however seem to be quite a stir on the internet in regards to decision, which I now will call the “What The Fuck Problem”. Most (if not all) public reactions on blogs, reddit, slashdot and other technology oriented sites can be summed up as “What The Fuck?”. While I’m not going to dwell into the psychological reasons for such a reaction, my guess is that it’s unusual, lacks familiarity for developers in other languages already supporting namespaces and that it might be hard to understand the reasoning behind such a decision.

The problem: to retrofit namespaces into a language which were not designed to support such a construct, without breaking backwards compability. The part about not breaking backwards compability is very important here, as it leaves out everything which could result in a breakage in existing code by simply using a new PHP version.

The discussions have been long, the attempts has been several (thanks to Greg Beaver’s repeated persistence in writing different implementations along the way) and each and every single time an issue has crept in which either breaks existing functionality or results in an ambigiuity when it comes to resolving the namespace accessed. Most issues and the explaination why they are issues, are documented at Namespace Issues in the PHP Wiki. This provides some insight into why the decision to use a separate identifier was chosen.

This seemed to get through in the flamewars discussions after a while, and people instead started to point out the “gigantic flaw”:

If you were to load a class or a namespace dynamically by referencing it in a string, you’d have to take care to escape your backslash:

spl_autoload_register(array("Foo\tBar", "loader"));

.. would mean Foo<tab>Bar. Yep. It actually would. And if that’s the _biggest_ problem with the implementation of using \ as a namespace separator, then I feared its introduction earlier for no apparent reason.

The other examples has plagued us with ambiguity issues, autoloading issues, no-apparent-way-of-putting-functions-and-constants-in-the-namespace-issues and other problems. This way we are left with the task that we, as usual, have to escape \ in a string — when we want to reference a namespace in a dynamic name — and that’s the biggest problem?

I just hope that people keep it sane and don’t implement any special behaviour in regards to how strings are parsed in regards to new, $className::. This _will_ cause problems and magic issues down the road if it ever gets into the engine.

PHP is free, it’s open and it’s yours for the taking. Fork it if you want to, or provide a patch which solves the problem in a better way. The issue has been discussed to death, and so far there is no apparent better solutions on the table. If you have one, you’ve already had three years to suggest it. You better hurry if you’re going to make people realize it now.

Additional observation: most people who has an issue with this, seems to be the same people who would rather be caught dead than writing something in PHP. Yes. Python and Java does it prettier. That’s not a solution to the problem discussed in regards to PHP.

Java printf and Booleans

The available stdout mapping in Java, System.out (PrintStream), supports printf (to format a string when outputting it) in the same manner as other languages such as C. By using System.out.printf instead of System.out.print, you can also provide a formatting string which the method can use to write the output in a custom format.

As Java supports several other native datatypes that weren’t available as regular C-types, you also have a few other options. One of these are the %b / %B identifier, which represents a boolean value.

The format string “%b” expects one argument, a boolean. This will be written as either ‘true’ or ‘false. You would (as of writing) get the same result by using the %s identifier as this would cast the boolean of a string, which in turn would give true or false, but it’s always better to be explicit about what you are doing than assuming that people understand that you understood something (which even could change..).

Keeping Your Code in Check: Part 1 – DRY

DRY – Don’t Repeat Yourself – is a central principle for everyone who wants to keep their code maintainable and in a clean and pristine state. It will save you from tedious tasks in the future, although it requires you to do a bit more thinking up front. The clue is to recognize when you’re about to commit the first programming sin: repeating parts of code.

Lets first be completely clear: not repeating code does not mean your code should be as general as possible. Doing that will only end up in a horrible mess that your only hope of selling is to label it “Enterprise” and push all the configuration options into XML. If your way to success is to craft a new buzzword and get the consulting business to sell your product, that might be how you do it. If you’re interesting in writing maintainable and solid software that other people can play around with and get the grip of quite quickly; don’t.

Keep it specialized to do what you need it do easily, but keep it general enough to allow you to reuse it for similar tasks.

The reason why I’m posting this now is unsurprisingly enough that I ran into this issue while coding a library at work. This issue is so common that any programmer will run into it several times a day, but the issue at hand today illustrates the problem perfectly. In a class that I was writing to return a set of data to the user, I decided to add three methods for sorting the data in different manners before returning it. The problem is that the data is kept in different array keys, and it’s the arrays themselves I want to sort. Example:

['displayValue' => 'Aloha Beach', 'count' => 13, 'value' => 'aloha']
['displayValue' => 'Norway', count => 3, 'value' => 'norway']
['displayValue' => 'Sweden Swapparoo', count => 7, 'value' => 'sweden']

To sort this in PHP, you’d use the usort function with a callback function / method that sorts the arrays by the value of the right key. My first version was something like:

private function sortByValue($a, $b)
    if (!isset($a['value'], $b['value']))
        if (isset($a['value']))
            return -1;

        if (isset($b['value']))
            return 1;
        return 0;

    return strcmp($a['value'], $b['value']);

private function sortByCount($a, $b)
    if (!isset($a['count'], $b['count']))
        if (isset($a['count']))
            return -1;

        if (isset($b['count']))
            return 1;
        return 0;

    return ($b['count'] - $a['count']);

As I typed the second if (isset()) in the second method, I finally realized that I were sitting at work and writing the exact same function twice, with just a little twist between them in regards to which field to sort by – and how to determine the sort. One field was numeric, the other was a string. Our goal is to re-use as much as the common code from both methods without typing it up — or much more important, maintaining — it twice.

As you can see, almost the whole function is identicial, except for the key name (‘value’ vs ‘count’) and how the comparison is done (numeric vs string). We need to handle these two issues to be able to use the same function for both purposes, so we rework the two functions into one function for general use for sorting on the value of a key:

private function sortByArrayField($a, $b, $field)
    if (!isset($a[$field], $b[$field]))
        if (isset($a[$field]))
            return -1;

        if (isset($b[$field]))
            return 1;
        return 0;

    if (is_numeric($a[$field]) && is_numeric($b[$field]))
        return ($b[$field] - $a[$field]);

    return strcmp($a[$field], $b[$field]);

This way we handle both the comparison method (if both values are numeric, we do the comparison as a numeric value by simply subtracting the first from the second) and the field to sort (as a third parameter).

Sadly usort does not provide that third parameter for us automagically, but by creating a few simple helper methods for using our refactored function, we get all “configuration” set in those three methods. The code base only contains one implementation of the method itself, but several ways to use it. Example of these three helper methods:

private function sortByValue($a, $b)
    return $this->sortByArrayField($a, $b, 'value');

private function sortByCount($a, $b)
    return $this->sortByArrayField($a, $b, 'count');

private function sortByDisplayValue($a, $b)
    return $this->sortByArrayField($a, $b, 'displayValue');

If someone now comes along and decides to add another column to our array, such as ‘price’, we can simply add another sorting callback to accomodate this:

private function sortByPrice($a, $b)
    return $this->sortByArrayField($a, $b, 'price');

The sortByArrayField method is already well proven and tried from our previous usage, and by simply changing the field we’re sorting by, we still get the power of the callback, get a completely new sort criteria and the maintainability of just one method.

The First Rule of Debugging

Seems like there are quite a lot of ideas about what the first rule of debugging are, and a quick Google Search gives you insightful suggestions such as:

These are all valuable insights that provide clear value about what and how you could attack the problem of the Bug That Wasn’t Supposed To Be There (there are surely bugs that actually were supposed to be there, because of invalid domain specifications, etc.). I did however find one small column from DDJ that led into the same thing I’m going to write now: “Before you go to fix it, be sure you’re fixing the right thing.

I will assume that you already have uncovered that you actually have a bug — you’re not getting the results you expected, and something is to blame. You start out by trying to isolate the problem, and try to recreate the broken situation with different inputs. You try it on another system to see if there’s something about the configuration, about the dataset, about the versions of your library or something, which we’ll disover later, is completely irrelevant.

I obviously forgot this rule at work today, and that’s why I’m writing this post now. Remember kids. Always make sure you’re fixing the right thing! I’ve started porting one of the front end services we use to federate searches from a borked Java library to a PHP-based implementation instead. This involves talking to an internal search server that I hadn’t written code to interface with before, other than the existing search service. The implementation went from zero to usable in almost no time, and I now have code that provides a nice foundation for the remaining work. The problem is that I obviously decided to fix another issue with the search server that I had looked into earlier, and it seems that I thought that since I now had new and improved experience with the service, I’d be better off this time.

So I set out trying to get a new sorting scheme working for the search results. At first I failed, but then I stumbled across some documentation that threw me in the right direction. I did a few minor changes, and the sorting changed. Woohooo! Well, it almost worked. The order was not quite as expected, but the results seemed to be close enough for the proximity search to be working. It just had to be something in the documentation or implementation that I had missed! Further digging and experimentation for the greater part of an hour or so left me blank, but I managed to get another search ordering which made me think that there was something peculiar about my input parameters that created the issue.

After almost two and a half hour of this, I happened to read four lines that were neatly tucked away in a “process” part of the resulting XML document from the search service. In the diffuse light glooming from my Dell monitor, I could read the now obvious words: “<subsystem> No license.”.

Yep. It never had anything to do with the actual result. It was never even active. The change of the sorting must have happened for some other reason (such as removing the default values or anything like that). I weren’t supposed to debug why the sorting was incorrect, I was supposed to find out why the subsystem didn’t load.


Mats’ (borrowed) first rule of debugging: Make sure you debug the right problem. Do not make assumptions.

(and a rule of documentation: if the feature may not be available in the installation, write how you can check this before actually giving examples and detailing how to use the feature)

Informa and Custom XML Namespaces in RSS

While integrating a custom search application into a Java-based web application, I came across the need to access properties in custom namespaces through the Informa RSS library. Or to put it in another way; i needed to access to properties, Informa had been used for RSS parsing in the previous versions of the web application. The people who developed the original version of the application had decided to extend the Informa library into their own version, and had added several methods for .get<NameOfCustomProperty> etc. After thinking about this for approximately 2 seconds, I decided that having to support and modify a custom version of Informa was not the right track for us.

My initial thought was that their decision to customize Informa to support these methods had to come from the idea that Informa did not support custom namespaces out of the box. I did a few searchas over at Google, and found nothing useful. Reading through the documentation for Informa didn’t do me any good either, so I tried to find an alternative library instead. Did a bit of searching here too, and stumbled across a hit for one of the util classes for Informa (.. again). This did support custom namespaces, so the backend support was there at least. Then it struck me while reading the documentation for Informa and ChannelIF again; Informa did support it, as it inherited the methods from further up in the hierarchy. The getElementValue and getElementValues methods of the ChannelIF and ItemIF classes allows you to fetch the contents of elements with custom namespaces in a very easy to like manner.


This simply returns the string contained between <exampleNS:field> and </exampleNS:field>

Hoooray! We now have support for these additional fields, and we do not have to keep Informa manually in sync with the version in our application. Why the original developers decided to fork the Informa library to add their own properties I may never know, but I’ll update this post if they decide to step forward!

Unit-Testing Code Which Uses a Database

How to unit-test code that interacts with a database appeared on the blog of Baron Schwartz, and to be really boring, I agree with what he’s writing. Unit-testing database connectivity and storage is not hard. If it is, it might be a good time to redo that architecture you’ve been talking about.

An important point that Baron mentions is that you _NEVER_ _EVER_ run your tests on your production servers. That will of course be disastrous, as your tests needs a predefined state of the database to be valid for testing. The solution I’ve been using to handle this, is to always set up my environment to use another database when doing the tests. This way, you’ll never end up with running the tests on a live database by accident. I handle this in my AllTests.php file, where the test suites and shared fixtures are set up. We dump the contents of the developer database (databasename), create a new database (databasename_test) and insert all the current table structures and indexes. This way we get an accurate copy of the table definitions currently defined by the developer (so that we don’t run the tests against an old set of tables), and we test that the code works as it should with the active definitions.

The simplest way to do this, is to use mysqldump and mysql through a call to exec. If you’re not in a trusted environment, please, please, please add the appropriate shell argument escape commands. It can however be argued that if you’re allowing random people to change your database login information, you probably have bigger problems than doing unit testing..

exec('mysqldump -u ' . $username . ' -p' . $password . ' ' . $dbname . ' | mysql -u ' . $username . ' -p' . $password . ' ' . $database . '_test');

It would be very interesting to get more information about which measures Baron advocates for detecting a production system. We have configuration settings for our applications which also defines if this is a development or production system, in addition to the fact that our testing code only touches databases which end in _test.

The Results of our Weekend Challenge

The weekend PHP size competiton I mentioned on friday has come to an end, with the results being as follows:

      CNU (253 bytes)
      Helge (261 bytes)
      Ymgve (267 bytes)
      Me (365 bytes)
      dibon&zep (which never delivered, but had working solutions)

For those who are interested in the strategies and tactics employed by the contestants, I’ve included a small write-up and analysis of the various contributions. We decided early on that using eval() and gzinflate on the content were something that everyone could apply, so the size would be counted for the decompressed code. If any contestant implemented their own decompressor in the user space code themselves, we would accept that. It would not be any advantage anyways, as the decompressor code would eat up more space than the best contributions used in total.

I’ve included the writers own comments if they had any. The biggest change in size happened when people started using one dimensional strings instead of arrays.

CNU’s contribution

';if($d==' ')$q[]=$x;$d=$c;}$q=$b='';}$b.=$l;$w=strlen($l);}

Helge’s contribution


Helge also included an annotated version with comments in norwegian:

 * $l = én linje (av maps.txt)
 * $m = ett map
 * $w = vidden til et kart
 * $d = distansen til et vilkårlig punkt
 * $v = open/closed-array (putter både squares 
        of interest og visited squares her)
 * $u = "current square", utgangspunktet til å 
        generere naboer
 * $n = nabo
 * $x = hjelpevariabel for å generere $n
 * $y = hjelpevariabel for å generere $u

/* Klarte aldri å finne en måte å skippe initialisering av 
   $m her.. (den resettes på siste linja og.. føles waste)
   * En idé var å bruke to for(;;)'s istedenfor en foreach 
   og continue, men jeg tror jeg hadde økt i kode da.. */

/* Foreach var nyttig fant jeg ut.. den stopper når filen er 
   ferdig lest av seg selv, så slipper kode på sjekking av det, 
   $f[$y++], osv. */
foreach(file("maps.txt") as $l) {
        /* Når $l=="\n" (som betyr slutten på et kart) skal 
           vi slutte å appende til $m og heller breake if'en 
           så vi kan starte pathfinding */
        $l != "\n" &&

        /* Bruker en 1-dimensjonal string istedenfor array 
           (= win!) */
        ($m .= $l) +

        /* Trenger vidden når jeg skal regne ut naboene til et 
           gitt felt. */
        $w = strlen($l)
        /* Gjør så $l blir feedet en ny linje av kartet vårt til 
           kartet er ferdig */

    /* Pathfinding start! */
        /* Setter $y og $d[starten] lik 0, og putter startposisjon 
           inn i $v i samma slengen. */
        $y = $d[ $v[] = strpos($m, '.') ] = 0;

         * Gjør så for(;;) breaker når sluttposisjon-$u blir 0 
         * (synonymt med $u == strpos($m, 'X');
         * Tar også neste $u fra $v i samma slengen */
        strpos($m, 'X') - $u = $v[ $y++ ];
        /* For hver nabo */
            /* Genererer naboer.
             * Den starter på -width (som blir opp), går så til -1 
               (som blir venstre), 1 (som blir høyre) og tilslutt 
               width (som blir ned) */
            $x = -$w;
            $x <= $w;
            $x += $x+1 ? $w-1 : 2
            /* Om naboen (summen av $u og $x) er i $v: continue */
            in_array($n = $u+$x, $v) | /* fancy bitwise or som kan 
                                          brukes fordi det er snakk 
                                          om 2 booleans */

            /* Om naboen er en '#' på kartet: continue */
            $m[$n] == "#" ||

            /* Hvis ikke, sett distansen til naboen lik distansen 
               til $u + 1 og putt naboen inn i $v */
            $d[ $v[] = $n ] = $d[$u]+1;

    /* Skriv ut distansen til $u (skal være sluttpunktet nå) */
    echo "$d[$u]\n";

    /* Disse må resettes.. */

Ymgve's contribution

foreach(file("maps.txt")as$l){for($i=0;-75<$e=ord($l[$i++])-88;$m[]=$e+56?$e:1e6,$z=array(-1,1,-$i,$i))$e+42||$q[0]=count($m);if($i<3){for($d=$k=$c=0;;$c++-$k||$i++&$k=$d)for($j=4;$j;$g>$i&&($g=$i)&$q[++$d]=$p)if(!$g=&$m[$p=$q[$c]+$z[--$j]])break 2;echo"$i

My own contribution

Breadth-first Search, Delivered Version, 365 bytes:


This is mostly your usual run-of-the-mill BFS, where the search itself is implemented as a simple queue.

We loop through all the lines in the file (observe that you can remove the space character on both sides of as, saving you two bytes!):


The part within the else condition builds the map to search for the exit point:

    // for each line in the file, we do this

The for() loops through each character on the line, until it hits the newline marker at the end. Instead of entering "\n", you simply use the actual newline. This way you save another byte. The map ($m) is then populated using the current y,x coordinate (y being the line, x being the character on that line).

$m[$y][$x]=$l[$x]=="#"?0:$l[$x] can be expanded to:

if ($l[$x] == "#")
    $m[$y][$x] = 0;
    $m[$y][$x] = $l[$x];

Meaning that if the character is a wall, we store the value zero; if any other value, we store the character itself. We'll use this attribute when searching to determine when we've reached our goal. All of this is kept inside the for()-construct itself. The only thing contained in the for()-loop, is the statement if($l[$x]=='.')$q[]=array($y,$x,0);. This checks if the current character is the starting point, and if that's the case, adds it to our queue of points to check (making it the starting point of our BFS).

When we hit an empty line in the file (through the if("
"==$l) check), we execute our BFS.

    /* while $q is valid (this handles empty labyrinths), we fetch the 
        current y and x coordinates, in addition to d, the distance to 
        the location we're fetching from the queue. No {} here, as the 
        for only contains the if() statement below. */

        /* if this spot has a value (remember that we assigned 0 to walls, 
            while we kept the character value for other fields when we 
            parsed the file. We use empty() as the coordinates may not be 
            valid indexes into the array, as we don't do any checks when 
            we add them. one call to empty() is better than four ifs and 
            duplicate array references). */
            // create a reference (saves one byte instead of referencing $m[$y][$x] twice.

            /* if we've found the X, echo the distance to the X. Under any 
                under circumstance, you'd break; out of the for/while here, 
                but as bytes count more than speed, we save six bytes by 
                searching the whole labyrinth instead of ending early. Funny 
                detail: as echo is a language construct, you can skip the space. 
                If the character is not 'X', we simply echo an empty string 
                instead of using a conditional echo. Saves a couple of bytes. */

            // unset the value, so that we don't visit this node again.

            /* add all neighbours to our queue. We pre-increment $d, instead 
                of adding +1 in all the adds. */
        /* Reset the values. PHP generates a warning if a value has never
            been set before when used, but will happily accept null values 
            as empty arrays or 0. The double assignment is also a trick worth 
            remembering to save bytes. */

Some of tricks used in the competition

  • Multi-variable assignment: $a=$b=0;
  • Actual newline when detecting a newline: if ($a=="
  • Using null to be able to reset both integers and arrays in one statement.
  • The more code you can fit into a for() statement, the better. You get three "free" line endings by doing this. for(;;) is the same length as while(), but you can get several statements evaluated.
  • You can skip the spaces in foreach().
  • Create references to avoid using long array indices several times.
  • The ternary operator (?:) is your friend!
  • Remember that {}'s can be dropped when you have a single statement that contains all subsequent code (as long as you don't get any conflicts with else etc. on different levels):
    for(;;) <-- no { here.
    if ()


  • Simulating arrays with strings

The scripts were run with a set of pre-made labyrinths and a collection of 20 random labyrinths (more like maps, but they still test the code the same). The script for creating the labyrinths is available here. All delivered contributions passed all tests.