The Power of Micro Languages

A “micro language” is a small, domain specific language. These small languages are often invented on a project basis or carried over from previous projects (or in some cases, a standardized language is used). The power of this comes as we’re able to move the rules of the application from the application logic, and instead can maintain a seperate set of rules independent of the application. This way you can add and remove business rules without having to re-test or rewrite your application. The concept has been around for years, and it’s in use more places than you could imagine, but I see people over and over again which does not see the benefits of separating the rules from the application itself.

These micro languages can be described in a markup language (such as XML), as a subset of other existing languages, or as your own simple-to-parse language. In this small introduction I’ll use an example from the current codebase of pwned.no, a site for arranging tournaments in several different online games.

When writing the engine that powers pwned.no I wanted to give the users several different options of tournaments to create. You have the regular single elimination tournaments, where winning a match moves you on to the next level, you have the double elimination events, where your team is not out before they’ve lost two matches, and you have the possibility of a group stage where the best teams (and possibly the worst) move on to the finals. Even after this, you realize that you might have situations where you want to combine these different sets of tournaments into a large one (example: elimination -> group stage -> double elimination). Embedding this logic into the application would create an unmaintainable mess of special cases and special handling, and each time I wanted to add a tournament, the possibility of breaking some other tournament were a possibility.

Instead I decided to implement a small language that describes tournaments, a parser to load and process the language and finally store it in an underlying database structure of how the tournament should progress when the next round starts. This meant that the job up front demanded a bit more planning and sketching, but the solution makes the system more flexible and more stable. When someone requests a new tournament format now, I simply create a small file describing the flow in the tournament, and everything works as it should. No code edits, no unit tests that need to be run, nothing at all, as long as the file is a valid tournament description file.

A tournament description file looks like this (for a tournament with a group stage, consisting of eight teams and with a single elimination stage for the two best teams from each group):

ID GROUP8
TEAMS 8
TOURNAMENTSTAGE 3 FINAL
MATCH
.ID FINALE
TOURNAMENTSTAGE 2 SEMIFINALS
MATCH
.ID SEMIFINALE1
.WIN FINALE
MATCH
.ID SEMIFINALE2
.WIN FINALE
TOURNAMENTSTAGE 1 PRELIMINARY_GROUPPLAY
GROUP
.ID GROUP1
.PLACETOMATCH 1 SEMIFINALE1
.PLACETOMATCH 2 SEMIFINALE2
GROUP
.ID GROUP2
.PLACETOMATCH 1 SEMIFINALE2
.PLACETOMATCH 2 SEMIFINALE1

The ID is an internal identifier which is resolved to a string in the currently active language, the TEAMS identifier tells the parser how many teams who can sign up for this tournament, and the rest of the lines are descriptions of the different stages of the tournament. The file can be parsed top-to-bottom, as the identifiers mentioned later in the file already exists higher up in the hierarchy. The .WIN commands tells the parser where the winning team should move next, and the .PLACETOMATCH commands indicates where the different places in the group stage should move next. If I wanted to add a lower bracket too, I’d simply add .PLACETOMATCH 3 LOWERBRACKET1 and a MATCH with .ID LOWERBRACKET1.

You could of course use XML for this instead, but this very simple and very easy to parse language has so far created a total of almost 25.000 tournaments, and everything has worked without a hitch. After resolving the first issues with the parser in my development version, there are now several new tournament formats that has been added (such as a tournament with 256 teams) in the years after the application was released.

This was just a very small introduction to the concept of micro languages, and you’ll find loads of more information about them online. You may ask what the difference between a micro language and a configuration file is, and the truth is that they’re quite similiar. But as the configuration file describes different settings for the application, the micro language describe different rules for the application. You may of course have configuration files that also contain rules, but you’ll need a well-defined and expressive language to define your rules. The domain where this language is used is so small and limited (there are only that many different concepts that are used in tournaments), so a simple language as this fits. If you’re in a situation where the micro language evolves into a full fledged programming language, you’d probably be better off with embedding an existing scripting language (such as Python), or moving the rules out into seperate modules in your application.

David Cummins on Fulltext Search as a Webservice

David Cummins has a neat little post up about replicating some of Solr’s features in a PHP based solution. His post “Fulltext search as a webservice” should sound familiar to Solr’s approach from the title, and David describes how they built a similiar solution on top of Zend_Search_Lucene (Solr also uses Lucene in the backend). Seems like it would be easier to just set up a dedicated Solr cluster instead, but hey, how often has “it would be easier to do something else” sparked innovation?

I’d also like to note that the coming Solr 1.3 supports php serialization as an output format, so you can just unserialize() the response from Solr. Should provide for even easier integration between PHP and Solr in the future. While on the subject, I’d like to suggest reading Stemming in Zend_Search_Lucene too, an introduction to adding filters to Zend_Search_Lucene. Also worth a look is the Search Tools in PHP presentation from phplondon.

Web Frontends for Xdebug

David Coallier has a post up about a Google Summer of Code project that has been launched to finally get a web frontend for Xdebug, something many people have been requesting for quite some time. Simultaneously Webgrind has been released, another web based frontend that replicates some of the features from kcachegrind in a web based fashion. Here we go several years without a decent front end, and then two gets announced in the same week. Neat! You can check out Webgrind at their Google Code page.

WordPress Usability Issues

After moving from s9y to WordPress, I encountered a few usability issues that I’d like to highlight. While WordPress has a very neat color scheme, large letters and in general does quite a good job of keeping everyone happy, there were a few issues that I stumbled across.

  • The “Save” button is only available at the right side of the post. When entering text in a post, the Save/Publish buttons are placed just to the right of the text field where you write your post. This is fine if I only were to save my post and be done with it, but usually I go through several sets of plugins, tags, categories etc. for the post. After doing this, my eyes and focus are on the bottom of the post column, and I have to searching for how I’m going to get my post posted. This still happens after writing several posts in wordpress.

    How about having just a save and publish button that replicates the function in the rightmost column on the bottom too?

  • After logging into the admin interface, the “Right now!” header with “write a page” and “write a post” is in a reddish (dark orange) background. Red should indicate errors, not “start here, do this!” as the current meaning is. I also wrote a post as a page once, as my eyes just read “write a post” for the first button. I’ll admit that that was me, but as I’ve learnt over the years, if I’m having trouble doing stuff, others will have too. “They’ll figure it out” means they never will.

    How about changing the color of the bar, and instead making the “Write a new post” link to a button all by itself?

    The “Write a Page” button could instead be moved beneath the other button to signify lesser importance. (just noticed that in the “create a new post” page, the bold button is to the right, the opposite of the first login page. Inconsistency is never good.)

  • I registered for an account at WordPress.com to receive my Akismet API key. After entering my details, I got a page with really large text telling me what my username and password was. If I were sitting at a computer lab or at work, people three rows behind me could probably see both my password and my username. I know my password, that’s why I’m writing it two times when registering (.. and then it only shows as ****** as usual…).

    Remove the password from the display page, and we nerds won’t be scared.

    (Am I the only one who finds it a bit akward to see their own password in clear text?)

  • When setting up the text on the right side of this blog, I used a simple “Text” widget from the regular wordpress repository. Works great, but it also have a few usability quirks. I entered my text, and clicked “Change”, and went happily along with my usual business. But the text never changed. Turns out there’s a “save” button further down (which was not visible when the Text widget was expanded). Unless you also press save, the text is actually not changed (even though pressing “Change” indicates that it should be changed).

    You already have a “cancel” link in the upper right corner, but that should probably be placed where “remove” is placed now. “Change” and “Cancel” has opposite meanings but work together, so they should be placed close to each other. Now you have two functions (store the post, don’t store the post) which are shown diagonally across from each other, in completely different context.

    Make the “Change” button actually change the text, and if needed, add an undo mode instead.

  • The last issue is an issue I had with the WordPress.org website. Yet again they use red buttons to indicate “download this!” links. While they do stand out neatly, a friend of mine had trouble actually finding the download link. But it’s right there, in the middle of the page with large red letters. But red isn’t a good color to indicate something we want, and it signifies more “HELP HELP EVERYTHING HAS GONE BANANAS!” than “eeyh! i want to start a new and better blogging life!”.

    My problem was something similiar, but I were lucky enough to already have downloaded WordPress. When trying to install the Highlight Source Pro Plugin for WordPress, I couldn’t for the love of <insert your choice of higher power> find the download link. Guess why? It’s placed as a red button, out to the right over the information about the plugin. I think my mind expected to find passive information there, not a download link. I searched the page visually four or five times trying to find a way to download the plugin, but my mind simply wouldn’t get it. I ended up going to the homepage of the plugin and clicking the “download” link there, which simply triggered the download from wordpress.com.

    Add a download-link in the menu at the top about the plugin and add a download button/link at the bottom together with the tags for the plugin.

    (after reading about the plugin, you usually want to download it or move away, the last option you’ll just use your back button for or scroll to the top to find something else on the same site).

I’m by no means an usability expert, but these were just my small annoyances when trying to use WordPress from scratch. I’ve tried it out a couple of times before, but this is the first time I’ve installed the 2.5 branch.

The Story of Migrating From Serendipity (s9y) to WordPress

As I posted a couple of days ago, I’ve moved the blog from Serendipity to WordPress. The reasen for this was detailed in the other post, as I simply weren’t too happy about how s9y worked in regards to pingbacks/trackbacks and the general quality of the code. Anyways, this is a general post for those who wish to make the same move, and are wondering about how.

After installing WordPress, delete all the existing posts in the database and reset the AUTO_INCREMENT value (needed if you want to keep your old links working). If you do not do this, your ID values may be skewed by 2 or more. Download and install the Serendipity importer for WordPress. After installing it you’ll have a new option under “Manage” -> Import that enables you to import your posts from s9y. Do the import, and check that your posts got the same IDs as they had under s9y.

Then you’ll have to do some fiddling with the .htaccess file, so that mod_rewrite knows about your old URL scheme. These rules worked for my installation of s9y, your milage may vary (and please, if you find any problems, post a comment or a pingback to your fixes). Add the following lines together with the existing WordPress rules:

RewriteRule ^archives/([0-9]+)\-[a-zA-Z\-_]+.html$ /index.php?p=$1 [L]
RewriteRule ^feeds/index.rss2$ /feed/ [L,R=301]
RewriteRule ^feeds/index.rss1$ /feed/ [L,R=301]
RewriteRule ^feeds/index.rss$ /feed/ [L,R=301]

This should also keep most of the RSS-readers out there happy, but you might want to change the feed addresses if you have several different formats of the RSS available. This does not include the category/ etc. links from s9y, but those will be reindexed at a later time by any search engine anyways. The most important thing is to ensure that people who are looking for your old posts still are able to find them.

PHP 5.2.6 Released!

Ilia just noted that PHP 5.2.6 was released yesterday! Lucky for me, as I have been away for a couple of days at a trackday in Sweden (pictures coming to my flickr page within the night). Among notable fixes are added metadata for the reflection API in the DOM classes and a few PDO-fixes for PostgreSQL. Several security vulernabilities were also fixed, and a complete list can be seen in the PHP 5.2.6 Changelog.

Moving to WordPress from Serendipity (s9y)

Oh well, I spent the last evening moving my blog from Serendipity (s9y) over to WordPress. The reason for this is that after Christer and I talked a bit, we discovered that something wasn’t working as it should with the trackbacks and pingbacks. As every good programmer does, I headed into the source instead of reading anything sensible about the subject, and well.. I wasn’t really impressed. As s9y is still catering to those who need PHP4 support, the code was a mess of functions here and functions there and no real separation, including Global variables and other messy things. WordPress isn’t much better in the “silly amount of functions” area, but I’ll leave that for the next blogpost or two about WordPress issues that’s getting on my nerves.

Anyways, after seeing their XML-RPC implementation .. which was a couple of large preg_match statements trying to look for the XML-RPC request, I called it a day and decided this wasn’t fixable to bring it up to my need. I’d very much like to patch it up and submit my own XML-RPC server for them to use, but it’s based on PHP5 and the reflection API, so no go there. All that frustration ended in installing wordpress yet again, so here we are, after spending the night of getting everything up to speed and looking like it was again. I’ll create a post later for those who want to take the same route, and I’ll include the RewriteRules to make all your old links still work in WordPress.

Orto – A Java VM Implemented in Javascript

Meant to publish this earlier, but it’s just been sitting in a tab in my browser for a day now. Anyways, John Resig (hm, familiar domain name / real name combination there…) has a post about Orto from Japan, a compiler that transformers Java bytecode to JavaScript statements . The end result is Java applications that do not depend on Java, but can be run with just JavaScript available in a browser. Not quite sure what I’d want to use this for, but on a scale for awesome, this ranks pretty high! John’s page has screenshots and links to several more interesting sources about Orto and is well worth the read.

77 Things To Do With Your iPod (.. other than just listen to music on it)

Travelhacker has an ingenious list with 77 alternative things to do with that old iPod you’ve just been ignoring for a while. If you’ve ever wondered how to convert your iPod into a sex toy, create an iPod Taser (iTaser?) or create your own iPod based pirate radio station, this guide is for you.

Marco’s Five PHP5 Features You Can’t Ignore

Marco has a neat list up with five different features about PHP5 which people are still not quite catching on to , and I agree with every single item that made it to the list. I’ve been using SimpleXML myself, and except for a few cryptic issues regarding namespaces and iterators (SimpleXML does quite a bit of magic..) it’s a breeze to work with. For simple .. XML .. parsing, it’s ingenious.

For PDO, we’ve already moved all our projects to PDO, and it’s been my preferred method of accessing databases for at least a year and a half already. Great stuff. We’ve also used the json module for quite some time, which neatly ties into jQuery , mootools and other JavaScript APIs. I still haven’t used SPL that much, but that might change soon. Anyways, a good read for anyone who still live in the PHP4 world..