Parse a DSN string in Python

January 31st, 2011

A simple hack to get the different parts of a DSN string (which are used in PDO in PHP):

  1. def parse_dsn(dsn):
  2.     m = re.search("([a-zA-Z0-9]+):(.*)", dsn)
  3.     values = {}
  4.    
  5.     if (m and m.group(1) and m.group(2)):
  6.         values['driver'] = m.group(1)
  7.         m_options = re.findall("([a-zA-Z0-9]+)=([a-zA-Z0-9]+)", m.group(2))
  8.        
  9.         for pair in m_options:
  10.             values[pair[0]] = pair[1]
  11.  
  12.     return values

The returned dictionary contains one entry for each of the entries in the DSN.

Update: helge also submitted a simplified version of the above:

  1. driver, rest = dsn.split(':', 1)
  2. values = dict(re.findall('(\w+)=(\w+)', rest), driver=driver)

Introducing Mismi – Amazon Price Comparison for Norwegian Customers

January 10th, 2011

My main project in December was Mismi – a service that compares the total price of items from Amazon.com and from Amazon.co.uk for Norwegians. The solution is built on top of the Zend_Service_Amazon class (with a few extensions of my own).

The reasoning behind making the service is that there are several factors that are in play when deciding whether to order a product from the US or from the UK: the exchange rate for GBP and USD, the shipping cost, the delivery situation for the item and whether the item is sold in the store at all.

The user enters a list of the URLs to the products they’re considering purchasing from an Amazon-store, press submit and get a list back of which items are in stock, where the item is the cheapest and what the total sum of an order placed at the store would be. In addition I added a alpha stage feature just before Christmas which will also tell you the “optimum” set of items for the orders – “order item 1,4,7,9 from .com, item 2,3,5,6,8 from .co.uk”. This took quite a bit of hacking – you also have to consider the initial price of shipping, shipping for each item and other fun things.

Feel free to play with it over at mismi.e-mats.org. It’s in Norwegian, but it should be easy to understand anyhow with the description above.

gnome-web-photo, Ubuntu and “Could not get gre path!”

October 26th, 2010

Another issue I came across in my current adventures in gnome-web-photo land was that gnome-web-photo just stopped running after upgrading up through four different Ubuntu releases. gnome-web-photo crapped out with an error message about the GRE path being wrong, and the only resources on the web were commit messages from when the error message was changed and committed to the gnome-web-photo project.

Digging, digging and a bit more digging led me to suspecting that xulrunner wasn’t behaving as it should. stracing the gnome-web-photo binary (named gnome-web-photo.real on Ubuntu, as gnome-web-photo is a simple wrapper script) revealed that it was attempting to load a difference version of xulrunner than what actually were present (it tried fstat-ing a non-existant directory).

Checking out the current configuration of gre in /etc/gre.d (.. where I’ve never ventured before) indicated that DPKG had left the new package configuration without switching it to the standard configuration. This meant that the old GRE configuration was used, and not the new one. Renaming the old configuration file to gibberish and then removing the suffix from the new one solved the issue.

  • Removed: 1.9.1.14.system.conf
  • Renamed: 1.9.2.11.system.conf.dpkg-bak to 1.9.2.11.system.conf

.. and now gnome-web-photo actually works again!

gnome-web-photo and GConf Error: Failed to contact configuration server

October 26th, 2010

This error was quite hard to track down, and was also the cause for gnome-web-photo timing out. Things worked when running from the shell, but when running it through our web service (as another user), gnome-web-photo went misbehaving!

Reports on the intarwebs suggested changing ownership of the .dbus directory (so dbus can work properly), but as we don’t have a regular home directory for the user (we actually have to set a fake one to fool mozilla when it’s creating its .mozilla directory) we didn’t have any .dbus directories.

After toying around and getting nowhere fast, I finally had a bit of luck with just running dbus-launch before gnome-web-photo. This worked once, then things went back to being crap. After reading through the dbus-launch man page I finally found out that you can make dbus-launch set up the environment for proper dbus access and then launch a command line you provide.

End solution:

  1. dbus-launch /usr/bin/gnome-web-photo -t 30mode=photo –format png ..

Space, Page Up, Page Down, Home, End Broken for Navigation in Firefox

August 18th, 2010

While I initially thought I had gone crazy or were still dreaming, my fear was true – all my keyboard page navigation had stopped working in Firefox (3.6.3). I tried to move onto to the next by pressing space or page down, my hands trembled as I discovered that neither arrow keys or home / end keys worked. The friendly internet suggested that I should turn Caret Browsing (F7) off (.. so try that if you’re in this situation), but that just told me that I were turning Caret Browsing on (with a proper warning). No luck.

After a bit of tinkering I disabled all my add-ons (previously known as extensions), and voilá, everything worked as it were supposed to! Further problem hunting revealed the culprit: The Del.icio.us Bookmarks Add-on. Curse you, del.icio.us! Disabling the Add-on and re-enabling all the other addons solved the issue, and I’m now happily browsing the internet one page size at the time again.

Trac and “ClearSilver not installed” message

July 10th, 2010

After updating our debian installation on one of our servers we were left with a non-functioning Trac installation. An exception were thrown with a message about ClearSilver suddenly missing from our installation.

Solution: Debian packages ClearSilver ready for python:

  1. apt-get install python-clearsilver

.. and party like it’s 2010!

Install rdiff-backup and librsync on RedHat Enterprise Linux 4

July 7th, 2010

We’ve just installed rdiff-backup on some of our older RedHat Enterprise Linux 64-bit servers. We decided to install the .tar.gz-distributions and you can build your own rpms from these source packages (I won’t cover that here). There are a few pitfalls you’ll have to avoid.

First: librsync needs to be built with the 64-bit lib directory as the standard linkage, and needs to be compiled as a shared module.

To do this, provide configure with a few hints:

  1. ./configure LDFLAGS=-L/usr/lib64 --enable-shared --enable-lib64

Then make and make install

Run ldconfig to be sure that your LD cache has been updated (make install should do this, but .. just to be sure.)

The “.. can not be used when making a shared object; recompile with -fPIC” error for librsync will occur if you build it without –enable-shared, while “/usr/lib/libpopt.so: could not read symbols: File in wrong format” will occur if you build the library with the standard 32 bit libraries.

After this rdiff-backup compiled as it should:

  1. python setup.py install

After rdiff-backup builds it python module and other small items, you might have to run ldconfig again (if you get “ImportError: librsync.so.1: cannot open shared object file: No such file or directory”, try this):

  1. ldconfig

If you still get the above error, check that the directory where librsync was installed is present in either one of the files in /etc/ld.so.conf.d/ or in /etc/ld.so.conf itself. After adding the path to the library directory (which by default will be /usr/local/lib), run ldconfig again.

rdiff-backup should now (hopefully) work as expected.

Solr, Tomcat and HTTP/1.1 505 HTTP Version Not Supported

May 19th, 2010

During today’s hacking aboot I came across the above error from our Solr query library. The error indicates that some part of Tomcat was unable to parse the “GET / HTTP/1.1″ string – where it is unable to determine the “HTTP/1.1″ part. A problem like this could be introduced by having a space in the query string (and it not being escaped properly), so that the request would have been for “GET /?a=b c HTTP/1.1″. After running through both the working and non-working query through ngrep and wireshark, this did however not seem to be the problem. My spaces were properly escaped using plus signs (GET /?a=b+c HTTP/1.1).

There does however seem to be a problem (at least with our version of Tomcat – 6.0.20) which results in the +-s being resolved before the request is handed off to the code that attempts to parse the header, so even though it is properly escaped using “+”, it still barfs.

The solution:

Use %20 to escape spaces instead of + signs; simply adding str_replace(” “, “%20″, ..); in our query layer solved the problem.

Making Friends With Emacs and UTF-8

May 5th, 2010

After having a few issues with UTF-8 support in Emacs earlier (I .. well .. err .. use emacs as my .. mail editor.), I finally found a solution that works with both putty and gterm as my terminals.

The important settings I ended up using in my .emacs:

(prefer-coding-system       'utf-8)
(set-default-coding-systems 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)

(setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING))

Archived for my own use at a later time.

Simple PHP Hack to Print an Integer in Binary Form

April 5th, 2010

Today: Using my own blog as a simple pastebin to remember a helper function I wrote while debugging a good selection of binary operations on integers in PHP. This will simply output the provided value as bit values (0 / 1). Wrap it in <pre>-s if you’re running it in a web context.

$len allows you do change how many bits it will print (starting from lsb – least significant bit), $blockSize adjust the amount of bits before throwing in a space.

  1. function printBinary($val, $len = 32, $blockSize = 8)
  2. {
  3.     print("\n");
  4.  
  5.     for($i = $len - 1; $i >= 0; $i--)
  6.     {
  7.         print(($val & 0x1<<$i) ? 1 : 0);
  8.  
  9.         if ($i%$blockSize == 0)
  10.         {
  11.             print (" ");
  12.         }
  13.     }
  14.  
  15.     print ("\n");
  16. }