Mats Lindh » PHP http://e-mats.org Where desperate is just another word for a regular day. Thu, 11 Mar 2010 22:00:40 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 Fixing Issue With PHPs SoapClient Overwriting Duplicate Attribute and Tag Names http://e-mats.org/2010/02/fixing-issue-with-phps-soapclient-overwriting-duplicate-attribute-and-tag-names/ http://e-mats.org/2010/02/fixing-issue-with-phps-soapclient-overwriting-duplicate-attribute-and-tag-names/#comments Fri, 05 Feb 2010 13:48:59 +0000 Mats http://e-mats.org/?p=750 The setting:

An SOAP request contains an Id attribute – and an element with the exact name in the response (directly beneath the element containing the attribute – an immediate child):

  1. <res z:Id="i123">
  2.   <Id>foobar</Id>
  3. </res>

The problem is that the generated result object from the SoapClient (at least of PHP 5.2.12) contains the attribute value, and not the element value. In our case we could ignore the z:Id attribute, as it was simply an Id to identify the element in the response (this might be something that ASP.NET or some other .NET component does).

Our solution is to subclass the internal SoapClient and handle the __doRequest method, stripping out the part of the request that gives the wrong value for the Id field:

  1. class Provider_SoapClient extends SoapClient
  2. {
  3.     public function __doRequest($request, $location, $action, $version)
  4.     {
  5.         $result = parent::__doRequest($request, $location, $action, $version);
  6.         $result = preg_replace('/ z:Id="i[0-9]+"/', '', $result);
  7.         return $result;
  8.     }
  9. }

This removes the attribute from all the values (there is no danger that the string will be present in any other of the elements. If there is – be sure to adjust the regular expression). And voilá, it works!

]]> http://e-mats.org/2010/02/fixing-issue-with-phps-soapclient-overwriting-duplicate-attribute-and-tag-names/feed/ 0 Avoid Escaping Spaces in the Query String in a Solr Query http://e-mats.org/2010/01/avoid-escaping-spaces-in-the-query-string-in-a-solr-query/ http://e-mats.org/2010/01/avoid-escaping-spaces-in-the-query-string-in-a-solr-query/#comments Sat, 30 Jan 2010 22:24:14 +0000 Mats http://e-mats.org/?p=744 Following up on the previous post about escaping values in a Solr query string, it’s important to note that you should not escape spaces in the query itself. The reason for this is that if you escape spaces in the query “foo bar”, the search will be performed on the term “foo bar” itself, and not with “foo” as one term and “bar” as the other. This will only return documents that has the string “foo bar” in sequence.

The solution is to either remove the space from the escape list in the previous function – and use another function for escaping values where you actually should escape the spaces – or break up the string into “escapable” parts.

The code included beneath performs the last task; it splits the string into different parts delimited by space and then escapes each part of the query by itself.

  1. $queryParts = explode(' ', $this->getQuery());
  2. $queryEscaped = array();
  3.  
  4. foreach($queryParts as $queryPart)
  5. {
  6.     $queryEscaped[] = self::escapeSolrValue($queryPart);
  7. }
  8.  
  9. $queryEscaped = join(' ', $queryEscaped);
]]> http://e-mats.org/2010/01/avoid-escaping-spaces-in-the-query-string-in-a-solr-query/feed/ 1 A Simple Smarty Modifier to Generate a Chart Through Google Chart API http://e-mats.org/2010/01/a-simple-smarty-modifier-to-generate-a-chart-through-google-chart-api/ http://e-mats.org/2010/01/a-simple-smarty-modifier-to-generate-a-chart-through-google-chart-api/#comments Fri, 29 Jan 2010 22:48:55 +0000 Mats http://e-mats.org/?p=739 After the longest title of my blog so far follows one of the shortest posts.

The function has two required parameters – the first one is provided automagically for you by smarty (it’s the value of the variable you’re applying the modifier to). This should be an array of objects containing the value you want to graph. The only required argument you have to provide to the modifier is the method to use for fetching the values for graphing.

Usage:
{$objects|googlechart:”getValue”}

This will dynamically load your plugin from the file modifier.googlechart.php in your Smarty plugins directory, or you can register the plugin manually by calling register_modifier on the template object after you’ve created it.

  1. function smarty_modifier_googlechart($points, $method, $size = "600×200", $low = 0, $high = 0)
  2. {
  3.     $pointStr = '';
  4.     $maxValue = 0;
  5.     $minValue = INT_MAX;
  6.    
  7.     foreach($points as $point)
  8.     {
  9.         if ($point->$method() > $maxValue)
  10.         {
  11.             $maxValue = $point->$method();
  12.         }
  13.  
  14.         if ($point->$method() < $minValue)
  15.         {
  16.             $minValue = $point->$method();
  17.         }
  18.     }
  19.  
  20.     if (!empty($high))
  21.     {
  22.         $maxValue = $high;
  23.     }
  24.  
  25.     $scale = 100 / $maxValue;
  26.  
  27.     foreach($points as $point)
  28.     {
  29.         $pointStr .= (int) ($point->$method() * $scale) . ',';
  30.     }
  31.  
  32.     $pointStr = substr($pointStr, 0, -1);
  33.  
  34.     // labels (5)
  35.     $labels = array();
  36.  
  37.     $steps = 4;
  38.     $interval = $maxValue / $steps;
  39.  
  40.     for($i = 0; $i < $steps; $i++)
  41.     {
  42.         $labels[] = (int) ($i * $interval);
  43.     }
  44.  
  45.     $labels[] = (int) $maxValue;
  46.  
  47.     return 'http://chart.apis.google.com/chart?cht=lc&amp;chd=t:' . $pointStr . '&amp;chs=' . $size . '&amp;chxt=y&amp;chxl=0:|' . join('|', $labels);
  48. }

The function does not support the short version of the Google Chart API Just Yet ™ as it is an simple proof of concept hack made a few months ago.

]]> http://e-mats.org/2010/01/a-simple-smarty-modifier-to-generate-a-chart-through-google-chart-api/feed/ 0 How To Dismantle An Atomic HTTP Query .. String. http://e-mats.org/2010/01/how-to-dismantle-an-atomic-http-query-string/ http://e-mats.org/2010/01/how-to-dismantle-an-atomic-http-query-string/#comments Thu, 28 Jan 2010 20:48:40 +0000 Mats http://e-mats.org/?p=737 Following up on yesterday’s gripe about PHPs (old and now useless) automagic translation of dots in GET and POST parameters to underscores, today’s edition manipulates the query string in place instead of returning it as an array.

This is useful if you have a query string you want to pass on to another service, and for some reason the default behaviour in PHP will barf barf and barf. That might happen because of the dot translation issue or that some services (such as Solr) rely on a parameter name being repeatable (in PHP the second parameter value will overwrite the first).

  1. function http_dismantle_query($queryString, $remove)
  2. {
  3.     $removeKeys = array();
  4.  
  5.     if (is_array($remove))
  6.     {
  7.         foreach($remove as $removeKey)
  8.         {
  9.             $removeKeys[$removeKey] = true;
  10.         }
  11.     }
  12.     else
  13.     {
  14.         $removeKeys[$remove] = true;
  15.     }
  16.  
  17.     $resultEntries = array();
  18.     $segments = explode("&", $queryString);
  19.  
  20.     foreach($segments as $segment)
  21.     {
  22.         $parts = explode('=', $segment);
  23.  
  24.         $key = urldecode(array_shift($parts));
  25.  
  26.         if (!isset($removeKeys[$key]))
  27.         {
  28.             $resultEntries[] = $segment;
  29.         }
  30.     }
  31.  
  32.     return join('&', $resultEntries);
  33. }

I’m not really sure what I’ll call the next function in this series, but there sure are loads of candidates out there.

]]> http://e-mats.org/2010/01/how-to-dismantle-an-atomic-http-query-string/feed/ 0 Getting Dots to Work in PHP and GET / POST / COOKIE Variable Names http://e-mats.org/2010/01/getting-dots-to-work-in-php-and-get-post-cookie-variable-names/ http://e-mats.org/2010/01/getting-dots-to-work-in-php-and-get-post-cookie-variable-names/#comments Wed, 27 Jan 2010 22:22:21 +0000 Mats http://e-mats.org/?p=735 One of the oldest and ugliest relics of the register_globals era of PHP are the fact that all dots in request variable names gets replaced with “_”. If your variable was named “foo.bar”, PHP will serve it to you as “foo_bar”. You cannot turn this off, you cannot use extract() or parse_str() to avoid it and you’re mostly left out in the dark. Luckily the QUERY_STRING enviornment (in _SERVER if you’re running mod_php, etc) contains the raw string, and this string contains the dots.

The following “”parser”" is a work in progress and does currently not support the array syntax for keys that PHP allow, but it solves the issue for regular vars. I will try to extend this later on to do actually replicate the functionality of the regular parser.

Here’s the code. No warranties. Ugly hack. You’re warned. Leave a comment if you have any good suggestions regarding this (.. or know of an existing library doing the same..).

  1. function http_demolish_query($queryString)
  2. {
  3.     $result = array();
  4.     $segments = explode("&", $queryString);
  5.  
  6.     foreach($segments as $segment)
  7.     {
  8.         $parts = explode('=', $segment);
  9.  
  10.         $key = urldecode(array_shift($parts));
  11.         $value = null;
  12.  
  13.         if ($parts)
  14.         {
  15.             $value = urldecode(join('=', $parts));
  16.         }
  17.  
  18.         $result[$key] = $value;
  19.     }
  20.  
  21.     return $result;
  22. }

(OK, that’s not the real function name, but it’s aptly named to be the nemesis of http_build_query)

]]> http://e-mats.org/2010/01/getting-dots-to-work-in-php-and-get-post-cookie-variable-names/feed/ 0 Retrieving URLs in Parallel With CURL and PHP http://e-mats.org/2010/01/retrieving-urls-in-parallel-with-curl-and-php/ http://e-mats.org/2010/01/retrieving-urls-in-parallel-with-curl-and-php/#comments Sun, 24 Jan 2010 10:00:50 +0000 Mats http://e-mats.org/?p=724 As we’ve recently added support for querying Solr servers in parallel, one of the things we added was a simple class to allow us to query several servers at the same time. The CURL library (which has a PHP extension) even provides an abstraction layer for doing the nitty gritty work for you, as long as you keep track of the resources. The code beneath is based on examples in the documentation and a few tweaks of my own.

The code beneath is licensed under a MIT license. You can also download the file (gzipped).

  1. class Footo_Content_Retrieve_HTTP_CURLParallel
  2. {
  3.     /**
  4.      * Fetch a collection of URLs in parallell using cURL. The results are
  5.      * returned as an associative array, with the URLs as the key and the
  6.      * content of the URLs as the value.
  7.      *
  8.      * @param array<string> $addresses An array of URLs to fetch.
  9.      * @return array<string> The content of each URL that we've been asked to fetch.
  10.      **/
  11.     public function retrieve($addresses)
  12.     {
  13.         $multiHandle = curl_multi_init();
  14.         $handles = array();
  15.         $results = array();
  16.  
  17.         foreach($addresses as $url)
  18.         {
  19.             $handle = curl_init($url);
  20.             $handles[$url] = $handle;
  21.  
  22.             curl_setopt_array($handle, array(
  23.                 CURLOPT_HEADER => false,
  24.                 CURLOPT_RETURNTRANSFER => true,
  25.             ));
  26.  
  27.             curl_multi_add_handle($multiHandle, $handle);
  28.         }
  29.  
  30.         //execute the handles
  31.         $result = CURLM_CALL_MULTI_PERFORM;
  32.         $running = false;
  33.  
  34.         // set up and make any requests..
  35.         while ($result == CURLM_CALL_MULTI_PERFORM)
  36.         {
  37.             $result = curl_multi_exec($multiHandle, $running);
  38.         }
  39.  
  40.         // wait until data arrives on all sockets
  41.         while($running && ($result == CURLM_OK))
  42.         {
  43.             if (curl_multi_select($multiHandle) > -1)
  44.             {
  45.                 $result = CURLM_CALL_MULTI_PERFORM;
  46.  
  47.                 // while we need to process sockets
  48.                 while ($result == CURLM_CALL_MULTI_PERFORM)
  49.                 {
  50.                     $result = curl_multi_exec($multiHandle, $running);
  51.                 }
  52.             }
  53.         }
  54.  
  55.         // clean up
  56.         foreach($handles as $url => $handle)
  57.         {
  58.             $results[$url] = curl_multi_getcontent($handle);
  59.  
  60.             curl_multi_remove_handle($multiHandle, $handle);
  61.             curl_close($handle);
  62.         }
  63.  
  64.         curl_multi_close($multiHandle);
  65.  
  66.         return $results;
  67.     }
  68. }

Download the file.

]]> http://e-mats.org/2010/01/retrieving-urls-in-parallel-with-curl-and-php/feed/ 0 Escaping Characters in a Solr Query / Solr URL http://e-mats.org/2010/01/escaping-characters-in-a-solr-query-solr-url/ http://e-mats.org/2010/01/escaping-characters-in-a-solr-query-solr-url/#comments Wed, 20 Jan 2010 10:00:13 +0000 Mats http://e-mats.org/?p=710 We’re using our own Solr library at Derdubor at the moment, but we’ve only been using it for indexing content. The query part was never standardized in our common library as we usually used an alternative output format, but during the last days that has changed. We now have a parser for the default XML outputter and we’re also supporting facets and field queries (or constraints as they’re abstracted as in our library).

This means that we’re feeding content into the query that may contain foreign characters, in particular those who have special meaning in a Solr query. You can find the complete list of characters that need to be escaped in a SOLR or Lucene query in the Lucene manual.

To escape the characters we use this very simple and stupid PHP method:

  1.     static public function escapeSolrValue($string)
  2.     {
  3.         $match = array('\\', '+', '-', '&', '|', '!', '(', ')', '{', '}', '[', ']', '^', '~', '*', '?', ':', '"', ';', ' ');
  4.         $replace = array('\\\\', '\\+', '\\-', '\\&', '\\|', '\\!', '\\(', '\\)', '\\{', '\\}', '\\[', '\\]', '\\^', '\\~', '\\*', '\\?', '\\:', '\\"', '\\;', '\\ ');
  5.         $string = str_replace($match, $replace, $string);
  6.  
  7.         return $string;
  8.     }

We used a regular expression first, but the sheer amount of backslashes made it a regular .. hell … to read. So to make it easier for the persons maintaining this in the future, we went the easy to read / easy to maintain road for this one.

]]> http://e-mats.org/2010/01/escaping-characters-in-a-solr-query-solr-url/feed/ 3 PHP: Fatal error: Can’t use method return value in write context http://e-mats.org/2010/01/php-fatal-error-cant-use-method-return-value-in-write-context/ http://e-mats.org/2010/01/php-fatal-error-cant-use-method-return-value-in-write-context/#comments Mon, 18 Jan 2010 10:00:38 +0000 Mats http://e-mats.org/?p=702 Just a quick post to help anyone struggling with this error message, as this issue gets raised from time to time on support forums.

The reason for the error is usually that you’re attempting to use empty or isset on a function instead of a variable. While it may be obvious that this doesn’t make sense for isset(), the same cannot be said for empty(). You simply meant to check if the value returned from the function was an empty value; why shouldn’t you be able to do just that?

The reason is that empty($foo) is more or less syntactic sugar for isset($foo) && $foo. When written this way you can see that the isset() part of the statement doesn’t make sense for functions. This leaves us with simply the $foo part. The solution is to actually just drop the empty() part:

Instead of:

  1. if (empty($obj->method()))
  2. {
  3. }

Simply drop the empty construct:

  1. if ($obj->method())
  2. {
  3. }
]]> http://e-mats.org/2010/01/php-fatal-error-cant-use-method-return-value-in-write-context/feed/ 0 Missed Schedule for Posts in Wordpress http://e-mats.org/2010/01/missed-schedule-for-posts-in-wordpress/ http://e-mats.org/2010/01/missed-schedule-for-posts-in-wordpress/#comments Tue, 12 Jan 2010 10:00:02 +0000 Mats http://e-mats.org/?p=685 As I started queuing the posts for the previous run of “Ready for 2010“-articles, I came across a problem with my Wordpress installation. The scheduled articles didn’t show up when they were scheduled, and the only thing shown in the Wordpress administration interface were a message about “Missed Schedule”. No shit, sherlock.

The reason behind the message is that the wp-cron.php file didn’t run as it should. Wordpress usually tries to run this every now and then by inserting a reference to the file through the web site. Apparently this behavior was borked on my blog. I have a perfectly working cron implementation on my server, so instead of relying on Wordpress to do some kind of magic to insert a reference to the file and kick off the processing with a web request, I added a reference to wp-cron.php in my usual crontab.

I have no idea how often wp-cron really should be run, but decided that a five minute resolution was enough for my use. The crontab entry is included here:

*/5 * * * * cd <directory of blog> && php wp-cron.php

This runs the cron script from the proper directory, and seems to work fine.

]]> http://e-mats.org/2010/01/missed-schedule-for-posts-in-wordpress/feed/ 0 Writing a Munin Plugin http://e-mats.org/2010/01/writing-a-munin-plugin/ http://e-mats.org/2010/01/writing-a-munin-plugin/#comments Sat, 09 Jan 2010 10:00:07 +0000 Mats http://e-mats.org/?p=671 I have to admit something. I’ve become addicted.

One of the things I finally got around to doing while living the quiet life over the christmas holiday was to dive a bit further into Munin – a simple framework for collecting information from your computers and servers and making nice graphs that you can watch while you’re bored.

I’m not going to write a lot about how you can create your own Munin plugin to create your own graphs, as they have a very simple tutorial giving you all the basics about writing Munin plugins themselves. The only thing you need to remember are these two tidbits:

  1. When Munin first registers your plugin, it runs your script with config as the only argument. This provides Munin with the name of the graph, the labels and names (keys) of the graphs you’re providing values for, information about the axis, etc.
  2. When Munin runs your script without the config argument, it expects you to give it values for the keys you provided it in the configuration.

You enable and disable plugins by creating symlinks in /etc/munin/plugins (at least under debian / ubuntu), and plugins are usually stored in /usr/share/munin/plugins.

I keep my plugins archived together with the rest of the repository for my web projects, and then either symlink the content into the plugins-directory or create a simple wrapper script that changes the current directory to the location of the script and then invokes it (to make the current working directory be correct).

A very simple bash script that does this – and passes through any parameters given to the script:

  1. #!/bin/bash
  2. cd <absolute path> && php ./<script name> "$@"

An example of a simple PHP script to provide information to Munin:

  1. <?php
  2. if ((count($argv) > 1) && ($argv[1] == 'config'))
  3. {
  4.     print("graph_title THE TITLE OF YOUR GRAPH
  5. graph_category THE CATEGORY / GROUP OF YOUR GRAPH
  6. graph_vlabel Count
  7. total.label Total
  8. other.label Other
  9. ");
  10.     exit();
  11. }
  12.  
  13. print('total.value ' . get_total_value() . "\n");
  14. print('other.value ' . get_other_value() . "\n");

Symlink everything, check that it runs properly when you execute the script from the plugins directory:

mats@xx:/usr/share/munin/plugins$ ./scriptname
total.value 37
other.value 13
mats@xx:/usr/share/munin/plugins$

Symlink it into the /etc/munin/plugins directory and reload or restart Munin.

To check that Munin runs your script properly, telnet into the Munin server from an approved host and type “fetch “. You should now see the same output as you got when you simply typed ./scriptname in the plugins directory.

If stuff doesn’t work and you’re having a hard time finding out why, be sure to check out the munin-node logfile: /var/log/munin/munin-node.log.

As soon as you have the basics down, you’re free to start graphing whatever numeric value you can think of. The most interesting uses are probably something that integrates with your web applications, such as the number of searches, the number of signed up users, the language selection of users, the popularity of certain categories, etc. The possibilities are endless, use your imagination!

And about the addiction: NEED MORE GRAPHS.

]]> http://e-mats.org/2010/01/writing-a-munin-plugin/feed/ 0