If the log shows that the client connects, but then gets disconnected after sending a few REQ packets (or at another part of the exchange), the cause is probably that there is an incompability between the version of the server and the version of the client or worker. If the client or worker sends a request that the server can’t understand (if the client/worker uses a part of the protocol that wasn’t defined when the server was compiled), the server will disconnect the client or worker.
While building gearman on our older RHEL4 servers, there was two issues that surfaced:
- The version of boost included in RHEL4 is too old (1.32) for gearman. I decided to download the new boost version (1.48.0 at the time of writing) and install it. Be sure to remove the old version with rpm -e boost-devel, so that you don’t get strange conflicts while attempting to build the benchmark tools:
benchmark/blobslap_worker.cc:89: undefined reference to `boost::program_options::options_description::m_default_line_length' benchmark/benchmark_blobslap_worker-blobslap_worker.o(.text+0x1d9): benchmark/blobslap_worker.cc:89: undefined reference to `boost::program_options::options_description::m_default_line_length'
Issues while trying to build the tests/ directory:
tests/stress_worker.cc: In function `test_return_t worker_ramp_TEST(void*)': tests/stress_worker.cc:113: error: `pthread_timedjoin_np' was not declared in this scope tests/stress_worker.cc:113: warning: unused variable 'pthread_timedjoin_np' make: *** [tests/stress_worker.o] Error 1
This can be solved by removing the whole section enclosed in the #ifdef _GNU_SOURCE
section. Let the content in the #else-part in place. Removing this will not affect the usual (and any important parts of it) gearman distribution in any way.
The configure / make process of gearman needs a way to exclude the benchmark/ and tests/ parts of the project from being built.
I spent the evening yesterday playing around a bit more with Gearman, a system for farming out tasks to workers across several servers. As my workstation at home still runs Windows, the only PHP library available is the Net_Gearman in PEAR. Net_Gearman supports tasks (something to do), sets (a collection of tasks), workers (the processes that performs the task) and clients (which requests tasks to be performed). The gearman protocol supports retrieving the current status of a task from the gearman server (which contains information about how the worker is progressing, reported by the worker itself), but Net_Gearman did not.
The reason for ‘did not’ is that I’ve created a small patchset to add the functionality to Net_Gearman. All internal methods and properties are still used as they were before, but I’ve added two helper methods for retrieving the socket connection for a particular gearman server (Net_Gearman usually just picks a random server, but we need to contact the server that’s responsible for the task) and a getStatus(server, handle) method to the Gearman Client. I’ve also added a property keeping the address of the server which were assigned the task to the Task class.
After submitting a task to be performed in the background (you do not need this to get the status for foreground tasks, as you can provide a callback to handle that), your Task object will have its
server properties set. These can be used to retrieve status information about the task later. You’ll still need to provide the possible servers to the Gearman client when creating the client (through the constructor).
Example of creating a task and retrieving the server / handle pair after starting the task:
require_once 'Net/Gearman/Client.php'; $client = new Net_Gearman_Client(array('host:4730')); $task = new Net_Gearman_Task('Reverse', range(1,5)); $task->type = Net_Gearman_Task::JOB_BACKGROUND; $set = new Net_Gearman_Set(); $set->addTask($task); $client->runSet($set); print("Status information: \n"); print($task->handle . "\n"); print($task->server . "\n");
Retrieving the status:
require_once 'Net/Gearman/Client.php'; $client = new Net_Gearman_Client(array('host:4730')); $status = $client->getStatus('host:4730', 'H:mats-ubuntu:1');
The array returned from the getStatus() method is the same array as returned from the gearman server and contains information about the current status (numerator, denominator, finished, etc, var_dump it to get the current structure). I’ve also added the patchset to the Issue tracker for Net_Gearman at github.
The patchset (created from the current master branch at github) can be downloaded here: GearmanGetStatusSupport.tar.gz.
UPDATE: I’ve finally gotten around to creating my own fork of NET_Gearman on github too. This fork features the patch mentioned above.