memcache

 

 Shared Memory replacement for Memcache in PHP

October 6, 2009

speed

I recently (tonight, actually) had a requirement for the usual set() get() and incr() memcache style key-based variable access in PHP (on Linux) without the usual mucking about of installing memcached, and php5-memcached on Debian. I’m busy writing some modules for OpenRADIUS and they need to be able to increment some shared statistics counters, and share other state information between multiple instances of the same module.

I didn’t need cross-machine sharing of the data, and needed a quick (fairly platform independant) way of sharing some variables between processes. Previously, I relied on memcache, but the socket and stream i/o overhead in a high-performance solution is simply too much for the simple sharing of a hashtable I require.

As any good linux programmer will know, shm_* is your friend. SYSV IPC and shared memory is a fairly light, mostly kernel-based implementation mechanism to share memory across Unix processes.

Without further ado, the class:

// This code is public domain
// Original author: Roelf Diedericks (rodent@rodent.za.net)

class sharedMemoryStore {

	private $shmk_key;
	private $shm_id;
	private $var_key=1;

	private $sem_id;

	public function __construct($key="",$size=0,$perm=0666) {

		if ($key=="")
			$key=__FILE__;

		// default 16KB size shared memory
		if ($size==0)
			$size=1024*16;

		$this->shm_key=$key;

		$this->shm_key=ftok($key,'N');
		$this->shm_id=@shm_attach($this->shm_key,$size,$perm);

		if ( empty($this->shm_id) )  {
			throw new Exception("shared memory allocation failed");
		}

		$this->sem_id=@sem_get($this->shm_key,1,0666,true);

		if ( empty($this->sem_id) ) {
			throw new Exception("sem_get failed");
		}
	}

	public function lock() {
		if ( !sem_acquire($this->sem_id) ) {
			throw new Exception("lock failed");
		}
	}

	public function unlock() {
		if (! @sem_release($this->sem_id) ) {
			throw new Exception("unlock failed");
		}
	}

	public function set($key,$value) {

		$this->lock();
		$res=@shm_get_var($this->shm_id,$this->var_key);
		$this->unlock();

		if ($res===FALSE)
			$res=array();

		$res[$key]=$value;

		if (!  shm_put_var($this->shm_id,$this->var_key,$res) ) {
			throw new Exception("shm_put_var failed");
		}

	}

	public function get($key) {
		$res=@shm_get_var($this->shm_id,$this->var_key);

		if ($res===false) {
			echo "warn array empty\n";
			return false;
		}

		return @$res[$key];
	}

	public function incr($key,$increment=1) {
		$this->lock();

		$res=@shm_get_var($this->shm_id,$this->var_key);

		if ($res===FALSE)
			$res=array();

		if ( empty($res[$key]) )
			$res[$key]=0;

		$res[$key]+=$increment;

		if (!  shm_put_var($this->shm_id,$this->var_key,$res) ) {
			$this->unlock();
			throw new Exception("shm_put_var failed");
		}

		$this->unlock();

		return $res[$key];

	}

	public function  __destruct() {
		$this->unlock();
	}

}

Using it is as simple as the following:

include_once("sharedMemoryStore.php");

$s=new sharedMemoryStore("some-identifier");

$s->set("foo","bar");   // set key "foo" to value "bar"
echo "getting foo:";
echo $s->get("foo");        // get the value of foo
echo "\n";

$s->incr("counter",2); //incremement "counter" with 2
$s->incr("counter"); //incremement "counter" with 1

echo "counter is now: " . $s->get("counter") . "\n";

Now, if you fire up multiple processes on the same machine, they can all share the same counters, or hashtable. The shared memory segment is appropriately locked, so that only a single process can access the variables at a time. incr() works atomically, as expected, so that multiple process can increment a counter without treading on each others’ toes.

The size of the shared memory segment is by default a bit small (16kb), because most Linux distributions don’t have decent shared memory defaults. If you need to increase the size, check the appropriate sysctl.conf setting for your distribution, and change the constructor to something like

$s=new sharedMemoryStore("some-identifier",1024*1024; //1 meg shared segment

I went from about 300 requests per second to 1500 requests per second on a low-spec virtual machine by simply dropping in the shared memory storage class.