Love Fuel?    Donate

FuelPHP Forums

Ask your question about FuelPHP in the appropriate forum, or help others by answering their questions.
Memcached expiration time
  • I'm having an odd problem using memcached with FuelPHP's Cache (using FuelPHP 1.4). For some reason, the expiration time seems to be out of sync by about 150 seconds. ie. I need to set expiration time to [number of seconds required] + 150. (So, for example, setting expiration to 160 would give me ~10s before the cache expires). Any value less than 150 isn't cached.

    I thought this might be a server date/time problem, but I've just tried using PHP's Memcached extension to directly set values (bypassing FuelPHP's Cache), and I've found that expiration times work exactly as expected (ie. 10 = 10s; 160 = 160s). So it seems like it must be something in the Cache class.

    Can anyone shed any light on this? I don't really want to have to write a custom Cache class just to get around this, but I do need to be able to accurately set expiration times.

    Thanks.

  • Update: I tried replacing Cache_Storage_Memcached with the latest version (1.7) and I'm still having the exact same problem.
  • I can not reproduce that here. I used this code to test:
    function microtime_float()
    {
        list($usec, $sec) = explode(" ", microtime());
        return ((float)$usec + (float)$sec);
    }

    \Cache::set('test', 'value', 10);

    $time_start = microtime_float();

    while(true)
    {
        try
        {
            $var = \Cache::get('test');
        }
        catch (\Exception #e)
        {
            $time_end = microtime_float();
            $time = $time_end - $time_start;

            echo "Cache expired in $time seconds\n";
            exit;
        }
    }
    which in your case won't cache at all (since it only caches for 10 seconds), but the response is:

    Cache expired in 9.9892930984497 seconds


    as expected.

    Also tried this with a timeout of 200 seconds (which in your case would expire in 50 seconds), but it doesn't:

    Cache expired in 200.19303607941 seconds


    So it must be something local that is causing this.
  • Thanks for that. Hmm, this is odd. I tried your code, and if I set the expiry time to, say, 170 I get:

        Cache expired in 7.133584022522 seconds

    Anything less than about 160 and cache expires in ~0.002 seconds (ie. immediately). This seems to be a problem with Memcached only: when I switch the Cache driver to "file" it works as expected (eg. 10 => ~10s)

    One thing that occurred to me: I ran "date" on the server, and it looks like the system clock is about 2 mins 15 seconds behind UTC, which I'm guessing could well have something to do with this problem (though it's odd that the difference doesn't seem to match that ~160s offset)

    However, as I said, when I call PHP's Memcached::set directly the expiry times work exactly as expected. It's as if Memcached is using relative/local times to expire, while FuelPHP's Cache is somehow locked to UTC. Could that make sense? (I had a quick look through Cache_Storage_Memcached, and I couldn't figure out where/why/how this would be happening) The fact that Memcached::set works fine does make me think that although the problem could be something to do with my local configuration, it must be specific to FuelPHP because I only see it when I access Memcached through FuelPHP's Cache.

  • (Oh, just a sidenote, in case you didn't know: as of PHP5 you can use "microtime(true)" to return microtime as a float - which is quite handy and probably avoids the need for that "microtime_float" custom function)
  • I know that, just quickly re-used an existing test controller that already had that function. I'm lazy by nature ;-).

    Cache drivers use time(), so local time, not UTC. And local means "in the configured timezone".

    All cache drivers use the same mechanism. The cache payload is a struct which contains, next to the actual data, some metadata fields. One of them is expiration timestamp.

    When the cache is read, this metadata is used to populate the cache object, the expiration timestamp is taken, the current time() is subtracted, and the result is stored. Meaning, if the outcome of the subtraction is less then zero, the item is expired.

    The file driver works exactly the same way, so I still find it unbelievable that the code is to blame here. Besides that, if it was a logic error, I would have the same issue here too.
  • Yes, it does seem odd...

    The thing is though that (a) I ran the exact same code (ie. your test function above) with driver set to "memcached", "file", and "apc" and the problem only occurs when the driver is set to memcached (ie. with the "file" and "apc" drivers it expired in just over 10s; with  "memcached" it expired in 0.002s); (b) like I said, this problem only occurs when I use FuelPHP's Cache: when I use Memcached::set($key, $value, 10) to cache something directly, the memcached value expires at the expected time (ie. after 10s).

    Given (a) and (b), I'm finding it hard to imagine where the problem could be other than in FuelPHP's Memcached driver, though of course that could be a failure of imagination on my part!

    I've actually got round the problem for now (I wrote my own simple cache class), so I'm no longer in urgent need of a solution, but do let me know if there's anything else you can think of that I can usefully test.

    For what it's worth, my code is running in PHP 5.4 on an Amazon EC2 instance, using an Amazon Elasticache node as the Memcached client.

  • Fuel doesn't use the expiration on the set() call (directly), the expiration in the metadata is used. I've just spend another half hour debugging the code, but I can't reproduce it.

    It would be interesting to know if the memcached->get() fails (on L#264) or if the metadata check returns a negative expiration (on L#224), perhaps you can add some debug code to the memcached storage driver?

  • Ok, I dumped the contents of $payload (returned by memcached->get()) after line 264, and it's false, which is causing a "Cache has bad formatting" exception (cf. line 212).

    $key looks fine though, so I went to the _set() method, and it turns out that if I try dumping the contents of $this->memcached->get($key) after it is set at line 242 (ie. just before "$this->_update_index($key);"), I still get "false". If I dump $this->memcached->getAllKeys() at this same point then I get an array of keys which includes $key so it seems like it's setting the value ok, but is then unable to retrieve it. I'm struggling to imagine why that would be.

    Hope that's of some use. Let me know if there's anything else you'd like me to try!


  • If $this->memcached-get() returns false, then memcached has already expired the key, so everything else is academic.

    The Memcached driver uses a unix timestamp as expiration, which is calculated as time() + expiration, to allow expirations of more then 30 days. This uses local server time, so if local time is different from the time (which is what you mentioned) on your memcached server, that will cause your problem.

    If you have tested it with an expiration value in seconds, the Memcached server will convert it to a unix timestamp for you, which uses local time of the Memcached server and not your webserver, so expiration will be correct even if your webserver time is incorrect.

    Only solution is to make sure your server time is correct (which you should always do, because for example cookies will have a similar issue).
  • Thanks for that. That clears up what's happening: there must be a discrepancy between the clock on my server and the server running memcached.

    While I could and should (and perhaps even will!) fix that by correcting the clocks, it occurred to me that there is perhaps another solution, which could make the memcached driver more robust: to use the relative time (eg. 30s) when the expiration time is less than 30 days, and only use a unix timestamp when it's greater (in line with how the Memcached PHP extension handles expiration times). This would mean that even where server clock was innacurate, the inaccuracy would only affect caches where expiration time > 30 days (and unless the clock is completely wrong, the inaccuracy would most likely be less significant at these timescales anyway). This should make the Cache behave as expected even where the clock on the server running FuelPHP doesn't quite match the clock on the server running Memcached.

    I tweaked the memcached storage driver to try the above, and it now seems to work as expected (eg. 10 --> ~10s, even on my server). In case you're interested, I've sent a pull request on Github with the updated version for you to take a look at. Hope it's of use! Do give me a shout if you have any thoughts / questions.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

In this Discussion