Love Fuel?    Donate

FuelPHP Forums

Ask your question about FuelPHP in the appropriate forum, or help others by answering their questions.
Caching results in high memory usage
  • CharlieCharlie
    Hi everybody, here's a deal: Lately i've noticed that my fuel installation on production server started to behave a bit slow. I opened Profiler and was shocked by the amount of memory fuel was using -- over 45 MB to generate simple pages. So, i started digging and managed to narrow problems to the Agent and Cache (file cache) class. Agent class uses in several places Cache class to generate/read/parse several files (browscap.cache and possible cache.cache) even though my config settings doesn't have caching set to true. So, here you are. It seems like this problem could be bypassed by creating agent.php config in config directory and set browscap->enabled to false or change expiry time in cache section.
  • Jelmer SchreuderJelmer Schreuder
    You're not going to get to 45 MB because of caching as you suggest, I have caching enabled and get about a memory usage of 2.5 to 4.48 MB. So that suggestion is pretty much pure nonsense, if your stuff goes up to 45MB there's more going on than caching. There might be a problem with parsing the browsecap file if that needs to be done on every request, which might be the case if the cached browsecap file expires too quickly (which may have been your problem judging by the last sentense).
    I woulnd't encourage disabling the browsecap file, it makes the Agent class output pretty useless if your PHP install doesn't have a browsecap included itself.
  • CharlieCharlie
    I wasn't reading Agent class thoroughly to understand how it works. But fact is a fact - my memory usage was over 45 MB and that was happening when Agent class was dealing with data from cache/fuel/agent directory. It's very much possible that problem was with browsecap parsing, but that was happening every time i made a request and only methods i've used were is_robot(), browser(), platform() and some others. Obviously something's wrong here, gonna try change agent cache expire time maybe that'll help. When i've cleared browscap.cache and cache.cache files from cache/fuel/agent/ dir memory usage dropped down to 2MB.
  • HarroHarro
    You're probably looking at peak memory. When the Agent class needs to load and parse the browscap file, and you have used the full file, a lot of memory is needed to process it. It is reduced as much as possible, and then written to a cache file. By default, the browscap file is only fetched and parsed once a week. I would suggest not to do it more often then that. Then, when you load the Agent class, it will check the clients user agent string against the cached browscap data. If found, it will be stored in a second cache file for re-use. This second cache file is used first, so the second time the same client connects, it will use the small second cache file, instead of the full browscap cache. In the assumption that in reality an average site doesn't get visited by more that a few dozen different user agent strings, this process is quite efficient. If you actually need to rely on the Agent class, and you have access to your php.ini, it's best to configure PHP to use the browscap file, and setup a cron job to update the file. It's much more efficient that parsing it in PHP code.
  • CharlieCharlie
    WanWizard, thanks, i'll see how fuel will behave after i've cleaned those cache files. If situation gets worse as the last time i will follow your suggestion and will use browscap.ini
  • CharlieCharlie
    Ok, after about 10 hours cache.cache file grew up to about ~350KB of data which increased PHP memory usage from 2 MB up to 3.8/4MB.
  • CharlieCharlie
    Looks like i've found where was a problem: Line 400 in Agent class (get_from_cache() method):
    return array_key_exists(static::$user_agent, $cache) ? $content[static::$user_agent] : false;

    Looks like it should actually be
    return array_key_exists(static::$user_agent, $content) ? $content[static::$user_agent] : false;
    EDIT. One more update. Just after one day of running fuel generated cache.cache file with weight over 310KB which is larger then browscap.cache file itself.
  • HarroHarro
    Has already been corrected in the 1.1/develop branch. Can you dump the cache file after it's been read, to see that's been stored, and report if you see something odd?
  • CharlieCharlie
    I rewrote Agent class a bit so it doesn't use caching anymore, but instead only browscap.cache. I'll restore it to original state and check cache file for odd stuff as soon as I can. Right now I have suspicion that cache file simply fills up with hundreds or even thouthands variations of different browser agents.
  • HarroHarro
    Possible. But we need you to check to be sure. And about what kind of load are we talking (in terms of unique visitors / day?). If caching results is pointless, it might be a good idea just to remove it, or at least have a config value to disable it.
  • CharlieCharlie
    About 6k unique visitors per day. Caching could be good, but maybe variations should be lowered? Right now as i understand Agent stores in cache associated array with agent strings as keys. Maybe in order to reduce amount of variations it would be better to use different data as keys. For example "browser+version+os" or something similar.
  • HarroHarro
    Is is possible to collect a few days in the two cache files, and then zip them and email them to me, so I have live data to test with and see if I can come up with something clever? You can reach me at wanwizard <at> fuelphp <dot> com.
  • 4etch4etch
    I realize this is an ancient thread but this helped me solve a major issue on my production server so I thought I'd make a note of it. 

    After doing some profiling I discovered that the Agent class was using 60M of memory when calling Agent::browser... 60M! I was making that call on every page load for logging purposes without realizing the cost and it was shutting down my oversized server with cpu usage. Each request was using 50% of a core for simple page rendering. By removing that logging I was able to cut my memory usage per request from somewhere over 100M per to ~15M and cpu usage down to like 5% per core. Deleting the browscap.cache did not help. Fuel 1.3

    Hope that helps someone else with a similar problem!
  • HarroHarro
    We're aware of this, there is an issue open for it.

    Currently the browscap file is getting so big that is has become difficult to handle. The short term workaround is, if you have access to your php.ini, use PHP's built-in support. Agent will automatically use it when enabled.
Add a Comment

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Apply for Membership

Categories

In this Discussion