I have noticed, that using az_AZ locale causes fuel autoloader to break. It does not properly load the required framework classes.
For what I see, running the following code:
setlocale("az_AZ.utf8");
Config::set("language", "az");
Causes PHP to treat all the strings in the code as multi-byte.
Simply changing all the occurrences inside core/classes/autoloader.php from:
strtolower
to
mb_strtolower
fixes the issue, but of course it requires mbstring extension.
The environment I have used:
Ubuntu 16.04 LTS fully updated,
PHP Version 7.0.22-0ubuntu0.16.04.1 (latest from stable repository)
with installed az_AZ UTF-8 locale
I am not sure if the problem lies inside the Fuel PHP code or inside Ubuntu/PHP. Maybe someone could check on the different environment if this bug can be reproduced. For example running simple:
\Form::csrf();
Should fail with the error (Form_Instance class not found).
Update: it seems that the PHP is automatically changing the string to multibyte, when the string contains "I" letter (uppercase i). That's why "Form_Instance" does not work, but "Config" works. I have also seen problems with strings "it_IT", "sl_SI". Of course when the "az_AZ UTF-8" locale is installed and active. Still would be good if someone could confirm.
The Fuel core was writting for PHP versions that automatically swap those functions for their MB equivalents when available.
In those versions, mbstring.func_overload in php.ini was enabled. As of PHP 7, it seems to be disabled by default, and as of PHP 7.2, this function is deprecated.
So the workaround is:
- make sure mbstring is installed
- use ini_set('mbstring.func_overload', 1) in your app bootstrap if it is not set in php.ini
Addressing this properly means checking and replacing each and every string function in the framework, which is a lot of work.
But if you say it fails on the I, it could well be that multibyte is already enabled, which would cause this problem if I is a multibyte character in Azerbaijani. And that isn't easily fixed I fear, because that would mean disabling the overloading in the framework, and deal with the functions line by line.
It gets even worse if the OS (and therefore the filesystem itself) also uses AZ_az, because then you could still have the porblem in the file names, but then originating from the filesystem.
I would check the the value of mbstring.func_overload in your php.ini, and set it to 0 if it currently has another value. If you can't, use ini_set('mbstring.func_overload', 0) in your app bootstrap.
You wrote "use ini_set('mbstring.func_overload', 1)" and later "use ini_set('mbstring.func_overload', 0)" but I bet you meant the first one.
Yes, that could be the temporary fix and yes, I can confirm that "I" character causes PHP to treat string as multibyte in az_AZ locale. However, I don't get it why "I" character is treated as multibyte causing so much trouble here, do you have any idea? Do you think it's intended by PHP authors/Ubuntu locale? it's the first locale that I have problem with and I have Chinese, Bengali, Hindi, Thay etc running without this multibyte problem.
When using PHP7.2+ it will get even worse, as my application uses different APIs like cubits, maxmind geoip, libphonenumber-php, sofort and all of them are using non-multibyte string functions, so neither using func_overload or not is a good choice.
No, it depends on what the problem is, as there are two possible causes.
The correct approach would be to disable it: ini_set('mbstring.func_overload', 0), but I can't guarantee that the current framework code is 100% multibyte safe. The Str class itself is.
I see, thanks for the test code, this clarifies things!
Two points.
I made a mistake, to overload strtolower(), you need to set func_overload to 2, not to 1. And appearently you can not set func_overload using ini_set(), you need to set it either in your php.ini (in which case it is global), or in your virtual host, using
php_admin_value mbstring.func_overload 2
Again, I can't say (and I am not that confident) if and what impact this will have on other string functions in the framework.
Yes, I noticed earlier the ini_set does not work in this case, that's why I changed it in php.ini and checked with phpinfo.
I can confirm that setting this to 2 works as supposed to work.
However, as you said, it's not the preferred way, because we don't know what would be the behavior of the scripts. But the same apply without setting func_overlay to 2.
Thanks for your help, I will try to understand why az_AZ locale is changing the "I" letter to multibyte and is it the expected PHP behavior as I am unsure about this.
The workaround is to set LC_CTYPE to en_US.utf8 while losing currency, string and money formatting so most of the things that matters when localizing the application (although for php_intl you can set the locale manually which every request, so no big deal). For the framework alone it may be sufficient to somehow set LC_CTYPE before loading the class and set back LC_CTYPE after loading the class. I don't know nothing about how the autoloading is handled in the FuelPHP anyway.
It looks like the bug isn't fixed (the ticket mentions it actually can't be fixed due to the inconsistency between lower- and uppercase I), but worked around.
The test code provided in this ticket indeed doesn't fail anymore from PHP 5.5 onwards, but your test code still does. So the question remains what else, besides autoloading, is affected?
I would suggest to change nothing in the framework itself. You, as the developer of the framework, should not be concerned about PHP inconsistency, especially if the framework is not a single point of failure in this case (each library included could fail too, as a lot of them are not using multibyte functions).
Every programmer that would want to deal with tr_* and az_AZ locales will just have to remember to set up the LC_CTYPE to en_US.utf8 or another default. It is always possible to change the locale only for specific function (intl or not), so basically there is no need for changes in the framework itself.
It is part of properly dealing with multibyte support anyway.
As I wrote earlier, all instances in the framework of the functions mentioned in http://php.net/manual/en/mbstring.overload.php need to be looked at, and to be made multibyte aware if needed.
I have now done this for the Autoloader class only, to address this particular issue, but every class needs to be checked and updated where needed.
Donate
