Test ConfirmImportControllerTest.testConvertHtmlToBbcode also fails only if run along with ConfirmImportControllerTest.testImportKOI_8. It runs clean by itself or with any of the other failures. The only error message I can see is:
Quote:
There are unreleased locks!
Test CustomFieldItemAdminControllerTest.testCustomFieldItemAdminController also fails:
[IBM][CLI Driver][DB2/NT] SQL0401N The data types of the operands for the operation "=" are not compatible. SQLSTATE=42818 SQLCODE=-401
ADOConnection._Execute(SELECT g2_ChildEntity.g_id FROM
g2_ChildEntity WHERE g2_ChildEntity.g_parentId='2659')
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2006-02-16 08:58
ok, my test script was not perfect. strlen() does not return the multi-byte character length. it returns the nr of bytes, i guess.
if you set the internal encoding of mb_substr / mb_strlen etc before using it, it shows that this string is 128 mulit-byte characters long.
specifically, \xc3\xa5 is a single multi-byte character and not 2 characters.
that to say: the truncation works correctly in G2. to make it work correctly in your test script, you probably have to add a few character encoding statements.
the full string that is truncated before inserting is:
"Western European (ISO)\r\n" .
"Hello World \r\n" .
"Norwegian Norsk\r\n" .
"Hallo Verden\r\n" .
"Gallery er s\xc3\xa5 utrolig bra at jeg har satt det opp til flere venner. "
which is clearly > 128 multi-byte characters.
e.g. to get mbstring working, get the php_mbstring.dll (if it isn't in your php ext/ / extensions/ folder yet) from http://www.php.net/downloads.php i guess the "collection of pecl modules" should include it. in php.ini change the ; extension=php_mbstring.dll line to extension=php_mbstring.dll and restart apache.
before the test script runs, you'll have to call mb_internal_encoding("UTF-8"); at the top.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2006-02-16 16:09
OK, I've enabled the php_mb_string.dll (and restarted Apache, even though I'm not using it right now), and added the mb_internal_encoding("UTF-8"); call. Now my script does find function mb_substr(), so it does use it instead of going through the other G2 functions. But even mb_substr() is returning a 129-byte-long string.
Doh! Never mind, I lost the mb_internal_encoding("UTF-8") call while editing my scripts. The non-database test script works now. Will now try my database-enabled test script, and then G2 itself.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2006-02-16 16:15
as i said, you need to add
mb_internal_encoding("UTF-8");
at the beginning of your script and then you'll see that the mb_strlen() is actually 128.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2006-02-16 16:22
Yes, I did so (you replied before I updated my posting) ;-).
DB2 still doesn't like it though. I'll keep looking into it.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2006-02-16 17:26
Valiant,
The PHP manual for mb_strlen() says:
Quote:
mb_strlen() returns number of characters in string str having character encoding encoding.
A multi-byte character is counted as 1.
Note: "characters", not "bytes". That must be our problem. VARCHAR(128) means 128 bytes, not 128 characters.
So again... what's a reasonable size to make the column?
To safely accommodate the extreme case (128 UTF-8 4-byte non-ASCII characters), we would need to use VARCHAR(1024) and reduce g_description accordingly.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2006-02-16 17:35
heh, i was there a few posts ago when i asked you:
Quote:
are you sure
- your db is utf8,
- the connection is utf8 (i don't know if the db2 connection needs to be configured) and
- that db2 can store 128 multybyte characters in a varchar(128) field?
because that is what i understand as a database supporting UTF-8. can you find out whether DB2 really can't store 128 UTF8 characters in a UTF-8 database table column?
edit: highlighting my own typos, i'm brave today
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2006-02-16 17:48
Sorry, I guess I didn't fully understand your question... I'm not that familiar with MBCS.
When working with character string data in UTF-8, one should not assume that each character is one byte.
In multibyte UTF-8 encoding, each ASCII character is one byte, but non-ASCII characters take two to four bytes each.
This should be taken into account when defining CHAR fields. Depending on the ratio of ASCII to non-ASCII characters,
a CHAR field of size n bytes can contain anywhere from n/4 to n characters.
that means for you: ignore the 3-4 migrate failures please for now. we'll fix it after G2.1.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2006-02-16 18:48
Just when we thought we had it figured out...
I really, really, really hate to say this, but... all of a sudden, my database-enabled test script is now working. But I have no idea why! I'm quite sure I haven't changed anything since it used to fail.
should we go with varchar(30000) for TEXT (session data and pluginparametermap value)?
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2006-02-16 21:03
> patch
Thanks. Are there still nightly builds being sent tp jmullan's page?
> text/clob
As I recall, you had suggested using 'TEXT-MEDIUM' => 'VARCHAR(32000)'. So that's what I'm currently using, and that seems to be working well.
I'm currently running the full suite of unit tests (except the 4 failing migrate tests). The good news is that the run no longer aborts after test 529 (perhaps it's related to the migrate errors). The bad news is that there are still a handful of new errors I need to look into. I'm currently as far as test #917 of 2171.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2006-02-16 22:12
nightly builds are made about 3 AM pacific standard time, so 14h from now.
i have changed TEXT and TEXT-MEDIUM to varchar(15000).
i have not changed the column types from TEXT to TEXT-MEDIUM.
this should be good for now. notice also that i committed another fix yesterday which should ensure that the session data doesn't skyrocket as badly as before.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2006-02-16 22:28
OK, I'll grab a fresh build tomorrow. Is it still the RC1 that I want?
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2006-02-16 22:43
just grab the next "latest" nightly snapshot, the one dated with February 17th or so.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2006-02-17 19:16
I grabbed the current build today and am running unit tests on it (all modules installed).
I'm having difficulty getting a complete run... if it doesn't die with functional problems, it dies with a 30-second timeout. Wonder if I need a beefier system to run all these unit tests. My current test system is a PIII 800 MHz, 384 Mb RAM. I guess if I have to, I can break the tests up into groups.
Will let you know how it goes. Maybe I'll give up trying to get a complete run and just start looking at specific failures.
Edit: 384 Mb, not 384 Kb.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2006-02-17 19:32
you can change the 30 second timeout to e.g. 600 in
lib/tools/phpunit/GalleryTestCase.class
line 280, $gallery->guaranteeTimeLimit(30);
could be changed to
$gallery->guaranteeTimeLimit(600);
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2006-02-17 19:41
Good to know... thanks!
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2006-02-17 22:57
I shut down another app on my test system, didn't have to change the timeout. I'm still unable to get a complete run though... but there's no error, no timeout, nothing. Despite running clean up until that point, it just ends abruptly, precisely between the PermissionSetTest and PermissionTest tests.
Then I did a run specifying the numeric range of the remaining tests, and finally there were no more abrupt exits.
I have results of every testcase, over a series of 4 partial runs.
I have determined that the failures are in the first set of asserts:
GalleryUtilities::putRequestVariable('isapiRewriteTest', 'pass');
list ($ret, $status, $trueStatus) = IsapiRewriteHelper::checkIsapiRewrite();
if ($ret) {
return $this->failWithStatus($ret->wrap(__FILE__, __LINE__));
}
$this->assertEquals(REWRITE_STATUS_OK, $status);
$this->assertEquals(REWRITE_STATUS_OK, $trueStatus);
Both these asserts fail. Both $status and $trueStatus = 31. The other 2 sets of asserts are successful (comment out the first set of asserts and the test runs clean).
Before I look into this any further, let me state up front that I do have a possible problem when configuring the "URL Rewrite" module. There's a 'Warning' message printed that support for Rewrite could not be confirmed. There are two manual test links there, they both return FAILURE (or something like that, I'm going from memory now). I have enabled mod_rewrite in my Apache.
Any suggestions as to this?
Thanks.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2006-02-18 00:35
i need sleep now. i asked pelle, the author of rewrite to assist you in the rewrite test failures. but he won't be there in the next 10h, or maybe even longer.
see my last post, i've fixed one of your failures.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2006-02-18 00:38
Thanks. Bon soir.
pelle
Joined: 2004-12-10
Posts: 389
Posted: Sat, 2006-02-18 09:14
The test should use a mock platform and has nothing to do with if Isapi Rewrite actually works or not. I'll see if I can reproduce the test failures on a windows machine (with mysql though).
As for the mod_rewrite test they are pretty accurate. If you get FAIL_NO_REWRITE chanses are that you have missed some part of the mod_rewrite configuration.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2006-02-18 09:17
i can't reproduce the failures and i'm always testing with windows xp / apache2 / mod_php 4 / 5 lately. but mod_rewrite and pathinfo work for me.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2006-02-18 15:10
Hi, Pelle. Thanks for looking into it. Let me describe what I go through in configuring the "URL Rewrite" module via "Site admin"->"Modules".
When I first go into "URL Rewrite" module configuration, I get the following:
Then when I click on "PHP Path Info", I'm just immediately returned to the "Modules" page (of "Site admin"), and the status of "URL Rewrite" is now "Activate" (not "Configure").
Then I click on "Activate", and the install/configuration/activation completes successfully.
So, what am I missing?
Edit: FYI, my .htaccess files are:
C:\MyServer>type gallery2\modules\rewrite\data\mod_rewrite\custom\.htaccess
# Testing mod_rewrite full functionality
Options +FollowSymLinks
RewriteEngine On
# Uncomment and edit this line as appropriate to reflect your Gallery installation.
#RewriteBase /<Your Gallery Location>/modules/rewrite/data/mod_rewrite/custom/
RewriteBase C:\MyServer\Gallery2\modules\rewrite\data\mod_rewrite\custom\
RewriteRule Rewrite.txt Works.txt [L]
C:\MyServer>type gallery2\modules\rewrite\data\mod_rewrite_no_options\custom\.htaccess
# Testing mod_rewrite full functionality
RewriteEngine On
# Uncomment and edit this line as appropriate to reflect your Gallery installation.
#RewriteBase /<Your Gallery Location>/modules/rewrite/data/mod_rewrite_no_options/custom/
RewriteBase C:\MyServer\Gallery2\modules\rewrite\data\mod_rewrite_no_options\custom\
RewriteRule Rewrite.txt Works.txt [L]
C:\MyServer>
No I hadn't seen that. I just added the lines to my httpd.conf, shut down all browsers, restarted the server, and I still get the same error. I swa some other talk of firewall interference, so I even tried shutting down Windows Firewall, but still no difference.
pelle
Joined: 2004-12-10
Posts: 389
Posted: Sat, 2006-02-18 18:29
Hmm, change rewrite base to
/Gallery2/modules/rewrite/data/mod_rewrite_no_options/custom/
if /Gallery2 is the server relative path...
Or, you should do well withouth it. It's realy only needed when the path differs like with /~user -> public_html URLs.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2006-02-18 22:09
You mean change the Windows backslashes to Unix forward slashes? OK, done. No difference.
I then commented out the RewriteBase lines completely. No difference.
The error doesn't seem to be affected at all by the .htaccess files.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2006-02-18 22:29
Pelle, I should also mention again that I am not using the default port 80 for this Gallery installation. To distinguish it from my 'production' server, I use port 800 (http://blah.blah.blah:800/Gallery2/...). Don't know if that matters.
pelle
Joined: 2004-12-10
Posts: 389
Posted: Sun, 2006-02-19 02:22
Larry Menard wrote:
You mean change the Windows backslashes to Unix forward slashes? OK, done. No difference.
No, I mean exactly what I wrote, try that rewrite base (wthouth C:\...\).
Also, make sure it reads the .htaccess (like put some random string in there and you should get a 501)
1. i've tested today to run the unit tests when using the port 8081 for my g2
2. i could successfully configure and activate the mod_rewrite based rewrite module, no manual steps required
3. i've got a lot of rewrite test failures: http://dev.nei.ch/Gallery%20Unit%20Tests_rewrite_with_non_standard_port.png
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2006-02-19 09:18
my test failures for non-standard-port test runs for the rewrite module are fixed now. it's in cvs.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2006-02-19 09:22
these tests that fail for you, pass for me, even when i use 8081 as port (nothing changed):
- IndexDotPhpTest.testHttpRedirect
- PhpVmTest.testHeader
- SessionTest.testSessionCookieHeader
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2006-02-19 09:39
pelle and i had a short talk and we came to the conclusion that the .htaccess file just doesn't get read in your case.
make sure that AllowOverride is set to "All" (and surely not set to "None") in your httpd conf and then restart the webserver.
if it is set to none, then .htaccess isn't even opened by apache. see: http://httpd.apache.org/docs/2.0/howto/htaccess.html
and if it still doesn't work, forget about it. it's not worth losing more time on this.
we just have to make sure that the tests pass and with my changes from above i hope they do now.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sun, 2006-02-19 15:14
> AllowOverride All
Done. Made no difference.
Here are all the <directory> and '.htaccess' entries in my httpd.conf:
AccessFileName .htaccess
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory "C:/MyServer">
Options Indexes FollowSymLinks
AllowOverride None
Allow from all
</Directory>
<Directory "C:/Program Files/Apache Group/Apache2/icons">
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
</Directory>
<Directory "C:/Program Files/Apache Group/Apache2/manual">
Options Indexes
AllowOverride None
Order allow,deny
Allow from all
<Files *.html>
SetHandler type-map
</Files>
SetEnvIf Request_URI ^/manual/(de|en|es|fr|ja|ko|ru)/ prefer-language=$1
RedirectMatch 301 ^/manual(?:/(de|en|es|fr|ja|ko|ru)){2,}(/.*)?$ /manual/$1$2
</Directory>
<Directory "C:/Program Files/Apache Group/Apache2/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
</Directory>
<Directory "C:\MyServer\gallery2">
AllowOverride All
</Directory>
I have tried changing "/", "C:/MyServer" and "C:\MyServer\Gallery2" to "AllowOverride All", and I tried changing all the forward slashes to backslashes, but none of it made any difference at all.
I'll try downloading the current build, but I'm pessimistic about it since you did not have the same problems I am having.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sun, 2006-02-19 15:30
Valiant, you ROCK! All the rewrite errors are now gone with the latest build. I'll never doubt you again. ;-)
Now I'll try running the whole suite again to make sure of exactly where I stand.
average for guest that has rated
Mismatch At: [] 2.6 !== 2
2.6 type:double
2 type:double
Average rating test failed.
Mismatch At: [] 2.6 !== 2
2.6 type:double
2 type:double
Looks like perhaps rounding errors in the rating tests. I'll look into them ASAP.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Mon, 2006-02-20 03:19
I'm looking into rating.RatingHelperTest.testFetchRatingsForGuestWithoutPersistentSession. I'm not sure I see what it's trying to do.
I see it insert a random number of random ratings into g2_RatingCacheMap. Then it SELECTs one of those rows from that table, and compares the value of g_AverageRating to the average that it calculated while inserting the rows.
But the values in column g_AverageRating are not averages... they are just raw values.
Also, if g_AverageRating is supposed to be an average, it should be a DECIMAL datatype, but it is in fact an INTEGER.
I really don't understand this. Any sage advice?
Thanks.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2006-02-20 05:22
values are stored in integer since the g2 storage abstraction only allows for integers and strings right now.
the floating point precision is achieved by computing
sum(rating) / "count(*) votes"
i'm away for a day now.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Mon, 2006-02-20 16:14
Are you talking about the line:
$average = round($totalRating / $votes, 3);
If so, then I know that, but that's only one side of the comparison.
The failing assertion is the comparison between:
- the calculated and rounded average, calculated based on all the rows as they are initially INSERTed (floating point)
and:
- the value SELECTed from g_AverageRating (integer).
The value SELECTed from g_AverageRating is never modified or manipulated in any way... it is immediately compared to the floating point calculated average.
And even if it were converted to a floating point number, it would still be wrong. Since the SELECT is returning only a single row, and the average of one row is the same as the original value, there's no way it can work.
The only way this can work is if the number of rows SELECTed was the same as the number initially INSERTed, then that average is rounded to 3 decimal places.
So now let's look at it from another angle. Why are we getting only one row returned? Because just a few lines earlier in the trace all contents of g2_RatingsCacheMap are DELETEd, then only one more row is INSERTed. I don't know where that DELETE or INSERT are being done, but if they are correct then we shouldn't be comparing to the $average that was calculated based on all rows.
Sorry if I'm missing something obvious.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2006-02-20 16:53
facts:
- data is stored as integer in both rating tables
- averages are computed in php, see modules/rating/classes/RatingHelper.class function fetchRatings() etc.
@unit test testFetchRatingsForGuestWithoutPersistentSession:
it adds about 5 ratings (for each of the test users one rating).
- Mismatch At: [] 3.077 !== 3
--> there's something wrong with db2 and the modules/rating/classes/RatingHelper.class function fetchRatings()
Posts: 757
I apparently do not have a mb_strlen() function, so I can't run your script. ;-)
In case it helps, here's the output of my script:
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/test_utf8Substr.out.txt
Posts: 757
Also, I have the exact same problem with tests:
- ConfirmImportControllerTest.testImportfr_FR
- ConfirmImportControllerTest.testImportKOI_8
Test ConfirmImportControllerTest.testConvertHtmlToBbcode also fails only if run along with ConfirmImportControllerTest.testImportKOI_8. It runs clean by itself or with any of the other failures. The only error message I can see is:
Test CustomFieldItemAdminControllerTest.testCustomFieldItemAdminController also fails:
[IBM][CLI Driver][DB2/NT] SQL0401N The data types of the operands for the operation "=" are not compatible. SQLSTATE=42818 SQLCODE=-401 ADOConnection._Execute(SELECT g2_ChildEntity.g_id FROM g2_ChildEntity WHERE g2_ChildEntity.g_parentId='2659')Posts: 32509
ok, my test script was not perfect. strlen() does not return the multi-byte character length. it returns the nr of bytes, i guess.
if you set the internal encoding of mb_substr / mb_strlen etc before using it, it shows that this string is 128 mulit-byte characters long.
specifically, \xc3\xa5 is a single multi-byte character and not 2 characters.
that to say: the truncation works correctly in G2. to make it work correctly in your test script, you probably have to add a few character encoding statements.
the full string that is truncated before inserting is:
which is clearly > 128 multi-byte characters.
e.g. to get mbstring working, get the php_mbstring.dll (if it isn't in your php ext/ / extensions/ folder yet) from http://www.php.net/downloads.php i guess the "collection of pecl modules" should include it. in php.ini change the ; extension=php_mbstring.dll line to extension=php_mbstring.dll and restart apache.
before the test script runs, you'll have to call mb_internal_encoding("UTF-8"); at the top.
Posts: 757
OK, I've enabled the php_mb_string.dll (and restarted Apache, even though I'm not using it right now), and added the mb_internal_encoding("UTF-8"); call. Now my script does find function mb_substr(), so it does use it instead of going through the other G2 functions. But even mb_substr() is returning a 129-byte-long string.
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/test_utf8Substr.php.txt
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/test_utf8Substr.out.txt
Doh! Never mind, I lost the mb_internal_encoding("UTF-8") call while editing my scripts. The non-database test script works now. Will now try my database-enabled test script, and then G2 itself.
Posts: 32509
as i said, you need to add
mb_internal_encoding("UTF-8");
at the beginning of your script and then you'll see that the mb_strlen() is actually 128.
Posts: 757
Yes, I did so (you replied before I updated my posting) ;-).
DB2 still doesn't like it though. I'll keep looking into it.
Posts: 757
Valiant,
The PHP manual for mb_strlen() says:
Note: "characters", not "bytes". That must be our problem. VARCHAR(128) means 128 bytes, not 128 characters.
So again... what's a reasonable size to make the column?
To safely accommodate the extreme case (128 UTF-8 4-byte non-ASCII characters), we would need to use VARCHAR(1024) and reduce g_description accordingly.
Posts: 32509
heh, i was there a few posts ago when i asked you:
because that is what i understand as a database supporting UTF-8. can you find out whether DB2 really can't store 128 UTF8 characters in a UTF-8 database table column?
edit: highlighting my own typos, i'm brave today
Posts: 757
Sorry, I guess I didn't fully understand your question... I'm not that familiar with MBCS.
I think we can confirm this is the problem. The DB2 doc (http://publib.boulder.ibm.com/infocenter/db2luw/v8//topic/com.ibm.db2.udb.doc/admin/c0004846.htm) says:
Posts: 32509
ok, that's a clear answer
since we already have an open bug to handle truncation correctly, i have added a note about fixing it for DB2 too:
http://sourceforge.net/tracker/index.php?func=detail&aid=1382037&group_id=7130&atid=107130
that means for you: ignore the 3-4 migrate failures please for now. we'll fix it after G2.1.
Posts: 757
Just when we thought we had it figured out...
I really, really, really hate to say this, but... all of a sudden, my database-enabled test script is now working. But I have no idea why! I'm quite sure I haven't changed anything since it used to fail.
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/testDb2Api.php.txt
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/g2_utf8.inc.php.txt
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/testDb2Api.out.txt
But my G2 unit test still fails with the same error.
This even seems to contradict the doc. I don't know what the #@%# is going on now.
Edit: Never mind, I take it back. I found the problem in my script (the string was hard-coded... not using the $string variable). Whew!
Posts: 32509
Posts: 757
Moving right along...
Test CustomFieldItemAdminControllerTest.testCustomFieldItemAdminController. Was getting datatype mismatches.
I first 'fixed' it by changing CustomFieldItemAdminController::handleRequest():
list ($adminMode, $itemId) = GalleryUtilities::getRequestVariables('cfAdmin', 'itemId'); $itemId = (int) $itemId;That did eliminate the DB2 errors, but it still left this error (twice):
Then I changed my 'fix' to:
//LJM list ($adminMode, $itemId) = GalleryUtilities::getRequestVariables('cfAdmin', 'itemId'); list ($adminMode, $itemId) = (int) GalleryUtilities::getRequestVariables('cfAdmin', 'itemId');Again, the DB2 errors are eliminated, but it still left errors:
and
(The 1st error occurs once, the 2nd error occurs twice.)
Are either of these 'fixes' a step in the right direction?
Posts: 32509
i have committed this patch:
Index: CustomFieldItemAdmin.inc =================================================================== RCS file: /cvsroot/gallery/gallery2/modules/customfield/CustomFieldItemAdmin.inc,v retrieving revision 1.8 diff -u -r1.8 CustomFieldItemAdmin.inc --- CustomFieldItemAdmin.inc 17 Jan 2006 01:44:12 -0000 1.8 +++ CustomFieldItemAdmin.inc 16 Feb 2006 20:38:01 -0000 @@ -48,6 +48,7 @@ list ($adminMode, $itemId) = GalleryUtilities::getRequestVariables('cfAdmin', 'itemId'); if (!empty($adminMode) && !empty($itemId)) { + $itemId = (int)$itemId; $ret = GalleryCoreApi::assertHasItemPermission($itemId, 'core.edit'); if ($ret) { return array($ret->wrap(__FILE__, __LINE__), null); Index: test/phpunit/CustomFieldItemAdminControllerTest.class =================================================================== RCS file: /cvsroot/gallery/gallery2/modules/customfield/test/phpunit/CustomFieldItemAdminControllerTest.class,v retrieving revision 1.10 diff -u -r1.10 CustomFieldItemAdminControllerTest.class --- test/phpunit/CustomFieldItemAdminControllerTest.class 10 Jan 2006 04:39:40 -0000 1.10 +++ test/phpunit/CustomFieldItemAdminControllerTest.class 16 Feb 2006 20:38:02 -0000 @@ -117,7 +117,7 @@ // Check adminMode in session.. $adminMode = $session->get(CUSTOM_FIELD_SESSION_KEY); - $this->assertEquals((string)$this->_album->getId(), $adminMode, 'Session admin mode'); + $this->assertEquals($this->_album->getId(), $adminMode, 'Session admin mode'); // Change admin mode.. GalleryUtilities::removeFormVariables('form'); @@ -138,7 +138,7 @@ 'status' => array(), 'error' => array()), $results, 'Switch to edit-settings'); $adminMode = $session->get(CUSTOM_FIELD_SESSION_KEY); - $this->assertEquals((string)$this->_album->getId(), $adminMode, 'Session admin mode'); + $this->assertEquals($this->_album->getId(), $adminMode, 'Session admin mode'); // Set values.. $ret = CustomFieldHelper::saveFieldValues($this->_item,Posts: 32509
what's our solution for TEXT / CLOB / ... ?
should we go with varchar(30000) for TEXT (session data and pluginparametermap value)?
Posts: 757
> patch
Thanks. Are there still nightly builds being sent tp jmullan's page?
> text/clob
As I recall, you had suggested using 'TEXT-MEDIUM' => 'VARCHAR(32000)'. So that's what I'm currently using, and that seems to be working well.
I'm currently running the full suite of unit tests (except the 4 failing migrate tests). The good news is that the run no longer aborts after test 529 (perhaps it's related to the migrate errors). The bad news is that there are still a handful of new errors I need to look into. I'm currently as far as test #917 of 2171.
Posts: 32509
nightly builds are made about 3 AM pacific standard time, so 14h from now.
i have changed TEXT and TEXT-MEDIUM to varchar(15000).
i have not changed the column types from TEXT to TEXT-MEDIUM.
this should be good for now. notice also that i committed another fix yesterday which should ensure that the session data doesn't skyrocket as badly as before.
Posts: 757
OK, I'll grab a fresh build tomorrow. Is it still the RC1 that I want?
Posts: 32509
just grab the next "latest" nightly snapshot, the one dated with February 17th or so.
Posts: 757
I grabbed the current build today and am running unit tests on it (all modules installed).
I'm having difficulty getting a complete run... if it doesn't die with functional problems, it dies with a 30-second timeout. Wonder if I need a beefier system to run all these unit tests. My current test system is a PIII 800 MHz, 384 Mb RAM. I guess if I have to, I can break the tests up into groups.
Will let you know how it goes. Maybe I'll give up trying to get a complete run and just start looking at specific failures.
Edit: 384 Mb, not 384 Kb.
Posts: 32509
you can change the 30 second timeout to e.g. 600 in
lib/tools/phpunit/GalleryTestCase.class
line 280, $gallery->guaranteeTimeLimit(30);
could be changed to
$gallery->guaranteeTimeLimit(600);
Posts: 757
Good to know... thanks!
Posts: 757
I shut down another app on my test system, didn't have to change the timeout. I'm still unable to get a complete run though... but there's no error, no timeout, nothing. Despite running clean up until that point, it just ends abruptly, precisely between the PermissionSetTest and PermissionTest tests.
Then I did a run specifying the numeric range of the remaining tests, and finally there were no more abrupt exits.
I have results of every testcase, over a series of 4 partial runs.
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/Gallery%20Unit%20Tests%20-%20all%20modules.htm
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/Gallery%20Unit%20Tests%20-%20all%20modules%20-%20exceptions.htm
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/Gallery%20Unit%20Tests%20-%20all%20modules%20-%20exceptions2.htm
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/Gallery%20Unit%20Tests%20-%20Specific.htm
Now I'm going to focus on those individual test failures.
Posts: 757
Valiant, could you please change:
modules/core/classes/GalleryStorage/db2Storage.class, getFunctionSql():
Posts: 32509
FETCH FIRST committed.
Posts: 32509
FIXED: RewriteHelperTest.testParseActiveRules
had to do with your :8080 port.
Posts: 757
FYI,
The FETCH FIRST n ROWS ONLY change will fix 6 ImageBlock failures. That will leave only:
- the 3 ConfirmImportControllerTest.testImport* tests (bug report opened)
- IndexDotPhpTest.testHttpRedirect (you've said I could ignore for now)
- IsapiRewriteHelperTest.testCheckIsapiRewrite
- IsapiRewriteParserTest.testNeedsConfigurationEmbedded
- IsapiRewriteParserTest.testNeedsConfiguration
- ModRewriteParserTest.testNeedsConfiguration
- PathInfoHelperTest.testCheckPathInfo
- PathInfoParserTest.testNeedsConfiguration
- RatingHelperTest.testMultiUserRateItem
- RewriteHelperTest.testParseActiveRules
Now, I'm currently looking into IsapiRewriteParserTest.testCheckIsapiRewrite... The error is:
I have determined that the failures are in the first set of asserts:
GalleryUtilities::putRequestVariable('isapiRewriteTest', 'pass'); list ($ret, $status, $trueStatus) = IsapiRewriteHelper::checkIsapiRewrite(); if ($ret) { return $this->failWithStatus($ret->wrap(__FILE__, __LINE__)); } $this->assertEquals(REWRITE_STATUS_OK, $status); $this->assertEquals(REWRITE_STATUS_OK, $trueStatus);Both these asserts fail. Both $status and $trueStatus = 31. The other 2 sets of asserts are successful (comment out the first set of asserts and the test runs clean).
Before I look into this any further, let me state up front that I do have a possible problem when configuring the "URL Rewrite" module. There's a 'Warning' message printed that support for Rewrite could not be confirmed. There are two manual test links there, they both return FAILURE (or something like that, I'm going from memory now). I have enabled mod_rewrite in my Apache.
Any suggestions as to this?
Thanks.
Posts: 32509
i need sleep now. i asked pelle, the author of rewrite to assist you in the rewrite test failures. but he won't be there in the next 10h, or maybe even longer.
see my last post, i've fixed one of your failures.
Posts: 757
Thanks. Bon soir.
Posts: 389
The test should use a mock platform and has nothing to do with if Isapi Rewrite actually works or not. I'll see if I can reproduce the test failures on a windows machine (with mysql though).
As for the mod_rewrite test they are pretty accurate. If you get FAIL_NO_REWRITE chanses are that you have missed some part of the mod_rewrite configuration.
Posts: 32509
i can't reproduce the failures and i'm always testing with windows xp / apache2 / mod_php 4 / 5 lately. but mod_rewrite and pathinfo work for me.
Posts: 757
Hi, Pelle. Thanks for looking into it. Let me describe what I go through in configuring the "URL Rewrite" module via "Site admin"->"Modules".
When I first go into "URL Rewrite" module configuration, I get the following:
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/mod_rewrite_1.jpg
Then when I click on "Apache mod_rewrite" I get:
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/mod_rewrite_2.jpg
Note the " the "Warning" beside "Apache mod_rewrite". When I click on either of the two manual tests I get "FAIL_NO_REWRITE".
Then I click on "Back", to go back to:
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/mod_rewrite_1.jpg
Then when I click on "PHP Path Info", I'm just immediately returned to the "Modules" page (of "Site admin"), and the status of "URL Rewrite" is now "Activate" (not "Configure").
Then I click on "Activate", and the install/configuration/activation completes successfully.
So, what am I missing?
Edit: FYI, my .htaccess files are:
Posts: 389
Your custom .htaccess files looks fine. Have you read this?
http://codex.gallery2.org/index.php/Gallery2:Modules:rewrite#I_know_mod_rewrite_is_loaded.2C_why_is_Gallery_telling_me_it.27s_not_working.3F
Posts: 757
No I hadn't seen that. I just added the lines to my httpd.conf, shut down all browsers, restarted the server, and I still get the same error. I swa some other talk of firewall interference, so I even tried shutting down Windows Firewall, but still no difference.
Posts: 389
Hmm, change rewrite base to
/Gallery2/modules/rewrite/data/mod_rewrite_no_options/custom/
if /Gallery2 is the server relative path...
Or, you should do well withouth it. It's realy only needed when the path differs like with /~user -> public_html URLs.
Posts: 757
You mean change the Windows backslashes to Unix forward slashes? OK, done. No difference.
I then commented out the RewriteBase lines completely. No difference.
The error doesn't seem to be affected at all by the .htaccess files.
Posts: 757
Pelle, I should also mention again that I am not using the default port 80 for this Gallery installation. To distinguish it from my 'production' server, I use port 800 (http://blah.blah.blah:800/Gallery2/...). Don't know if that matters.
Posts: 389
No, I mean exactly what I wrote, try that rewrite base (wthouth C:\...\).
Also, make sure it reads the .htaccess (like put some random string in there and you should get a 501)
Posts: 757
OK, I've tried:
#RewriteBase /<Your Gallery Location>/modules/rewrite/data/mod_rewrite/custom/
#RewriteBase C:\MyServer\Gallery2\modules\rewrite\data\mod_rewrite\custom\
#RewriteBase C:/MyServer/Gallery2/modules/rewrite/data/mod_rewrite/custom/
#RewriteBase /Gallery2/modules/rewrite/data/mod_rewrite/custom/
#RewriteBase Gallery2/modules/rewrite/data/mod_rewrite/custom/
#RewriteBase Fiddle-Dee-Friggin-Dee
#FuddleDuddle
FuddleDuddle JackShit
Nothing makes any difference at all.
Posts: 32509
1. i've tested today to run the unit tests when using the port 8081 for my g2
2. i could successfully configure and activate the mod_rewrite based rewrite module, no manual steps required
3. i've got a lot of rewrite test failures:
http://dev.nei.ch/Gallery%20Unit%20Tests_rewrite_with_non_standard_port.png
Posts: 32509
my test failures for non-standard-port test runs for the rewrite module are fixed now. it's in cvs.
Posts: 32509
these tests that fail for you, pass for me, even when i use 8081 as port (nothing changed):
- IndexDotPhpTest.testHttpRedirect
- PhpVmTest.testHeader
- SessionTest.testSessionCookieHeader
Posts: 32509
pelle and i had a short talk and we came to the conclusion that the .htaccess file just doesn't get read in your case.
make sure that AllowOverride is set to "All" (and surely not set to "None") in your httpd conf and then restart the webserver.
if it is set to none, then .htaccess isn't even opened by apache. see: http://httpd.apache.org/docs/2.0/howto/htaccess.html
and if it still doesn't work, forget about it. it's not worth losing more time on this.
we just have to make sure that the tests pass and with my changes from above i hope they do now.
Posts: 757
> AllowOverride All
Done. Made no difference.
Here are all the <directory> and '.htaccess' entries in my httpd.conf:
AccessFileName .htaccess <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory "C:/MyServer"> Options Indexes FollowSymLinks AllowOverride None Allow from all </Directory> <Directory "C:/Program Files/Apache Group/Apache2/icons"> Options Indexes MultiViews AllowOverride None Order allow,deny Allow from all </Directory> <Directory "C:/Program Files/Apache Group/Apache2/manual"> Options Indexes AllowOverride None Order allow,deny Allow from all <Files *.html> SetHandler type-map </Files> SetEnvIf Request_URI ^/manual/(de|en|es|fr|ja|ko|ru)/ prefer-language=$1 RedirectMatch 301 ^/manual(?:/(de|en|es|fr|ja|ko|ru)){2,}(/.*)?$ /manual/$1$2 </Directory> <Directory "C:/Program Files/Apache Group/Apache2/cgi-bin"> AllowOverride None Options None Order allow,deny Allow from all </Directory> <Directory "C:\MyServer\gallery2"> AllowOverride All </Directory>I have tried changing "/", "C:/MyServer" and "C:\MyServer\Gallery2" to "AllowOverride All", and I tried changing all the forward slashes to backslashes, but none of it made any difference at all.
I'll try downloading the current build, but I'm pessimistic about it since you did not have the same problems I am having.
Posts: 757
Valiant, you ROCK! All the rewrite errors are now gone with the latest build. I'll never doubt you again. ;-)
Now I'll try running the whole suite again to make sure of exactly where I stand.
Thanks guys.
Posts: 757
OK, run is done. Had to do it in 3 parts.
Part 1:
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/Gallery%20Unit%20Tests%20-%20part%201.htm
Failures: 4
migrate.ConfirmImportControllerTest.testImportno_NO (Known problem with UTF-8 string truncation.)
migrate.ConfirmImportControllerTest.testImportfr_FR (Known problem with UTF-8 string truncation.)
migrate.ConfirmImportControllerTest.testImportKOI_8 (Known problem with UTF-8 string truncation.)
migrate ConfirmImportControllerTest testConvertHtmlToBbcode (Runs clean in isolation. Only fails if run with testImportKOI_8)
Part 2:
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/Gallery%20Unit%20Tests%20-%20part%202.htm
Failures: 2
core.IndexDotPhpTest.testHttpRedirect (You've said we can ignore this for now.)
rating.RatingHelperTest.testFetchRatingsForGuestWithoutPersistentSession
Part 3:
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/Gallery%20Unit%20Tests%20-%20part%203.htm
Failures: None.
Looks like perhaps rounding errors in the rating tests. I'll look into them ASAP.
Posts: 757
I'm looking into rating.RatingHelperTest.testFetchRatingsForGuestWithoutPersistentSession. I'm not sure I see what it's trying to do.
I see it insert a random number of random ratings into g2_RatingCacheMap. Then it SELECTs one of those rows from that table, and compares the value of g_AverageRating to the average that it calculated while inserting the rows.
But the values in column g_AverageRating are not averages... they are just raw values.
Look for "fetching" in this (large) trace:
http://cpe0013102da23b-cm0f0079804905.cpe.net.cable.rogers.com/misc/Gallery2/Gallery%20Unit%20Tests%20-%20rating.htm
Also, if g_AverageRating is supposed to be an average, it should be a DECIMAL datatype, but it is in fact an INTEGER.
I really don't understand this. Any sage advice?
Thanks.
Posts: 32509
values are stored in integer since the g2 storage abstraction only allows for integers and strings right now.
the floating point precision is achieved by computing
sum(rating) / "count(*) votes"
i'm away for a day now.
Posts: 757
Are you talking about the line:
$average = round($totalRating / $votes, 3);If so, then I know that, but that's only one side of the comparison.
The failing assertion is the comparison between:
- the calculated and rounded average, calculated based on all the rows as they are initially INSERTed (floating point)
and:
- the value SELECTed from g_AverageRating (integer).
The value SELECTed from g_AverageRating is never modified or manipulated in any way... it is immediately compared to the floating point calculated average.
And even if it were converted to a floating point number, it would still be wrong. Since the SELECT is returning only a single row, and the average of one row is the same as the original value, there's no way it can work.
The only way this can work is if the number of rows SELECTed was the same as the number initially INSERTed, then that average is rounded to 3 decimal places.
So now let's look at it from another angle. Why are we getting only one row returned? Because just a few lines earlier in the trace all contents of g2_RatingsCacheMap are DELETEd, then only one more row is INSERTed. I don't know where that DELETE or INSERT are being done, but if they are correct then we shouldn't be comparing to the $average that was calculated based on all rows.
Sorry if I'm missing something obvious.
Posts: 32509
facts:
- data is stored as integer in both rating tables
- averages are computed in php, see modules/rating/classes/RatingHelper.class function fetchRatings() etc.
@unit test testFetchRatingsForGuestWithoutPersistentSession:
it adds about 5 ratings (for each of the test users one rating).
- Mismatch At: [] 3.077 !== 3
--> there's something wrong with db2 and the modules/rating/classes/RatingHelper.class function fetchRatings()