Microsoft SQL Support

Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sat, 2006-07-01 17:18

> StorageTest.testSqlInjectionWithNullCharacter

Not as simple as what I thought... I guess I was thinking of something else.

The problem is that SQL Server is choking while parsing the SQL statement:

Quote:
ado_mssql error: [102: [Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near 'The NULL character should be escaped '.] in EXECUTE("
INSERT INTO g2_PluginParameterMap (g_pluginType, g_pluginId, g_itemId, g_parameterName, g_parameterValue)
VALUES ('module','unitTestModule',1,'test19476','The NULL character should be escaped !')")

I'm trying to create a simple script that repros the scenario, but I'm not sure if I've reproed the same problem or have hit a different one:

Quote:
<?php

$localhost = exec("hostname");
$database = "gallery2";
$uid = "g2user";
$pwd = "g2pwd";
$connectString = "Host=PROVIDER=MSDASQL;DRIVER={SQL Server};";
$connectString .= "SERVER=$localhost\\sqlexpress;";
$connectString .= "DATABASE=$database;";
$connectString .= "UID=$uid;PWD=$pwd";

$sqlTableDrop = "drop table ljmtemp";
$sqlTableCreate = "create table ljmtemp (col1 nvarchar(100))";
$sqlTableQuery = "select len(col1) from ljmtemp";

// Connect to the db
$db = new COM("ADODB.Connection") or die("Cannot start ADO");
$db->open($connectString);

// Drop & recreate the table
$db->Execute ($sqlTableDrop);
$db->Execute ($sqlTableCreate);

// Insert the test data
$testString = "The NULL character should be escaped \0 !";
//$testString = "This is a test string.";
$res = $db->Execute("insert into ljmtemp (col1) values ('$testString')");
if (!$res) die ("INSERT failed");

// Query the table
$rs = $db->Execute("select col1, len(col1) as length from ljmtemp");
if (!$rs) die ("SELECT failed");

while (!$rs->EOF) {
$fv1 = $rs->Fields("col1");
$fv2 = $rs->Fields("length");
printf("COL1 = \"$fv1->value\", LENGTH = \"$fv2->value\"\n");
$rs->MoveNext();
}
$rs->Close();

// Disconnect from the db
$db->Close();

?>

Results in:

Quote:
C:\MyServer>php testMsSqlInsertNull.php
PHP Fatal error: Uncaught exception 'com_exception' with message 'Source: Microsoft OLE DB Provider for ODBC Drivers
Description: [Microsoft][ODBC SQL Server Driver][SQL Server]Unclosed quotation mark after the character string 'The NULL character should be escaped '.'
in C:\MyServer\testMsSqlInsertNull.php:27
Stack trace:
#0 C:\MyServer\testMsSqlInsertNull.php(27): com->Execute('insert into ljm...')
#1 {main}
thrown in C:\MyServer\testMsSqlInsertNull.php on line 27

Perhaps PHP(?) is processing the \0 as a string terminator. But if I double the backslash the actual literal '\0' gets inserted into the db.

Suggestions?

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Sat, 2006-07-01 19:15

valiant worked on this issue with adodb/postgres. maybe email -devel to get his input on this.. it's possible a fix in adodb will be needed.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2006-07-02 01:27

$testString = "The NULL character should be escaped \0 !"; creates a PHP string which actually includes the NULL character.
var_dump, print, strlen, ... all read the string until to the very end ("!").

if the MS SQL driver chokes up on this original string without escaping, fine. if it still chokes up when you escape it or even escape it twice, then the driver is buggy.
the MSDN guys should be able to help you. maybe there are other resources too.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Tue, 2006-07-04 23:48

> ModuleTest.testModuleCycle and ModuleTest.testPermissionsCleanup

These are new errors, and are happening because of the lack of BITOR support in MSSQL. The error is ERROR_UNSUPPORTED_FUNCTION from the BITOR case in getFunctionSql(). I haven't checked mindless' change yet, but why are we going into this code? How does G2 determine whether to do the BITOR in the database or PHP?

Update: I found the problem. Apparently these two tests call getFunctionSql() and have their own copy of _fetchPermissionBitsForItem(). I modified it as follows:

    function _fetchPermissionBitsForItem($itemId) {
	global $gallery;

	$userId = $gallery->getActiveUserId();
	list ($ret, $groupIds) = GalleryCoreApi::fetchGroupsForUser($userId);
	if ($ret) {
	    return array($ret, null);
	}

	$groupIds = array_keys($groupIds);
	$userOrGroupIdMarkers = GalleryUtilities::makeMarkers(count($groupIds) + 1);

	$storage =& $gallery->getStorage();
	list ($ret, $bitOr) = $storage->getFunctionSql(
				'BIT_OR', array('[GalleryAccessMap::permission]',
						'[GalleryAccessSubscriberMap::itemId]'));
	if ($ret) {
//<LJM>
	    /* Try to OR permissions using aggregate function in DB; fallback to OR in PHP */
	    if ($ret->getErrorCode() & ERROR_UNSUPPORTED_OPERATION) {
		$bitOr = '[GalleryAccessMap::permission]';
		$groupBy = '';
	    } else {
//</LJM>
	    return array($ret, null);
	}
//<LJM>
}
//</LJM>

	$query = '
	SELECT
	  ' . $bitOr . '
	FROM
	  [GalleryAccessMap], [GalleryAccessSubscriberMap]
	WHERE
	  [GalleryAccessSubscriberMap::itemId] = ?
	  AND
	  [GalleryAccessSubscriberMap::accessListId] = [GalleryAccessMap::accessListId]
	  AND
	  [GalleryAccessMap::userOrGroupId] IN (' . $userOrGroupIdMarkers . ')
	';

and both these tests now run clean. Mindless, if you could please check this in too?

> RatingHelperTest.testFetchRatingsForGuestWithoutPersistentSession


We had this same problem in DB2, and it was because DB2's AVG() function was returning an INTEGER datatype when the subject column was of type INTEGER. We fixed it by casting to type double. According to the MSSQL doc, MSSQL's AVG() function behaves similarly, but that same change doesn't seem to be valid on MSSQL (SQL syntax error). Then I tried using CAST(x AS y):

$sql = sprintf('cast(AVG(%s) as decimal(4,3))', $args[0]);

This seems to be syntactically OK, but it still hasn't fixed the original problem... it's applying the CAST to the output of the AVG() call, which already has lost the decimal remainder. I can't see how to apply the CAST() to the base column before the AVG() is applied.

Correction... I've found the solution:

$sql = sprintf('AVG(CAST(%s AS FLOAT))', $args[0]);

Unit test RatingHelperTest.testFetchRatingsForGuestWithoutPersistentSession now runs clean. Mindless, could you please checkin this fix?

> StorageTest.testSqlInjectionWithNullCharacter

Still no responses to this in the MSSQL forum. I've now also sent it to a couple of PHP mailing lists.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2006-07-05 01:36

Mindless, I've been meaning to ask you this...

I haven't been activating the "Multiroot" plugin for these test runs because I get an error when I try to configure it. (I'm not even sure I'm doing it right.)

On the configuration page I enter "Gallery2/vroot" in the "URI for new guest view:" field, then click on "Generate Files". But I just get error:

Quote:
Warning: file_get_contents(C:\MyServer\gallery2\.htaccess) [function.file-get-contents]: failed to open stream: No such file or directory in C:\MyServer\gallery2\modules\core\classes\GalleryPlatform.class on line 339

Sure enough I have no .htaccess file in my Gallery root. Should I have one?

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Wed, 2006-07-05 16:14

fixes above committed.. sorry about the BIT_OR one.. I scanned all G2 code for uses of BIT_OR but didn't look at unit tests.

Multiroot module is a setup assistanat and cannot be activated.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2006-07-05 19:00

Thanks, Mindless.

So I assume that Multiroot does not have any Unit Tests that are not being run. Which means the only Unit Test failures I have are now:

- known mb_strlen() issues
- should be run only on svn builds
- unable to insert \0 (still hoping for responses to various forum postings)

I'll now:

- put some documentation in http://codex.gallery2.org/index.php/Gallery2:MSSQL,
- get a fresh build and reverify the results are still good,
- update the Test Matrix.

Is the StorageTest.testSqlInjectionWithNullCharacter test failure a big enough problem to prevent MSSQL from being formally supported in G2.2?

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Wed, 2006-07-05 20:58

would be nice to figure that last one out, but no i don't think it will hold up support for mssql.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Thu, 2006-07-06 20:05

I upgraded my build to revision 13921, re-ran all unit tests, and achieved the same results... only the 1 unresolved error (StorageTest.testSqlInjectionWithNullCharacter).

There is now doc in http://codex.gallery2.org/index.php/Gallery2:MSSQL.

While updating the Test Matrix I realized that I don't know if the matrix is supposed to reflect current Production builds or Development builds. Until I know for sure, I've removed the MSSQL information from the matrix.

Mindless, could you please remove the "Experimental" warning from the Installer Step 5 selection for the MS SQL Server database type? The complete description of the database server on which I did my dev and test is "Microsoft SQL Server 2005 Express Edition". Thanks.

FYI: h0bbel has volunteered to try running the unit tests using a commercial edition of SQL Server 2005. If that is successful, we could remove the "Express Edition" from the Installer description.

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Sat, 2006-07-08 17:02

test matrix is for current svn. i'm just putting "2005 or newer" for mssql support. thanks!

 
aavdberg
aavdberg's picture

Joined: 2002-09-29
Posts: 100
Posted: Sat, 2006-11-04 17:02

Hello,

I have installed the latest build on test machine with the db on a other server and now i'm running the Gallery v2 on SQL 2k5 Std. I ask myself is there any way to export the MySQL db and import it into the SQL 2k5 Std db?

Greetings from,

André van den Berg.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2006-11-04 18:06

We don't have any tool / script to help you in this transition.
Also, note that MS SQL support in G2 does not support international characters yet.

It's much too early to consider switching from MySQL to MS SQL when it comes to G2.