Support for DB2 databases?

Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sat, 2005-10-01 17:43

Folks,

I'm brand new to Gallery, just downloaded G2.0. Haven't been able to install it yet because I don't have a currently-supported database. I was hoping I could make it work with a DB2 database, but the install program hard-codes the choices.. Any chance it could be made to work with a DB2 database? I'm not a hardcore developer, but I have years of experience with DB2 and would be happy to help G2 developers implement support for DB2.

I'm running Windows XP SP2, Internet Explorer 6.0.2900.2180.xpsp_sp2_gdr.050301-1519, PHP 4.3.7, Apache 2.0.49, and DB2 V8.2.2.

Thanks.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2005-10-01 20:53

it would be cool if we could could add DB2 support.
and you're more than welcome helping us there.
i'd prefer if we could guide you to do the necessary steps :)

in G2, the DDL (CREATE TABLE, ALTER TABLE, sequences, keys, ..) SQL is generated automatically.
we use XML to describe our PHP classes, then we use XML parsers to extract this information from the php files and generate directly SQL CREATE TABLE statements.

basically, you just need to create 2 files. 1 file to translate XML to DDL SQL (create table,..) and 1 file for DML SQL (special SQL functions like concat, bit_and, ... are not implemented the same way in all DBMS).

first get the latest g2 dev version, not the 2.0 release since exactly this automatic sql generation has changed since then.
get it via cvs or the latest nightly snapshot.

ok, let's first start with the create table SQL generation:

/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform/
- add db2 as a subfolder parallel to mysql, postgresql and oracle
- in GNUMakefile, in the scrub section, add an analogous line for db2
- open /gallery2/lib/tools/bin/generate-sql.php
- copy the class that represents the most similar DBMS to db2 (either Postgres, mysql or oracle class)
- change the class name and begin your customization
- i'm sure there will be questions, don't hesitate joining irc.freenode.net #gallery to chat with bharat or another core developer and asking your questions

- to finally generate sql, you need to run gmake in /gallery2/modules/core/classes/ and this has some dependencies (gmake, rxp, perl, php) but you can submit us your changes and we can run gmake for you too..

for the DML, you need to cd to
/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/
and copy again the most similar dbms file to a Db2DatabaseStorage.class and begin customizing it.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sat, 2005-10-01 22:34

Thanks for the response, Valiant.

> i'd prefer if we could guide you to do the necessary steps

That's the *only* way this would happen. ;-)

OK, will follow your instructions ASAP (maybe even tomorrow) and get back to you. Note that I can only test this on my Windows environment, I don't currently have a Linux/UNIX environment at my disposal (never mind one with DB2 installed ;-) ).

Let me know (larry.menard@rogers.com) if you want to continue this discussion offline as opposed to via the forum.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 03:20

Update: Please bear with me, as I'm a Windows user I'm not particularly well versed in Gnu-type things, and I have no idea what you mean by "joining irc.freenode.net #gallery".

So far, I've 'Gnu-ized' my Windows system to the point that I can build this puppy with the changes you described.

Now when I do the Install, I get a warning in the "System Checks", particularly in the "Gallery file integrity" step. The message says: "Modified files (4)". I gather that's expected (it's still considered successful by the install program), so I continue.

But then in the "Database setup" step, I still don't have a DB2 selection in the drop-down field (not too surprisingly).

Next suggestion?

As I said, we can take this offline onto e-mail (larry.menard@rogers.com) if we don't want to bore or annoy the sandbaggers. ;-)

Thanks.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-10-02 10:25

@IRC:
IRC is internet relay chat. so i offered you to use a irc client to join our chat channel called #gallery on this irc server irc.freenode.net.
most people use Gaim or mIRC to join chat rooms, i use opera, because it's so easy with opera. Gaim is also quite easy, mIRC is very powerful, but IMO, not very intuitive.

@file integrity:
that's fine. since you changed now some files, the checksums we've computed before don't match anymore. you can ignore this warning.

@database setup step:
i didn't expect you to be so fast, so my instructions were incomplete :)

what you still have to do:
- installer: install/steps/DatabaseSetupStep.class
after:

	    case 'mysqlt':
		if (!function_exists('mysql_connect')) {
		    $templateData['error']['phpDbMissing'] =
			_('You must have the MySQL PHP module installed');
		}
		$dbPlatformType = 'mysql';
		break;

add an analogous block for db2. function_exists('db2_connect') should be fine as a test.
in

foreach (array('mysql'  => _('MySQL (all versions)'),
			   'mysqlt' => _('MySQL with Transactions (v3.23.34a and newer)'),
			   'postgres7' => _('PostgreSQL v7.x'),
			   'postgres' => _('PostgreSQL v6.x (not well tested)'),
			   'oci8po' => _('Oracle (9i and newer)'))

add something like 'db2' => _('DB2 v...'),

- not important, but nice: in install/config.php-template add a comment about db2 in

 * The possible database types are:
 *  mysql        Standard MySQL
 *  mysqlt       MySQL with transactions (3.23.34a and newer)
 *  postgres     PostgreSQL 6.x (not rigorously tested)
 *  postgres7    PostgreSQL 7.x
 *  oci8po       Oracle (9i and newer)
 */

- in modules/core/classes/GalleryStorage.class
add after

case 'postgres':
	case 'postgres7':
	    GalleryCoreApi::relativeRequireOnce(
		'modules/core/classes' . $base . 'DatabaseStorage/PostgreSqlDatabaseStorage.class');
	    $this->_impl = new PostgreSqlDatabaseStorage($config);
	    break;

an analogous case for db2.

- finally add the adodb db2 driver to G2. adodb supports a lot of dbs, we only included the drivers in G2 that we actually need.
get the latest php adodb from http://adodb.sourceforge.net/
and copy the adodb-db2.inc.php (or whatever its name is) driver from this downloaded archive (drivers/ folder) to gallery2/lib/adodb/drivers/adodb-db2.inc.php.

i guess that's it.

@email:
actually, i prefer the forums. plus if someone else wants to add ms sql server support, the person could just read our discussion here.
but i'm moving the topic to the development forums :)

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 14:16

OK, done.

Then when I tried the installer, I got the error caused by "function_exists('db2_connect')" failing.

I see in the DB2 sample applications that they use function "odbc_connect()" to connect to their databases, so I changed "db2_connect" to "odbc_connect" in DatabaseSetupStep.class.

Now I no longer get the error, but neither do I get to the next step in the install... all I get is a blank browser. Nothing between the BODY and /BODY tags.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 14:47

If it helps, I put a couple of debug prontf()s in the code, and I can see that it seems to go away in the call to ADONewConnection():

printf("I get here, config type = %s\n", $this->_config['type']);
$this->_db =& ADONewConnection($this->_config['type']);
printf("But I never get here\n");

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-10-02 15:00

right, a test for odbc_connect should be used, not db2_connect. just adodb-db2.inc.php and they use odbc stuff their too.
but odbc_connect isn't really db2 specific. maybe you find a better test to detect db2 support. after all, the function_exists thing is just there to find out whether the php installation supports db2.

i'd keep debugging DatabaseSetupStep.class, add there some print statements...
and make sure your php error_reporting level is set to E_ALL plus check your apache error log.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 15:45

error_reporting is set to E_ALL, and nothing in the Apache logs in the last 2 days.

I plugged a few more printf()s in it, and in "adodb.inc.php" it seems to never return from:

printf("<br>In ADOLoadCode, db = %s, class = %s, file = %s.\n", $db, $class, $file);
@include_once($file);
printf("<br>In ADOLoadCode, I do not get here.\n");

Result:

In file DatabaseSetupStep.class, I get here, config type = db2.
In ADONewConnection, db = db2.
In ADOLoadCode, db = db2, class = db2, file = C:\My Server\gallery2\lib\adodb/drivers/adodb-db2.inc.php.
In adodb-db2.inc.php, I get here.

But that's all I get. It seems to never return to ADOLoadCode() from include_once().

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 16:06

In fact, even if I remove *all* the code from the "adodb-db2.inc.php" file (except the ADODB_DB2{} class and constructor), it still gets the same problem. Am I wrong in expecting that it should return to ADOLoadCode()?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-10-02 16:16

remove the @ in front of the include once, or even replace @include_once with require_once

@ will suppress any errors. and require is even more restrictive = better for debugging.
maybe spaces are a problem in dirnames on windows, no idea.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 16:29

OK, it *was* finding the "adodb-db2.inc.php" file (as I thought) but it turns out I also had to copy "adodb-odbc.inc.php" into "gallery2\lib\adodb\drivers".

I'm back on the rails, will let you know if/when I hit another wall.

Thanks again. :-)

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 17:06

Well, that didn't take long, did it? :-)

The database connect is failing. Messages are:

----------------------------
For odbc Connect(), gallery2 is not used. Place dsn in 1st parameter.

Warning: odbc_connect(): SQL error: [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified, SQL state IM002 in SQLConnect in C:\My Server\gallery2\lib\adodb\drivers\adodb-odbc.inc.php on line 61
localhost: Connection error to server 'localhost' with user 'lmenard'
----------------------------

Looking at that code in "adodb-odbc.inc.php", I presume the "not used" message is informational, not fatal (because debug is enabled). And on the odbc_connect() call, $argDSN *is* specified first.

So I printed out the value of $argDSN, and for some reason it is "localhost".

Shouldn't the DSN be the same as the database name?

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 17:42

After reconsideration, I think maybe I'm misinterpreting the message. Maybe it is telling me that whoever *calls* the "_connect()" function in "adodb-odbc.inc.php" needs to do the arg manipulation. But who calls it? I don't see where it's being called from... not even in "adodb-db2.inc.php".

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-10-02 17:43

it has been some time since i last worked with ODBC on windows.
but as far as i remember, i had to setup all ODBC connections in a windows odbc manager. e.g. like http://www.databasejournal.com/features/mssql/article.php/2238221

so instead of db host you'd use the DSN in the g2 db host field in the database setup step.
try entering your dsn there instead of localhost.

does that make sense? or should the db name be used as dsn?

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 17:54

Well, I did think of that, but rather than force the User to do something different in some cases, I would think that it makes more sense for G2 or ADODB to handle the difference under the covers when required.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-10-02 18:19

well, first get it working, then let's discuss about usability issues :)
we can change the html form / add notices etc of course.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 18:24

Yeah, I did so, and the installer does at least proceed to the next page. I'm now getting:

-----------------------------------------
The database privileges test did not complete successfully.
SQL test file "C:\My Server\gallery2\install\steps\..\..\modules\core\classes\GalleryStorage\DatabaseStorage\schema\platform\db2\InstallerTest_sql" not found.
-----------------------------------------

Sure enough, that directory is empty. Did I miss a step somewhere?

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-02 18:45

Just for completeness... yes I did run gmake in modules/core/classes as you said originally. Here's the optput of that command (it doesn't say anything about the db2 directory):

cd interfaces && gmake - --unix
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/interfaces'
perl ../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../lib/tools/dtd/GalleryClass2.0.dtd --stub-ok --quiet --out-dir=tmp ../*.class
php ../../../../lib/tools/bin/generate-interfaces.php GalleryCore
Content-type: text/html
X-Powered-By: PHP/4.3.7

gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/interfaces'
cd GalleryStorage/DatabaseStorage/schema && gmake - --unix
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema'
cd xml-src && gmake - --unix -w xml
gmake[2]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/xml-src'
perl ../../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp ../../../../*.class
cp *.xml ../xml-out;
php -f ../../../../../../../lib/tools/bin/generate-dbxml.php
gmake[2]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/xml-src'
cd platform && gmake - --unix -w sql
gmake[2]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform'
php -f ../../../../../../../lib/tools/bin/generate-sql.php ../xml-out
gmake[2]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform'
gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema'

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2005-10-03 18:55

please delete all files in gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform/mysql/

then do the gmake in gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform/ again (or in gallery2/modules/core/classes/).

is mysql/ populated with sql files afterwards?
and do you use the latest g2 snapshots / cvs version (from friday last week or newer or)?

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Mon, 2005-10-03 19:48

Nope, there is nothing in the "mysql" after running gmake.

I now believe I may have grabbed the wrong build. I couldn't find an obvious link to nightly build images from the Gallery home page. I found a bunch of nightly builds on http://galleryupdates.jpmullan.com, but wasn't sure if they were developer builds. So I grabbed the developer build from the home page, which seems to be from Sept. 12th.

I'll try grabbing the latest build from http://galleryupdates.jpmullan.com... hopefully they are developer builds.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2005-10-03 20:28

i guess the nightly builds should include everything you need. else use cvs. but i don't think cvs is necessary.
to run gmake successfully, i guess you also need rxp.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Tue, 2005-10-04 02:27

OK, I got "gallery-2.0+.zip" from http://galleryupdates.jpmullan.com.

Now I'm unable to get the gmake to work. First it squawked about:

Quote:
PHP Warning: main(../../../../lib/smarty/Smarty.class.php): failed to open stream: No such file or directory in c:\My Server\gallery2\lib\tools\bin\generate-interfaces.php on line 28

but I resolved that by hard-coding the entire path to that file. Now it gets past that, but fails with:

Quote:
cd interfaces && gmake - --unix
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/interfaces'
perl ../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../lib/tools/dtd/GalleryClass2.0.dtd --stub-ok --quiet --out-dir=tmp ../*.class
php ../../../../lib/tools/bin/generate-interfaces.php GalleryCore
Content-type: text/html
X-Powered-By: PHP/4.3.7

Usage: generate-interfaces.php <PackageName>
PHP Notice: Undefined variable: argv in c:\My Server\gallery2\lib\tools\bin\generate-interfaces.php on line 42
gmake[1]: *** [inc] Error 1
gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/interfaces'
gmake: *** [all] Error 2

I'm at a loss to explain that one.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-04 08:14

maybe just put a patch file somewhere online and we can then gmake on one of our linux boxes.
do you already have cvs working on your windows box?
either use cygwin (highly recommended linux emulation for windows) or http://www.tortoisecvs.org/index.shtml

with cygwin, you could maybe get everything to work. or at least cvs.
with tortoisecvs you could install a windows cvs client. with cvs, you could create a patch file (diff -Nuw).

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Tue, 2005-10-04 13:46

I figured this one out too. Needed to set "register_argc_argv = On" in my php.ini.

It then squawked about the unix-style 'find' command. It was finding the Windows version of the command. I pointed it to my copy of the unix-style 'find.exe' and it got past that.

It then squawked again about a relative path "../../../../../../../lib/smarty/Smarty.class.php" in "generate-dbxml.php", so I again hard-coded the entire path.

The gmake is now successful,

Quote:
cd interfaces && gmake - --unix
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/interfaces'
perl ../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../lib/tools/dtd/GalleryClass2.0.dtd --stub-ok --quiet --out-dir=tmp ../*.class
php ../../../../lib/tools/bin/generate-interfaces.php GalleryCore
Content-type: text/html
X-Powered-By: PHP/4.3.7

gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/interfaces'
cd GalleryStorage/DatabaseStorage/schema && gmake - --unix
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema'
cd xml-src && gmake - --unix -w xml
gmake[2]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/xml-src'
perl ../../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp ../../../../*.class
c:/util/find.exe . -name '*.xml' -exec cp {} ../xml-out \;
php -f ../../../../../../../lib/tools/bin/generate-dbxml.php
gmake[2]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/xml-src'
cd platform && gmake - --unix -w sql
gmake[2]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform'
php -f ../../../../../../../lib/tools/bin/generate-sql.php ../xml-out
gmake[2]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform'
gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema'

but I still have no files in "gallery2\modules\core\classes\GalleryStorage\DatabaseStorage\schema\platform\db2". I then renamed the "mysql" directory and re-created an empty one, and after re-running gmake there is nothing in there either. So somebody somewhere is apparently not reporting an error.

I'm trying to attach my patched files here, but am getting errors about "invalid extension". It seems to insist on adding ".txt" to the file name.

So screw that, here's a link to my updated files.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-04 16:44

please rename your .php file to .txt, else i cannot download it.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-04 16:48

and in generate-sql.php, don't forget to replace
foreach (array('mysql', 'postgres', 'oracle') as $db) {
with
foreach (array('mysql', 'postgres', 'oracle', 'db2') as $db) {

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Tue, 2005-10-04 17:34

Done. Still doesn't work.

I've updated the file on my server, use the same link as above.

FYI, I haven't actually modified any of the SQL yet. I've just copied the Oracle code as-is until I can at least see the generated statements... it'll be easier for me to read the generated SQL instead of the generation code.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-04 21:17

yep, works for me. the db2 sql is generated too. but i don't send it to you, since as you said, it's just the oracle sql at the moment.

what about your gmake flags? is the unix thing required?

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Tue, 2005-10-04 22:12

The "--unix" is apparently coming from $(MAKEFLAGS). It looks like that's the default unless I specify "--win32" on my "gmake" command. But when I do so, it just breaks other things further down:

Quote:
C:\My Server\gallery2\modules\core\classes>gmake --win32 2>&1 | tee gmake.out
cd interfaces && gmake - --win32
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/interfaces'
! was unexpected at this time.
gmake[1]: *** [inc] Error 1
gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/interfaces'
gmake: *** [all] Error 2

Could you send me the generated DB2 SQL files? I'll reverse-engineer them and come up with a "generate-sql.php" that generates vlid DB2 SQL.

Sorry, but it looks like unless we can get these makefiles to work on Windows, I might need to ask you to do the occasional build for me. :-(

Thanks.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-04 22:20

as i said, the db2/*.sql will look like the oracle/*.sql files if you just have copied the oracle class in generate-sql.php.

maybe bharat can tell you why gmake doesn't work for you correctly.
but you should definitely try cygwin, if you haven't already.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Tue, 2005-10-04 22:39

Right, sorry, I didn't make the mental connection.

I'll look into cygwin, but to be honest I have a low degree of confidence that it will help.

Thanks again.

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Wed, 2005-10-05 00:22

Hi, Larry. I've been reworking this framework quite a bit in the past week so things are in a state of flux right now. But we'll sort all of this out. Sorry for the hassle! Also, the "generate-sql.php" script that I recently created is really a tangled mess right now. It was my first attempt, and I improved the model with the generate-dbxml and generate-interfaces scripts, but haven't gone back and fixed generate-sql yet.

The first thing I do is to stop doing builds in modules/core/classes and instead do them in modules/core/classes/GalleryStorage/DatabaseStorage/schema. That'll cut out the step where we build interfaces, which you don't care about now.

I've tweaked generate-sql.php a little bit so that it generates db2 files that are exactly like Oracle. Here's my version:
http://www.menalto.com/.outgoing/gallery/generate-sql.php.txt

I haven't tested this on Windows yet (my windows box is at home) but if you replace your generate-sql.php script with this one and then run make in the schema directory, you should get the appropriate DB2 SQL files, which you can then tweak. If that doesn't work, can you provide me with a link to a text file with the output from your make run?

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2005-10-05 00:53

Hi Bharat.

I copied your 'generate-sql.php' file, and ran 'gmake' in 'modules\core\classes\GalleryStorage\DatabaseStorage\schema' both with and without '--win32'. No difference. I'll attach both outputs here.

Thanks.

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Wed, 2005-10-05 06:04

I'm pretty sure you want the --unix flag. That's generating the correct output, so theoretically it should have worked. I installed cygwin on a new windows box, set up xampp, installed g2, and ran my modified script and it generated the db2 files for me. It had issues with missing make/gmake/xmllint/rxp and some other issues, for which I've improved the scripts a little bit, but none of those things should be affecting you.

Let's break it down further. Go to the core schema directory and do a "make" and verify that it creates an xml-out directory with a bunch of xml files in it. This is stage 1 -- if that doesn't work then there's a problem with generate-dbxml.php. Then if that works go to the platform directory and look to see if you have a db2 directory. If you don't, then try:

  • Verify that you've got my tweaked script in lib/tools/bin/generate-sql.php
  • Delete the contents of the mysql directory and run the script again -- does it recreate the .sql files there?
  • put some debug statements in generate-sql.php and see if/where it's going wrong

something in there should work!

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2005-10-05 17:08

I've now downloaded and installed a brand new shiny "cygwin"... problem persists, but I'll continue working in that environment.

I've verified that the xml files are being created, and I am using your tweaked 'generate-sql.php' script.

I put some debug printf()s in 'generate-sql.php', and it looks to me like it isn't finding the xml files, even though they are there.

Quote:
if (!empty($_SERVER['SERVER_NAME'])) {^M
print "You must run this from the command line\n";^M
exit(1);^M
}^M
^M
require_once(dirname(__FILE__) . '/XmlParser.inc');^M
printf("Larry is here.\n");
^M
foreach (glob('../xml-out/*.xml') as $xmlFile) {^M
printf("Larry never gets here.\n");
$p =& new XmlParser();^M
$root = $p->parse($xmlFile);^M
^M
foreach (array('mysql', 'postgres', 'oracle', 'db2') as $db) {^M
if (!file_exists($db)) {^M
mkdir($db);^M
}^M
$generatorClass = "${db}Generator";^M
$generator = new $generatorClass;^M
$output = $generator->createSql($root[0], 0, 0, null) . "\n";^M
$base = basename($xmlFile);^M
$base = preg_replace('/\.[^\.]*$/', '', $base);^M
^M
$fd = fopen("$db/$base.sql", "w");^M
fwrite($fd, $output);^M
fclose($fd);^M
}^M
printf("Larry never gets here either.\n");
}^M

Output looks like:

Quote:
lmenard@larryhome /cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema
$ gmake 2>&1 | tee gmake.out.cygwin
cd xml-src && /cygdrive/c/UTIL/gmake - --unix xml
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classe
s/GalleryStorage/DatabaseStorage/schema/xml-src'
perl ../../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp ../../../../*.class
find . -name '*.xml' -exec cp {} ../xml-out \;
php -f ../../../../../../../lib/tools/bin/generate-dbxml.php
gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/xml-src'
cd platform && /cygdrive/c/UTIL/gmake - --unix sql
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform'
php -f ../../../../../../../lib/tools/bin/generate-sql.php ../xml-out
Larry is here.
gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform'

Proof that the xml files do exist:

Quote:
lmenard@larryhome /cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema
$ ls xml-out
A_GalleryAlbumItem_1.0.xml GalleryGroup.xml
A_GalleryDerivative_1.0.xml GalleryItem.xml
A_GalleryEntity_1.0.xml GalleryItemAttributesMap.xml
A_GalleryGroup_1.0.xml GalleryMaintenanceMap.xml
A_GalleryItem_1.0.xml GalleryMimeTypeMap.xml
A_GalleryPluginMap_1.0.xml GalleryMovieItem.xml
A_GalleryPluginParameterMap_1.0.xml GalleryPermissionSetMap.xml
A_GalleryPluginParameterMap_1.1.xml GalleryPhotoItem.xml
A_GalleryPluginParameterMap_1.2.xml GalleryPluginMap.xml
ExternalIdMap.xml GalleryPluginParameterMap.xml
GalleryAccessMap.xml GalleryRecoverPasswordMap.xml
GalleryAccessSubscriberMap.xml GalleryToolkitOperationMap.xml
GalleryAlbumItem.xml GalleryToolkitOperationMimeTypeMap.xml
GalleryAnimationItem.xml GalleryToolkitOperationParameterMap.xml
GalleryChildEntity.xml GalleryToolkitPropertyMap.xml
GalleryDataItem.xml GalleryToolkitPropertyMimeTypeMap.xml
GalleryDerivative.xml GalleryUnknownItem.xml
GalleryDerivativeImage.xml GalleryUser.xml
GalleryDerivativePreferencesMap.xml GalleryUserGroupMap.xml
GalleryDescendentCountsMap.xml Lock.xml
GalleryEntity.xml R_GalleryItemPropertiesMap_1.0.xml
GalleryFactoryMap.xml R_GalleryPermissionMap_1.0.xml
GalleryFileSystemEntity.xml Schema.xml

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-10-05 17:23

but you used gmake in 'modules\core\classes\GalleryStorage\DatabaseStorage\schema' right?
maybe add
var_dump(getcwd());
var_dump(glob('../xml-out/*.xml'));

right before the foreach statement for debugging.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2005-10-05 20:51

Oh, now this is interesting. :-)

Yes my current directory when I run 'gmake' is 'gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema'. But I gather the following means that the current directory when 'generate-sql.php' is run is 'gallery2/lib/tools/bin'.

Quote:
lmenard@larryhome /cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema
$ gmake 2>&1 | tee gmake.out.cygwin
cd xml-src && /cygdrive/c/UTIL/gmake - --unix xml
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/xml-src'
perl ../../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp ../../../../*.class
find . -name '*.xml' -exec cp {} ../xml-out \;
php -f ../../../../../../../lib/tools/bin/generate-dbxml.php
gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/xml-src'
cd platform && /cygdrive/c/UTIL/gmake - --unix sql
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classe
s/GalleryStorage/DatabaseStorage/schema/platform'
php -f ../../../../../../../lib/tools/bin/generatesql.php ../xml-out
Larry is here.
string(35) "c:\My Server\gallery2\lib\tools\bin"
array(0) {
}

gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform'

lmenard@larryhome /cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema
$

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2005-10-05 21:07

Yup, confirmed it with a little test script. The current directory in a given script is the directory in which the script exists. I gather this is not the case in php on Linux/UNIX?

Quote:
C:\temp>"c:\program files\php\php" -f test.php
string(7) "C:\temp"

C:\temp>cd ..

C:\>"c:\program files\php\php" -f temp\test.php
string(7) "C:\temp"

Perhaps it would be wise to avoid using relative paths. How about using a variable called something like GALLERY_ROOT to qualify all path references?

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Wed, 2005-10-05 21:37

Wow, that's not good. That sounds like a bug in PHP to me. On my windows box, I see:

Quote:
c:\Documents and Settings\mediratta>"\Program Files\xampp\php\php.exe" -f tmp\test.php
string(35) "C:\Documents and Settings\mediratta"

Php should not change the current working directory to match that of the script -- that's wrong (and will break other parts of our code also). What version of PHP are you using? I'm using 5.0.4 that comes with the latest version of http://www.apachefriends.org/en/xampp.html. You could try installing that and see if that works.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2005-10-05 21:46

Ah-haa, I just found the following:

http://ca3.php.net/getcwd

Quote:
If your PHP cli binary is built as a cgi binary (check with php_sapi_name), the cwd functions differently than you might expect.

say you have a script /usr/local/bin/purge
you are in /home/username

php CLI: getcwd() gives you /home/username
php CGI: getcwd() gives you /usr/local/bin

This can trip you up if you're writing command line scripts with php. You can override the CGI behavior by adding -C to the php call:

#!/usr/local/bin/php -Cq

and then getcwd() behaves as it does in the CLI-compiled version.

There are 2 copies of "php.exe" in the distribution I have:

Quote:
C:\Program Files\PHP>dir php.exe /s
Volume in drive C has no label.
Volume Serial Number is CC7D-89B5

Directory of C:\Program Files\PHP

02/06/2004 04:03 PM 45,056 php.exe
1 File(s) 45,056 bytes

Directory of C:\Program Files\PHP\cli

02/06/2004 04:03 PM 24,576 php.exe
1 File(s) 24,576 bytes

If I prepend '/cygdrive/c/Program Files/php/cli' to my PATH, it now reports the correct current directory, but seems it still can't find the xml files.

Quote:
lmenard@larryhome /cygdrive/c/My Server/gallery2/modules/core/classes/GallerySto
rage/DatabaseStorage/schema
$ echo $PATH
/cygdrive/c/Program Files/php/cli:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/cygdrive/c/Perl/bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/UTIL:/cygdrive/c/Program Files/Common Files/Roxio Shared/DLLShared:/cygdrive/c/PROGRA~1/IBM/SQLLIB/BIN:/cygdrive/c/PROGRA~1/IBM/SQLLIB/FUNCTION:/cygdrive/c/PROGRA~1/IBM/SQLLIB/SAMPLES/REPL:/cygdrive/c/PROGRA~1/IBM/SQLLIB/java/jdk/bin:/cygdrive/c/util:/cygdrive/c/Program:/cygdrive/c/Program Files/php
lmenard@larryhome /cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema
$ gmake 2>&1 | tee gmake.out.cygwin-cli
cd xml-src && /cygdrive/c/UTIL/gmake - --unix xml
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/xml-src'
perl ../../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp ../../../../*.class
find . -name '*.xml' -exec cp {} ../xml-out \;
php -f ../../../../../../../lib/tools/bin/generate-dbxml.php
gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes
/GalleryStorage/DatabaseStorage/schema/xml-src'
cd platform && /cygdrive/c/UTIL/gmake - --unix sql
gmake[1]: Entering directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform'
php -f ../../../../../../../lib/tools/bin/generate-sql.php ../xml-out
Larry is here.
string(89) "c:\My Server\gallery2\modules\core\classes\GalleryStorage\DatabaseStorage\schema\platform"
array(0) {
}

gmake[1]: Leaving directory `/cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform'

lmenard@larryhome /cygdrive/c/My Server/gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2005-10-05 22:09

This is still weirder... even in my test script the ".." reference seems to be going all the way back to the root:

Quote:
<?php
/* */

var_dump(getcwd());

foreach (glob('../*.*') as $xmlFile) {
printf("%s\n", $xmlFile);
}
?>

Quote:
C:\Program Files\PHP>cli\php.exe -f \temp\test.php
string(20) "C:\Program Files\PHP"
../AUTOEXEC.BAT
../CONFIG.SYS
../FileMonk.cfg
../IO.SYS
../MSDOS.SYS
../NTDETECT.COM
../boot.ini
../pagefile.sys
../search.log
../setuplog.txt

C:\Program Files\PHP>

And I've confirmed that this is also what's happening in 'generate-sql.php'... the '..' is somehow being resolved to the root directory. If I copy the 'xml-out' directory to immediately beneath the root, it works and the db2 files are created in the right place.

So why is ".." being misunderstood?

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2005-10-05 22:11

FYI, my php version is:

Quote:
$ php -v
PHP 4.3.7 (cli) (built: Jun 2 2004 15:49:44)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies

I'll try upgrading and we'll see what happens.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Wed, 2005-10-05 22:26

OK, I upgraded my PHP to:

Quote:
$ php -v
PHP 5.0.5 (cli) (built: Sep 5 2005 15:54:44)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v2.0.5, Copyright (c) 1998-2004 Zend Technologies

and if I'm not mistaken, all seems to be working now. Sigh.

Onwards and upwards. Thanks for the guidance.

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Wed, 2005-10-05 23:55

What a pain! So even the 4.3.7 CLI version of PHP doens't work? I'll investigate that in more detail and see if I can figure out what's going on there. In the meantime I'm glad to see that you've got it working and can move forward! I'll be very happy to see a DB2 version of the SQL!

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Thu, 2005-10-06 00:03

OK, I've re-implemented everything described above. Now I get file integrity warnings:

Quote:
+ Missing files (2)
modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform/mysql/InstallerTest_sql
modules/core/classes/GalleryStorage/DatabaseStorage/schema/xml-out/InstallerTest.xml

+ Modified files (11)
install/config.php-template
install/steps/DatabaseSetupStep.class
lib/tools/bin/generate-dbxml.php
lib/tools/bin/generate-interfaces.php
lib/tools/bin/generate-sql.php
modules/core/classes/GalleryStorage.class
modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform/GNUmakefile
modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform/postgres/A_GalleryGroup_1.0.sql
modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform/postgres/A_GalleryPluginMap_1.0.sql
modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform/postgres/A_GalleryPluginParameterMap_1.0.sql
modules/core/classes/GalleryStorage/DatabaseStorage/schema/platform/postgres/A_GalleryPluginParameterMap_1.2.sql

I proceed anyway, and the database setup eventually fails with:

Quote:
The database privileges test did not complete successfully.
Unknown DB type or no known tables information.

So I guess we have to resolve that missing file problem. Any ideas why they're missing?

Thanks.

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Thu, 2005-10-06 00:22

Hm .. the missing InstallerTest_sql files indicates that you're using a build from last week (or older). The latest stuff no longer builds that file, which is why when you deleted the mysql dir it didn't come back. Really the best thing to do is to get the latest code from CVS ( see http://gallery.menalto.com/wiki/DownloadingGalleryUsingCVS ) and then use my tweaked script and see if that gets you by the problem. Then when we make fixes to help you along, you can get them quickly by just updating your code.

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Thu, 2005-10-06 00:34

Larry, heads up. I've begun working on a change that will combine all .sql files into one large file. This will cut us down from 130+ .sql files in the core module to 1 file, reducing our disk footprint considerably. This will involve changes to a lot of the stuff that you're working on now so if you cvs up and you see something weird happening along those lines, check back here and I'll help you get your stuff in the right shape.

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Thu, 2005-10-06 00:39

Great... something else I have to learn, install, configure, and debug. Sigh...

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Thu, 2005-10-06 01:19

:-( Don't worry, it won't be very traumatic. The first thing I'm doing is getting rid of all the excess directory structure and makefiles in the schema directories. This will involve a little restructuring and I hope to commit it tonight. If the current scheme works for you (which it should, now that your PHP cli issues are resolved) then the new code will too.

If you want to avoid dealing with any of this hassle, that's fine too. Grab yesterday's nightly from here:
http://galleryupdates.jpmullan.com/G2/gallery-2.0+.tar.gz

And use that. Or cvs update to the current code as of now and don't cvs up again until you have it working. When you get something that approximates what db2 does, I can help you merge it into the current code so that you don't have to deal with the infrastructural changes...

 
Larry Menard
Larry Menard's picture

Joined: 2005-10-01
Posts: 757
Posted: Thu, 2005-10-06 01:38

Well I've already taken a bite of the CVS apple, so I may as well finish it.

I found a web page that describes how to install & use CVS on Windows
(http://www.ifm-services.com/people/jamesk/papers/cms/cvs-win32-client.html).
I've now installed CVSNT, WINCVS, and Python, but then the web page goes into detail
that I suspect may not be necessary.

Any chance anyone can provide a bare-minimum 'CVS For Windows For Dummies'?