G3: Migrating from G2 and G1

Hi,

I have been running Gallery for a long time. First G1, then G2 and now G3.

G2 supported migrating from G1, and G3 support migratiog from G3. But does
G3 support migrating from a G2, that was supporting migrating from G1?

I have been running G2 and G3 in parallel, G2 because I want all old picture
links to work, but I'd like to go fully into G3 and retire old G2 install.
But this requires that also the G1-era links continue to work perfectly.

Any help will be greatly appreciated, I have 39GB/32000pcs pictures, and
keeping all thsi duplicated is not something I want to continue forever.

-Paavo

1) - 5) done, here is my G2 .htaccess:

and I get a lot of this:

Should I remove (comment out) the original G2 part of the .htaccess?

-Paavo

Hi,

I have been running G1 and G2 for a long time in HOST_OLD, which
is disappearing soon. I installed G3 in it, but I soon copied G3
to a new host, HOST_NEW.

I want to migrate G1/G2/G3 from HOST_OLD to G3 in HOST_NEW.

Should this do the trick? Part of HOST_OLD apache configuration:

Directories albums, gallery and gallery3 in HOST_OLD have been
renamed to something else, if all works fine I'll nuke those later.

Comments please, I'm not an expert with apache redirects, just got
rewrite to work within one host...

-Paavo

I don't understand the last two posts.
Are you changing domain names, or just moving to a new host?

If changing to a new host, you shouldn't need any of that.

If changing domain names, you have to still have access to the old domain name, and just one redirect (in the root folder of the old domain) would do it:
Redirect 301 /gallery3 http://new-domain-name/gallery3

Done changes in the above post.

These still don't work (just examples, these are not the only ones):

- http://HOST_OLD/gallery/d/98574-1/20110526_006.jpg -> http://HOST_NEW/gallery3/index.php/g2/map?path=d/98574-1/20110526_006.jpg results an error page.

- http://HOST_OLD/albums/album63/IMG_0913.sized.jpg -> http://HOST_NEW/gallery3/index.php/g2/map?path=album63/IMG_0913.sized.jpg results an error page.

So now I'm good with redirecting to the new host/domain, and this is again a generic G1/G2 to G3 migrating topic.

Any help will be GREATLY appreciated, I have a lot of pics in my gallery, and I'd very much like also the old ones to display properly also with G3.

Should I somehow have also the old G2 .htaccess onboard?

-Paavo

Copy of my old G2 .htaccess

I have been lurking and watching for this since December, as I have the same problem G2, with G1 migrated to it, then imported to G3. From my observations, and I'm no expert, but:

1. G1, G2, and G3 appear to be totally different programs

2. G1-->G2, G2-->G3, and G1-->G3 all make Rewrite commands in the .htaccess and create and use tables in the database to link the 'old' gallery locations to the 'new' gallery location.

3. The G1-->G3 module may do this, if the G1 files are still available, but that would (?) re-import the G1 albums into G3 duplicating the pics already imported to G2 from G1 and then to G3.

4. IMHO, the 'g2_G1MigrateMap' table in G2, has to be read in conjunction with the 'g2_maps' table in G3 to create a new table (g1_maps like the G1-->G3 module?) in G3 to remap the G1 addresses to G3. Does that make any sense? Could the G1-->G3 module be modified to read the database tables to do this? Will the G1-->G2 migration module be needed (rewritten?) to make this work?

My head hurts.

Hi,

The host is now insignificant, I have the same G1/G2 -> G3 migration problems in HOST_OLD and HOST_NEW.

G1 -> G2 migration worked OK, but G1/G2 -> G3 migration is not working properly.

This is the reason why I have been running G2 and G3 in parallel in the HOST_OLD (duplicating all pics), but I don't have enough disc space in HOST_NEW to do so, and have to be able to use G3 only.

G1/G2 to G3 migration is the problem.

I can provide non-generalised details if needed, but I think these two examples should provide a good starting point.

Just forget about the host, it does not matter.

-Paavo

"Albums" is the directory gallery1 stored the pictures in. Using gallery2's "g1-Migration module" the "Albums" folder must remain in place with only an ".htaccess" to redirect gallery1 url's to the gallery2 folder. Then gallery2's g1-migration module can do it's magic and map the url's to the current pic location in gallery2.

When this setup is further imported to gallery3, gallery1 url's fail. There is no procedure to remap the url's of those gallery1 pics that were migrated to gallery2 and then further imported to gallery3. Some examples at my test site.(Gallery 3 not active at this time, but I saved the url's)

Gallery1 url: http://64.131.78.133/albums/album356/Picture_094.jpg

Gallery2 url: http://64.131.78.133/gallery/d/44136-2/Picture_094.jpg

gallery3 url: http://64.131.78.133/gallery3/var/resizes/wingmans/album356/Picture_094.jpg?m=1293195269

gallery 1 to gallery 2 redirected url: http://64.131.78.133/gallery/main.php?g2_view=core.DownloadItem&g2_itemId=44136

gallery 2 to gallery 3 redirected url: http://64.131.78.133/gallery3/var/albums/wingmans/album356/Picture_094.jpg?m=1293195268

gallery 1 to gallery 3 url: http://64.131.78.133/gallery3/g2/map?path=main.php&g2_view=core.DownloadItem&g2_itemId=44136 (don't work)

Hi,

If this is really true, this is a major problem for me, and also for everybody else who started with G1 and continued with G2 and are considering G3.

Can this be solved with mod_rewrite, or is something like database-remapping-table needed?

Not sure if my skills are good enough to start any implementation, but I certainly want to help getting this done.

WHAT IS NEEDED, AND HOW CAN I HELP? (sorry for yelling, but this really important for me)

I still have all G2 files/directories in good backups, and G2 database is also available. And lots of test material, just google for "stiletti.wipsl.com/albums/" or "stiletti.wipsl.com/gallery/" and you get some hundreds of hits. Which I'd very very much like to work properly with G3 (in ptl.kapsi.fi host).

-Paavo

My problem is Gallery is used heavily within the forum on the same site. I just did a search of the posts table in the forum database and there are 6961 G1 url's posted, and there are over 2500 G2 url's posted.

I've been active in the maintenance of the website for 2 and a half years, and as I understand, Gallery1 was operational on the site for five years before we upgraded to Gallery2 in Sept 2009.

I'm not a programmer, but it seems to me the line "RewriteRule (.*)$ /gallery/main.php?g2_controller=migrate.Redirect&g2_path=$1 [QSA]" is calling something in G2 to access the G2 database table "g2_G1MigrateMap" and find out what the new G2 url equivalent is. When G2 is deleted and G3 up and running, G1 url's CAN'T work due to the G2 software and database being gone. How can a redirect be done if it don't know which picture to redirect to?[img]http://www.ngwclub.com/gallery/d/139739-1/g1_g2_g3.png[/img]

Can this be done in .htaccess with with a Rewrite? Or am I correct in assuming a database table and directions to look up the data are required?

Nice, completely unrelated thread jack....
http://codex.gallery2.org/Gallery3:Modules:serveradd

If that doesn't get you going, don't reply here, start a new thread please.
____________________________________________
Like Gallery? Like the support? Donate now!

If you need more help, just start a new thread and we'll be glad to help you.
____________________________________________
Like Gallery? Like the support? Donate now!

OK,

case study, lets pick this link:

It has been used e.g. here http://forums.offipalsta.com/showthread.php?t=1634 (1st link to a picture, in 4st post).

Again, stiletti.wipsl.com is the old host, the address will disappear soon. This host was originally running G1, and was upgraded to G2. G3 was installed, and everything from G2 were imported. G3 was then copied to new host, ptl.kapsi.fi, which is running just G3.

OK, lets see... using stiletti.wipsl.com, which still has G2 and G3 installed.

Using "hand re-writing" (and new host) http://stiletti.wipsl.com/albums/album63/IMG_0913.sized.jpg becomes
http://ptl.kapsi.fi/gallery3/index.php/g2/map?path=d/65312-2/IMG_0913.jpg which displays the picture OK.

So G3 seems to have all the information how to display old G1 links!!!

I just don't know how this can be done using apache rewrites, IMHO database access is needed.

But since the information is available, could G3 index.php be enhanced so that every old G1/G2 link would display correctly (with the help of some rewrites)?

-Paavo

@roncar: I'm looking back at your post above ( http://gallery.menalto.com/node/103597#comment-381211 ). This should be too difficult to solve. The only issue is the "wingman" part; is that the same for all photos? If not, how many different directories are there?

@roncar: Yeah, I'm skipping the database stuff. Keeping it simple. My thought is to just do manual htaccess rewrites/redirects for the 6 directories (Gallery1 to Gallery3) and to do a migration to cover the Gallery2 to Gallery3 rewrites/redirects.

You'll need 2 rewrites to cover the Gallery1 to Gallery3 stuff (maybe a couple more if you've done something complicated--like rename a bunch of files--that hasn't been mentioned yet).
The migration tool should get the Gallery2 to Gallery3 stuff squared away for you.

The one concern is that your Gallery3 urls have a folder that wasn't in the Gallery1 urls.

Gallery1 url: http://64.131.78.133/albums/album356/Picture_094.jpg
Gallery3 url: http://64.131.78.133/gallery3/var/resizes/wingmans/album356/Picture_094.jpg?m=1293195269

If all of the Gallery1 albums ended up in the same Gallery3 folder, that'll be easy.
Else, if there weren't many Gallery1 albums and you know where they all are now, that'll take a few more rewrites, but is totally doable.
Otherwise, if there were a lot of Gallery1 albums are now all over the place in Gallery3, that's doable but likely to be aggravating.

Let me know and I'll give you some code to try.

@tempg: From what I remember, when using gallery1 (I don't have an operational copy running) moving around inside the gallery there were folders, sub-folders, sub-sub-folders, etc. However when physically stored on the drive, ALL the folders are in the root folder 'albums.' To figure which are which would be a monumental task.

ie gallery1 /albums/album277/70fl78.jpg
in gallery 3 becomes /gallery3/var/resizes/wingmans/roncarGL/misc-pics/album277/70fl78.jpg

Looking at my backup of gallery1, there are 877 folders and 46,189 files in the albums folder.

@tempg:

This means what? ssh access? just let me know, stiletti is all mine, and I can arrange anything you need.

I don't have G1 any more, all of it was thrown away when G2 was taken into use.

@floridave:

Me thinks the same.

-Paavo

Hi,

any progesss on this?

Another example what's broken:

This http://www.offroadoulu.com/smf_forum/index.php?topic=53.msg14005#msg14005 post has two links

http://stiletti.wipsl.com/gallery/d/99687-1/20110805_012.jpg (1st link)
http://ptl.kapsi.fi/gallery3/index.php/g2/map?path=d/99687-1/20110805_012.jpg (redirected to G3 in new host, broken)
http://ptl.kapsi.fi/gallery3/index.php/g2/map?path=d/99687-2/20110805_012.jpg (replace -1 with -2 and it works fine)
http://ptl.kapsi.fi/gallery3/var/resizes/range-misc/summer_2011/20110805_012.jpg?m=1312826526 (picture shows OK)

http://stiletti.wipsl.com/gallery/d/99690-1/20110805_013.jpg (2nd link)
http://ptl.kapsi.fi/gallery3/index.php/g2/map?path=d/99690-1/20110805_013.jpg (redirected to G3 in new host, broken)
http://ptl.kapsi.fi/gallery3/index.php/g2/map?path=d/99690-2/20110805_013.jpg (replace -1 with -2 and it works fine)
http://ptl.kapsi.fi/gallery3/var/resizes/range-misc/summer_2011/20110805_013.jpg?m=1312826527 (picture shows OK)

Here host in insignificant, functionality would be thw same in one host only.

G2 must have had a logic for this, but based on my data it looks like a "yes".

-Paavo

@ptl: I don't fully get some of what you're saying, but try using this in Gallery's htaccess (make sure you put it in the right place):

<IfModule mod_rewrite.c>
Options +FollowSymLinks
RewriteEngine On
RewriteBase /gallery3/
RewriteRule (.*)=d/([0-9]+)-1/(.*).jpg$ $1=d/$2-2/$3.jpg [L,R=301]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
...

Some server configs require the full path in the rewrite, so you may have to change the line to
RewriteRule (.*)=d/([0-9]+)-1/(.*).jpg$ http://ptl.kapsi.fi/gallery3/$1=d/$2-2/$3.jpg [L,R=301]

One each of what?

Just to make it a tangible fix proposal, could index.php/g2/map be modified like this:

for ( $i = 0; $i < 10; $i++ ) // is 0-9 exactly the right range? 1-9? 1-something? 0-something?
{
    $fixed_url=check_url("gallery3/index.php/g2/map?path=d/98574-".$i."/20110526_006.jpg") and break;
}

check_url(STRING) returns true if STRING is OK, false if it's not.

Of course, "gallery3/index.php/g2/map?path=d/98574-" and "/20110526_006.jpg" would need to be exploded to variables or something, but I hope this gives and idea. Maybe the first part should be "d/98574-" only, because then it's very easy to check what is the value for g2_url for that particular g2_id from g3tab_g2_maps table.

Hmm...

$fixed_url="gallery3/index.php/g2/map?path=".get_g2_url_from_g3tab_g2_maps_table($g2_id);

Maybe it's better that I stop "coding" and let somebody who knows G3 (and PHP!) implement this.

-Paavo

OK,

I'll try that. Just feels a bit silly to write a lot of stuff to .htaccess instead of one simple database query, but if I'm alone with this problem then .htaccess is easier for me that hack&branch core G3 php code (also because kohana is as clear a black magic to me).

RewriteCond $0 !-f

Will this work? Wrong resize-url will non return 404, but the G3 oops-page, so will apache know -2 did not display properly?

I was not able to use RewriteLog and RewriteLogLevel in the "production" server, so I need to set up a test system. Firsts tests with the real G3 server failed, since

RewriteRule (.*)=d/([0-9]+)-1/(.*).jpg$ $1=d/$2-2/$3.jpg [L,R=301]

did not rewrite http://ptl.kapsi.fi/gallery3/index.php/g2/map?path=d/98574-1/20110526_006.jpg. I also tried

  RewriteRule ^(.*)=d/([0-9]+)-1/(.*)\.jpg$ $1=d/$2-2/$3.jpg [L,R=301]

  RewriteCond %{QUERY_STRING} ^(.*)=d/([0-9]+)-1/(.*)\.jpg$
  RewriteRule $1=d/$2-2/$3.jpg

  RewriteCond %{REQUEST_URI} ^(.*)=d/([0-9]+)-1/(.*).jpg$
  RewriteRule $1=d/$2-2/$3.jpg [L,R=301]

  RewriteRule (.*)=d/([0-9]+)-1/(.*).jpg$ $1=d/$2-2/$3.jpg

  RewriteRule (.*)-1/(.*) $1-2/$2

And several other variants, but nothing (except RewriteRule ^ http://example.com/) worked... Debugging rewrite-rules is not one of my stongerst areas.

-Paavo

Hi,

I could not get RewriteRule (.*)=d/([0-9]+)-1/(.*).jpg$ $1=d/$2-2/$3.jpg [L,R=301] to match http://ptl.kapsi.fi/gallery3/index.php/g2/map?path=d/98574-1/20110526_006.jpg. Frustrated, also because I felt stupid, rewrite rules debugging is not my game. Also because rewrite rules are the "wrong" medicine (IMHO).

Took another path, and studied G3 php a bit. Found modules/g2_import/controllers/g2.php.

After

} else if ($path) {

I have something like this

$tmp1=explode("/", $path);
// should check 1st that this request is in the form of
// path=d/98574-1/20110526_006.jpg
// $tmp1[0] must be "d"
$tmp2=explode("-", $tmp1[1]);
$tmp_id=$tmp2[0];

And then we need something like this:

$tmp3=$database->query("select g2_url from g3tab_g2_maps where g2_id=".$tmp_id);
$tmp4=$database->last_query();

and then need to replace this

$where = array(array("g2_url", "IN", array($path, str_replace(" ", "+", $path))));

with something like this:

$where = array(array("g2_url", "IN", array($path, str_replace(" ", "+", $path), $tmp4)));

I.e. add 3rd element to the list ($tmp4).

And yes, this is very quick and dirty, the variable names and all that is horrible, please ignore the code quality, this just demos the principle.

Need a bit help (had not time to find all answers bu myself, and now I need to go for the weekend) with the database query, and conformation that "where g2_url IN <LIST>" LIST can have more than two candidates.

-Paavo

Hi,

just came back to modern world. I'm still going the php path, just did not have time to implement it yet.

But since this thread has so heavily been about rewrites, here are the answers if somebody else could benefit from rewrite approach.

Lets make this real simple, and use just one URL.

(1) http://ptl.kapsi.fi/gallery/d/98574-1/20110526_006.jpg

(2) http://ptl.kapsi.fi/gallery3/index.php/g2/map?path=d/98574-2/20110526_006.jpg

(3) http://ptl.kapsi.fi/gallery3/index.php/g2/map?path=d/98574-1/20110526_006.jpg

gallery/.htaccess is standard:

gallery $ cat .htaccess
<IfModule mod_rewrite.c>
  # Options +FollowSymLinks
  RewriteEngine On
  RewriteBase /gallery/
  RewriteRule ^(.*)$ /gallery3/index.php/g2/map?path=$1 [QSA,L,R=301]
</IfModule>

gallery3/.htaccess is modified:

gallery3 $ diff .htaccess .htaccess.orig
18,26d17
< # see http://gallery.menalto.com/node/103597
< #
< <IfModule mod_rewrite.c>
<   # Options +FollowSymLinks
<   RewriteEngine On
<   RewriteBase /gallery3/
<   RewriteRule (.*)=d/([0-9]+)-1/(.*).jpg$ $1=d/$2-2/$3.jpg [L,R=301]
< </IfModule>
<

My service provider does not allow FollowSymLinks, but FollowSymLinksIfOwnerMatch is on by default. Every single file and directory in the www directory (where gallery and gallery3 live) is owned by my userid.

Directory gallery is totally empty except the .htaccess file.

I also tried http://martinmelin.se/rewrite-rule-tester/ but could not get the rewrite rule to match. I used URL "gallery3/index.php/g2/map?path=d/98574-1/20110526_006.jpg" and rewrite rule "RewriteRule (.*)=d/([0-9]+)-1/(.*).jpg$ $1=d/$2-2/$3.jpg [L,R=301]".

Are there other/better rewrite-rule testing sites?

-Paavo

Initial testing indicates this might work, now need to start looking for more test cases.

$ diff g2.php g2.php.orig
51,80d50
<         $tmp1=explode("/", $path);
<         if($tmp1[0] == "d") {
<           $tmp2=explode("-", $tmp1[1]);
<           if(ctype_digit($tmp2[0])) {
<             $tmp3=DB::select()->from('g2_maps')->where('g2_id', '=', $tmp2[0]);
<             $tmp4=$tmp3->execute(); $tmp5=$tmp4->as_array();
<             if((count($tmp5) == 1) && (isset($tmp5[0]['g2_url'])))
<               $where[0][2][]=$tmp5[0]['g2_url'];
<           }
<         } else {
<           if($path == "main.php") $tmp6=$input->get("g2_path"); else $tmp6=$path;
<           $tmp7=explode("/", $tmp6);
<           if(count($tmp7) == 2) {
<             $tmp8=explode(".", $tmp7[1]);
<             $tmp9=DB::select()->from('g1_maps')->where('g2col_g1album', '=', $tmp7[0])
<             ->and_where('g2col_g1item', '=', $tmp8[0]);
<             $tmpa=$tmp9->execute(); $tmpb=$tmpa->as_array();
<             if((count($tmpb) == 1) && (isset($tmpb[0]['g2col_itemId']))
<               && (ctype_digit($tmpb[0]['g2col_itemId']))) {
<               $tmpc=$tmpb[0]['g2col_itemId'];
<               $tmpd=DB::select()->from('g2_maps')->where('g2_id', '=', $tmpc)
<                 ->and_where('resource_type', '=', 'file');
<               $tmpe=$tmpd->execute(); $tmpf=$tmpe->as_array();
<               if((count($tmpf) == 1) && (isset($tmpf[0]['g2_url'])))
<                 $tmpg=$tmpf[0]['g2_url'];
<                 $where[0][2][0]=$tmpg;
<                 $where[0][2][1]=str_replace(" ", "+", $tmpg);
<             }
<           }
<         }

This is still 100% trial-and-error method based, I don't have a glue how G3/kohana internals work (e.g. $path/$input->get("g2_path") for $tmp6), so I might have broken something.

One needs to export g2tab_G1MigrateMap table from G2 database, I imported it to the same database with the rest of G3 tables, and renamed it to 'g3tab_g1_maps' as can be seen from the code (G3/kohana auto-inserts the table prefix, g3tab_ in my case).

But even if this is pre-beta, I thought I'll post this, if everybody in the G3 developer community is not upset by this super-long thread, somebody might find some problems in this code, or even make some use of it.

I still think G3 support for G1 and G2 could be better, hopefully this is a small bit to improve it.

-Paavo

PS. I copied /gallery/.htaccess to /albums/.htaccess, just changed the RewriteBase, I was not sure if rewrites after rewrites work or are efficient. Original /albums/.htaccess rule was "RewriteRule (.*)$ /gallery/main.php?g2_controller=migrate.Redirect&g2_path=$1 [QSA]". Also /albums is completely empty, except the .htaccess file.

Done. Think this case might be impossible with rewrite, at least with my rewrite "skills".

$ diff g2.php g2.php.orig
51,85d50
<         $tmp1=explode("/", $path);
<         if($tmp1[0] == "d") {
<           $tmp2=explode("-", $tmp1[1]);
<           if(ctype_digit($tmp2[0])) {
<             $tmp3=DB::select()->from('g2_maps')->where('g2_id', '=', $tmp2[0]);
<             $tmp4=$tmp3->execute(); $tmp5=$tmp4->as_array();
<             if((count($tmp5) == 1) && (isset($tmp5[0]['g2_url']))) {
<               $where[0][2][0]=$tmp5[0]['g2_url'];
<               $where[0][2][1]=str_replace(" ", "+", $tmp5[0]['g2_url']);
<             }
<           }
<         } else {
<           if($path == "main.php") $tmp6=$input->get("g2_path"); else $tmp6=$path;
<           $tmp7=explode("/", $tmp6);
<           if(count($tmp7) == 1) {
<             $tmp9=DB::select()->from('g1_maps')->where('g2col_g1album', '=', $tmp7[0])
<               ->and_where('g2col_g1item', 'is', NULL);
<           } elseif(count($tmp7) == 2) {
<             $tmp8=explode(".", $tmp7[1]);
<             $tmp9=DB::select()->from('g1_maps')->where('g2col_g1album', '=', $tmp7[0])
<             ->and_where('g2col_g1item', '=', $tmp8[0]);
<           }
<           $tmpa=$tmp9->execute(); $tmpb=$tmpa->as_array();
<           if((count($tmpb) == 1) && (isset($tmpb[0]['g2col_itemId']))
<             && (ctype_digit($tmpb[0]['g2col_itemId']))) {
<             $tmpc=DB::select()->from('g2_maps')->where('g2_id', '=', $tmpb[0]['g2col_itemId']);
<             if(count($tmp7) == 1) $tmpc->and_where('resource_type', '=', 'album');
<             elseif(count($tmp7) == 2) $tmpc->and_where('resource_type', '=', 'file');
<             $tmpd=$tmpc->execute(); $tmpe=$tmpd->as_array();
<             if((count($tmpe) == 1) && (isset($tmpe[0]['g2_url']))) {
<               $where[0][2][0]=$tmpe[0]['g2_url'];
<               $where[0][2][1]=str_replace(" ", "+", $tmpe[0]['g2_url']);
<             }
<           }
<         }

I still don't like how the code looks like, but it works with all the test cases I've found so far.

-Paavo