1.2.1 - upload corrupts .jpg

Dynomite2910
Dynomite2910's picture

Joined: 2004-02-09
Posts: 7
Posted: Mon, 2004-02-09 17:44

I am running Gallery Remote 1.2.1 on a Windows XP machine and everytime I try to upload new photos the images are corrupted.

You can see an example of what I mean here...
http://www.gnomesontour.com/gallery/view_album.php?set_albumName=test

When I upload the same images from my laptop which has Gallery Remote 1.1 they seem to work fine.

Also, if I upload directly from the gallery pages it seem to work fine also.

Anyone else seen this behavior ?

Thx,
David

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Mon, 2004-02-09 19:33

Yes, I'd heard about that once before, but nobody was every able to reproduce/fix it.

Best guess is to uninstall GR, remove all traces of ImageMagick in the registry, then reinstall.

Also: which version of Java are you using, and are you using "resize before upload"?

 
Dynomite2910
Dynomite2910's picture

Joined: 2004-02-09
Posts: 7
Posted: Mon, 2004-02-09 19:49

Ok, thanks - I will try to do a reinstall and clean out all references to ImageMagick.

I am using j2re1.4.2_03 (I just upgraded to the latest before doing a reinstall in case it was the jvm).

I am not using "resize before upload" because I find this in my log:

62|TRACE|UsrProps |resizeBeforeUpload= |false|

And the upload appears to occur quickly and then it says that the server is processing the image.

But I do find the following call in my log also but I assume that ImageMagick is also being used to do resizes for the preview and upload windows. (The images looked fine in the preview and upload windows btw)

22859|TRACE|ImageUtils|Executing [imagemagick/win32/convert.exe, -size, 64x64, -filter, Box, C:\Documents and Settings\All Users\Documents\My Pictures\Canon S400\2004_02_07\121_2182.JPG, -resize, 64x64, +profile, *, C:\DOCUME~1\DAVIDA~1\LOCALS~1\Temp\thumbs\thumb-1281754974.gif]
23797|TRACE|ImageUtils|Returned with value 0
23797|TRACE|ImageUtils|Time: 938 - Avg: 938

This would seem to indicate to me that it is ImageMagick on my server that is causing the error but only when it is sent an image from GR 1.2.1 ....strange.

Thanks,
David

 
Dynomite2910
Dynomite2910's picture

Joined: 2004-02-09
Posts: 7
Posted: Mon, 2004-02-09 20:02

After looking at this a little bit more...I don't think this has anything to do with ImageMagick because the original version of the photo on my server is also corrupted.

What seems to be happening is that GR 1.2.1 is uploading a corrupted version of the image for some reason (or it is getting corrupted during the upload) and then ImageMagick is just making resized versions of the same photo.

So problem does indeed appear to be with GR 1.2.1 - it doesn't seem to see any problems with what it is doing though.

David

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Mon, 2004-02-09 20:07

OK, thanks for all the info.

I have no idea why GR would corrupt the image, but we can test a bit more to narrow down the problem.

It seems, as you pointed out, that ImageMagick on the client-side is indeed not to blame, since it is not used by GR for the upload (it is used, however, for the thumbnails).

Is GR running on the same machine Gallery is also running on?

Can you enable "resize before upload" and see what happens with that? What are you using for the web server (Apache/IIS)?

 
Dynomite2910
Dynomite2910's picture

Joined: 2004-02-09
Posts: 7
Posted: Mon, 2004-02-09 20:30

No, GR 1.2.1 is not running on the same machine as the server in this case.

The web server is Apache running on W2K - I just reinstalled PHP Home 2.3.2 recently thinking it may have been the problem.

I tried uploading an image after turning on "resize before upload" as you suggested and it seems to work fine.

You can see it worked here...
http://www.gnomesontour.com/gallery/view_photo.php?set_albumName=test&id=res57332

But, of course, in this case the full size image never gets into my gallery - which is what I want so people can get to and download the full size image if they want to.

Here's another interesting thing - I was going to attach 2 images (the original and the corrupt) but the files from my first test were too big so I tried uploading a small photo and was going to attach the original and the corrupt but there was no corrupt - perhaps size factors into the equation here ? (i.e. any jpg over a certain size transfer incorrectly ? - I will test that theory).

 
Dynomite2910
Dynomite2910's picture

Joined: 2004-02-09
Posts: 7
Posted: Mon, 2004-02-09 20:41

Ok, so I think it definitely has something to do with the size of the .jpg being sent to the server.

Here's what I did...

I turned "resize before upload" on but then instead of saying to "album default", I said to resize it to 1024 x 768 and then image is corrupted again.

So it looks like no matter whether you use "resize before upload" or not, it is the size of the image that is being sent to the server that matters. As soon as you get over some threshold (no idea what) you will see some corruption in the image.

I can send you an original image and the corrupt version of the same if that would help. Just give me and ftp or email address to send them.

Thanks,
David

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Mon, 2004-02-09 20:46

Is your Gallery set to "optimize" image size by trying different compression levels to keep the image size smaller than a certain size?

I'll PM you my email address.

 
Dynomite2910
Dynomite2910's picture

Joined: 2004-02-09
Posts: 7
Posted: Mon, 2004-02-09 20:50

I don't think my gallery is set to "optimze" unless that is the default - I never set anything like that.

Also, the original file and the corrupt version are the exact same number of bytes.

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Mon, 2004-02-09 20:55

Wow, very odd... Do you have a binary diff software that would allow you to see the difference? Maybe a packet was lost, etc.

 
Dynomite2910
Dynomite2910's picture

Joined: 2004-02-09
Posts: 7
Posted: Mon, 2004-02-09 20:59

I opened the 2 files in UltraEdit and did a file comparision, it does find a few differences but they mean nothing to me.

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Mon, 2004-02-09 21:55

Still stumped: I was able to upload the picture to my test server without problems, both resized and non-resized. I guess the next step it to figure out if I can upload to your server. Can you email me the login info?

I really appreciate your help trying to resolve this.

Also, where was this pciture taken? I lived in Ottawa for 2 years and I loved the Bal des Neiges and this looks similar.

 
joel558
joel558's picture

Joined: 2003-04-10
Posts: 93
Posted: Tue, 2004-02-10 01:11

if you have access to them, perhaps check the apache and php error logs. Might be some information in there, although I'm not sure.

Also maybe the apache access log, although less likely there to be something in there.

 
angelo94

Joined: 2003-05-29
Posts: 7
Posted: Mon, 2004-02-16 05:42
paour wrote:
Still stumped: I was able to upload the picture to my test server without problems, both resized and non-resized. I guess the next step it to figure out if I can upload to your server. Can you email me the login info?

I really appreciate your help trying to resolve this.

Also, where was this pciture taken? I lived in Ottawa for 2 years and I loved the Bal des Neiges and this looks similar.

was this issue ever resolved? i'm having the same problem.

thanks,
angelo

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Tue, 2004-02-17 21:27

angelo94, no, I'm still puzzling over this. Can you try joel558's suggestion?

Btw, I also lived in Silver Spring years ago :-)

 
martin_halford

Joined: 2004-03-22
Posts: 2
Posted: Mon, 2004-03-22 00:15

Here's my recent post to the Ensim Forum. I'm convinced this is purely a PHP problem, not Gallery (or Squirrelmail in my case also). Uploaded files are larger than they should be.

I'm on Ensim Pro 3.5.21 with Apache 1.3.27 and PHP 4.3.3 using Gallery and Squirrelmail, both having problems uploading images.

Here's what happens in Gallery 1.3.4 in debug mode:

Processing status...
- Adding Sunset.jpg

Executing:

/usr/X11R6/bin/convert -quality 95 -size 150x150 /home/virtual/site4/fst/var/www/html/albums/album18/Sunset.jpg -geometry 150x150 +profile '\*' /home/virtual/site4/fst/var/www/html/albums/album18/Sunset.thumb.jpg

Results:

none

Error messages:

convert: Corrupt JPEG data: 2 extraneous bytes before marker 0xd4 (/home/virtual/site4/fst/var/www/html/albums/album18/Sunset.jpg) [No such file or directory].
convert: Corrupt JPEG data: 2 extraneous bytes before marker 0xd4 (/home/virtual/site4/fst/var/www/html/albums/album18/Sunset.jpg) [No such file or directory].

Status: 1 (expected 0)IN UTIL ITEMCAPTUREDATE = 2004

- Resizing Sunset.jpg

Any ideas what is causing this problem. I've tried googling a couple of times without success. I think something must have changed with the php and apache configuration in the 3.5.21 upgrade.

This thread on the Gallery forum gives some clues, but in Ensim I'm never sure which configuration files need updating?

http://gallery.menalto.com/modules....ic&topic=4612&8

Any tips would be appreciated.

Martin

 
martin_halford

Joined: 2004-03-22
Posts: 2
Posted: Sun, 2004-04-04 13:36

Ignore my last post unless you are having the same problem It turns out to have been a proxy server problem on my Broadband connection. Even more embarrassing, I found the answer on eBay!!! See the following link...

http://forums.ebay.co.uk/thread.jsp?forum=1004&thread=300018603&start=35&msRange=

Martin

 
mtt1853

Joined: 2004-05-27
Posts: 5
Posted: Thu, 2004-05-27 22:15

I have the same problems as the original poster, I am running Apache 2.0.48 & PHP 4.3.4 with Gallery 1.4.3. I've checked all error logs and nothing shows up. I did the same comparison in Ultraedit and it appears there IS a pattern... but there is some randomness to it too. It looks like every couple of Kilobytes there are a few bytes that are incorrect... The interesting part I found is that they are a sequence of repeated bytes from somewhere earlier in the file. These erroneous bytes replace the bytes that should be there, so that in the end the file size is still the same as the original file, just the contents are gibberish.

If I upload a file multiple times, I get different levels of junk... it's definitely somewhat random.

I originally thought this was a problem with Apache or PHP, but now I'm wondering if it's a problem with Java! I can upload the files over the HTML form just fine, they show up perfect. But once I use either of the GR applets or the GR application, everything gets garbled. Can others with this problem verify that their HTML form uploads correctly?

 
mtt1853

Joined: 2004-05-27
Posts: 5
Posted: Fri, 2004-05-28 04:54

OK, after some more research I have found that this is a bug most likely form the combination of Windows XP and Java. I figured this out by testing the same version of GR on my Linux machine downstairs. Worked without a hitch!

Then I remembered I have a dual-boot with Windows 98 on my current machine... Using the same installation of java that is on my hard drive from WinXP, the uploads worked perfectly in Win 98!

I've tried 3 different VM's in Win XP, including the new 1.5.0 beta, and all of them seem to corrupt images. So I'm wondering if it's a problem in WinXP's Winsock. Any thoughts?

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Fri, 2004-05-28 05:00

Thanks a lot for this analysis. This is a very elusive bug, because few people experience it, and I've never been able to replicate it, even though I also use WinXP.

Let's hope that by gradually tightening the description of the problem we'll end up finding the culprit.

Can you post more details about your config, such as CPU, chipset and network card?

 
mtt1853

Joined: 2004-05-27
Posts: 5
Posted: Sat, 2004-05-29 00:06

I'm running Win XP Professional SP1, on a Pentium III 800 MHz... But I don't think it's my hardware. I just tried using GR from my laptop, which obviously has completely different hardware than my desktop, and I still get the same garbled images. It's running Win XP Home Edition SP1. I wonder if it's a combination of Win XP and my server? I could give you a temp account on my gallery if you want to test that theory... send me your email address if you'd like to do this.

Thanks,
Mike

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Sat, 2004-05-29 00:09

You can send me a PM with the access info.

 
mtt1853

Joined: 2004-05-27
Posts: 5
Posted: Fri, 2004-06-11 04:22

I sent the login info-- were you able to determine whether the problem persisted?

Mike

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Mon, 2004-06-21 23:08

mtt1853, I tried uploading with GR 1.4-beta and it worked fine. I don't think the version number (of GR) matters, this is a Java/OS issue.

 
blumpy

Joined: 2004-04-29
Posts: 7
Posted: Sun, 2004-08-01 17:16

I have this problem as well, and after many hours of frustration, I've come up with a temporary solution: Use a proxy server when communicating with the gallery.

My guess is that there's a possiblity that the problem lies somewhere in the networking code or the JVM networking libraries, and I'm guessing by choosing to use a proxy server, the applet/browser uses different networking libraries?

I dunno, i could be talking out of my butt, but that's my findings so far, hope that helps. I'll post whatever else I can find.

 
blumpy

Joined: 2004-04-29
Posts: 7
Posted: Sun, 2004-08-01 17:45

Oops, or possibly be the OS (win2k) networking code? I do notice the chances of the "streaking" are higher if the computer is busy doing other stuff. It appears the problem happens less frequently (but almost always still does) when I have nothing open other than GR. Hope that helps.

 
pivot_enabled

Joined: 2004-09-06
Posts: 1
Posted: Mon, 2004-09-06 17:39

I have this same streaking problem. It is absolutely maddening. I have been using GR for years. A couple of weeks ago all was well. Near as I can tell this problem started manifesting itsself for me as soon as I installed XP SP2.

Like the other posters I am able to successfully upload using HTML. Whether I have GR size the images locally or on the server doesn't matter, if I am using GR the problem shows up. I tried updating the Java VM. No Joy.

 
mtt1853

Joined: 2004-05-27
Posts: 5
Posted: Wed, 2004-09-15 06:51

AHA! Well I fixed the problem... I can't guarantee it will work for anyone else, but strangely the problem turned out to be an odd combination of Windows XP (since it worked in Linux and Win 98), Java (since it worked in publishing wizard and not in GR), and MY ROUTER. I got a new router today, and lo and behold, it uploads from GR like a charm in XP.

My old router was a Linksys 4-port (BEFSR41). The new one is a Linksys Wireless-G (WRT54GS). BTW, I am still using ethernet cable and not wireless. With the old one I did notice that occasionally I would get 400 (bad request) and "malformed header" errors while internet browsing. Not sure if it was gradually crapping out on me, but I have had it for years and years.

So, for others, I can't guarantee that the problem is the same for you but it's worth looking into!

Mike

 
blumpy

Joined: 2004-04-29
Posts: 7
Posted: Sun, 2004-10-03 08:02

Confirmed, I was using a BEFSR41 as well... switched to a different router (borrowed a Dlink from a friend) and the problem went away. I read about a problem with the BEFSR41 when having multiple different OS's behind the router... this could be related. Regardless, it works now... what a frustrating problem. Thanks mtt1853 for the solution!

 
philjung

Joined: 2004-05-18
Posts: 19
Posted: Sun, 2005-04-17 08:28
blumpy wrote:
Confirmed, I was using a BEFSR41 as well... switched to a different router (borrowed a Dlink from a friend) and the problem went away. I read about a problem with the BEFSR41 when having multiple different OS's behind the router... this could be related. Regardless, it works now... what a frustrating problem. Thanks mtt1853 for the solution!

Just wanted to chime in here. I also had corrupt image upload problem with the applets, but had no problems with the other upload methods.

I was using a Linksys BEFSR81 (8-port versoin of the BEFSR41). I hooked up and old US Robotics DSL router (model USR8011) and voila, it works like a champ now.