WebDAV module, image Upload broken?
|
damato
Joined: 2004-10-30
Posts: 29 |
Posted: Tue, 2007-02-27 18:11
|
|
Hi, I just updated my gallery2 installation to the very latest version in the subversion repository (revision 15879) and enabled the WebDAV module. I am perfectly able to mount albums via the OSX WebDAV client and I can also perfectly rename albums/images as well as drag&drop new images into the WebDAV connected folder under MacOSX. However, as soon as I try to view those dragged images on my gallery2 website all I see is just the replacement image which is normally displayed if the image is broken. In addition, the size of the image is set to 92x92 pixels only. No matter what, if I look at the actual server's file directory I can perfectly see the uploaded image file and I am also able to verify that the file is not corrupt and displays perfectly. So does anyone have an idea why the upload succeeds, but gallery2 itself doesn't seem to trigger imagemagick/gd or netpbm for actually putting the image into the gallery? I tried to set NetPBM or GD as the default image tool, but without success. No matter which image I try to upload via WebDAV, all I can see is the 'broken image' in my gallery2. And ideas? Thoughts? |
|
Posts: 32509
- what OSX version are you using? and are you using its built-in webdav client or another program to connect to g2?
- let's focus on webav and forget about the broken thumbnails and broken resizes for now.
when you upload files via webdav, are the files uploaded successfully and in your albums afterwards?
you say the file is in g2data/albums/, but can you see / download the file when browsing via g2 as well?
- can you post a url to such a file / album maybe?
topic moved to the support forums.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 29
I am using the latest MacOSX 10.4.8 version and I am connecting to gallery2 via the builtin WebDAV client, yes.
As I said above, the raw image files are uploaded correctly to the hard disk of my server (/g2data/albums) yes and I also verified that they are working fine and aren't broken after the upload. However, they show up broken in the gallery2 web frontend. Both, the thumbnails are broken, but also the highest resolution image is shown as broken in the gallery2 web frontend.
Sorry, can't post a link yet. But as I said, g2 only shows the "broken image thumbnail" no matter what resolution of the image I choose. To me it looks like the image tools (imagemagick, etc.) are never executed and/or the database not updated properly.
The data of my server installation is:
-- cut here --
Gallery version = 2.2-rc-1 core 1.1.29
PHP version = 5.1.6-pl8-gentoo apache2handler
Webserver = Apache/2.0.59 (Gentoo) mod_ssl/2.0.59 OpenSSL/0.9.8d DAV/2 PHP/5.1.6-pl8-gentoo
Database = mysqlt 5.0.32-log, lock.system=flock
Toolkits = Gd, Dcraw, NetPBM, ArchiveUpload, Exif, ImageMagick, Getid3, LinkItemToolkit, Ffmpeg, Thumbnail
Acceleration = full/900, none
Operating system = Linux gandalf 2.6.20-gentoo #3 Sat Feb 24 18:31:57 CET 2007 x86_64
Default theme = matrix
gettext = enabled
Locale = en_GB
Browser = Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en; rv:1.8.1.2pre) Gecko/20070223 Camino/1.1b
Rows in GalleryAccessMap table = 41
Rows in GalleryAccessSubscriberMap table = 8753
Rows in GalleryUser table = 4
Rows in GalleryItem table = 8882
Rows in GalleryAlbumItem table = 232
Rows in GalleryCacheMap table = 420
-- cut here --
cheers,
jens
Posts: 32509
> but also the highest resolution image is shown as broken in the gallery2 web frontend.
fullsize images are not generated by g2. are you using watermarks? else this doesn't make a lot of sense.
it looks like a graphics toolkit issue and doesn't seem to be related to webdav.
please deactivate all graphics toolkits (netpbm, gd, imagemagick) and upload another item via webdav.
i expect that the image doesn't have a thumbnail, no resize, but that the fullsize shows correctly in the g2 UI.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 29
No, I am not using watermarks at all. However, please note that normal image upload works fine (e.g. using the web frontend or via ftp upload to the server and then import to the g2 UI).
Of course I don't expect that the webdav protocol is to blame here. However, the WebDAV plugin must call certain scripts and stuff which notifies g2 about the webdav copy operation. And here probably something fails.
I did this right now. Unfortunatly the very same result, however no thumbnail image is shown. As soon as I uploaded the image via WebDAV I can see the new image entry in the g2 UI, and it says "no thumbnail" as expected (due to the disabled image toolkits).
But when I click on the "no thumbnail" g2 forwards me to the image page of the uploaded image. Unfortunatly, the g2 UI doesn't show the uploaded raw image at all, but a "Download photo" link. If I click on that link I am pefectly able to download the image via my webbrowser.
So image upload via WebDAV works, however thumbnails are not generated and the g2 UI only presents me with a "Download photo" link. Could this also possibly be a MIME type problem or such? However, as I said, if I upload the very same image via the g2 UI instead of WebDAV everything will be finde.
cheers,
jens
Posts: 32509
I didn't notice that we're talking about RAW images before. That explains why fullsize images are a result of graphics toolkit operations.
I thought you were uploading normal jpg / gif images.
Please enable your graphics toolkit modules again and try to upload a .jpg image via webdav.
Do you get thumbnails etc?
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 29
Sorry for the confusion. Of course I am trying to upload jpg files here and not real RAW files. With 'raw' image I meant the image exactly as I uploaded it via webdav and not a generated thumbnail for example.
Posts: 32509
In that case I don't understand this bit:
since you're using the matrix theme, it shouldn't show the "download photo" text at all. it should just display the image itself, even if no graphics toolkit module is active.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 32509
BTW: just added some images with webdav. still works fine.
adding images with webdav is not different from other add item methods when it comes to detecting files as photos, generating thumbnails and stuff like that.
it's all using the same code.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 29
Well, I don't doubt that. However, for me it doesn't seem to be the very same. Here I can perfectly reproduce it with the version I just grab from the SVN this morning. As soon as I add an image via WebDAV I can see the file in g2date/albums, however I am never able to view the actual image inline in the g2 UI. All I see is the broken thumbnail replacement and if I click on it I see the "Download photo" link which allows me to download the file via HTTP.
So there must be something wrong. Any chance of giving me an advice on how to debug that situation?
Posts: 11
Hey guys, I'm having a similar issue, broken images / thumbs after uploading via Webdav. Clean install of G2.2rc2 on Dreamhost, fairly experienced gallery user. Mounted my root album via Webdav, ran some smaller tests, which worked fine. Then uploaded my entire photo tree, about 2000 photos and it seemed to work ok, but some albums contain broken images.
From what I read above all symptoms are the same. Although I'm connecting to Gallery via WinXP using Novell's netdrive for Webdav support.
What I noticed about the broken images is that if I click "Show debug tree" while trying to view the item it appears that the mime type is messed up. It shows up as application/unknown, although the file has the correct extension (.jpg)
Further, I can't seem to delete the thumbs using /lib/support/ cache del derivatives, although I'm new to that page so I might be doing it wrong. So it sucks, half of my 3 gigs of images are broken, I hate to re-upload them with the same results (took 3 days).
Anyone have any thoughts? Thanks!!!
Here is the full debug tree for that item:
* 7: (GalleryAlbumItem) [browse]
o 639: (GalleryAlbumItem) [browse]
+ 2008: (GalleryAlbumItem) [browse]
# 2115: (GalleryPhotoItem) [browse]
width 2272
height 1704
mimeType application/unknown <======================
size 1336994
canContainChildren 0
description
keywords
ownerId 6
renderer
summary
title IMG_6256.JPG
viewedSinceTimestamp 1173460795
originationTimestamp 1159647670
pathComponent IMG_6256.JPG
parentId 2008
id 2115
creationTimestamp 1173460795
isLinkable 1
linkId
linkedEntity
modificationTimestamp 1173509197
serialNumber 2
entityType GalleryPhotoItem
onLoadHandlers
_persistentStatus Array
* 2116: (GalleryDerivativeImage)
* 2117: (GalleryDerivativeImage)
Here is my Gallery info:
Gallery version = 2.2-rc-2 core 1.1.30
PHP version = 5.2.1 cgi-fcgi
Webserver = Apache/2.0.54 (Unix) PHP/4.4.4 mod_ssl/2.0.54 OpenSSL/0.9.7e mod_fastcgi/2.4.2 DAV/2 SVN/1.3.2
Database = mysqli 5.0.18-standard-log, lock.system=database
Toolkits = Exif, Getid3, LinkItemToolkit, Thumbnail, Gd, SquareThumb
Acceleration = full/21600, partial/900
Operating system = Linux redhot 2.6.18.1-grsec+e+f6b+gr219+opt+c4a+gr2b-grsec #1 SMP Thu Oct 19 12:50:29 PDT 2006 i686
Default theme = matrix
gettext = enabled
Locale = en_US
Browser = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2
Rows in GalleryAccessMap table = 56
Rows in GalleryAccessSubscriberMap table = 1969
Rows in GalleryUser table = 2
Rows in GalleryItem table = 1968
Rows in GalleryAlbumItem table = 49
Rows in GalleryCacheMap table = 19
Posts: 32509
Triumph
your case doesn't sound exactly the same as the one discussed in this thread.
the discussed case is:
- when adding images with other methods, damato gets thumbs and resizes
- when adding images with webdav, damato gets the "broken image" placeholder for the thumbs and resizes
your case:
- "half of my 3 gigs of images are broken,"
-> some files/images worked, some didn't.
but you mentioned messed-up mime-types, which certainly sounds like the above discussed issue.
so the question is:
can you reproduce the issue?
- pick an image that uploaded fine. can you upload it with webdav again with the same result (still fine)?
- pick an image that didn't uploaded fine (has broken thumb). can you upload it with webdav again with the same result (again a broken thumb)?
if so, what's the difference between those files? file-type? size? dimensions? colorspace, exif data, ...? lower / upper case? special characters in the file name?
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 11
Hi guys, just replied but I think it got lost.
Looking at it again, I agree that it's not exactly the same problem. I ended up deleting all of the broken images with bad mime types and I'm re-uploading now (via webdav again). It seems to be working ok, I'm not seeing the same issue. Will report back if I run into it again.
As for differences in the files that worked vs didn't, I couldn't honestly find any differences at all, but I will report it if I ever figure it out.
I'm hoping damato got his issue resolved as well as I will be using webdav a lot in the future if it holds up! Thx!
Posts: 10
I can reproduce this issue, with 2.2 RC2
When uploading a jpeg via WebDAV, the image uploads fine, but as Damato reports, no thumbnails or resizes are displayed, just the broken image icon.
The full size image is fine.
If I upload the same images via another method, like the GR Applet, thumbs and resizes display fine.
Other things of note - the UI calls the images "files" not "Photos". ie. "Edit this File", not "Edit this Photo".
Could it be that the mimetype is not being set correctly when the files are uploaded?
Here's a URL with images uploaded via WebDAV: http://gallery.stuartherring.com/v/Staging/ and one uploaded via the web browser.
Posts: 21
The problem is changing an item's entity type. You can reproduce a version of this problem with Windows or cadaver by writing a movie to a GalleryPhotoItem, or a photo to a GalleryUnknownItem. The mime type should be updated correctly, but the entity type remains GalleryPhotoItem or GalleryUnknownItem, respectively.
Because Mac OS X and GNOME each first PUT an empty file (GalleryUnknownItem), then fill it with data, you get all the correct data on the server, but it's a GalleryUnknownItem vs. a GalleryPhotoItem. Consequently the thumbnail is broken.
I experimented in ItemAddWebDav::_replaceItem with $item->setEntityType($newEntityType), without success. How can we change an item's entity type?
Posts: 32509
jablko
I don't understand.
- None of these users is overwriting a file, it's an initial upload.
- Even if there's an empty PUT first, the PUT is for the right filename -> our file extension based mime-type mapping should create an item with of the right type.
Theory:
For the empty PUT, after detecting the right mime-type based on the file extensionSome, we run an identify operation (toolkit) and that operation fails upon which we change the item to unknown item.
Can you confirm this theory or what's the problem?
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 21
Confirmed. Your theory is what's happening.
Posts: 32509
Thanks for testing!
Why isn't this happening for all users? Isn't the Windows Webdav client doing the same thing? It's probably toolkit-dependent.
What's the fix? In ItemAddWebDav, should we add a check for zero-sized files before we actually call addItem / replaceItem?
Alternatively, we could check for zero size after the item has been added and fix the mime-type and entityType afterwards. But that's complicated. There's no GCA function yet to preserve the itemId but change the entityType. So the above workaround would be better.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 10
I've updated my example to show the results of uploading from Windows (XP SP2), Konqueror, and Nautilus.
It appears that KDE's WebDAV KIO-Slave works fine, but the others result in no resizes or thumbs.
Also - the images that get thumbs and resizes in Gallery are also recognized as images by Konqueror, and it generates a preview for them - the others are not.
Posts: 32509
mabinogi
it's not that general. it works fine for me using Windows XP's built-in webdav client.
when you deactivate your graphics toolkit modules (gd, netpbm, imagemagick), does g2 recognize the files as images? you should see the item in g2 as images without thumbs and resizes.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 32509
mabinogi
Can you reproduce the problem with Windows XP's built-in webdav client (Windows Explorer)? I can't and you shouldn't be able to do that.
The problem jablko describes doesn't apply to this webdav client.
----
FYI: I files a bug about the OS X issue at http://sourceforge.net/tracker/index.php?func=detail&aid=1681406&group_id=7130&atid=107130 .
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 10
The previous Windows example was in fact done with the built-in webdav client - I mapped the gallery to a drive. I have no third party webdav software on this machine.
I have now also created a Network Place for the gallery via My Network Places and uploaded an image via that, which appears to give the same result as using the mapped drive.
I also disabled all the toolkits and uploaded two more images - one via the mapped drive, and one via the network place. Those ones get recognised as photos.
(EDIT: removed the stuff about the edit conflict, as I sorted that out and it's probably not relevant to this issue)
Posts: 5
I'm having the same problems described by damato, only on OS X 10.4.9 and Gallery2.2. It sounds like the problem was found, but a fix has yet to be implemented. If there's something more that needs debugging, please reply and let me know. I'd be glad to help out.
Posts: 6
I have the same problem using the Gnome VFS client. The pictures never have thumbnails and are identified as 'items'. It works fine from KDE and Windows webdav (when that behaves)
Posts: 20
I am also having a webdav problem with OS X 10.4.9 and Gallery 2.2.
I can connect fine, create folders, and delete pictures and folders.
However, when I upload a file, it times out on trying to close the file. I get messages like: "Copied: 3.9 MB of 3.9 MB (Closing file...) - Please wait..." in a Finder dialog box which eventually times out with an error. (See attachments for dialog boxes).
Any idea what this could be? Related to the problems others are having?
Posts: 20
I want to add that I tested webdav with Windows XP and it works fine, so the problem I am having is specific to OS X's WebDAVFS. Even though the symptoms are identical, I'm guessing it's caused by the same problems mentioned above.
Does anyone know a good webdav upload client for OS X besides the one built into the Finder? Maybe I can use that as a workaround.
Posts: 32509
That's for the feedback.
jablko, our webdav expert, will work on a fix for OS X / gnome / kde soon.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 42
FWIW, I'm having the exact same problem as the OP with Gallery 2.2 and Mac OS X 10.4.9.
Posts: 10
I wonder what's different in my case that I'm seeing it with Windows when others aren't?
I guess I'll just wait and see if the fix for OS X and Gnome fixes my problem too...
Posts: 5
sneeper, I downloaded an application called Goliath http://www.webdav.org/goliath/ that works fine in OS X. However the real reason I wanted to do WebDAV was to run a script against my iPhoto library to sync all albums, and I can only do that once it is physically mounted as a filesystem. Too bad for now! Goliath will suffice.
Posts: 42
Valiant.... also talk to jablko about having Gallery ignore the ".DS_Store" files that are automatically created in every folder with Mac OS X. Gallery treats those hidden folders as new albums.
Posts: 20
Ditto on the .DS_Store files! They should be ignored by Gallery if possible.
.DS_Store is where Finder saves how the person last viewed a folder.. For example, if it was in column view, or with giant icons, or with a custom background, etc. It can safely be deleted/ignored and Finder will just view the folder as if it hadn't seen it before.
Posts: 32509
sneeper, JustinHoMi
could one of you guys please post a bug report about DS_Store files at http://sf.net/projects/gallery/ ?
thanks!
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 42
Doing it now....
Posts: 1
Same problem here, can add files to the gallery through WebDAV (in the Finder, OS X 10.4.9) but they don't show up as pictures, hence no thumbnails etc.
I created a tcpflow dump of a simple upload transaction which is available here:
http://www.typo.nl/misc/gallery2-osx-webdav.txt
[edit]
I also created a logfile of an upload transaction:
http://www.typo.nl/misc/gallery2-osx-webdav-log.txt
It seems to recognise the file as image/jpeg and tries to create a thumbnail for it, a while later it changes the image to application/unknown (just like Valiant predicted).
[/edit]
Note that I have .DS_store files switched of on WebDAV mounts ( http://www.macosxhints.com/article.php?story=2005070300463515 )
I'm using Gallery 2.2.1
Posts: 3
Gallery 2.2.1 and uploading a jpg with webdav...
Using Gnome in fc6, I can upload, rename, delete, etc. but the image thumbnails are not created and the image is unviewable as the filetype is "unknown" as far as gallery is concerned. "There are no graphics toolkits enabled that support this type of item, so we cannot create or modify a thumbnail."
Using WindowsXP, the webdav feature now works as expected.
Posts: 1
Roger that, same problem. OS X 10.4.9. WebDAV .DS_Store files = off.
Posts: 32509
Thanks to jablko, the fix is in.
@ALL:
Upgrade to the next nightly snapshot (May 30th, 2007) or do a svn update
to r16462 in case you're using subversion.
We need people to test this before we can release G2.2.2 with this
bugfix.
Please let us know whether it works for you.
--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 5
It's hard to tell if this nightly fixed uploads because for some reason a directory listing shows up blank once you've authenticated. Has anyone else seen this problem?
Posts: 32509
Sorry about that. webdav is indeed broken in SVN. It will be fixed in a day. I'll update this thread when it's fixed.
--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 32509
Webdav's fixed again. please grab the latest nightly snapshot / svn update.
Posts: 5
Things still look empty to me when I connect in OSX. I can go in there and create a new directory, but the minute it creates the directory it disappears and I can't actually give the directory a name. It's odd too because it doesn't even try to authenticate me against gallery's login/pass until I go and create the folder. You'd think it would want that authentication before it even does a directory listing.
Posts: 32509
please also test with your OS X / GNOME _webdav_ client against this test installation. it's a G2.2.2 preview:
http://cordic.org/g2_2_2/
user: demo
password: upload
--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 5
Cool, your 2.2.2 preview works. Now you have me completely envious ;)
Posts: 2
I have the new webdav plugin installed and it's working fine, except that Mac mime types are not recognized and so an Mac generated jpeg file without the .jpg extensions are not being recoginzed. The Mac file system identifies file types using a resource fork in the file encoding. I know the other 'clients' recognize and assign the correct mime type to these files so it would be nice if the the webdav client was able to handle the Mac mime types too.
Steve
Posts: 1
Hi,
I have the same problem as the ones above.
Got the G2 Experimental 16566 build on a Fedora box with Apache 2.2.2 and php 5.1.6.
When I open up a Webdav mapping on Windows XP SP2, the display name do not show the extension (I guess this is what you have said before is because it have the MIME type set?).
Was there a fix on this somewhere, like we could tweak the WebDav on G2 to actually send whole name and not only the display name?
I have a file in one of the albums, it is named 293479614.jpg and the Display name is set to: Suicide, but in the Windows XP Client the file is shown as: Suicide
Is that the way it should be? (as the other ones I can without a problem upload a file and get it to work, but I guess that is because it sets the display name to the actual file name).
If you want more input I'll be glad to help, or if you can give some output I am still glad
Posts: 32509
@cruxis:
no, there's no such feature.
g2's webdav module submits the item title and filename to the webdav client.
the windows xp client only shows the item's title, not the filename.
--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 32509
@AustinChronicle:
it's not really common to deal with filenames without extension, is it?
anyhow, g2 also looks at the content-type header submitted by the webdav client. deriving the mime-type from the extension is just the fallback code. so it should work.
--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 2
Not commom except for OS X which uses a resource fork on files without an extension, a legacy hack so there would be better legacy support for OS 9 files. If you want Gallery to be fully inter-operable with Mac systems then the webdav client should support Mac mime types. For me it's no big deal but I work with a many Mac creative types who cannot be bothered with file extensions, especially when uploading folders of multipe images via webdav. When we upload Mac files via the web browswer client they are assigned the correct mime types, why not with the weddav client.
I'll keep trying but right now files uploaded without extensions are not recoginzed as the correct mime type.
Thanks for all the great work on the webdav client and the obvious workaround on this end is to make sure files have extensions, but the Lowest Common Denominator will not be served, and my users will complain.
Posts: 29
Hi guys,
I'm partly back and have updated my SVN checkout of G2 to r16603 and it seems the WebDAV problems (uploaded file, no thumbnail) are gone on MacOSX. Thanks a lot for the fix in G2 and the continued pressure to the developers in here ;-) Now I can finally use WebDAV to manage my gallery. Great.
However, as it has been reported here as well, the ".DS_Store" files are still added everytimes I am connecting to the WebDAV and browsing through the folder. They should really be ignored if this is possible as it is annoying to always have to remove them manually all over the place.
In addition, is it also possible to do a secure "https://" connection to the WebDAV because I really don't want to have my passwords sent in plain authentication through the internet. Would really be great to have that.
Also, I would like to restrict the WebDAV access only for certain users in G2, is that possible as well somewhere?
Posts: 32509
- We'll fix the DS_Store issue soon.
- @https: please file a feature request.
--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage