[WORKAROUND] Thumbnails not displayed for FLV videos

jsroques

Joined: 2006-12-03
Posts: 4
Posted: Sun, 2006-12-03 17:49

Hi,
I have some troubles displaying thumbnails for flv videos. I need to add some custom thumbnail and the remove it to have it displayed.

I am fairly new to Gallery.

Here is some tests I did :
1 - Add a video : thumbnail is not displayed (I have the generic video one)
2 - Go to "Thumbnail" tab in "Edit Movie",set a new offset abd save : thumbnail still not displayed
3 - Go to "General" tab, select a custom thumbnail and save. The thumbnail I chose is displayed correctly
4 - Still in "General" tab, tick "Remove custom thumbnail for this item" and here is the magic : movie thumbnail (the one corresponding to the offset) is now displayed !
5 - And if I change the offset, thumbnail is changed accordingly.

So my guess is that ffmpeg is able to generate the thumbnail but there is a "missing link" somewhere (in the gallery db ?) that is "repaired" when using a custom thumbnail.

I attached 2 debug logs : for step 1 and step 5. In the first one, ffmpeg is executed once using -vstat to get some info and in the 2nd one, it is executed 3 times and once for creating the thumbnail.

I am no developper so I don't really know where to look next.

Any idea on how to correct this ? What am I missing ?

Thank you in anticipation for your help

JS


================== My gallery install ========================
Gallery version = 2.2-svn noyau 1.1.23
PHP version = 4.4.4 cgi
Serveur Web = Apache/1.3.33 (Unix)
Base de données = mysqlt 5.0.26-standard-log, lock.system=flock
Boîtes à outils = ArchiveUpload, Exif, Getid3, LinkItemToolkit, NetPBM, Thumbnail, Gd, Ffmpeg, SquareThumb
Accélération = none, none
Système d'exploitation = Linux infong 2.4 #1 SMP Thu Jan 13 08:59:31 CET 2005 i686 unknown
Thème par défaut = matrix
gettext = activé
Langage = fr_FR
Navigateur Web = Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) Gecko/20061010 Firefox/2.0
Rows in GalleryAccessMap table = 26
Rows in GalleryAccessSubscriberMap table = 52
Rows in GalleryUser table = 8
Rows in GalleryItem table = 32
Rows in GalleryAlbumItem table = 6
Rows in GalleryCacheMap table = 0

AttachmentSize
logs.zip20.12 KB
 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2006-12-03 18:29

your ffmpeg binary needs to support flv. from the 1st debug log i see your ffmpeg can read flv information.
that doesn't mean that your ffmpeg can extract a single frame from a flv movie and convert it to a jpeg image.

from the 2nd debug log i gather this:

Quote:
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Executing: (nice "/kunden/homepages/35/d144959315/htdocs/tools/ffmpeg" "-i" "/homepages/35/d144959315/htdocs/videos/cache/derivative/2/2/221.dat" "-ss" "15" "-f" "mjpeg" "-t" "0.001" "-y" "/homepages/35/d144959315/htdocs/videos/tmp/fmpg_kMJYX1" ) 2>/homepages/35/d144959315/htdocs/videos/tmp/g2dbgdOpQky
...
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Regular Output:
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Error Output:
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] FFmpeg version SVN-rUNKNOWN, Copyright (c) 2000-2006 Fabrice Bellard, et al.
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] configuration:
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] libavutil version: 49.1.0
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] libavcodec version: 51.25.0
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] libavformat version: 51.6.0
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] built on Nov 28 2006 23:44:16, gcc: 2.95.4 20011002 (Debian prerelease)
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Seems that stream 0 comes from film source: 1000.00 (1000/1) -> 25.00 (25/1)
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Input #0, flv, from '/homepages/35/d144959315/htdocs/videos/cache/derivative/2/2/221.dat':
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Duration: 00:03:39.2, start: 0.000000, bitrate: 56 kb/s
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Stream #0.0: Video: flv, yuv420p, 320x240, 25.00 fps(r)
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Stream #0.1: Audio: mp3, 22050 Hz, mono, 56 kb/s
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Output #0, mjpeg, to '/homepages/35/d144959315/htdocs/videos/tmp/fmpg_kMJYX1':
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Stream #0.0: Video: mjpeg, yuvj420p, 320x240, q=2-31, 200 kb/s, 25.00 fps(c)
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Stream mapping:
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Stream #0.0 -> #0.0
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Press [q] to stop encoding
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] frame= 1 q=3.5 Lsize= 6kB time=0.0 bitrate=1247.4kbits/s
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] video:6kB audio:0kB global headers:0kB muxing overhead 0.000000%
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Status: 0 (expected 0)

seems that extracting a jpeg from a flv movie was successful.

then there are a few operations to add the film-reel watermark on the jpeg (netpbm)
the last one is:

Quote:
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Executing: (nice "/kunden/homepages/35/d144959315/htdocs/tools/netpbm/jpegtopnm" "--quiet" "/homepages/35/d144959315/htdocs/videos/cache/derivative/2/2/221.dat" | nice "/kunden/homepages/35/d144959315/htdocs/tools/netpbm/pnmcomp" "-align=left" "-valign=top" "-alpha=/homepages/35/d144959315/htdocs/videos/tmp/npbm_ff8Jo5" "/homepages/35/d144959315/htdocs/videos/tmp/npbm_Mmqsua" | nice "/kunden/homepages/35/d144959315/htdocs/tools/netpbm/pnmcomp" "-align=right" "-valign=top" "-alpha=/homepages/35/d144959315/htdocs/videos/tmp/npbm_lUmzJT" "/homepages/35/d144959315/htdocs/videos/tmp/npbm_ehXpjW" | nice "/kunden/homepages/35/d144959315/htdocs/tools/netpbm/pnmtojpeg" "--quality=75" > "/homepages/35/d144959315/htdocs/videos/tmp/npbm_yx1ndR" ) 2>/homepages/35/d144959315/htdocs/videos/tmp/g2dbgX5DPPQ
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] file_exists(/homepages/35/d144959315/htdocs/videos/tmp/g2dbgX5DPPQ)
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] filesize(/homepages/35/d144959315/htdocs/videos/tmp/g2dbgX5DPPQ)
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] unlink(/homepages/35/d144959315/htdocs/videos/tmp/g2dbgX5DPPQ)
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Regular Output:
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Error Output:
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] Status: 0 (expected 0)
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] getParameter jheadPath for netpbm plugin
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] rename(/homepages/35/d144959315/htdocs/videos/tmp/npbm_yx1ndR, /homepages/35/d144959315/htdocs/videos/cache/derivative/2/2/221.dat)
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] chmod(/homepages/35/d144959315/htdocs/videos/cache/derivative/2/2/221.dat, )
2006-12-03 17:49:17 [d76f40aa85bdca1d7a4cf573c4ec96ee] is_dir(/homepages/35/d144959315/htdocs/videos/cache/derivative/2/2/221.dat)

which shows that everything should be perfect. no error and the final image file for the thumbnail exists as well.

ok, that's what you describe in step 4. i guess the debug output is from there.

was the movie maybe added before the ffmpeg module has been activated?

 
jsroques

Joined: 2006-12-03
Posts: 4
Posted: Sun, 2006-12-03 22:47

Hi,

I was wondering what netpbm was used for in this case, now I know !

To answer your question, ffmpeg was up & running before importing the movie. And I do believe the thumbnail is created but not "linked" correctly to the item. I did some further testing and here is what I found out :

  • I added a new view. At this point, the default video thumbnail is diplayed (although I did check "create thumbnail now". I forgot to mention that I am using "from local server" option instead of directly downloading the video from my browser.
  • I added a custom thumbnail and delete it immediately : then the thumbnail it set to the first frame of the video, as it should be !

This would confirm that the thumbnail is properly extracted from video but not displayed... But I am not even sure. May the thumbnail is calculated again when removing the custom one.

I also checked in the DB (but the schema is a bit complex for a newbie so I may be on the wrong path...)
In the g2_Derivative table I noticed that the field g_derivativeOperations was set to :

Quote:
thumbnail|150;convert-to-image/jpeg;composite|plugins_data/modules/ffmpeg/filmreel.png,image/png,12,399,top-left,0,0;composite|plugins_data/modules/ffmpeg/filmreel.png,image/png,12,399,top-right,0,0

before adding/removing the default thumbnail and the set to

Quote:
convert-to-image/jpeg;thumbnail|150;composite|plugins_data/modules/ffmpeg/filmreel.png,image/png,12,399,top-left,0,0;composite|plugins_data/modules/ffmpeg/filmreel.png,image/png,12,399,top-right,0,0

after.

I am not sure this is linked as changing it manually in the db did not make the thumbnail to appear but there may be some other changes that are made (there must be !).

Anyway... this would mean that some fields are incorrectly set at import time and this would be corrected when adding / removing custom thumbnail ? But I could'nt figure out where it is said : "here is the thumbnail this item should use"

Thank you for your help.

JS

 
jsroques

Joined: 2006-12-03
Posts: 4
Posted: Sun, 2006-12-03 23:14

Hi again,

I said something incorrect. I enabled debug and when removing the custom thumbnail, ffmpeg is executed and generates the thumbnail. On the contrary, when adding a video, ffmpeg do not generates the thumbnail. It is executed once with the -vstat option.

To get the thumbnail calculated from the video I have first to add/remove a custom thumbnail.

The question is why ? What could prevent gallery to run ffmpeg for generating the thumbnail ? I could find where it should be executed in Gallery php code...

JS

 
jsroques

Joined: 2006-12-03
Posts: 4
Posted: Tue, 2006-12-05 22:46

Hi,
Just to let you know that if I don't check "generate thumbnail" when importing a video... the thumbnail is generated correctly !
Thank you for the help.
Jean-Sébastien.

 
mabinogi

Joined: 2004-12-18
Posts: 10
Posted: Sat, 2007-01-20 12:09

I have had a similar problem - nothing I can do will make ffmpeg create thumbnails.
I've verified that ffmpeg supports the file formats, and that I can extract individual frames from them, but g2 just doesn't do anything with it.

So I dug through the code (SVN from last week and today), and from what I can tell, it should never work.
ffmpeg registers itself as a toolkit provider for convert-to-image/jpeg, whereas the thumbnailing process appears to be looking for 'thumbnail'.
In addition to that, the thumbnailing process seems to be deliberately avoiding the possibility of calling ffmpeg, even if it was registered to handle thumbnail - ThumbnailToolkit hard codes the input mimetype to image/jpeg and copies something (I'm not sure what) first before passing that through to the retrieved toolkit.

Unfortunately I'm unfamiliar with the G2 code, and php in general, so I wasn't able to dig much further. But from what I can figure out, it's not even supposed to work, and has been like that since mid way through 2006.
The thing that mystifies me is that it actually does appear to work for some people.

The add a custom thumbnail then remove it trick did work for me though, so there must be two different paths that generate thumbnails for videos - and the second one is called in the case of removing custom thumbnails, and in that case is looking for 'convert-to-image/jpeg' instead of 'thumbnail'

It's also entirely possible that I was looking in the wrong place (ThumbnailTookit::performOperation) for the code that would call the ffmpeg module to create the thumbnail.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2007-01-20 15:23

> But from what I can figure out, it's not even supposed to work, and has been like that since mid way through 2006.

FYI:
it works.

how do you come to the conclusion that it's not supposed to work?

Quote:
In addition to that, the thumbnailing process seems to be deliberately avoiding the possibility of calling ffmpeg, even if it was registered to handle thumbnail - ThumbnailToolkit hard codes the input mimetype to image/jpeg and copies something (I'm not sure what) first before passing that through to the retrieved toolkit.

in g2, you can register toolkits which are then responsible to process certain operations.
multiple toolits can be registered to do the same operation. each such toolkit registers an operation with a certain priority.
if g2 needs to process a certain operation, it checks if there is any toolkit available for that operation. if there is more than one such toolkit, it uses the toolkit that has the highest priority for this operation.

note: priority 0 has a higher precedence than priority 100. g2 picks the one with the lowest priority-number.

the thumbnail toolkit (responsible for custom thumbnails and predefined mime-type thumbnails) is usually registered with priority 50.
this is higher than what other toolkits are registered with, thus the thumbnail toolkit should always be just a fallback solution if no other toolkit is registered to handle the same operation / mimetype.

------------

back to your case. what happens if you disable the thumbnail module?

--------------
Enter the Gallery 2 Theme Contest today!

 
mabinogi

Joined: 2004-12-18
Posts: 10
Posted: Sat, 2007-01-20 22:06

Success!

Quote:
FYI:
it works.

Yup, I figured it must do, I just couldn't see how it could.

Quote:
how do you come to the conclusion that it's not supposed to work?

Through attempting to trace back to find out where the ffmpeg module was going to be called from. However, it turns out that I was looking in the wrong places.

Quote:
the thumbnail toolkit (responsible for custom thumbnails and predefined mime-type thumbnails) is usually registered with priority 50.
this is higher than what other toolkits are registered with, thus the thumbnail toolkit should always be just a fallback solution if no other toolkit is registered to handle the same operation / mimetype.

Yes, that was probably the crux of my misunderstanding, I assumed the ThumbnailToolkit was entirely responsible for generating thumbnails, and that it then delegated to the appropriate module to do the job - so as a result I didn't trace back far enough to see what was calling ThumbnailToolkit.

Quote:
back to your case. what happens if you disable the thumbnail module?

I didn't try disabling the thumbnail module - but I did actually look at the thumbnail settings page, which instantly showed me what I was missing. (With a big warning in nice red letters that clearly explain the problem even). I simply hadn't removed the default thumbnail for the video types. I would suspect that is the cause of jsroques's problem too.
I don't know why I never looked at the thumbnail manager settings until now - I looked pretty much everywhere else, but in any case, that's my failing not Gallery's.

One thing that was lacking though, was the ffmpeg documentation page - it made no mention of the fact that you'd need to go remove the default thumbnail association. But I've updated the page so hopefully no one else will make my mistake.

Thank you for your response, Valiant - it provided exactly what I needed to figure out my mistake :)