[Module] Image Firewall Bypass Module
|
Dayo
Joined: 2005-11-04
Posts: 1642 |
Posted: Thu, 2010-08-19 00:19
|
|
Image Firewall Bypass Module ********STATUS UPDATE******** Notes Although I intend to personally continue running Gallery 2, I advise all Gallery 2 users to convert their installations to Gallery 3. ********UPDATE Version 0.1.0******** Change Log ********UPDATE Version 0.0.0******** This are just the instructions on how to manually bypass the image firewall. Change Log -- |
|
Posts: 27300
I'm heads down in G3 but this looks great. Good job on the docs. Looking forward to a final module.
Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team
Posts: 36
Great stuff, for me this issue was the only one forcing me to switch to G3
otherwise i'm really happy with G2 and what the community has added in this version,
it would really be very useful once its done
Posts: 4342
This is a fantastic idea! One think I'm uncertain about though: if images are served directly by the webserver then I don't see how "this is achieved without sacrificing security of the storage folder and that permission settings are respected."
Particularly: (user 1 with permissions) does cut-and-paste on the direct image url to the g2data directory (now inside the webroot) and sends it to user 2 (who doesn't have permissions, or isn't even logged in to G2) - what stops user 2 downloading the image?
Or, what stops user 3 (also not logged in to G2, and without a username, and without permission to view any images) from spoofing image urls? Derivative ids are trivial to predict.
Posts: 1642
In the case of security, which I interpret to be access to original files or data in the g2data folder not meant for display, this is 100% the case. You can't get to them.
For permissions on items meant for display such as resizes and thumbnails, in the case of User 1 with permissions sending to his mate, User 2, I'll say if he is inclined to do that, he could also just as easily download the image and email that under the standard configuration. There is no way of stopping someone with permissions who wishes to share the image under any configuration.
As for User 3 spoofing urls, after going through this process I'll disagree with you and say that on the contrary, derivative ids are actually extremely difficult to pick out of the blue correctly and say the chances of making a correct pick of the item id is very low.
You are correct though that not going through the php script means that it is technically possible for a user without permissions to view an image as long as they somehow know the image url before hand. It is important that they know the url beforehand as if you deny access to an image to the "Everybody" group for instance, then this image will not be shown in G2.
I am not familiar with G3 but if it is not passing images through php, then I'll think implementing this should have the same level of permissions control as G3 does now or G1 did before it.
--
dakanji.com
Posts: 4342
It's hard to pick an individual URL, but with a G2 installation full of images it's trivial to pick *a* url that reveals an image, and then by incrementing the id one, or two, or three each time (depends on the number of derivatives created at upload time) you can scan the whole gallery. A php script to download the whole G2 contents is only about three lines long.
Your method is ingenious but reverts to the G1 security model, which is "security by url-hiding" - aka security by obscurity and widely recognised as no security at all.
G3 allows Apache directly to serve images only where user permissions allow public access. As soon as an image is made private in any way G3 inserts .htaccess entry to prevent direct access and inserts its own image firewall (like G2).
Don't get me wrong - I think this is a *great* idea for G2 and will help a lot of people for whom public access is permitted; but for those who do need security I think your documentation is misleading and you should be upfront about the implications of making these changes.
Posts: 1642
I suppose I disagree with the specifics of your point as I have an interpretation of security (access to originals and other files) which is controlled and totally inaccessible.
There is no attempt to hide urls, since they are there in full view. I suppose the bottom line is that various people have different ideas of how much "security" is needed for an image gallery ranging from what I would describe as unreasonable paranoia to reasonable impediments.
In terms of someone jumping through the gallery to download derivatives, I believe it is a bit of a stretch and an extreme case but I suppose it is technically possible as said before.
As an aside, can you try accessing my site to see:
1. What you are able to get.
2. Whether what you get will be something that is not already publicly available. I.E. Whether you can find an item from a private album.
3. How much effort is involved.
4. Whether you are able to get a specific item you want.
Evaluating what I believe to be the likely results of this is how I have determined what I consider to be a reasonable level of security.
Perhaps you might want to edit the documentation to your satisfaction though since I am reasonably satisfied with its accuracy ... as of the three original points raised, we are down to one long shot process.
Cheers
--
dakanji.com
Posts: 4342
We can each have our own interpretation, what's important is that anyone who takes on your mods understands what the implications for their security is.
"Security" is provided by make the url for an image hard to guess; if you can guess, or you have the url, then there's no security. That's not what I understand by security.
Sure - will have a look later.
Yep - will do that too.
Posts: 1642
Fair enough. I suppose we can agree to disagree.
I have added some clarification anyway and looking forward to your report. You can pick a start url for the process by looking at the source.
--
dakanji.com
Posts: 1642
Never looked at G3 code. Can you point me to where this is implemented ... might be some ideas there to take up.
--
dakanji.com
Posts: 4342
I've not looked at G3 at all. I just understood that was the way it worked, from developer discussions. It's one of the reasons that G3 doesn't have item permissions, only album permissions.
Posts: 1642
OK. I can see how this can be done where the images are stored on an album basis.
As for the test, I just put created a brand new private album, named after you, waiting for you to breach ... should be a piece of cake for you to grab the images in that album from what you wrote.
Let us all know when you finish.
Cheers.
--
dakanji.com
Posts: 4342
I'm not going to take part in any timed hacking contests (unless there's a substantial reward, of course) - but as I said, I will take a look at your site when convenient, and see how easy it is to dig out images by url-fabrication.
For anyone following, there's a big hole in the G2 security (as released) in respect of watermarked images too; a hole of which I have personally been a victim. I posted a fix here:
http://gallery.menalto.com/node/87322
Posts: 1642
Sorry. My mistake. I was mislead by the statements above into not seeing this as a timed hacking contest but a trivial matter that would be accomplished by three short lines of code. I see now that they must be three very, very long lines of code.
Let us all know when you are able to find some time to put those three lines together.
I'll take the album down for now and put it back up when you are ready. Just send a PM.
Cheers.
--
dakanji.com
Posts: 4342
It sounds like you've taken umbridge with me in this thread; if I've caused offence then please accept my apologies, and be assured that it wasn't my intention to cause insult.
I've spent a few minutes running urls against your site, as asked. I went to an album from the bottom of the list and picked out a likely starting point - image 6968, I think it was and from there just by incrementing numbers I can dig out images: eg:
http://dakanji.com/imagery/main.php?g2_view=core.DownloadItem&g2_itemId=6986
http://dakanji.com/imagery/main.php?g2_view=core.DownloadItem&g2_itemId=6990
http://dakanji.com/imagery/main.php?g2_view=core.DownloadItem&g2_itemId=6994
I've no idea if those are public, or private; I'm afraid you'll have to take my word that (if they are posted in an album somewhere) I didn't copy the numbers off a page generated by Gallery.
But - if I've understood the mechanism of your speed-increase correctly, then all derivatives (not originals) are exposed to url-guessing, regardless of the permissions set on them. All I have to do is keep trying numbers in sequence, and I'm guaranteed to find all derivatives images in the gallery installation, private ones included.
A security "flaw" doesn't have to be pin-point-specific still to be a flaw. I don't have to decide in advance which private album I'm looking for before running an attack; it's just as valid to scoop *everything* and examine it later.
For example - let's say those private images are naked pictures of my boyfriend (or girlfriend) and I'm using G2 security to protect them from everyone's eyes except his (or hers). A hacker doesn't have to know which album exactly he's looking for when passing a casual eye over the entire contents of my site that he's just downloaded.
Now, do you still want me to write those three lines of code to scan incremental urls and report when the result isn't an error page?
Posts: 1642
Those urls are standard G2 urls (main.php etc etc) and are thus processed through the standard G2 process (although they can be crafted to match the g2data pattern) and are from the publicly accessible album you started from (I had taken the temporary private one I created down).
I suppose it is down to what each person regards as an acceptable level of security v performance. In my case, I'll take this speed increase and reduced server load.
In any case, this has a bit gone off track. I always stated that it is technically possible to cycle through the item ids to hit images and yes, the all requests for derivatives are served (originals are not) although I can lock specific ones down at the webserver level as well for instance. This requires access to the webserver config - Nginx users will have such access.
With that in mind, the module is there for those with galleries for which it is suitable. These are:
1. People with open galleries. All albums accessible and the important thing is that original images should not be downloadable
2. People without overly sensitive images on the gallery. The obscurity of the urls offers a sufficient level of security
3. People with access to webserver configs to lock down particularly sensitive images or albums.
I can put up the private album again although I think that would probably just be belabouring things for little gain.
--
dakanji.com
Posts: 27300
That was more than 80% of our users so we rethought the G2 permissions system and also found through our extensive user survey that most users only set permissions on a per album basis. That is why we went with the permissions system in G3.
You can't call obscurity, security, but like you said most people will find the obscurity enough and the speed increase will help as well.
I like what you have done and most people with performance issues that don't want to go to G3, because of other features, will appreciate your modifications.
At lease we can agree that obscurity is not true security but it might be enough for most users that want to keep out the honest people.
Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team
Posts: 1642
Cheers chaps.
I decided to pull the module after a review of the comments and also, the End of Life status of G2.
The EOL status also means I wouldn't be doing any more work of the zencart bridge although that has not been pulled.
Not getting into a strop, just a realisation that G2 development for public consumption is essentially a waste of time.
--
dakanji.com
Posts: 8339
Good going guys, talking Dayo out of further G2 development.
Personally I love G2, its extremely powerful and for me very easy to integrate into other apps/sites.
Its downfalls would be horrible error handling. I mean why not display the rest of the page if one or two items fail, instead of killing the whole app?
The other would be exactly what Dayo was trying to develop. Short, quick image rendering for those who desire it.(my method might be to alter the urlGenerator)
G2 development might be done for the core team, but I'm still here and producing what interests me.(Uploadify)
As for G3... I started working with the REST api a while back, when all was changed and my time wasted. Waiting for a more solid jumping point.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 1642
Hi Wayne
To to clarify before this becomes another storm in a tea cup lol.
It is not because of the specific comments here that I decided to stop and I had hooking into the UrlGenerator class as my next step for a full release version (this was v0.x and that would have been a v1.x).
I prefer G2 but the reality is that the devs have not just stopped active coding, which is fair enough, but basically any support even before G3 has been officially released.
I can understand that it is human nature that since they are working on G3, that is all they want to hear of although I would have thought that since G2 is still at this stage the official stable release, there would at least be a small amount of interest in answering development queries. I would have thought this would last until say G3.1 but this is not the case.
So the bottomline is that G2 development is essentially a painful exercise as you are on your own due to lack of dev interest in answering queries, and a waste of time due to lack of public interest. The realisation of this fact is what drove my decision.
If I need something, I'll knock it up for myself. I started looking at this bypass because my tiny VPS was creaking and by doing this, I now have spare capacity but what is fine for me is not necessarily ok for general consumption particularly in a declining userbase. By just doing it as I need it, I don't need to go through the hassle of the level of care needed for general release.
As for pulling this specific module, I knew I would not have the time to spend trying to amend to meet general release requirements that's all. So I will just take it forward for my own use which would be specific requirements implemented with full understanding of the issues.
So again, not due to the specifics of the comments here which have their validity in specific contexts as clearly set out.
--
dakanji.com
Posts: 4342
I was going to have a close look at the code, and see if I could make some helpful suggestions (probably obvious, but anyway) things like how to get the g2data folder location out of the api (it was hard coded when I looked).
An alternative approach would be to hack the downloadItem view (instead of the render function) to redirect to the file in the cache location.
I'm also curious about how to turn this into a module as opposed to some code mods. It would be nice if Dayo would re-post the code, especially having gone to the trouble of writing, installing and testing it. Even if he doesn't want to carry on with it, someone else might.
Alternatively, the code is still in the codex history: latest revision as of this time is here, for anyone interested:
http://codex.gallery2.org/index.php?title=Gallery2:Modules:Bypass&oldid=21962
Posts: 36
this is kind of sad.. since I was really looking forward for this.. oh well.
Im not a developer just a consumer, but I have this strong opinion that the bells and whistles that G2 community has developed out weighs the pros of G3 at the moment, and it would take substantial time till an end-consumer like me starts thinking of porting to G3, since I don't see all the features .. doing small things like making square thumbs are very hard for me at the moment in G3.
I think this module would have made gallery 2 more practical maybe a bit more popular till G3 catches on with its strength,
as pointed out before even if it causes security lapse it helps a large number of open gallery users where site responsiveness is the main factor.
------------------
on the side note.. Can any of guys explain errors in my event log. is this something to do with g2 image firewall or something unrelated
I use Database locking system because my gallery becomes crazy if I use File locking
Type Gallery Error
Location http://www.farhan-tauheed.com/gallery2/main.php?g2_view=core.DownloadItem&g2_itemId=39&g2_serialNumber=6
User Id 5
Client 128.178.77.146
Summary ERROR_LOCK_TIMEOUT
Referer http://www.farhan-tauheed.com/gallery2/main.php?g2_itemId=49
Details
Error (ERROR_LOCK_TIMEOUT)in modules/core/classes/GalleryStorage/GalleryStorageExtras.class at line 1980 (GalleryCoreApi::error)
in modules/core/classes/GalleryStorage/GalleryStorageExtras.class at line 683 (GalleryStorageExtras::_getLockClearance)
in modules/core/classes/GalleryStorage.class at line 397 (GalleryStorageExtras::acquireWriteLock)
in modules/core/classes/DatabaseLockSystem.class at line 42 (GalleryStorage::acquireWriteLock)
in modules/core/classes/GalleryLockSystem.class at line 147 (DatabaseLockSystem::_acquireLock)
in modules/core/classes/helpers/GalleryLockHelper_simple.class at line 176 (GalleryLockSystem::acquireWriteLock)
in modules/core/classes/GalleryCoreApi.class at line 2291 (GalleryLockHelper_simple::acquireWriteLock)
in modules/core/classes/helpers/GalleryDerivativeHelper_advanced.class at line 833 (GalleryCoreApi::acquireWriteLock)
in modules/core/classes/GalleryCoreApi.class at line 997 (GalleryDerivativeHelper_advanced::rebuildCache)
in modules/core/classes/helpers/GalleryDerivativeHelper_simple.class at line 50 (GalleryCoreApi::rebuildDerivativeCache)
in modules/core/classes/GalleryCoreApi.class at line 984 (GalleryDerivativeHelper_simple::rebuildCacheIfNotCurrent)
in modules/core/DownloadItem.inc at line 109 (GalleryCoreApi::rebuildDerivativeCacheIfNotCurrent)
in main.php at line 450 (DownloadItemView::renderImmediate)
in main.php at line 104
in main.php at line 88
Request variables: Array
(
[view] => core.DownloadItem
[itemId] => 39
[serialNumber] => 6
)