Change default charset?..

Lufa

Joined: 2005-04-11
Posts: 33
Posted: Thu, 2005-06-30 21:45

Hi!

I'm trying to use G2b3, integrated in Mambo CMS.

Default codepage for my website is windows-1251. But when I'm entering the embeded Gallery2 page, gallery looks fine but other characters (web sites menus and other CMS mdules) are "scrambled" because of Gallery2 UTF charset.

What can I do? I dont want to convert whole site because a new gallery version... (((

Any advise?
Thank you.

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-06-30 22:09

G2 requires utf-8, period.
if possible, i'd change the charset in the mambo head section to utf-8. what's wrong with utf-8?

Login or register to post comments
Lufa

Joined: 2005-04-11
Posts: 33
Posted: Fri, 2005-07-01 04:33
valiant wrote:
G2 requires utf-8, period.
if possible, i'd change the charset in the mambo head section to utf-8. what's wrong with utf-8?

But my Mambo content have windows-1251 charset! :)

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-07-01 08:52

have you tried using utf-8 as the charset for mambo pages? what does the result look like?

Login or register to post comments
Lufa

Joined: 2005-04-11
Posts: 33
Posted: Fri, 2005-07-01 12:35
valiant wrote:
have you tried using utf-8 as the charset for mambo pages? what does the result look like?

Looked fine...

But I can't use UTF-8 charset, because my web-page is already formded and have too many materials in windows-1251 charset, and it is very hard to "migrate" from UTF to windows-1251.

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-07-01 12:43

how does your windows-1251 content look when you set the <meta content-type header of mambo to utf-8 ?

you can try to start hacking your g2 to output in windows-1251 though...don't know if it's feasible.

Login or register to post comments
Lufa

Joined: 2005-04-11
Posts: 33
Posted: Fri, 2005-07-01 12:53
valiant wrote:
how does your windows-1251 content look when you set the <meta content-type header of mambo to utf-8 ?

Every symbol like "?" in a circle.

valiant wrote:
you can try to start hacking your g2 to output in windows-1251 though...don't know if it's feasible.

Searching this forum - no luck for now...(((

It's horrible! How can be Gallery 2 used as embedded in CSS when only UTF charset is accepted. I won't believe it! Realy!

I loved Gallery...(

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-07-01 13:12

UTF-8 is so general, there's no real reason to use another charset. or is there?

you'd have to convert all g2 output from utf-8 to the other encoding. we have the necessary functions already in place etc. feel free to try your luck.

Login or register to post comments
Lufa

Joined: 2005-04-11
Posts: 33
Posted: Fri, 2005-07-01 13:21
valiant wrote:
UTF-8 is so general, there's no real reason to use another charset. or is there?

Working under M$ Windows RUS/UKR, with simple text editors like Notepad wich are creating cp-1251 text .

valiant wrote:
we have the necessary functions already in place etc. feel free to try your luck.

Sorry, I dont get it. Which functions? Can you explain this?

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-07-01 13:38

hmm, it might be not that easy. i'd say G2 is not ready to switch to non utf-8 output encoding easily.
the easiest fix would have been if smarty would accept a parameter for the encoding. smarty does the rendering so it would be the right place to convert everything. if you convert the strings before, somewhere in G2, you'd get different problems because G2 expects all internal strings to be unicode.

unless another dev has a (better) idea, i'd say you can't switch to non utf-8 output without hacking your G2. no embedded G2 for you then...
(a dirty fix would be to have charset = utf-8 in embedded g2 pages (com_gallery) and the other charset on all other pages of your mambo install.

Login or register to post comments
Lufa

Joined: 2005-04-11
Posts: 33
Posted: Fri, 2005-07-01 15:31

OK. I think it's hopeless...(((

But I just can't imagine that G2, in fact, is writen with CMS neglecting.

90+ % of russian/ukrainian sites are using cp-1251 or koi8 charset. No one of them (!!!) can't use G2 embedding.

It's can be a true... Looks like some kind of spiteful joke...

Login or register to post comments
paparaci

Joined: 2005-06-01
Posts: 21
Posted: Wed, 2005-07-06 09:47

Lufa,

And what you can say about storing data in mysql database? It looks like scrambled characters, isntead of russian symbols..
When my russian users write russian comments, they are shown correctly, but can not be recognized by G2 search functions, or by search engine.

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-07-06 10:32

paparaci, please report this as a bug on http://sf.net/projects/gallery/
thanks!

Login or register to post comments
paparaci

Joined: 2005-06-01
Posts: 21
Posted: Wed, 2005-07-06 10:37

Hmmm, Valiant..
I'm not sure it's the bug infact.. I have not tuned up G2 to use proper language...

'Your webserver does not support localization. Please instruct your system administrator to reconfigure PHP with the gettext option enabled'

I'm not familiar with gettext func, so i'm to see whether my tuning is right or not.. I can't understand this function.. So now i'm not ready to report a bug without being sure that i'm right for sure and it's a real bug..

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-07-06 10:42

paparaci, yeah, better wait until you have gettext support in your php before you report that bug. plus, make sure you use the proper encoding for the database (unicode or ASCII).

Login or register to post comments
Lufa

Joined: 2005-04-11
Posts: 33
Posted: Wed, 2005-07-06 11:24
paparaci wrote:
Lufa,

And what you can say about storing data in mysql database? It looks like scrambled characters, isntead of russian symbols..

Guys look:

I have some site in CP1251 charset. I'm using Mambo CMS. It looks great for me.

I have fresh G2 install. When I go to http://mysite.com/gallery2/ I can see my gallery (imported from G1). I like it! Everything is OK. My pictures are creatю All text data looks fine (UTF charset).

Now! I'm trying to use G2 embedded in my site using Mambo G2 Component.
Remember? Site Data are in CP1251, and Gallery Text Data in UTF.

It's terrible! My CMS trying to interpritate G2 UTF Data like CP1251 (HTML Meta tag "saying" that)! Or, sometimes I can see G2 data well, but all html page is showing like UTF. You see, my pages data are the charecter code according some character table (cp1251). In embedded G2 wich somewhow "overwriting" page charset (meta tag), all codes are searching in UTF character table. Finally I'm getting totally different characters.

That's is my problem.(((

No database problems.

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-07-06 12:06

Lufa, i know that paparaci reported a different issue than yours.
There's not much we can do right now about your issue (embedded G2 when emApp uses different charset). I've described what the options are. We don't have time to work on this now, if you have, please feel free and submit a patch that allows switching the output charset.

Login or register to post comments
Lufa

Joined: 2005-04-11
Posts: 33
Posted: Thu, 2005-07-07 07:15

Sorry, I dont think I'm good enaught in PHP...(((
But I wonder that only my have the such problem.

Thank all of you for trying to help!

Login or register to post comments
drk8

Joined: 2005-07-26
Posts: 3
Posted: Tue, 2005-07-26 07:39

Lufa, was you able to solve the problem yet?

I have the same problem as you. Can anyone help?

I cannot change the character set of Mambo to utf-8 since the whole site will be destroyed with funny character.

Is there any place I can change the charset=utf-8 to charset=iso-8859-1? Please help. I've been searching lot but still could not find any solution.

Login or register to post comments
Lufa

Joined: 2005-04-11
Posts: 33
Posted: Tue, 2005-07-26 07:55

No, I did not...((((

Login or register to post comments
Puntukas

Joined: 2005-07-28
Posts: 2
Posted: Thu, 2005-07-28 11:46

i changed charset=utf-8 to charset=windows-1257 in modules/core/classes/GalleryTranslator.class (270 line) and my problems gone.

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-07-28 11:57

Puntukas, while your hack isn't a real solution (G2 will still output in utf-8, but your browser interprets it as windows-1257), it brought me to an idea.

Additional to your one line change, we could use G2's translation functions to change the internal utf-8 data to any output encoding/charset!

Login or register to post comments
Lufa

Joined: 2005-04-11
Posts: 33
Posted: Thu, 2005-07-28 13:25
valiant wrote:
Additional to your one line change, we could use G2's translation functions to change the internal utf-8 data to any output encoding/charset!

? !!!
Please clarify...

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-07-28 13:38

most g2 content is fed through GalleryTranslater::translateDomain().
in this function, we could change the encoding type of the text to a non-utf8 type before returning the text.

but i just realized that this won't work. without further changes.

1. we'd have to run everything to a filter, not just the text that we translate. e.g. text that comes from the database has to be converted too, but only when outputting
2. translate is also used when inserting certain data into the database (e.g. the registered users group at install time).

i don't have time to work on it any time soon, just pointing out that there are some challenges and one had to make sure only to convert when outputting.

Login or register to post comments
cusper

Joined: 2005-10-04
Posts: 1
Posted: Tue, 2005-10-04 23:59

Hello Everybody!

I have just installed mambo component and faced the same problem. win1251 is my site's charset and gallery2 charset is utf-8. So I get messy content when displaying gallery2 (utf-8) under mambo (win1251).

Are there any results made in this direction?

ps. Gallery2 is great!

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-10-05 06:27

no, no progress / development in this direction.
but if someone has time, please submit a patch to switch the g2 output / input charset to anything possible.
also see:
http://gallery.menalto.com/node/24999#comment-137375

Login or register to post comments
yard

Joined: 2005-08-05
Posts: 6
Posted: Wed, 2005-10-26 10:15

Way to use Unicode charset everywhere is correct. It is good for multilanguages project. But, there is many old project with non-Unicode output.

I have the same problem with Joomla 1.0.3 (Mambo's child) and Gallery2 integration.

And...
joomla seems to go utf-8 also - http://developer.joomla.org/sf/scm/do/viewCommit/projects.joomla/scm.joomla/cmmt2778

Login or register to post comments
skorp

Joined: 2005-12-15
Posts: 4
Posted: Thu, 2005-12-15 03:59

I wrote this code for my webpage.
I have Gallery2 with UTF-8 and Mambo with ISO-8859-2 and I use com_gallery2 and mod_gallery2_image to show my albums.
You can change function utf82iso88592 to convert UTF-8 to different charset.

Rafal

AttachmentSize
galleryISO.php.txt2 KB
Login or register to post comments
skorp

Joined: 2005-12-15
Posts: 4
Posted: Thu, 2005-12-15 04:04

Sorry, remove ';' from

print utf82iso88592($g2moddata['bodyHtml'];);

it schould be

print utf82iso88592($g2moddata['bodyHtml']);

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-12-15 05:10

thanks for sharing.

but you'll face some problems:

- now you changed the html output from utf8 to iso8859-2, which is cool.
- but G2 still expects input to be in UTF-8 (all form data input), so you'd have to convert the GET/POST data too but the other way around, from iso8859-2 to utf8.

Login or register to post comments
skorp

Joined: 2005-12-15
Posts: 4
Posted: Thu, 2005-12-15 12:30

Yes, I understand that I can't edit/add data to my gallery using mambo component.
But I can display correct characters from my Gallery in Mambo :) and I can edit my albums,users etc. directly in my Gallery using UTF-8.

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-12-15 16:02

btw:
your idea could be a good workaround / simple solution for most embedded G2 users. just run the whole html output through a conversion function.
the input convertToUtf8 call could easily be added in GalleryUtilities::sanitize* or ::getRequestVariables

Login or register to post comments
skorp

Joined: 2005-12-15
Posts: 4
Posted: Thu, 2005-12-15 21:49

Thank You for advice. Now I am trying to convert user registration form to add new gallery users from mambo.
I have new functions (in attachment) and now I can convert data from and to utf-8.
but, I am not sure where, in class GalleryUtilities, could I convert data from iso to utf-8 ? Any suggestions?

AttachmentSize
convertCharset.php.txt2.01 KB
Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-12-15 22:19

GalleryCoreApi::convertFromUtf8
GalleryCoreApi::convertToUtf8

call it in GalleryUtilities::getRequestVariables

Login or register to post comments
webmaster33

Joined: 2006-11-22
Posts: 29
Posted: Wed, 2006-11-22 12:11

What to do in G2 v2.1.2 ?

These solutions are not working there :(

Login or register to post comments
jay_ar

Joined: 2006-05-31
Posts: 6
Posted: Tue, 2006-12-05 10:59

Hi, i come a time ago to a similar solution and faced bad bad problems - see my post half a year ago. My whole databasis was corrupted and i had to face a absolute new installation of g2.

i absolutly can't recommend that solution.

btw: Joomla is on the way to UTF-8 - so i will just sit this problem out...

Login or register to post comments
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2006-12-05 12:28

> What to do in G2 v2.1.2 ?

same thing.

@jay_ar:
yeah, looking forward to have everyone on utf8.
but if you cleanly convert between different charsets everywhere neccessary, it should work that way even if joomla isn't utf8.

here's some docs i added lately:
http://codex.gallery2.org/index.php/Gallery2:Embedding:Integration#Character_Set_Conversion

Login or register to post comments
webmaster33

Joined: 2006-11-22
Posts: 29
Posted: Tue, 2006-12-05 13:08
valiant wrote:
> What to do in G2 v2.1.2 ?

same thing.

@jay_ar:
yeah, looking forward to have everyone on utf8.
but if you cleanly convert between different charsets everywhere neccessary, it should work that way even if joomla isn't utf8.

here's some docs i added lately:
http://codex.gallery2.org/index.php/Gallery2:Embedding:Integration#Character_Set_Conversion

A more detailed instruction would be needed about how to add ISO-8859-2 support...
I was not able to do the modding based on the mentioned docs you created. :(

Login or register to post comments