TinyXml with Unicode thread

[thomas]

What we use xml for, DOM is the one and only good thing. SAX would indeed make my life a lot less enjoyable. Whether it uses a little more or less memory really does not matter - the data needs to be stored anyway, and the extra overhead per node is 28 bytes on a 32 bit machine, not so much really

A config file has typically 1200-1500 nodes, so we're talking about 30-50 kilobytes.

[Michael]

A config file has typically 1200-1500 nodes, so we're talking about 30-50 kilobytes.

Yes, that is not too much. And also it should not be an issue if the config file would grove a bit in the future.

Michael

[takeshimiya]

Back on topic: Unicode support.

@280Z28: I've applied your patch and it's working a lot better.

Here are the things that works (tried with Japanese):
-Typing in the cbEditor Unicode chars.
-Copying and pasting Unicode chars between cbEditors.
-Copying Unicode chars between cbEditor and external editors.
-Dragging Unicode chars between cbEditors.
-Dragging Unicode chars between cbEditor and external editors.
-Saving a file that contain the recently typed Unicode chars (It's saved in UTF-8 without BOM).

Here are the things that doesn't works: :cry:
-Loading a file that contain Unicode chars. It loads ok, but it's displayed as it were ASCII, not Unicode, so the Unicode chars becomes garbage.
-Pasting Unicode chars from an external editor (Notepad) to cbEditor. Somehow, it pastes the word "Hello". :shock:
-Doesn't haves any notion of what a BOM is.
-Other Unicode encodings such as UTF-16 doesn't works. Only UTF-8 works.

[280Z28]

Back on topic: Unicode support.

@280Z28: I've applied your patch and it's working a lot better.

Here are the things that works (tried with Japanese):
-Typing in the cbEditor Unicode chars.
-Copying and pasting Unicode chars between cbEditors.
-Copying Unicode chars between cbEditor and external editors.
-Dragging Unicode chars between cbEditors.
-Dragging Unicode chars between cbEditor and external editors.
-Saving a file that contain the recently typed Unicode chars (It's saved in UTF-8 without BOM).

Here are the things that doesn't works: :cry:
-Loading a file that contain Unicode chars. It loads ok, but it's displayed as it were ASCII, not Unicode, so the Unicode chars becomes garbage.
-Pasting Unicode chars from an external editor (Notepad) to cbEditor. Somehow, it pastes the word "Hello". :shock:
-Doesn't haves any notion of what a BOM is.
-Other Unicode encodings such as UTF-16 doesn't works. Only UTF-8 works.

:lol: :lol: :lol: I had "Hello" hard coded in the version you downloaded. That's one of the things I fixed for when I re-posted a patch. There is a problem with pasting data from other editors that shows up on my computer here at work but not at home and I was trying to see if a certain section of code was being executed.

The BOM stuff, etc. is going to be a nightmare.

In the end the user MUST (using ricks terms ) have a way to say "this file opened in the wrong encoding. reopen it in _____."

Files MUST be saved in the same encoding that they were opened with.

The BOM MUST be preserved if it was present when the file was opened.

[takeshimiya]

Yes, just like SciTE does this: it let you override (once the file is opened) the encoding (ASCII, UTF-8, UTF-16LE, UTF-16BE).

If the file haves BOM, it's easy, we read that, and when the file is going to be saved we check the actual Encoding.

It only will requiere UTF-8 to UTF-16, type of conversions, which I think wxWidgets provides, but if not, they're only small functions.

The only difficult case will be when we load a file that doesn't have BOM, but the most simple solution would be to assume either that it's UTF-8, or ASCII with the current locale.

[takeshimiya]

The link http://www.280z28.org/CodeBlocks/patches/complete-rev1565u.patch does not works.

Anyways, there is a problem with your patches, you provide some here but not all on the sf.net tracker?

I'll want to see instead is more small patches, posted and updated in sf.net

[280Z28]

The link http://www.280z28.org/CodeBlocks/patches/complete-rev1565u.patch does not works.

Anyways, there is a problem with your patches, you provide some here but not all on the sf.net tracker?

I'll want to see instead is more small patches, posted and updated in sf.net

I don't have what I consider a "workable solution to the problem." Only once I'm sure they work, I post them at SF.

I have posted quite a few of the Unicode fixes at SF already.

By the way, CodeBlocks\src\sdk\wxscintilla\src\scintilla\src\Editor.cxx is ASCII. On top of that, there is a character at line 1195 that prevents Code::Blocks from opening it (it opens as blank) if UTF8 is specified. As a workaround, I changed lines 364, 390, and 405 of codeblocks\src\sdk\globals.cpp from wxConvUTF8 to wxConvLibc. I have no idea if that was a good idea (so I haven't posted a patch), but it worked on my system so I left it.

Edit: I make no guarantees concerning my multi-patches I post on here. :lol:

[takeshimiya]

No, I wasn't saying your patches didn't work.

The link http://www.280z28.org/CodeBlocks/patches/complete-rev1565u.patch  still doesn't works.

[280Z28]

No, I wasn't saying your patches didn't work.

The link http://www.280z28.org/CodeBlocks/patches/complete-rev1565u.patch  still doesn't works.

:oops: Oh well 1565's history anyway :lol: