Author Topic: Weird foreign letters problem...  (Read 600 times)

Offline 5Nick7

  • Single posting newcomer
  • *
  • Posts: 6
Weird foreign letters problem...
« on: November 21, 2018, 08:07:04 pm »
Hello,

I am a beginner with Code::Blocks and have recently ran into a problem which I can not seem to resolve. The actions that I do until the problem occurs goes something like this:
1. I open up Code::Blocks;
2. I write some code (using foreign Lithuanian letters "ą, č, ę...");
3. I save & exit;
4. Problem. After opening the .cpp file the Lithuanian letters change into other ones. (For example: 'ą' changes to ''; 'č' changes to ''; etc.).

Why is this? How do I fix it? I'm guessing the problem is in my code. When the problem occurs, the code goes from this:
Code: [Select]
#include <iostream>
#include <iomanip>
using namespace std;
int main ()
{
    setlocale(LC_ALL, "Lithuanian");
cout << ("Ąą Čč Ęę Ėė Įį Ųų Ūū ") << endl;
return 0;
}
to this:
Code: [Select]
#include <iostream>
#include <iomanip>
using namespace std;
int main ()
{
    setlocale(LC_ALL, "Lithuanian");
cout << (" ") << endl;
return 0;
}
The 'setlocale' is there so that the Lithuanian letters wouldn't turn into unknown (for me) symbols when i build and run the code, so I must also use system default file encoding.

I couldn't find anyone with a similar problem in the forums, sorry if this is post is purposeless.

Thank you.

Offline raynebc

  • Almost regular
  • **
  • Posts: 161
Re: Weird foreign letters problem...
« Reply #1 on: November 21, 2018, 09:04:21 pm »
Try saving your source files in Unicode (ie. UTF-8) format.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 2105
Re: Weird foreign letters problem...
« Reply #2 on: November 21, 2018, 09:59:09 pm »
Try saving your source files in Unicode (ie. UTF-8) format.
You can do this wiht Edit->File encoding->UTF-8

Offline 5Nick7

  • Single posting newcomer
  • *
  • Posts: 6
Re: Weird foreign letters problem...
« Reply #3 on: November 21, 2018, 10:40:50 pm »
Try saving your source files in Unicode (ie. UTF-8) format.
You can do this wiht Edit->File encoding->UTF-8

While this saves all the written code and everything looks great, when I run it, the console prints symbols instead of the normal letters:
From
Code: [Select]
Aą Čč Ęę Ėė Įį Ųų Ūū To
Code: [Select]
Ą ?? ?t -- Į
Any other ideas?

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 11279
    • Travis build status
Re: Weird foreign letters problem...
« Reply #4 on: November 21, 2018, 11:42:18 pm »
You have to tell the compiler that what is the encoding of the file you're give it.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 2105
Re: Weird foreign letters problem...
« Reply #5 on: November 22, 2018, 12:29:41 am »
next time use the search. 6 posts ago someone had the exact same problem as you, and there you would find this link: http://forums.codeblocks.org/index.php/topic,22168.msg150872.html#msg150872

Offline 5Nick7

  • Single posting newcomer
  • *
  • Posts: 6
Re: Weird foreign letters problem...
« Reply #6 on: November 22, 2018, 04:46:33 pm »
next time use the search. 6 posts ago someone had the exact same problem as you, and there you would find this link: http://forums.codeblocks.org/index.php/topic,22168.msg150872.html#msg150872

My problem is that the letters don't save, and change into other foreign letters when I close(, save) and reopen the .cpp file. I looked up the problem someone had ~6 posts ago, but their problem is foreign letters printing other symbols. Though I posted that it happens to me as well, it only happens when I set my file encoding to UTF-8, so as long as I don't do that, the only problem is the problem that is described in my first post.

While your fix provided in the link works, I can not use it, because I must use 'cout << "  " << endl;', 'cin >> x;', etc.

Offline 5Nick7

  • Single posting newcomer
  • *
  • Posts: 6
Re: Weird foreign letters problem...
« Reply #7 on: November 22, 2018, 04:52:20 pm »
You have to tell the compiler that what is the encoding of the file you're give it.

Sorry, I do not understand what you want me to do. If you want to know the compiler that I am using - I think it is called 'GNU GCC Compiler', but I do not know the difference in using other compilers or even what they do. Is the problem the compiler that I am using?

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 11279
    • Travis build status
Re: Weird foreign letters problem...
« Reply #8 on: November 22, 2018, 05:49:04 pm »
I guess writing message just after I've woken is not a good idea.

What I've meant in this post is that the compiler doesn't know the encoding of the source file (if it is different from the default one).
You have to pass an option which tells the compiler this information.
The exact option which has to be passed to your compiler can be found in the documentation of the compiler you're using.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline 5Nick7

  • Single posting newcomer
  • *
  • Posts: 6
Re: Weird foreign letters problem...
« Reply #9 on: November 22, 2018, 07:17:46 pm »
I guess writing message just after I've woken is not a good idea.

What I've meant in this post is that the compiler doesn't know the encoding of the source file (if it is different from the default one).
You have to pass an option which tells the compiler this information.
The exact option which has to be passed to your compiler can be found in the documentation of the compiler you're using.

Alright, now I understand. But also, I don't. Could you give me a link to some kind of tutorial on how to do this? Like I have said previously, I'm a beginner with Code::Blocks, not much of a 'computer-whiz' either. What you said just sounds like some top-class scientist conspiracy theories to me. :-[

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 11279
    • Travis build status
Re: Weird foreign letters problem...
« Reply #10 on: November 22, 2018, 08:21:02 pm »
It is not a conspiracy just complexity. Generally it is not a good idea to do what you want to do. At least in c++.

The link posted by bluehazzard looks good.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 2105
Re: Weird foreign letters problem...
« Reply #11 on: November 23, 2018, 09:59:02 am »
Quote
Quote
While your fix provided in the link works, I can not use it, because I must use 'cout << "  " << endl;', 'cin >> x;', etc.
From the linked forum thread:
Quote
[EDIT:] You can use std::cout with this method. If it is not working you have to update your compiler, or use utf8 literals that are supported by c++11

So to sum up, you have 3 problems:
1) The Letters you enter in codeblocks show as squares
2) The letters you enter in your code do not show correctly in the console window
3) You write letters in codeblocks, they seem fine, you save and after reopening the file, the letters are not the same as you saved?

Is this right?

All this are unicode problems that are not easy... Unicode is a hard problem... that is why i only use ascii, because on different systems and on different places in the world this is handled different. The problem is that at this time on normal programming no one has standardized one way to go. (On the web luckily we have UTF-8 as standard). I will not go into details about this, but it is not easy to solve....


1) This is an ecoding and font problem. You have to save the files with an encoding that support your letters (code points) and your font has to support this encoding and has to contain symbols for your code points (letters). On windows "Lucida Console" is a UTF-8 font with many supported code points, so i suggest you use this font for your code, or you download some from the google  font page https://www.google.com/get/noto/
2) This is also a problem with unicode and fonts, i think i have described in the other forum thread what you should do..
3) This is an encoding problem. At this point you have to decide what encoding you want to use. Your windows system encoding (some garbage utf 16 s***t) or the proper way to utf-8 glorious master race.
If you decide the utf-8 way set all your editor encodings to this:
Settings->Editor->General settings->Encode settings->Use encoding when opening files: utf-8
Make sure your file are utf-8 encoded: Open each file, look at the bottom in the status bar, if there is written "utf-8" or some garbage "windows-xx" encoding. If there is later go to Edit->File encoding->UTF-8

REMEMBER: You can not copy and paste things around like you want with unicode signs. If you have a text document encoded in some strange windows codepage and you copy and paste letters from there to an utf-8 encoded file i do not think that it would work. I do not know if codeblocks makes an automatic transformation between encodings in the copy and paste part....


I should really collect all this in one place... There are a lot unicode questions lately...

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 11279
    • Travis build status
Re: Weird foreign letters problem...
« Reply #12 on: November 23, 2018, 10:25:07 am »
I should really collect all this in one place... There are a lot unicode questions lately...
A wiki page would be great. It would be also useful if you test what happens with copy-pasting... Probably we should at least warn that something is wrong.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline 5Nick7

  • Single posting newcomer
  • *
  • Posts: 6
Re: Weird foreign letters problem...
« Reply #13 on: November 23, 2018, 08:00:43 pm »
So to sum up, you have 3 problems:
1) The Letters you enter in codeblocks show as squares
2) The letters you enter in your code do not show correctly in the console window
3) You write letters in codeblocks, they seem fine, you save and after reopening the file, the letters are not the same as you saved?

Is this right?

Correct, the 3) problem happens when I use the block of code in my very first post, nothing about 2) and 1). But, if I change my encoding to UTF-8, then problem 3) disappears, instead 2) and 1) come. Does this make sense?

Quote
Quote
While your fix provided in the link works, I can not use it, because I must use 'cout << "  " << endl;', 'cin >> x;', etc.
From the linked forum thread:
Quote
[EDIT:] You can use std::cout with this method. If it is not working you have to update your compiler, or use utf8 literals that are supported by c++11

Sorry for being so... what's the word? Demanding? But I need specifically 'cout', not 'std::cout' or 'wcout'...

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 2105
Re: Weird foreign letters problem...
« Reply #14 on: November 24, 2018, 11:56:19 pm »
Ok, i have retried this and was not able to make the cout part working in the normal console... Only in ConEMU it worked, but there only the cout part. Cin never worked.
And i think it will never be possible, with utf8. Windows made a big pile of shit here.... I read of some rumors that with the latest build of  windows 10 they introduced a true utf8 codepage, but i can not test it, because i do not have windows 10...

What can you do? If you want to use unciode i am not sure. You can probably try to use the codepage of your language. I do not know if this 128 characters are enough for your language. If not you will need wcout and a lot of other trash...
If you want to use your codepage, all files have to be the same encoding. And also the console has to use this encoding, you have to set it with the system("chcp 65001  > nul");  call.
You will have to try it.... The most important thing is to keep in sync all encoding settings, from the file in codeblocks, to the console font and encoding setting...

Would be nice if you could report back if you succeed...