Author Topic: The 30 December 2018 build (11543) is out.  (Read 46303 times)

Offline UberNewb2

  • Multiple posting newcomer
  • *
  • Posts: 11
Re: The 30 December 2018 build (11543) is out.
« Reply #45 on: January 11, 2019, 05:27:39 am »
Can't say this is related to CodeBlocks but I was running 11543 at the time, and this is a pretty serious error and problem.

My Windows 10 pc running my project Blue Screened right when I ran my project's executable from within CodeBlocks using Build->Run. It started up and immediately died. Now I can't tell whether my program caused the blue screen, or Code Blocks, or some other process, but I lost a file in the process. It was open in the Code Blocks editor, and after Windows recovered from the blue screen and restarted, the file was empty. I opened it with a hex editor and discovered it was a bunch of zeros. The entire file contents were wiped out. And I probably had several hundred lines of code that were three days old since the last commit.

I am in the process of recovery right now. I am going to attempt to disassemble the .o file and get the source back that way. If that fails, I don't know what else to try. I can't attach any  kind of crash report because there wasn't one. I will attempt to find the memory dump created by the crash recovery thing a ma jiggie and see what I can dredge up.

Wish me luck.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 30 December 2018 build (11543) is out.
« Reply #46 on: January 11, 2019, 08:23:27 am »
Can't say this is related to CodeBlocks but I was running 11543 at the time, and this is a pretty serious error and problem.
No, it is not codeblocks.

My Windows 10 pc running my project Blue Screened right when I ran my project's executable from within CodeBlocks using Build->Run.
You have hardware or OS/driver problem! User software cannot cause a BSOD on a well working computer! I doubt you're using Windows 98.

..., but I lost a file in the process. It was open in the Code Blocks editor, and after Windows recovered from the blue screen and restarted, the file was empty....

For these cases we use backup solutions. In the software development world these are called version control systems. The one which won the battle is called git. Start using it, even for personal projects and you won't have problems like these in the future.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: The 30 December 2018 build (11543) is out.
« Reply #47 on: January 11, 2019, 12:39:51 pm »
Quote
It was open in the Code Blocks editor, and after Windows recovered from the blue screen and restarted, the file was empty. I opened it with a hex editor and discovered it was a bunch of zeros. The entire file contents were wiped out.
Also codeblocks uses some special save procedure to prevent exactly this. It saves the file first in a temporary file, copies the temporary file over the old file and then removes the temp file after the copy process was successful. So there is always a valid file with your content on the system if something bad is happening

Offline Miguel Gimenez

  • Developer
  • Lives here!
  • *****
  • Posts: 1553
Re: The 30 December 2018 build (11543) is out.
« Reply #48 on: January 11, 2019, 01:04:52 pm »
An useful feature can be warn the user if one of those .temp files is found when loading the project or, if a file fails to load, look for an associated  .temp file and restore it.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 30 December 2018 build (11543) is out.
« Reply #49 on: January 11, 2019, 07:14:54 pm »
Doing the save-to-temp-and-move prevents problems only if cb crashes. If the os crashes the file could be in the write caches of the os and it could be lost. In theory the journal should handle such cases, but there is always a chance of total failure.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline UberNewb2

  • Multiple posting newcomer
  • *
  • Posts: 11
Re: The 30 December 2018 build (11543) is out.
« Reply #50 on: January 11, 2019, 11:29:31 pm »
..., but I lost a file in the process. It was open in the Code Blocks editor, and after Windows recovered from the blue screen and restarted, the file was empty....

For these cases we use backup solutions. In the software development world these are called version control systems. The one which won the battle is called git. Start using it, even for personal projects and you won't have problems like these in the future.

If you're going to chastise me, at least read what I said.
And I probably had several hundred lines of code that were three days old since the last commit.

Quote
It was open in the Code Blocks editor, and after Windows recovered from the blue screen and restarted, the file was empty. I opened it with a hex editor and discovered it was a bunch of zeros. The entire file contents were wiped out.
Also codeblocks uses some special save procedure to prevent exactly this. It saves the file first in a temporary file, copies the temporary file over the old file and then removes the temp file after the copy process was successful. So there is always a valid file with your content on the system if something bad is happening
The file had already been saved I believe, as I was in the process of running the project right after building it.

Something happened during the crash that overwrote the file.

However, if you say CB creates temporary files, perhaps I could recover one of those. Are they stored in the same directory as the source file in question?

Doing the save-to-temp-and-move prevents problems only if cb crashes. If the os crashes the file could be in the write caches of the os and it could be lost. In theory the journal should handle such cases, but there is always a chance of total failure.
I read that the change journal of NTFS is not able to recover old versions of the file.

The problem is not that the file was ever deleted, it was physically overwritten during the blue screen.

Like I said, my latest commit for that file is like three days old, and I lost a lot of changes since then.



Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 30 December 2018 build (11543) is out.
« Reply #51 on: January 12, 2019, 12:16:11 am »
The problem is not that the file was ever deleted, it was physically overwritten during the blue screen.
There are multiple levels of caches, both software and hardware. If the file hasn't been flushed fully to the storage media, there is nothing that would prevent a corruption.

C::B deletes the original with a move operation. So after a successful save there is no trace of the temporary file.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline UberNewb2

  • Multiple posting newcomer
  • *
  • Posts: 11
Re: The 30 December 2018 build (11543) is out.
« Reply #52 on: January 12, 2019, 12:56:59 am »
The problem is not that the file was ever deleted, it was physically overwritten during the blue screen.
There are multiple levels of caches, both software and hardware. If the file hasn't been flushed fully to the storage media, there is nothing that would prevent a corruption.

C::B deletes the original with a move operation. So after a successful save there is no trace of the temporary file.
Well I'm screwed then. :/ No temporary file to undelete means no back up copy. Dang. I'd honestly prefer you litter my source directory with temporary deleted files then make it as if they never existed. :/


Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 30 December 2018 build (11543) is out.
« Reply #53 on: January 12, 2019, 11:44:32 am »
Well I'm screwed then. :/ No temporary file to undelete means no back up copy. Dang. I'd honestly prefer you litter my source directory with temporary deleted files then make it as if they never existed. :/
Sorry, we're not making a backup software. For such purposes there are better alternatives (like using git and commit every small change you make every hour or two; this have other benefits which a backup solution won't provide).

You can look at the auto save plugin, it has such features, but very limited.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline UberNewb2

  • Multiple posting newcomer
  • *
  • Posts: 11
Re: The 30 December 2018 build (11543) is out.
« Reply #54 on: January 12, 2019, 06:33:23 pm »
Well the autosave plugin might be what I'm looking for, but I can't tell what it does or where it saves things. The wiki page ;

http://wiki.codeblocks.org/index.php/Autosave_plugin

doesn't say much...

I can tell from the settings in the environment settings that I can make a .save file every X minutes, which works.

That would at least save my butt in the case of another BSOD.

And yes, I'll try to commit more often.


Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 30 December 2018 build (11543) is out.
« Reply #55 on: January 21, 2019, 09:18:58 pm »
Doesn't seem like working 100% reliably:

Code
=================================================================
==27797==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6100011229fc at pc 0x7fffe061e899 bp 0x7fffccd7f070 sp 0x7fffccd7f060
READ of size 4 at 0x6100011229fc thread T11
    #0 0x7fffe061e898 in Tokenizer::KMP_Find(wchar_t const*, int, wchar_t const*, int) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/tokenizer.cpp:1716
    #1 0x7fffe061f5ee in Tokenizer::GetFirstTokenPosition(wchar_t const*, unsigned long, wchar_t const*, unsigned long) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/tokenizer.cpp:1929
    #2 0x7fffe061ef1d in Tokenizer::GetMacroExpandedText(Token const*, wxString&) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/tokenizer.cpp:1843
    #3 0x7fffe061e2a6 in Tokenizer::ReplaceMacroUsage(Token const*) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/tokenizer.cpp:1665
    #4 0x7fffe061aeb4 in Tokenizer::CheckMacroUsageAndReplace() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/tokenizer.cpp:1089
    #5 0x7fffe061a404 in Tokenizer::DoGetToken() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/tokenizer.cpp:966
    #6 0x7fffe0619c55 in Tokenizer::PeekToken() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/tokenizer.cpp:904
    #7 0x7fffe05eb56a in ParserThread::HandleFunction(wxString&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:2463
    #8 0x7fffe05e3630 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:1188
    #9 0x7fffe05e93ce in ParserThread::HandleClass(ParserThread::EClassType) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:2112
    #10 0x7fffe05e25c6 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:882
    #11 0x7fffe05e0961 in ParserThread::Parse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:533
    #12 0x7fffe05c369c in Parser::Parse(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:345
    #13 0x7fffe05c8a45 in Parser::ParseFile(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:670
    #14 0x7fffe05e6f08 in ParserThread::HandleIncludes() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:1738
    #15 0x7fffe05e14e2 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:653
    #16 0x7fffe05e0961 in ParserThread::Parse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:533
    #17 0x7fffe05c369c in Parser::Parse(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:345
    #18 0x7fffe05c8a45 in Parser::ParseFile(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:670
    #19 0x7fffe05e6f08 in ParserThread::HandleIncludes() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:1738
    #20 0x7fffe05e14e2 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:653
    #21 0x7fffe05e0961 in ParserThread::Parse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:533
    #22 0x7fffe05c369c in Parser::Parse(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:345
    #23 0x7fffe05c8a45 in Parser::ParseFile(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:670
    #24 0x7fffe05e6f08 in ParserThread::HandleIncludes() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:1738
    #25 0x7fffe05e14e2 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:653
    #26 0x7fffe05e0961 in ParserThread::Parse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:533
    #27 0x7fffe05c369c in Parser::Parse(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:345
    #28 0x7fffe05c8a45 in Parser::ParseFile(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:670
    #29 0x7fffe05e6f08 in ParserThread::HandleIncludes() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:1738
    #30 0x7fffe05e14e2 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:653
    #31 0x7fffe05e0961 in ParserThread::Parse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:533
    #32 0x7fffe05c369c in Parser::Parse(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:345
    #33 0x7fffe05c8a45 in Parser::ParseFile(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:670
    #34 0x7fffe05e6f08 in ParserThread::HandleIncludes() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:1738
    #35 0x7fffe05e14e2 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:653
    #36 0x7fffe05e0961 in ParserThread::Parse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:533
    #37 0x7fffe05c369c in Parser::Parse(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:345
    #38 0x7fffe05c8a45 in Parser::ParseFile(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:670
    #39 0x7fffe05e6f08 in ParserThread::HandleIncludes() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:1738
    #40 0x7fffe05e14e2 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:653
    #41 0x7fffe05e0961 in ParserThread::Parse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:533
    #42 0x7fffe05c369c in Parser::Parse(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:345
    #43 0x7fffe05c8a45 in Parser::ParseFile(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:670
    #44 0x7fffe05e6f08 in ParserThread::HandleIncludes() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:1738
    #45 0x7fffe05e14e2 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:653
    #46 0x7fffe05e0961 in ParserThread::Parse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:533
    #47 0x7fffe05c369c in Parser::Parse(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:345
    #48 0x7fffe05c8a45 in Parser::ParseFile(wxString const&, bool, bool) /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parser.cpp:670
    #49 0x7fffe05e6f08 in ParserThread::HandleIncludes() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:1738
    #50 0x7fffe05e14e2 in ParserThread::DoParse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:653
    #51 0x7fffe05e0961 in ParserThread::Parse() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.cpp:533
    #52 0x7fffe05f595c in ParserThread::Execute() /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/parserthread.h:192
    #53 0x7ffff1aa05cf  (libcodeblocks.so+0x6b35cf)
    #54 0x7ffff2978642 in wxThread::CallEntry() ../git/include/wx/thrimpl.cpp:356
    #55 0x7ffff2982d55 in wxThreadInternal::PthreadStart(wxThread*) ../git/src/unix/threadpsx.cpp:869
    #56 0x7ffff29831d1 in wxPthreadStart ../git/src/unix/threadpsx.cpp:819
    #57 0x7ffff083215b  (/lib64/libpthread.so.0+0x815b)
    #58 0x7ffff05650ae in clone (/lib64/libc.so.6+0x1040ae)

0x6100011229fc is located 0 bytes to the right of 188-byte region [0x610001122940,0x6100011229fc)
allocated by thread T11 here:
    #0 0x7ffff6efde60 in operator new(unsigned long) /var/tmp/portage/sys-devel/gcc-6.4.0-r5/work/gcc-6.4.0/libsanitizer/asan/asan_new_delete.cc:60
    #1 0x7ffff114e3ae in std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::_M_assign(std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const&) /var/tmp/portage/sys-devel/gcc-8.2.0-r6/work/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:265

Thread T11 created by T0 here:
    #0 0x7ffff6e67448 in __interceptor_pthread_create /var/tmp/portage/sys-devel/gcc-6.4.0-r5/work/gcc-6.4.0/libsanitizer/asan/asan_interceptors.cc:236
    #1 0x7ffff297d739 in wxThreadInternal::Create(wxThread*, unsigned int) ../git/src/unix/threadpsx.cpp:1121

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/obfuscated/projects/codeblocks/git/src/plugins/codecompletion/parser/tokenizer.cpp:1716 in Tokenizer::KMP_Find(wchar_t const*, int, wchar_t const*, int)
Shadow bytes around the buggy address:
  0x0c208021c4e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c208021c4f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c208021c500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c208021c510: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c208021c520: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
=>0x0c208021c530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04]
  0x0c208021c540: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c208021c550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04
  0x0c208021c560: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c208021c570: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c208021c580: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==27797==ABORTING

This from a minimlal wxwidgets type of project...
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline Miguel Gimenez

  • Developer
  • Lives here!
  • *****
  • Posts: 1553
Re: The 30 December 2018 build (11543) is out.
« Reply #56 on: January 22, 2019, 10:32:30 am »
Is this report from revision 11543? KMP-related routines were fixed in revisions 11544 to 11548.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5910
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: The 30 December 2018 build (11543) is out.
« Reply #57 on: January 22, 2019, 04:17:53 pm »
Is this report from revision 11543? KMP-related routines were fixed in revisions 11544 to 11548.

Hi, OBF, as Miguel Gimenez said, it should be fixed in trunk already. Are you using an old svn revision?
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: The 30 December 2018 build (11543) is out.
« Reply #58 on: January 22, 2019, 07:34:35 pm »
I am at 11552...  8) It fails parsing some unichar.h file from wx 3.0.4...
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline Miguel Gimenez

  • Developer
  • Lives here!
  • *****
  • Posts: 1553
Re: The 30 December 2018 build (11543) is out.
« Reply #59 on: January 22, 2019, 08:50:52 pm »
Got it, revision 11548 corrected the issue in all lines but one. Patch is attached.