Author Topic: Clang CC  (Read 260272 times)

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13406
    • Travis build status
Re: Clang CC
« Reply #225 on: March 07, 2017, 09:54:06 pm »
Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety.
How are you doing the deep-copies? I'm thinking of adding something like cbDeepCopyWxString that is reliable for both wx2.8 and wx3.0.
Also switching to std::string is not really a good idea - you'll do many conversions and possibly loose unicode support (you'll have to use wstring for that).
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3352
Re: Clang CC
« Reply #226 on: March 07, 2017, 11:58:19 pm »
A other problem i noted quite frequently is if you open a new cpp file, enter code at the end of the file (no line text after the last line code) and this code has errors, so they get reported by the instant error reporting wxScilite will report some error about writing past the buffer, or writing non printable characters or something like this. I am actively investigating in this if i find some time.
This is no crash, but a assert()

I don't recall how i can reproduce this, but i will report as soon i am able to reproduce it...
Sounds familiar, I fixed something similar in the past, but it was only reproducible on Windows. Sadly I don't test it often on Windows. It's hard enough as it is to test on wx28, wx30, ubuntu, centos, ...

Yves

This seems to fix the error:

Code
diff --git a/src/clangrefactoring.cpp b/src/clangrefactoring.cpp
index b3913ac..f88b76d 100644
--- a/src/clangrefactoring.cpp
+++ b/src/clangrefactoring.cpp
@@ -217,8 +217,8 @@ void ClangRefactoring::BeginHighlightOccurrences(cbEditor* ed)
     // Set Styling:
     // clear all style indications set in a previous run (is also done once after text gets unselected)
     stc->IndicatorClearRange(0, stc->GetLength());
-
-    if (stc->GetTextRange(pos - 1, pos + 1).Strip().IsEmpty())
+    int max_length =  std::min(pos + 1 ,stc->GetLength());
+    if (stc->GetTextRange(pos - 1, max_length).Strip().IsEmpty())
         return;

     const int line = stc->LineFromPosition(pos);
i honestly don't know if it changes the behavior of the function, but at least it does not give this annoying message ;) . The decision if this is the right fix i let for the developer...

Offline yvesdm3000

  • Almost regular
  • **
  • Posts: 225
Re: Clang CC
« Reply #227 on: March 08, 2017, 06:47:48 am »
Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety.
How are you doing the deep-copies? I'm thinking of adding something like cbDeepCopyWxString that is reliable for both wx2.8 and wx3.0.
Also switching to std::string is not really a good idea - you'll do many conversions and possibly loose unicode support (you'll have to use wstring for that).

I do a c_str() on the source wxString when the operation or event is copied. I also avoided using wxCommandEvent and built a ClangEvent object that does this deep copy of every member.

Yves
Clang based code completion for Code::Blocks:   http://github.com/yvesdm3000/ClangLib

Offline yvesdm3000

  • Almost regular
  • **
  • Posts: 225
Re: Clang CC
« Reply #228 on: March 08, 2017, 07:21:07 am »
A other problem i noted quite frequently is if you open a new cpp file, enter code at the end of the file (no line text after the last line code) and this code has errors, so they get reported by the instant error reporting wxScilite will report some error about writing past the buffer, or writing non printable characters or something like this. I am actively investigating in this if i find some time.
This is no crash, but a assert()

I don't recall how i can reproduce this, but i will report as soon i am able to reproduce it...
Sounds familiar, I fixed something similar in the past, but it was only reproducible on Windows. Sadly I don't test it often on Windows. It's hard enough as it is to test on wx28, wx30, ubuntu, centos, ...

Yves

This seems to fix the error:

Code
diff --git a/src/clangrefactoring.cpp b/src/clangrefactoring.cpp
index b3913ac..f88b76d 100644
--- a/src/clangrefactoring.cpp
+++ b/src/clangrefactoring.cpp
@@ -217,8 +217,8 @@ void ClangRefactoring::BeginHighlightOccurrences(cbEditor* ed)
     // Set Styling:
     // clear all style indications set in a previous run (is also done once after text gets unselected)
     stc->IndicatorClearRange(0, stc->GetLength());
-
-    if (stc->GetTextRange(pos - 1, pos + 1).Strip().IsEmpty())
+    int max_length =  std::min(pos + 1 ,stc->GetLength());
+    if (stc->GetTextRange(pos - 1, max_length).Strip().IsEmpty())
         return;

     const int line = stc->LineFromPosition(pos);
i honestly don't know if it changes the behavior of the function, but at least it does not give this annoying message ;) . The decision if this is the right fix i let for the developer...

I applied the fix. Thanks !

Yves
Clang based code completion for Code::Blocks:   http://github.com/yvesdm3000/ClangLib

Offline yvesdm3000

  • Almost regular
  • **
  • Posts: 225
Re: Clang CC
« Reply #229 on: March 08, 2017, 08:14:48 am »
Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety.
How are you doing the deep-copies? I'm thinking of adding something like cbDeepCopyWxString that is reliable for both wx2.8 and wx3.0.
Also switching to std::string is not really a good idea - you'll do many conversions and possibly loose unicode support (you'll have to use wstring for that).

I want to use std::string because Clang uses utf8, otherwise i'm stuck with a conversion of 16 bits into 8 bits (for most of the cases). I know std::string isn't ideal for utf8 but when you know the limitations, it's doable since I don't process the strings all that much.

Yves
Clang based code completion for Code::Blocks:   http://github.com/yvesdm3000/ClangLib

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6034
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang CC
« Reply #230 on: March 11, 2017, 09:40:44 am »
In the file "include\sdk_events.h", there is a class named "CodeBlocksThreadEvent" which I introduced years ago to implement deep copy of it's wxString member. It is for wx 2.8.12, since wx 3.x does not such issue, and this class is used for thread safety in some plugins.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13406
    • Travis build status
Re: Clang CC
« Reply #231 on: March 11, 2017, 10:25:38 am »
In wx3.0 in most implementations (gcc until 5.x) the string is still reference counted, but it is protected by an atomic. So deep copying is useful there, too.

Also in wx3.0 they've added a clone method: http://docs.wxwidgets.org/3.1.0/classwx_string.html#afe63f53ecaa197333c405ca985f733fe wxString::Clone.
« Last Edit: March 11, 2017, 10:27:11 am by oBFusCATed »
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6034
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Clang CC
« Reply #232 on: March 11, 2017, 02:46:09 pm »
In wx3.0 in most implementations (gcc until 5.x) the string is still reference counted, but it is protected by an atomic. So deep copying is useful there, too.

Also in wx3.0 they've added a clone method: http://docs.wxwidgets.org/3.1.0/classwx_string.html#afe63f53ecaa197333c405ca985f733fe wxString::Clone.
If I remember correctly, if the wxString(std::string) is atomic reference counted, it is theadsafe. I mean the wxCommandEvent should be threadsafe. What's what the atomic reference counter used for, in "copy on write" mode.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline yvesdm3000

  • Almost regular
  • **
  • Posts: 225
Re: Clang CC
« Reply #233 on: March 18, 2017, 03:18:46 pm »
I pushed a new feature called "Call hierarchy" that allows you to lookup call references and where a specific method/function is being called. And then each function that referenced a function can simply be looked up again in a new level in the tree. It's very usefull for understanding large codebases and is more accurate than the thread-search plugin.

There are still a lot of limitations that will be solved in the next updates:
- Doesn't list overrides
- Doesn't list callers that are calling a pure or not function that the selected call overrides
- Doesn't work on anything else than functions/methods. Member variables, global variables, class instances all come to mind tho be useful for this feature.
- List to the right should become optional

And obviously small bugs since it's all new code that will be solved every time I come across one (or someone indicates one).

As always open to feedback, suggestions or other excellent ideas.

Yves
Clang based code completion for Code::Blocks:   http://github.com/yvesdm3000/ClangLib

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3352
Re: Clang CC
« Reply #234 on: March 18, 2017, 09:42:27 pm »
Hi!
good work!

i tested your plugin on linux and have an assert from wx:

Code
ASSERT INFO:
/usr/include/wx-3.0/wx/datetime.h(876): assert "IsValid() && dt.IsValid()" failed in operator!=(): invalid wxDateTime

BACKTRACE:
[1] ClangToolbar::OnEditorHook(cbEditor*, wxScintillaEvent&)
[2] EditorHooks::CallHooks(cbEditor*, wxScintillaEvent&)
[3] cbEditor::OnScintillaEvent(wxScintillaEvent&)
[4] cbEditor::OnEditorUpdateUI(wxScintillaEvent&)
[5] wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const
[6] wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)
[7] wxEvtHandler::SearchDynamicEventTable(wxEvent&)
[8] wxEvtHandler::TryHereOnly(wxEvent&)
[9] wxEvtHandler::ProcessEventLocally(wxEvent&)
[10] wxEvtHandler::ProcessEvent(wxEvent&)
[11] wxWindowBase::TryAfter(wxEvent&)
[12] wxScintilla::NotifyParent(SCNotification*)
[13] ScintillaWX::NotifyParent(SCNotification)
[14] Editor::NotifyUpdateUI()
[15] Editor::Paint(Surface*, PRectangle)
[16] ScintillaWX::DoPaint(wxDC*, wxRect)
[17] wxScintilla::OnPaint(wxPaintEvent&)
[18] wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const
[19] wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)
[20] wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*)
[21] wxEvtHandler::TryHereOnly(wxEvent&)
[22] wxEvtHandler::DoTryChain(wxEvent&)
[23] wxEvtHandler::ProcessEvent(wxEvent&)
[24] wxEvtHandler::SafelyProcessEvent(wxEvent&)
[25] wxWindow::GTKSendPaintEvents(_GdkRegion const*)
[26] g_closure_invoke
[27] g_signal_emit_valist
[28] g_signal_emit
[29] gtk_main_do_event
[30] gdk_window_process_all_updates
[31] g_main_context_dispatch
[32] g_main_context_iteration
[33] gtk_main_iteration
[34] wxWindow::DoPopupMenu(wxMenu*, int, int)
[35] wxWindowBase::PopupMenu(wxMenu*, int, int)
[36] wxWindowBase::PopupMenu(wxMenu*, wxPoint const&) /usr/include/wx-3.0/wx/window.h:1216
[37] EditorBase::DisplayContextMenu(wxPoint const&, ModuleType)
[38] cbStyledTextCtrl::OnContextMenu(wxContextMenuEvent&)
[39] wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const
[40] wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)
[41] wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*)
[42] wxEvtHandler::TryHereOnly(wxEvent&)
[43] wxEvtHandler::DoTryChain(wxEvent&)
[44] wxEvtHandler::ProcessEvent(wxEvent&)
[45] wxEvtHandler::SafelyProcessEvent(wxEvent&)
[46] g_closure_invoke
[47] g_signal_emit_valist
[48] g_signal_emit
[49] gtk_propagate_event
[50] gtk_main_do_event
[51] g_main_context_dispatch
[52] g_main_loop_run
[53] gtk_main
[54] wxGUIEventLoop::DoRun()
[55] wxEventLoopBase::Run()
[56] wxAppConsoleBase::MainLoop()
[57] CodeBlocksApp::OnRun() codeblocks_sf/src/src/app.cpp:850
[58] wxEntry(int&, wchar_t**)
[59] CodeBlocks codeblocks_sf/src/src/app.cpp:322
[60] __libc_start_main
[61] _start

The source is from here:
clangtoolbar.cpp:196
Code
if ( (ed->GetLastModificationTime() != m_CurrentState.m_CurrentEditorModificationTime)||(m_Function&&(m_Function->GetCount()==0)))

it seems that m_CurrentState.m_CurrentEditorModificationTime is not initialized by inspecting with the debugger...

Also i can't test your new function? How do i call it? In the right click menu i can't find a entry?
[edit:] Found it... Somehow the menu entry was not present, but after a restart all worked

greetings
« Last Edit: March 18, 2017, 09:45:54 pm by BlueHazzard »

Offline yvesdm3000

  • Almost regular
  • **
  • Posts: 225
Re: Clang CC
« Reply #235 on: March 19, 2017, 04:40:11 am »
CurrentState.m_CurrentEditorModificationTime  initialization should now be fixed

Yves
Clang based code completion for Code::Blocks:   http://github.com/yvesdm3000/ClangLib

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3352
Re: Clang CC
« Reply #236 on: April 14, 2017, 10:25:08 pm »
How is the update algorithm of the cc? If i open the codeblocks project file one of my processors is at 100%  and the console outputs
Code
Reindex on tokenindexdb 0x3831db0 project=codeblocks_sf/src/CodeBlocks_wx30-unix.cbp /codeblocks_sf/src/include/configurationpanel.h
ClTranslationUnit::Parse id=127
[Thread 0x7fffb6732700 (LWP 20185) exited]
ClTranslationUnit::UpdateTokenDatabase 127 finished: 74779 tokens processed

could this be a recursive loop?
Does the parser indexes all files all the time, or does it stop after all project files are parsed?

thank you for your work!

greetings

Offline yvesdm3000

  • Almost regular
  • **
  • Posts: 225
Re: Clang CC
« Reply #237 on: April 15, 2017, 12:08:46 pm »
How is the update algorithm of the cc? If i open the codeblocks project file one of my processors is at 100%  and the console outputs
Code
Reindex on tokenindexdb 0x3831db0 project=codeblocks_sf/src/CodeBlocks_wx30-unix.cbp /codeblocks_sf/src/include/configurationpanel.h
ClTranslationUnit::Parse id=127
[Thread 0x7fffb6732700 (LWP 20185) exited]
ClTranslationUnit::UpdateTokenDatabase 127 finished: 74779 tokens processed

could this be a recursive loop?
Does the parser indexes all files all the time, or does it stop after all project files are parsed?

thank you for your work!

greetings

The indexer checks if the timestamp of a file is more recent than the timestamp in the database. If it does, it reindexes and updates the timestamp in the database. Someday I want to extend this with the filesize just in case a file has been reverted (a checksum is probably overkill) .

I actually fixed an issue on this in commit d0c4f433b0b1e619f7fd14794c05d84379db7c1b on 21 of march for an issue where it continuously tried to reindex the same file. I don't know if your build has this fix yet?

Yves

Clang based code completion for Code::Blocks:   http://github.com/yvesdm3000/ClangLib

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3352
Re: Clang CC
« Reply #238 on: August 01, 2017, 02:48:40 pm »
Code
#0 0xa6ed440	__cxa_throw() ( output\share\codeblocks\plugins\clanglib.dll:??)
#1 0xa6ec266 operator new(unsigned int) () ( output\share\codeblocks\plugins\clanglib.dll:??)
#2 0xa7bae18 typeinfo for std::time_put<wchar_t, std::ostreambuf_iterator<wchar_t, std::char_traits<wchar_t> > >() ( output\share\codeblocks\plugins\clanglib.dll:??)
#3 0xa6f0200 std::bad_alloc::what() const() ( output\share\codeblocks\plugins\clanglib.dll:??)
#4 0xa7784c9 std::allocator_traits<std::allocator<ClAbstractToken> >::max_size(__a=...) (mingw32/4.9.2/include/c++/bits/alloc_traits.h:422)
#5 0xa7784a8 std::allocator_traits<std::allocator<ClAbstractToken> >::allocate(__a=..., __n=131072) (mingw32/4.9.2/include/c++/bits/alloc_traits.h:357)
#6 0xa762a99 std::_Vector_base<ClAbstractToken, std::allocator<ClAbstractToken> >::_M_allocate(this=0x55a1d0bc, __n=131072) ( mingw32/4.9.2/include/c++/bits/stl_vector.h:170)
#7 0xa786642 std::vector<ClAbstractToken, std::allocator<ClAbstractToken> >::_M_emplace_back_aux<ClAbstractToken const&>(this=0x55a1d0bc) ( mingw32/4.9.2/include/c++/bits/vector.tcc:412)
#8 0xa786a06 std::vector<ClAbstractToken, std::allocator<ClAbstractToken> >::push_back(this=0x55a1d0bc, __x=...) ( mingw32/4.9.2/include/c++/bits/stl_vector.h:923)
#9 0xa723c8c ClTreeMap<ClAbstractToken>::Insert(this=0x55a1d0b8, key=..., value=...) ( cclang-plugin/src/ClangLib/src/treemap.h:37)
#10 0xa6e3c77 ClTokenDatabase::InsertToken(this=0xb00fdc8, token=...) ( cclang-plugin\src\ClangLib\src\tokendatabase.cpp:960)
#11 0xa6e9bf0 ClImportClangToken(cursor=..., scopeCursor=..., typ=ClTokenType_FuncDecl, client_data=0xb00fcc8) ( cclang-plugin\src\ClangLib\src\translationunit.cpp:886)
#12 0xa6ea1ea ClAST_Visitor(cursor=..., parent=..., client_data=0xb00fcc8) ( cclang-plugin\src\ClangLib\src\translationunit.cpp:983)
#13 0xf212b76 libclang!clang_disposeTranslationUnit() ( output\libclang.dll:??)
#14 0x42f42028 ?? () (??:??)
#15 0xf215041 libclang!clang_isTranslationUnit() ( output\libclang.dll:??)
#16 0x8 ?? () (??:??)
#17 ?? ?? () (??:??)

a other crash back trace. It happened after some time idle running