Clang CC

[oBFusCATed]

Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety.

How are you doing the deep-copies? I'm thinking of adding something like cbDeepCopyWxString that is reliable for both wx2.8 and wx3.0.
Also switching to std::string is not really a good idea - you'll do many conversions and possibly loose unicode support (you'll have to use wstring for that).

[BlueHazzard]

A other problem i noted quite frequently is if you open a new cpp file, enter code at the end of the file (no line text after the last line code) and this code has errors, so they get reported by the instant error reporting wxScilite will report some error about writing past the buffer, or writing non printable characters or something like this. I am actively investigating in this if i find some time.
This is no crash, but a assert()

I don't recall how i can reproduce this, but i will report as soon i am able to reproduce it...

Sounds familiar, I fixed something similar in the past, but it was only reproducible on Windows. Sadly I don't test it often on Windows. It's hard enough as it is to test on wx28, wx30, ubuntu, centos, ...

Yves

This seems to fix the error:

Code

diff --git a/src/clangrefactoring.cpp b/src/clangrefactoring.cpp
index b3913ac..f88b76d 100644
--- a/src/clangrefactoring.cpp
+++ b/src/clangrefactoring.cpp
@@ -217,8 +217,8 @@ void ClangRefactoring::BeginHighlightOccurrences(cbEditor* ed)
     // Set Styling:
     // clear all style indications set in a previous run (is also done once after text gets unselected)
     stc->IndicatorClearRange(0, stc->GetLength());
-
-    if (stc->GetTextRange(pos - 1, pos + 1).Strip().IsEmpty())
+    int max_length =  std::min(pos + 1 ,stc->GetLength());
+    if (stc->GetTextRange(pos - 1, max_length).Strip().IsEmpty())
         return;

     const int line = stc->LineFromPosition(pos);

i honestly don't know if it changes the behavior of the function, but at least it does not give this annoying message . The decision if this is the right fix i let for the developer...

[yvesdm3000]

Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety.

How are you doing the deep-copies? I'm thinking of adding something like cbDeepCopyWxString that is reliable for both wx2.8 and wx3.0.
Also switching to std::string is not really a good idea - you'll do many conversions and possibly loose unicode support (you'll have to use wstring for that).

I do a c_str() on the source wxString when the operation or event is copied. I also avoided using wxCommandEvent and built a ClangEvent object that does this deep copy of every member.

Yves

[yvesdm3000]

A other problem i noted quite frequently is if you open a new cpp file, enter code at the end of the file (no line text after the last line code) and this code has errors, so they get reported by the instant error reporting wxScilite will report some error about writing past the buffer, or writing non printable characters or something like this. I am actively investigating in this if i find some time.
This is no crash, but a assert()

I don't recall how i can reproduce this, but i will report as soon i am able to reproduce it...

Sounds familiar, I fixed something similar in the past, but it was only reproducible on Windows. Sadly I don't test it often on Windows. It's hard enough as it is to test on wx28, wx30, ubuntu, centos, ...

Yves

This seems to fix the error:

Code

diff --git a/src/clangrefactoring.cpp b/src/clangrefactoring.cpp
index b3913ac..f88b76d 100644
--- a/src/clangrefactoring.cpp
+++ b/src/clangrefactoring.cpp
@@ -217,8 +217,8 @@ void ClangRefactoring::BeginHighlightOccurrences(cbEditor* ed)
     // Set Styling:
     // clear all style indications set in a previous run (is also done once after text gets unselected)
     stc->IndicatorClearRange(0, stc->GetLength());
-
-    if (stc->GetTextRange(pos - 1, pos + 1).Strip().IsEmpty())
+    int max_length =  std::min(pos + 1 ,stc->GetLength());
+    if (stc->GetTextRange(pos - 1, max_length).Strip().IsEmpty())
         return;

     const int line = stc->LineFromPosition(pos);

i honestly don't know if it changes the behavior of the function, but at least it does not give this annoying message . The decision if this is the right fix i let for the developer...

I applied the fix. Thanks !

Yves

[yvesdm3000]

Normally I try to keep these wxString objects created&destroyed on the same thread or deep-copied if they need to be handed over from one thread to another. It's probably better if I switch to std::string for anything that crosses these threads and force c++11 to assure thread-safety.

How are you doing the deep-copies? I'm thinking of adding something like cbDeepCopyWxString that is reliable for both wx2.8 and wx3.0.
Also switching to std::string is not really a good idea - you'll do many conversions and possibly loose unicode support (you'll have to use wstring for that).

I want to use std::string because Clang uses utf8, otherwise i'm stuck with a conversion of 16 bits into 8 bits (for most of the cases). I know std::string isn't ideal for utf8 but when you know the limitations, it's doable since I don't process the strings all that much.

Yves

[ollydbg]

In the file "include\sdk_events.h", there is a class named "CodeBlocksThreadEvent" which I introduced years ago to implement deep copy of it's wxString member. It is for wx 2.8.12, since wx 3.x does not such issue, and this class is used for thread safety in some plugins.

[oBFusCATed]

In wx3.0 in most implementations (gcc until 5.x) the string is still reference counted, but it is protected by an atomic. So deep copying is useful there, too.

Also in wx3.0 they've added a clone method: http://docs.wxwidgets.org/3.1.0/classwx_string.html#afe63f53ecaa197333c405ca985f733fe wxString::Clone.

[ollydbg]

In wx3.0 in most implementations (gcc until 5.x) the string is still reference counted, but it is protected by an atomic. So deep copying is useful there, too.

Also in wx3.0 they've added a clone method: http://docs.wxwidgets.org/3.1.0/classwx_string.html#afe63f53ecaa197333c405ca985f733fe wxString::Clone.

If I remember correctly, if the wxString(std::string) is atomic reference counted, it is theadsafe. I mean the wxCommandEvent should be threadsafe. What's what the atomic reference counter used for, in "copy on write" mode.

[yvesdm3000]

I pushed a new feature called "Call hierarchy" that allows you to lookup call references and where a specific method/function is being called. And then each function that referenced a function can simply be looked up again in a new level in the tree. It's very usefull for understanding large codebases and is more accurate than the thread-search plugin.

There are still a lot of limitations that will be solved in the next updates:
- Doesn't list overrides
- Doesn't list callers that are calling a pure or not function that the selected call overrides
- Doesn't work on anything else than functions/methods. Member variables, global variables, class instances all come to mind tho be useful for this feature.
- List to the right should become optional

And obviously small bugs since it's all new code that will be solved every time I come across one (or someone indicates one).

As always open to feedback, suggestions or other excellent ideas.

Yves

[BlueHazzard]

Hi!
good work!

i tested your plugin on linux and have an assert from wx:

Code

ASSERT INFO:
/usr/include/wx-3.0/wx/datetime.h(876): assert "IsValid() && dt.IsValid()" failed in operator!=(): invalid wxDateTime

BACKTRACE:
[1] ClangToolbar::OnEditorHook(cbEditor*, wxScintillaEvent&)
[2] EditorHooks::CallHooks(cbEditor*, wxScintillaEvent&)
[3] cbEditor::OnScintillaEvent(wxScintillaEvent&)
[4] cbEditor::OnEditorUpdateUI(wxScintillaEvent&)
[5] wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const
[6] wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)
[7] wxEvtHandler::SearchDynamicEventTable(wxEvent&)
[8] wxEvtHandler::TryHereOnly(wxEvent&)
[9] wxEvtHandler::ProcessEventLocally(wxEvent&)
[10] wxEvtHandler::ProcessEvent(wxEvent&)
[11] wxWindowBase::TryAfter(wxEvent&)
[12] wxScintilla::NotifyParent(SCNotification*)
[13] ScintillaWX::NotifyParent(SCNotification)
[14] Editor::NotifyUpdateUI()
[15] Editor::Paint(Surface*, PRectangle)
[16] ScintillaWX::DoPaint(wxDC*, wxRect)
[17] wxScintilla::OnPaint(wxPaintEvent&)
[18] wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const
[19] wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)
[20] wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*)
[21] wxEvtHandler::TryHereOnly(wxEvent&)
[22] wxEvtHandler::DoTryChain(wxEvent&)
[23] wxEvtHandler::ProcessEvent(wxEvent&)
[24] wxEvtHandler::SafelyProcessEvent(wxEvent&)
[25] wxWindow::GTKSendPaintEvents(_GdkRegion const*)
[26] g_closure_invoke
[27] g_signal_emit_valist
[28] g_signal_emit
[29] gtk_main_do_event
[30] gdk_window_process_all_updates
[31] g_main_context_dispatch
[32] g_main_context_iteration
[33] gtk_main_iteration
[34] wxWindow::DoPopupMenu(wxMenu*, int, int)
[35] wxWindowBase::PopupMenu(wxMenu*, int, int)
[36] wxWindowBase::PopupMenu(wxMenu*, wxPoint const&) /usr/include/wx-3.0/wx/window.h:1216
[37] EditorBase::DisplayContextMenu(wxPoint const&, ModuleType)
[38] cbStyledTextCtrl::OnContextMenu(wxContextMenuEvent&)
[39] wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const
[40] wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&)
[41] wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*)
[42] wxEvtHandler::TryHereOnly(wxEvent&)
[43] wxEvtHandler::DoTryChain(wxEvent&)
[44] wxEvtHandler::ProcessEvent(wxEvent&)
[45] wxEvtHandler::SafelyProcessEvent(wxEvent&)
[46] g_closure_invoke
[47] g_signal_emit_valist
[48] g_signal_emit
[49] gtk_propagate_event
[50] gtk_main_do_event
[51] g_main_context_dispatch
[52] g_main_loop_run
[53] gtk_main
[54] wxGUIEventLoop::DoRun()
[55] wxEventLoopBase::Run()
[56] wxAppConsoleBase::MainLoop()
[57] CodeBlocksApp::OnRun() codeblocks_sf/src/src/app.cpp:850
[58] wxEntry(int&, wchar_t**)
[59] CodeBlocks codeblocks_sf/src/src/app.cpp:322
[60] __libc_start_main
[61] _start

The source is from here:
clangtoolbar.cpp:196

Code

if ( (ed->GetLastModificationTime() != m_CurrentState.m_CurrentEditorModificationTime)||(m_Function&&(m_Function->GetCount()==0)))

it seems that m_CurrentState.m_CurrentEditorModificationTime is not initialized by inspecting with the debugger...

Also i can't test your new function? How do i call it? In the right click menu i can't find a entry?
[edit:] Found it... Somehow the menu entry was not present, but after a restart all worked

greetings

[yvesdm3000]

CurrentState.m_CurrentEditorModificationTime  initialization should now be fixed

Yves

[BlueHazzard]

How is the update algorithm of the cc? If i open the codeblocks project file one of my processors is at 100%  and the console outputs

Code

Reindex on tokenindexdb 0x3831db0 project=codeblocks_sf/src/CodeBlocks_wx30-unix.cbp /codeblocks_sf/src/include/configurationpanel.h
ClTranslationUnit::Parse id=127
[Thread 0x7fffb6732700 (LWP 20185) exited]
ClTranslationUnit::UpdateTokenDatabase 127 finished: 74779 tokens processed

could this be a recursive loop?
Does the parser indexes all files all the time, or does it stop after all project files are parsed?

thank you for your work!

greetings

[yvesdm3000]

How is the update algorithm of the cc? If i open the codeblocks project file one of my processors is at 100%  and the console outputs

Code

Reindex on tokenindexdb 0x3831db0 project=codeblocks_sf/src/CodeBlocks_wx30-unix.cbp /codeblocks_sf/src/include/configurationpanel.h
ClTranslationUnit::Parse id=127
[Thread 0x7fffb6732700 (LWP 20185) exited]
ClTranslationUnit::UpdateTokenDatabase 127 finished: 74779 tokens processed

could this be a recursive loop?
Does the parser indexes all files all the time, or does it stop after all project files are parsed?

thank you for your work!

greetings

The indexer checks if the timestamp of a file is more recent than the timestamp in the database. If it does, it reindexes and updates the timestamp in the database. Someday I want to extend this with the filesize just in case a file has been reverted (a checksum is probably overkill) .

I actually fixed an issue on this in commit d0c4f433b0b1e619f7fd14794c05d84379db7c1b on 21 of march for an issue where it continuously tried to reindex the same file. I don't know if your build has this fix yet?

Yves

[BlueHazzard]

Code

#0 0xa6ed440	__cxa_throw() ( output\share\codeblocks\plugins\clanglib.dll:??)
#1 0xa6ec266	operator new(unsigned int) () ( output\share\codeblocks\plugins\clanglib.dll:??)
#2 0xa7bae18	typeinfo for std::time_put<wchar_t, std::ostreambuf_iterator<wchar_t, std::char_traits<wchar_t> > >() ( output\share\codeblocks\plugins\clanglib.dll:??)
#3 0xa6f0200	std::bad_alloc::what() const() ( output\share\codeblocks\plugins\clanglib.dll:??)
#4 0xa7784c9	std::allocator_traits<std::allocator<ClAbstractToken> >::max_size(__a=...) (mingw32/4.9.2/include/c++/bits/alloc_traits.h:422)
#5 0xa7784a8	std::allocator_traits<std::allocator<ClAbstractToken> >::allocate(__a=..., __n=131072) (mingw32/4.9.2/include/c++/bits/alloc_traits.h:357)
#6 0xa762a99	std::_Vector_base<ClAbstractToken, std::allocator<ClAbstractToken> >::_M_allocate(this=0x55a1d0bc, __n=131072) ( mingw32/4.9.2/include/c++/bits/stl_vector.h:170)
#7 0xa786642	std::vector<ClAbstractToken, std::allocator<ClAbstractToken> >::_M_emplace_back_aux<ClAbstractToken const&>(this=0x55a1d0bc) ( mingw32/4.9.2/include/c++/bits/vector.tcc:412)
#8 0xa786a06	std::vector<ClAbstractToken, std::allocator<ClAbstractToken> >::push_back(this=0x55a1d0bc, __x=...) ( mingw32/4.9.2/include/c++/bits/stl_vector.h:923)
#9 0xa723c8c	ClTreeMap<ClAbstractToken>::Insert(this=0x55a1d0b8, key=..., value=...) ( cclang-plugin/src/ClangLib/src/treemap.h:37)
#10 0xa6e3c77	ClTokenDatabase::InsertToken(this=0xb00fdc8, token=...) ( cclang-plugin\src\ClangLib\src\tokendatabase.cpp:960)
#11 0xa6e9bf0	ClImportClangToken(cursor=..., scopeCursor=..., typ=ClTokenType_FuncDecl, client_data=0xb00fcc8) ( cclang-plugin\src\ClangLib\src\translationunit.cpp:886)
#12 0xa6ea1ea	ClAST_Visitor(cursor=..., parent=..., client_data=0xb00fcc8) ( cclang-plugin\src\ClangLib\src\translationunit.cpp:983)
#13 0xf212b76	libclang!clang_disposeTranslationUnit() ( output\libclang.dll:??)
#14 0x42f42028	?? () (??:??)
#15 0xf215041	libclang!clang_isTranslationUnit() ( output\libclang.dll:??)
#16 0x8	?? () (??:??)
#17 ??	?? () (??:??)

a other crash back trace. It happened after some time idle running