Developer forums (C::B DEVELOPMENT STRICTLY!) > CodeCompletion redesign
event handling of our CC, any new idea of the task queue design?
ollydbg:
We have many timers in our CC code. For example, editor activated event.
The first thing we know is that we don't directly response to those timers, because they happens too often. For example, if user open several editors in the same time, we may receive many such events, but what we are interest in is the last event. To implement this, we have a delay timer for the editor activated event.
We have many other timers, such as editor content changed, mouse position moved, editor closed, project loaded... And you can see that we have a lot of timers, which do quite similar things, they just start a delayed timer, if a similar event happens again(such as a new editor is activated again), then we just restart the timer gain, and all the actual operation is done in the timer event handler. This is the way much similar like the mouse dwell event, we only response to stable events, and thus have better performance.
Events are related, I mean if we first get a mouse position changed event, and when its delayed timer is running, we may received the new editor opened events, at this time, we should cancel the job associated with the mouse position changed event, since we have a new editor opened, and the mouse position is invalid for the new editor. In this situation, we should just sweep the job for mouse position changed event. There are a lot of similar cases of such event relationship.
Idea: I think we need a unified way to implement a TaskQueue, it may have such features:
* when an event happens, we register a new job, the job may run immediately, or we should run the job in a worker thread pool
* when an event happens, we register a new job, but a timer is started
* always keep the last same kind of event, such as editor active event, this means a new same event will cancel a same job
* event may have relations, so an important event may sweep all the previous jobs
* when the timer reached, the job should be run
* when there is not jobs in the queue, the timer may stopped
Any ideas?
Are there some code we can directly use?
EDIT: I think not the native CC, but other CC such as CC for Fortran, Clang CC may benefit.
yvesdm3000:
I've actually redone and removed a couple of the timers that were originally from Alpha, mostly because there is now a JobQueue that performs some lengtly operations and when that operation is done, the result is, when a lot of UI events come in, often no longer desired and is therefore dropped. You must however keep in mind that most of the Clang-operations are not read-only (sadly not even the code-completion operation since it does a simplified reparse) so they cannot be parallelized. You can parallelize on different translation units, but generally we're only interested in 1 translation unit: The one we're editing. Experiments using 2 translation units for the same document showed that Clang doesn't like it, and we can't copy a translation unit. So we're stuck with 1 JobQueue (except for indexing, could be totally separated because there is no interaction with the UI needed so ThreadPool comes to mind)
I currently designed it so to keep lock contention to a minimum by sending a message to the one job queue and sending a message back for event handling but also for destruction purposes.
Patches and/or suggestions for improvements on ClangCC are always welcome. :-)
Yves
ollydbg:
--- Quote from: yvesdm3000 on February 19, 2016, 07:29:21 am ---I've actually redone and removed a couple of the timers that were originally from Alpha, mostly because there is now a JobQueue that performs some lengtly operations and when that operation is done, the result is, when a lot of UI events come in, often no longer desired and is therefore dropped.
--- End quote ---
Hi, yvesdm3000, thanks for the reply. You said that old events may be dropped when they are not desired, that's the same thing I said, great! I will look at your git code repo to see how JobQueue works in your clangCC.
--- Quote ---You must however keep in mind that most of the Clang-operations are not read-only (sadly not even the code-completion operation since it does a simplified reparse) so they cannot be parallelized.
--- End quote ---
What do you mean by "not read-only", you mean you can not call two functions inside the clang shared library at the same time from two worker thread?
--- Quote ---You can parallelize on different translation units, but generally we're only interested in 1 translation unit: The one we're editing. Experiments using 2 translation units for the same document showed that Clang doesn't like it, and we can't copy a translation unit. So we're stuck with 1 JobQueue (except for indexing, could be totally separated because there is no interaction with the UI needed so ThreadPool comes to mind)
--- End quote ---
Do you means you have one JobQueue for one translation unit?
Under the native CC, we have one ThreadPool running a lot of parser threads to index all the files inside a cbp project.
--- Quote ---I currently designed it so to keep lock contention to a minimum by sending a message to the one job queue and sending a message back for event handling but also for destruction purposes.
--- End quote ---
Does the JobQueue works on a worker thread?
--- Quote ---Patches and/or suggestions for improvements on ClangCC are always welcome. :-)
--- End quote ---
OK, if I can build the ClangCC plugin, and test it. ;)
I haven't used clang code completion for several years, I see the last time I use clangCC was around 2011, at that time, I use the git repo https://github.com/Lalaland/ClangComplete.git, but I see this project is halt around 2012.
yvesdm3000:
--- Quote from: ollydbg on February 19, 2016, 09:50:20 am ---Hi, yvesdm3000, thanks for the reply. You said that old events may be dropped when they are not desired, that's the same thing I said, great! I will look at your git code repo to see how JobQueue works in your clangCC.
--- End quote ---
An example is probably better here:
- Document 1 is activated
- translation unit creation job started
- Document 2 is activated
- translation unit creation job finished
==> the translation unit created is no longer current for the activated document
--- Quote ---What do you mean by "not read-only", you mean you can not call two functions inside the clang shared library at the same time from two worker thread?
--- End quote ---
When things are read-only (and const), you could read from it from multiple threads. This is not at all applicable here as I explained.
--- Quote ---Do you means you have one JobQueue for one translation unit?
--- End quote ---
No reason to have a JobQueue for one translation unit. It would only help when you switch from one document to another and I don't consider that a problem at all currently.
--- Quote ---Under the native CC, we have one ThreadPool running a lot of parser threads to index all the files inside a cbp project.
--- End quote ---
I'll probably have the same once the indexing code of ALL files is in place, but I start with 1 thread to keep development simple and move to a ThreadPool later.
--- Quote ---Does the JobQueue works on a worker thread?
[/quote/
Yes. It's the C::B implementation.
--- Quote ---
--- Quote ---Patches and/or suggestions for improvements on ClangCC are always welcome. :-)
--- End quote ---
OK, if I can build the ClangCC plugin, and test it. ;)
I haven't used clang code completion for several years, I see the last time I use clangCC was around 2011, at that time, I use the git repo https://github.com/Lalaland/ClangComplete.git, but I see this project is halt around 2012.
--- End quote ---
--- End quote ---
I suggest using the one from the 'staging' branch, it's nearly ready to be merged into master, only a double diagnostics-event needs to be fixed in it, which is not too shaby.
Also don't enable the ClangLib toolbar on windows, it makes C::B unstable. I need to look into that shortly (don't understand why, it's an exact copy of the builtin-cc).
https://github.com/yvesdm3000/ClangLib
Yves
ollydbg:
@Yves, thanks. (I see you forgot adding a quote tag in your reply so that you words are quoted)
I just looked at clanglib's code, I see that
1, when the sdk(ccmanager) ask the suggestion list, you just send the request(a synchronous job) to a worker thread, and wait for the job get done in a limited time
2, I see you use the background thread class, which is a single thread queue
3, you have many components, such as:
--- Code: --- ClangCodeCompletion m_CodeCompletion;
ClangDiagnostics m_Diagnostics;
ClangToolbar m_Toolbar;
--- End code ---
so that when you get an c::b event, you can search on the component list, and see which one can handle such event.
But I haven't see some code that you can drop some already queued job from the thread queue. So, maybe, I need to implement one...
Navigation
[0] Message Index
[#] Next page
Go to full version