Developer forums (C::B DEVELOPMENT STRICTLY!) > Development
Revamping Code Completion with little changes
zieQ:
Yes, but what is the problem if you let the tasks ending naturally? (I still suppose that tasks are living for a short time heh). Anyway I think there is a potential bug with the way delete of tasks is implemented!
Mandrav, you didn't answer my question about wxExecute and threads: is there a way to execute a command in a thread instead of a process? I can't find any... In fact, I found out that C::B is very resource consuming when used as a background task (compared to MSVC), I'm wondering if it's not related to the way compiler commands are executed (in processes instead of threads)...
thomas:
wxExecute must not be used from anything but the main thread. First, it will rise an assertion if you try to, and second, wxExecute calls wxYield() at the most impossible of times, possibly causing recursive Yields.
wxExecute always runs on processes, but this is really not the reason why it is so inefficient. The main reason why it is so bad is that you must regularly Yield or you will never get anything from the pipe. The pipe will fill up and the child process will block. Consequently, your application will block, too, waiting for the child to die.
There are two modes of operation for wxExecute, one is while(!done){wxYield();} - this is what wxExecute does internally when running synchronously, it is *horribly* inefficient. The other way (the recommended way for asynchronous execution, and also the way it is done in code::blocks) is to use a wxTimer, which wastes less CPU, but means that no matter what you do, the Timer's minimum latency (around 50ms) plus the time needed to pass messages through the wx message queue (50-250ms?) always add to the execution time of your process. If you spawn off 200 processes, this can be quite a few seconds. There is nothing you can do, though, it is like this by design.
Apart from wxExecute, there is no way to run a command in a thread, either. It obviously has to be process (except if you load the executable image like a dll and do some hokuspokus maybe, but let's not even think about that...). However since the question was about Linux originally, process and thread is not really a different thing.
I would also advocate to be really gratious with threads (i.e. have several thread pools, and keep them sleeping rather than ending them). Thread scheduling is O(0) under Linux, and although the Windows scheduler sucks, it is still better to keep a few threads sleeping than spawn new ones regularly, as this is a lot more expensive.
I would also not make the thread pool a singleton (it is tempting, admitted!) because a thread pool is a queue, so if it is a singleton, you may get undesirable effects. For example if you decide to run "search in files" in its own thread, then it will have to wait until the last syntax parsing job is finished - this does not make sense since these are not related tasks.
A singleton with a priority queue might be a consideration. You could queue "interactive" tasks to the queue with a higher priority. However, this is a lot more complicated (keep things simple!), and the management of the priority queue is probably more expensive than just spawning two or three more threads.
When advocating many threads, one important thing to keep in mind is memory. If you spawn off 50 threads with 1 MB of stack each, then you need 50 MB of memory only for these, so obviously the total number should not be a lot more than maybe 6-8.
mandrav:
--- Quote from: zieQ on August 23, 2005, 08:30:59 am ---Mandrav, you didn't answer my question about wxExecute and threads: is there a way to execute a command in a thread instead of a process? I can't find any... In fact, I found out that C::B is very resource consuming when used as a background task (compared to MSVC), I'm wondering if it's not related to the way compiler commands are executed (in processes instead of threads)...
--- End quote ---
Unfortunately there is no way. This is the root of the problem (from wx docs):
--- Quote ---NB: Currently wxExecute() can only be used from the main thread, calling this function from another thread will result in an assert failure in debug build and won't work.
--- End quote ---
Yiannis.
grv575:
So why not make a couple of thread pools, each one for a specific cb component (compiler, code completion, other plugins). Have each pool allocate (by default) the same # of threads as cpus/cores. Now the only thing you don't want is to have more *resource intensive* threads than the number of cpus. If there are 2 cpus and only two intensive threads (with code completion and ui events not running for very long periods of time) then everything is optimal. So really, there shouldn't be any problem with > 1 pool, considering what CB does in terms of processing.
zieQ:
The term optimality I used previously refer to the article that nobody seems to have read, not a guess of what would be the better way to implement it and a guess on the number of worker threads to use. I'll stop making some comments if nobody read what I say :?
As I said,
- no need for many pools, it will degrade performance anyway: if one of the pool is already consuming all the processing power, the other pool will have some difficulties to run, and otherwise there will be many thread idle which consume (a little but...) processing power
- the number of thread should depend on the workload and workload type, not a fixed value which provide good performance for somebody but bad for others
Hope somebody is going to read the article from the link I posted above :x
So I agree about the wxExecute, and we will not be able to use the pool to perform compilation tasks
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version