* The tree construction needs to be split in several parts so it can be done inside idle events. The way to do it
is simple: Instead of building the tree topdown (from the root to the leaves), grab all the tokens and add them to
the tree. If their parent isn't there, add the parent. This way, we can keep an index to the last added item, and
add around 100 items at a time.
* The tokens need to be organized per file (actually per workspace), and not per project. It's those duplicated
tokens that add so much memory in big workspaces.
So we need to add inside the TokensTree (more than a tree it's a whole set of information), a map of project
numbers to project names, and for each file, keep a list (an STL set) of all the projects (numbers) it belongs to.
When searching for the included files, just query the first project that the file belongs to (we could add to the
path all the projects).
Then, the tokens need to become independent from the files. There are two dependencies in the Token structure that
hinder us from doing this optimization. The ancestors (and descendants), and the implementation file / number.
We should get rid of the ancestor/descendant indexes, and just keep the names. When searching for the ancestors,
just make a variation of the "LinkInheritance" that works on only one token. Searching the tokens by name is easy
with my SearchTree, i forgot which tree was used to map the names and tokens, but it's somewhere in there
The implementation file / number is a bit more tricky. Actually the implementation file / number applies only to
functions and methods. So we need to create a new token kind, which I'll call "function implementation". The
function tokens become just function declarations. So to search for the declaration / implementation, just find
the name among the known tokens.
There is a catch, however. We need to identify a token by listing all its parent or namespaces it belongs to. So
the token might need a new function, "CanonicalName" to identify the tokens.
Suppose we have this:
int myNamespace::myClass::myFunction(int myparam);
myfunction's canonical name is "myNamespace::myClass::myFunction".
But suppose we have this:
(Example #2)
namespace myNamespace
{
int myClass::myFunction(int myparam);
}
it's canonical namespace is the same.
So, We might need another function, "surname".
so myFunction's surName is "myClass::". myFunction's canonical name is calculated in the following way:
if(function has parent)
canonical name = parent's canonical name + "::" + token's plain name (name).
else
canonical name = token's surname + token's plain name.
It's recursive, this way we can easily search for the implementation / declaration without having to put the links inside the Token class.
---
Now there are other changes to be done: The code completion functionality is obsolete by now. It uses Token's m_bool member. But we can use the TokenIdxSet class (it's an STL set) to list the currently elligible tokens. But I'll write about that in another post.
