I did something similar to the plugin. It works on startup / shutdown, but when loading/unloading project, it's still blind.
However I did manage to speed it up significantly now, added some flags for reentrancy, file modification checks, and memory usage only goes to 33MB.
Unfortunately, the memory leak is STILL present. Everytime i hit "Refresh list", memory consumption goes up by 20 MB. I need memory debugging tools or something.
I'll commit tomorrow anyway, it seems the plugin's 100% usable now.