Packages are global to Sublime Text. They apply to every window and every file. It would be extremely hard to unload plugins from memory in a sane when when switching tabs in a window (different files may or may not be part of a project).
That completely ignores the fact that code from plugins could be running in threads - so Sublime Text would have to try and guess what threads are things that should keep going and which ones are ones that should be stopped. Generically trying to handle complex situations like this would be a support nightmare, and it would probably never work the way users wanted it to.
In short, I wouldn’t expect to see project-specific package loading ever. Instead, ideally packages should be written in such a way that they do not become intrusive for other types of projects. Sometimes, however, this can be hard or impossible.