The problem with this plugin and large files is currently how this plugin is implemented. It currently searches until it finds a match, so if you are between two brackets that are not far apart, then it is quick. If you are between no brackets in a large file, you will search the entire 7000 lines every time you move your cursor.
It is incorrect to say that it is something you just have to live with. I have created a branch that merged sublimeBracket and sublimeTagmatcher and does not have this problem. I fixed it by one: debouncing the events that trigger searching, two: implementing a search threshold so that by default you only search so far, and three: I allow blacklisting certain languages for matching via settings (you will never need to search for tags in CSS); if you would like to search the entire file until you find a match, that has been attached to an on demand shortcut.