Package Control: A full-featured package manager

[wbond]

I'm not sure what you meant but calling thread.join definitely does NOT block the other threads from running, at least not for me.

Yes. I'm not sure why I had it in mind that joining one thread would stop the others. It will just stop the original spawning thread.

I'm going to try you solution again to see if I was just getting a fluke with the failures and timeouts. I suppose with a 5 second timeout that a couple of failures on github URL could easily make the process take a long time.

I'm not sure why I've experienced so many timeout issues when testing GitHub and BitBucket. I can browse the web no problem, and like you've mentioned, chrome has no trouble downloading 4+ assets from github at the same time, so I'm not sure why I have trouble with urllib2 and curl. I suppose maybe the fact that I need to spawn curl on Linux to download https URLs could be affecting performance, but that wouldn't explain why I was getting similar timeout issues on my Win 7 vm.

Anyway, I've at least identified bizoo's issue. I'll try to work around that for now and then try your downloader solution and see if I can get that working any faster. Hopefully I'll have a new, much more stable version out tonight or tomorrow.

[wbond]

Hrmmm, so I don't think it is a subprocess/hg bug but rather some weird condition comprised of showing the quick_panel from a timeout inside a thread when the Sublime window doesn't have focus. I guess only Jon would really know what the issue is there.

I got curious after going to bitbucket when in my notifications I saw a commit Guillermoo made to his Hg plugin. He was using windowless subprocess also. I then watched process explorer while alt - tabbing between it and Sublime and noticed Sublime crashing when git was the subprocess. Hrmm...

Anyway, so I found that it crashed fairly consistently at the same point, that being right at the end of PackageInstaller.make_package_list, after the very lastest package which in my group of plugins happens to be git controlled.

Hrmmm. So it seems to happen when `switching windows`. What if we make Sublime wait until it's focused? Does it still crash?

Code: Select all

        if os.name == 'nt':
            from ctypes import windll
            getwh = windll.user32.GetForegroundWindow

            while getwh() != self.window.hwnd():
                time.sleep(0.001)


It doesn't seem to in the 4-5 manual tests I've done. These crashes are only effecting us windows (lo|u)sers you say?

Thanks for the tip about the quick panel. I found that this simple example shows the crash off.

Code: Select all

# coding=utf-8
import sublime
import sublime_plugin
import threading
import time


class InstallPackageCommand(sublime_plugin.WindowCommand):
    def run(self):
        thread = InstallPackageThread(self.window)
        thread.start()


class InstallPackageThread(threading.Thread):
    def __init__(self, window):
        self.window = window
        threading.Thread.__init__(self)

    def run(self):
        time.sleep(3)
        def show_quick_panel():
            self.window.show_quick_panel([['foo'], ['bar']], self.on_done)
        sublime.set_timeout(show_quick_panel, 0)
   
    def on_done(self, num):
        print num


sublime.active_window().run_command('install_package')


Basically if a set_timeout of 0 that shows a quick panel is called in a thread and Sublime isn't the foremost window (on Windows) then it will crash. Changing the timeout to anything larger than 0 fixes the crash. Another interesting thing is that if the sublime window is not focused, the quick panel never shows, but instead fires the on_done callback with -1, like if the user had hit escape.

[wbond]

So the crash is NOT happening on linux/osx?

You are getting the -1 cancelling behaviour though?

On Linux and OS X I do not get a crash and the quick panel does show even though another window has the focus. I can return focus to the ST2 window and interact with the quick panel.

I kind of *like* having set_timeout *not* queue up/halt when Sublime is unactive.

It *should* show the quickpanel though eh and wait for input rather than cancel.

Yeah, I would not expect set_timeout to queue up or halt when it is not active. I would expect the timers to run as normal. And I agree, I think the behavior on Linux and OS X seems correct with the quick panel showing and waiting for input.

[wbond]

I just pushed version 1.2.5 out.

Thanks sublimator for all of the help. I ended up testing and getting the 150ms staggering to work well and I am seeing faster performance.

For a short term fix that isn't too messy I updated all of the sublime.set_timeout() calls to 10ms instead of 0ms. This seems to prevent the crashing issue on Windows that bizoo was having and I think some other crashing issues that Windows users have mentioned when trying to install and upgrade packages.

[queonda]

Just tried this on windows and get long delays when repos are refreshing. Also, it only seems to show about 5 available packages. Any idea what is going on?

[wbond]

Just tried this on windows and get long delays when repos are refreshing. Also, it only seems to show about 5 available packages. Any idea what is going on?

It sounds like https URLs are timing out. All of the packages are served over https, except for mine since I host my own packages.json file. It just so happens I have 6 packages available. The rest of the packages are hosted on GitHub and BitBucket, which force use of SSL.

If you look at the console (ctrl+`) you will probably see some errors about downloading URLs. Let me know what you find and I can try to help resolve it.

[queonda]

You are probably right - i am behind a proxy, which has caused issues with other source control programs. Do you know of a system environment variable or some other setting I can change, probably related to proxies, to allow this to go through?

[wbond]

You are probably right - i am behind a proxy, which has caused issues with other source control programs. Do you know of a system environment variable or some other setting I can change, probably related to proxies, to allow this to go through?

Package Control has two settings, http_proxy and https_proxy. If you set http_proxy and not https_proxy, the http_proxy will be used for https also.

You can see the settings to copy by going to the Preferences > Package Settings > Package Control > Settings - Default menu entry. Copy those two settings to the Preferences > Package Settings > Package Control > Settings - User file and customize there.

[bizoo]

I want to add "https://bitbucket.org/wuub/sublimerepl" to Package Control but I want the "default" branch, not the "pexpect" one that run only in linux.
How I could specify the branch in the config ?

Edit:
Didn't find a way in the api.bitbucket.org documentation to find the last changeset of a branch, so actually it's the last changeset of the repository that is downloaded, which is not the one I want.

[wbond]

I want to add "https://bitbucket.org/wuub/sublimerepl" to Package Control but I want the "default" branch, not the "pexpect" one that run only in linux.
How I could specify the branch in the config ?

Edit:
Didn't find a way in the api.bitbucket.org documentation to find the last changeset of a branch, so actually it's the last changeset of the repository that is downloaded, which is not the one I want.

Yeah, it looks like I need to update Package Control to explicitly specify the default branch when grabbing changesets. https://api.bitbucket.org/1.0/repositor ... lt?limit=1

I should be able to do this later today.