Structuring Packages into directory trees?

[SteveCooperOrg]

I've got a recurring problem managing a lot of packages. Right now I consume two subversion repositories of packages; the google code one, and my own.

I'd like to be able to organize my packages something like this;

Code: Select all

  \Packages
      \Google
          \AAALoadFirstExtension
          ...
          \Zoom
      \Personal
          \Steve1
          ...
          \SteveN


The idea being that I can just check out two roots, and then keep everything up-to-date with two `svn up` commands.

Problem is, it doesn't work. To get a package to work, it has to be a direct child of `\Packages`, so the organizing subfolders cause the packages to load incorrectly -- keybindings load, menus load, but commands bound do not work. I'm not sure why.

I can't check both package repositories out directly under \Packages because the subversion information clashes.

What I'm wondering is if anyone has had any success in managing more than one Subversion repository of pacakages?

Any thoughts appreciated.

[SteveCooperOrg]

Having a full checkout of each repo might not be that useful in practice. How often do you really want to indiscriminately update each and every Package from the repository?

Actually, I want all the code on my machine. I don't necessarily want all the packages loaded, but that's not quite the same thing.

What I'm looking to do, really, is have all of the code on my machine, and then turn packages on and off. I don't know how yet, but that was the desire.

Would you not also on `svn update` get new packages as well or `updates` to packages you have deleted ?

Yeah, I'm looking to get all the updates available. I wouldn't want to delete packages -- rather, I'd turn them off. Using the magic method that I haven't defined

In your `Packages` you could have cuttings from a master checkout of each package you used ( from n repositories ) and then have one single update.py script that managed. It would likely be gloriously slow as there would be a lot of redundant network usage compared to just one update command per repo.

This is what I was doing before. It is slow. Oh, boy. It also means you have to go through quite a rigmarole to start watching a new package, including changing your update script, your commit script, doing the initial checkout into the right location... That's why I wanted to move to a single operation.

I like the idea of looking for .svn files -- I was just keeping a master list, but the .svn search makes things much neater.

--

I actually thought last night that what I really want is multiple directories to serve as package folders; so I could set an option like this;

Code: Select all

packageDirectories <userdata>/Packages;c:\src\google-sublime;c:\sr\personal-sublime;


and have sublime monitor all three folders. Hmmm...

--

I wonder if you could use an __init__.py file to turn off packages? Not sure what you'd need to do -- my python-fu is weak!

[jps]

svn externals are another option, still a bit of rigmarole to setup, but should work as desired afterwards.

[gpfsmurf]

My far from perfect (but simple) solution is to have the whole SVN repo in a separate directory.
I copy the folders of the packages I want to use to my Packages folder (including the .svn subdirectory)
Then I can update them separately.

For instance, I have E:\sublime-text-community-packages\ which contains the whole repo
I copied E:\sublime-text-community-packages\CTags to X:\PortableApps\Sublime Text Data\Packages\CTags to be able to use CTags.

In essence it means each folder under Packages is either 1) a normal folder or 2) a subfolder of the SVN repo, and I think it could be used for several different SVN sources since in the end I don't have any .svn folder right under Packages. I only have .svn folders in the subfolders (such as Packages\CTags)

[EJ12N]

My far from perfect (but simple) solution is to have the whole SVN repo in a separate directory.
I copy the folders of the packages I want to use to my Packages folder (including the .svn subdirectory)
Then I can update them separately.

For instance, I have E:\sublime-text-community-packages\ which contains the whole repo
I copied E:\sublime-text-community-packages\CTags to X:\PortableApps\Sublime Text Data\Packages\CTags to be able to use CTags.

In essence it means each folder under Packages is either 1) a normal folder or 2) a subfolder of the SVN repo, and I think it could be used for several different SVN sources since in the end I don't have any .svn folder right under Packages. I only have .svn folders in the subfolders (such as Packages\CTags)

Thats exactly what I do too

[gpfsmurf]

I use tortoise so I just select all the directories and click on Update.
It opens a single dialog but issues separate SVN update commands.

I can see how more automation will be useful in the future when there are more plugins and more activity on the repo though; I especially like the idea off 'turning off' a particular package without removing the directory

[SteveCooperOrg]

Here's what I've ended up doing.

- I'm checking out folders individually into my packages dir.
- I've written a script which will bulk-update and bulk-commit changes to the svn packages in the packages dir.
- I run the bulk-update whenever I want to update things.

It's multithreaded, so it's pretty fast at updating a lot of small package folders.

The library code (svnlib.py) is this;

Code: Select all

import os, sys, subprocess
from threading import Thread

#
# These directories contain subdirectories which are subversion repositories.
# These are, then, collections of SVN repos, rather than repos themselves.
#
dirs = [
    "c:\\usr",
    "c:\\missing",
    "C:\\Documents and Settings\\Steve\\Application Data\\Sublime Text\\Packages"
    ]

def exist(x): return os.path.exists(x) # does a file/folder x exist?
def children(d): return [os.path.join(d, x) for x in os.listdir(d)] # subdirs/files of d
def issvn(x): return exist(os.path.join(x, ".svn")) # is x a subversion folder?
def flatmap(f, L): # like map, but for a fn of type a->[b], converts [a]->[b], not [a]->[[b]]
  for l in L:
    for r in f(l):
      yield r

def update(direc):
  """starts a subversion update thread for the given directory"""
  updater = Updater(direc)
  updater.start()
  return updater

def commit(direc):
  """starts a subversion commit thread for the given directory"""
  committer = Committer(direc)
  committer.start()
  return committer

class Committer(Thread):
  """Commit thread"""
 
  def __init__(self, direc):
    Thread.__init__(self)
    self.direc = direc
   
  def run(self):
    print "Committing", self.direc
    self.result = subprocess.call(['svn', 'commit', '-m', '', self.direc ])

class Updater(Thread):
  """Update thread"""
  def __init__(self, direc):
    Thread.__init__(self)
    self.direc = direc
   
  def run(self):
    print "Updating", self.direc
    self.result = subprocess.call(['svn', 'up', self.direc ])

def existingSvnDirs(dirs):
  """gives subversion directories within the dirs directories."""
  subs = filter(issvn, flatmap(children, map(os.path.abspath, filter(exist,dirs))))
  return subs

def runSvnProcess(proc):
  """Run a subversion process (update, commit) against a set of subversion repos"""
 
  # start the update/commit process
  threads = []
  for sub in existingSvnDirs(dirs):
    threads.append(proc(sub))
 
  # wait for the threads to join and show the results. 
  for thread in threads:
    thread.join()
    print "finished", thread.direc, "with", thread.result



and it's invoked like this;

Code: Select all

import svnlib
svnlib.runSvnProcess(svnlib.update)


The top of the library contains a list of my subversion-controlled folders; you'll want to edit this before you use it.