Home Download Buy Blog Forum Support

Structuring Packages into directory trees?

Structuring Packages into directory trees?

Postby SteveCooperOrg on Wed Jul 01, 2009 4:12 pm

I've got a recurring problem managing a lot of packages. Right now I consume two subversion repositories of packages; the google code one, and my own.

I'd like to be able to organize my packages something like this;

Code: Select all
  \Packages
      \Google
          \AAALoadFirstExtension
          ...
          \Zoom
      \Personal
          \Steve1
          ...
          \SteveN


The idea being that I can just check out two roots, and then keep everything up-to-date with two `svn up` commands.

Problem is, it doesn't work. To get a package to work, it has to be a direct child of `\Packages`, so the organizing subfolders cause the packages to load incorrectly -- keybindings load, menus load, but commands bound do not work. I'm not sure why.

I can't check both package repositories out directly under \Packages because the subversion information clashes.

What I'm wondering is if anyone has had any success in managing more than one Subversion repository of pacakages?

Any thoughts appreciated.
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Re: Structuring Packages into directory trees?

Postby SteveCooperOrg on Thu Jul 02, 2009 11:34 am

sublimator wrote:Having a full checkout of each repo might not be that useful in practice. How often do you really want to indiscriminately update each and every Package from the repository?


Actually, I want all the code on my machine. I don't necessarily want all the packages loaded, but that's not quite the same thing.

What I'm looking to do, really, is have all of the code on my machine, and then turn packages on and off. I don't know how yet, but that was the desire.

sublimator wrote:Would you not also on `svn update` get new packages as well or `updates` to packages you have deleted ?


Yeah, I'm looking to get all the updates available. I wouldn't want to delete packages -- rather, I'd turn them off. Using the magic method that I haven't defined ;)

sublimator wrote:In your `Packages` you could have cuttings from a master checkout of each package you used ( from n repositories ) and then have one single update.py script that managed. It would likely be gloriously slow as there would be a lot of redundant network usage compared to just one update command per repo.


This is what I was doing before. It is slow. Oh, boy. It also means you have to go through quite a rigmarole to start watching a new package, including changing your update script, your commit script, doing the initial checkout into the right location... That's why I wanted to move to a single operation.

I like the idea of looking for .svn files -- I was just keeping a master list, but the .svn search makes things much neater.

--

I actually thought last night that what I really want is multiple directories to serve as package folders; so I could set an option like this;

Code: Select all
packageDirectories <userdata>/Packages;c:\src\google-sublime;c:\sr\personal-sublime;


and have sublime monitor all three folders. Hmmm...

--

I wonder if you could use an __init__.py file to turn off packages? Not sure what you'd need to do -- my python-fu is weak!
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Re: Structuring Packages into directory trees?

Postby jps on Mon Jul 06, 2009 12:59 pm

svn externals are another option, still a bit of rigmarole to setup, but should work as desired afterwards.
jps
Site Admin
 
Posts: 3062
Joined: Wed Mar 19, 2008 12:33 pm

Re: Structuring Packages into directory trees?

Postby gpfsmurf on Mon Jul 06, 2009 5:44 pm

My far from perfect (but simple) solution is to have the whole SVN repo in a separate directory.
I copy the folders of the packages I want to use to my Packages folder (including the .svn subdirectory)
Then I can update them separately.

For instance, I have E:\sublime-text-community-packages\ which contains the whole repo
I copied E:\sublime-text-community-packages\CTags to X:\PortableApps\Sublime Text Data\Packages\CTags to be able to use CTags.

In essence it means each folder under Packages is either 1) a normal folder or 2) a subfolder of the SVN repo, and I think it could be used for several different SVN sources since in the end I don't have any .svn folder right under Packages. I only have .svn folders in the subfolders (such as Packages\CTags)
gpfsmurf
 
Posts: 211
Joined: Mon Jun 23, 2008 6:31 pm

Re: Structuring Packages into directory trees?

Postby EJ12N on Tue Jul 07, 2009 2:29 pm

gpfsmurf wrote:My far from perfect (but simple) solution is to have the whole SVN repo in a separate directory.
I copy the folders of the packages I want to use to my Packages folder (including the .svn subdirectory)
Then I can update them separately.

For instance, I have E:\sublime-text-community-packages\ which contains the whole repo
I copied E:\sublime-text-community-packages\CTags to X:\PortableApps\Sublime Text Data\Packages\CTags to be able to use CTags.

In essence it means each folder under Packages is either 1) a normal folder or 2) a subfolder of the SVN repo, and I think it could be used for several different SVN sources since in the end I don't have any .svn folder right under Packages. I only have .svn folders in the subfolders (such as Packages\CTags)


Thats exactly what I do too :)
EJ12N
 
Posts: 281
Joined: Tue Apr 21, 2009 5:23 pm

Re: Structuring Packages into directory trees?

Postby gpfsmurf on Wed Jul 08, 2009 3:08 am

I use tortoise so I just select all the directories and click on Update.
It opens a single dialog but issues separate SVN update commands.

I can see how more automation will be useful in the future when there are more plugins and more activity on the repo though; I especially like the idea off 'turning off' a particular package without removing the directory
gpfsmurf
 
Posts: 211
Joined: Mon Jun 23, 2008 6:31 pm

Re: Structuring Packages into directory trees?

Postby SteveCooperOrg on Thu Jul 09, 2009 9:04 am

Here's what I've ended up doing.

- I'm checking out folders individually into my packages dir.
- I've written a script which will bulk-update and bulk-commit changes to the svn packages in the packages dir.
- I run the bulk-update whenever I want to update things.

It's multithreaded, so it's pretty fast at updating a lot of small package folders.

The library code (svnlib.py) is this;

Code: Select all
import os, sys, subprocess
from threading import Thread

#
# These directories contain subdirectories which are subversion repositories.
# These are, then, collections of SVN repos, rather than repos themselves.
#
dirs = [
    "c:\\usr",
    "c:\\missing",
    "C:\\Documents and Settings\\Steve\\Application Data\\Sublime Text\\Packages"
    ]

def exist(x): return os.path.exists(x) # does a file/folder x exist?
def children(d): return [os.path.join(d, x) for x in os.listdir(d)] # subdirs/files of d
def issvn(x): return exist(os.path.join(x, ".svn")) # is x a subversion folder?
def flatmap(f, L): # like map, but for a fn of type a->[b], converts [a]->[b], not [a]->[[b]]
  for l in L:
    for r in f(l):
      yield r

def update(direc):
  """starts a subversion update thread for the given directory"""
  updater = Updater(direc)
  updater.start()
  return updater

def commit(direc):
  """starts a subversion commit thread for the given directory"""
  committer = Committer(direc)
  committer.start()
  return committer

class Committer(Thread):
  """Commit thread"""
 
  def __init__(self, direc):
    Thread.__init__(self)
    self.direc = direc
   
  def run(self):
    print "Committing", self.direc
    self.result = subprocess.call(['svn', 'commit', '-m', '', self.direc ])

class Updater(Thread):
  """Update thread"""
  def __init__(self, direc):
    Thread.__init__(self)
    self.direc = direc
   
  def run(self):
    print "Updating", self.direc
    self.result = subprocess.call(['svn', 'up', self.direc ])

def existingSvnDirs(dirs):
  """gives subversion directories within the dirs directories."""
  subs = filter(issvn, flatmap(children, map(os.path.abspath, filter(exist,dirs))))
  return subs

def runSvnProcess(proc):
  """Run a subversion process (update, commit) against a set of subversion repos"""
 
  # start the update/commit process
  threads = []
  for sub in existingSvnDirs(dirs):
    threads.append(proc(sub))
 
  # wait for the threads to join and show the results. 
  for thread in threads:
    thread.join()
    print "finished", thread.direc, "with", thread.result



and it's invoked like this;

Code: Select all
import svnlib
svnlib.runSvnProcess(svnlib.update)


The top of the library contains a list of my subversion-controlled folders; you'll want to edit this before you use it.
Maybe what you're looking for is at the http://www.sublimetextwiki.com
SteveCooperOrg
 
Posts: 193
Joined: Thu Mar 20, 2008 9:53 am

Next

Return to Plugin Development

Who is online

Users browsing this forum: No registered users and 6 guests