Home Download Buy Blog Forum Support

Bug: Directories not expanding on linux x64

Bug: Directories not expanding on linux x64

Postby ajpalkovic on Fri Jun 24, 2011 1:38 am

Seemingly randomly, when I try to expand a folder in the project pane, the directory will not open and display the list of files. The arrow will change to be pointing down so it appears that the directory was expanded, it just is empty. If I ls -l the directory though, I can see four files. The permissions on the files are 640 and the directory is 750. If I restart sublime, sometimes it will work, sometimes it doesn't. Lemme know if you need more information.

Also, do you have a preferred way for listing bugs or just forums posts?
ajpalkovic
 
Posts: 142
Joined: Fri Jun 10, 2011 10:21 pm

Re: Bug: Directories not expanding on linux x64

Postby ajpalkovic on Fri Jun 24, 2011 1:41 am

Also, I'm seeing pretty high memory usage too. Sublime launches at 50 megs, and just expanding directories and leaving the app running, I had it up to 2.5 gigs of ram today :)
ajpalkovic
 
Posts: 142
Joined: Fri Jun 10, 2011 10:21 pm

Re: Bug: Directories not expanding on linux x64

Postby ajpalkovic on Fri Jun 24, 2011 1:46 am

Also, I'm not sure it's exiting correctly either. I'll run it from the terminal next time, but I just saw I had 6 instances of sublime running, but no open sublime windows :)
I tested it again by closing them all, opening and closing sublime once, and ps -ef | grep sublime still showed one running sublime process.
ajpalkovic
 
Posts: 142
Joined: Fri Jun 10, 2011 10:21 pm

Re: Bug: Directories not expanding on linux x64

Postby jps on Fri Jun 24, 2011 12:56 pm

With regards to the missing files issue, is there an entry in the console along the lines of "...has been seen before, skipping"?

Can you let me know what file system you're using, and if it's local or remote? Are there recursive symlinks involved?
jps
Site Admin
 
Posts: 3071
Joined: Wed Mar 19, 2008 12:33 pm

Re: Bug: Directories not expanding on linux x64

Postby ajpalkovic on Fri Jun 24, 2011 4:32 pm

No recursive symlinks.
It is a network drive though.

I'll check on the other questions when I get a chance, but that might not happen till Monday.
ajpalkovic
 
Posts: 142
Joined: Fri Jun 10, 2011 10:21 pm

Re: Bug: Directories not expanding on linux x64

Postby ajpalkovic on Mon Jun 27, 2011 7:38 pm

It was printing the 'has been seen before" message. But, there were no recursive symlinks.
Instead, apparently this is a cron job that basically copies the home directory in ~/.snapshots.

I added my home folder to my sublime project and I later added a subdirectory.

So, for instance, I had two folders in the project:
home directory
project root

I'd expand the project root to something like
project root/fe/some file

And I'd see multiple copies of the message in the console
.snapshots/.hourly0/project root/fe/some file has been seen

And then the directory wouldn't expand.

I removed my home directory and that mostly fixes it. It looks like the project does have a few recursive symlinks so I can still see this, but it's no where near as common now.
ajpalkovic
 
Posts: 142
Joined: Fri Jun 10, 2011 10:21 pm

Re: Bug: Directories not expanding on linux x64

Postby ajpalkovic on Tue Jun 28, 2011 1:52 am

I don't see the high memory usage or improper shutdown issues anymore either. I guess that was just because the snapshot directory was confusing it?
ajpalkovic
 
Posts: 142
Joined: Fri Jun 10, 2011 10:21 pm

Re: Bug: Directories not expanding on linux x64

Postby jps on Tue Jun 28, 2011 3:05 am

When a folder is added to the sidebar, its contents are recursively enumerated in the background - it's done this way, rather than when sub-folders are expanded (as happens in finder and explorer, for example), so that a list of files can be generated for Goto Anything to consider.

Scanning folders recursively, and not including the same file twice, nor getting stuck in a loop, is a surprisingly tricky thing to do in the face of hard links and symlinks, and cross platform vagaries. The general strategy used is to build a dictionary of directory fingerprints, and not scan a directory if we've already seen its fingerprint before.

On platforms where inode ids are available (Linux and OS X), the inode/device id pair is used as a fingerprint for the directory. On windows (or on a posix system when accessing an inode-less file system), the hash of the contents of the directory (file names, sizes, and write times) is used is the fingerprint.

If the .snapshots directory either has no inodes, or returns the same inodes as regular files in your home directory (possible if there's some form of copy on write happening), then I would expect to see the "has been seen before" message, due to the contents matching other directories. From memory, home directories at Google are NFS mounted from NetApp filers, so I wouldn't be surprised to see strange things happening with the inodes.

High memory usage and improper shutdown (pre-2070) indicates that the scanning was still happening in the background. In turn, this implies that either it was stuck in an infinite loop, or there were just a huge amount of files included, and it was just taking that much time+memory to scan them all. Either are a possibility, but I'm not currently aware of any situations that will cause the folder scanning to get stuck in an infinite loop.
jps
Site Admin
 
Posts: 3071
Joined: Wed Mar 19, 2008 12:33 pm

Re: Bug: Directories not expanding on linux x64

Postby jps on Tue Jun 28, 2011 4:44 am

I should add to that, there is a known scenario where directory scanning will take an unbounded amount of time:

- Must be falling back to content fingerprints (i.e., stat reports all inodes as 0)
- Must have a recursive file system, where the contents are changing faster than the time it takes to scan the directory tree

In this case, because the contents of the directory are continuously changing, the content fingerprint based approach will not realise that it's scanned the directory in question before.
jps
Site Admin
 
Posts: 3071
Joined: Wed Mar 19, 2008 12:33 pm

Re: Bug: Directories not expanding on linux x64

Postby ajpalkovic on Tue Jun 28, 2011 6:07 pm

jps wrote:I should add to that, there is a known scenario where directory scanning will take an unbounded amount of time:

- Must be falling back to content fingerprints (i.e., stat reports all inodes as 0)
- Must have a recursive file system, where the contents are changing faster than the time it takes to scan the directory tree

In this case, because the contents of the directory are continuously changing, the content fingerprint based approach will not realise that it's scanned the directory in question before.

Yea, it looks like stat is reporting all the inodes in my home directory as 0, but ls -i also shows the correct inodes. The inodes in the .snapshot/hourly* directories are the exact same, ie inode(~/.snapshot/hourly0/sublime) == inode(~/subime) so I'd guess if those were used it would appear to sublime as just a link?

sublimator wrote:Do you work for Google ajpalkovic?

Uh, yea, I start a week or so ago.
ajpalkovic
 
Posts: 142
Joined: Fri Jun 10, 2011 10:21 pm

Next

Return to Technical Support

Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 23 guests