Sublime Forum

Bug: Directories not expanding on linux x64

#1

Seemingly randomly, when I try to expand a folder in the project pane, the directory will not open and display the list of files. The arrow will change to be pointing down so it appears that the directory was expanded, it just is empty. If I ls -l the directory though, I can see four files. The permissions on the files are 640 and the directory is 750. If I restart sublime, sometimes it will work, sometimes it doesn’t. Lemme know if you need more information.

Also, do you have a preferred way for listing bugs or just forums posts?

0 Likes

Files skipped; missing in Goto Anything, Side Bar; Windows
#2

Also, I’m seeing pretty high memory usage too. Sublime launches at 50 megs, and just expanding directories and leaving the app running, I had it up to 2.5 gigs of ram today :smile:

0 Likes

#3

Also, I’m not sure it’s exiting correctly either. I’ll run it from the terminal next time, but I just saw I had 6 instances of sublime running, but no open sublime windows :smile:
I tested it again by closing them all, opening and closing sublime once, and ps -ef | grep sublime still showed one running sublime process.

0 Likes

#4

With regards to the missing files issue, is there an entry in the console along the lines of “…has been seen before, skipping”?

Can you let me know what file system you’re using, and if it’s local or remote? Are there recursive symlinks involved?

0 Likes

#5

No recursive symlinks.
It is a network drive though.

I’ll check on the other questions when I get a chance, but that might not happen till Monday.

0 Likes

#6

It was printing the 'has been seen before" message. But, there were no recursive symlinks.
Instead, apparently this is a cron job that basically copies the home directory in ~/.snapshots.

I added my home folder to my sublime project and I later added a subdirectory.

So, for instance, I had two folders in the project:
home directory
project root

I’d expand the project root to something like
project root/fe/some file

And I’d see multiple copies of the message in the console
.snapshots/.hourly0/project root/fe/some file has been seen

And then the directory wouldn’t expand.

I removed my home directory and that mostly fixes it. It looks like the project does have a few recursive symlinks so I can still see this, but it’s no where near as common now.

0 Likes

#7

I don’t see the high memory usage or improper shutdown issues anymore either. I guess that was just because the snapshot directory was confusing it?

0 Likes

#8

When a folder is added to the sidebar, its contents are recursively enumerated in the background - it’s done this way, rather than when sub-folders are expanded (as happens in finder and explorer, for example), so that a list of files can be generated for Goto Anything to consider.

Scanning folders recursively, and not including the same file twice, nor getting stuck in a loop, is a surprisingly tricky thing to do in the face of hard links and symlinks, and cross platform vagaries. The general strategy used is to build a dictionary of directory fingerprints, and not scan a directory if we’ve already seen its fingerprint before.

On platforms where inode ids are available (Linux and OS X), the inode/device id pair is used as a fingerprint for the directory. On windows (or on a posix system when accessing an inode-less file system), the hash of the contents of the directory (file names, sizes, and write times) is used is the fingerprint.

If the .snapshots directory either has no inodes, or returns the same inodes as regular files in your home directory (possible if there’s some form of copy on write happening), then I would expect to see the “has been seen before” message, due to the contents matching other directories. From memory, home directories at Google are NFS mounted from NetApp filers, so I wouldn’t be surprised to see strange things happening with the inodes.

High memory usage and improper shutdown (pre-2070) indicates that the scanning was still happening in the background. In turn, this implies that either it was stuck in an infinite loop, or there were just a huge amount of files included, and it was just taking that much time+memory to scan them all. Either are a possibility, but I’m not currently aware of any situations that will cause the folder scanning to get stuck in an infinite loop.

0 Likes

Directory not loading in sidebar
Can't open directory in sidebar--not symlinks
#9

I should add to that, there is a known scenario where directory scanning will take an unbounded amount of time:

  • Must be falling back to content fingerprints (i.e., stat reports all inodes as 0)
  • Must have a recursive file system, where the contents are changing faster than the time it takes to scan the directory tree

In this case, because the contents of the directory are continuously changing, the content fingerprint based approach will not realise that it’s scanned the directory in question before.

0 Likes

#10

[quote=“jps”]I should add to that, there is a known scenario where directory scanning will take an unbounded amount of time:

  • Must be falling back to content fingerprints (i.e., stat reports all inodes as 0)
  • Must have a recursive file system, where the contents are changing faster than the time it takes to scan the directory tree

In this case, because the contents of the directory are continuously changing, the content fingerprint based approach will not realise that it’s scanned the directory in question before.[/quote]

Yea, it looks like stat is reporting all the inodes in my home directory as 0, but ls -i also shows the correct inodes. The inodes in the .snapshot/hourly* directories are the exact same, ie inode(~/.snapshot/hourly0/sublime) == inode(~/subime) so I’d guess if those were used it would appear to sublime as just a link?

Uh, yea, I start a week or so ago.

0 Likes

#11

Any updates regarding this? I’m running into the same problem. I have a network filesystem (linux/samba) and I’m accessing it via Windows 7(64) and running into odd problems

  • Taking forever to scan files
  • Intermittent empty directories.

I don’t have symlinks within this directory tree. I have the following set in user settings: ““ignore_inodes”: true”

I really like this editor, but if this is a continued problem, this may be a deal breaker for me.

Hope someone can help.

Thanks!
hanji

0 Likes

#12

I experienced this yesterday (actually the day before considering the actual “date”) on Windows, too.

I just added a simple subdir to my project with about 15 files and another dir with about 8 files, no links anywhere. ST would not show any files when expanding the new folder in the side bar, after restarting ST it worked. I don’t exactly know whether I could access the files using Goto Anywhere and I did not look into the console. Just leaving this here in case it helps.

OS: Windows 7 Ult x64, ST Build 2220

0 Likes