Dieter Plaetinck | 15 Mar 2012 14:18
Picon

gazzilion lookups when cd'ing into mountpoint + assuring responsiveness

Hi,
I've used sshfs with success in various previous use cases,
although I'm now having an issue I haven't seen before.
I mount the sshfs like so:
sshfs -d -o debug -o idmap=user $server:/home/dieter/public_html/master /home/dieter/workspaces/eclipse/master

I can ls the contents of the mountpoint well, and read files, but as soon as I want to cd into the mount,
sshfs seems to perform a LOOKUP/getattr for pretty much every single file contained within the remote
directory (recursively).
This causes the cd call to stall.

My other question is, how is responsiveness assured? I know sshfs performs predictive readahead and
caching, but when are caches invalidated?
Does sshfs use something like inotify?
I'm working on a remote codebase (editing it locally, but storing and running it remotely, although a full
local cache is acceptable)
and I'm considering two options.  One is just using sshfs.  The other one is more involved but possibly more responsive:
I would just do an initial rsync of the directory to my local pc, and then use inotify to perform an rsync to the
remote host when files change.

any thoughts?

thanks,
Dieter

------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here 
http://p.sf.net/sfu/sfd2d-msazure
(Continue reading)

Miklos Szeredi | 19 Mar 2012 14:49
Picon

Re: gazzilion lookups when cd'ing into mountpoint + assuring responsiveness

Dieter Plaetinck <dieter <at> plaetinck.be> writes:

> Hi,
> I've used sshfs with success in various previous use cases,
> although I'm now having an issue I haven't seen before.
> I mount the sshfs like so:
> sshfs -d -o debug -o idmap=user $server:/home/dieter/public_html/master /home/dieter/workspaces/eclipse/master
>
> I can ls the contents of the mountpoint well, and read files, but as soon as I want to cd into the mount,
> sshfs seems to perform a LOOKUP/getattr for pretty much every single file contained within the remote
directory (recursively).
> This causes the cd call to stall.

This usually happens if LD_LIBRARY_PATH contains an entry for "."

>
> My other question is, how is responsiveness assured? I know sshfs performs predictive readahead and
caching, but when are caches invalidated?
> Does sshfs use something like inotify?

No, this would need support in the sftp-server and int the protocol.
Extending the protocol is theoretically possible, but I don't think
such extension makes sense, it's too far from the original purpose of
SFTP.

Thanks,
Miklos

------------------------------------------------------------------------------
This SF email is sponsosred by:
(Continue reading)

Dieter Plaetinck | 19 Mar 2012 14:56
Picon

Re: gazzilion lookups when cd'ing into mountpoint + assuring responsiveness

On Mon, 19 Mar 2012 14:49:10 +0100
Miklos Szeredi <miklos <at> szeredi.hu> wrote:

> Dieter Plaetinck <dieter <at> plaetinck.be> writes:
> 
> > Hi,
> > I've used sshfs with success in various previous use cases,
> > although I'm now having an issue I haven't seen before.
> > I mount the sshfs like so:
> > sshfs -d -o debug -o idmap=user $server:/home/dieter/public_html/master /home/dieter/workspaces/eclipse/master
> >
> > I can ls the contents of the mountpoint well, and read files, but as soon as I want to cd into the mount,
> > sshfs seems to perform a LOOKUP/getattr for pretty much every single file contained within the remote
directory (recursively).
> > This causes the cd call to stall.
> 
> This usually happens if LD_LIBRARY_PATH contains an entry for "."

this doesn't seem to be the case:
$ echo $LD_LIBRARY_PATH

$ echo "a${LD_LIBRARY_PATH}b"
ab
any other idea?

> > My other question is, how is responsiveness assured? I know sshfs performs predictive readahead and
caching, but when are caches invalidated?
> > Does sshfs use something like inotify?
> 
> No, this would need support in the sftp-server and int the protocol.
(Continue reading)

Miklos Szeredi | 19 Mar 2012 15:20
Picon

Re: gazzilion lookups when cd'ing into mountpoint + assuring responsiveness

Dieter Plaetinck <dieter <at> plaetinck.be> writes:

> On Mon, 19 Mar 2012 14:49:10 +0100
> Miklos Szeredi <miklos <at> szeredi.hu> wrote:
>
>> Dieter Plaetinck <dieter <at> plaetinck.be> writes:
>> 
>> > Hi,
>> > I've used sshfs with success in various previous use cases,
>> > although I'm now having an issue I haven't seen before.
>> > I mount the sshfs like so:
>> > sshfs -d -o debug -o idmap=user $server:/home/dieter/public_html/master /home/dieter/workspaces/eclipse/master
>> >
>> > I can ls the contents of the mountpoint well, and read files, but as soon as I want to cd into the mount,
>> > sshfs seems to perform a LOOKUP/getattr for pretty much every single file contained within the remote
directory (recursively).
>> > This causes the cd call to stall.
>> 
>> This usually happens if LD_LIBRARY_PATH contains an entry for "."
>
> this doesn't seem to be the case:
> $ echo $LD_LIBRARY_PATH
>
> $ echo "a${LD_LIBRARY_PATH}b"
> ab
> any other idea?

No ideas.  Can you send the debug log when entering the directory:

  sshfs -d -osshfs_debug ...
(Continue reading)

Dieter Plaetinck | 19 Mar 2012 15:59
Picon

Re: gazzilion lookups when cd'ing into mountpoint + assuring responsiveness

On Mon, 19 Mar 2012 15:20:59 +0100
Miklos Szeredi <miklos <at> szeredi.hu> wrote:

> Dieter Plaetinck <dieter <at> plaetinck.be> writes:
> 
> > On Mon, 19 Mar 2012 14:49:10 +0100
> > Miklos Szeredi <miklos <at> szeredi.hu> wrote:
> >
> >> Dieter Plaetinck <dieter <at> plaetinck.be> writes:
> >> 
> >> > Hi,
> >> > I've used sshfs with success in various previous use cases,
> >> > although I'm now having an issue I haven't seen before.
> >> > I mount the sshfs like so:
> >> > sshfs -d -o debug -o idmap=user $server:/home/dieter/public_html/master /home/dieter/workspaces/eclipse/master
> >> >
> >> > I can ls the contents of the mountpoint well, and read files, but as soon as I want to cd into the mount,
> >> > sshfs seems to perform a LOOKUP/getattr for pretty much every single file contained within the remote
directory (recursively).
> >> > This causes the cd call to stall.
> >> 
> >> This usually happens if LD_LIBRARY_PATH contains an entry for "."
> >
> > this doesn't seem to be the case:
> > $ echo $LD_LIBRARY_PATH
> >
> > $ echo "a${LD_LIBRARY_PATH}b"
> > ab
> > any other idea?
> 
(Continue reading)

Miklos Szeredi | 19 Mar 2012 18:13
Picon

Re: gazzilion lookups when cd'ing into mountpoint + assuring responsiveness

Dieter Plaetinck <dieter <at> plaetinck.be> writes:

> On Mon, 19 Mar 2012 15:20:59 +0100
> Miklos Szeredi <miklos <at> szeredi.hu> wrote:
>
>> Dieter Plaetinck <dieter <at> plaetinck.be> writes:
>> 
>> > On Mon, 19 Mar 2012 14:49:10 +0100
>> > Miklos Szeredi <miklos <at> szeredi.hu> wrote:
>> >
>> >> Dieter Plaetinck <dieter <at> plaetinck.be> writes:
>> >> 
>> >> > Hi,
>> >> > I've used sshfs with success in various previous use cases,
>> >> > although I'm now having an issue I haven't seen before.
>> >> > I mount the sshfs like so:
>> >> > sshfs -d -o debug -o idmap=user $server:/home/dieter/public_html/master /home/dieter/workspaces/eclipse/master
>> >> >
>> >> > I can ls the contents of the mountpoint well, and read files, but as soon as I want to cd into the mount,
>> >> > sshfs seems to perform a LOOKUP/getattr for pretty much every single file contained within the
remote directory (recursively).
>> >> > This causes the cd call to stall.
>> >> 
>> >> This usually happens if LD_LIBRARY_PATH contains an entry for "."
>> >
>> > this doesn't seem to be the case:
>> > $ echo $LD_LIBRARY_PATH
>> >
>> > $ echo "a${LD_LIBRARY_PATH}b"
>> > ab
(Continue reading)

Dieter Plaetinck | 21 Mar 2012 13:26
Picon

Re: gazzilion lookups when cd'ing into mountpoint + assuring responsiveness

On Mon, 19 Mar 2012 18:13:47 +0100
Miklos Szeredi <miklos <at> szeredi.hu> wrote:

> Dieter Plaetinck <dieter <at> plaetinck.be> writes:
> 
> > On Mon, 19 Mar 2012 15:20:59 +0100
> > Miklos Szeredi <miklos <at> szeredi.hu> wrote:
> >
> >> Dieter Plaetinck <dieter <at> plaetinck.be> writes:
> >> 
> >> > On Mon, 19 Mar 2012 14:49:10 +0100
> >> > Miklos Szeredi <miklos <at> szeredi.hu> wrote:
> >> >
> >> >> Dieter Plaetinck <dieter <at> plaetinck.be> writes:
> >> >> 
> >> >> > Hi,
> >> >> > I've used sshfs with success in various previous use cases,
> >> >> > although I'm now having an issue I haven't seen before.
> >> >> > I mount the sshfs like so:
> >> >> > sshfs -d -o debug -o idmap=user $server:/home/dieter/public_html/master /home/dieter/workspaces/eclipse/master
> >> >> >
> >> >> > I can ls the contents of the mountpoint well, and read files, but as soon as I want to cd into the mount,
> >> >> > sshfs seems to perform a LOOKUP/getattr for pretty much every single file contained within the
remote directory (recursively).
> >> >> > This causes the cd call to stall.
> >> >> 
> >> >> This usually happens if LD_LIBRARY_PATH contains an entry for "."
> >> >
> >> > this doesn't seem to be the case:
> >> > $ echo $LD_LIBRARY_PATH
(Continue reading)


Gmane