headers
Emmanuel Florac | 27 Jan 2011 19:07
Favicon

can someone please explain the performance difference between mount.cifs and smbclient?


Using mount -t cifs //server/share /mnt/share between two big servers
connected with 10GigE, I've got : 115 MB/s reading, 132 MB/s writing. 
Using smbclient, I've got 450 MB/s reading, 132 MB/s writing (NFS gives
~ 260 MB/s write, 550 MB/s read on the same setup, with absolutely zero
optimisation).

Why this huge difference? BTW, why such a discrepancy between read and
write speed? 

--

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac <at> intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------
Permalink | Reply |
headers
Volker Lendecke | 27 Jan 2011 19:39
Picon
Favicon

Re: can someone please explain the performance difference between mount.cifs and smbclient?

On Thu, Jan 27, 2011 at 07:07:19PM +0100, Emmanuel Florac wrote:
> Using mount -t cifs //server/share /mnt/share between two big servers
> connected with 10GigE, I've got : 115 MB/s reading, 132 MB/s writing. 
> Using smbclient, I've got 450 MB/s reading, 132 MB/s writing (NFS gives
> ~ 260 MB/s write, 550 MB/s read on the same setup, with absolutely zero
> optimisation).
> 
> Why this huge difference? BTW, why such a discrepancy between read and
> write speed? 

Good question. There has been a subtle bug in smbclient in
that it did not send out requests as fast as it could (the
events system did not prefer writes over reads). You could
be hitting this bug. But I'd need to analyze this much
closer with an strace -ttT of smbclient and smbd, and a
wireshark trace.

With best regards,

Volker Lendecke

--

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
Permalink | Reply |
headers
Emmanuel Florac | 27 Jan 2011 23:11
Favicon

Re: [linux-cifs-client] can someone please explain the performance difference between mount.cifs and smbclient?

Le Thu, 27 Jan 2011 19:39:35 +0100 vous écriviez:

> > Why this huge difference? BTW, why such a discrepancy between read
> > and write speed?   
> 
> Good question. There has been a subtle bug in smbclient in
> that it did not send out requests as fast as it could (the
> events system did not prefer writes over reads). You could
> be hitting this bug. But I'd need to analyze this much
> closer with an strace -ttT of smbclient and smbd, and a
> wireshark trace.

Thank you, I'll post the strace output tomorrow. Note that I'm running
debian stable, so we're talking about rotten^w mature  versions :)

--

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@...>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------
Permalink | Reply |
headers
Jeff Layton | 27 Jan 2011 19:54
Picon
Favicon

Re: [linux-cifs-client] can someone please explain the performance difference between mount.cifs and smbclient?

On Thu, 27 Jan 2011 19:07:19 +0100
Emmanuel Florac <eflorac@...> wrote:

> 
> Using mount -t cifs //server/share /mnt/share between two big servers
> connected with 10GigE, I've got : 115 MB/s reading, 132 MB/s writing. 
> Using smbclient, I've got 450 MB/s reading, 132 MB/s writing (NFS gives
> ~ 260 MB/s write, 550 MB/s read on the same setup, with absolutely zero
> optimisation).
> 
> Why this huge difference? BTW, why such a discrepancy between read and
> write speed? 

FWIW, linux-cifs-client@... is deprecated.

cc'ing linux-cifs@... which is the new mailing list...

Most likely you're seeing the result of a lack of parallelism in the
Linux kernel cifs client. It does not do async writes (yet), which
really hinders throughput.

I'm hoping to get to work on that in the next few months unless someone
else beats me to it...

--

-- 
Jeff Layton <jlayton@...>
Permalink | Reply |
headers
Steve French | 27 Jan 2011 20:25
Picon
Gravatar

Re: [linux-cifs-client] can someone please explain the performance difference between mount.cifs and smbclient?

On Thu, Jan 27, 2011 at 12:54 PM, Jeff Layton <jlayton@...> wrote:
> On Thu, 27 Jan 2011 19:07:19 +0100
> Emmanuel Florac <eflorac@...> wrote:
>
>>
>> Using mount -t cifs //server/share /mnt/share between two big servers
>> connected with 10GigE, I've got : 115 MB/s reading, 132 MB/s writing.
>> Using smbclient, I've got 450 MB/s reading, 132 MB/s writing (NFS gives
>> ~ 260 MB/s write, 550 MB/s read on the same setup, with absolutely zero
>> optimisation).
>>
>> Why this huge difference? BTW, why such a discrepancy between read and
>> write speed?
>
> FWIW, linux-cifs-client@... is deprecated.
>
> cc'ing linux-cifs@... which is the new mailing list...
>
> Most likely you're seeing the result of a lack of parallelism in the
> Linux kernel cifs client. It does not do async writes (yet), which
> really hinders throughput.

The Linux cifs client does a good job of sending requests for
different files in parallel, and there are typically not "big kernel lock"
type problems - but for i/o to the same inode ... cifs_writepages and
cifs_readpages are serialized so you have "dead time" on the network
while the server (and later the client) is processing the request and
no request is in flight for part of the time.

The reason that cifs read performance is relatively worse - is that the
(Continue reading)

Permalink | Reply |
headers
Emmanuel Florac | 27 Jan 2011 23:03
Favicon

Re: [linux-cifs-client] can someone please explain the performance difference between mount.cifs and smbclient?

Le Thu, 27 Jan 2011 13:54:10 -0500 vous écriviez:

> cc'ing linux-cifs@... which is the new mailing list...

Damn, that's why I wasn't receiving messages from the list for a few
months :)

> Most likely you're seeing the result of a lack of parallelism in the
> Linux kernel cifs client. It does not do async writes (yet), which
> really hinders throughput.
> 
> I'm hoping to get to work on that in the next few months unless
> someone else beats me to it...

Thank you. In the meanwhile I'll probably stick to NFS :)

--

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@...>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------
Permalink | Reply |
headers
Steve French | 28 Jan 2011 04:19
Picon
Gravatar

Re: [linux-cifs-client] can someone please explain the performance difference between mount.cifs and smbclient?

On Thu, Jan 27, 2011 at 4:03 PM, Emmanuel Florac <eflorac@...m> wrote:
> Le Thu, 27 Jan 2011 13:54:10 -0500 vous écriviez:
>
>> cc'ing linux-cifs@... which is the new mailing list...
>
> Damn, that's why I wasn't receiving messages from the list for a few
> months :)
>
>> Most likely you're seeing the result of a lack of parallelism in the
>> Linux kernel cifs client. It does not do async writes (yet), which
>> really hinders throughput.
>>
>> I'm hoping to get to work on that in the next few months unless
>> someone else beats me to it...
>
> Thank you. In the meanwhile I'll probably stick to NFS :)
>

With NFSv4 you get similar caching to cifs, but there are
cases where our testing shows CIFS faster - but not for
single process, single file i/o.

--

-- 
Thanks,

Steve
Permalink | Reply |

Gmane