Network Appliance Fileserver discussion

headers Milazzo Giacomo | 17 Jun 2012 17:23

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

We've been wanting C-Mode for a while but there are some standard features that we rely on that aren't there
yet.  
This email got cut off so I sent another.  My basic thought on it is - sure, there are those that can move to
C-Mode but that doesn't mean NetApp should force everyone to C-Mode.  We were actually going to go to it on a
new cluster earlier this year and then were advised against it due to a bug (which has been fixed).

We'll probably consider it next year sometime.  Just not for existing infrastructure.
C-

Bohemen, Anton van wrote:
> Well, to be honest, I think it is time to (at least) start thinking 
> about moving to cluster mode. A FAS2220 is a bit on the positive side as 
> you can’t do SAN with it in C-Mode, but a 2240 can, so then you are 
> fine. I’ve been testing quite extensive with the simulator and it 
> actually works fine. A bit to my surprise as well I must admit, I was 
> just as skeptical as you all. The concepts are great and the VServer 
> virtualization layer is a great addition to the WAFL storage 
> virtualization layer. VServers work so much better than vFilers and with 
> c-mode you get all the scalability advantages you need.
> 
>  
> 
> So, I’m not surprised all the commercial people at NetApp and partners 
> are pushing to C-Mode. If you have a technology refresh in a few years, 
> you should definitely consider it. If you have one this year, I’m not 
> really sure yet. Not because not all features are supported in c-mode 
> (yet), but right now it is quite a steep learning curve if you’re used 
> to 7-mode. You need to re-think most of your storage setup but once you 
> do, I think c-mode will fit a lot of environments.
> 
>  
> 
> Bottom line: start reading and testing, I think c-mode actually is a 
> good product.
> 
>  
> 
> Regards,
> 
>  
> 
> Anton
> 
>  
> 
>  
> 
> *From:* toasters-bounces <at> teaparty.net 
> [mailto:toasters-bounces <at> teaparty.net] *On Behalf Of *Blackmor, Chris
> *Sent:* zondag 17 juni 2012 17:45
> *To:* Milazzo Giacomo
> *Cc:* toasters <at> teaparty.net
> *Subject:* Re: Cluster mode - Market vs field
> 
>  
> 
> They can say that all they want but I think NA has had way too many 
> missteps with C-mode to be pushing it down anyone's 
> 
> Sent from my iPhone
> 
>  
> 
> 
> On Jun 17, 2012, at 10:24 AM, "Milazzo Giacomo" <G.Milazzo <at> sinergy.it 
> <mailto:G.Milazzo <at> sinergy.it>> wrote:
> 
>     Hi all,
> 
>      
> 
>     I’ve recently attended an official event where people from NetApp in
>     few words said:
> 
>      
> 
>     “/DOT 8 7-mode has done its time, it’s over, it has been a great os
>     but It’s now time to move and push DOT 8 Cluster Mode, everytime,
>     everywhere…/” and this also if with a simple two node HA system,
>     including 2220!
> 
>      
> 
>     What do you think about? Also let us assume that cluster mode can (I
>     add ‘theoretically’) do all what 7 mode does, don’t you think that
>     is quite premature to trumpet this out?
> 
>      
> 
>     Regards,
> 
>      
> 
>     _______________________________________________
>     Toasters mailing list
>     Toasters <at> teaparty.net <mailto:Toasters <at> teaparty.net>
>     http://www.teaparty.net/mailman/listinfo/toasters
> 
> This e-mail is personal. For our full disclaimer, please visit www.centric.eu/disclaimer.
> 

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

Darren Sykes wrote:
> While I wouldn't disagree that it's a good product, having run GX, then 
> 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be 
> less reliable with little benefit over 7 mode at this point in time.

Quite... Functionality wise, C-Mode isn't finished yet either, there's quite 
some work for NTAP to do still.  And it's not stable in the proper sense of 
the word -- 8.1 7-mode isn't stable yet either.

> In fact, I'd probably wager that there will be a minimal performance 
> degradation too.

There is. But it's not minimal, not in any sense of the word. With C-mode 
there's "local" and "remote" array access (the back-ends below the heads so 
to speak) and the additional latency induced by the cluster network is big. 
Really big. In use cases where you need to be down at 1 ms avg internal 
latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, 
you can just forget about C-Mode yet.  *Maybe* some time in the future, 
maybe it can be polished until 8.2Px or 8.2.1 is out...  We'll see.

> Less speed and reliability is a hard sell and that's 
> before you even think about how difficult it'd be to move to C mode 
> without buying more hardware.

Somehow "moving" to C-Mode without deploying a whole new system from scratch 
is unthinkable from there I stand.  But of course it depends on the use case 
and other details.

I'd argue that at this point at least and for the next 24 months C-mode is 
pretty much like deploying a completely new (from a different vendor) 
storage system at your site in your environment, effectively potentiually 
doubling the complexity to handle for your storage ops ppl. Remember the CLI 
is completely different as well (for the large group of us who need/want to 
use that a lot, suppose a no of customers use the Web GUI way of controlling 
Filers to 100%)

/M
_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

In fairness, I used C-Mode at my last place and I can say a few things.

- the new cli isn't has hard to pick up as you might think.  I feel
within a 3 weeks or a month I was pretty comfortable with it.
   - there is a back way into the old ontap cli (it's not exactly the
same, but close) for some debugging things.
- the latency isn't nearly as bad as you might think.  We had Nexus
5k's and many 10g links, and worked pretty smoothly.
- While stability wasn't 100%, the failovers worked as expected and
filesystems never stopped being served.  This was 8.0 C-Mode
- Moving data between cfo pairs is slick and awesome.  Highly
recommend 64bit aggrs.

All that said, I don't think a C-Mode cluster should be pushed unless
4 nodes are in it, two cfo pairs is what every c-mode cluster should
start as.  Two nodes doesn't make a heck of a lot of sense.  I now
live in a sea of Isilon and comparing and contrasting the two
platforms is pretty interesting.

Hope that helps,

-Blake

On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman
<michael.bergman <at> ericsson.com> wrote:
> Darren Sykes wrote:
>> While I wouldn't disagree that it's a good product, having run GX, then
>> 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be
>> less reliable with little benefit over 7 mode at this point in time.
>
> Quite... Functionality wise, C-Mode isn't finished yet either, there's quite
> some work for NTAP to do still.  And it's not stable in the proper sense of
> the word -- 8.1 7-mode isn't stable yet either.
>
>> In fact, I'd probably wager that there will be a minimal performance
>> degradation too.
>
> There is. But it's not minimal, not in any sense of the word. With C-mode
> there's "local" and "remote" array access (the back-ends below the heads so
> to speak) and the additional latency induced by the cluster network is big.
> Really big. In use cases where you need to be down at 1 ms avg internal
> latency for NFS (with the help of PAM-II etc), for a fair NFSops load point,
> you can just forget about C-Mode yet.  *Maybe* some time in the future,
> maybe it can be polished until 8.2Px or 8.2.1 is out...  We'll see.
>
>> Less speed and reliability is a hard sell and that's
>> before you even think about how difficult it'd be to move to C mode
>> without buying more hardware.
>
> Somehow "moving" to C-Mode without deploying a whole new system from scratch
> is unthinkable from there I stand.  But of course it depends on the use case
> and other details.
>
> I'd argue that at this point at least and for the next 24 months C-mode is
> pretty much like deploying a completely new (from a different vendor)
> storage system at your site in your environment, effectively potentiually
> doubling the complexity to handle for your storage ops ppl. Remember the CLI
> is completely different as well (for the large group of us who need/want to
> use that a lot, suppose a no of customers use the Web GUI way of controlling
> Filers to 100%)
>
> /M
> _______________________________________________
> Toasters mailing list
> Toasters <at> teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

I was trying to be diplomatic, seems Michael has lowered the bar on that count :)

I agree with you both. I was very concerned about the latency when accessing remote dblades in our
environment since we have lots of very small files where the difference is highlighted. There was
supposed to be significant improvements in 8.1 - I should really re-run the tests to prove that. 

At the moment we attempt to work around the problem by tracking where volumes live and mounting the nblade
local to the dblade. Admittedly, that's not the most elegant solution and there is a lag as volumes move. 

I wonder how that latency will impact the performance of infinite volumes in 8.1C and beyond? 

We also have Isilons and their infiniband backend proved extremely good in reducing the overhead of access
remote nodes. Not that they don't have other performance gotchas. 

Darren

-----Original Message-----
From: toasters-bounces <at> teaparty.net [mailto:toasters-bounces <at> teaparty.net] On Behalf Of Blake Golliher
Sent: 19 June 2012 17:43
To: Michael Bergman
Cc: Toasters
Subject: Re: Cluster mode - Market vs field

In fairness, I used C-Mode at my last place and I can say a few things.

- the new cli isn't has hard to pick up as you might think.  I feel
within a 3 weeks or a month I was pretty comfortable with it.
   - there is a back way into the old ontap cli (it's not exactly the
same, but close) for some debugging things.
- the latency isn't nearly as bad as you might think.  We had Nexus
5k's and many 10g links, and worked pretty smoothly.
- While stability wasn't 100%, the failovers worked as expected and
filesystems never stopped being served.  This was 8.0 C-Mode
- Moving data between cfo pairs is slick and awesome.  Highly
recommend 64bit aggrs.

All that said, I don't think a C-Mode cluster should be pushed unless
4 nodes are in it, two cfo pairs is what every c-mode cluster should
start as.  Two nodes doesn't make a heck of a lot of sense.  I now
live in a sea of Isilon and comparing and contrasting the two
platforms is pretty interesting.

Hope that helps,

-Blake

On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman
<michael.bergman <at> ericsson.com> wrote:
> Darren Sykes wrote:
>> While I wouldn't disagree that it's a good product, having run GX, then
>> 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be
>> less reliable with little benefit over 7 mode at this point in time.
>
> Quite... Functionality wise, C-Mode isn't finished yet either, there's quite
> some work for NTAP to do still.  And it's not stable in the proper sense of
> the word -- 8.1 7-mode isn't stable yet either.
>
>> In fact, I'd probably wager that there will be a minimal performance
>> degradation too.
>
> There is. But it's not minimal, not in any sense of the word. With C-mode
> there's "local" and "remote" array access (the back-ends below the heads so
> to speak) and the additional latency induced by the cluster network is big.
> Really big. In use cases where you need to be down at 1 ms avg internal
> latency for NFS (with the help of PAM-II etc), for a fair NFSops load point,
> you can just forget about C-Mode yet.  *Maybe* some time in the future,
> maybe it can be polished until 8.2Px or 8.2.1 is out...  We'll see.
>
>> Less speed and reliability is a hard sell and that's
>> before you even think about how difficult it'd be to move to C mode
>> without buying more hardware.
>
> Somehow "moving" to C-Mode without deploying a whole new system from scratch
> is unthinkable from there I stand.  But of course it depends on the use case
> and other details.
>
> I'd argue that at this point at least and for the next 24 months C-mode is
> pretty much like deploying a completely new (from a different vendor)
> storage system at your site in your environment, effectively potentiually
> doubling the complexity to handle for your storage ops ppl. Remember the CLI
> is completely different as well (for the large group of us who need/want to
> use that a lot, suppose a no of customers use the Web GUI way of controlling
> Filers to 100%)
>
> /M
> _______________________________________________
> Toasters mailing list
> Toasters <at> teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

 To report this email as spam click https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .

Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number
4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ,
United Kingdom
More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and
read our blog at www.csr.com/blog

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

Someone mentioned below "With C-mode there's "local" and "remote" array
access". Actually, "remote" access would never happen when LIFs are
configured on each node (which is the recommended best practice
configuration). The only case when you'd get remote traffic thru the
interconnect 10GigE network is when you need to shutdown a node for an
upgrade for example, or when you move a volume to a node where you don't
have a LIF configured (but you could also move the LIF along with the
volume or just create a new LIF on the destination node and that would
provide a direct path to the new destination node for the volume; one LIF
on each node is the bottom line recommended best practice).

Regarding the "performance impact" of cluster-mode, you may want to check
the latest SPC-1 benchmark results with FAS 6240 in cluster-mode:
http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp/A001
15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executive-sum
mary.pdf
Regarding the new CLI, I really like it. Conceptually, it's similar to
Cisco's CLI (contextual commands, contextual help, tab to complete the
command or get help). The tab and help features are really great.

I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out.
Perhaps, download the cluster-mode vSim first and try it on your laptop.
ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system
anymore. There's a lot of goodness to it now and it's stable and really
easy to work with. You could check the Data ONTAP Cluster-Mode
Administration course begin taught by NetApp training partners to test
drive it further if you wish.

Cheers,
Sil Angelescu

On 6/19/12 12:57 PM, "Darren Sykes" <Darren.Sykes <at> csr.com> wrote:

>I was trying to be diplomatic, seems Michael has lowered the bar on that
>count :)
>
>I agree with you both. I was very concerned about the latency when
>accessing remote dblades in our environment since we have lots of very
>small files where the difference is highlighted. There was supposed to be
>significant improvements in 8.1 - I should really re-run the tests to
>prove that. 
>
>At the moment we attempt to work around the problem by tracking where
>volumes live and mounting the nblade local to the dblade. Admittedly,
>that's not the most elegant solution and there is a lag as volumes move.
>
>I wonder how that latency will impact the performance of infinite volumes
>in 8.1C and beyond?
>
>We also have Isilons and their infiniband backend proved extremely good
>in reducing the overhead of access remote nodes. Not that they don't have
>other performance gotchas.
>
>Darren
>
>
>-----Original Message-----
>From: toasters-bounces <at> teaparty.net
>[mailto:toasters-bounces <at> teaparty.net] On Behalf Of Blake Golliher
>Sent: 19 June 2012 17:43
>To: Michael Bergman
>Cc: Toasters
>Subject: Re: Cluster mode - Market vs field
>
>In fairness, I used C-Mode at my last place and I can say a few things.
>
>- the new cli isn't has hard to pick up as you might think.  I feel
>within a 3 weeks or a month I was pretty comfortable with it.
>   - there is a back way into the old ontap cli (it's not exactly the
>same, but close) for some debugging things.
>- the latency isn't nearly as bad as you might think.  We had Nexus
>5k's and many 10g links, and worked pretty smoothly.
>- While stability wasn't 100%, the failovers worked as expected and
>filesystems never stopped being served.  This was 8.0 C-Mode
>- Moving data between cfo pairs is slick and awesome.  Highly
>recommend 64bit aggrs.
>
>All that said, I don't think a C-Mode cluster should be pushed unless
>4 nodes are in it, two cfo pairs is what every c-mode cluster should
>start as.  Two nodes doesn't make a heck of a lot of sense.  I now
>live in a sea of Isilon and comparing and contrasting the two
>platforms is pretty interesting.
>
>Hope that helps,
>
>-Blake
>
>
>On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman
><michael.bergman <at> ericsson.com> wrote:
>> Darren Sykes wrote:
>>> While I wouldn't disagree that it's a good product, having run GX, then
>>> 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be
>>> less reliable with little benefit over 7 mode at this point in time.
>>
>> Quite... Functionality wise, C-Mode isn't finished yet either, there's
>>quite
>> some work for NTAP to do still.  And it's not stable in the proper
>>sense of
>> the word -- 8.1 7-mode isn't stable yet either.
>>
>>> In fact, I'd probably wager that there will be a minimal performance
>>> degradation too.
>>
>> There is. But it's not minimal, not in any sense of the word. With
>>C-mode
>> there's "local" and "remote" array access (the back-ends below the
>>heads so
>> to speak) and the additional latency induced by the cluster network is
>>big.
>> Really big. In use cases where you need to be down at 1 ms avg internal
>> latency for NFS (with the help of PAM-II etc), for a fair NFSops load
>>point,
>> you can just forget about C-Mode yet.  *Maybe* some time in the future,
>> maybe it can be polished until 8.2Px or 8.2.1 is out...  We'll see.
>>
>>> Less speed and reliability is a hard sell and that's
>>> before you even think about how difficult it'd be to move to C mode
>>> without buying more hardware.
>>
>> Somehow "moving" to C-Mode without deploying a whole new system from
>>scratch
>> is unthinkable from there I stand.  But of course it depends on the use
>>case
>> and other details.
>>
>> I'd argue that at this point at least and for the next 24 months C-mode
>>is
>> pretty much like deploying a completely new (from a different vendor)
>> storage system at your site in your environment, effectively
>>potentiually
>> doubling the complexity to handle for your storage ops ppl. Remember
>>the CLI
>> is completely different as well (for the large group of us who
>>need/want to
>> use that a lot, suppose a no of customers use the Web GUI way of
>>controlling
>> Filers to 100%)
>>
>> /M
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>
>_______________________________________________
>Toasters mailing list
>Toasters <at> teaparty.net
>http://www.teaparty.net/mailman/listinfo/toasters
>
>
> To report this email as spam click
>https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
>
>
>Member of the CSR plc group of companies. CSR plc registered in England
>and Wales, registered number 4187346, registered office Churchill House,
>Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
>More information can be found at www.csr.com. Follow CSR on Twitter at
>http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
>
>_______________________________________________
>Toasters mailing list
>Toasters <at> teaparty.net
>http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

Angelescu, Silviu (NetApp) wrote:
> Someone mentioned below "With C-mode there's "local" and "remote" array
> access". Actually, "remote" access would never happen when LIFs are
> configured on each node (which is the recommended best practice
> configuration). 

That someone was me. If you consider an NFSv3 based (not NFSv4 with 
referrals or v4.1 with pNFS extension) environment with many 100s of NFS 
clients and they have > 100 NFS mounts each to various places in the GNS 
from the C-mode cluster, and they cannot really ever be restarted all at 
once except under exceptional circumstances (meaning: re-mount happens very 
rarely for a large portion of the data...)

Even if LIFs are configured on each node, as soon as you move things around 
in the back-end, you'll get lots of cross Cluster Network accesses. It's 
pretty much unavoidable the way I see things

> The only case when you'd get remote traffic thru the
> interconnect 10GigE network is when you need to shutdown a node for an
> upgrade for example, or when you move a volume to a node where you don't
> have a LIF configured (but you could also move the LIF along with the
> volume or just create a new LIF on the destination node and that would
> provide a direct path to the new destination node for the volume; one LIF
> on each node is the bottom line recommended best practice).

That's fine but in the above scenario i described it won't really help that much

> Regarding the "performance impact" of cluster-mode, you may want to check
> the latest SPC-1 benchmark results with FAS 6240 in cluster-mode:
> http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp/A001
> 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executive-sum
> mary.pdf

It's a nice result, comparable to any other major "Tier 1" Block Storage 
vendor out there

The latency impact of C-mode compared to 7-mode cannot be juged from that 
data though, simply because there's no published SPC-1 benchmark for a 
7-mode "single node" (or HA cluster) FAS6240.  If there were, then I would 
absolutely do that comparison for FC Block Storage use case as well, not 
just SPECsfs2008

Regards,
/M
_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

I'll back up what has already been said - there's absolutely no guarantee you'll access a local dblade
without pnfs.

The other thing to consider is infinite volumes - with no nfs4 support, by nature you'll use the cluster
network most of the time. 

There's no need for a sim - we're running 8.1p1 in our live environment, and will run 8.1.1RC in dev.

Darren 
Sent from my iPhone

On 19 Jun 2012, at 18:33, "Angelescu, Silviu" <Silviu.Angelescu <at> netapp.com> wrote:

> Someone mentioned below "With C-mode there's "local" and "remote" array
> access". Actually, "remote" access would never happen when LIFs are
> configured on each node (which is the recommended best practice
> configuration). The only case when you'd get remote traffic thru the
> interconnect 10GigE network is when you need to shutdown a node for an
> upgrade for example, or when you move a volume to a node where you don't
> have a LIF configured (but you could also move the LIF along with the
> volume or just create a new LIF on the destination node and that would
> provide a direct path to the new destination node for the volume; one LIF
> on each node is the bottom line recommended best practice).
> 
> Regarding the "performance impact" of cluster-mode, you may want to check
> the latest SPC-1 benchmark results with FAS 6240 in cluster-mode:
> http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp/A001
> 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executive-sum
> mary.pdf
> Regarding the new CLI, I really like it. Conceptually, it's similar to
> Cisco's CLI (contextual commands, contextual help, tab to complete the
> command or get help). The tab and help features are really great.
> 
> I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out.
> Perhaps, download the cluster-mode vSim first and try it on your laptop.
> ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system
> anymore. There's a lot of goodness to it now and it's stable and really
> easy to work with. You could check the Data ONTAP Cluster-Mode
> Administration course begin taught by NetApp training partners to test
> drive it further if you wish.
> 
> Cheers,
> Sil Angelescu
> 
> 
> 
> 
> On 6/19/12 12:57 PM, "Darren Sykes" <Darren.Sykes <at> csr.com> wrote:
> 
>> I was trying to be diplomatic, seems Michael has lowered the bar on that
>> count :)
>> 
>> I agree with you both. I was very concerned about the latency when
>> accessing remote dblades in our environment since we have lots of very
>> small files where the difference is highlighted. There was supposed to be
>> significant improvements in 8.1 - I should really re-run the tests to
>> prove that. 
>> 
>> At the moment we attempt to work around the problem by tracking where
>> volumes live and mounting the nblade local to the dblade. Admittedly,
>> that's not the most elegant solution and there is a lag as volumes move.
>> 
>> I wonder how that latency will impact the performance of infinite volumes
>> in 8.1C and beyond?
>> 
>> We also have Isilons and their infiniband backend proved extremely good
>> in reducing the overhead of access remote nodes. Not that they don't have
>> other performance gotchas.
>> 
>> Darren
>> 
>> 
>> -----Original Message-----
>> From: toasters-bounces <at> teaparty.net
>> [mailto:toasters-bounces <at> teaparty.net] On Behalf Of Blake Golliher
>> Sent: 19 June 2012 17:43
>> To: Michael Bergman
>> Cc: Toasters
>> Subject: Re: Cluster mode - Market vs field
>> 
>> In fairness, I used C-Mode at my last place and I can say a few things.
>> 
>> - the new cli isn't has hard to pick up as you might think.  I feel
>> within a 3 weeks or a month I was pretty comfortable with it.
>>  - there is a back way into the old ontap cli (it's not exactly the
>> same, but close) for some debugging things.
>> - the latency isn't nearly as bad as you might think.  We had Nexus
>> 5k's and many 10g links, and worked pretty smoothly.
>> - While stability wasn't 100%, the failovers worked as expected and
>> filesystems never stopped being served.  This was 8.0 C-Mode
>> - Moving data between cfo pairs is slick and awesome.  Highly
>> recommend 64bit aggrs.
>> 
>> All that said, I don't think a C-Mode cluster should be pushed unless
>> 4 nodes are in it, two cfo pairs is what every c-mode cluster should
>> start as.  Two nodes doesn't make a heck of a lot of sense.  I now
>> live in a sea of Isilon and comparing and contrasting the two
>> platforms is pretty interesting.
>> 
>> Hope that helps,
>> 
>> -Blake
>> 
>> 
>> On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman
>> <michael.bergman <at> ericsson.com> wrote:
>>> Darren Sykes wrote:
>>>> While I wouldn't disagree that it's a good product, having run GX, then
>>>> 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be
>>>> less reliable with little benefit over 7 mode at this point in time.
>>> 
>>> Quite... Functionality wise, C-Mode isn't finished yet either, there's
>>> quite
>>> some work for NTAP to do still.  And it's not stable in the proper
>>> sense of
>>> the word -- 8.1 7-mode isn't stable yet either.
>>> 
>>>> In fact, I'd probably wager that there will be a minimal performance
>>>> degradation too.
>>> 
>>> There is. But it's not minimal, not in any sense of the word. With
>>> C-mode
>>> there's "local" and "remote" array access (the back-ends below the
>>> heads so
>>> to speak) and the additional latency induced by the cluster network is
>>> big.
>>> Really big. In use cases where you need to be down at 1 ms avg internal
>>> latency for NFS (with the help of PAM-II etc), for a fair NFSops load
>>> point,
>>> you can just forget about C-Mode yet.  *Maybe* some time in the future,
>>> maybe it can be polished until 8.2Px or 8.2.1 is out...  We'll see.
>>> 
>>>> Less speed and reliability is a hard sell and that's
>>>> before you even think about how difficult it'd be to move to C mode
>>>> without buying more hardware.
>>> 
>>> Somehow "moving" to C-Mode without deploying a whole new system from
>>> scratch
>>> is unthinkable from there I stand.  But of course it depends on the use
>>> case
>>> and other details.
>>> 
>>> I'd argue that at this point at least and for the next 24 months C-mode
>>> is
>>> pretty much like deploying a completely new (from a different vendor)
>>> storage system at your site in your environment, effectively
>>> potentiually
>>> doubling the complexity to handle for your storage ops ppl. Remember
>>> the CLI
>>> is completely different as well (for the large group of us who
>>> need/want to
>>> use that a lot, suppose a no of customers use the Web GUI way of
>>> controlling
>>> Filers to 100%)
>>> 
>>> /M
>>> _______________________________________________
>>> Toasters mailing list
>>> Toasters <at> teaparty.net
>>> http://www.teaparty.net/mailman/listinfo/toasters
>> 
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>> 
>> 
>> To report this email as spam click
>> https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
>> 
>> 
>> Member of the CSR plc group of companies. CSR plc registered in England
>> and Wales, registered number 4187346, registered office Churchill House,
>> Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
>> More information can be found at www.csr.com. Follow CSR on Twitter at
>> http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
>> 
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
> 

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

With the access methods available today (either RRDNS or the onboard load balancing) a client can land on
any lif regardless of the intended access target.  pNFS fixes that, but there's no telling how far away that
really is.

We see more traffic on the intercluster network than on the data network, regularly.  Granted, we move
volumes all the time (being able to load balance and expand capacity at will without 7 mode restraints is
fantastic) and as we add nodes that traffic increases, but even without that there is and will be a
significant amount of cluster traffic that is due to non-local lif's.

If your need is truly very high performance, cluster mode is not for you, at least not now.  For marginally
high performance and lower it performs well enough.  If you are able/willing to trade some of the features
of 7 mode (ok, let's be honest, it's quite a few features still) for the primary benefits of CM, you are
unlikely to be disappointed.  For the data set we put on CM we can do without those features, for now, but the
benefits of the namespace and transparent volume migrations for capacity or load balancing is a major
benefit.  Enough to outweigh all the features we've yet to see on CM.

Jeff Kennedy
Qualcomm, Incorporated
QCT Engineering Compute
858-651-6592

-----Original Message-----
From: toasters-bounces <at> teaparty.net [mailto:toasters-bounces <at> teaparty.net] On Behalf Of Darren Sykes
Sent: Tuesday, June 19, 2012 12:57 PM
To: Angelescu, Silviu
Cc: Michael Bergman; Toasters
Subject: Re: Cluster mode - Market vs field

I'll back up what has already been said - there's absolutely no guarantee you'll access a local dblade
without pnfs.

The other thing to consider is infinite volumes - with no nfs4 support, by nature you'll use the cluster
network most of the time. 

There's no need for a sim - we're running 8.1p1 in our live environment, and will run 8.1.1RC in dev.

Darren
Sent from my iPhone

On 19 Jun 2012, at 18:33, "Angelescu, Silviu" <Silviu.Angelescu <at> netapp.com> wrote:

> Someone mentioned below "With C-mode there's "local" and "remote" 
> array access". Actually, "remote" access would never happen when LIFs 
> are configured on each node (which is the recommended best practice 
> configuration). The only case when you'd get remote traffic thru the 
> interconnect 10GigE network is when you need to shutdown a node for an 
> upgrade for example, or when you move a volume to a node where you 
> don't have a LIF configured (but you could also move the LIF along 
> with the volume or just create a new LIF on the destination node and 
> that would provide a direct path to the new destination node for the 
> volume; one LIF on each node is the bottom line recommended best practice).
> 
> Regarding the "performance impact" of cluster-mode, you may want to 
> check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode:
> http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp
> /A001 
> 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executiv
> e-sum
> mary.pdf
> Regarding the new CLI, I really like it. Conceptually, it's similar to 
> Cisco's CLI (contextual commands, contextual help, tab to complete the 
> command or get help). The tab and help features are really great.
> 
> I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out.
> Perhaps, download the cluster-mode vSim first and try it on your laptop.
> ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system 
> anymore. There's a lot of goodness to it now and it's stable and 
> really easy to work with. You could check the Data ONTAP Cluster-Mode 
> Administration course begin taught by NetApp training partners to test 
> drive it further if you wish.
> 
> Cheers,
> Sil Angelescu
> 
> 
> 
> 
> On 6/19/12 12:57 PM, "Darren Sykes" <Darren.Sykes <at> csr.com> wrote:
> 
>> I was trying to be diplomatic, seems Michael has lowered the bar on 
>> that count :)
>> 
>> I agree with you both. I was very concerned about the latency when 
>> accessing remote dblades in our environment since we have lots of 
>> very small files where the difference is highlighted. There was 
>> supposed to be significant improvements in 8.1 - I should really 
>> re-run the tests to prove that.
>> 
>> At the moment we attempt to work around the problem by tracking where 
>> volumes live and mounting the nblade local to the dblade. Admittedly, 
>> that's not the most elegant solution and there is a lag as volumes move.
>> 
>> I wonder how that latency will impact the performance of infinite 
>> volumes in 8.1C and beyond?
>> 
>> We also have Isilons and their infiniband backend proved extremely 
>> good in reducing the overhead of access remote nodes. Not that they 
>> don't have other performance gotchas.
>> 
>> Darren
>> 
>> 
>> -----Original Message-----
>> From: toasters-bounces <at> teaparty.net
>> [mailto:toasters-bounces <at> teaparty.net] On Behalf Of Blake Golliher
>> Sent: 19 June 2012 17:43
>> To: Michael Bergman
>> Cc: Toasters
>> Subject: Re: Cluster mode - Market vs field
>> 
>> In fairness, I used C-Mode at my last place and I can say a few things.
>> 
>> - the new cli isn't has hard to pick up as you might think.  I feel 
>> within a 3 weeks or a month I was pretty comfortable with it.
>>  - there is a back way into the old ontap cli (it's not exactly the 
>> same, but close) for some debugging things.
>> - the latency isn't nearly as bad as you might think.  We had Nexus 
>> 5k's and many 10g links, and worked pretty smoothly.
>> - While stability wasn't 100%, the failovers worked as expected and 
>> filesystems never stopped being served.  This was 8.0 C-Mode
>> - Moving data between cfo pairs is slick and awesome.  Highly 
>> recommend 64bit aggrs.
>> 
>> All that said, I don't think a C-Mode cluster should be pushed unless
>> 4 nodes are in it, two cfo pairs is what every c-mode cluster should 
>> start as.  Two nodes doesn't make a heck of a lot of sense.  I now 
>> live in a sea of Isilon and comparing and contrasting the two 
>> platforms is pretty interesting.
>> 
>> Hope that helps,
>> 
>> -Blake
>> 
>> 
>> On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman 
>> <michael.bergman <at> ericsson.com> wrote:
>>> Darren Sykes wrote:
>>>> While I wouldn't disagree that it's a good product, having run GX, 
>>>> then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode 
>>>> would be less reliable with little benefit over 7 mode at this point in time.
>>> 
>>> Quite... Functionality wise, C-Mode isn't finished yet either, 
>>> there's quite some work for NTAP to do still.  And it's not stable 
>>> in the proper sense of the word -- 8.1 7-mode isn't stable yet 
>>> either.
>>> 
>>>> In fact, I'd probably wager that there will be a minimal 
>>>> performance degradation too.
>>> 
>>> There is. But it's not minimal, not in any sense of the word. With 
>>> C-mode there's "local" and "remote" array access (the back-ends 
>>> below the heads so to speak) and the additional latency induced by 
>>> the cluster network is big.
>>> Really big. In use cases where you need to be down at 1 ms avg 
>>> internal latency for NFS (with the help of PAM-II etc), for a fair 
>>> NFSops load point, you can just forget about C-Mode yet.  *Maybe* 
>>> some time in the future, maybe it can be polished until 8.2Px or 
>>> 8.2.1 is out...  We'll see.
>>> 
>>>> Less speed and reliability is a hard sell and that's before you 
>>>> even think about how difficult it'd be to move to C mode without 
>>>> buying more hardware.
>>> 
>>> Somehow "moving" to C-Mode without deploying a whole new system from 
>>> scratch is unthinkable from there I stand.  But of course it depends 
>>> on the use case and other details.
>>> 
>>> I'd argue that at this point at least and for the next 24 months 
>>> C-mode is pretty much like deploying a completely new (from a 
>>> different vendor) storage system at your site in your environment, 
>>> effectively potentiually doubling the complexity to handle for your 
>>> storage ops ppl. Remember the CLI is completely different as well 
>>> (for the large group of us who need/want to use that a lot, suppose 
>>> a no of customers use the Web GUI way of controlling Filers to 100%)
>>> 
>>> /M
>>> _______________________________________________
>>> Toasters mailing list
>>> Toasters <at> teaparty.net
>>> http://www.teaparty.net/mailman/listinfo/toasters
>> 
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>> 
>> 
>> To report this email as spam click
>> https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
>> 
>> 
>> Member of the CSR plc group of companies. CSR plc registered in 
>> England and Wales, registered number 4187346, registered office 
>> Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 
>> 0WZ, United Kingdom More information can be found at www.csr.com. 
>> Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog 
>> at www.csr.com/blog
>> 
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
> 

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

I'm really glad that my questions started a so interesting discussion.
And of course I'm also glad to read that speaking about "cluster mode" has to be based on a minimum of a two
nodes (four heads) because it could be a really nonsense to speak about GNS, grid storage, HPC and so on in a
"small" environment where budget and needs are no more that a 2220 with 10k SAS disks...
Isilon (that I know) has also be cited as the really competitor to compare with - and the same opinion about
NetApp comes from Isilon, part of but not from EMC of course .
But just because I know a lot of things about Isilon technology (from field) I still continue to think that
disk ownership can be the real constraint from a pure performance (in HPC word) point of view.

Thank you everybody

-----Messaggio originale-----
Da: toasters-bounces <at> teaparty.net [mailto:toasters-bounces <at> teaparty.net] Per conto di Kennedy, Jeffrey
Inviato: mercoledì 20 giugno 2012 00:47
A: Darren Sykes; Angelescu, Silviu
Cc: Michael Bergman; Toasters
Oggetto: RE: Cluster mode - Market vs field

With the access methods available today (either RRDNS or the onboard load balancing) a client can land on
any lif regardless of the intended access target.  pNFS fixes that, but there's no telling how far away that
really is.

We see more traffic on the intercluster network than on the data network, regularly.  Granted, we move
volumes all the time (being able to load balance and expand capacity at will without 7 mode restraints is
fantastic) and as we add nodes that traffic increases, but even without that there is and will be a
significant amount of cluster traffic that is due to non-local lif's.

If your need is truly very high performance, cluster mode is not for you, at least not now.  For marginally
high performance and lower it performs well enough.  If you are able/willing to trade some of the features
of 7 mode (ok, let's be honest, it's quite a few features still) for the primary benefits of CM, you are
unlikely to be disappointed.  For the data set we put on CM we can do without those features, for now, but the
benefits of the namespace and transparent volume migrations for capacity or load balancing is a major
benefit.  Enough to outweigh all the features we've yet to see on CM.

Jeff Kennedy
Qualcomm, Incorporated
QCT Engineering Compute
858-651-6592

-----Original Message-----
From: toasters-bounces <at> teaparty.net [mailto:toasters-bounces <at> teaparty.net] On Behalf Of Darren Sykes
Sent: Tuesday, June 19, 2012 12:57 PM
To: Angelescu, Silviu
Cc: Michael Bergman; Toasters
Subject: Re: Cluster mode - Market vs field

I'll back up what has already been said - there's absolutely no guarantee you'll access a local dblade
without pnfs.

The other thing to consider is infinite volumes - with no nfs4 support, by nature you'll use the cluster
network most of the time. 

There's no need for a sim - we're running 8.1p1 in our live environment, and will run 8.1.1RC in dev.

Darren
Sent from my iPhone

On 19 Jun 2012, at 18:33, "Angelescu, Silviu" <Silviu.Angelescu <at> netapp.com> wrote:

> Someone mentioned below "With C-mode there's "local" and "remote" 
> array access". Actually, "remote" access would never happen when LIFs 
> are configured on each node (which is the recommended best practice 
> configuration). The only case when you'd get remote traffic thru the 
> interconnect 10GigE network is when you need to shutdown a node for an 
> upgrade for example, or when you move a volume to a node where you 
> don't have a LIF configured (but you could also move the LIF along 
> with the volume or just create a new LIF on the destination node and 
> that would provide a direct path to the new destination node for the 
> volume; one LIF on each node is the bottom line recommended best practice).
> 
> Regarding the "performance impact" of cluster-mode, you may want to 
> check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode:
> http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp
> /A001
> 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executiv
> e-sum
> mary.pdf
> Regarding the new CLI, I really like it. Conceptually, it's similar to 
> Cisco's CLI (contextual commands, contextual help, tab to complete the 
> command or get help). The tab and help features are really great.
> 
> I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out.
> Perhaps, download the cluster-mode vSim first and try it on your laptop.
> ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system 
> anymore. There's a lot of goodness to it now and it's stable and 
> really easy to work with. You could check the Data ONTAP Cluster-Mode 
> Administration course begin taught by NetApp training partners to test 
> drive it further if you wish.
> 
> Cheers,
> Sil Angelescu
> 
> 
> 
> 
> On 6/19/12 12:57 PM, "Darren Sykes" <Darren.Sykes <at> csr.com> wrote:
> 
>> I was trying to be diplomatic, seems Michael has lowered the bar on 
>> that count :)
>> 
>> I agree with you both. I was very concerned about the latency when 
>> accessing remote dblades in our environment since we have lots of 
>> very small files where the difference is highlighted. There was 
>> supposed to be significant improvements in 8.1 - I should really 
>> re-run the tests to prove that.
>> 
>> At the moment we attempt to work around the problem by tracking where 
>> volumes live and mounting the nblade local to the dblade. Admittedly, 
>> that's not the most elegant solution and there is a lag as volumes move.
>> 
>> I wonder how that latency will impact the performance of infinite 
>> volumes in 8.1C and beyond?
>> 
>> We also have Isilons and their infiniband backend proved extremely 
>> good in reducing the overhead of access remote nodes. Not that they 
>> don't have other performance gotchas.
>> 
>> Darren
>> 
>> 
>> -----Original Message-----
>> From: toasters-bounces <at> teaparty.net
>> [mailto:toasters-bounces <at> teaparty.net] On Behalf Of Blake Golliher
>> Sent: 19 June 2012 17:43
>> To: Michael Bergman
>> Cc: Toasters
>> Subject: Re: Cluster mode - Market vs field
>> 
>> In fairness, I used C-Mode at my last place and I can say a few things.
>> 
>> - the new cli isn't has hard to pick up as you might think.  I feel 
>> within a 3 weeks or a month I was pretty comfortable with it.
>>  - there is a back way into the old ontap cli (it's not exactly the 
>> same, but close) for some debugging things.
>> - the latency isn't nearly as bad as you might think.  We had Nexus 
>> 5k's and many 10g links, and worked pretty smoothly.
>> - While stability wasn't 100%, the failovers worked as expected and 
>> filesystems never stopped being served.  This was 8.0 C-Mode
>> - Moving data between cfo pairs is slick and awesome.  Highly 
>> recommend 64bit aggrs.
>> 
>> All that said, I don't think a C-Mode cluster should be pushed unless
>> 4 nodes are in it, two cfo pairs is what every c-mode cluster should 
>> start as.  Two nodes doesn't make a heck of a lot of sense.  I now 
>> live in a sea of Isilon and comparing and contrasting the two 
>> platforms is pretty interesting.
>> 
>> Hope that helps,
>> 
>> -Blake
>> 
>> 
>> On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman 
>> <michael.bergman <at> ericsson.com> wrote:
>>> Darren Sykes wrote:
>>>> While I wouldn't disagree that it's a good product, having run GX, 
>>>> then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode 
>>>> would be less reliable with little benefit over 7 mode at this point in time.
>>> 
>>> Quite... Functionality wise, C-Mode isn't finished yet either, 
>>> there's quite some work for NTAP to do still.  And it's not stable 
>>> in the proper sense of the word -- 8.1 7-mode isn't stable yet 
>>> either.
>>> 
>>>> In fact, I'd probably wager that there will be a minimal 
>>>> performance degradation too.
>>> 
>>> There is. But it's not minimal, not in any sense of the word. With 
>>> C-mode there's "local" and "remote" array access (the back-ends 
>>> below the heads so to speak) and the additional latency induced by 
>>> the cluster network is big.
>>> Really big. In use cases where you need to be down at 1 ms avg 
>>> internal latency for NFS (with the help of PAM-II etc), for a fair 
>>> NFSops load point, you can just forget about C-Mode yet.  *Maybe* 
>>> some time in the future, maybe it can be polished until 8.2Px or
>>> 8.2.1 is out...  We'll see.
>>> 
>>>> Less speed and reliability is a hard sell and that's before you 
>>>> even think about how difficult it'd be to move to C mode without 
>>>> buying more hardware.
>>> 
>>> Somehow "moving" to C-Mode without deploying a whole new system from 
>>> scratch is unthinkable from there I stand.  But of course it depends 
>>> on the use case and other details.
>>> 
>>> I'd argue that at this point at least and for the next 24 months 
>>> C-mode is pretty much like deploying a completely new (from a 
>>> different vendor) storage system at your site in your environment, 
>>> effectively potentiually doubling the complexity to handle for your 
>>> storage ops ppl. Remember the CLI is completely different as well 
>>> (for the large group of us who need/want to use that a lot, suppose 
>>> a no of customers use the Web GUI way of controlling Filers to 100%)
>>> 
>>> /M
>>> _______________________________________________
>>> Toasters mailing list
>>> Toasters <at> teaparty.net
>>> http://www.teaparty.net/mailman/listinfo/toasters
>> 
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>> 
>> 
>> To report this email as spam click
>> https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
>> 
>> 
>> Member of the CSR plc group of companies. CSR plc registered in 
>> England and Wales, registered number 4187346, registered office 
>> Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 
>> 0WZ, United Kingdom More information can be found at www.csr.com.
>> Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog 
>> at www.csr.com/blog
>> 
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
> 

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

I think that's the Netapp argument - regardless of the number of nodes, C mode is the future. 

I'm also not sure I understand your terminology. A cluster is either one or an even number of nodes. A node is
effectively a filer or a head (a node certainly isn't two heads/filers). 

Isilon is slightly different - the minimum cluster size is 3 nodes but you can have an odd number of nodes and
add to the cluster in singles. Isilon also doesn't have the concept of disk ownership as each node has its
own disks and doesn't share with any others. Resilience is achieved by either mirroring or adding parity
information for all data that is written. 

Netapp on the other hand will have the problem of disk ownership because data isn't replicated - resilience
is gained from using RAID within a pair of nodes. 

Darren

-----Original Message-----
From: Milazzo Giacomo [mailto:G.Milazzo <at> sinergy.it] 
Sent: 20 June 2012 13:23
To: Kennedy, Jeffrey; Darren Sykes; Angelescu, Silviu
Cc: Michael Bergman; Toasters
Subject: R: Cluster mode - Market vs field

I'm really glad that my questions started a so interesting discussion.
And of course I'm also glad to read that speaking about "cluster mode" has to be based on a minimum of a two
nodes (four heads) because it could be a really nonsense to speak about GNS, grid storage, HPC and so on in a
"small" environment where budget and needs are no more that a 2220 with 10k SAS disks...
Isilon (that I know) has also be cited as the really competitor to compare with - and the same opinion about
NetApp comes from Isilon, part of but not from EMC of course .
But just because I know a lot of things about Isilon technology (from field) I still continue to think that
disk ownership can be the real constraint from a pure performance (in HPC word) point of view.

Thank you everybody

-----Messaggio originale-----
Da: toasters-bounces <at> teaparty.net [mailto:toasters-bounces <at> teaparty.net] Per conto di Kennedy, Jeffrey
Inviato: mercoledì 20 giugno 2012 00:47
A: Darren Sykes; Angelescu, Silviu
Cc: Michael Bergman; Toasters
Oggetto: RE: Cluster mode - Market vs field

With the access methods available today (either RRDNS or the onboard load balancing) a client can land on
any lif regardless of the intended access target.  pNFS fixes that, but there's no telling how far away that
really is.

We see more traffic on the intercluster network than on the data network, regularly.  Granted, we move
volumes all the time (being able to load balance and expand capacity at will without 7 mode restraints is
fantastic) and as we add nodes that traffic increases, but even without that there is and will be a
significant amount of cluster traffic that is due to non-local lif's.

If your need is truly very high performance, cluster mode is not for you, at least not now.  For marginally
high performance and lower it performs well enough.  If you are able/willing to trade some of the features
of 7 mode (ok, let's be honest, it's quite a few features still) for the primary benefits of CM, you are
unlikely to be disappointed.  For the data set we put on CM we can do without those features, for now, but the
benefits of the namespace and transparent volume migrations for capacity or load balancing is a major
benefit.  Enough to outweigh all the features we've yet to see on CM.

Jeff Kennedy
Qualcomm, Incorporated
QCT Engineering Compute
858-651-6592

-----Original Message-----
From: toasters-bounces <at> teaparty.net [mailto:toasters-bounces <at> teaparty.net] On Behalf Of Darren Sykes
Sent: Tuesday, June 19, 2012 12:57 PM
To: Angelescu, Silviu
Cc: Michael Bergman; Toasters
Subject: Re: Cluster mode - Market vs field

I'll back up what has already been said - there's absolutely no guarantee you'll access a local dblade
without pnfs.

The other thing to consider is infinite volumes - with no nfs4 support, by nature you'll use the cluster
network most of the time. 

There's no need for a sim - we're running 8.1p1 in our live environment, and will run 8.1.1RC in dev.

Darren
Sent from my iPhone

On 19 Jun 2012, at 18:33, "Angelescu, Silviu" <Silviu.Angelescu <at> netapp.com> wrote:

> Someone mentioned below "With C-mode there's "local" and "remote" 
> array access". Actually, "remote" access would never happen when LIFs 
> are configured on each node (which is the recommended best practice 
> configuration). The only case when you'd get remote traffic thru the 
> interconnect 10GigE network is when you need to shutdown a node for an 
> upgrade for example, or when you move a volume to a node where you 
> don't have a LIF configured (but you could also move the LIF along 
> with the volume or just create a new LIF on the destination node and 
> that would provide a direct path to the new destination node for the 
> volume; one LIF on each node is the bottom line recommended best practice).
> 
> Regarding the "performance impact" of cluster-mode, you may want to 
> check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode:
> http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp
> /A001
> 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executiv
> e-sum
> mary.pdf
> Regarding the new CLI, I really like it. Conceptually, it's similar to 
> Cisco's CLI (contextual commands, contextual help, tab to complete the 
> command or get help). The tab and help features are really great.
> 
> I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out.
> Perhaps, download the cluster-mode vSim first and try it on your laptop.
> ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system 
> anymore. There's a lot of goodness to it now and it's stable and 
> really easy to work with. You could check the Data ONTAP Cluster-Mode 
> Administration course begin taught by NetApp training partners to test 
> drive it further if you wish.
> 
> Cheers,
> Sil Angelescu
> 
> 
> 
> 
> On 6/19/12 12:57 PM, "Darren Sykes" <Darren.Sykes <at> csr.com> wrote:
> 
>> I was trying to be diplomatic, seems Michael has lowered the bar on 
>> that count :)
>> 
>> I agree with you both. I was very concerned about the latency when 
>> accessing remote dblades in our environment since we have lots of 
>> very small files where the difference is highlighted. There was 
>> supposed to be significant improvements in 8.1 - I should really 
>> re-run the tests to prove that.
>> 
>> At the moment we attempt to work around the problem by tracking where 
>> volumes live and mounting the nblade local to the dblade. Admittedly, 
>> that's not the most elegant solution and there is a lag as volumes move.
>> 
>> I wonder how that latency will impact the performance of infinite 
>> volumes in 8.1C and beyond?
>> 
>> We also have Isilons and their infiniband backend proved extremely 
>> good in reducing the overhead of access remote nodes. Not that they 
>> don't have other performance gotchas.
>> 
>> Darren
>> 
>> 
>> -----Original Message-----
>> From: toasters-bounces <at> teaparty.net
>> [mailto:toasters-bounces <at> teaparty.net] On Behalf Of Blake Golliher
>> Sent: 19 June 2012 17:43
>> To: Michael Bergman
>> Cc: Toasters
>> Subject: Re: Cluster mode - Market vs field
>> 
>> In fairness, I used C-Mode at my last place and I can say a few things.
>> 
>> - the new cli isn't has hard to pick up as you might think.  I feel 
>> within a 3 weeks or a month I was pretty comfortable with it.
>>  - there is a back way into the old ontap cli (it's not exactly the 
>> same, but close) for some debugging things.
>> - the latency isn't nearly as bad as you might think.  We had Nexus 
>> 5k's and many 10g links, and worked pretty smoothly.
>> - While stability wasn't 100%, the failovers worked as expected and 
>> filesystems never stopped being served.  This was 8.0 C-Mode
>> - Moving data between cfo pairs is slick and awesome.  Highly 
>> recommend 64bit aggrs.
>> 
>> All that said, I don't think a C-Mode cluster should be pushed unless
>> 4 nodes are in it, two cfo pairs is what every c-mode cluster should 
>> start as.  Two nodes doesn't make a heck of a lot of sense.  I now 
>> live in a sea of Isilon and comparing and contrasting the two 
>> platforms is pretty interesting.
>> 
>> Hope that helps,
>> 
>> -Blake
>> 
>> 
>> On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman 
>> <michael.bergman <at> ericsson.com> wrote:
>>> Darren Sykes wrote:
>>>> While I wouldn't disagree that it's a good product, having run GX, 
>>>> then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode 
>>>> would be less reliable with little benefit over 7 mode at this point in time.
>>> 
>>> Quite... Functionality wise, C-Mode isn't finished yet either, 
>>> there's quite some work for NTAP to do still.  And it's not stable 
>>> in the proper sense of the word -- 8.1 7-mode isn't stable yet 
>>> either.
>>> 
>>>> In fact, I'd probably wager that there will be a minimal 
>>>> performance degradation too.
>>> 
>>> There is. But it's not minimal, not in any sense of the word. With 
>>> C-mode there's "local" and "remote" array access (the back-ends 
>>> below the heads so to speak) and the additional latency induced by 
>>> the cluster network is big.
>>> Really big. In use cases where you need to be down at 1 ms avg 
>>> internal latency for NFS (with the help of PAM-II etc), for a fair 
>>> NFSops load point, you can just forget about C-Mode yet.  *Maybe* 
>>> some time in the future, maybe it can be polished until 8.2Px or
>>> 8.2.1 is out...  We'll see.
>>> 
>>>> Less speed and reliability is a hard sell and that's before you 
>>>> even think about how difficult it'd be to move to C mode without 
>>>> buying more hardware.
>>> 
>>> Somehow "moving" to C-Mode without deploying a whole new system from 
>>> scratch is unthinkable from there I stand.  But of course it depends 
>>> on the use case and other details.
>>> 
>>> I'd argue that at this point at least and for the next 24 months 
>>> C-mode is pretty much like deploying a completely new (from a 
>>> different vendor) storage system at your site in your environment, 
>>> effectively potentiually doubling the complexity to handle for your 
>>> storage ops ppl. Remember the CLI is completely different as well 
>>> (for the large group of us who need/want to use that a lot, suppose 
>>> a no of customers use the Web GUI way of controlling Filers to 100%)
>>> 
>>> /M
>>> _______________________________________________
>>> Toasters mailing list
>>> Toasters <at> teaparty.net
>>> http://www.teaparty.net/mailman/listinfo/toasters
>> 
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>> 
>> 
>> To report this email as spam click
>> https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
>> 
>> 
>> Member of the CSR plc group of companies. CSR plc registered in 
>> England and Wales, registered number 4187346, registered office 
>> Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 
>> 0WZ, United Kingdom More information can be found at www.csr.com.
>> Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog 
>> at www.csr.com/blog
>> 
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
> 

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

Pardon the interjection, but I thought this may be of interest to those of you using 7 and looking to get a
quick introduction to Cluster-Mode.

"The new Introduction to Cluster-Mode for 7-Mode Administrators course provides an introduction to
Cluster-Mode, and is being offered  free of charge.  This 1 hour  Web-based training is a great way to
introduce 7-Mode familiar NetApp customers, employees and partners to the basics of Cluster-Mode, an
important Data ONTAP concept and key NetApp differentiator."

 Course Description:
http://learningcenter.netapp.com/content/public/production/course_descriptions/CRS-002697-01-A_CourseDescription.pdf 
 Enrollment: http://learningcenter.netapp.com/LC?ObjectType=WBT&ObjectID=00236611 (NOW login required)

Alex McDonald
NetApp, CTO Office
Industry Stds and Forums
twitter:  <at> alextangent
phone: +44 7795 046686

-----Original Message-----
From: toasters-bounces <at> teaparty.net [mailto:toasters-bounces <at> teaparty.net] On Behalf Of
Angelescu, Silviu
Sent: Tuesday, June 19, 2012 6:34 PM
To: Darren Sykes; Blake Golliher; Michael Bergman
Cc: Toasters
Subject: Re: Cluster mode - Market vs field

Someone mentioned below "With C-mode there's "local" and "remote" array access". Actually, "remote"
access would never happen when LIFs are configured on each node (which is the recommended best practice
configuration). The only case when you'd get remote traffic thru the interconnect 10GigE network is when
you need to shutdown a node for an upgrade for example, or when you move a volume to a node where you don't have
a LIF configured (but you could also move the LIF along with the volume or just create a new LIF on the
destination node and that would provide a direct path to the new destination node for the volume; one LIF on
each node is the bottom line recommended best practice).

Regarding the "performance impact" of cluster-mode, you may want to check the latest SPC-1 benchmark
results with FAS 6240 in cluster-mode:
http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp/A001
15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executive-sum
mary.pdf
Regarding the new CLI, I really like it. Conceptually, it's similar to Cisco's CLI (contextual commands,
contextual help, tab to complete the command or get help). The tab and help features are really great.

I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out.
Perhaps, download the cluster-mode vSim first and try it on your laptop.
ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system anymore. There's a lot of goodness
to it now and it's stable and really easy to work with. You could check the Data ONTAP Cluster-Mode
Administration course begin taught by NetApp training partners to test drive it further if you wish.

Cheers,
Sil Angelescu

On 6/19/12 12:57 PM, "Darren Sykes" <Darren.Sykes <at> csr.com> wrote:

>I was trying to be diplomatic, seems Michael has lowered the bar on 
>that count :)
>
>I agree with you both. I was very concerned about the latency when 
>accessing remote dblades in our environment since we have lots of very 
>small files where the difference is highlighted. There was supposed to 
>be significant improvements in 8.1 - I should really re-run the tests 
>to prove that.
>
>At the moment we attempt to work around the problem by tracking where 
>volumes live and mounting the nblade local to the dblade. Admittedly, 
>that's not the most elegant solution and there is a lag as volumes move.
>
>I wonder how that latency will impact the performance of infinite 
>volumes in 8.1C and beyond?
>
>We also have Isilons and their infiniband backend proved extremely good 
>in reducing the overhead of access remote nodes. Not that they don't 
>have other performance gotchas.
>
>Darren
>
>
>-----Original Message-----
>From: toasters-bounces <at> teaparty.net
>[mailto:toasters-bounces <at> teaparty.net] On Behalf Of Blake Golliher
>Sent: 19 June 2012 17:43
>To: Michael Bergman
>Cc: Toasters
>Subject: Re: Cluster mode - Market vs field
>
>In fairness, I used C-Mode at my last place and I can say a few things.
>
>- the new cli isn't has hard to pick up as you might think.  I feel 
>within a 3 weeks or a month I was pretty comfortable with it.
>   - there is a back way into the old ontap cli (it's not exactly the 
>same, but close) for some debugging things.
>- the latency isn't nearly as bad as you might think.  We had Nexus 
>5k's and many 10g links, and worked pretty smoothly.
>- While stability wasn't 100%, the failovers worked as expected and 
>filesystems never stopped being served.  This was 8.0 C-Mode
>- Moving data between cfo pairs is slick and awesome.  Highly recommend 
>64bit aggrs.
>
>All that said, I don't think a C-Mode cluster should be pushed unless
>4 nodes are in it, two cfo pairs is what every c-mode cluster should 
>start as.  Two nodes doesn't make a heck of a lot of sense.  I now live 
>in a sea of Isilon and comparing and contrasting the two platforms is 
>pretty interesting.
>
>Hope that helps,
>
>-Blake
>
>
>On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman 
><michael.bergman <at> ericsson.com> wrote:
>> Darren Sykes wrote:
>>> While I wouldn't disagree that it's a good product, having run GX, 
>>> then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode 
>>> would be less reliable with little benefit over 7 mode at this point in time.
>>
>> Quite... Functionality wise, C-Mode isn't finished yet either, 
>>there's quite  some work for NTAP to do still.  And it's not stable in 
>>the proper sense of  the word -- 8.1 7-mode isn't stable yet either.
>>
>>> In fact, I'd probably wager that there will be a minimal performance 
>>> degradation too.
>>
>> There is. But it's not minimal, not in any sense of the word. With 
>>C-mode  there's "local" and "remote" array access (the back-ends below 
>>the heads so  to speak) and the additional latency induced by the 
>>cluster network is big.
>> Really big. In use cases where you need to be down at 1 ms avg 
>>internal  latency for NFS (with the help of PAM-II etc), for a fair 
>>NFSops load point,  you can just forget about C-Mode yet.  *Maybe* 
>>some time in the future,  maybe it can be polished until 8.2Px or 
>>8.2.1 is out...  We'll see.
>>
>>> Less speed and reliability is a hard sell and that's before you even 
>>> think about how difficult it'd be to move to C mode without buying 
>>> more hardware.
>>
>> Somehow "moving" to C-Mode without deploying a whole new system from 
>>scratch  is unthinkable from there I stand.  But of course it depends 
>>on the use case  and other details.
>>
>> I'd argue that at this point at least and for the next 24 months 
>>C-mode is  pretty much like deploying a completely new (from a 
>>different vendor)  storage system at your site in your environment, 
>>effectively potentiually  doubling the complexity to handle for your 
>>storage ops ppl. Remember the CLI  is completely different as well 
>>(for the large group of us who need/want to  use that a lot, suppose a 
>>no of customers use the Web GUI way of controlling  Filers to 100%)
>>
>> /M
>> _______________________________________________
>> Toasters mailing list
>> Toasters <at> teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>
>_______________________________________________
>Toasters mailing list
>Toasters <at> teaparty.net
>http://www.teaparty.net/mailman/listinfo/toasters
>
>
> To report this email as spam click
>https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
>
>
>Member of the CSR plc group of companies. CSR plc registered in England 
>and Wales, registered number 4187346, registered office Churchill 
>House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United 
>Kingdom More information can be found at www.csr.com. Follow CSR on 
>Twitter at http://twitter.com/CSR_PLC and read our blog at 
>www.csr.com/blog
>
>_______________________________________________
>Toasters mailing list
>Toasters <at> teaparty.net
>http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

On Jun 17, 2012 12:08 PM, "Blackmor, Chris" <Chris.Blackmor <at> amd.com> wrote:
>
> With all the missteps NA has had getting C-mode out the door I don't think they can really push it on anyone.
Remember Auspex 1.8.  Where's Auspex now?

A very fond memory, if triste; the months that felt like years that I spent trying to master the art of
handholding the recovery of lock state after failover, and appreciating the cleverness of the multicast
tricks and nfs protocol.shortcuts, taught me lessons about how NFS really works (when it does) that will
last me.

H-A is great when it works, but I've rarely seen implementations that aren't seriously lower availability
than the single-point-of-failure system they try to replace.

Though I think what did for Auspex wasn't their ambitious H-A, it was NetApp's ability to cling tenatiously
to the Juggernaut as Moore's Law drove costs of processing, memory, and bandwidth down, and down, and ever down.
_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

I agree with you and Todd but I've another point of view.
I know very well other competitor (really ones) in the field of 'cluster' such as Isilon (and this one
considers a real competitor only NetApp). 
For Isilon and such companies cluster mean over all HPC, parallel f/s, single name spaces, grid computing,
they mean also to share network and other resources in a scale-out point of view.

I think for that reason that to propose CM to justify a 'never poweroff', 'move a single vol without
interruption' and so on is a very limited vision maybe because in the scale-out sharing of resource the
NetApp disk ownership is a constraint? Or more realistically because to speak of CM thinking to 2200
systems could be a nonsense (customer with limited budget and needs..

regards

-----Messaggio originale-----
Da: Bennett Todd [mailto:bent <at> latency.net] Per conto di Bennett Todd
Inviato: domenica 17 giugno 2012 18:27
A: Blackmor, Chris; Milazzo Giacomo
Cc: toasters <at> teaparty.net
Oggetto: Re: Cluster mode - Market vs field

On Jun 17, 2012 12:08 PM, "Blackmor, Chris" <Chris.Blackmor <at> amd.com> wrote:
>
> With all the missteps NA has had getting C-mode out the door I don't think they can really push it on anyone.
Remember Auspex 1.8.  Where's Auspex now?

A very fond memory, if triste; the months that felt like years that I spent trying to master the art of
handholding the recovery of lock state after failover, and appreciating the cleverness of the multicast
tricks and nfs protocol.shortcuts, taught me lessons about how NFS really works (when it does) that will
last me.

H-A is great when it works, but I've rarely seen implementations that aren't seriously lower availability
than the single-point-of-failure system they try to replace.

Though I think what did for Auspex wasn't their ambitious H-A, it was NetApp's ability to cling tenatiously
to the Juggernaut as Moore's Law drove costs of processing, memory, and bandwidth down, and down, and ever down.

_______________________________________________
Toasters mailing list
Toasters <at> teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters