headers
timothyhobbs | 10 Nov 2012 20:49
Picon
Favicon
Gravatar

Hard drive thrashing with modern controllers

Hello,
I have a very general question that doesn't exactly relate to Haskell.  Do I need to worry about hard drive thrashing with a modern controller?  I understand, that such "thrashing" can significantly reduce the lifespan of modern SSD drives.

For example, if I do:

import Control.Monad
foo = do
 forever $ writeFile "filename.foo" "Hello world!"

will that destroy those sectors of my SSD after the rated 3000 write cycles?

The reason I ask, is that I have a program here where I would like to save a set of files after every change/operation and I'm wondering if I should check if the files I'm saving have actually changed or if this will be handled by the OS...

Thank you
Timothy
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Permalink | Reply |
headers
Brandon Allbery | 10 Nov 2012 20:55
Picon
Gravatar

Re: Hard drive thrashing with modern controllers

On Sat, Nov 10, 2012 at 2:49 PM, <timothyhobbs <at> seznam.cz> wrote:
import Control.Monad
foo = do
 forever $ writeFile "filename.foo" "Hello world!"

will that destroy those sectors of my SSD after the rated 3000 write cycles?

Check your OS; while the firmware of modern SSD devices does much of the work of rotating blocks of Flash around to mitigate this, the OS can help by using a TRIM operation.  http://en.wikipedia.org/wiki/TRIM

I personally would consider that rapidly changing files should be kept somewhere else such as tmpfs with periodic snapshots to nonvolatile storage.

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix/linux, openafs, kerberos, infrastructure          http://sinenomine.net 

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Permalink | Reply |
headers
timothyhobbs | 10 Nov 2012 21:02
Picon
Favicon
Gravatar

Re: Hard drive thrashing with modern controllers

Hmm.  I was hoping for good news that things had changed for the better :( .  I want these files to be on the disk so I don't lose data in the case of failure.  A common solution here is to acidify the program, but that is not acceptable from a usability standpoint.  I don't want to have the user mess around with swap files and the like.  When something goes wrong, I want to seamlessly start up where we left off without the user even knowing that something out of the ordinary happened.  A tmpfs will do nothing for this case :)

Timothy


---------- Původní zpráva ----------
Od: Brandon Allbery <allbery.b <at> gmail.com>
Datum: 10. 11. 2012
Předmět: Re: [Haskell-cafe] Hard drive thrashing with modern controllers

On Sat, Nov 10, 2012 at 2:49 PM, <timothyhobbs <at> seznam.cz> wrote:
import Control.Monad
foo = do
 forever $ writeFile "filename.foo" "Hello world!"

will that destroy those sectors of my SSD after the rated 3000 write cycles?

Check your OS; while the firmware of modern SSD devices does much of the work of rotating blocks of Flash around to mitigate this, the OS can help by using a TRIM operation.  http://en.wikipedia.org/wiki/TRIM

I personally would consider that rapidly changing files should be kept somewhere else such as tmpfs with periodic snapshots to nonvolatile storage.

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix/linux, openafs, kerberos, infrastructure          http://sinenomine.net 

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Permalink | Reply |
headers
Brandon Allbery | 10 Nov 2012 21:26
Picon
Gravatar

Re: Hard drive thrashing with modern controllers

On Sat, Nov 10, 2012 at 3:02 PM, <timothyhobbs <at> seznam.cz> wrote:
Hmm.  I was hoping for good news that things had changed for the better :( .  I want these files to be on the disk so I don't lose data in the case of failure.  A common solution here is to acidify the program, but that is not acceptable from a usability standpoint.  I don't want to have the user mess around with swap files and the like.  When something goes wrong, I want to seamlessly start up where we left off without the user even knowing that something out of the ordinary happened.  A tmpfs will do nothing for this case :)

Maybe what you want to do is to write data every [blocksize] (often 4K but it may depend to some extent on the SSD's internal block size, which unfortunately you may not be able to determine easily) in the file and periodically truncate and start from the beginning.  To restart, seek backwards from the end of the file.

Alternatively this may be the point where you try to write stuff to an easily and cheaply replaced thumb drive or SD card.

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix/linux, openafs, kerberos, infrastructure          http://sinenomine.net 

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Permalink | Reply |
headers
Ketil Malde | 12 Nov 2012 10:21
Gravatar

Re: Hard drive thrashing with modern controllers

timothyhobbs <at> seznam.cz writes:

> import Control.Monad
> foo = do
>   forever $ writeFile "filename.foo" "Hello world!"

I could be wrong, but I suspect this is unlikely to result in (hardly)
any disk operations at all, as long as there is _any_ write caching in
the system.

> will that destroy those sectors of my SSD after the rated 3000 write cycles?

The SSD firmware will even this out by shuffling around the exact flash
blocks that are used, so that's 3K write cycles for each block, spread
out over hundreds of thousands of blocks - so your SSD should survive
perhaps a billion writes in total.

> The reason I ask, is that I have a program here where I would like to save a
> set of files after every change/operation and I'm wondering if I should 
> check if the files I'm saving have actually changed or if this will be 
> handled by the OS...

Unless the loss of a file would be crucial, I wouldn't worry about it.
If/when you wear out the SSD, a replacement will probably be a trivial
cost.

-k
Permalink | Reply |
headers
Brandon Allbery | 12 Nov 2012 21:54
Picon
Gravatar

Re: Hard drive thrashing with modern controllers

On Mon, Nov 12, 2012 at 4:21 AM, Ketil Malde <ketil <at> malde.org> wrote:
timothyhobbs <at> seznam.cz writes:
> import Control.Monad
> foo = do
>   forever $ writeFile "filename.foo" "Hello world!"

I could be wrong, but I suspect this is unlikely to result in (hardly)
any disk operations at all, as long as there is _any_ write caching in
the system.

Maybe no data blocks, but the metadata will be updated.

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix/linux, openafs, kerberos, infrastructure          http://sinenomine.net 

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Permalink | Reply |

Gmane