Hi, Is it possible to have ZFS or UFS to zero out the deallocated blocks to facilitate VM disk compaction? Pete _______________________________________________ freebsd-fs <at> freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe <at> freebsd.org"
Sounds like you may be talking about delete / trim support?
If so UFS supports BIO_DELETE via -t on newfs or on an existing
FS via tunefs -t enable
ZFS doesn't currently BIO_DELETE but should soon as we've sponsored
a project to add it, just waiting on it being committed. I've
got a patch against 8.3-RELEASE if your interested in testing.
It does require support from the underlying layers. cam ata
supports it but cam da doesn't but again we have a patch for
that if you want to test.
Regards
Steve
----- Original Message -----
From: "Petri Helenius" <petri <at> helenius.fi>
To: <freebsd-fs <at> freebsd.org>
Sent: Monday, August 13, 2012 7:01 AM
Subject: zero deleted blocks
Hi,
Is it possible to have ZFS or UFS to zero out the deallocated blocks to facilitate VM disk compaction?
Pete
_______________________________________________
freebsd-fs <at> freebsd.org mailing list
(Continue reading)
On Mon, Aug 13, 2012 at 09:01:57AM +0300, Petri Helenius wrote: > > Hi, > > Is it possible to have ZFS or UFS to zero out the deallocated blocks > to facilitate VM disk compaction? I think TRIM support in UFS can be extended to zero-out the deleted blocks, with relatively small amount of work. Basically, in ufs/ffs/ffs_alloc.c:ffs_blkfree(), a BIO_WRITE of zero buffer shall be issued instead of BIO_DELETE. More, it seems that zero_region can be often used as a source of zeroes.
In message <20120813084120.GA2352 <at> deviant.kiev.zoral.com.ua>, Konstantin Belous ov writes: >I think TRIM support in UFS can be extended to zero-out the deleted >blocks, with relatively small amount of work. Basically, in >ufs/ffs/ffs_alloc.c:ffs_blkfree(), a BIO_WRITE of zero buffer shall be >issued instead of BIO_DELETE. It would be a better idea to give geom_disk a per-disk option to convert BIO_DELETE to writes of zeros. -- -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk <at> FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. _______________________________________________ freebsd-fs <at> freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe <at> freebsd.org"
----- Original Message -----
From: "Poul-Henning Kamp" <phk <at> phk.freebsd.dk>
>>I think TRIM support in UFS can be extended to zero-out the deleted
>>blocks, with relatively small amount of work. Basically, in
>>ufs/ffs/ffs_alloc.c:ffs_blkfree(), a BIO_WRITE of zero buffer shall be
>>issued instead of BIO_DELETE.
>
> It would be a better idea to give geom_disk a per-disk option to
> convert BIO_DELETE to writes of zeros.
cam da already has this option with the sysctl:-
kern.cam.da.X.delete_method = ZERO
This however requires the underlying SCSI device supports
Write Same (WS10 or WS16) which seems to fairly rare.
I'm not aware of any an equivalent ATA command, but it would
be possible to have both use a standard write to achieve that.
If this was done care should be take with regards performance
as this will be significantly slower than the current methods.
That said if the underlying device "supports" trim / unmap
it could interpret that how it sees fit.
Regards
Steve
================================================
(Continue reading)
Obviously the most elegant way would be for KVM, Xen and VMware to support TRIM/UNMAP... Pete On Aug 13, 2012, at 11:58 , "Steven Hartland" <killing <at> multiplay.co.uk> wrote: > ----- Original Message ----- From: "Poul-Henning Kamp" <phk <at> phk.freebsd.dk> > > >>> I think TRIM support in UFS can be extended to zero-out the deleted >>> blocks, with relatively small amount of work. Basically, in >>> ufs/ffs/ffs_alloc.c:ffs_blkfree(), a BIO_WRITE of zero buffer shall be >>> issued instead of BIO_DELETE. >> It would be a better idea to give geom_disk a per-disk option to >> convert BIO_DELETE to writes of zeros. > > cam da already has this option with the sysctl:- > kern.cam.da.X.delete_method = ZERO > > This however requires the underlying SCSI device supports > Write Same (WS10 or WS16) which seems to fairly rare. > > I'm not aware of any an equivalent ATA command, but it would > be possible to have both use a standard write to achieve that. > > If this was done care should be take with regards performance > as this will be significantly slower than the current methods. > > That said if the underlying device "supports" trim / unmap(Continue reading)
----- Original Message -----
From: "Petri Helenius" <petri <at> helenius.fi>
> Obviously the most elegant way would be for KVM, Xen and
> VMware to support TRIM/UNMAP...
Indeed, I believe as thats exactly what thin provisioning
(unmap) was designed for.
Regards
Steve
================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is
addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or
otherwise disseminating it or any information contained in it.
In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster <at> multiplay.co.uk.
_______________________________________________
freebsd-fs <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe <at> freebsd.org"
2012/8/13 Petri Helenius <petri <at> helenius.fi>: > > Hi, > > Is it possible to have ZFS or UFS to zero out the deallocated blocks to facilitate VM disk compaction? > > Pete Did you try a classic "dd if=/dev/zero of=a-file-on-the-pool" ? This won't give good results if you do this on a filesystem with compression enabled. -- -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier <at> gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." _______________________________________________ freebsd-fs <at> freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe <at> freebsd.org"
On Aug 13, 2012, at 2:27 AM, Olivier Smedts <olivier <at> gid0.org> wrote: > 2012/8/13 Petri Helenius <petri <at> helenius.fi>: >> >> Hi, >> >> Is it possible to have ZFS or UFS to zero out the deallocated blocks to facilitate VM disk compaction? >> >> Pete > > Did you try a classic "dd if=/dev/zero of=a-file-on-the-pool" ? This > won't give good results if you do this on a filesystem with > compression enabled. ZFS uses copy on write (http://en.wikipedia.org/wiki/ZFS#Copy-on-write_transactional_model) so overwriting with zeroes will actually allocate new space to hold all the zeroed data and leave the old data mostly untouched. _______________________________________________ freebsd-fs <at> freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe <at> freebsd.org"
2012/8/13 Steven Schlansker <stevenschlansker <at> gmail.com>: > > On Aug 13, 2012, at 2:27 AM, Olivier Smedts <olivier <at> gid0.org> wrote: >> Did you try a classic "dd if=/dev/zero of=a-file-on-the-pool" ? This >> won't give good results if you do this on a filesystem with >> compression enabled. > > ZFS uses copy on write (http://en.wikipedia.org/wiki/ZFS#Copy-on-write_transactional_model) so overwriting with zeroes will actually allocate new space to hold all the zeroed data and leave the old data mostly untouched. Only until all the free space is consumed ! -- -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier <at> gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." _______________________________________________ freebsd-fs <at> freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe <at> freebsd.org"
----- Original Message ----- From: "Olivier Smedts" <olivier <at> gid0.org> > 2012/8/13 Steven Schlansker <stevenschlansker <at> gmail.com>: >> >> On Aug 13, 2012, at 2:27 AM, Olivier Smedts <olivier <at> gid0.org> wrote: >>> Did you try a classic "dd if=/dev/zero of=a-file-on-the-pool" ? This >>> won't give good results if you do this on a filesystem with >>> compression enabled. >> >> ZFS uses copy on write (http://en.wikipedia.org/wiki/ZFS#Copy-on-write_transactional_model) so overwriting with zeroes will >> actually allocate new space to hold all the zeroed data and leave the old data mostly untouched. > > Only until all the free space is consumed ! Also need to ensure compression isnt turned on and its not an device that does compression on disk / disk controller such as Sandforce based SSD's as that could resulting in something other than the intended behavour. ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster <at> multiplay.co.uk. _______________________________________________(Continue reading)
2012/8/13 Steven Hartland <killing <at> multiplay.co.uk>: > Also need to ensure compression isnt turned on and its not an device > that does compression on disk / disk controller such as Sandforce > based SSD's as that could resulting in something other than the > intended behavour. I think the "device" we're speaking of here is a flat file, as the OP wants to compact virtual machines' disks. -- -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier <at> gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." _______________________________________________ freebsd-fs <at> freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe <at> freebsd.org"
RSS Feed