On Nov 10, 2012, at 4:55 AM, Jim Klimov <jimklimov <at> cos.ru
> On 2012-11-10 04:55, Garrett D'Amore wrote:
>> unlikely that any SHA-256 collision will be found in my life time,
>> unless there is a deeper flaw found in the SHA algorithm itself. I'm
> Then again, I found and George Wilson (I believe) fixed a bug
> in ZFS itself which caused problems with unverified writes on
> a deduped pool. So even if the hash collisions are not likely
> mathematically, there are situations which look the same and
> dedup verification is not something "too paranoid"- even if
> only to catch ZFS bugs.
> To recap the "anecdotal reference", my userdata got corrupted
> somehow so that some blocks from several files could not be
> recovered by raidz2 over 6 disks (I still have little rigid
> idea what went wrong, and I think I have some of those corrupted
> files left dormant on my old box so I could try to research
> their sectors manually some day).
> While the block-pointer tree walk (like scrub or normal file
> I/O) did find the presence of an error - a block did not match
> its checksum - the entry in DDT for this checksum existed and
> was not invalidated because of scrub's findings.
> Further unverified writes into the pool just incremented the DDT
> counter and new files (restored from backup) remained corrupted.
> Writes with verification did find that the on-disk blocks differ
> (they still did not care that one of them actually mismatches
> the checksum) and decreased the DDT counter for "bad block" and
> allocated a new unique one for valid data. I believe that for
> such collided entries as recovery blocks with valid data, new
> normal unique blocks are allocated and never deduped.
> Ultimately, there was a panic (the cause of which was what George
> fixed in https://www.illumos.org/issues/2649
) because ZFS did
> not do proper housekeeping for such unique blocks, and they were
> still regarded as deduped - but not entered into the DDT. So the
> further writes into the "dedup=verify" file that tried to release
> the unique block from DDT (and which was not there) panicked.
> My other related reported bugs on this subject were
> My fix for not-panicking due to already having metadata corruption
> of this type, and just releasing the "non-DDT" deduped blocks when
> asked at a risk of "leaks" in the block tree was in the end the
> essence of my 2024, and ranks between cosmetic fix and a usability
> one (the storage system no longer panics in this case). I don't
> think it was integrated - or, after the discussions, if it should
> be - but the patch is on the issue tracker for those in need.
> The corrupted DDT entries were never addressed, to my knowledge,
> so it is still possible to have DDT point to unreadable garbage.
> Most of these "corrupted" blocks on my system were flushed when
> the DDT counter went to 0, after all copies of the file (block)
> were "recovered" from backup or other sources, or deleted.
> Archives: https://www.listbox.com/member/archive/182191/=now