Haldun ALTAN | 4 May 2012 17:45
Picon

[CinCV] grand'ma proxy / RAID

Well that was no more, for a long time, about Grand'ma proxy but about RAID

May be this not the exact place to talk about RAID but indirectly offers
a better use of cinelerra. I think :))

I just wanted to send a link about a very good tuto on how to RAID on
ubuntu or others. It's just like the vidéo but very much more specifique.

http://www.dedoimedo.com/computers/linux-raid.html

Hope helps.

Haldun.
Christian Thaeter | 4 May 2012 18:32

Re: [CinCV] grand'ma proxy / RAID

Am Fri, 04 May 2012 17:45:21 +0200
schrieb Haldun ALTAN <altan@...>:

> Well that was no more, for a long time, about Grand'ma proxy but
> about RAID
> 
> May be this not the exact place to talk about RAID but indirectly
> offers a better use of cinelerra. I think :))
> 
> I just wanted to send a link about a very good tuto on how to RAID on
> ubuntu or others. It's just like the vidéo but very much more
> specifique.
> 
> http://www.dedoimedo.com/computers/linux-raid.html

apart from that tutorial, here are some tips for the raid level and
setup from my experience:

for anything valuable (aka, your work) you want raid10
with 3 or more drives. Thats not mentioned on the tutorial above but the
linux kernel can do raid10 since quite some time.

This gives fast read speed, average write speed and redundancy on the
cost of 50% storage utilization. Speeds scale up as more drives you add.

If you have only 2 Drives then choose Raid1, Performance and safety is
comparable to the Raid10 above.

If you have a fast computer with lots of RAM and lots of HDD's (4+)
then you may try to evaluate raid5 or raid6, but be aware that write
(Continue reading)

Tim Copeland | 5 May 2012 23:10

Re: [CinCV] grand'ma proxy / RAID

On Fri, 2012-05-04 at 18:32 +0200, Christian Thaeter wrote:
Am Fri, 04 May 2012 17:45:21 +0200 schrieb Haldun ALTAN <altan-39ZsbGIQGT5GWvitb5QawA@public.gmane.org>: > Well that was no more, for a long time, about Grand'ma proxy but > about RAID > > May be this not the exact place to talk about RAID but indirectly > offers a better use of cinelerra. I think :)) > > I just wanted to send a link about a very good tuto on how to RAID on > ubuntu or others. It's just like the vidéo but very much more > specifique. > > http://www.dedoimedo.com/computers/linux-raid.html apart from that tutorial, here are some tips for the raid level and setup from my experience: for anything valuable (aka, your work) you want raid10 with 3 or more drives. Thats not mentioned on the tutorial above but the linux kernel can do raid10 since quite some time.

You can use RAID 10 on only 2 drives.

This gives fast read speed, average write speed and redundancy on the cost of 50% storage utilization. Speeds scale up as more drives you add. If you have only 2 Drives then choose Raid1, Performance and safety is comparable to the Raid10 above.
This is not the case. Yes you get the safety but no you don't get the performance.
Linux RAID 1 only reads from a single disk, so the performance is comparable to a single drive.
It was thought at one time that Linux RAID 1 did parallel reads, but this is false. I have personally
done the benchmarking and my findings were conclusive. Plus you can find modern documentation
to support my findings.

If you seek redundancy and performance, use RAID 10.
Linux RAID 10 is supported on as few as 2 drives.



If you have a fast computer with lots of RAM and lots of HDD's (4+) then you may try to evaluate raid5 or raid6, but be aware that write speeds are quite low and benchmark this before you using it seriously. Reading scales well with the number of disks you have. Makes a good vault for archiving files but might be too slow for a working area (considering video work). Anyways this once worked for me. Finally stay away from Raid0, it brings double (or more) the speed for double (or more) the risk. HDD's *WILL* fail eventually, its only matter of time and Murphy's law tells this happens when you are least expect it! This makes only sense for data you can *extremely* easy recover like volatile cache files (Background rendering), Files you grabbed from a cam and which are still available on a fast medium (no you don't want to grab all tapes again!). For anything else, don't even consider it. Don't forget to make regular Disk checks (regular badblocks (readonly) checks and then Raid-resyncs with mdadm in daemon mode) which will try to repair damaged data. Having a smartd running is nice to find out about if one of your drives will fail soon in few cases, but doesn't substitute for the disk checks above as smartchecks don't enforce a repair of damaged data and abort on the first found error, leaving any potential bad data ahead undiscovered. And finally: RAID redundancy is only an insurance against Harddisk failures but does *not* substitute for a backup, one manual mishap destroying data can not be undone. Christian > > Hope helps. > > Haldun. > > _______________________________________________ > Cinelerra mailing list > Cinelerra-0LiWvn2yMWjUKW2QJMybhA@public.gmane.org > https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra _______________________________________________ Cinelerra mailing list Cinelerra-0LiWvn2yMWjUKW2QJMybhA@public.gmane.org https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra

Christian Thaeter | 5 May 2012 23:54

Re: [CinCV] grand'ma proxy / RAID

Am Sat, 05 May 2012 16:10:27 -0500
schrieb Tim Copeland <tim@...>:

> On Fri, 2012-05-04 at 18:32 +0200, Christian Thaeter wrote:
> 
> > Am Fri, 04 May 2012 17:45:21 +0200
> > schrieb Haldun ALTAN <altan@...>:
> > 
> > > Well that was no more, for a long time, about Grand'ma proxy but
> > > about RAID
> > > 
> > > May be this not the exact place to talk about RAID but indirectly
> > > offers a better use of cinelerra. I think :))
> > > 
> > > I just wanted to send a link about a very good tuto on how to
> > > RAID on ubuntu or others. It's just like the vidéo but very much
> > > more specifique.
> > > 
> > > http://www.dedoimedo.com/computers/linux-raid.html
> > 
> > apart from that tutorial, here are some tips for the raid level and
> > setup from my experience:
> > 
> > for anything valuable (aka, your work) you want raid10
> > with 3 or more drives. Thats not mentioned on the tutorial above
> > but the linux kernel can do raid10 since quite some time.
> 
> 
> You can use RAID 10 on only 2 drives.
> 
> > 
> > This gives fast read speed, average write speed and redundancy on
> > the cost of 50% storage utilization. Speeds scale up as more drives
> > you add.
> > 
> > If you have only 2 Drives then choose Raid1, Performance and safety
> > is comparable to the Raid10 above.
> 
> 
> 
> This is not the case. Yes you get the safety but no you don't get the
> performance.
> Linux RAID 1 only reads from a single disk, so the performance is
> comparable to a single drive.
> It was thought at one time that Linux RAID 1 did parallel reads, but
> this is false. I have personally 
> done the benchmarking and my findings were conclusive. Plus you can
> find modern documentation
> to support my findings.
> 
> If you seek redundancy and performance, use RAID 10.
> Linux RAID 10 is supported on as few as 2 drives.

nice to know that, I never benchmarked raid1 as I only use it for
boot/system partitions but not data (except on our server ..
maybe I shall change that? do you know if raid1 is upgradeable to
raid10? i am being to lazy to rtfm now :)).

> 
> 
> 
> 
> > If you have a fast computer with lots of RAM and lots of HDD's (4+)
> > then you may try to evaluate raid5 or raid6, but be aware that write
> > speeds are quite low and benchmark this before you using it
> > seriously. Reading scales well with the number of disks you have.
> > Makes a good vault for archiving files but might be too slow for a
> > working area (considering video work). Anyways this once worked for
> > me.
> > 
> > Finally stay away from Raid0, it brings double (or more) the speed
> > for double (or more) the risk. HDD's *WILL* fail eventually, its
> > only matter of time and Murphy's law tells this happens when you
> > are least expect it! This makes only sense for data you can
> > *extremely* easy recover like volatile cache files (Background
> > rendering), Files you grabbed from a cam and which are still
> > available on a fast medium (no you don't want to grab all tapes
> > again!). For anything else, don't even consider it.
> > 
> > Don't forget to make regular Disk checks (regular badblocks
> > (readonly) checks and then Raid-resyncs with mdadm in daemon mode)
> > which will try to repair damaged data. Having a smartd running is
> > nice to find out about if one of your drives will fail soon in few
> > cases, but doesn't substitute for the disk checks above as
> > smartchecks don't enforce a repair of damaged data and abort on the
> > first found error, leaving any potential bad data ahead
> > undiscovered.
> > 
> > And finally: RAID redundancy is only an insurance against Harddisk
> > failures but does *not* substitute for a backup, one manual mishap
> > destroying data can not be undone.
> > 
> > 	Christian
> > 
> > > 
> > > Hope helps.
> > > 
> > > Haldun.
> > > 
> > > _______________________________________________
> > > Cinelerra mailing list
> > > Cinelerra@...
> > > https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra
> > 
> > _______________________________________________
> > Cinelerra mailing list
> > Cinelerra@...
> > https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra
> 
> 
Tim Copeland | 6 May 2012 20:17

Re: [CinCV] grand'ma proxy / RAID

On Sat, 2012-05-05 at 23:54 +0200, Christian Thaeter wrote:
Am Sat, 05 May 2012 16:10:27 -0500 schrieb Tim Copeland <tim-pq2sHRfyv0yACnhNS6V4X+TW4wlIGRCZ@public.gmane.org>: > On Fri, 2012-05-04 at 18:32 +0200, Christian Thaeter wrote: > > > Am Fri, 04 May 2012 17:45:21 +0200 > > schrieb Haldun ALTAN <altan-39ZsbGIQGT5GWvitb5QawA@public.gmane.org>: > > > > > Well that was no more, for a long time, about Grand'ma proxy but > > > about RAID > > > > > > May be this not the exact place to talk about RAID but indirectly > > > offers a better use of cinelerra. I think :)) > > > > > > I just wanted to send a link about a very good tuto on how to > > > RAID on ubuntu or others. It's just like the vidéo but very much > > > more specifique. > > > > > > http://www.dedoimedo.com/computers/linux-raid.html > > > > apart from that tutorial, here are some tips for the raid level and > > setup from my experience: > > > > for anything valuable (aka, your work) you want raid10 > > with 3 or more drives. Thats not mentioned on the tutorial above > > but the linux kernel can do raid10 since quite some time. > > > You can use RAID 10 on only 2 drives. > > > > > This gives fast read speed, average write speed and redundancy on > > the cost of 50% storage utilization. Speeds scale up as more drives > > you add. > > > > If you have only 2 Drives then choose Raid1, Performance and safety > > is comparable to the Raid10 above. > > > > This is not the case. Yes you get the safety but no you don't get the > performance. > Linux RAID 1 only reads from a single disk, so the performance is > comparable to a single drive. > It was thought at one time that Linux RAID 1 did parallel reads, but > this is false. I have personally > done the benchmarking and my findings were conclusive. Plus you can > find modern documentation > to support my findings. > > If you seek redundancy and performance, use RAID 10. > Linux RAID 10 is supported on as few as 2 drives. nice to know that, I never benchmarked raid1 as I only use it for boot/system partitions but not data (except on our server .. maybe I shall change that? do you know if raid1 is upgradeable to raid10? i am being to lazy to rtfm now :)).

Thats a good question. Unfortunately, that is not possible as far as I know.  That will require
a lot of scary data juggling. Your question prompted me to poke around a bit to see what I
could find on the topic. Here is an interesting post from others doing that type of migration.

http://www.burgundywall.com/tech/convert-raid1-to-raid10-with-lvm/

On a side note:
When the new btrfs file system goes main stream, it supports many neat RAID and LVM type features natively.
I do believe converting from one RAID type to another on the fly will be possible then. Heavy development is surrounding
btrfs , and is looking to be nothing short of impressive. Kernel support already exists, so its possible to play around with it
on a test system. However, it wont be viable for real world until they get the check disk finished for it. I highly recommend
keeping look out for this. I cant wait for this file system to mature.


> > > If you have a fast computer with lots of RAM and lots of HDD's (4+) > > then you may try to evaluate raid5 or raid6, but be aware that write > > speeds are quite low and benchmark this before you using it > > seriously. Reading scales well with the number of disks you have. > > Makes a good vault for archiving files but might be too slow for a > > working area (considering video work). Anyways this once worked for > > me. > > > > Finally stay away from Raid0, it brings double (or more) the speed > > for double (or more) the risk. HDD's *WILL* fail eventually, its > > only matter of time and Murphy's law tells this happens when you > > are least expect it! This makes only sense for data you can > > *extremely* easy recover like volatile cache files (Background > > rendering), Files you grabbed from a cam and which are still > > available on a fast medium (no you don't want to grab all tapes > > again!). For anything else, don't even consider it. > > > > Don't forget to make regular Disk checks (regular badblocks > > (readonly) checks and then Raid-resyncs with mdadm in daemon mode) > > which will try to repair damaged data. Having a smartd running is > > nice to find out about if one of your drives will fail soon in few > > cases, but doesn't substitute for the disk checks above as > > smartchecks don't enforce a repair of damaged data and abort on the > > first found error, leaving any potential bad data ahead > > undiscovered. > > > > And finally: RAID redundancy is only an insurance against Harddisk > > failures but does *not* substitute for a backup, one manual mishap > > destroying data can not be undone. > > > > Christian > > > > > > > > Hope helps. > > > > > > Haldun. > > > > > > _______________________________________________ > > > Cinelerra mailing list > > > Cinelerra-0LiWvn2yMWjUKW2QJMybhA@public.gmane.org > > > https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra > > > > _______________________________________________ > > Cinelerra mailing list > > Cinelerra-0LiWvn2yMWjUKW2QJMybhA@public.gmane.org > > https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra > > _______________________________________________ Cinelerra mailing list Cinelerra-0LiWvn2yMWjUKW2QJMybhA@public.gmane.org https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra


Gmane