Mark-Willem Jansen | 18 Jul 2012 10:20
Picon
Favicon
Gravatar

Picking up development of dmraid

Dear dmraid developers,

Sometime in this mail-list it was said that the program dmraid was in maintaining mode and not further developed anymore. In the meantime the dm-developement team has put out new dm-target, which can be used by the tool.

I would like to fork the latest RC and put on github, to continue developing the tool. I will give it a slightly new name, so people will not confuse it with the original. My plan is to add the support for new dm-targets and also implement more partition tables, starting with GPT.

I am not really good at generating new names, but here are some ideas.

dmraid-fbmw (forked by Mark-Willem)
dmraid-fu (follow-up)
dmraid-ext (extended version)

So my question which name you think is a good one for the forked?

And who can I connect if I have some questions about the tool.

Greetings,

Mark-Willem Jansen

_______________________________________________
Ataraid-list mailing list
Ataraid-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list
Danny Wood | 18 Jul 2012 11:08
Picon

Re: Picking up development of dmraid

Hi Mark-Willem Jansen

You may want to speak with Phillip Susi of the Ubuntu Dmraid team.
He built a set of patches a long while ago that I don't think got included in the stable dmraid.
He knows the ins and outs of dmraid and has spends a lot of time bug fixing during Ubuntu release cycles.

I think the main reason that this project has died is because it is a very niche market.
It's usually only used by the people who run a dual boot with Windows as Mdadm is far superior for pure Linux installs.

Also GPT can already be used on top of dmraid, as far as I know you use dmraid to initialise the block devs and kpartx to deal with partitions.

Good luck and best regards,
Danny


On 18/07/12 09:20, Mark-Willem Jansen wrote:
Dear dmraid developers,

Sometime in this mail-list it was said that the program dmraid was in maintaining mode and not further developed anymore. In the meantime the dm-developement team has put out new dm-target, which can be used by the tool.

I would like to fork the latest RC and put on github, to continue developing the tool. I will give it a slightly new name, so people will not confuse it with the original. My plan is to add the support for new dm-targets and also implement more partition tables, starting with GPT.

I am not really good at generating new names, but here are some ideas.

dmraid-fbmw (forked by Mark-Willem)
dmraid-fu (follow-up)
dmraid-ext (extended version)

So my question which name you think is a good one for the forked?

And who can I connect if I have some questions about the tool.

Greetings,

Mark-Willem Jansen



_______________________________________________ Ataraid-list mailing list Ataraid-list <at> redhat.com https://www.redhat.com/mailman/listinfo/ataraid-list


_______________________________________________
Ataraid-list mailing list
Ataraid-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list
Mark-Willem Jansen | 18 Jul 2012 12:11
Picon
Favicon
Gravatar

RE: Picking up development of dmraid

Hi Danny,

Thanks for the info.

I will try to contact Phillip Susi directly. I already had some contact with him in the past. Lets see if we can make this tool fully functional.

For my own system I implemented dm-raid.ko support into dmraid and the kpartx work-around. But I got some minor errors with gdisk when adding partitions to the raid. And I also find it dangerous that such an essential tool is dependent on another tool.

Kind regards,

Mark-Willem



Date: Wed, 18 Jul 2012 10:08:28 +0100
From: danwood76 <at> gmail.com
To: ataraid-list <at> redhat.com
Subject: Re: Picking up development of dmraid

Hi Mark-Willem Jansen

You may want to speak with Phillip Susi of the Ubuntu Dmraid team.
He built a set of patches a long while ago that I don't think got included in the stable dmraid.
He knows the ins and outs of dmraid and has spends a lot of time bug fixing during Ubuntu release cycles.

I think the main reason that this project has died is because it is a very niche market.
It's usually only used by the people who run a dual boot with Windows as Mdadm is far superior for pure Linux installs.

Also GPT can already be used on top of dmraid, as far as I know you use dmraid to initialise the block devs and kpartx to deal with partitions.

Good luck and best regards,
Danny
_______________________________________________
Ataraid-list mailing list
Ataraid-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list
Heinz Mauelshagen | 19 Jul 2012 10:26
Picon
Favicon

Re: Picking up development of dmraid

On 07/18/2012 11:08 AM, Danny Wood wrote:
Hi Mark-Willem Jansen

You may want to speak with Phillip Susi of the Ubuntu Dmraid team.
He built a set of patches a long while ago that I don't think got included in the stable dmraid.
He knows the ins and outs of dmraid and has spends a lot of time bug fixing during Ubuntu release cycles.

I think the main reason that this project has died is because it is a very niche market.

Seconded WRT the niche market.

It's usually only used by the people who run a dual boot with Windows as Mdadm is far superior for pure Linux installs.

The later is exactly why things move to MD and eg. we're doing a device-mapper target wrapper
to access the MD kernel runtime in order to make it accessible in LVM.

Because the MD runtime has the long established field record it has, major FAKERAID OEMs decided
to use it (namly Intel with their Intel Matrix RAID, isw in dmraid).
mdadm gained external metadata format support along the lines of dmraid to allow for that and
thus supports isw for long time now.

As a result of that, Red Hat decided to not further develop dmraid. Actually we already asked publically,
if dmraid is still mandatory to support the other metadata formats than DDF, Intel Matrix RAID and MD,
which are supported by mdadm now anyway.

No arguments it's still needed resulted from that so far.


Also GPT can already be used on top of dmraid, as far as I know you use dmraid to initialise the block devs and kpartx to deal with partitions.

There's no need to have code duplication for partitioinig support in another tool.
For the record: the DOS partitioning support got added way back in time before kpartx addressed it
(and never got pulled out).

So use kpartx for activating partitionins on RAID sets.


The most important question (as mentioned above) still persists though: is dmraid still needed or
is any further development adequate  to support the Adaptec, Highpoint, Jmicron, LSI, NVidia, Promise,
Silicon Image and VIA metadata formats? Are they still being used that much in the field or are users
just happy with dmraid access to those in their mixed Linux/Windows environments?
Requirement for pure Linux environment is MD/LVM anyway.

We better get field feedback which we didn't get so far to answer that question.

Heinz


Good luck and best regards,
Danny


On 18/07/12 09:20, Mark-Willem Jansen wrote:
Dear dmraid developers,

Sometime in this mail-list it was said that the program dmraid was in maintaining mode and not further developed anymore. In the meantime the dm-developement team has put out new dm-target, which can be used by the tool.

I would like to fork the latest RC and put on github, to continue developing the tool. I will give it a slightly new name, so people will not confuse it with the original. My plan is to add the support for new dm-targets and also implement more partition tables, starting with GPT.

I am not really good at generating new names, but here are some ideas.

dmraid-fbmw (forked by Mark-Willem)
dmraid-fu (follow-up)
dmraid-ext (extended version)

So my question which name you think is a good one for the forked?

And who can I connect if I have some questions about the tool.

Greetings,

Mark-Willem Jansen



_______________________________________________ Ataraid-list mailing list Ataraid-list <at> redhat.com https://www.redhat.com/mailman/listinfo/ataraid-list




_______________________________________________ Ataraid-list mailing list Ataraid-list <at> redhat.com https://www.redhat.com/mailman/listinfo/ataraid-list

_______________________________________________
Ataraid-list mailing list
Ataraid-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list
Mark-Willem Jansen | 19 Jul 2012 15:33
Picon
Favicon
Gravatar

RE: Picking up development of dmraid

Hi Heinz,

Concerning the partitioning support. I now feel that kpartx or (partx which could be made compatible) would be the way to go in stead of dmraid. So it would be a good idea to remove the partitioning support from dmraid.  This will mean the package maintainer will need to make dmraid dependent on kpartx for it to work.

One remark on this: I found that on Debian to make kpartx work with dmraid during boot, one needs to make some changes to the multipath-tools packages.

On a side note: Why does mdadm support MBR and GPT?

Concerning the usage of mdadm instead of dmraid. Me and probably and a large amount of AMD users will have a AMD chip-set which means a Promise FAKERAID controller. So in my idea I had two options update dmraid to work with the dm-raid target(probably the device-mapper target wrapper you are talking about) to setup my RAID-5 or add support for the Promise metadata format to mdadm. I picked the former as the code of dmraid look easier to understand. ;-)

If you say that adding a super-promise.c to mdadm is doable, I could change my mind.

Just one last question I never really got an answer to. Can one use mdadm on a dual boot system(MS and Linux) were the RAID partitions are shared? In other words will mdadm leave the metadata on the disks unchanged or in a state the the MS drivers can still recognize the RAID.

Would it be an idea to clean-up dmraid? Remove what is not needed anymore, or does not fit the purpose of the tool.

Kind regards,


Mark-Willem

Date: Thu, 19 Jul 2012 10:26:49 +0200
From: heinzm <at> redhat.com
To: ataraid-list <at> redhat.com
Subject: Re: Picking up development of dmraid

On 07/18/2012 11:08 AM, Danny Wood wrote:
Hi Mark-Willem Jansen

You may want to speak with Phillip Susi of the Ubuntu Dmraid team.
He built a set of patches a long while ago that I don't think got included in the stable dmraid.
He knows the ins and outs of dmraid and has spends a lot of time bug fixing during Ubuntu release cycles.

I think the main reason that this project has died is because it is a very niche market.

Seconded WRT the niche market.

It's usually only used by the people who run a dual boot with Windows as Mdadm is far superior for pure Linux installs.

The later is exactly why things move to MD and eg. we're doing a device-mapper target wrapper
to access the MD kernel runtime in order to make it accessible in LVM.

Because the MD runtime has the long established field record it has, major FAKERAID OEMs decided
to use it (namly Intel with their Intel Matrix RAID, isw in dmraid).
mdadm gained external metadata format support along the lines of dmraid to allow for that and
thus supports isw for long time now.

As a result of that, Red Hat decided to not further develop dmraid. Actually we already asked publically,
if dmraid is still mandatory to support the other metadata formats than DDF, Intel Matrix RAID and MD,
which are supported by mdadm now anyway.

No arguments it's still needed resulted from that so far.


Also GPT can already be used on top of dmraid, as far as I know you use dmraid to initialise the block devs and kpartx to deal with partitions.

There's no need to have code duplication for partitioinig support in another tool.
For the record: the DOS partitioning support got added way back in time before kpartx addressed it
(and never got pulled out).

So use kpartx for activating partitionins on RAID sets.


The most important question (as mentioned above) still persists though: is dmraid still needed or
is any further development adequate  to support the Adaptec, Highpoint, Jmicron, LSI, NVidia, Promise,
Silicon Image and VIA metadata formats? Are they still being used that much in the field or are users
just happy with dmraid access to those in their mixed Linux/Windows environments?
Requirement for pure Linux environment is MD/LVM anyway.

We better get field feedback which we didn't get so far to answer that question.

Heinz


Good luck and best regards,
Danny


On 18/07/12 09:20, Mark-Willem Jansen wrote:
Dear dmraid developers,

Sometime in this mail-list it was said that the program dmraid was in maintaining mode and not further developed anymore. In the meantime the dm-developement team has put out new dm-target, which can be used by the tool.

I would like to fork the latest RC and put on github, to continue developing the tool. I will give it a slightly new name, so people will not confuse it with the original. My plan is to add the support for new dm-targets and also implement more partition tables, starting with GPT.

I am not really good at generating new names, but here are some ideas.

dmraid-fbmw (forked by Mark-Willem)
dmraid-fu (follow-up)
dmraid-ext (extended version)

So my question which name you think is a good one for the forked?

And who can I connect if I have some questions about the tool.

Greetings,

Mark-Willem Jansen



_______________________________________________ Ataraid-list mailing list Ataraid-list <at> redhat.com https://www.redhat.com/mailman/listinfo/ataraid-list




_______________________________________________ Ataraid-list mailing list Ataraid-list <at> redhat.com https://www.redhat.com/mailman/listinfo/ataraid-list


_______________________________________________ Ataraid-list mailing list Ataraid-list <at> redhat.com https://www.redhat.com/mailman/listinfo/ataraid-list
_______________________________________________
Ataraid-list mailing list
Ataraid-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list
Bryn M. Reeves | 19 Jul 2012 16:02
Picon
Favicon
Gravatar

Re: Picking up development of dmraid


On 07/19/2012 02:33 PM, Mark-Willem Jansen wrote:
> Concerning the partitioning support. I now feel that kpartx or 
> (partx which could be made compatible) would be the way to go in 
> stead of

The partx command only works for devices that use the in-kernel
partition support (IDE, SCSI, MD, cciss, loop etc.).

Device-mapper devices are not partitionable which is why kpartx was
written in the first place.

> dmraid. So it would be a good idea to remove the partitioning 
> support from dmraid.  This will mean the package maintainer will 
> need to make dmraid dependent on kpartx for it to work.

It already does in most (all?) distributions that ship it.

> One remark on this: I found that on Debian to make kpartx work with
> dmraid during boot, one needs to make some changes to the 
> multipath-tools packages.

What were the changes? The kpartx command is part of multipath-tools
and although it's common to have it in a separate sub-package (all
current Red Hat and Fedora distros do this) they are part of the same
project upstream.

> On a side note: Why does mdadm support MBR and GPT?

Not sure what you're asking here? The kernel MD driver creates
partitionable devices so you can use any of the label formats that are
enabled in the kernel you're running (although really, MBR and GPT are
the only ones that make sense for most systems today).

> Concerning the usage of mdadm instead of dmraid. Me and probably 
> and a large amount of AMD users will have a AMD chip-set which 
> means a Promise FAKERAID controller. So in my idea I had two 
> options update dmraid to work with the dm-raid target(probably the 
> device-mapper target wrapper you are talking about) to setup my 
> RAID-5 or add support for the Promise metadata format to mdadm. I 
> picked the former as the code of dmraid look easier to understand. 
> ;-)
> 
> If you say that adding a super-promise.c to mdadm is doable, I 
> could change my mind.

I think adding new format handlers to MD is a much better idea; the
dominant formats backed by major OEMs are already using it so if
there's interest in the less commonly used formats I think they would
see much better maintenance and continued development in an active
project like mdadm than they would in a revived dmraid.

Aside from the other matters MD and mdadm are well-integrated in every
modern distro. Although some releases managed to get dmraid to work
well out of the box most others did not and it caused quite some pain
for users trying to get it working.

> Just one last question I never really got an answer to. Can one use
> mdadm on a dual boot system(MS and Linux) were the RAID partitions
> are shared? In other words will mdadm leave the metadata on the
> disks unchanged or in a state the the MS drivers can still 
> recognize the RAID.

Assuming that MD supports the format handler you need: yes.

All drivers (whether user/kernel on Linux or Windows) need to respect
the requirements of the on-disk formats and leave them in an
appropriate state on shutdown.

That said, a RAID subsystem also needs to be robust and to tolerate
sudden failures so it's also true that both sides need to be tolerant
in dealing with a system in a failed state (it's Postel's law again:
"be conservative in what you do, be liberal in what you accept from
others").

> Would it be an idea to clean-up dmraid? Remove what is not needed 
> anymore, or does not fit the purpose of the tool.

I think the time would be better spent learning or contributing to MD
and mdadm development and adding support for other format handlers
that have users wanting native Linux support.

Regards,
Bryn.

Curtis Gedak | 18 Jul 2012 18:34
Picon

Re: Picking up development of dmraid

Hi Mark,

This is great news that you are interested in furthering the development 
of dmraid.

Ideally, the current maintainer of dmraid would consider migrating the 
maintenance duties to you.  As far as I know, Heinz Mauelshagen is the 
current maintainer.

      Heinz,

      If you are following this list it would be good to hear from you.

Phillip Susi has created some dmraid patches that have been awaiting 
review for a long while.  As such it would be great to have a more 
active maintainer for dmraid.

If no response is received from Heinz, then a fork might be the next 
best way to proceed.  For the util-linux package, the fork was named 
util-linux-ng where I think "ng" stood for "next generation". Eventually 
the util-linux-ng project took over from the unmaintained util-linux 
project to become util-linux once again.

I believe that Karel Zak is the maintainer for util-linux (ng) so he 
might have some good advice regarding forking a project.

Regards,
Curtis Gedak
(Maintainer of GParted)

On 12-07-18 02:20 AM, Mark-Willem Jansen wrote:
> Dear dmraid developers,
>
> Sometime in this mail-list it was said that the program dmraid was in 
> maintaining mode and not further developed anymore. In the meantime 
> the dm-developement team has put out new dm-target, which can be used 
> by the tool.
>
> I would like to fork the latest RC and put on github, to continue 
> developing the tool. I will give it a slightly new name, so people 
> will not confuse it with the original. My plan is to add the support 
> for new dm-targets and also implement more partition tables, starting 
> with GPT.
>
> I am not really good at generating new names, but here are some ideas.
>
> dmraid-fbmw (forked by Mark-Willem)
> dmraid-fu (follow-up)
> dmraid-ext (extended version)
>
> So my question which name you think is a good one for the forked?
>
> And who can I connect if I have some questions about the tool.
>
> Greetings,
>
> Mark-Willem Jansen
>
>
>
> _______________________________________________
> Ataraid-list mailing list
> Ataraid-list <at> redhat.com
> https://www.redhat.com/mailman/listinfo/ataraid-list
Mark-Willem Jansen | 18 Jul 2012 19:48
Picon
Favicon
Gravatar

RE: Picking up development of dmraid

Hi Curtis,



It looks like I am in good company here. Now I just need to find a way to contact Karel Zak. I have found a redhat email adres on the internet, but maybe you know a better way to contact him.


Regards,


Mark-Willem Jansen


> Date: Wed, 18 Jul 2012 10:34:58 -0600
> From: gedakc <at> gmail.com
> To: ataraid-list <at> redhat.com
> CC: markwillem <at> hotmail.com
> Subject: Re: Picking up development of dmraid
>
> Hi Mark,
>
> This is great news that you are interested in furthering the development
> of dmraid.
>
> Ideally, the current maintainer of dmraid would consider migrating the
> maintenance duties to you. As far as I know, Heinz Mauelshagen is the
> current maintainer.
>
> Heinz,
>
> If you are following this list it would be good to hear from you.
>
>
> Phillip Susi has created some dmraid patches that have been awaiting
> review for a long while. As such it would be great to have a more
> active maintainer for dmraid.
>
> If no response is received from Heinz, then a fork might be the next
> best way to proceed. For the util-linux package, the fork was named
> util-linux-ng where I think "ng" stood for "next generation". Eventually
> the util-linux-ng project took over from the unmaintained util-linux
> project to become util-linux once again.
>
> I believe that Karel Zak is the maintainer for util-linux (ng) so he
> might have some good advice regarding forking a project.
>
> Regards,
> Curtis Gedak
> (Maintainer of GParted)
>
> On 12-07-18 02:20 AM, Mark-Willem Jansen wrote:
> > Dear dmraid developers,
> >
> > Sometime in this mail-list it was said that the program dmraid was in
> > maintaining mode and not further developed anymore. In the meantime
> > the dm-developement team has put out new dm-target, which can be used
> > by the tool.
> >
> > I would like to fork the latest RC and put on github, to continue
> > developing the tool. I will give it a slightly new name, so people
> > will not confuse it with the original. My plan is to add the support
> > for new dm-targets and also implement more partition tables, starting
> > with GPT.
> >
> > I am not really good at generating new names, but here are some ideas.
> >
> > dmraid-fbmw (forked by Mark-Willem)
> > dmraid-fu (follow-up)
> > dmraid-ext (extended version)
> >
> > So my question which name you think is a good one for the forked?
> >
> > And who can I connect if I have some questions about the tool.
> >
> > Greetings,
> >
> > Mark-Willem Jansen
> >
> >
> >
> > _______________________________________________
> > Ataraid-list mailing list
> > Ataraid-list <at> redhat.com
> > https://www.redhat.com/mailman/listinfo/ataraid-list
>
>
_______________________________________________
Ataraid-list mailing list
Ataraid-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list
Petr Uzel | 18 Jul 2012 21:04
Picon
Gravatar

Re: Picking up development of dmraid

On Wed, Jul 18, 2012 at 07:48:25PM +0200, Mark-Willem Jansen wrote:
> Hi Curtis,
> 
> It looks like I am in good company here. Now I just need to find a
> way to contact Karel Zak. I have found a redhat email adres on the
> internet, but maybe you know a better way to contact him.

kzak <at> redhat.cz or kzak  <at>  freenode IRC work fine.

BTW, I'd also vote for dmraid-ng - this seems to be
preferred naming for taking over no-longer-maintained projects.
(iptraf-ng and procps-ng are other projects which come to my mind).

Regards,

Petr

--

-- 
Petr Uzel
IRC: ptr_uzl  <at>  freenode
_______________________________________________
Ataraid-list mailing list
Ataraid-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list
Alasdair G Kergon | 18 Jul 2012 21:59
Picon
Favicon

Re: Picking up development of dmraid

On Wed, Jul 18, 2012 at 10:20:10AM +0200, Mark-Willem Jansen wrote:
> Sometime in this mail-list it was said that the program dmraid was in
> maintaining mode and not further developed anymore. 

What features of dmraid are still missing and aren't available from other
tools already?

> also implement more partition tables, starting with GPT.

What advantage would dmraid have over kpartx which already does this?

Alasdair
Picon
Favicon

Re: Picking up development of dmraid

On 18/07/12 03:59 PM, Alasdair G Kergon wrote:
> On Wed, Jul 18, 2012 at 10:20:10AM +0200, Mark-Willem Jansen wrote:
>> also implement more partition tables, starting with GPT.
>  
> What advantage would dmraid have over kpartx which already does this?
> 

I would have concerns on this one too -- imo dmraid should just handle
what it does now, which is to parse the raid metadata and configure the
device-mapper appropriately.  Dmraid shouldn't take over for kpartx (or
whatever other tool handles GPT), but if there is a conflict which is
keeping dmraid or kpartx from doing its job with GPT partitions then
this should be resolved.
Bad Bod | 19 Jul 2012 09:33
Gravatar

Re: Picking up development of dmraid

Hi,

  Any update on whether the checksum error has been fixed for VIA chipsets?


Regards
David



On Thu, Jul 19, 2012 at 3:04 AM, Ian Stakenvicius, Aerobiology Research <ian <at> aerobiology.ca> wrote:
On 18/07/12 03:59 PM, Alasdair G Kergon wrote:
> On Wed, Jul 18, 2012 at 10:20:10AM +0200, Mark-Willem Jansen wrote:
>> also implement more partition tables, starting with GPT.
>
> What advantage would dmraid have over kpartx which already does this?
>

I would have concerns on this one too -- imo dmraid should just handle
what it does now, which is to parse the raid metadata and configure the
device-mapper appropriately.  Dmraid shouldn't take over for kpartx (or
whatever other tool handles GPT), but if there is a conflict which is
keeping dmraid or kpartx from doing its job with GPT partitions then
this should be resolved.

_______________________________________________
Ataraid-list mailing list
Ataraid-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list

_______________________________________________
Ataraid-list mailing list
Ataraid-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list

Gmane