headers
John McMonagle | 29 Jun 2012 17:32
Favicon

live migration iscsi and lvm

I'm experimenting setting up san and live migrations.

Running debian squeeze and usually create domains with xentools although I 
will consider other possibilities.

Saw a tutorial iscsi + lvm.
Setup a iscsi target for use as a volume group.
I made it into a volume group and as far as that seems to work fine.

What are my possibilities for destruction :-)
Like to test the limits now when nothing that really matters is on the vg.
If I create a new logical volume on one node it  shows up on the other node 
without and issues.

Should I be using clvm?
I tried to install it but it  seems to want a setup in /etc/cluster.
Or should I do the sharing in a different way?

John
Permalink | Reply |
headers
Javier Guerra Giraldez | 29 Jun 2012 22:29
Gravatar

Re: live migration iscsi and lvm

On Fri, Jun 29, 2012 at 10:32 AM, John McMonagle <johnm <at> advocap.org> wrote:
> Should I be using clvm?

not necessarily; but it does have some advantages.

in most VM setups you don't use cluster filesystems (GFS, OCFS2, etc),
so you should never mount the same volume on two machines.  that
extends to virtual machines too, so you must not start the same VM
image on two hosts.  most setups help you enforce this, and live
migration makes sure the target VM isn't resumed until the original VM
is no longer working.

so far, no need for any 'cluster' setup.

LVM reads all the physical volume, volume group and logical volume
layouts (what it calls metadata) from the shared storage to RAM at
startup and then works from there.  It's only rewritten to disk when
modified (creating/deleting volumes, resizing them, adding PVs, etc).

That means that if you start more than one box connected to the same
PVs they'll be able to reach the same VGs and LVs, and if you don't
modify anything, it would run perfectly.  But, if you want to do any
metadata change, you have to:

1) choose one machine to do the change
2) on every other machine, disconnect from the VG (vgchange -a n)
3) do any needed change on the only machine that's still connected to the VG
4) reread the metadata on all machines (lvscan)

if you have periodic planned downtimes, you can schedule things and
(Continue reading)

Permalink | Reply |
headers
John McMonagle | 30 Jun 2012 14:06
Favicon

Re: live migration iscsi and lvm

On Friday 29 June 2012 3:29:47 pm Javier Guerra Giraldez wrote:
> On Fri, Jun 29, 2012 at 10:32 AM, John McMonagle <johnm <at> advocap.org> wrote:
> > Should I be using clvm?
> 
> not necessarily; but it does have some advantages.
> 
> in most VM setups you don't use cluster filesystems (GFS, OCFS2, etc),
> so you should never mount the same volume on two machines.  that
> extends to virtual machines too, so you must not start the same VM
> image on two hosts.  most setups help you enforce this, and live
> migration makes sure the target VM isn't resumed until the original VM
> is no longer working.
> 
> so far, no need for any 'cluster' setup.
> 
> LVM reads all the physical volume, volume group and logical volume
> layouts (what it calls metadata) from the shared storage to RAM at
> startup and then works from there.  It's only rewritten to disk when
> modified (creating/deleting volumes, resizing them, adding PVs, etc).
> 
> That means that if you start more than one box connected to the same
> PVs they'll be able to reach the same VGs and LVs, and if you don't
> modify anything, it would run perfectly.  But, if you want to do any
> metadata change, you have to:
> 
> 1) choose one machine to do the change
> 2) on every other machine, disconnect from the VG (vgchange -a n)
> 3) do any needed change on the only machine that's still connected to the
> VG 4) reread the metadata on all machines (lvscan)
>
(Continue reading)

Permalink | Reply |
headers
Fajar A. Nugraha | 30 Jun 2012 14:13

Re: live migration iscsi and lvm

On Sat, Jun 30, 2012 at 7:06 PM, John McMonagle <johnm <at> advocap.org> wrote:

> I have been trying to get clvm working but it's dependent on cman and have not
> had  much luck figuring out cluster.conf.
>
> I see the new version in testing has no dependency on cman so I think I'll try
> that one.
> Looks like will have to upgrade lvm2 also.
>
> I have seen references saying that you can not do snapshots.
> And others saying the the snapshot  and snapshotted volume have to be on one
> node.  That I can live with. Is that the case?

Since you're using an iscsi target anyway, why not just setup a
separate LUN for each VM? That you don't have to worry about LVM on
the dom0 nodes.

--

-- 
Fajar
Permalink | Reply |
headers
Stephan Seitz | 3 Jul 2012 11:10
Picon

Re: live migration iscsi and lvm


Am Samstag, den 30.06.2012, 19:13 +0700 schrieb Fajar A. Nugraha:
> On Sat, Jun 30, 2012 at 7:06 PM, John McMonagle <johnm <at> advocap.org> wrote:
> 
> > I have been trying to get clvm working but it's dependent on cman and have not
> > had  much luck figuring out cluster.conf.
> >
> > I see the new version in testing has no dependency on cman so I think I'll try
> > that one.
> > Looks like will have to upgrade lvm2 also.
> >
> > I have seen references saying that you can not do snapshots.
> > And others saying the the snapshot  and snapshotted volume have to be on one
> > node.  That I can live with. Is that the case?
> 
> Since you're using an iscsi target anyway, why not just setup a
> separate LUN for each VM? That you don't have to worry about LVM on
> the dom0 nodes.
> 

I've tried that, but was never able to figure out how a smoothly working
multipath setup could be done for direct LUN/VM mapping. After some
xm/xl create and migrate either dm-multipath or open-iscsi always had
issues with locked devices which in most cases couldn't be freed. I
assume better scripting skills for my xen/scripts/block-* attach/detach
scripts could've mitigate that.
Additionally, using a fixed set of LUNs connected via fixed amount of
paths over fixed dedicated links and combined as PVs together to VGs
gave me the chance of setting any tcp and iscsi (target as well as
initiator) value to ideally fit to latency and bandwidth. Having
(Continue reading)

Permalink | Reply |

Gmane