Marko Sutic | 25 Aug 12:42 2011
Picon

Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Freek,

you are correct - heartbeat fatal messages are there due to the missing voting disk.

I have another database up and running on second node and this database is using same ocfs2 volume for Oracle database files as the first one.
This database is running without any error so I suppose that other OCFS2 volumes were accessible in the time of the failure.

In this configuration are 3 voting disk files located on 3 different luns and separate OCFS2 volumes. When failure occurs two of three voting devices hang.

It is also worth to mention that nothing else is running on that node except import.


I simply can't figure out why two of three voting disks hang.


Regards,
Marko


On Thu, Aug 25, 2011 at 11:08 AM, D'Hooge Freek <Freek.DHooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org> wrote:
Marco,

I don't know the error timings for the other node, but I think the heartbeat fatal messages are coming after the first node has terminated due to the missing voting disk.

This would indicate that there is no general problem with the voting disk itself, but that the problem is specific to the first node.
Either the connection itself or the load or an ocfs2 bug would then be the cause of the error.

Do you know if at the time of the failure the other OCFS2 volumes where still accessible?
Are your voting disks placed on the same luns as your database files or are they on a separate ocfs2 volume?

Regards,


Freek D'Hooge
Uptime
Oracle Database Administrator
email: freek.dhooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org
tel +32(0)3 451 23 82
http://www.uptime.be
disclaimer: www.uptime.be/disclaimer
---
From: Marko Sutic [mailto:marko.sutic-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]
Sent: donderdag 25 augustus 2011 10:51
To: D'Hooge Freek
Cc: oracle-l-uGLqWuYN4qMgsBAKwltoeQ@public.gmane.org
Subject: Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Errors messages from another node:

2011-08-25 10:38:33.563
[cssd(18117)]CRS-1612:node l01ora3 (1) at 50% heartbeat fatal, eviction in 14.000 seconds
2011-08-25 10:38:40.558
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 7.010 seconds
2011-08-25 10:38:41.560
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 6.010 seconds
2011-08-25 10:38:45.558
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 2.010 seconds
2011-08-25 10:38:46.560
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 1.010 seconds
2011-08-25 10:38:47.562
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 0.010 seconds
2011-08-25 10:38:47.574
[cssd(18117)]CRS-1607:CSSD evicting node l01ora3. Details in /u01/app/crs/log/l01ora4/cssd/ocssd.log.
2011-08-25 10:39:01.579
[cssd(18117)]CRS-1601:CSSD Reconfiguration complete. Active nodes are l01ora4 .


Regards,
Marko



--
Marko Sutic, dipl.ing.rač.
My LinkedIn Profile

David Barbour | 26 Aug 00:14 2011
Picon

Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Anything in /var/log/messages? 

On Thu, Aug 25, 2011 at 5:42 AM, Marko Sutic <marko.sutic-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Freek,

you are correct - heartbeat fatal messages are there due to the missing voting disk.

I have another database up and running on second node and this database is using same ocfs2 volume for Oracle database files as the first one.
This database is running without any error so I suppose that other OCFS2 volumes were accessible in the time of the failure.

In this configuration are 3 voting disk files located on 3 different luns and separate OCFS2 volumes. When failure occurs two of three voting devices hang.

It is also worth to mention that nothing else is running on that node except import.


I simply can't figure out why two of three voting disks hang.


Regards,
Marko


On Thu, Aug 25, 2011 at 11:08 AM, D'Hooge Freek <Freek.DHooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org> wrote:
Marco,

I don't know the error timings for the other node, but I think the heartbeat fatal messages are coming after the first node has terminated due to the missing voting disk.

This would indicate that there is no general problem with the voting disk itself, but that the problem is specific to the first node.
Either the connection itself or the load or an ocfs2 bug would then be the cause of the error.

Do you know if at the time of the failure the other OCFS2 volumes where still accessible?
Are your voting disks placed on the same luns as your database files or are they on a separate ocfs2 volume?

Regards,


Freek D'Hooge
Uptime
Oracle Database Administrator
email: freek.dhooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org
tel +32(0)3 451 23 82
http://www.uptime.be
disclaimer: www.uptime.be/disclaimer
---
From: Marko Sutic [mailto:marko.sutic-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]
Sent: donderdag 25 augustus 2011 10:51
To: D'Hooge Freek
Cc: oracle-l <at> freelists.org
Subject: Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Errors messages from another node:

2011-08-25 10:38:33.563
[cssd(18117)]CRS-1612:node l01ora3 (1) at 50% heartbeat fatal, eviction in 14.000 seconds
2011-08-25 10:38:40.558
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 7.010 seconds
2011-08-25 10:38:41.560
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 6.010 seconds
2011-08-25 10:38:45.558
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 2.010 seconds
2011-08-25 10:38:46.560
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 1.010 seconds
2011-08-25 10:38:47.562
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 0.010 seconds
2011-08-25 10:38:47.574
[cssd(18117)]CRS-1607:CSSD evicting node l01ora3. Details in /u01/app/crs/log/l01ora4/cssd/ocssd.log.
2011-08-25 10:39:01.579
[cssd(18117)]CRS-1601:CSSD Reconfiguration complete. Active nodes are l01ora4 .


Regards,
Marko



--
Marko Sutic, dipl.ing.rač.

Marko Sutic | 26 Aug 10:34 2011
Picon

Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Hi David,


/var/log/messages is stuffed with various messages and I cannot identify what is important to look for.

I will attach excerpt of log file from the period during import and when failure occurred.

If you notice something odd please let me know.

Regards,
Marko



On Fri, Aug 26, 2011 at 12:14 AM, David Barbour <david.barbour1-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Anything in /var/log/messages? 

On Thu, Aug 25, 2011 at 5:42 AM, Marko Sutic <marko.sutic-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Freek,

you are correct - heartbeat fatal messages are there due to the missing voting disk.

I have another database up and running on second node and this database is using same ocfs2 volume for Oracle database files as the first one.
This database is running without any error so I suppose that other OCFS2 volumes were accessible in the time of the failure.

In this configuration are 3 voting disk files located on 3 different luns and separate OCFS2 volumes. When failure occurs two of three voting devices hang.

It is also worth to mention that nothing else is running on that node except import.


I simply can't figure out why two of three voting disks hang.


Regards,
Marko


On Thu, Aug 25, 2011 at 11:08 AM, D'Hooge Freek <Freek.DHooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org> wrote:
Marco,

I don't know the error timings for the other node, but I think the heartbeat fatal messages are coming after the first node has terminated due to the missing voting disk.

This would indicate that there is no general problem with the voting disk itself, but that the problem is specific to the first node.
Either the connection itself or the load or an ocfs2 bug would then be the cause of the error.

Do you know if at the time of the failure the other OCFS2 volumes where still accessible?
Are your voting disks placed on the same luns as your database files or are they on a separate ocfs2 volume?

Regards,


Freek D'Hooge
Uptime
Oracle Database Administrator
email: freek.dhooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org
tel +32(0)3 451 23 82
http://www.uptime.be
disclaimer: www.uptime.be/disclaimer
---
From: Marko Sutic [mailto:marko.sutic-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]
Sent: donderdag 25 augustus 2011 10:51
To: D'Hooge Freek
Cc: oracle-l <at> freelists.org
Subject: Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Errors messages from another node:

2011-08-25 10:38:33.563
[cssd(18117)]CRS-1612:node l01ora3 (1) at 50% heartbeat fatal, eviction in 14.000 seconds
2011-08-25 10:38:40.558
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 7.010 seconds
2011-08-25 10:38:41.560
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 6.010 seconds
2011-08-25 10:38:45.558
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 2.010 seconds
2011-08-25 10:38:46.560
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 1.010 seconds
2011-08-25 10:38:47.562
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 0.010 seconds
2011-08-25 10:38:47.574
[cssd(18117)]CRS-1607:CSSD evicting node l01ora3. Details in /u01/app/crs/log/l01ora4/cssd/ocssd.log.
2011-08-25 10:39:01.579
[cssd(18117)]CRS-1601:CSSD Reconfiguration complete. Active nodes are l01ora4 .


Regards,
Marko




Aug 25 19:06:49 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 295 seconds.
Aug 25 19:11:43 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:11:44 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:11:44 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 248 seconds.
Aug 25 19:15:52 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:15:52 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:15:52 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 266 seconds.
Aug 25 19:20:18 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:20:18 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:20:18 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 294 seconds.
Aug 25 19:25:12 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:25:12 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:25:12 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 238 seconds.
Aug 25 19:29:09 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:29:10 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:29:10 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 291 seconds.
Aug 25 19:34:00 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:34:01 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:34:01 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 270 seconds.
Aug 25 19:38:30 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:38:31 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:38:31 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 241 seconds.
Aug 25 19:42:31 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:42:32 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:42:32 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 266 seconds.
Aug 25 19:46:58 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:46:58 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:46:58 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 239 seconds.
Aug 25 19:50:56 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:50:57 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:50:57 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 232 seconds.
Aug 25 19:54:49 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:54:49 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:54:49 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 294 seconds.
Aug 25 19:59:42 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 19:59:43 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 19:59:43 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 296 seconds.
Aug 25 20:04:39 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:04:39 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:04:39 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 264 seconds.
Aug 25 20:09:02 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:09:03 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:09:03 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 268 seconds.
Aug 25 20:13:31 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:13:31 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:13:31 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 285 seconds.
Aug 25 20:18:15 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:18:16 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:18:16 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 291 seconds.
Aug 25 20:23:06 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:23:07 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:23:07 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 294 seconds.
Aug 25 20:28:01 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:28:01 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:28:01 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 242 seconds.
Aug 25 20:28:44 l01ora3 avahi-daemon[15490]: Registering new address record for 10.2.10.117 on eth2.
Aug 25 20:28:44 l01ora3 avahi-daemon[15490]: Withdrawing address record for 10.2.10.117 on eth2.
Aug 25 20:28:44 l01ora3 avahi-daemon[15490]: Registering new address record for 10.2.10.117 on eth2.
Aug 25 20:32:03 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:32:03 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:32:03 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 228 seconds.
Aug 25 20:35:50 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:35:51 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:35:51 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 243 seconds.
Aug 25 20:35:59 l01ora3 kernel: FS-Cache: Loaded
Aug 25 20:39:53 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:39:54 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:39:54 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 276 seconds.
Aug 25 20:44:30 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:44:30 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:44:30 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 291 seconds.
Aug 25 20:49:20 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:49:21 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:49:21 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 270 seconds.
Aug 25 20:53:50 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:53:51 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:53:51 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 231 seconds.
Aug 25 20:57:42 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 20:57:42 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 20:57:42 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 264 seconds.
Aug 25 21:02:05 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:02:06 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:02:06 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 249 seconds.
Aug 25 21:06:15 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:06:15 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:06:15 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 238 seconds.
Aug 25 21:10:13 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:10:13 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:10:13 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 290 seconds.
Aug 25 21:15:02 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:15:03 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:15:03 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 231 seconds.
Aug 25 21:18:53 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:18:54 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:18:54 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 229 seconds.
Aug 25 21:22:42 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:22:43 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:22:43 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 266 seconds.
Aug 25 21:25:08 l01ora3 auditd[14006]: Audit daemon rotating log files
Aug 25 21:27:09 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:27:09 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:27:09 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 283 seconds.
Aug 25 21:31:52 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:31:52 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:31:52 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 235 seconds.
Aug 25 21:31:52 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:9): Abort command issued -- 1 686d0 2002.
Aug 25 21:31:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:9): Abort command issued -- 1 686d1 2002.
Aug 25 21:31:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:10): Abort command issued -- 1 686d2 2002.
Aug 25 21:31:54 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:2): Abort command issued -- 1 686d3 2002.
Aug 25 21:31:55 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:9): Abort command issued -- 1 686d4 2002.
Aug 25 21:31:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:3): Abort command issued -- 1 686d5 2002.
Aug 25 21:31:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:7): Abort command issued -- 1 686d6 2002.
Aug 25 21:31:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:5): Abort command issued -- 1 686d7 2002.
Aug 25 21:31:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:8): Abort command issued -- 1 686d8 2002.
Aug 25 21:31:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:1): Abort command issued -- 1 686da 2002.
Aug 25 21:31:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 686db 2002.
Aug 25 21:33:28 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:2): Abort command issued -- 1 6a440 2002.
Aug 25 21:33:29 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:9): Abort command issued -- 1 6a442 2002.
Aug 25 21:33:29 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:9): Abort command issued -- 1 6a443 2002.
Aug 25 21:33:29 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:5): Abort command issued -- 1 6a444 2002.
Aug 25 21:33:30 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:12): Abort command issued -- 1 6a445 2002.
Aug 25 21:33:30 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:1): Abort command issued -- 1 6a446 2002.
Aug 25 21:33:30 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:9): Abort command issued -- 1 6a447 2002.
Aug 25 21:33:31 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:2): Abort command issued -- 1 6a448 2002.
Aug 25 21:33:31 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:2): Abort command issued -- 1 6a449 2002.
Aug 25 21:33:31 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:6): Abort command issued -- 1 6a44a 2002.
Aug 25 21:33:32 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:6): Abort command issued -- 1 6a44b 2002.
Aug 25 21:33:32 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:6): Abort command issued -- 1 6a44c 2002.
Aug 25 21:34:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:3): Abort command issued -- 1 6b788 2002.
Aug 25 21:34:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 6b789 2002.
Aug 25 21:34:54 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:8): Abort command issued -- 1 6b78a 2002.
Aug 25 21:34:54 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:1): Abort command issued -- 1 6b78b 2002.
Aug 25 21:34:55 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:4): Abort command issued -- 1 6b78c 2002.
Aug 25 21:34:55 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:10): Abort command issued -- 1 6b78d 2002.
Aug 25 21:34:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:7): Abort command issued -- 1 6b78e 2002.
Aug 25 21:34:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:11): Abort command issued -- 1 6b78f 2002.
Aug 25 21:34:57 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 6b790 2002.
Aug 25 21:35:46 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:35:47 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:35:47 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 294 seconds.
Aug 25 21:37:50 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 7cfce 2002.
Aug 25 21:37:50 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 7cfcf 2002.
Aug 25 21:37:51 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:10): Abort command issued -- 1 7cfd0 2002.
Aug 25 21:37:52 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:8): Abort command issued -- 1 7cfd1 2002.
Aug 25 21:37:52 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:12): Abort command issued -- 1 7cfd2 2002.
Aug 25 21:37:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 7cfd3 2002.
Aug 25 21:37:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:7): Abort command issued -- 1 7cfd4 2002.
Aug 25 21:37:54 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:4): Abort command issued -- 1 7cfd5 2002.
Aug 25 21:37:54 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:11): Abort command issued -- 1 7cfd6 2002.
Aug 25 21:37:55 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:5): Abort command issued -- 1 7cfd7 2002.
Aug 25 21:37:55 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:1): Abort command issued -- 1 7cfd8 2002.
Aug 25 21:37:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 7cfd9 2002.
Aug 25 21:37:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:9): Abort command issued -- 1 7cfda 2002.
Aug 25 21:37:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:9): Abort command issued -- 1 7cfdb 2002.
Aug 25 21:37:57 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:9): Abort command issued -- 1 7cfdc 2002.
Aug 25 21:40:41 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:40:41 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:40:41 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 301 seconds.
Aug 25 21:42:17 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 8e40b 2002.
Aug 25 21:42:17 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:6): Abort command issued -- 1 8e40c 2002.
Aug 25 21:42:18 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:6): Abort command issued -- 1 8e40d 2002.
Aug 25 21:42:18 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:6): Abort command issued -- 1 8e40e 2002.
Aug 25 21:42:18 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:1): Abort command issued -- 1 8e40f 2002.
Aug 25 21:42:19 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:3): Abort command issued -- 1 8e410 2002.
Aug 25 21:42:19 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 8e411 2002.
Aug 25 21:42:20 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:9): Abort command issued -- 1 8e77d 2002.
Aug 25 21:42:20 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:9): Abort command issued -- 1 8e77e 2002.
Aug 25 21:42:21 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:4): Abort command issued -- 1 8e77f 2002.
Aug 25 21:42:22 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:9): Abort command issued -- 1 8e780 2002.
Aug 25 21:42:22 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:7): Abort command issued -- 1 8e781 2002.
Aug 25 21:43:50 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 9017c 2002.
Aug 25 21:43:50 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 9017d 2002.
Aug 25 21:43:51 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 9017e 2002.
Aug 25 21:43:51 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:1): Abort command issued -- 1 9017f 2002.
Aug 25 21:43:52 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:4): Abort command issued -- 1 90180 2002.
Aug 25 21:43:52 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:7): Abort command issued -- 1 90181 2002.
Aug 25 21:43:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:12): Abort command issued -- 1 90182 2002.
Aug 25 21:43:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:5): Abort command issued -- 1 90183 2002.
Aug 25 21:43:54 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 90184 2002.
Aug 25 21:45:42 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:45:42 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:45:42 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 264 seconds.
Aug 25 21:47:13 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 a0ff3 2002.
Aug 25 21:47:14 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 a0ff4 2002.
Aug 25 21:47:14 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 a0ff5 2002.
Aug 25 21:47:14 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:11): Abort command issued -- 1 a0ff6 2002.
Aug 25 21:47:14 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:1): Abort command issued -- 1 a0ff7 2002.
Aug 25 21:47:14 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:5): Abort command issued -- 1 a0ff8 2002.
Aug 25 21:47:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:4): Abort command issued -- 1 a0ff9 2002.
Aug 25 21:47:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:7): Abort command issued -- 1 a0ffa 2002.
Aug 25 21:47:16 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:10): Abort command issued -- 1 a0ffb 2002.
Aug 25 21:47:16 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:8): Abort command issued -- 1 a0ffc 2002.
Aug 25 21:47:16 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 a0ffd 2002.
Aug 25 21:47:17 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 a0ffe 2002.
Aug 25 21:47:17 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:6): Abort command issued -- 1 a0fff 2002.
Aug 25 21:49:45 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 a34b6 2002.
Aug 25 21:49:46 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 a34b7 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:6): Abort command issued -- 1 a34b8 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:6): Abort command issued -- 1 a34b9 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 a34ba 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:5): Abort command issued -- 1 a34bb 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:8): Abort command issued -- 1 a34bc 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:7): Abort command issued -- 1 a34bd 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:11): Abort command issued -- 1 a34be 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:1): Abort command issued -- 1 a34bf 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:4): Abort command issued -- 1 a34c0 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:6): Abort command issued -- 1 a34c1 2002.
Aug 25 21:49:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 a34c2 2002.
Aug 25 21:50:05 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:50:06 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:50:06 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 283 seconds.
Aug 25 21:53:27 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:9): Abort command issued -- 1 b4cf8 2002.
Aug 25 21:53:27 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:6): Abort command issued -- 1 b4cf9 2002.
Aug 25 21:53:28 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:9): Abort command issued -- 1 b4cfa 2002.
Aug 25 21:53:28 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:6): Abort command issued -- 1 b4cfb 2002.
Aug 25 21:53:29 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 b4cfc 2002.
Aug 25 21:53:29 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 b4cfd 2002.
Aug 25 21:53:30 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:8): Abort command issued -- 1 b4cff 2002.
Aug 25 21:53:30 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:11): Abort command issued -- 1 b4d00 2002.
Aug 25 21:53:30 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:3): Abort command issued -- 1 b4d03 2002.
Aug 25 21:54:48 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:54:49 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:54:49 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 253 seconds.
Aug 25 21:54:50 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 b66f3 2002.
Aug 25 21:54:51 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 b66f5 2002.
Aug 25 21:54:51 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:6): Abort command issued -- 1 b66f6 2002.
Aug 25 21:54:52 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:6): Abort command issued -- 1 b66f7 2002.
Aug 25 21:54:52 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:9): Abort command issued -- 1 b66f8 2002.
Aug 25 21:54:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:9): Abort command issued -- 1 b66f9 2002.
Aug 25 21:54:53 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:6): Abort command issued -- 1 b66fa 2002.
Aug 25 21:54:54 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:9): Abort command issued -- 1 b66fb 2002.
Aug 25 21:54:54 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:10): Abort command issued -- 1 b66fc 2002.
Aug 25 21:54:55 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:1): Abort command issued -- 1 b66fd 2002.
Aug 25 21:54:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:4): Abort command issued -- 1 b66fe 2002.
Aug 25 21:54:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:12): Abort command issued -- 1 b66ff 2002.
Aug 25 21:54:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:5): Abort command issued -- 1 b6700 2002.
Aug 25 21:54:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:7): Abort command issued -- 1 b6701 2002.
Aug 25 21:54:56 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 b6702 2002.
Aug 25 21:54:57 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:12): Abort command issued -- 1 b6703 2002.
Aug 25 21:54:57 l01ora3 kernel: sd 3:0:3:12: timing out command, waited 60s
Aug 25 21:58:26 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 c7b2f 2002.
Aug 25 21:58:27 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 c7b30 2002.
Aug 25 21:58:27 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:2): Abort command issued -- 1 c7b31 2002.
Aug 25 21:58:28 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:6): Abort command issued -- 1 c7b32 2002.
Aug 25 21:58:28 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:6): Abort command issued -- 1 c7b33 2002.
Aug 25 21:58:28 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:8): Abort command issued -- 1 c7b34 2002.
Aug 25 21:58:28 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:7): Abort command issued -- 1 c7b35 2002.
Aug 25 21:58:28 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:6): Abort command issued -- 1 c7b36 2002.
Aug 25 21:58:28 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:11): Abort command issued -- 1 c7b37 2002.
Aug 25 21:58:29 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:4): Abort command issued -- 1 c7b38 2002.
Aug 25 21:58:30 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:1): Abort command issued -- 1 c7b39 2002.
Aug 25 21:58:30 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 c7b3a 2002.
Aug 25 21:58:31 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:9): Abort command issued -- 1 c7b3b 2002.
Aug 25 21:58:31 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:9): Abort command issued -- 1 c7b3c 2002.
Aug 25 21:59:02 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 21:59:02 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 21:59:02 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 269 seconds.
Aug 25 21:59:44 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 c811a 2002.
Aug 25 21:59:44 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 c811b 2002.
Aug 25 21:59:44 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:6): Abort command issued -- 1 c811c 2002.
Aug 25 21:59:45 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:6): Abort command issued -- 1 c811d 2002.
Aug 25 21:59:46 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:5): Abort command issued -- 1 c811e 2002.
Aug 25 21:59:46 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:3): Abort command issued -- 1 c811f 2002.
Aug 25 21:59:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:6): Abort command issued -- 1 c8120 2002.
Aug 25 21:59:47 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:8): Abort command issued -- 1 c8121 2002.
Aug 25 21:59:48 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 c8122 2002.
Aug 25 21:59:48 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:11): Abort command issued -- 1 c8123 2002.
Aug 25 21:59:49 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:10): Abort command issued -- 1 c8124 2002.
Aug 25 21:59:49 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:12): Abort command issued -- 1 c8125 2002.
Aug 25 21:59:49 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 c8126 2002.
Aug 25 21:59:50 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 c8127 2002.
Aug 25 21:59:50 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 c8128 2002.
Aug 25 22:02:10 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 cb426 2002.
Aug 25 22:02:11 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:2): Abort command issued -- 1 cb428 2002.
Aug 25 22:02:11 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:9): Abort command issued -- 1 cb429 2002.
Aug 25 22:02:12 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:6): Abort command issued -- 1 cb42a 2002.
Aug 25 22:02:12 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:9): Abort command issued -- 1 cb42b 2002.
Aug 25 22:02:13 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:6): Abort command issued -- 1 cb42c 2002.
Aug 25 22:02:14 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:10): Abort command issued -- 1 cb42d 2002.
Aug 25 22:02:14 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 cb42e 2002.
Aug 25 22:02:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:6): Abort command issued -- 1 cb42f 2002.
Aug 25 22:02:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:8): Abort command issued -- 1 cb430 2002.
Aug 25 22:02:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:9): Abort command issued -- 1 cb431 2002.
Aug 25 22:02:16 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:3:11): Abort command issued -- 1 cb432 2002.
Aug 25 22:03:30 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:03:31 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:03:31 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 296 seconds.
Aug 25 22:05:18 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): Abort command issued -- 1 d9191 2002.
Aug 25 22:07:30 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 dfc08 2002.
Aug 25 22:07:31 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:12): Abort command issued -- 1 dfc0b 2002.
Aug 25 22:07:31 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 dfc0c 2002.
Aug 25 22:07:31 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:10): Abort command issued -- 1 dfc0d 2002.
Aug 25 22:07:32 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:4): Abort command issued -- 1 dfc0e 2002.
Aug 25 22:07:32 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:5): Abort command issued -- 1 dfc0f 2002.
Aug 25 22:07:32 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:7): Abort command issued -- 1 dfc10 2002.
Aug 25 22:07:33 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:11): Abort command issued -- 1 dfc11 2002.
Aug 25 22:07:33 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:8): Abort command issued -- 1 dfc12 2002.
Aug 25 22:07:34 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:1:1): Abort command issued -- 1 dfc13 2002.
Aug 25 22:07:34 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:3): Abort command issued -- 1 dfc14 2002.
Aug 25 22:07:35 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:2): Abort command issued -- 1 dfc15 2002.
Aug 25 22:08:26 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:08:27 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:08:27 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 254 seconds.
Aug 25 22:08:45 l01ora3 kernel: qla2xxx 0000:24:00.0: LOOP DOWN detected (2 0 0).
Aug 25 22:09:15 l01ora3 kernel:  rport-3:0-0: blocked FC remote port time out: saving binding
Aug 25 22:09:15 l01ora3 kernel:  rport-3:0-1: blocked FC remote port time out: saving binding
Aug 25 22:09:15 l01ora3 kernel:  rport-3:0-2: blocked FC remote port time out: saving binding
Aug 25 22:09:15 l01ora3 kernel:  rport-3:0-3: blocked FC remote port time out: saving binding
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:1): DEVICE RESET ISSUED.
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: qla2xxx_eh_device_reset: device reset failed
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:2): DEVICE RESET ISSUED.
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: qla2xxx_eh_device_reset: device reset failed
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:3): DEVICE RESET ISSUED.
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: qla2xxx_eh_device_reset: device reset failed
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:4): DEVICE RESET ISSUED.
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: qla2xxx_eh_device_reset: device reset failed
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:5): DEVICE RESET ISSUED.
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: qla2xxx_eh_device_reset: device reset failed
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:7): DEVICE RESET ISSUED.
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: qla2xxx_eh_device_reset: device reset failed
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:2:10): DEVICE RESET ISSUED.
Aug 25 22:09:15 l01ora3 kernel: qla2xxx 0000:24:00.0: qla2xxx_eh_device_reset: device reset failed
Aug 25 22:09:16 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:4): LOOP RESET ISSUED.
Aug 25 22:09:16 l01ora3 kernel: qla2xxx 0000:24:00.0: qla2xxx_eh_bus_reset: reset failed
Aug 25 22:09:16 l01ora3 kernel: qla2xxx 0000:24:00.0: scsi(3:0:4): ADAPTER RESET ISSUED.
Aug 25 22:09:16 l01ora3 kernel: qla2xxx 0000:24:00.0: Performing ISP error recovery - ha= ffff81107dac44f8.
Aug 25 22:09:36 l01ora3 kernel: qla2xxx 0000:24:00.0: Cable is unplugged...
Aug 25 22:09:36 l01ora3 kernel: qla2xxx 0000:24:00.0: qla2xxx_eh_host_reset: reset succeeded
Aug 25 22:13:34 l01ora3 syslogd 1.4.1: restart.
Aug 25 22:13:34 l01ora3 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Aug 25 22:13:34 l01ora3 kernel: Linux version 2.6.18-238.el5 (mockbuild <at> ca-build10.us.oracle.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Tue Jan 4 15:41:11 EST 2011
Aug 25 22:13:34 l01ora3 kernel: Command line: ro root=/dev/VolGroup00/LogVol00 rhgb quiet
Aug 25 22:13:34 l01ora3 kernel: BIOS-provided physical RAM map:
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 0000000000010000 - 000000000009c000 (usable)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000000009c000 - 00000000000a0000 (reserved)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 0000000000100000 - 000000007cff4000 (usable)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000007cff4000 - 000000007d0b8000 (reserved)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000007d0b8000 - 000000007d585000 (usable)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000007d585000 - 000000007d635000 (reserved)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000007d635000 - 000000007f68f000 (usable)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000007f68f000 - 000000007f6df000 (reserved)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000007f6df000 - 000000007f7df000 (ACPI NVS)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000007f7df000 - 000000007f7ff000 (ACPI data)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000007f7ff000 - 000000007f800000 (usable)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 000000007f800000 - 0000000090000000 (reserved)
Aug 25 22:13:34 l01ora3 cpuspeed: Enabling ondemand cpu frequency scaling governor
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 00000000fc000000 - 00000000fd000000 (reserved)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 00000000fed1c000 - 00000000fed20000 (reserved)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 00000000ff800000 - 0000000100000000 (reserved)
Aug 25 22:13:34 l01ora3 kernel:  BIOS-e820: 0000000100000000 - 0000001080000000 (usable)
Aug 25 22:13:34 l01ora3 kernel: DMI 2.5 present.
Aug 25 22:13:35 l01ora3 rpc.statd[11895]: Version 1.0.9 Starting
Aug 25 22:13:35 l01ora3 rpc.statd[11895]: statd running as root. chown /var/lib/nfs/statd/sm to choose different user 
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 0 -> APIC 0 -> Node 0
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 0 -> APIC 2 -> Node 0
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 0 -> APIC 18 -> Node 0
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 0 -> APIC 20 -> Node 0
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 1 -> APIC 32 -> Node 1
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 1 -> APIC 34 -> Node 1
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 1 -> APIC 50 -> Node 1
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 1 -> APIC 52 -> Node 1
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 0 -> APIC 1 -> Node 0
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 0 -> APIC 3 -> Node 0
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 0 -> APIC 19 -> Node 0
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 0 -> APIC 21 -> Node 0
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 1 -> APIC 33 -> Node 1
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 1 -> APIC 35 -> Node 1
Aug 25 22:13:35 l01ora3 kdump: No crashkernel parameter specified for running kernel
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 1 -> APIC 51 -> Node 1
Aug 25 22:13:35 l01ora3 kdump: failed to start up
Aug 25 22:13:35 l01ora3 kernel: SRAT: PXM 1 -> APIC 53 -> Node 1
Aug 25 22:13:35 l01ora3 kernel: SRAT: Node 0 PXM 0 0-80000000
Aug 25 22:13:35 l01ora3 kernel: SRAT: Node 0 PXM 0 0-880000000
Aug 25 22:13:36 l01ora3 kernel: SRAT: Node 1 PXM 1 880000000-1080000000
Aug 25 22:13:36 l01ora3 kernel: Bootmem setup node 0 0000000000000000-0000000880000000
Aug 25 22:13:36 l01ora3 kernel: Bootmem setup node 1 0000000880000000-0000001080000000
Aug 25 22:13:36 l01ora3 kernel: Memory for crash kernel (0x0 to 0x0) notwithin permissible range
Aug 25 22:13:36 l01ora3 kernel: disabling kdump
Aug 25 22:13:36 l01ora3 modprobe: FATAL: Module ocfs2_stackglue not found. 
Aug 25 22:13:36 l01ora3 hcid[12095]: Bluetooth HCI daemon
Aug 25 22:13:36 l01ora3 kernel: ACPI: PM-Timer IO Port: 0x588
Aug 25 22:13:36 l01ora3 hcid[12095]: Register path:/org/bluez fallback:1
Aug 25 22:13:36 l01ora3 sdpd[12099]: Bluetooth SDP daemon 
Aug 25 22:13:36 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Aug 25 22:13:36 l01ora3 kernel: Processor #0 6:12 APIC version 21
Aug 25 22:13:36 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
Aug 25 22:13:36 l01ora3 kernel: Processor #2 6:12 APIC version 21
Aug 25 22:13:36 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] disabled)
Aug 25 22:13:36 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x03] lapic_id[0x10] disabled)
Aug 25 22:13:36 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x04] lapic_id[0x12] enabled)
Aug 25 22:13:36 l01ora3 kernel: Processor #18 6:12 APIC version 21
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x05] lapic_id[0x14] enabled)
Aug 25 22:13:37 l01ora3 kernel: Processor #20 6:12 APIC version 21
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x06] lapic_id[0x20] enabled)
Aug 25 22:13:37 l01ora3 kernel: Processor #32 6:12 APIC version 21
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x07] lapic_id[0x22] enabled)
Aug 25 22:13:37 l01ora3 kernel: Processor #34 6:12 APIC version 21
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x08] lapic_id[0x24] disabled)
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x09] lapic_id[0x30] disabled)
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x32] enabled)
Aug 25 22:13:37 l01ora3 kernel: Processor #50 6:12 APIC version 21
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x34] enabled)
Aug 25 22:13:37 l01ora3 kernel: Processor #52 6:12 APIC version 21
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x01] enabled)
Aug 25 22:13:37 l01ora3 kernel: Processor #1 6:12 APIC version 21
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x03] enabled)
Aug 25 22:13:37 l01ora3 kernel: Processor #3 6:12 APIC version 21
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x05] disabled)
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x11] disabled)
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x10] lapic_id[0x13] enabled)
Aug 25 22:13:37 l01ora3 kernel: Processor #19 6:12 APIC version 21
Aug 25 22:13:37 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x11] lapic_id[0x15] enabled)
Aug 25 22:13:38 l01ora3 kernel: Processor #21 6:12 APIC version 21
Aug 25 22:13:38 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x12] lapic_id[0x21] enabled)
Aug 25 22:13:38 l01ora3 kernel: Processor #33 6:12 APIC version 21
Aug 25 22:13:38 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x13] lapic_id[0x23] enabled)
Aug 25 22:13:38 l01ora3 kernel: Processor #35 6:12 APIC version 21
Aug 25 22:13:38 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x14] lapic_id[0x25] disabled)
Aug 25 22:13:38 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x15] lapic_id[0x31] disabled)
Aug 25 22:13:38 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x16] lapic_id[0x33] enabled)
Aug 25 22:13:38 l01ora3 kernel: Processor #51 6:12 APIC version 21
Aug 25 22:13:38 l01ora3 kernel: ACPI: LAPIC (acpi_id[0x17] lapic_id[0x35] enabled)
Aug 25 22:13:38 l01ora3 kernel: Processor #53 6:12 APIC version 21
Aug 25 22:13:38 l01ora3 kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
Aug 25 22:13:38 l01ora3 kernel: ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
Aug 25 22:13:38 l01ora3 kernel: IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
Aug 25 22:13:38 l01ora3 kernel: ACPI: IOAPIC (id[0x09] address[0xfec80000] gsi_base[24])
Aug 25 22:13:38 l01ora3 kernel: IOAPIC[1]: apic_id 9, version 32, address 0xfec80000, GSI 24-47
Aug 25 22:13:38 l01ora3 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Aug 25 22:13:38 l01ora3 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Aug 25 22:13:38 l01ora3 kernel: Setting APIC routing to physical flat
Aug 25 22:13:38 l01ora3 kernel: ACPI: HPET id: 0x8086a301 base: 0xfed00000
Aug 25 22:13:38 l01ora3 kernel: Using ACPI (MADT) for SMP configuration information
Aug 25 22:13:38 l01ora3 kernel: Nosave address range: 000000000009c000 - 00000000000a0000
Aug 25 22:13:38 l01ora3 kernel: Nosave address range: 00000000000a0000 - 00000000000e0000
Aug 25 22:13:38 l01ora3 kernel: Nosave address range: 00000000000e0000 - 0000000000100000
Aug 25 22:13:38 l01ora3 kernel: Nosave address range: 000000007cff4000 - 000000007d0b8000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 000000007d585000 - 000000007d635000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 000000007f68f000 - 000000007f6df000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 000000007f6df000 - 000000007f7df000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 000000007f7df000 - 000000007f7ff000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 000000007f800000 - 0000000090000000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 0000000090000000 - 00000000fc000000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 00000000fc000000 - 00000000fd000000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 00000000fd000000 - 00000000fed1c000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 00000000fed1c000 - 00000000fed20000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 00000000fed20000 - 00000000ff800000
Aug 25 22:13:39 l01ora3 kernel: Nosave address range: 00000000ff800000 - 0000000100000000
Aug 25 22:13:39 l01ora3 kernel: Allocating PCI resources starting at 98000000 (gap: 90000000:6c000000)
Aug 25 22:13:39 l01ora3 kernel: SMP: Allowing 24 CPUs, 8 hotplug CPUs
Aug 25 22:13:39 l01ora3 kernel: Built 2 zonelists.  Total pages: 16536215
Aug 25 22:13:39 l01ora3 kernel: Kernel command line: ro root=/dev/VolGroup00/LogVol00 rhgb quiet
Aug 25 22:13:39 l01ora3 kernel: Initializing CPU#0
Aug 25 22:13:39 l01ora3 kernel: PID hash table entries: 4096 (order: 12, 32768 bytes)
Aug 25 22:13:39 l01ora3 kernel: Console: colour VGA+ 80x25
Aug 25 22:13:39 l01ora3 kernel: Dentry cache hash table entries: 8388608 (order: 14, 67108864 bytes)
Aug 25 22:13:39 l01ora3 kernel: Inode-cache hash table entries: 4194304 (order: 13, 33554432 bytes)
Aug 25 22:13:39 l01ora3 kernel: Checking aperture...
Aug 25 22:13:39 l01ora3 kernel: ACPI: DMAR not present
Aug 25 22:13:39 l01ora3 kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Aug 25 22:13:39 l01ora3 kernel: Placing software IO TLB between 0x707f000 - 0xb07f000
Aug 25 22:13:39 l01ora3 kernel: Memory: 65974624k/69206016k available (2592k kernel code, 1122624k reserved, 1649k data, 224k init)
Aug 25 22:13:39 l01ora3 kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 5333.68 BogoMIPS (lpj=2666844)
Aug 25 22:13:39 l01ora3 kernel: Security Framework v1.0.0 initialized
Aug 25 22:13:39 l01ora3 kernel: SELinux:  Initializing.
Aug 25 22:13:39 l01ora3 kernel: selinux_register_security:  Registering secondary module capability
Aug 25 22:13:39 l01ora3 kernel: Capability LSM initialized as secondary
Aug 25 22:13:39 l01ora3 kernel: Mount-cache hash table entries: 256
Aug 25 22:13:39 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:39 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:39 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:40 l01ora3 kernel: CPU 0/0 -> Node 0
Aug 25 22:13:40 l01ora3 kernel: using mwait in idle threads.
Aug 25 22:13:40 l01ora3 kernel: CPU: Physical Processor ID: 0
Aug 25 22:13:40 l01ora3 kernel: CPU: Processor Core ID: 0
Aug 25 22:13:40 l01ora3 kernel: CPU0: Thermal monitoring enabled (TM1)
Aug 25 22:13:40 l01ora3 kernel: SMP alternatives: switching to UP code
Aug 25 22:13:40 l01ora3 kernel: ACPI: Core revision 20060707
Aug 25 22:13:40 l01ora3 kernel: Using local APIC timer interrupts.
Aug 25 22:13:40 l01ora3 kernel: Detected 8.333 MHz APIC timer.
Aug 25 22:13:40 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:40 l01ora3 kernel: Booting processor 1/16 APIC 0x2
Aug 25 22:13:40 l01ora3 kernel: Initializing CPU#1
Aug 25 22:13:40 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.40 BogoMIPS (lpj=2666703)
Aug 25 22:13:40 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:40 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:40 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:40 l01ora3 kernel: CPU 1/2 -> Node 0
Aug 25 22:13:40 l01ora3 kernel: CPU: Physical Processor ID: 0
Aug 25 22:13:40 l01ora3 kernel: CPU: Processor Core ID: 1
Aug 25 22:13:40 l01ora3 kernel: CPU1: Thermal monitoring enabled (TM1)
Aug 25 22:13:40 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:40 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:40 l01ora3 kernel: Booting processor 2/16 APIC 0x12
Aug 25 22:13:40 l01ora3 kernel: Initializing CPU#2
Aug 25 22:13:40 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.48 BogoMIPS (lpj=2666742)
Aug 25 22:13:40 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:40 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:40 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:40 l01ora3 kernel: CPU 2/12 -> Node 0
Aug 25 22:13:40 l01ora3 kernel: CPU: Physical Processor ID: 0
Aug 25 22:13:40 l01ora3 kernel: CPU: Processor Core ID: 9
Aug 25 22:13:40 l01ora3 kernel: CPU2: Thermal monitoring enabled (TM1)
Aug 25 22:13:40 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:40 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:40 l01ora3 kernel: Booting processor 3/16 APIC 0x14
Aug 25 22:13:40 l01ora3 kernel: Initializing CPU#3
Aug 25 22:13:40 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.63 BogoMIPS (lpj=2666818)
Aug 25 22:13:41 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:41 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:41 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:41 l01ora3 kernel: CPU 3/14 -> Node 0
Aug 25 22:13:41 l01ora3 kernel: CPU: Physical Processor ID: 0
Aug 25 22:13:41 l01ora3 kernel: CPU: Processor Core ID: 10
Aug 25 22:13:41 l01ora3 kernel: CPU3: Thermal monitoring enabled (TM1)
Aug 25 22:13:41 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:41 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:41 l01ora3 kernel: Booting processor 4/16 APIC 0x20
Aug 25 22:13:41 l01ora3 kernel: Initializing CPU#4
Aug 25 22:13:41 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.47 BogoMIPS (lpj=2666738)
Aug 25 22:13:41 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:41 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:41 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:41 l01ora3 kernel: CPU 4/20 -> Node 1
Aug 25 22:13:41 l01ora3 kernel: CPU: Physical Processor ID: 1
Aug 25 22:13:41 l01ora3 kernel: CPU: Processor Core ID: 0
Aug 25 22:13:41 l01ora3 kernel: CPU4: Thermal monitoring enabled (TM1)
Aug 25 22:13:41 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:41 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:41 l01ora3 kernel: Booting processor 5/16 APIC 0x22
Aug 25 22:13:41 l01ora3 kernel: Initializing CPU#5
Aug 25 22:13:41 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.47 BogoMIPS (lpj=2666737)
Aug 25 22:13:41 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:41 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:41 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:41 l01ora3 kernel: CPU 5/22 -> Node 1
Aug 25 22:13:41 l01ora3 kernel: CPU: Physical Processor ID: 1
Aug 25 22:13:41 l01ora3 kernel: CPU: Processor Core ID: 1
Aug 25 22:13:41 l01ora3 kernel: CPU5: Thermal monitoring enabled (TM1)
Aug 25 22:13:41 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:41 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:41 l01ora3 kernel: Booting processor 6/16 APIC 0x32
Aug 25 22:13:41 l01ora3 kernel: Initializing CPU#6
Aug 25 22:13:41 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.69 BogoMIPS (lpj=2666846)
Aug 25 22:13:42 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:42 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:42 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:42 l01ora3 kernel: CPU 6/32 -> Node 1
Aug 25 22:13:42 l01ora3 kernel: CPU: Physical Processor ID: 1
Aug 25 22:13:42 l01ora3 kernel: CPU: Processor Core ID: 9
Aug 25 22:13:42 l01ora3 kernel: CPU6: Thermal monitoring enabled (TM1)
Aug 25 22:13:42 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:42 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:42 l01ora3 kernel: Booting processor 7/16 APIC 0x34
Aug 25 22:13:42 l01ora3 kernel: Initializing CPU#7
Aug 25 22:13:42 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.45 BogoMIPS (lpj=2666729)
Aug 25 22:13:42 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:42 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:42 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:42 l01ora3 kernel: CPU 7/34 -> Node 1
Aug 25 22:13:42 l01ora3 kernel: CPU: Physical Processor ID: 1
Aug 25 22:13:42 l01ora3 kernel: CPU: Processor Core ID: 10
Aug 25 22:13:42 l01ora3 kernel: CPU7: Thermal monitoring enabled (TM1)
Aug 25 22:13:42 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:42 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:42 l01ora3 kernel: Booting processor 8/16 APIC 0x1
Aug 25 22:13:42 l01ora3 kernel: Initializing CPU#8
Aug 25 22:13:42 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.45 BogoMIPS (lpj=2666726)
Aug 25 22:13:42 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:42 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:42 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:42 l01ora3 kernel: CPU 8/1 -> Node 0
Aug 25 22:13:42 l01ora3 kernel: CPU: Physical Processor ID: 0
Aug 25 22:13:42 l01ora3 kernel: CPU: Processor Core ID: 0
Aug 25 22:13:42 l01ora3 kernel: CPU8: Thermal monitoring enabled (TM1)
Aug 25 22:13:42 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:42 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:42 l01ora3 kernel: Booting processor 9/16 APIC 0x3
Aug 25 22:13:42 l01ora3 kernel: Initializing CPU#9
Aug 25 22:13:42 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.42 BogoMIPS (lpj=2666712)
Aug 25 22:13:43 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:43 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:43 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:43 l01ora3 kernel: CPU 9/3 -> Node 0
Aug 25 22:13:43 l01ora3 kernel: CPU: Physical Processor ID: 0
Aug 25 22:13:43 l01ora3 kernel: CPU: Processor Core ID: 1
Aug 25 22:13:43 l01ora3 kernel: CPU9: Thermal monitoring enabled (TM1)
Aug 25 22:13:43 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:43 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:43 l01ora3 kernel: Booting processor 10/16 APIC 0x13
Aug 25 22:13:43 l01ora3 kernel: Initializing CPU#10
Aug 25 22:13:43 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.68 BogoMIPS (lpj=2666841)
Aug 25 22:13:43 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:43 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:43 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:43 l01ora3 kernel: CPU 10/13 -> Node 0
Aug 25 22:13:43 l01ora3 kernel: CPU: Physical Processor ID: 0
Aug 25 22:13:43 l01ora3 kernel: CPU: Processor Core ID: 9
Aug 25 22:13:43 l01ora3 kernel: CPU10: Thermal monitoring enabled (TM1)
Aug 25 22:13:43 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:43 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:43 l01ora3 kernel: Booting processor 11/16 APIC 0x15
Aug 25 22:13:43 l01ora3 kernel: Initializing CPU#11
Aug 25 22:13:43 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.45 BogoMIPS (lpj=2666725)
Aug 25 22:13:43 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:43 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:43 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:43 l01ora3 kernel: CPU 11/15 -> Node 0
Aug 25 22:13:43 l01ora3 kernel: CPU: Physical Processor ID: 0
Aug 25 22:13:43 l01ora3 kernel: CPU: Processor Core ID: 10
Aug 25 22:13:43 l01ora3 kernel: CPU11: Thermal monitoring enabled (TM1)
Aug 25 22:13:43 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:43 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:43 l01ora3 kernel: Booting processor 12/16 APIC 0x21
Aug 25 22:13:43 l01ora3 kernel: Initializing CPU#12
Aug 25 22:13:44 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.46 BogoMIPS (lpj=2666730)
Aug 25 22:13:44 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:44 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:44 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:44 l01ora3 kernel: CPU 12/21 -> Node 1
Aug 25 22:13:44 l01ora3 kernel: CPU: Physical Processor ID: 1
Aug 25 22:13:44 l01ora3 kernel: CPU: Processor Core ID: 0
Aug 25 22:13:44 l01ora3 kernel: CPU12: Thermal monitoring enabled (TM1)
Aug 25 22:13:44 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:44 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:44 l01ora3 kernel: Booting processor 13/16 APIC 0x23
Aug 25 22:13:44 l01ora3 kernel: Initializing CPU#13
Aug 25 22:13:44 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.69 BogoMIPS (lpj=2666845)
Aug 25 22:13:44 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:44 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:44 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:44 l01ora3 kernel: CPU 13/23 -> Node 1
Aug 25 22:13:44 l01ora3 kernel: CPU: Physical Processor ID: 1
Aug 25 22:13:44 l01ora3 kernel: CPU: Processor Core ID: 1
Aug 25 22:13:44 l01ora3 kernel: CPU13: Thermal monitoring enabled (TM1)
Aug 25 22:13:44 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:44 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:44 l01ora3 kernel: Booting processor 14/16 APIC 0x33
Aug 25 22:13:44 l01ora3 kernel: Initializing CPU#14
Aug 25 22:13:44 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.53 BogoMIPS (lpj=2666767)
Aug 25 22:13:44 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:44 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:44 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:44 l01ora3 kernel: CPU 14/33 -> Node 1
Aug 25 22:13:44 l01ora3 kernel: CPU: Physical Processor ID: 1
Aug 25 22:13:44 l01ora3 kernel: CPU: Processor Core ID: 9
Aug 25 22:13:44 l01ora3 kernel: CPU14: Thermal monitoring enabled (TM1)
Aug 25 22:13:44 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:44 l01ora3 kernel: SMP alternatives: switching to SMP code
Aug 25 22:13:44 l01ora3 kernel: Booting processor 15/16 APIC 0x35
Aug 25 22:13:44 l01ora3 kernel: Initializing CPU#15
Aug 25 22:13:44 l01ora3 kernel: Calibrating delay using timer specific routine.. 5333.45 BogoMIPS (lpj=2666728)
Aug 25 22:13:44 l01ora3 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Aug 25 22:13:44 l01ora3 kernel: CPU: L2 cache: 256K
Aug 25 22:13:45 l01ora3 kernel: CPU: L3 cache: 12288K
Aug 25 22:13:45 l01ora3 kernel: CPU 15/35 -> Node 1
Aug 25 22:13:45 l01ora3 kernel: CPU: Physical Processor ID: 1
Aug 25 22:13:45 l01ora3 kernel: CPU: Processor Core ID: 10
Aug 25 22:13:45 l01ora3 kernel: CPU15: Thermal monitoring enabled (TM1)
Aug 25 22:13:45 l01ora3 kernel: Intel(R) Xeon(R) CPU           E5640   <at>  2.67GHz stepping 02
Aug 25 22:13:45 l01ora3 kernel: Brought up 16 CPUs
Aug 25 22:13:45 l01ora3 kernel: NMI watchdog testing PASSED.
Aug 25 22:13:45 l01ora3 kernel: time.c: Using 14.318180 MHz WALL HPET GTOD HPET/TSC timer.
Aug 25 22:13:45 l01ora3 kernel: time.c: Detected 2666.844 MHz processor.
Aug 25 22:13:45 l01ora3 kernel: migration_cost=1,23,1475
Aug 25 22:13:45 l01ora3 kernel: checking if image is initramfs... it is
Aug 25 22:13:45 l01ora3 kernel: Freeing initrd memory: 4637k freed
Aug 25 22:13:45 l01ora3 kernel: NET: Registered protocol family 16
Aug 25 22:13:45 l01ora3 kernel: ACPI: bus type pci registered
Aug 25 22:13:45 l01ora3 kernel: PCI: Using MMCONFIG at 80000000
Aug 25 22:13:45 l01ora3 kernel: ACPI: Interpreter enabled
Aug 25 22:13:45 l01ora3 kernel: ACPI: Using IOAPIC for interrupt routing
Aug 25 22:13:45 l01ora3 kernel: ACPI: No dock devices found.
Aug 25 22:13:45 l01ora3 kernel: ACPI: PCI Root Bridge [PCI0] (0000:00)
Aug 25 22:13:45 l01ora3 kernel: PCI: Transparent bridge - 0000:00:1e.0
Aug 25 22:13:45 l01ora3 kernel: ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
Aug 25 22:13:45 l01ora3 kernel: ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
Aug 25 22:13:45 l01ora3 kernel: ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 11 12 14 *15)
Aug 25 22:13:45 l01ora3 kernel: ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
Aug 25 22:13:45 l01ora3 kernel: ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
Aug 25 22:13:45 l01ora3 kernel: ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
Aug 25 22:13:45 l01ora3 kernel: ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
Aug 25 22:13:46 l01ora3 kernel: ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
Aug 25 22:13:46 l01ora3 kernel: Linux Plug and Play Support v0.97 (c) Adam Belay
Aug 25 22:13:46 l01ora3 kernel: pnp: PnP ACPI init
Aug 25 22:13:46 l01ora3 kernel: pnp: PnP ACPI: found 13 devices
Aug 25 22:13:46 l01ora3 kernel: usbcore: registered new driver usbfs
Aug 25 22:13:46 l01ora3 kernel: usbcore: registered new driver hub
Aug 25 22:13:46 l01ora3 kernel: PCI: Using ACPI for IRQ routing
Aug 25 22:13:46 l01ora3 kernel: PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
Aug 25 22:13:46 l01ora3 kernel: NetLabel: Initializing
Aug 25 22:13:46 l01ora3 kernel: NetLabel:  domain hash size = 128
Aug 25 22:13:46 l01ora3 kernel: NetLabel:  protocols = UNLABELED CIPSOv4
Aug 25 22:13:46 l01ora3 kernel: NetLabel:  unlabeled traffic allowed by default
Aug 25 22:13:46 l01ora3 kernel: hpet0: at MMIO 0xfed00000 (virtual 0xffffffffff5fe000), IRQs 2, 8, 0, 0
Aug 25 22:13:46 l01ora3 kernel: hpet0: 4 64-bit timers, 14318180 Hz
Aug 25 22:13:46 l01ora3 kernel: ACPI: DMAR not present
Aug 25 22:13:46 l01ora3 kernel: PCI-GART: No AMD northbridge found.
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:05: iomem range 0xfed00000-0xfed003ff has been reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:06: ioport range 0x400-0x43f has been reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0a: iomem range 0xfed1c000-0xfed1ffff could not be reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0a: iomem range 0xfec00000-0xfecfffff has been reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0a: iomem range 0xfee00000-0xfeefffff has been reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0a: iomem range 0x80000000-0x8fffffff could not be reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0b: ioport range 0xca0-0xca1 has been reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0b: ioport range 0xca4-0xca9 has been reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0b: ioport range 0xcaa-0xcab has been reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0b: ioport range 0xcac-0xccb has been reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0c: ioport range 0xca2-0xca2 has been reserved
Aug 25 22:13:46 l01ora3 kernel: pnp: 00:0c: ioport range 0xca3-0xca3 has been reserved
Aug 25 22:13:46 l01ora3 kernel: PCI: Bridge: 0000:00:01.0
Aug 25 22:13:46 l01ora3 kernel:   IO window: 3000-3fff
Aug 25 22:13:46 l01ora3 kernel:   MEM window: 95f00000-95ffffff
Aug 25 22:13:46 l01ora3 kernel:   PREFETCH window 0x0000000098000000-0x00000000981fffff
Aug 25 22:13:46 l01ora3 kernel: PCI: Bridge: 0000:00:03.0
Aug 25 22:13:46 l01ora3 kernel:   IO window: 2000-2fff
Aug 25 22:13:46 l01ora3 kernel:   MEM window: 95a00000-95efffff
Aug 25 22:13:46 l01ora3 kernel:   PREFETCH window 0x0000000098200000-0x00000000982fffff
Aug 25 22:13:46 l01ora3 kernel: PCI: Bridge: 0000:00:05.0
Aug 25 22:13:46 l01ora3 kernel:   IO window: disabled.
Aug 25 22:13:46 l01ora3 kernel:   MEM window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   PREFETCH window: disabled.
Aug 25 22:13:47 l01ora3 kernel: PCI: mem resource #6:800 <at> fc000000 for 0000:10:00.0 was not allocated.
Aug 25 22:13:47 l01ora3 kernel: PCI: Bridge: 0000:00:07.0
Aug 25 22:13:47 l01ora3 kernel:   IO window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   MEM window: 92000000-93ffffff
Aug 25 22:13:47 l01ora3 kernel:   PREFETCH window 0x00000000fa000000-0x00000000fbffffff
Aug 25 22:13:47 l01ora3 kernel: PCI: Bridge: 0000:00:08.0
Aug 25 22:13:47 l01ora3 kernel:   IO window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   MEM window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   PREFETCH window: disabled.
Aug 25 22:13:47 l01ora3 kernel: PCI: Bridge: 0000:00:09.0
Aug 25 22:13:47 l01ora3 kernel:   IO window: 1000-1fff
Aug 25 22:13:47 l01ora3 kernel:   MEM window: 95900000-959fffff
Aug 25 22:13:47 l01ora3 kernel:   PREFETCH window 0x0000000098300000-0x00000000983fffff
Aug 25 22:13:47 l01ora3 kernel: PCI: Bridge: 0000:00:1c.0
Aug 25 22:13:47 l01ora3 kernel:   IO window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   MEM window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   PREFETCH window: disabled.
Aug 25 22:13:47 l01ora3 kernel: PCI: Bridge: 0000:06:00.0
Aug 25 22:13:47 l01ora3 kernel:   IO window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   MEM window: 95000000-958fffff
Aug 25 22:13:47 l01ora3 kernel:   PREFETCH window 0x0000000094000000-0x0000000094ffffff
Aug 25 22:13:47 l01ora3 kernel: PCI: Bridge: 0000:00:1c.4
Aug 25 22:13:47 l01ora3 kernel:   IO window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   MEM window: 95000000-958fffff
Aug 25 22:13:47 l01ora3 kernel:   PREFETCH window 0x0000000094000000-0x0000000094ffffff
Aug 25 22:13:47 l01ora3 kernel: PCI: Bridge: 0000:00:1e.0
Aug 25 22:13:47 l01ora3 kernel:   IO window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   MEM window: disabled.
Aug 25 22:13:47 l01ora3 kernel:   PREFETCH window: disabled.
Aug 25 22:13:47 l01ora3 kernel: GSI 16 sharing vector 0xA9 and IRQ 16
Aug 25 22:13:47 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:01.0[A] -> GSI 28 (level, low) -> IRQ 169
Aug 25 22:13:47 l01ora3 kernel: GSI 17 sharing vector 0xB1 and IRQ 17
Aug 25 22:13:47 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:03.0[A] -> GSI 24 (level, low) -> IRQ 177
Aug 25 22:13:48 l01ora3 kernel: GSI 18 sharing vector 0xB9 and IRQ 18
Aug 25 22:13:48 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:05.0[A] -> GSI 26 (level, low) -> IRQ 185
Aug 25 22:13:48 l01ora3 kernel: GSI 19 sharing vector 0xC1 and IRQ 19
Aug 25 22:13:48 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:07.0[A] -> GSI 30 (level, low) -> IRQ 193
Aug 25 22:13:48 l01ora3 kernel: GSI 20 sharing vector 0xC9 and IRQ 20
Aug 25 22:13:48 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:08.0[A] -> GSI 31 (level, low) -> IRQ 201
Aug 25 22:13:48 l01ora3 kernel: GSI 21 sharing vector 0xD1 and IRQ 21
Aug 25 22:13:48 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:09.0[A] -> GSI 32 (level, low) -> IRQ 209
Aug 25 22:13:48 l01ora3 kernel: GSI 22 sharing vector 0xD9 and IRQ 22
Aug 25 22:13:48 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI 20 (level, low) -> IRQ 217
Aug 25 22:13:48 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:1c.4[A] -> GSI 20 (level, low) -> IRQ 217
Aug 25 22:13:48 l01ora3 kernel: ACPI: PCI Interrupt 0000:06:00.0[A] -> GSI 20 (level, low) -> IRQ 217
Aug 25 22:13:48 l01ora3 kernel: NET: Registered protocol family 2
Aug 25 22:13:48 l01ora3 kernel: IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
Aug 25 22:13:48 l01ora3 kernel: TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
Aug 25 22:13:48 l01ora3 kernel: TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
Aug 25 22:13:48 l01ora3 kernel: TCP: Hash tables configured (established 262144 bind 65536)
Aug 25 22:13:48 l01ora3 kernel: TCP reno registered
Aug 25 22:13:48 l01ora3 kernel: audit: initializing netlink socket (disabled)
Aug 25 22:13:48 l01ora3 kernel: type=2000 audit(1314303134.462:1): initialized
Aug 25 22:13:48 l01ora3 kernel: Total HugeTLB memory allocated, 0
Aug 25 22:13:48 l01ora3 kernel: VFS: Disk quotas dquot_6.5.1
Aug 25 22:13:48 l01ora3 kernel: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Aug 25 22:13:48 l01ora3 kernel: Initializing Cryptographic API
Aug 25 22:13:48 l01ora3 kernel: alg: No test for crc32c (crc32c-generic)
Aug 25 22:13:48 l01ora3 kernel: ksign: Installing public key data
Aug 25 22:13:48 l01ora3 kernel: Loading keyring
Aug 25 22:13:48 l01ora3 kernel: - Added public key 895817073326F70
Aug 25 22:13:48 l01ora3 kernel: - User ID: Red Hat, Inc. (Kernel Module GPG key)
Aug 25 22:13:48 l01ora3 kernel: io scheduler noop registered
Aug 25 22:13:48 l01ora3 kernel: io scheduler anticipatory registered
Aug 25 22:13:48 l01ora3 kernel: io scheduler deadline registered
Aug 25 22:13:48 l01ora3 kernel: io scheduler cfq registered (default)
Aug 25 22:13:49 l01ora3 kernel: pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Aug 25 22:13:49 l01ora3 kernel: Real Time Clock Driver v1.12ac
Aug 25 22:13:49 l01ora3 kernel: Non-volatile memory driver v1.2
Aug 25 22:13:49 l01ora3 kernel: Linux agpgart interface v0.101 (c) Dave Jones
Aug 25 22:13:49 l01ora3 kernel: Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
Aug 25 22:13:49 l01ora3 kernel: serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
Aug 25 22:13:49 l01ora3 kernel: serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
Aug 25 22:13:49 l01ora3 kernel: 00:07: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
Aug 25 22:13:49 l01ora3 kernel: 00:08: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
Aug 25 22:13:49 l01ora3 kernel: brd: module loaded
Aug 25 22:13:49 l01ora3 kernel: Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
Aug 25 22:13:49 l01ora3 kernel: ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Aug 25 22:13:49 l01ora3 kernel: ide-floppy driver 0.99.newide
Aug 25 22:13:49 l01ora3 kernel: usbcore: registered new driver hiddev
Aug 25 22:13:49 l01ora3 kernel: usbcore: registered new driver usbhid
Aug 25 22:13:49 l01ora3 kernel: drivers/usb/input/hid-core.c: v2.6:USB HID core driver
Aug 25 22:13:49 l01ora3 kernel: PNP: No PS/2 controller found. Probing ports directly.
Aug 25 22:13:49 l01ora3 kernel: serio: i8042 KBD port at 0x60,0x64 irq 1
Aug 25 22:13:49 l01ora3 kernel: mice: PS/2 mouse device common for all mice
Aug 25 22:13:49 l01ora3 kernel: md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
Aug 25 22:13:49 l01ora3 kernel: md: bitmap version 4.39
Aug 25 22:13:49 l01ora3 kernel: TCP bic registered
Aug 25 22:13:49 l01ora3 kernel: Initializing IPsec netlink socket
Aug 25 22:13:49 l01ora3 kernel: NET: Registered protocol family 1
Aug 25 22:13:49 l01ora3 kernel: NET: Registered protocol family 17
Aug 25 22:13:49 l01ora3 kernel: ACPI: (supports S0 S1 S5)
Aug 25 22:13:49 l01ora3 kernel: Initalizing network drop monitor service
Aug 25 22:13:49 l01ora3 kernel: Freeing unused kernel memory: 224k freed
Aug 25 22:13:49 l01ora3 kernel: Write protecting the kernel read-only data: 519k
Aug 25 22:13:49 l01ora3 kernel: GSI 23 sharing vector 0x6A and IRQ 23
Aug 25 22:13:49 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:1a.7[C] -> GSI 16 (level, low) -> IRQ 106
Aug 25 22:13:49 l01ora3 kernel: ehci_hcd 0000:00:1a.7: EHCI Host Controller
Aug 25 22:13:49 l01ora3 kernel: ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 1
Aug 25 22:13:49 l01ora3 kernel: ehci_hcd 0000:00:1a.7: debug port 1
Aug 25 22:13:49 l01ora3 kernel: ehci_hcd 0000:00:1a.7: irq 106, io mem 0x96021000
Aug 25 22:13:49 l01ora3 kernel: ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
Aug 25 22:13:49 l01ora3 kernel: usb usb1: configuration #1 chosen from 1 choice
Aug 25 22:13:49 l01ora3 kernel: hub 1-0:1.0: USB hub found
Aug 25 22:13:49 l01ora3 kernel: hub 1-0:1.0: 6 ports detected
Aug 25 22:13:49 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI 20 (level, low) -> IRQ 217
Aug 25 22:13:50 l01ora3 kernel: ehci_hcd 0000:00:1d.7: EHCI Host Controller
Aug 25 22:13:50 l01ora3 kernel: ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2
Aug 25 22:13:50 l01ora3 kernel: ehci_hcd 0000:00:1d.7: debug port 1
Aug 25 22:13:50 l01ora3 kernel: ehci_hcd 0000:00:1d.7: irq 217, io mem 0x96020000
Aug 25 22:13:50 l01ora3 kernel: ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
Aug 25 22:13:50 l01ora3 kernel: usb usb2: configuration #1 chosen from 1 choice
Aug 25 22:13:50 l01ora3 kernel: hub 2-0:1.0: USB hub found
Aug 25 22:13:50 l01ora3 kernel: hub 2-0:1.0: 6 ports detected
Aug 25 22:13:50 l01ora3 kernel: USB Universal Host Controller Interface driver v3.0
Aug 25 22:13:50 l01ora3 kernel: GSI 24 sharing vector 0x72 and IRQ 24
Aug 25 22:13:50 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:1a.0[A] -> GSI 22 (level, low) -> IRQ 114
Aug 25 22:13:50 l01ora3 kernel: uhci_hcd 0000:00:1a.0: UHCI Host Controller
Aug 25 22:13:50 l01ora3 kernel: uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
Aug 25 22:13:50 l01ora3 kernel: uhci_hcd 0000:00:1a.0: irq 114, io base 0x00004080
Aug 25 22:13:50 l01ora3 kernel: usb usb3: configuration #1 chosen from 1 choice
Aug 25 22:13:50 l01ora3 kernel: hub 3-0:1.0: USB hub found
Aug 25 22:13:50 l01ora3 kernel: hub 3-0:1.0: 2 ports detected
Aug 25 22:13:50 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 20 (level, low) -> IRQ 217
Aug 25 22:13:50 l01ora3 kernel: uhci_hcd 0000:00:1d.0: UHCI Host Controller
Aug 25 22:13:50 l01ora3 kernel: uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 4
Aug 25 22:13:50 l01ora3 kernel: uhci_hcd 0000:00:1d.0: irq 217, io base 0x00004060
Aug 25 22:13:50 l01ora3 kernel: usb usb4: configuration #1 chosen from 1 choice
Aug 25 22:13:50 l01ora3 kernel: hub 4-0:1.0: USB hub found
Aug 25 22:13:50 l01ora3 kernel: hub 4-0:1.0: 2 ports detected
Aug 25 22:13:50 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 20 (level, low) -> IRQ 217
Aug 25 22:13:50 l01ora3 kernel: uhci_hcd 0000:00:1d.1: UHCI Host Controller
Aug 25 22:13:50 l01ora3 kernel: uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 5
Aug 25 22:13:50 l01ora3 kernel: uhci_hcd 0000:00:1d.1: irq 217, io base 0x00004040
Aug 25 22:13:50 l01ora3 kernel: usb usb5: configuration #1 chosen from 1 choice
Aug 25 22:13:50 l01ora3 kernel: hub 5-0:1.0: USB hub found
Aug 25 22:13:50 l01ora3 kernel: hub 5-0:1.0: 2 ports detected
Aug 25 22:13:50 l01ora3 kernel: usb 2-3: new high speed USB device using ehci_hcd and address 2
Aug 25 22:13:50 l01ora3 kernel: GSI 25 sharing vector 0x7A and IRQ 25
Aug 25 22:13:50 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI 21 (level, low) -> IRQ 122
Aug 25 22:13:51 l01ora3 kernel: uhci_hcd 0000:00:1d.2: UHCI Host Controller
Aug 25 22:13:51 l01ora3 kernel: uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 6
Aug 25 22:13:51 l01ora3 kernel: uhci_hcd 0000:00:1d.2: irq 122, io base 0x00004020
Aug 25 22:13:51 l01ora3 kernel: usb usb6: configuration #1 chosen from 1 choice
Aug 25 22:13:51 l01ora3 kernel: hub 6-0:1.0: USB hub found
Aug 25 22:13:51 l01ora3 kernel: hub 6-0:1.0: 2 ports detected
Aug 25 22:13:51 l01ora3 kernel: usb 2-3: configuration #1 chosen from 1 choice
Aug 25 22:13:51 l01ora3 kernel: hub 2-3:1.0: USB hub found
Aug 25 22:13:51 l01ora3 kernel: hub 2-3:1.0: 2 ports detected
Aug 25 22:13:51 l01ora3 kernel: SCSI subsystem initialized
Aug 25 22:13:51 l01ora3 kernel: Fusion MPT base driver 3.04.15rh
Aug 25 22:13:51 l01ora3 kernel: Copyright (c) 1999-2008 LSI Corporation
Aug 25 22:13:51 l01ora3 kernel: Fusion MPT SAS Host driver 3.04.15rh
Aug 25 22:13:51 l01ora3 kernel: ACPI: PCI Interrupt 0000:0b:00.0[A] -> GSI 28 (level, low) -> IRQ 169
Aug 25 22:13:51 l01ora3 kernel: mptbase: ioc0: Initiating bringup
Aug 25 22:13:51 l01ora3 kernel: usb 3-2: new full speed USB device using uhci_hcd and address 2
Aug 25 22:13:51 l01ora3 kernel: ioc0: LSISAS1064E: Capabilities={Initiator}
Aug 25 22:13:51 l01ora3 kernel: usb 3-2: configuration #1 chosen from 2 choices
Aug 25 22:13:51 l01ora3 kernel: scsi0 : ioc0: LSISAS1064E, FwRev=011e0600h, Ports=1, MaxQ=277, IRQ=169
Aug 25 22:13:51 l01ora3 kernel: mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 3, phy 0, sas_addr 0x5000c500398074f1
Aug 25 22:13:51 l01ora3 kernel:   Vendor: IBM-ESXS  Model: ST9146803SS       Rev: B53B
Aug 25 22:13:51 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:51 l01ora3 kernel: mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 2, phy 1, sas_addr 0x5000c5003975b325
Aug 25 22:13:51 l01ora3 kernel:   Vendor: IBM-ESXS  Model: ST9146803SS       Rev: B53B
Aug 25 22:13:51 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:51 l01ora3 kernel: mptsas: ioc0: attaching raid volume, channel 1, id 1
Aug 25 22:13:51 l01ora3 kernel:   Vendor: LSILOGIC  Model: Logical Volume    Rev: 3000
Aug 25 22:13:51 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 02
Aug 25 22:13:51 l01ora3 kernel: SCSI device sda: 285155328 512-byte hdwr sectors (146000 MB)
Aug 25 22:13:51 l01ora3 kernel: sda: Write Protect is off
Aug 25 22:13:51 l01ora3 kernel: SCSI device sda: drive cache: write through
Aug 25 22:13:51 l01ora3 kernel: SCSI device sda: 285155328 512-byte hdwr sectors (146000 MB)
Aug 25 22:13:51 l01ora3 kernel: sda: Write Protect is off
Aug 25 22:13:51 l01ora3 kernel: SCSI device sda: drive cache: write through
Aug 25 22:13:51 l01ora3 kernel:  sda: sda1 sda2
Aug 25 22:13:51 l01ora3 kernel: sd 0:1:1:0: Attached scsi disk sda
Aug 25 22:13:51 l01ora3 kernel: shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
Aug 25 22:13:51 l01ora3 kernel: QLogic Fibre Channel HBA Driver
Aug 25 22:13:51 l01ora3 kernel: GSI 26 sharing vector 0x82 and IRQ 26
Aug 25 22:13:52 l01ora3 kernel: ACPI: PCI Interrupt 0000:15:00.2[C] -> GSI 35 (level, low) -> IRQ 130
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.2: Found an ISP8001, irq 130, iobase 0xffffc2000002e000
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.2: Configuring PCI space...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.2: Configure NVRAM parameters...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.2: Verifying loaded RISC code...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.2: Allocated (64 KB) for EFT...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.2: Allocated (1414 KB) for firmware dump...
Aug 25 22:13:52 l01ora3 kernel: scsi1 : qla2xxx
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.2: 
Aug 25 22:13:52 l01ora3 kernel:  QLogic Fibre Channel HBA Driver: 8.03.01.05.05.06-k
Aug 25 22:13:52 l01ora3 kernel:   QLogic QMI8142 - QLogic 2-Port 10Gb CNA (CFFh) for IBM BladeCenter
Aug 25 22:13:52 l01ora3 kernel:   ISP8001: PCIe (5.0Gb/s x4)  <at>  0000:15:00.2 hdma+, host#=1, fw=5.03.05 (8d4)
Aug 25 22:13:52 l01ora3 kernel: GSI 27 sharing vector 0x9A and IRQ 27
Aug 25 22:13:52 l01ora3 kernel: ACPI: PCI Interrupt 0000:15:00.3[D] -> GSI 36 (level, low) -> IRQ 154
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.3: Found an ISP8001, irq 154, iobase 0xffffc20000032000
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.3: Configuring PCI space...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.3: Configure NVRAM parameters...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.3: Verifying loaded RISC code...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.3: Allocated (64 KB) for EFT...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.3: Allocated (1414 KB) for firmware dump...
Aug 25 22:13:52 l01ora3 kernel: scsi2 : qla2xxx
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:15:00.3: 
Aug 25 22:13:52 l01ora3 kernel:  QLogic Fibre Channel HBA Driver: 8.03.01.05.05.06-k
Aug 25 22:13:52 l01ora3 kernel:   QLogic QMI8142 - QLogic 2-Port 10Gb CNA (CFFh) for IBM BladeCenter
Aug 25 22:13:52 l01ora3 kernel:   ISP8001: PCIe (5.0Gb/s x4)  <at>  0000:15:00.3 hdma+, host#=2, fw=5.03.05 (8d4)
Aug 25 22:13:52 l01ora3 kernel: ACPI: PCI Interrupt 0000:24:00.0[A] -> GSI 32 (level, low) -> IRQ 209
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:24:00.0: Found an ISP2532, irq 209, iobase 0xffffc20000036000
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:24:00.0: Configuring PCI space...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:24:00.0: Configure NVRAM parameters...
Aug 25 22:13:52 l01ora3 kernel: qla2xxx 0000:24:00.0: Verifying loaded RISC code...
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.0: Allocated (64 KB) for EFT...
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.0: Allocated (1414 KB) for firmware dump...
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.0: Unable to read FCP priority data.
Aug 25 22:13:53 l01ora3 kernel: scsi3 : qla2xxx
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.0: 
Aug 25 22:13:53 l01ora3 kernel:  QLogic Fibre Channel HBA Driver: 8.03.01.05.05.06-k
Aug 25 22:13:53 l01ora3 kernel:   QLogic QMI2582 - QLogic 8Gb Fibre Channel Expansion Card (CIOv) for IBM BladeCenter
Aug 25 22:13:53 l01ora3 kernel:   ISP2532: PCIe (5.0Gb/s x4)  <at>  0000:24:00.0 hdma+, host#=3, fw=5.03.02 (d5)
Aug 25 22:13:53 l01ora3 kernel: GSI 28 sharing vector 0xC2 and IRQ 28
Aug 25 22:13:53 l01ora3 kernel: ACPI: PCI Interrupt 0000:24:00.1[B] -> GSI 42 (level, low) -> IRQ 194
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.1: Found an ISP2532, irq 194, iobase 0xffffc2000003a000
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.1: Configuring PCI space...
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.1: Configure NVRAM parameters...
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.1: Verifying loaded RISC code...
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.1: Allocated (64 KB) for EFT...
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.1: Allocated (1414 KB) for firmware dump...
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.1: Unable to read FCP priority data.
Aug 25 22:13:53 l01ora3 kernel: scsi4 : qla2xxx
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.1: 
Aug 25 22:13:53 l01ora3 kernel:  QLogic Fibre Channel HBA Driver: 8.03.01.05.05.06-k
Aug 25 22:13:53 l01ora3 kernel:   QLogic QMI2582 - QLogic 8Gb Fibre Channel Expansion Card (CIOv) for IBM BladeCenter
Aug 25 22:13:53 l01ora3 kernel:   ISP2532: PCIe (5.0Gb/s x4)  <at>  0000:24:00.1 hdma+, host#=4, fw=5.03.02 (d5)
Aug 25 22:13:53 l01ora3 kernel: Initializing USB Mass Storage driver...
Aug 25 22:13:53 l01ora3 kernel: usbcore: registered new driver usb-storage
Aug 25 22:13:53 l01ora3 kernel: USB Mass Storage support registered.
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.0: LOOP UP detected (8 Gbps).
Aug 25 22:13:53 l01ora3 kernel: device-mapper: uevent: version 1.0.3
Aug 25 22:13:53 l01ora3 kernel: device-mapper: ioctl: 4.11.5-ioctl (2007-12-12) initialised: dm-devel <at> redhat.com
Aug 25 22:13:53 l01ora3 kernel: device-mapper: dm-raid45: initialized v0.2594l
Aug 25 22:13:53 l01ora3 kernel: qla2xxx 0000:24:00.1: LOOP UP detected (8 Gbps).
Aug 25 22:13:53 l01ora3 kernel: EXT3-fs: INFO: recovery required on readonly filesystem.
Aug 25 22:13:53 l01ora3 kernel: EXT3-fs: write access will be enabled during recovery.
Aug 25 22:13:53 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:13:53 l01ora3 kernel: EXT3-fs: dm-0: orphan cleanup on readonly fs
Aug 25 22:13:53 l01ora3 kernel: EXT3-fs: dm-0: 1 orphan inode deleted
Aug 25 22:13:53 l01ora3 kernel: EXT3-fs: recovery complete.
Aug 25 22:13:53 l01ora3 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Aug 25 22:13:53 l01ora3 kernel: SELinux:  Disabled at runtime.
Aug 25 22:13:54 l01ora3 kernel: type=1404 audit(1314303181.027:2): selinux=0 auid=4294967295 ses=4294967295
Aug 25 22:13:54 l01ora3 kernel: 802.1Q VLAN Support v1.8 Ben Greear <greearb <at> candelatech.com>
Aug 25 22:13:54 l01ora3 kernel: All bugs added by David S. Miller <davem <at> redhat.com>
Aug 25 22:13:54 l01ora3 kernel: input: PC Speaker as /class/input/input0
Aug 25 22:13:54 l01ora3 kernel: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.0.8-rh (Oct 11, 2010)
Aug 25 22:13:54 l01ora3 kernel: ACPI: PCI Interrupt 0000:10:00.0[A] -> GSI 30 (level, low) -> IRQ 193
Aug 25 22:13:54 l01ora3 kernel: eth0: Broadcom NetXtreme II BCM5709 1000Base-SX (C0) PCI Express found at mem fa000000, IRQ 193, node addr 5cf3fc2148f4
Aug 25 22:13:54 l01ora3 kernel: GSI 29 sharing vector 0xDA and IRQ 29
Aug 25 22:13:54 l01ora3 kernel: ACPI: PCI Interrupt 0000:10:00.1[B] -> GSI 37 (level, low) -> IRQ 218
Aug 25 22:13:54 l01ora3 kernel: eth1: Broadcom NetXtreme II BCM5709 1000Base-SX (C0) PCI Express found at mem 92000000, IRQ 218, node addr 5cf3fc2148f6
Aug 25 22:13:54 l01ora3 kernel: scsi 0:0:0:0: Attached scsi generic sg0 type 0
Aug 25 22:13:54 l01ora3 kernel: scsi 0:0:1:0: Attached scsi generic sg1 type 0
Aug 25 22:13:54 l01ora3 kernel: sd 0:1:1:0: Attached scsi generic sg2 type 0
Aug 25 22:13:54 l01ora3 kernel: ACPI: PCI Interrupt 0000:15:00.0[A] -> GSI 24 (level, low) -> IRQ 177
Aug 25 22:13:54 l01ora3 kernel: qlge 0000:15:00.0: QLogic 10 Gigabit PCI-E Ethernet Driver 
Aug 25 22:13:54 l01ora3 kernel: qlge 0000:15:00.0: Driver name: qlge, Version: 1.00.00.25.
Aug 25 22:13:54 l01ora3 kernel: qlge 0000:15:00.0: Patch version: 2.6.16-2.6.18-p25, Release date: 100706.
Aug 25 22:13:54 l01ora3 kernel: qlge 0000:15:00.0: ql_display_dev_info: Function #0, Port #0, Rev ID = 20001010.
Aug 25 22:13:54 l01ora3 kernel: qlge 0000:15:00.0: ql_display_dev_info: MAC address 00:c0:dd:18:85:c4
Aug 25 22:13:54 l01ora3 kernel: GSI 30 sharing vector 0xE2 and IRQ 30
Aug 25 22:13:54 l01ora3 kernel: ACPI: PCI Interrupt 0000:15:00.1[B] -> GSI 34 (level, low) -> IRQ 226
Aug 25 22:13:54 l01ora3 kernel: qlge 0000:15:00.1: ql_display_dev_info: Function #1, Port #1, Rev ID = 20001010.
Aug 25 22:13:54 l01ora3 kernel: qlge 0000:15:00.1: ql_display_dev_info: MAC address 00:c0:dd:18:85:c6
Aug 25 22:13:54 l01ora3 kernel: usb0: register 'cdc_ether' at usb-0000:00:1a.0-2, CDC Ethernet Device, 5e:f3:fc:23:48:f7
Aug 25 22:13:54 l01ora3 kernel: usbcore: registered new driver cdc_ether
Aug 25 22:13:54 l01ora3 kernel: EDAC MC: Ver: 2.0.1 Jan  4 2011
Aug 25 22:13:54 l01ora3 kernel: tpm_tis 00:09: 1.2 TPM (device-id 0xFE, rev-id 70)
Aug 25 22:13:54 l01ora3 kernel: ACPI: PCI Interrupt 0000:00:1f.3[B] -> GSI 22 (level, low) -> IRQ 114
Aug 25 22:13:54 l01ora3 kernel: EDAC i7core: Driver loaded.
Aug 25 22:13:54 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV-LUN-0     Rev: 10.2
Aug 25 22:13:54 l01ora3 kernel:   Type:   RAID                               ANSI SCSI revision: 05
Aug 25 22:13:54 l01ora3 kernel: scsi 4:0:0:0: Attached scsi generic sg3 type 12
Aug 25 22:13:54 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:54 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:54 l01ora3 kernel: SCSI device sdb: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:13:54 l01ora3 kernel: sdb: Write Protect is off
Aug 25 22:13:54 l01ora3 kernel: SCSI device sdb: drive cache: write through
Aug 25 22:13:54 l01ora3 kernel: SCSI device sdb: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:13:54 l01ora3 kernel: sdb: Write Protect is off
Aug 25 22:13:55 l01ora3 kernel: SCSI device sdb: drive cache: write through
Aug 25 22:13:55 l01ora3 kernel:  sdb: sdb1
Aug 25 22:13:55 l01ora3 kernel: sd 4:0:0:1: Attached scsi disk sdb
Aug 25 22:13:55 l01ora3 kernel: sd 4:0:0:1: Attached scsi generic sg4 type 0
Aug 25 22:13:55 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:55 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:55 l01ora3 kernel: SCSI device sdc: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:13:55 l01ora3 kernel: sdc: Write Protect is off
Aug 25 22:13:55 l01ora3 kernel: SCSI device sdc: drive cache: write through
Aug 25 22:13:55 l01ora3 kernel: SCSI device sdc: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:13:55 l01ora3 kernel: sdc: Write Protect is off
Aug 25 22:13:55 l01ora3 kernel: SCSI device sdc: drive cache: write through
Aug 25 22:13:55 l01ora3 kernel:  sdc: sdc1
Aug 25 22:13:55 l01ora3 kernel: sd 4:0:0:2: Attached scsi disk sdc
Aug 25 22:13:55 l01ora3 kernel: sd 4:0:0:2: Attached scsi generic sg5 type 0
Aug 25 22:13:55 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:55 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:55 l01ora3 kernel: SCSI device sdd: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:55 l01ora3 kernel: sdd: Write Protect is off
Aug 25 22:13:55 l01ora3 kernel: SCSI device sdd: drive cache: write through
Aug 25 22:13:55 l01ora3 kernel: SCSI device sdd: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:55 l01ora3 kernel: sdd: Write Protect is off
Aug 25 22:13:55 l01ora3 kernel: SCSI device sdd: drive cache: write through
Aug 25 22:13:55 l01ora3 kernel:  sdd: sdd1
Aug 25 22:13:55 l01ora3 kernel: sd 4:0:0:3: Attached scsi disk sdd
Aug 25 22:13:55 l01ora3 kernel: sd 4:0:0:3: Attached scsi generic sg6 type 0
Aug 25 22:13:55 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:55 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:55 l01ora3 kernel: SCSI device sde: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:55 l01ora3 kernel: sde: Write Protect is off
Aug 25 22:13:55 l01ora3 kernel: SCSI device sde: drive cache: write through
Aug 25 22:13:55 l01ora3 kernel: SCSI device sde: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:55 l01ora3 kernel: sde: Write Protect is off
Aug 25 22:13:56 l01ora3 kernel: SCSI device sde: drive cache: write through
Aug 25 22:13:56 l01ora3 kernel:  sde: sde1
Aug 25 22:13:56 l01ora3 kernel: sd 4:0:0:4: Attached scsi disk sde
Aug 25 22:13:56 l01ora3 kernel: sd 4:0:0:4: Attached scsi generic sg7 type 0
Aug 25 22:13:56 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:56 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdf: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:13:56 l01ora3 kernel: sdf: Write Protect is off
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdf: drive cache: write through
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdf: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:13:56 l01ora3 kernel: sdf: Write Protect is off
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdf: drive cache: write through
Aug 25 22:13:56 l01ora3 kernel:  sdf: sdf1
Aug 25 22:13:56 l01ora3 kernel: sd 4:0:0:5: Attached scsi disk sdf
Aug 25 22:13:56 l01ora3 kernel: sd 4:0:0:5: Attached scsi generic sg8 type 0
Aug 25 22:13:56 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:56 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdg: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:13:56 l01ora3 kernel: sdg: Write Protect is off
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdg: drive cache: write through
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdg: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:13:56 l01ora3 kernel: sdg: Write Protect is off
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdg: drive cache: write through
Aug 25 22:13:56 l01ora3 kernel:  sdg: sdg1
Aug 25 22:13:56 l01ora3 kernel: sd 4:0:0:6: Attached scsi disk sdg
Aug 25 22:13:56 l01ora3 kernel: sd 4:0:0:6: Attached scsi generic sg9 type 0
Aug 25 22:13:56 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:56 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdh: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:56 l01ora3 kernel: sdh: Write Protect is off
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdh: drive cache: write through
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdh: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:56 l01ora3 kernel: sdh: Write Protect is off
Aug 25 22:13:56 l01ora3 kernel: SCSI device sdh: drive cache: write through
Aug 25 22:13:56 l01ora3 kernel:  sdh: sdh1
Aug 25 22:13:56 l01ora3 kernel: sd 4:0:0:7: Attached scsi disk sdh
Aug 25 22:13:56 l01ora3 kernel: sd 4:0:0:7: Attached scsi generic sg10 type 0
Aug 25 22:13:56 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:57 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdi: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:57 l01ora3 kernel: sdi: Write Protect is off
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdi: drive cache: write through
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdi: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:57 l01ora3 kernel: sdi: Write Protect is off
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdi: drive cache: write through
Aug 25 22:13:57 l01ora3 kernel:  sdi: sdi1
Aug 25 22:13:57 l01ora3 kernel: sd 4:0:0:8: Attached scsi disk sdi
Aug 25 22:13:57 l01ora3 kernel: sd 4:0:0:8: Attached scsi generic sg11 type 0
Aug 25 22:13:57 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:57 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdj: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:57 l01ora3 kernel: sdj: Write Protect is off
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdj: drive cache: write through
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdj: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:57 l01ora3 kernel: sdj: Write Protect is off
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdj: drive cache: write through
Aug 25 22:13:57 l01ora3 kernel:  sdj: sdj1
Aug 25 22:13:57 l01ora3 kernel: sd 4:0:0:9: Attached scsi disk sdj
Aug 25 22:13:57 l01ora3 kernel: sd 4:0:0:9: Attached scsi generic sg12 type 0
Aug 25 22:13:57 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:57 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdk: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:57 l01ora3 kernel: sdk: Write Protect is off
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdk: drive cache: write through
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdk: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:57 l01ora3 kernel: sdk: Write Protect is off
Aug 25 22:13:57 l01ora3 kernel: SCSI device sdk: drive cache: write through
Aug 25 22:13:57 l01ora3 kernel:  sdk: sdk1
Aug 25 22:13:57 l01ora3 kernel: sd 4:0:0:10: Attached scsi disk sdk
Aug 25 22:13:57 l01ora3 kernel: sd 4:0:0:10: Attached scsi generic sg13 type 0
Aug 25 22:13:57 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:58 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdl: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:58 l01ora3 kernel: sdl: Write Protect is off
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdl: drive cache: write through
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdl: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:58 l01ora3 kernel: sdl: Write Protect is off
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdl: drive cache: write through
Aug 25 22:13:58 l01ora3 kernel:  sdl: sdl1
Aug 25 22:13:58 l01ora3 kernel: sd 4:0:0:11: Attached scsi disk sdl
Aug 25 22:13:58 l01ora3 kernel: sd 4:0:0:11: Attached scsi generic sg14 type 0
Aug 25 22:13:58 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:58 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdm: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:58 l01ora3 kernel: sdm: Write Protect is off
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdm: drive cache: write through
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdm: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:58 l01ora3 kernel: sdm: Write Protect is off
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdm: drive cache: write through
Aug 25 22:13:58 l01ora3 kernel:  sdm: sdm1
Aug 25 22:13:58 l01ora3 kernel: sd 4:0:0:12: Attached scsi disk sdm
Aug 25 22:13:58 l01ora3 kernel: sd 4:0:0:12: Attached scsi generic sg15 type 0
Aug 25 22:13:58 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV-LUN-0     Rev: 10.2
Aug 25 22:13:58 l01ora3 kernel:   Type:   RAID                               ANSI SCSI revision: 05
Aug 25 22:13:58 l01ora3 kernel: scsi 4:0:1:0: Attached scsi generic sg16 type 12
Aug 25 22:13:58 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:58 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdn: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:13:58 l01ora3 kernel: sdn: Write Protect is off
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdn: drive cache: write through
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdn: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:13:58 l01ora3 kernel: sdn: Write Protect is off
Aug 25 22:13:58 l01ora3 kernel: SCSI device sdn: drive cache: write through
Aug 25 22:13:58 l01ora3 kernel:  sdn: sdn1
Aug 25 22:13:58 l01ora3 kernel: sd 4:0:1:1: Attached scsi disk sdn
Aug 25 22:13:58 l01ora3 kernel: sd 4:0:1:1: Attached scsi generic sg17 type 0
Aug 25 22:13:58 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:59 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdo: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:13:59 l01ora3 kernel: sdo: Write Protect is off
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdo: drive cache: write through
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdo: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:13:59 l01ora3 kernel: sdo: Write Protect is off
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdo: drive cache: write through
Aug 25 22:13:59 l01ora3 kernel:  sdo: sdo1
Aug 25 22:13:59 l01ora3 kernel: sd 4:0:1:2: Attached scsi disk sdo
Aug 25 22:13:59 l01ora3 kernel: sd 4:0:1:2: Attached scsi generic sg18 type 0
Aug 25 22:13:59 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:59 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdp: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:59 l01ora3 kernel: sdp: Write Protect is off
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdp: drive cache: write through
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdp: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:59 l01ora3 kernel: sdp: Write Protect is off
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdp: drive cache: write through
Aug 25 22:13:59 l01ora3 kernel:  sdp: sdp1
Aug 25 22:13:59 l01ora3 kernel: sd 4:0:1:3: Attached scsi disk sdp
Aug 25 22:13:59 l01ora3 kernel: sd 4:0:1:3: Attached scsi generic sg19 type 0
Aug 25 22:13:59 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:59 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdq: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:59 l01ora3 kernel: sdq: Write Protect is off
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdq: drive cache: write through
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdq: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:13:59 l01ora3 kernel: sdq: Write Protect is off
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdq: drive cache: write through
Aug 25 22:13:59 l01ora3 kernel:  sdq: sdq1
Aug 25 22:13:59 l01ora3 kernel: sd 4:0:1:4: Attached scsi disk sdq
Aug 25 22:13:59 l01ora3 kernel: sd 4:0:1:4: Attached scsi generic sg20 type 0
Aug 25 22:13:59 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:13:59 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:13:59 l01ora3 kernel: SCSI device sdr: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:13:59 l01ora3 kernel: sdr: Write Protect is off
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdr: drive cache: write through
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdr: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:14:00 l01ora3 kernel: sdr: Write Protect is off
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdr: drive cache: write through
Aug 25 22:14:00 l01ora3 kernel:  sdr: sdr1
Aug 25 22:14:00 l01ora3 kernel: sd 4:0:1:5: Attached scsi disk sdr
Aug 25 22:14:00 l01ora3 kernel: sd 4:0:1:5: Attached scsi generic sg21 type 0
Aug 25 22:14:00 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:00 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:00 l01ora3 kernel: SCSI device sds: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:14:00 l01ora3 kernel: sds: Write Protect is off
Aug 25 22:14:00 l01ora3 kernel: SCSI device sds: drive cache: write through
Aug 25 22:14:00 l01ora3 kernel: SCSI device sds: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:14:00 l01ora3 kernel: sds: Write Protect is off
Aug 25 22:14:00 l01ora3 kernel: SCSI device sds: drive cache: write through
Aug 25 22:14:00 l01ora3 kernel:  sds: sds1
Aug 25 22:14:00 l01ora3 kernel: sd 4:0:1:6: Attached scsi disk sds
Aug 25 22:14:00 l01ora3 kernel: sd 4:0:1:6: Attached scsi generic sg22 type 0
Aug 25 22:14:00 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:00 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdt: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:00 l01ora3 kernel: sdt: Write Protect is off
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdt: drive cache: write through
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdt: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:00 l01ora3 kernel: sdt: Write Protect is off
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdt: drive cache: write through
Aug 25 22:14:00 l01ora3 kernel:  sdt: sdt1
Aug 25 22:14:00 l01ora3 kernel: sd 4:0:1:7: Attached scsi disk sdt
Aug 25 22:14:00 l01ora3 kernel: sd 4:0:1:7: Attached scsi generic sg23 type 0
Aug 25 22:14:00 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:00 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdu: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:00 l01ora3 kernel: sdu: Write Protect is off
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdu: drive cache: write through
Aug 25 22:14:00 l01ora3 kernel: SCSI device sdu: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:00 l01ora3 kernel: sdu: Write Protect is off
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdu: drive cache: write through
Aug 25 22:14:01 l01ora3 kernel:  sdu: sdu1
Aug 25 22:14:01 l01ora3 kernel: sd 4:0:1:8: Attached scsi disk sdu
Aug 25 22:14:01 l01ora3 kernel: sd 4:0:1:8: Attached scsi generic sg24 type 0
Aug 25 22:14:01 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:01 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdv: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:01 l01ora3 kernel: sdv: Write Protect is off
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdv: drive cache: write through
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdv: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:01 l01ora3 kernel: sdv: Write Protect is off
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdv: drive cache: write through
Aug 25 22:14:01 l01ora3 kernel:  sdv: sdv1
Aug 25 22:14:01 l01ora3 kernel: sd 4:0:1:9: Attached scsi disk sdv
Aug 25 22:14:01 l01ora3 kernel: sd 4:0:1:9: Attached scsi generic sg25 type 0
Aug 25 22:14:01 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:01 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdw: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:01 l01ora3 kernel: sdw: Write Protect is off
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdw: drive cache: write through
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdw: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:01 l01ora3 kernel: sdw: Write Protect is off
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdw: drive cache: write through
Aug 25 22:14:01 l01ora3 kernel:  sdw: sdw1
Aug 25 22:14:01 l01ora3 kernel: sd 4:0:1:10: Attached scsi disk sdw
Aug 25 22:14:01 l01ora3 kernel: sd 4:0:1:10: Attached scsi generic sg26 type 0
Aug 25 22:14:01 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:01 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdx: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:01 l01ora3 kernel: sdx: Write Protect is off
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdx: drive cache: write through
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdx: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:01 l01ora3 kernel: sdx: Write Protect is off
Aug 25 22:14:01 l01ora3 kernel: SCSI device sdx: drive cache: write through
Aug 25 22:14:01 l01ora3 kernel:  sdx: sdx1
Aug 25 22:14:01 l01ora3 kernel: sd 4:0:1:11: Attached scsi disk sdx
Aug 25 22:14:02 l01ora3 kernel: sd 4:0:1:11: Attached scsi generic sg27 type 0
Aug 25 22:14:02 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:02 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdy: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:02 l01ora3 kernel: sdy: Write Protect is off
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdy: drive cache: write through
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdy: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:02 l01ora3 kernel: sdy: Write Protect is off
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdy: drive cache: write through
Aug 25 22:14:02 l01ora3 kernel:  sdy: sdy1
Aug 25 22:14:02 l01ora3 kernel: sd 4:0:1:12: Attached scsi disk sdy
Aug 25 22:14:02 l01ora3 kernel: sd 4:0:1:12: Attached scsi generic sg28 type 0
Aug 25 22:14:02 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV-LUN-0     Rev: 10.2
Aug 25 22:14:02 l01ora3 kernel:   Type:   RAID                               ANSI SCSI revision: 05
Aug 25 22:14:02 l01ora3 kernel: scsi 4:0:2:0: Attached scsi generic sg29 type 12
Aug 25 22:14:02 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:02 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdz: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:14:02 l01ora3 kernel: sdz: Write Protect is off
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdz: drive cache: write through
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdz: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:14:02 l01ora3 kernel: sdz: Write Protect is off
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdz: drive cache: write through
Aug 25 22:14:02 l01ora3 kernel:  sdz: sdz1
Aug 25 22:14:02 l01ora3 kernel: sd 4:0:2:1: Attached scsi disk sdz
Aug 25 22:14:02 l01ora3 kernel: sd 4:0:2:1: Attached scsi generic sg30 type 0
Aug 25 22:14:02 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:02 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdaa: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:14:02 l01ora3 kernel: sdaa: Write Protect is off
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdaa: drive cache: write through
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdaa: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:14:02 l01ora3 kernel: sdaa: Write Protect is off
Aug 25 22:14:02 l01ora3 kernel: SCSI device sdaa: drive cache: write through
Aug 25 22:14:02 l01ora3 kernel:  sdaa: sdaa1
Aug 25 22:14:02 l01ora3 kernel: sd 4:0:2:2: Attached scsi disk sdaa
Aug 25 22:14:02 l01ora3 kernel: sd 4:0:2:2: Attached scsi generic sg31 type 0
Aug 25 22:14:03 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:03 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdab: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:03 l01ora3 kernel: sdab: Write Protect is off
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdab: drive cache: write through
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdab: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:03 l01ora3 kernel: sdab: Write Protect is off
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdab: drive cache: write through
Aug 25 22:14:03 l01ora3 kernel:  sdab: sdab1
Aug 25 22:14:03 l01ora3 kernel: sd 4:0:2:3: Attached scsi disk sdab
Aug 25 22:14:03 l01ora3 kernel: sd 4:0:2:3: Attached scsi generic sg32 type 0
Aug 25 22:14:03 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:03 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdac: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:03 l01ora3 kernel: sdac: Write Protect is off
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdac: drive cache: write through
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdac: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:03 l01ora3 kernel: sdac: Write Protect is off
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdac: drive cache: write through
Aug 25 22:14:03 l01ora3 kernel:  sdac: sdac1
Aug 25 22:14:03 l01ora3 kernel: sd 4:0:2:4: Attached scsi disk sdac
Aug 25 22:14:03 l01ora3 kernel: sd 4:0:2:4: Attached scsi generic sg33 type 0
Aug 25 22:14:03 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:03 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdad: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:14:03 l01ora3 kernel: sdad: Write Protect is off
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdad: drive cache: write through
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdad: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:14:03 l01ora3 kernel: sdad: Write Protect is off
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdad: drive cache: write through
Aug 25 22:14:03 l01ora3 kernel:  sdad: sdad1
Aug 25 22:14:03 l01ora3 kernel: sd 4:0:2:5: Attached scsi disk sdad
Aug 25 22:14:03 l01ora3 kernel: sd 4:0:2:5: Attached scsi generic sg34 type 0
Aug 25 22:14:03 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:03 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:03 l01ora3 kernel: SCSI device sdae: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:14:03 l01ora3 kernel: sdae: Write Protect is off
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdae: drive cache: write through
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdae: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:14:04 l01ora3 kernel: sdae: Write Protect is off
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdae: drive cache: write through
Aug 25 22:14:04 l01ora3 kernel:  sdae: sdae1
Aug 25 22:14:04 l01ora3 kernel: sd 4:0:2:6: Attached scsi disk sdae
Aug 25 22:14:04 l01ora3 kernel: sd 4:0:2:6: Attached scsi generic sg35 type 0
Aug 25 22:14:04 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:04 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdaf: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:04 l01ora3 kernel: sdaf: Write Protect is off
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdaf: drive cache: write through
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdaf: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:04 l01ora3 kernel: sdaf: Write Protect is off
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdaf: drive cache: write through
Aug 25 22:14:04 l01ora3 kernel:  sdaf: sdaf1
Aug 25 22:14:04 l01ora3 kernel: sd 4:0:2:7: Attached scsi disk sdaf
Aug 25 22:14:04 l01ora3 kernel: sd 4:0:2:7: Attached scsi generic sg36 type 0
Aug 25 22:14:04 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:04 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdag: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:04 l01ora3 kernel: sdag: Write Protect is off
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdag: drive cache: write through
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdag: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:04 l01ora3 kernel: sdag: Write Protect is off
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdag: drive cache: write through
Aug 25 22:14:04 l01ora3 kernel:  sdag: sdag1
Aug 25 22:14:04 l01ora3 kernel: sd 4:0:2:8: Attached scsi disk sdag
Aug 25 22:14:04 l01ora3 kernel: sd 4:0:2:8: Attached scsi generic sg37 type 0
Aug 25 22:14:04 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:04 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdah: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:04 l01ora3 kernel: sdah: Write Protect is off
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdah: drive cache: write through
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdah: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:04 l01ora3 kernel: sdah: Write Protect is off
Aug 25 22:14:04 l01ora3 kernel: SCSI device sdah: drive cache: write through
Aug 25 22:14:05 l01ora3 kernel:  sdah: sdah1
Aug 25 22:14:05 l01ora3 kernel: sd 4:0:2:9: Attached scsi disk sdah
Aug 25 22:14:05 l01ora3 kernel: sd 4:0:2:9: Attached scsi generic sg38 type 0
Aug 25 22:14:05 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:05 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdai: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:05 l01ora3 kernel: sdai: Write Protect is off
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdai: drive cache: write through
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdai: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:05 l01ora3 kernel: sdai: Write Protect is off
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdai: drive cache: write through
Aug 25 22:14:05 l01ora3 kernel:  sdai: sdai1
Aug 25 22:14:05 l01ora3 kernel: sd 4:0:2:10: Attached scsi disk sdai
Aug 25 22:14:05 l01ora3 kernel: sd 4:0:2:10: Attached scsi generic sg39 type 0
Aug 25 22:14:05 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:05 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdaj: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:05 l01ora3 kernel: sdaj: Write Protect is off
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdaj: drive cache: write through
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdaj: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:05 l01ora3 kernel: sdaj: Write Protect is off
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdaj: drive cache: write through
Aug 25 22:14:05 l01ora3 kernel:  sdaj: sdaj1
Aug 25 22:14:05 l01ora3 kernel: sd 4:0:2:11: Attached scsi disk sdaj
Aug 25 22:14:05 l01ora3 kernel: sd 4:0:2:11: Attached scsi generic sg40 type 0
Aug 25 22:14:05 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:05 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdak: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:05 l01ora3 kernel: sdak: Write Protect is off
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdak: drive cache: write through
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdak: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:05 l01ora3 kernel: sdak: Write Protect is off
Aug 25 22:14:05 l01ora3 kernel: SCSI device sdak: drive cache: write through
Aug 25 22:14:05 l01ora3 kernel:  sdak: sdak1
Aug 25 22:14:05 l01ora3 kernel: sd 4:0:2:12: Attached scsi disk sdak
Aug 25 22:14:06 l01ora3 kernel: sd 4:0:2:12: Attached scsi generic sg41 type 0
Aug 25 22:14:06 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV-LUN-0     Rev: 10.2
Aug 25 22:14:06 l01ora3 kernel:   Type:   RAID                               ANSI SCSI revision: 05
Aug 25 22:14:06 l01ora3 kernel: scsi 4:0:3:0: Attached scsi generic sg42 type 12
Aug 25 22:14:06 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:06 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdal: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:14:06 l01ora3 kernel: sdal: Write Protect is off
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdal: drive cache: write through
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdal: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:14:06 l01ora3 kernel: sdal: Write Protect is off
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdal: drive cache: write through
Aug 25 22:14:06 l01ora3 kernel:  sdal: sdal1
Aug 25 22:14:06 l01ora3 kernel: sd 4:0:3:1: Attached scsi disk sdal
Aug 25 22:14:06 l01ora3 kernel: sd 4:0:3:1: Attached scsi generic sg43 type 0
Aug 25 22:14:06 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:06 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdam: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:14:06 l01ora3 kernel: sdam: Write Protect is off
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdam: drive cache: write through
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdam: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:14:06 l01ora3 kernel: sdam: Write Protect is off
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdam: drive cache: write through
Aug 25 22:14:06 l01ora3 kernel:  sdam: sdam1
Aug 25 22:14:06 l01ora3 kernel: sd 4:0:3:2: Attached scsi disk sdam
Aug 25 22:14:06 l01ora3 kernel: sd 4:0:3:2: Attached scsi generic sg44 type 0
Aug 25 22:14:06 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:06 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdan: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:06 l01ora3 kernel: sdan: Write Protect is off
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdan: drive cache: write through
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdan: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:06 l01ora3 kernel: sdan: Write Protect is off
Aug 25 22:14:06 l01ora3 kernel: SCSI device sdan: drive cache: write through
Aug 25 22:14:06 l01ora3 kernel:  sdan: sdan1
Aug 25 22:14:06 l01ora3 kernel: sd 4:0:3:3: Attached scsi disk sdan
Aug 25 22:14:06 l01ora3 kernel: sd 4:0:3:3: Attached scsi generic sg45 type 0
Aug 25 22:14:06 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:07 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdao: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:07 l01ora3 kernel: sdao: Write Protect is off
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdao: drive cache: write through
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdao: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:07 l01ora3 kernel: sdao: Write Protect is off
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdao: drive cache: write through
Aug 25 22:14:07 l01ora3 kernel:  sdao: sdao1
Aug 25 22:14:07 l01ora3 kernel: sd 4:0:3:4: Attached scsi disk sdao
Aug 25 22:14:07 l01ora3 kernel: sd 4:0:3:4: Attached scsi generic sg46 type 0
Aug 25 22:14:07 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:07 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdap: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:14:07 l01ora3 kernel: sdap: Write Protect is off
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdap: drive cache: write through
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdap: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:14:07 l01ora3 kernel: sdap: Write Protect is off
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdap: drive cache: write through
Aug 25 22:14:07 l01ora3 kernel:  sdap: sdap1
Aug 25 22:14:07 l01ora3 kernel: sd 4:0:3:5: Attached scsi disk sdap
Aug 25 22:14:07 l01ora3 kernel: sd 4:0:3:5: Attached scsi generic sg47 type 0
Aug 25 22:14:07 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:07 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdaq: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:14:07 l01ora3 kernel: sdaq: Write Protect is off
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdaq: drive cache: write through
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdaq: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:14:07 l01ora3 kernel: sdaq: Write Protect is off
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdaq: drive cache: write through
Aug 25 22:14:07 l01ora3 kernel:  sdaq: sdaq1
Aug 25 22:14:07 l01ora3 kernel: sd 4:0:3:6: Attached scsi disk sdaq
Aug 25 22:14:07 l01ora3 kernel: sd 4:0:3:6: Attached scsi generic sg48 type 0
Aug 25 22:14:07 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:07 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:07 l01ora3 kernel: SCSI device sdar: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:08 l01ora3 kernel: sdar: Write Protect is off
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdar: drive cache: write through
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdar: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:08 l01ora3 kernel: sdar: Write Protect is off
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdar: drive cache: write through
Aug 25 22:14:08 l01ora3 kernel:  sdar: sdar1
Aug 25 22:14:08 l01ora3 kernel: sd 4:0:3:7: Attached scsi disk sdar
Aug 25 22:14:08 l01ora3 kernel: sd 4:0:3:7: Attached scsi generic sg49 type 0
Aug 25 22:14:08 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:08 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdas: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:08 l01ora3 kernel: sdas: Write Protect is off
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdas: drive cache: write through
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdas: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:08 l01ora3 kernel: sdas: Write Protect is off
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdas: drive cache: write through
Aug 25 22:14:08 l01ora3 kernel:  sdas: sdas1
Aug 25 22:14:08 l01ora3 kernel: sd 4:0:3:8: Attached scsi disk sdas
Aug 25 22:14:08 l01ora3 kernel: sd 4:0:3:8: Attached scsi generic sg50 type 0
Aug 25 22:14:08 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:08 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdat: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:08 l01ora3 kernel: sdat: Write Protect is off
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdat: drive cache: write through
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdat: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:08 l01ora3 kernel: sdat: Write Protect is off
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdat: drive cache: write through
Aug 25 22:14:08 l01ora3 kernel:  sdat: sdat1
Aug 25 22:14:08 l01ora3 kernel: sd 4:0:3:9: Attached scsi disk sdat
Aug 25 22:14:08 l01ora3 kernel: sd 4:0:3:9: Attached scsi generic sg51 type 0
Aug 25 22:14:08 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:08 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdau: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:08 l01ora3 kernel: sdau: Write Protect is off
Aug 25 22:14:08 l01ora3 kernel: SCSI device sdau: drive cache: write through
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdau: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:09 l01ora3 kernel: sdau: Write Protect is off
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdau: drive cache: write through
Aug 25 22:14:09 l01ora3 kernel:  sdau: sdau1
Aug 25 22:14:09 l01ora3 kernel: sd 4:0:3:10: Attached scsi disk sdau
Aug 25 22:14:09 l01ora3 kernel: sd 4:0:3:10: Attached scsi generic sg52 type 0
Aug 25 22:14:09 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:09 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdav: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:09 l01ora3 kernel: sdav: Write Protect is off
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdav: drive cache: write through
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdav: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:09 l01ora3 kernel: sdav: Write Protect is off
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdav: drive cache: write through
Aug 25 22:14:09 l01ora3 kernel:  sdav: sdav1
Aug 25 22:14:09 l01ora3 kernel: sd 4:0:3:11: Attached scsi disk sdav
Aug 25 22:14:09 l01ora3 kernel: sd 4:0:3:11: Attached scsi generic sg53 type 0
Aug 25 22:14:09 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:14:09 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdaw: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:09 l01ora3 kernel: sdaw: Write Protect is off
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdaw: drive cache: write through
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdaw: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:14:09 l01ora3 kernel: sdaw: Write Protect is off
Aug 25 22:14:09 l01ora3 kernel: SCSI device sdaw: drive cache: write through
Aug 25 22:14:09 l01ora3 kernel:  sdaw: sdaw1
Aug 25 22:14:09 l01ora3 kernel: sd 4:0:3:12: Attached scsi disk sdaw
Aug 25 22:14:09 l01ora3 kernel: sd 4:0:3:12: Attached scsi generic sg54 type 0
Aug 25 22:14:09 l01ora3 kernel: floppy0: no floppy controllers found
Aug 25 22:14:09 l01ora3 kernel: work still pending
Aug 25 22:14:09 l01ora3 kernel: lp: driver loaded but no devices found
Aug 25 22:14:09 l01ora3 kernel: mtrr: no more MTRRs available
Aug 25 22:14:09 l01ora3 last message repeated 7 times
Aug 25 22:14:09 l01ora3 kernel: ACPI: Power Button (FF) [PWRF]
Aug 25 22:14:09 l01ora3 kernel: ACPI: Mapper loaded
Aug 25 22:14:10 l01ora3 kernel: dell-wmi: No known WMI GUID found
Aug 25 22:14:10 l01ora3 kernel: md: Autodetecting RAID arrays.
Aug 25 22:14:10 l01ora3 kernel: md: autorun ...
Aug 25 22:14:10 l01ora3 kernel: md: ... autorun DONE.
Aug 25 22:14:10 l01ora3 kernel: EXT3 FS on dm-0, internal journal
Aug 25 22:14:10 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:10 l01ora3 kernel: EXT3 FS on sda1, internal journal
Aug 25 22:14:10 l01ora3 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Aug 25 22:14:10 l01ora3 kernel: Adding 16777208k swap on /dev/VolGroup00/LogVol01.  Priority:-1 extents:1 across:16777208k
Aug 25 22:14:10 l01ora3 kernel: IA-32 Microcode Update Driver: v1.14a <tigran <at> veritas.com>
Aug 25 22:14:10 l01ora3 kernel: Loading iSCSI transport class v2.0-871.
Aug 25 22:14:10 l01ora3 kernel: cxgb3i: tag itt 0x1fff, 13 bits, age 0xf, 4 bits.
Aug 25 22:14:10 l01ora3 kernel: iscsi: registered transport (cxgb3i)
Aug 25 22:14:10 l01ora3 kernel: IPv6: Loaded, but administratively disabled, reboot required to enable
Aug 25 22:14:10 l01ora3 kernel: Broadcom NetXtreme II CNIC Driver cnic v2.1.2 (May 26, 2010)
Aug 25 22:14:10 l01ora3 kernel: cnic: Added CNIC device: eth0
Aug 25 22:14:10 l01ora3 kernel: cnic: Added CNIC device: eth1
Aug 25 22:14:10 l01ora3 kernel: device-mapper: multipath: version 1.0.5 loaded
Aug 25 22:14:10 l01ora3 kernel: device-mapper: multipath round-robin: version 1.0.0 loaded
Aug 25 22:14:10 l01ora3 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.3 (Aug 10, 2010)
Aug 25 22:14:10 l01ora3 kernel: iscsi: registered transport (bnx2i)
Aug 25 22:14:10 l01ora3 kernel: scsi5 : Broadcom Offload iSCSI Initiator
Aug 25 22:14:10 l01ora3 kernel: scsi6 : Broadcom Offload iSCSI Initiator
Aug 25 22:14:10 l01ora3 kernel: iscsi: registered transport (tcp)
Aug 25 22:14:10 l01ora3 kernel: iscsi: registered transport (iser)
Aug 25 22:14:10 l01ora3 kernel: iscsi: registered transport (be2iscsi)
Aug 25 22:14:10 l01ora3 kernel: bnx2: eth0: using MSIX
Aug 25 22:14:10 l01ora3 kernel: bnx2i: iSCSI not supported, dev=eth0
Aug 25 22:14:10 l01ora3 kernel: bnx2: eth0 NIC SerDes Link is Up, 1000 Mbps full duplex
Aug 25 22:14:10 l01ora3 kernel: bnx2i: iSCSI not supported, dev=eth0
Aug 25 22:14:10 l01ora3 kernel: qlge 0000:15:00.0: ql_init_fw_done: Firmware Revision  = 0x00012800.
Aug 25 22:14:10 l01ora3 kernel: qlge 0000:15:00.0: ql_enable_msix: MSI-X Enabled, got 8 vectors.
Aug 25 22:14:10 l01ora3 kernel: qlge 0000:15:00.0: ql_link_on: eth2: Link is Up.
Aug 25 22:14:10 l01ora3 kernel: OCFS2 Node Manager 1.4.8 Thu Jan  6 10:55:34 EST 2011 (build e8aac2f2bdee501fb1932596c706cca4)
Aug 25 22:14:10 l01ora3 kernel: OCFS2 DLM 1.4.8 Thu Jan  6 10:55:34 EST 2011 (build 160daa35825b01991fd7b803a9e47661)
Aug 25 22:14:10 l01ora3 kernel: OCFS2 DLMFS 1.4.8 Thu Jan  6 10:55:34 EST 2011 (build 160daa35825b01991fd7b803a9e47661)
Aug 25 22:14:10 l01ora3 kernel: OCFS2 User DLM kernel interface loaded
Aug 25 22:14:10 l01ora3 kernel: Bluetooth: Core ver 2.10
Aug 25 22:14:10 l01ora3 kernel: NET: Registered protocol family 31
Aug 25 22:14:10 l01ora3 kernel: Bluetooth: HCI device and connection manager initialized
Aug 25 22:14:10 l01ora3 kernel: Bluetooth: HCI socket layer initialized
Aug 25 22:14:10 l01ora3 kernel: Bluetooth: L2CAP ver 2.8
Aug 25 22:14:11 l01ora3 kernel: Bluetooth: L2CAP socket layer initialized
Aug 25 22:14:11 l01ora3 kernel: Bluetooth: RFCOMM socket layer initialized
Aug 25 22:14:11 l01ora3 kernel: Bluetooth: RFCOMM TTY layer initialized
Aug 25 22:14:11 l01ora3 kernel: Bluetooth: RFCOMM ver 1.8
Aug 25 22:14:11 l01ora3 kernel: OCFS2 1.4.8 Thu Jan  6 10:55:30 EST 2011 (build 27d6d0b45170348d3948369890be76c7)
Aug 25 22:14:11 l01ora3 kernel: o2dlm: Nodes in domain AC09C750FF9B4898BAF413F48E669B06: 0 
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12192,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12192,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:11 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:11 l01ora3 kernel: ocfs2: Mounting device (253,20) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:11 l01ora3 kernel: (ocfs2rec,12220,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,20)
Aug 25 22:14:11 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:11 l01ora3 kernel: o2dlm: Nodes in domain 46B7D2E34F044DA0B8E326113BFB1370: 0 
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12228,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12228,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:11 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:11 l01ora3 kernel: ocfs2: Mounting device (253,23) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:11 l01ora3 kernel: (ocfs2rec,12237,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,23)
Aug 25 22:14:11 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:11 l01ora3 kernel: o2dlm: Nodes in domain BEA8108F34864DADBDED1BCE3311B1BA: 0 
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12245,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12245,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:11 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:11 l01ora3 kernel: ocfs2: Mounting device (253,21) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:11 l01ora3 kernel: (ocfs2rec,12254,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,21)
Aug 25 22:14:11 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:11 l01ora3 kernel: o2dlm: Nodes in domain 98CFB9F661C6493AB748882C71B3E577: 0 
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12262,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12262,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:11 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:11 l01ora3 kernel: ocfs2: Mounting device (253,25) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:11 l01ora3 kernel: (ocfs2rec,12271,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,25)
Aug 25 22:14:11 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:11 l01ora3 kernel: o2dlm: Nodes in domain DD148B8FB5AA49E2AC7C74EF9631A2DF: 0 
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12279,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:11 l01ora3 kernel: (mount.ocfs2,12279,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:12 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:12 l01ora3 kernel: ocfs2: Mounting device (253,16) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:12 l01ora3 kernel: (ocfs2rec,12288,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,16)
Aug 25 22:14:12 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:12 l01ora3 kernel: o2dlm: Nodes in domain 4ACE425288044AC2A81C430EEE76DCB8: 0 
Aug 25 22:14:12 l01ora3 kernel: (mount.ocfs2,12296,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:12 l01ora3 kernel: (mount.ocfs2,12296,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:12 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:12 l01ora3 kernel: ocfs2: Mounting device (253,22) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:12 l01ora3 kernel: (ocfs2rec,12305,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,22)
Aug 25 22:14:12 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:12 l01ora3 kernel: o2dlm: Nodes in domain C41746218FA74548B4DB170071B518BA: 0 
Aug 25 22:14:12 l01ora3 kernel: (mount.ocfs2,12313,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:12 l01ora3 kernel: (mount.ocfs2,12313,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:12 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:12 l01ora3 kernel: ocfs2: Mounting device (253,18) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:12 l01ora3 kernel: (ocfs2rec,12322,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,18)
Aug 25 22:14:12 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:14 l01ora3 kernel: o2dlm: Nodes in domain ACF122DDA58A49D58F355EF220EB51AF: 0 
Aug 25 22:14:14 l01ora3 kernel: (mount.ocfs2,12330,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:14 l01ora3 kernel: (mount.ocfs2,12330,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:14 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:14 l01ora3 kernel: ocfs2: Mounting device (253,19) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:14 l01ora3 kernel: (ocfs2rec,12342,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,19)
Aug 25 22:14:15 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:18 l01ora3 kernel: o2dlm: Nodes in domain 7F814AB666A74F25BCB65491D59BD579: 0 
Aug 25 22:14:18 l01ora3 kernel: (mount.ocfs2,12350,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:18 l01ora3 kernel: (mount.ocfs2,12350,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:18 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:18 l01ora3 kernel: ocfs2: Mounting device (253,14) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:18 l01ora3 kernel: (ocfs2rec,12359,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,14)
Aug 25 22:14:21 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:23 l01ora3 kernel: o2dlm: Nodes in domain FA2E1CE55CCD48129448D2267299604C: 0 
Aug 25 22:14:23 l01ora3 kernel: (mount.ocfs2,12367,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:23 l01ora3 kernel: (mount.ocfs2,12367,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:23 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:23 l01ora3 kernel: ocfs2: Mounting device (253,24) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:23 l01ora3 kernel: (ocfs2rec,12376,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,24)
Aug 25 22:14:26 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:27 l01ora3 kernel: o2dlm: Nodes in domain 86054DCFCAF4486AB77BB3FD9E1BCF90: 0 
Aug 25 22:14:27 l01ora3 kernel: (mount.ocfs2,12384,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:27 l01ora3 kernel: (mount.ocfs2,12384,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:27 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:27 l01ora3 kernel: ocfs2: Mounting device (253,17) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:27 l01ora3 kernel: (ocfs2rec,12393,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,17)
Aug 25 22:14:29 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:31 l01ora3 kernel: o2dlm: Nodes in domain 4E379E75DD4B468DA7F55B0CF6359FED: 0 
Aug 25 22:14:31 l01ora3 kernel: (mount.ocfs2,12401,0):ocfs2_find_slot:249 slot 0 is already allocated to this node!
Aug 25 22:14:31 l01ora3 kernel: (mount.ocfs2,12401,0):ocfs2_check_volume:1944 File system was not unmounted cleanly, recovering volume.
Aug 25 22:14:31 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:31 l01ora3 kernel: ocfs2: Mounting device (253,15) on (node 0, slot 0) with ordered data mode.
Aug 25 22:14:31 l01ora3 kernel: (ocfs2rec,12410,0):ocfs2_replay_journal:1183 Recovering node 1 from slot 1 on device (253,15)
Aug 25 22:14:31 l01ora3 pcscd: pcscdaemon.c:507:main() pcsc-lite 1.4.4 daemon ready.
Aug 25 22:14:32 l01ora3 pcscd: hotplug_libusb.c:402:HPEstablishUSBNotifications() Driver ifd-egate.bundle does not support IFD_GENERATE_HOTPLUG. Using active polling instead.
Aug 25 22:14:32 l01ora3 pcscd: hotplug_libusb.c:411:HPEstablishUSBNotifications() Polling forced every 1 second(s)
Aug 25 22:14:34 l01ora3 kernel: kjournald starting.  Commit interval 5 seconds
Aug 25 22:14:49 l01ora3 kernel: Bluetooth: HIDP (Human Interface Emulation) ver 1.1
Aug 25 22:14:49 l01ora3 hidd[12605]: Bluetooth HID daemon
Aug 25 22:14:51 l01ora3 automount[12653]: lookup_read_master: lookup(nisplus): couldn't locate nis+ table auto.master
Aug 25 22:14:51 l01ora3 hpiod: 1.6.7 accepting connections at 2208... 
Aug 25 22:14:52 l01ora3 ntpd[12726]: ntpd 4.2.2p1 <at> 1.1570-o Tue Dec  8 20:30:45 UTC 2009 (1)
Aug 25 22:14:52 l01ora3 ntpd[12727]: precision = 1.000 usec
Aug 25 22:14:52 l01ora3 ntpd[12727]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Aug 25 22:14:52 l01ora3 ntpd[12727]: Listening on interface lo, 127.0.0.1#123 Enabled
Aug 25 22:14:52 l01ora3 ntpd[12727]: Listening on interface eth0, 192.168.0.103#123 Enabled
Aug 25 22:14:52 l01ora3 ntpd[12727]: Listening on interface eth2, 10.2.10.143#123 Enabled
Aug 25 22:14:52 l01ora3 ntpd[12727]: Listening on interface usb0, 169.254.95.120#123 Enabled
Aug 25 22:14:52 l01ora3 ntpd[12727]: kernel time sync status 0040
Aug 25 22:14:52 l01ora3 ntpd[12727]: getaddrinfo: "::1" invalid host address, ignored
Aug 25 22:14:53 l01ora3 ntpd[12727]: frequency initialized -10.533 PPM from /var/lib/ntp/drift
Aug 25 22:14:53 l01ora3 gpm[12771]: *** info [startup.c(95)]: 
Aug 25 22:14:53 l01ora3 gpm[12771]: Started gpm successfully. Entered daemon mode.
Aug 25 22:14:53 l01ora3 xinetd[12715]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in.
Aug 25 22:14:53 l01ora3 xinetd[12715]: Started working: 0 available services
Aug 25 22:14:54 l01ora3 logger: Oracle Cluster Ready Services starting up automatically.
Aug 25 22:14:54 l01ora3 avahi-daemon[13019]: Found user 'avahi' (UID 70) and group 'avahi' (GID 70).
Aug 25 22:14:54 l01ora3 avahi-daemon[13019]: Successfully dropped root privileges.
Aug 25 22:14:54 l01ora3 avahi-daemon[13019]: avahi-daemon 0.6.16 starting up.
Aug 25 22:14:54 l01ora3 avahi-daemon[13019]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
Aug 25 22:14:54 l01ora3 avahi-daemon[13019]: Successfully called chroot().
Aug 25 22:14:54 l01ora3 avahi-daemon[13019]: Successfully dropped remaining capabilities.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: No service found in /etc/avahi/services.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: socket() failed: Address family not supported by protocol
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: Failed to create IPv6 socket, proceeding in IPv4 only mode
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: socket() failed: Address family not supported by protocol
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: New relevant interface usb0.IPv4 for mDNS.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: Joining mDNS multicast group on interface usb0.IPv4 with address 169.254.95.120.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: New relevant interface eth2.IPv4 for mDNS.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: Joining mDNS multicast group on interface eth2.IPv4 with address 10.2.10.143.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: New relevant interface eth0.IPv4 for mDNS.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.0.103.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: Network interface enumeration completed.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: Registering new address record for 169.254.95.120 on usb0.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: Registering new address record for 10.2.10.143 on eth2.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: Registering new address record for 192.168.0.103 on eth0.
Aug 25 22:14:55 l01ora3 avahi-daemon[13019]: Registering HINFO record with values 'X86_64'/'LINUX'.
Aug 25 22:14:56 l01ora3 avahi-daemon[13019]: Server startup complete. Host name is l01ora3.local. Local service cookie is 1313219373.
Aug 25 22:16:11 l01ora3 kernel: Hangcheck: starting hangcheck timer 0.9.0 (tick is 1 seconds, margin is 10 seconds).
Aug 25 22:16:11 l01ora3 kernel: Hangcheck: Using monotonic_clock().
Aug 25 22:16:11 l01ora3 smartd[13789]: smartd version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen 
Aug 25 22:16:11 l01ora3 smartd[13789]: Home page is http://smartmontools.sourceforge.net/  
Aug 25 22:16:11 l01ora3 smartd[13789]: Opened configuration file /etc/smartd.conf 
Aug 25 22:16:11 l01ora3 smartd[13789]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices 
Aug 25 22:16:11 l01ora3 smartd[13789]: Problem creating device name scan list 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sda, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sda, Bad IEC (SMART) mode page, err=4, skip device 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdb, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdb, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdb' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdc, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdc, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdc' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdd, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdd, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdd' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sde, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sde, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sde' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdf, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdf, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdf' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdg, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdg, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdg' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdh, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdh, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdh' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdi, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdi, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdi' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdj, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdj, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdj' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdk, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdk, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdk' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdl, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdl, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdl' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdm, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdm, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdm' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdn, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdn, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdn' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdo, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdo, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdo' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdp, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdp, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdp' to turn on SMART features 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdq, opened 
Aug 25 22:16:11 l01ora3 smartd[13789]: Device: /dev/sdq, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdq' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdr, opened 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdr, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdr' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sds, opened 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sds, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sds' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdt, opened 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdt, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdt' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdu, opened 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdu, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdu' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdv, opened 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdv, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdv' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdw, opened 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdw, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdw' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdx, opened 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdx, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdx' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdy, opened 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdy, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdy' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdz, opened 
Aug 25 22:16:12 l01ora3 smartd[13789]: Device: /dev/sdz, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdz' to turn on SMART features 
Aug 25 22:16:12 l01ora3 smartd[13789]: Monitoring 0 ATA and 0 SCSI devices 
Aug 25 22:16:12 l01ora3 smartd[13791]: smartd has fork()ed into background mode. New PID=13791. 
Aug 25 22:16:14 l01ora3 kernel: mtrr: no more MTRRs available
Aug 25 22:16:14 l01ora3 last message repeated 3 times
Aug 25 22:16:14 l01ora3 pcscd: winscard.c:304:SCardConnect() Reader E-Gate 0 0 Not Found
Aug 25 22:16:14 l01ora3 last message repeated 3 times
Aug 25 22:16:14 l01ora3 logger: Cluster Ready Services completed waiting on dependencies.
Aug 25 22:16:14 l01ora3 last message repeated 2 times
Aug 25 22:16:14 l01ora3 logger: Oracle CSS Family monitor starting.
Aug 25 22:16:15 l01ora3 logger: Running CRSD with TZ = 
Aug 25 22:16:15 l01ora3 kernel: mtrr: no more MTRRs available
Aug 25 22:16:15 l01ora3 last message repeated 3 times
Aug 25 22:16:16 l01ora3 logger: Oracle CSS restart. 0, 1
Aug 25 22:16:17 l01ora3 logger: Oracle CSSD waiting for OPROCD to start
Aug 25 22:18:01 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:18:01 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:18:01 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 259 seconds.
Aug 25 22:18:07 l01ora3 ntpd[12727]: synchronized to LOCAL(0), stratum 10
Aug 25 22:18:07 l01ora3 ntpd[12727]: kernel time sync enabled 0001
Aug 25 22:18:40 l01ora3 kernel: qla2xxx 0000:24:00.0: LOOP DOWN detected (4 0 0).
Aug 25 22:19:09 l01ora3 ntpd[12727]: synchronized to 161.53.131.232, stratum 2
Aug 25 22:19:36 l01ora3 kernel: qla2xxx 0000:24:00.0: LOOP UP detected (8 Gbps).
Aug 25 22:19:58 l01ora3 avahi-daemon[13019]: Registering new address record for 10.2.10.113 on usb0.
Aug 25 22:19:58 l01ora3 avahi-daemon[13019]: Withdrawing address record for 10.2.10.113 on usb0.
Aug 25 22:19:58 l01ora3 avahi-daemon[13019]: Registering new address record for 10.2.10.113 on usb0.
Aug 25 22:20:08 l01ora3 avahi-daemon[13019]: Registering new address record for 10.2.10.114 on usb0.
Aug 25 22:20:08 l01ora3 avahi-daemon[13019]: Withdrawing address record for 10.2.10.114 on usb0.
Aug 25 22:20:08 l01ora3 avahi-daemon[13019]: Registering new address record for 10.2.10.114 on usb0.
Aug 25 22:20:29 l01ora3 avahi-daemon[13019]: Registering new address record for 10.2.10.115 on eth2.
Aug 25 22:20:29 l01ora3 avahi-daemon[13019]: Withdrawing address record for 10.2.10.115 on eth2.
Aug 25 22:20:29 l01ora3 avahi-daemon[13019]: Registering new address record for 10.2.10.115 on eth2.
Aug 25 22:20:30 l01ora3 avahi-daemon[13019]: Registering new address record for 10.2.10.117 on eth2.
Aug 25 22:20:30 l01ora3 avahi-daemon[13019]: Withdrawing address record for 10.2.10.117 on eth2.
Aug 25 22:20:30 l01ora3 avahi-daemon[13019]: Registering new address record for 10.2.10.117 on eth2.
Aug 25 22:22:19 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:22:20 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:22:20 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 243 seconds.
Aug 25 22:26:23 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:26:23 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:26:23 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 255 seconds.
Aug 25 22:30:37 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:30:38 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:30:38 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 289 seconds.
Aug 25 22:35:27 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:35:27 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:35:27 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 236 seconds.
Aug 25 22:37:22 l01ora3 kernel: qla2xxx 0000:24:00.0: LOOP DOWN detected (2 0 0).
Aug 25 22:37:44 l01ora3 kernel: qla2xxx 0000:24:00.0: LOOP UP detected (8 Gbps).
Aug 25 22:38:35 l01ora3 ntpd[12727]: time reset +0.485676 s
Aug 25 22:39:23 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:39:23 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:39:23 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 287 seconds.
Aug 25 22:42:39 l01ora3 ntpd[12727]: synchronized to LOCAL(0), stratum 10
Aug 25 22:43:21 l01ora3 ntpd[12727]: synchronized to 66.96.98.9, stratum 2
Aug 25 22:44:09 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:44:10 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:44:10 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 286 seconds.
Aug 25 22:46:16 l01ora3 ntpd[12727]: synchronized to 161.53.131.232, stratum 2
Aug 25 22:48:55 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:48:56 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:48:56 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 278 seconds.
Aug 25 22:49:18 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV-LUN-0     Rev: 10.2
Aug 25 22:49:18 l01ora3 kernel:   Type:   RAID                               ANSI SCSI revision: 05
Aug 25 22:49:18 l01ora3 kernel: scsi 3:0:0:0: Attached scsi generic sg55 type 12
Aug 25 22:49:18 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:18 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:18 l01ora3 kernel: SCSI device sdax: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:49:18 l01ora3 kernel: sdax: Write Protect is off
Aug 25 22:49:18 l01ora3 kernel: SCSI device sdax: drive cache: write through
Aug 25 22:49:18 l01ora3 kernel: SCSI device sdax: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:49:18 l01ora3 kernel: sdax: Write Protect is off
Aug 25 22:49:18 l01ora3 kernel: SCSI device sdax: drive cache: write through
Aug 25 22:49:18 l01ora3 kernel:  sdax: sdax1
Aug 25 22:49:18 l01ora3 kernel: sd 3:0:0:1: Attached scsi disk sdax
Aug 25 22:49:18 l01ora3 kernel: sd 3:0:0:1: Attached scsi generic sg56 type 0
Aug 25 22:49:18 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:18 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:18 l01ora3 kernel: SCSI device sday: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:49:18 l01ora3 kernel: sday: Write Protect is off
Aug 25 22:49:18 l01ora3 kernel: SCSI device sday: drive cache: write through
Aug 25 22:49:18 l01ora3 kernel: SCSI device sday: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:49:18 l01ora3 kernel: sday: Write Protect is off
Aug 25 22:49:18 l01ora3 kernel: SCSI device sday: drive cache: write through
Aug 25 22:49:18 l01ora3 kernel:  sday: sday1
Aug 25 22:49:18 l01ora3 kernel: sd 3:0:0:2: Attached scsi disk sday
Aug 25 22:49:18 l01ora3 kernel: sd 3:0:0:2: Attached scsi generic sg57 type 0
Aug 25 22:49:18 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:18 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:18 l01ora3 kernel: SCSI device sdaz: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:18 l01ora3 kernel: sdaz: Write Protect is off
Aug 25 22:49:18 l01ora3 kernel: SCSI device sdaz: drive cache: write through
Aug 25 22:49:18 l01ora3 kernel: SCSI device sdaz: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:18 l01ora3 kernel: sdaz: Write Protect is off
Aug 25 22:49:18 l01ora3 kernel: SCSI device sdaz: drive cache: write through
Aug 25 22:49:19 l01ora3 kernel:  sdaz: sdaz1
Aug 25 22:49:19 l01ora3 multipathd: sdbe: add path (uevent) 
Aug 25 22:49:19 l01ora3 kernel: sd 3:0:0:3: Attached scsi disk sdaz
Aug 25 22:49:19 l01ora3 multipathd: mpathg: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:192 15 66:0 15 65:64 15 8:128 15 67:128 15] 
Aug 25 22:49:19 l01ora3 kernel: sd 3:0:0:3: Attached scsi generic sg58 type 0
Aug 25 22:49:19 l01ora3 multipathd: sdbe path added to devmap mpathg 
Aug 25 22:49:19 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:19 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:19 l01ora3 multipathd: sdaz: add path (uevent) 
Aug 25 22:49:19 l01ora3 kernel: SCSI device sdba: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:19 l01ora3 multipathd: mpathc: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:112 15 65:176 15 8:240 15 8:48 15 67:48 15] 
Aug 25 22:49:19 l01ora3 kernel: sdba: Write Protect is off
Aug 25 22:49:19 l01ora3 multipathd: sdaz path added to devmap mpathc 
Aug 25 22:49:19 l01ora3 multipathd: sdax: add path (uevent) 
Aug 25 22:49:19 l01ora3 kernel: SCSI device sdba: drive cache: write through
Aug 25 22:49:19 l01ora3 multipathd: mpathe: load table [0 2013265920 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:80 15 65:144 15 8:208 15 8:16 15 67:16 15] 
Aug 25 22:49:19 l01ora3 kernel: SCSI device sdba: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:19 l01ora3 multipathd: sdax path added to devmap mpathe 
Aug 25 22:49:19 l01ora3 kernel: sdba: Write Protect is off
Aug 25 22:49:19 l01ora3 multipathd: sdbf: add path (uevent) 
Aug 25 22:49:19 l01ora3 multipathd: mpathi: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:208 15 66:16 15 65:80 15 8:144 15 67:144 15] 
Aug 25 22:49:20 l01ora3 kernel: SCSI device sdba: drive cache: write through
Aug 25 22:49:20 l01ora3 multipathd: sdbf path added to devmap mpathi 
Aug 25 22:49:20 l01ora3 kernel:  sdba: sdba1
Aug 25 22:49:20 l01ora3 multipathd: sdbj: add path (uevent) 
Aug 25 22:49:20 l01ora3 kernel: sd 3:0:0:4: Attached scsi disk sdba
Aug 25 22:49:20 l01ora3 multipathd: mpathe: load table [0 2013265920 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:80 15 65:144 15 8:208 15 8:16 15 67:16 15 67:208 15] 
Aug 25 22:49:20 l01ora3 kernel: sd 3:0:0:4: Attached scsi generic sg59 type 0
Aug 25 22:49:20 l01ora3 multipathd: sdbj path added to devmap mpathe 
Aug 25 22:49:20 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:20 l01ora3 multipathd: sdbp: add path (uevent) 
Aug 25 22:49:20 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:20 l01ora3 multipathd: mpathh: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:176 15 65:240 15 65:48 15 8:112 15 68:48 15] 
Aug 25 22:49:20 l01ora3 kernel: SCSI device sdbb: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:49:20 l01ora3 multipathd: sdbp path added to devmap mpathh 
Aug 25 22:49:20 l01ora3 kernel: sdbb: Write Protect is off
Aug 25 22:49:20 l01ora3 multipathd: sdbo: add path (uevent) 
Aug 25 22:49:20 l01ora3 multipathd: mpathf: load table [0 1006632960 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:160 15 65:224 15 65:32 15 8:96 15 68:32 15] 
Aug 25 22:49:20 l01ora3 kernel: SCSI device sdbb: drive cache: write through
Aug 25 22:49:20 l01ora3 multipathd: sdbo path added to devmap mpathf 
Aug 25 22:49:20 l01ora3 kernel: SCSI device sdbb: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:49:20 l01ora3 multipathd: dm-13: add map (uevent) 
Aug 25 22:49:20 l01ora3 kernel: sdbb: Write Protect is off
Aug 25 22:49:20 l01ora3 multipathd: dm-13: devmap already registered 
Aug 25 22:49:20 l01ora3 multipathd: sdbc: add path (uevent) 
Aug 25 22:49:20 l01ora3 kernel: SCSI device sdbb: drive cache: write through
Aug 25 22:49:20 l01ora3 multipathd: mpathf: load table [0 1006632960 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:160 15 65:224 15 65:32 15 8:96 15 68:32 15 67:96 15] 
Aug 25 22:49:20 l01ora3 multipathd: sdbc path added to devmap mpathf 
Aug 25 22:49:20 l01ora3 multipathd: sdbi: add path (uevent) 
Aug 25 22:49:20 l01ora3 kernel:  sdbb: sdbb1
Aug 25 22:49:20 l01ora3 multipathd: mpathk: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 67:0 15 66:64 15 65:128 15 8:192 15 67:192 15] 
Aug 25 22:49:20 l01ora3 kernel: sd 3:0:0:5: Attached scsi disk sdbb
Aug 25 22:49:20 l01ora3 multipathd: sdbi path added to devmap mpathk 
Aug 25 22:49:20 l01ora3 kernel: sd 3:0:0:5: Attached scsi generic sg60 type 0
Aug 25 22:49:20 l01ora3 multipathd: sdbd: add path (uevent) 
Aug 25 22:49:20 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:20 l01ora3 multipathd: mpathh: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:176 15 65:240 15 65:48 15 8:112 15 68:48 15 67:112 15] 
Aug 25 22:49:20 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:21 l01ora3 multipathd: sdbd path added to devmap mpathh 
Aug 25 22:49:21 l01ora3 kernel: SCSI device sdbc: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:49:21 l01ora3 multipathd: sday: add path (uevent) 
Aug 25 22:49:21 l01ora3 kernel: sdbc: Write Protect is off
Aug 25 22:49:21 l01ora3 multipathd: mpathd: load table [0 4026531840 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:96 15 65:160 15 8:224 15 8:32 15 67:32 15] 
Aug 25 22:49:21 l01ora3 multipathd: sday path added to devmap mpathd 
Aug 25 22:49:21 l01ora3 kernel: SCSI device sdbc: drive cache: write through
Aug 25 22:49:21 l01ora3 multipathd: sdbl: add path (uevent) 
Aug 25 22:49:21 l01ora3 kernel: SCSI device sdbc: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:49:21 l01ora3 multipathd: mpathc: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:112 15 65:176 15 8:240 15 8:48 15 67:48 15 67:240 15] 
Aug 25 22:49:21 l01ora3 kernel: sdbc: Write Protect is off
Aug 25 22:49:21 l01ora3 multipathd: sdbl path added to devmap mpathc 
Aug 25 22:49:21 l01ora3 multipathd: sdbb: add path (uevent) 
Aug 25 22:49:21 l01ora3 kernel: SCSI device sdbc: drive cache: write through
Aug 25 22:49:21 l01ora3 multipathd: mpatha: load table [0 201326592 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:144 15 65:208 15 65:16 15 8:80 15 67:80 15] 
Aug 25 22:49:21 l01ora3 kernel:  sdbc: sdbc1
Aug 25 22:49:21 l01ora3 multipathd: sdbb path added to devmap mpatha 
Aug 25 22:49:21 l01ora3 kernel: sd 3:0:0:6: Attached scsi disk sdbc
Aug 25 22:49:21 l01ora3 multipathd: dm-6: add map (uevent) 
Aug 25 22:49:21 l01ora3 kernel: sd 3:0:0:6: Attached scsi generic sg61 type 0
Aug 25 22:49:21 l01ora3 multipathd: dm-6: devmap already registered 
Aug 25 22:49:21 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:21 l01ora3 multipathd: dm-7: add map (uevent) 
Aug 25 22:49:21 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:21 l01ora3 multipathd: dm-7: devmap already registered 
Aug 25 22:49:21 l01ora3 kernel: SCSI device sdbd: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:21 l01ora3 multipathd: dm-9: add map (uevent) 
Aug 25 22:49:21 l01ora3 kernel: sdbd: Write Protect is off
Aug 25 22:49:21 l01ora3 multipathd: dm-9: devmap already registered 
Aug 25 22:49:21 l01ora3 multipathd: dm-5: add map (uevent) 
Aug 25 22:49:21 l01ora3 kernel: SCSI device sdbd: drive cache: write through
Aug 25 22:49:21 l01ora3 multipathd: dm-5: devmap already registered 
Aug 25 22:49:21 l01ora3 kernel: SCSI device sdbd: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:21 l01ora3 multipathd: dm-2: add map (uevent) 
Aug 25 22:49:21 l01ora3 kernel: sdbd: Write Protect is off
Aug 25 22:49:21 l01ora3 multipathd: dm-2: devmap already registered 
Aug 25 22:49:22 l01ora3 multipathd: sdcq: add path (uevent) 
Aug 25 22:49:22 l01ora3 kernel: SCSI device sdbd: drive cache: write through
Aug 25 22:49:22 l01ora3 multipathd: mpathj: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:224 15 66:32 15 65:96 15 8:160 15 69:224 15] 
Aug 25 22:49:22 l01ora3 kernel:  sdbd: sdbd1
Aug 25 22:49:22 l01ora3 multipathd: sdcq path added to devmap mpathj 
Aug 25 22:49:22 l01ora3 kernel: sd 3:0:0:7: Attached scsi disk sdbd
Aug 25 22:49:22 l01ora3 multipathd: sdbw: add path (uevent) 
Aug 25 22:49:22 l01ora3 kernel: sd 3:0:0:7: Attached scsi generic sg62 type 0
Aug 25 22:49:22 l01ora3 multipathd: mpathd: load table [0 4026531840 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:96 15 65:160 15 8:224 15 8:32 15 67:32 15 68:160 15] 
Aug 25 22:49:22 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:22 l01ora3 multipathd: sdbw path added to devmap mpathd 
Aug 25 22:49:22 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:22 l01ora3 kernel: SCSI device sdbe: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:22 l01ora3 multipathd: dm-4: add map (uevent) 
Aug 25 22:49:22 l01ora3 kernel: sdbe: Write Protect is off
Aug 25 22:49:22 l01ora3 multipathd: dm-4: devmap already registered 
Aug 25 22:49:22 l01ora3 multipathd: sdba: add path (uevent) 
Aug 25 22:49:22 l01ora3 kernel: SCSI device sdbe: drive cache: write through
Aug 25 22:49:22 l01ora3 multipathd: mpathb: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:128 15 65:192 15 65:0 15 8:64 15 67:64 15] 
Aug 25 22:49:22 l01ora3 kernel: SCSI device sdbe: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:22 l01ora3 multipathd: sdba path added to devmap mpathb 
Aug 25 22:49:22 l01ora3 kernel: sdbe: Write Protect is off
Aug 25 22:49:22 l01ora3 multipathd: sdbg: add path (uevent) 
Aug 25 22:49:22 l01ora3 multipathd: mpathj: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:224 15 66:32 15 65:96 15 8:160 15 69:224 15 67:160 15] 
Aug 25 22:49:22 l01ora3 kernel: SCSI device sdbe: drive cache: write through
Aug 25 22:49:22 l01ora3 multipathd: sdbg path added to devmap mpathj 
Aug 25 22:49:22 l01ora3 kernel:  sdbe: sdbe1
Aug 25 22:49:22 l01ora3 multipathd: sdcb: add path (uevent) 
Aug 25 22:49:22 l01ora3 kernel: sd 3:0:0:8: Attached scsi disk sdbe
Aug 25 22:49:22 l01ora3 multipathd: mpathh: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:176 15 65:240 15 65:48 15 8:112 15 68:48 15 67:112 15 68:240 15] 
Aug 25 22:49:22 l01ora3 kernel: sd 3:0:0:8: Attached scsi generic sg63 type 0
Aug 25 22:49:22 l01ora3 multipathd: sdcb path added to devmap mpathh 
Aug 25 22:49:22 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:22 l01ora3 multipathd: sdbh: add path (uevent) 
Aug 25 22:49:22 l01ora3 multipathd: mpathl: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 5 1 66:240 15 66:48 15 65:112 15 8:176 15 67:176 15] 
Aug 25 22:49:22 l01ora3 multipathd: sdbh path added to devmap mpathl 
Aug 25 22:49:22 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:22 l01ora3 multipathd: dm-11: add map (uevent) 
Aug 25 22:49:22 l01ora3 kernel: SCSI device sdbf: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:22 l01ora3 multipathd: dm-11: devmap already registered 
Aug 25 22:49:23 l01ora3 kernel: sdbf: Write Protect is off
Aug 25 22:49:23 l01ora3 multipathd: sdbk: add path (uevent) 
Aug 25 22:49:23 l01ora3 multipathd: mpathd: load table [0 4026531840 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:96 15 65:160 15 8:224 15 8:32 15 67:32 15 68:160 15 67:224 15] 
Aug 25 22:49:23 l01ora3 kernel: SCSI device sdbf: drive cache: write through
Aug 25 22:49:23 l01ora3 multipathd: sdbk path added to devmap mpathd 
Aug 25 22:49:23 l01ora3 kernel: SCSI device sdbf: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:23 l01ora3 multipathd: dm-8: add map (uevent) 
Aug 25 22:49:23 l01ora3 kernel: sdbf: Write Protect is off
Aug 25 22:49:23 l01ora3 multipathd: dm-8: devmap already registered 
Aug 25 22:49:23 l01ora3 multipathd: dm-12: add map (uevent) 
Aug 25 22:49:23 l01ora3 kernel: SCSI device sdbf: drive cache: write through
Aug 25 22:49:23 l01ora3 multipathd: dm-12: devmap already registered 
Aug 25 22:49:23 l01ora3 multipathd: dm-10: add map (uevent) 
Aug 25 22:49:23 l01ora3 multipathd: dm-10: devmap already registered 
Aug 25 22:49:23 l01ora3 kernel:  sdbf: sdbf1
Aug 25 22:49:23 l01ora3 multipathd: dm-8: add map (uevent) 
Aug 25 22:49:23 l01ora3 kernel: sd 3:0:0:9: Attached scsi disk sdbf
Aug 25 22:49:23 l01ora3 multipathd: dm-8: devmap already registered 
Aug 25 22:49:23 l01ora3 kernel: sd 3:0:0:9: Attached scsi generic sg64 type 0
Aug 25 22:49:23 l01ora3 multipathd: dm-11: add map (uevent) 
Aug 25 22:49:23 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:23 l01ora3 multipathd: dm-11: devmap already registered 
Aug 25 22:49:23 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:23 l01ora3 multipathd: dm-4: add map (uevent) 
Aug 25 22:49:23 l01ora3 kernel: SCSI device sdbg: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:23 l01ora3 multipathd: dm-4: devmap already registered 
Aug 25 22:49:23 l01ora3 kernel: sdbg: Write Protect is off
Aug 25 22:49:23 l01ora3 multipathd: dm-13: add map (uevent) 
Aug 25 22:49:23 l01ora3 multipathd: dm-13: devmap already registered 
Aug 25 22:49:23 l01ora3 kernel: SCSI device sdbg: drive cache: write through
Aug 25 22:49:23 l01ora3 multipathd: sdce: add path (uevent) 
Aug 25 22:49:23 l01ora3 kernel: SCSI device sdbg: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:23 l01ora3 multipathd: mpathj: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:224 15 66:32 15 65:96 15 8:160 15 69:224 15 67:160 15 69:32 15] 
Aug 25 22:49:23 l01ora3 kernel: sdbg: Write Protect is off
Aug 25 22:49:24 l01ora3 multipathd: sdce path added to devmap mpathj 
Aug 25 22:49:24 l01ora3 multipathd: sdbz: add path (uevent) 
Aug 25 22:49:24 l01ora3 kernel: SCSI device sdbg: drive cache: write through
Aug 25 22:49:24 l01ora3 multipathd: mpatha: load table [0 201326592 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:144 15 65:208 15 65:16 15 8:80 15 67:80 15 68:208 15] 
Aug 25 22:49:24 l01ora3 multipathd: sdbz path added to devmap mpatha 
Aug 25 22:49:24 l01ora3 multipathd: sdca: add path (uevent) 
Aug 25 22:49:24 l01ora3 kernel:  sdbg: sdbg1
Aug 25 22:49:24 l01ora3 multipathd: mpathf: load table [0 1006632960 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:160 15 65:224 15 65:32 15 8:96 15 68:32 15 67:96 15 68:224 15] 
Aug 25 22:49:24 l01ora3 kernel: sd 3:0:0:10: Attached scsi disk sdbg
Aug 25 22:49:24 l01ora3 multipathd: sdca path added to devmap mpathf 
Aug 25 22:49:24 l01ora3 kernel: sd 3:0:0:10: Attached scsi generic sg65 type 0
Aug 25 22:49:24 l01ora3 multipathd: sdcg: add path (uevent) 
Aug 25 22:49:24 l01ora3 multipathd: mpathk: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 67:0 15 66:64 15 65:128 15 8:192 15 67:192 15 69:64 15] 
Aug 25 22:49:24 l01ora3 multipathd: sdcg path added to devmap mpathk 
Aug 25 22:49:24 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:24 l01ora3 multipathd: sdbn: add path (uevent) 
Aug 25 22:49:24 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:24 l01ora3 multipathd: mpatha: load table [0 201326592 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:144 15 65:208 15 65:16 15 8:80 15 67:80 15 68:208 15 68:16 15] 
Aug 25 22:49:24 l01ora3 kernel: SCSI device sdbh: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:24 l01ora3 multipathd: sdbn path added to devmap mpatha 
Aug 25 22:49:24 l01ora3 kernel: sdbh: Write Protect is off
Aug 25 22:49:24 l01ora3 multipathd: sdbt: add path (uevent) 
Aug 25 22:49:24 l01ora3 multipathd: mpathl: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:240 15 66:48 15 65:112 15 8:176 15 67:176 15 68:112 15] 
Aug 25 22:49:24 l01ora3 kernel: SCSI device sdbh: drive cache: write through
Aug 25 22:49:24 l01ora3 multipathd: sdbt path added to devmap mpathl 
Aug 25 22:49:24 l01ora3 kernel: SCSI device sdbh: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:24 l01ora3 multipathd: dm-7: add map (uevent) 
Aug 25 22:49:24 l01ora3 kernel: sdbh: Write Protect is off
Aug 25 22:49:24 l01ora3 multipathd: dm-7: devmap already registered 
Aug 25 22:49:24 l01ora3 multipathd: dm-3: add map (uevent) 
Aug 25 22:49:24 l01ora3 kernel: SCSI device sdbh: drive cache: write through
Aug 25 22:49:24 l01ora3 multipathd: dm-3: devmap already registered 
Aug 25 22:49:24 l01ora3 multipathd: dm-12: add map (uevent) 
Aug 25 22:49:24 l01ora3 multipathd: dm-12: devmap already registered 
Aug 25 22:49:24 l01ora3 kernel:  sdbh: sdbh1
Aug 25 22:49:24 l01ora3 multipathd: sdbs: add path (uevent) 
Aug 25 22:49:25 l01ora3 kernel: sd 3:0:0:11: Attached scsi disk sdbh
Aug 25 22:49:25 l01ora3 multipathd: mpathj: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:224 15 66:32 15 65:96 15 8:160 15 69:224 15 67:160 15 69:32 15 68:96 15] 
Aug 25 22:49:25 l01ora3 kernel: sd 3:0:0:11: Attached scsi generic sg66 type 0
Aug 25 22:49:25 l01ora3 multipathd: sdbs path added to devmap mpathj 
Aug 25 22:49:25 l01ora3 multipathd: dm-4: add map (uevent) 
Aug 25 22:49:25 l01ora3 multipathd: dm-4: devmap already registered 
Aug 25 22:49:25 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:25 l01ora3 multipathd: dm-2: add map (uevent) 
Aug 25 22:49:25 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:25 l01ora3 multipathd: dm-2: devmap already registered 
Aug 25 22:49:25 l01ora3 kernel: SCSI device sdbi: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:25 l01ora3 multipathd: dm-9: add map (uevent) 
Aug 25 22:49:25 l01ora3 kernel: sdbi: Write Protect is off
Aug 25 22:49:25 l01ora3 multipathd: dm-9: devmap already registered 
Aug 25 22:49:25 l01ora3 multipathd: sdcp: add path (uevent) 
Aug 25 22:49:25 l01ora3 kernel: SCSI device sdbi: drive cache: write through
Aug 25 22:49:25 l01ora3 multipathd: mpathi: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:208 15 66:16 15 65:80 15 8:144 15 67:144 15 69:208 15] 
Aug 25 22:49:25 l01ora3 kernel: SCSI device sdbi: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:25 l01ora3 multipathd: sdcp path added to devmap mpathi 
Aug 25 22:49:25 l01ora3 kernel: sdbi: Write Protect is off
Aug 25 22:49:25 l01ora3 multipathd: dm-8: add map (uevent) 
Aug 25 22:49:25 l01ora3 multipathd: dm-8: devmap already registered 
Aug 25 22:49:25 l01ora3 kernel: SCSI device sdbi: drive cache: write through
Aug 25 22:49:25 l01ora3 multipathd: sdby: add path (uevent) 
Aug 25 22:49:25 l01ora3 multipathd: mpathb: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:128 15 65:192 15 65:0 15 8:64 15 67:64 15 68:192 15] 
Aug 25 22:49:25 l01ora3 multipathd: sdby path added to devmap mpathb 
Aug 25 22:49:25 l01ora3 kernel:  sdbi: sdbi1
Aug 25 22:49:25 l01ora3 multipathd: dm-5: add map (uevent) 
Aug 25 22:49:25 l01ora3 kernel: sd 3:0:0:12: Attached scsi disk sdbi
Aug 25 22:49:25 l01ora3 multipathd: dm-5: devmap already registered 
Aug 25 22:49:25 l01ora3 kernel: sd 3:0:0:12: Attached scsi generic sg67 type 0
Aug 25 22:49:25 l01ora3 multipathd: dm-10: add map (uevent) 
Aug 25 22:49:25 l01ora3 multipathd: dm-10: devmap already registered 
Aug 25 22:49:25 l01ora3 multipathd: sdcj: add path (uevent) 
Aug 25 22:49:25 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV-LUN-0     Rev: 10.2
Aug 25 22:49:25 l01ora3 multipathd: mpathc: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:112 15 65:176 15 8:240 15 8:48 15 67:48 15 67:240 15 69:112 15] 
Aug 25 22:49:25 l01ora3 kernel:   Type:   RAID                               ANSI SCSI revision: 05
Aug 25 22:49:26 l01ora3 multipathd: sdcj path added to devmap mpathc 
Aug 25 22:49:26 l01ora3 kernel: scsi 3:0:1:0: Attached scsi generic sg68 type 12
Aug 25 22:49:26 l01ora3 multipathd: dm-11: add map (uevent) 
Aug 25 22:49:26 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:26 l01ora3 multipathd: dm-11: devmap already registered 
Aug 25 22:49:26 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:26 l01ora3 multipathd: sdbr: add path (uevent) 
Aug 25 22:49:26 l01ora3 kernel: SCSI device sdbj: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:49:26 l01ora3 multipathd: mpathi: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:208 15 66:16 15 65:80 15 8:144 15 67:144 15 69:208 15 68:80 15] 
Aug 25 22:49:26 l01ora3 kernel: sdbj: Write Protect is off
Aug 25 22:49:26 l01ora3 multipathd: sdbr path added to devmap mpathi 
Aug 25 22:49:26 l01ora3 multipathd: sdcl: add path (uevent) 
Aug 25 22:49:26 l01ora3 kernel: SCSI device sdbj: drive cache: write through
Aug 25 22:49:26 l01ora3 multipathd: mpatha: load table [0 201326592 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:144 15 65:208 15 65:16 15 8:80 15 67:80 15 68:208 15 68:16 15 69:144 15] 
Aug 25 22:49:26 l01ora3 kernel: SCSI device sdbj: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:49:26 l01ora3 multipathd: sdcl path added to devmap mpatha 
Aug 25 22:49:26 l01ora3 kernel: sdbj: Write Protect is off
Aug 25 22:49:26 l01ora3 multipathd: sdcn: add path (uevent) 
Aug 25 22:49:26 l01ora3 multipathd: mpathh: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:176 15 65:240 15 65:48 15 8:112 15 68:48 15 67:112 15 68:240 15 69:176 15] 
Aug 25 22:49:26 l01ora3 kernel: SCSI device sdbj: drive cache: write through
Aug 25 22:49:26 l01ora3 multipathd: sdcn path added to devmap mpathh 
Aug 25 22:49:26 l01ora3 kernel:  sdbj: sdbj1
Aug 25 22:49:26 l01ora3 multipathd: sdck: add path (uevent) 
Aug 25 22:49:26 l01ora3 kernel: sd 3:0:1:1: Attached scsi disk sdbj
Aug 25 22:49:26 l01ora3 multipathd: mpathb: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:128 15 65:192 15 65:0 15 8:64 15 67:64 15 68:192 15 69:128 15] 
Aug 25 22:49:26 l01ora3 kernel: sd 3:0:1:1: Attached scsi generic sg69 type 0
Aug 25 22:49:26 l01ora3 multipathd: sdck path added to devmap mpathb 
Aug 25 22:49:26 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:26 l01ora3 multipathd: dm-4: add map (uevent) 
Aug 25 22:49:26 l01ora3 multipathd: dm-4: devmap already registered 
Aug 25 22:49:26 l01ora3 multipathd: dm-9: add map (uevent) 
Aug 25 22:49:26 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:26 l01ora3 multipathd: dm-9: devmap already registered 
Aug 25 22:49:26 l01ora3 kernel: SCSI device sdbk: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:49:26 l01ora3 multipathd: dm-10: add map (uevent) 
Aug 25 22:49:27 l01ora3 kernel: sdbk: Write Protect is off
Aug 25 22:49:27 l01ora3 multipathd: dm-10: devmap already registered 
Aug 25 22:49:27 l01ora3 multipathd: dm-9: add map (uevent) 
Aug 25 22:49:27 l01ora3 kernel: SCSI device sdbk: drive cache: write through
Aug 25 22:49:27 l01ora3 multipathd: dm-9: devmap already registered 
Aug 25 22:49:27 l01ora3 kernel: SCSI device sdbk: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:49:27 l01ora3 multipathd: dm-3: add map (uevent) 
Aug 25 22:49:27 l01ora3 kernel: sdbk: Write Protect is off
Aug 25 22:49:27 l01ora3 multipathd: dm-3: devmap already registered 
Aug 25 22:49:27 l01ora3 multipathd: dm-5: add map (uevent) 
Aug 25 22:49:27 l01ora3 kernel: SCSI device sdbk: drive cache: write through
Aug 25 22:49:27 l01ora3 multipathd: dm-5: devmap already registered 
Aug 25 22:49:27 l01ora3 multipathd: dm-7: add map (uevent) 
Aug 25 22:49:27 l01ora3 multipathd: dm-7: devmap already registered 
Aug 25 22:49:27 l01ora3 kernel:  sdbk: sdbk1
Aug 25 22:49:27 l01ora3 multipathd: sdci: add path (uevent) 
Aug 25 22:49:27 l01ora3 kernel: sd 3:0:1:2: Attached scsi disk sdbk
Aug 25 22:49:27 l01ora3 multipathd: mpathd: load table [0 4026531840 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:96 15 65:160 15 8:224 15 8:32 15 67:32 15 68:160 15 67:224 15 69:96 15] 
Aug 25 22:49:27 l01ora3 kernel: sd 3:0:1:2: Attached scsi generic sg70 type 0
Aug 25 22:49:27 l01ora3 multipathd: sdci path added to devmap mpathd 
Aug 25 22:49:27 l01ora3 multipathd: dm-12: add map (uevent) 
Aug 25 22:49:27 l01ora3 multipathd: dm-12: devmap already registered 
Aug 25 22:49:27 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:27 l01ora3 multipathd: sdcf: add path (uevent) 
Aug 25 22:49:27 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:27 l01ora3 multipathd: mpathl: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:240 15 66:48 15 65:112 15 8:176 15 67:176 15 68:112 15 69:48 15] 
Aug 25 22:49:27 l01ora3 kernel: SCSI device sdbl: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:27 l01ora3 multipathd: sdcf path added to devmap mpathl 
Aug 25 22:49:27 l01ora3 kernel: sdbl: Write Protect is off
Aug 25 22:49:27 l01ora3 multipathd: sdbu: add path (uevent) 
Aug 25 22:49:27 l01ora3 multipathd: mpathk: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 67:0 15 66:64 15 65:128 15 8:192 15 67:192 15 69:64 15 68:128 15] 
Aug 25 22:49:27 l01ora3 kernel: SCSI device sdbl: drive cache: write through
Aug 25 22:49:27 l01ora3 multipathd: sdbu path added to devmap mpathk 
Aug 25 22:49:27 l01ora3 kernel: SCSI device sdbl: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:27 l01ora3 multipathd: dm-3: add map (uevent) 
Aug 25 22:49:27 l01ora3 kernel: sdbl: Write Protect is off
Aug 25 22:49:27 l01ora3 multipathd: dm-3: devmap already registered 
Aug 25 22:49:28 l01ora3 multipathd: dm-12: add map (uevent) 
Aug 25 22:49:28 l01ora3 kernel: SCSI device sdbl: drive cache: write through
Aug 25 22:49:28 l01ora3 multipathd: dm-12: devmap already registered 
Aug 25 22:49:28 l01ora3 multipathd: dm-2: add map (uevent) 
Aug 25 22:49:28 l01ora3 multipathd: dm-2: devmap already registered 
Aug 25 22:49:28 l01ora3 kernel:  sdbl: sdbl1
Aug 25 22:49:28 l01ora3 multipathd: dm-7: add map (uevent) 
Aug 25 22:49:28 l01ora3 kernel: sd 3:0:1:3: Attached scsi disk sdbl
Aug 25 22:49:28 l01ora3 multipathd: dm-7: devmap already registered 
Aug 25 22:49:28 l01ora3 kernel: sd 3:0:1:3: Attached scsi generic sg71 type 0
Aug 25 22:49:28 l01ora3 multipathd: sdco: add path (uevent) 
Aug 25 22:49:28 l01ora3 multipathd: mpathg: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 6 1 66:192 15 66:0 15 65:64 15 8:128 15 67:128 15 69:192 15] 
Aug 25 22:49:28 l01ora3 multipathd: sdco path added to devmap mpathg 
Aug 25 22:49:28 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:28 l01ora3 multipathd: dm-6: add map (uevent) 
Aug 25 22:49:28 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:28 l01ora3 multipathd: dm-6: devmap already registered 
Aug 25 22:49:28 l01ora3 kernel: SCSI device sdbm: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:28 l01ora3 multipathd: sdcd: add path (uevent) 
Aug 25 22:49:28 l01ora3 kernel: sdbm: Write Protect is off
Aug 25 22:49:28 l01ora3 multipathd: mpathi: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:208 15 66:16 15 65:80 15 8:144 15 67:144 15 69:208 15 68:80 15 69:16 15] 
Aug 25 22:49:28 l01ora3 multipathd: sdcd path added to devmap mpathi 
Aug 25 22:49:28 l01ora3 kernel: SCSI device sdbm: drive cache: write through
Aug 25 22:49:28 l01ora3 multipathd: dm-5: add map (uevent) 
Aug 25 22:49:28 l01ora3 kernel: SCSI device sdbm: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:28 l01ora3 multipathd: dm-5: devmap already registered 
Aug 25 22:49:28 l01ora3 kernel: sdbm: Write Protect is off
Aug 25 22:49:28 l01ora3 multipathd: sdch: add path (uevent) 
Aug 25 22:49:28 l01ora3 multipathd: mpathe: load table [0 2013265920 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:80 15 65:144 15 8:208 15 8:16 15 67:16 15 67:208 15 69:80 15] 
Aug 25 22:49:29 l01ora3 kernel: SCSI device sdbm: drive cache: write through
Aug 25 22:49:29 l01ora3 multipathd: sdch path added to devmap mpathe 
Aug 25 22:49:29 l01ora3 multipathd: dm-13: add map (uevent) 
Aug 25 22:49:29 l01ora3 multipathd: dm-13: devmap already registered 
Aug 25 22:49:29 l01ora3 kernel:  sdbm: sdbm1
Aug 25 22:49:29 l01ora3 multipathd: sdbq: add path (uevent) 
Aug 25 22:49:29 l01ora3 kernel: sd 3:0:1:4: Attached scsi disk sdbm
Aug 25 22:49:29 l01ora3 multipathd: mpathg: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 7 1 66:192 15 66:0 15 65:64 15 8:128 15 67:128 15 69:192 15 68:64 15] 
Aug 25 22:49:29 l01ora3 kernel: sd 3:0:1:4: Attached scsi generic sg72 type 0
Aug 25 22:49:29 l01ora3 multipathd: sdbq path added to devmap mpathg 
Aug 25 22:49:29 l01ora3 multipathd: dm-6: add map (uevent) 
Aug 25 22:49:29 l01ora3 multipathd: dm-6: devmap already registered 
Aug 25 22:49:29 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:29 l01ora3 multipathd: sdbv: add path (uevent) 
Aug 25 22:49:29 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:29 l01ora3 multipathd: mpathe: load table [0 2013265920 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:80 15 65:144 15 8:208 15 8:16 15 67:16 15 67:208 15 69:80 15 68:144 15] 
Aug 25 22:49:29 l01ora3 kernel: SCSI device sdbn: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:49:29 l01ora3 multipathd: sdbv path added to devmap mpathe 
Aug 25 22:49:29 l01ora3 kernel: sdbn: Write Protect is off
Aug 25 22:49:29 l01ora3 multipathd: dm-13: add map (uevent) 
Aug 25 22:49:29 l01ora3 multipathd: dm-13: devmap already registered 
Aug 25 22:49:29 l01ora3 kernel: SCSI device sdbn: drive cache: write through
Aug 25 22:49:29 l01ora3 multipathd: sdcr: add path (uevent) 
Aug 25 22:49:29 l01ora3 kernel: SCSI device sdbn: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:49:29 l01ora3 multipathd: mpathl: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:240 15 66:48 15 65:112 15 8:176 15 67:176 15 68:112 15 69:48 15 69:240 15] 
Aug 25 22:49:29 l01ora3 kernel: sdbn: Write Protect is off
Aug 25 22:49:29 l01ora3 multipathd: sdcr path added to devmap mpathl 
Aug 25 22:49:29 l01ora3 multipathd: sdcc: add path (uevent) 
Aug 25 22:49:29 l01ora3 kernel: SCSI device sdbn: drive cache: write through
Aug 25 22:49:29 l01ora3 multipathd: mpathg: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:192 15 66:0 15 65:64 15 8:128 15 67:128 15 69:192 15 68:64 15 69:0 15] 
Aug 25 22:49:29 l01ora3 multipathd: sdcc path added to devmap mpathg 
Aug 25 22:49:29 l01ora3 multipathd: dm-3: add map (uevent) 
Aug 25 22:49:29 l01ora3 kernel:  sdbn: sdbn1
Aug 25 22:49:29 l01ora3 multipathd: dm-3: devmap already registered 
Aug 25 22:49:30 l01ora3 kernel: sd 3:0:1:5: Attached scsi disk sdbn
Aug 25 22:49:30 l01ora3 multipathd: dm-6: add map (uevent) 
Aug 25 22:49:30 l01ora3 kernel: sd 3:0:1:5: Attached scsi generic sg73 type 0
Aug 25 22:49:30 l01ora3 multipathd: dm-6: devmap already registered 
Aug 25 22:49:30 l01ora3 multipathd: sdcm: add path (uevent) 
Aug 25 22:49:30 l01ora3 multipathd: mpathf: load table [0 1006632960 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:160 15 65:224 15 65:32 15 8:96 15 68:32 15 67:96 15 68:224 15 69:160 15] 
Aug 25 22:49:30 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:30 l01ora3 multipathd: sdcm path added to devmap mpathf 
Aug 25 22:49:30 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:30 l01ora3 multipathd: dm-8: add map (uevent) 
Aug 25 22:49:30 l01ora3 kernel: SCSI device sdbo: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:49:30 l01ora3 multipathd: dm-8: devmap already registered 
Aug 25 22:49:30 l01ora3 kernel: sdbo: Write Protect is off
Aug 25 22:49:30 l01ora3 multipathd: sdcs: add path (uevent) 
Aug 25 22:49:30 l01ora3 multipathd: mpathk: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 67:0 15 66:64 15 65:128 15 8:192 15 67:192 15 69:64 15 68:128 15 70:0 15] 
Aug 25 22:49:30 l01ora3 kernel: SCSI device sdbo: drive cache: write through
Aug 25 22:49:30 l01ora3 multipathd: sdcs path added to devmap mpathk 
Aug 25 22:49:30 l01ora3 kernel: SCSI device sdbo: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:49:30 l01ora3 multipathd: dm-2: add map (uevent) 
Aug 25 22:49:30 l01ora3 kernel: sdbo: Write Protect is off
Aug 25 22:49:30 l01ora3 multipathd: dm-2: devmap already registered 
Aug 25 22:49:30 l01ora3 multipathd: sdbm: add path (uevent) 
Aug 25 22:49:30 l01ora3 kernel: SCSI device sdbo: drive cache: write through
Aug 25 22:49:30 l01ora3 multipathd: mpathb: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:128 15 65:192 15 65:0 15 8:64 15 67:64 15 68:192 15 69:128 15 68:0 15] 
Aug 25 22:49:30 l01ora3 multipathd: sdbm path added to devmap mpathb 
Aug 25 22:49:30 l01ora3 multipathd: dm-10: add map (uevent) 
Aug 25 22:49:30 l01ora3 kernel:  sdbo: sdbo1
Aug 25 22:49:30 l01ora3 multipathd: dm-10: devmap already registered 
Aug 25 22:49:30 l01ora3 kernel: sd 3:0:1:6: Attached scsi disk sdbo
Aug 25 22:49:30 l01ora3 multipathd: sdbx: add path (uevent) 
Aug 25 22:49:30 l01ora3 kernel: sd 3:0:1:6: Attached scsi generic sg74 type 0
Aug 25 22:49:30 l01ora3 multipathd: mpathc: load table [0 33554432 multipath 1 queue_if_no_path 0 1 1 round-robin 0 8 1 66:112 15 65:176 15 8:240 15 8:48 15 67:48 15 67:240 15 69:112 15 68:176 15] 
Aug 25 22:49:30 l01ora3 multipathd: sdbx path added to devmap mpathc 
Aug 25 22:49:30 l01ora3 multipathd: dm-11: add map (uevent) 
Aug 25 22:49:30 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:31 l01ora3 multipathd: dm-11: devmap already registered 
Aug 25 22:49:31 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbp: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:31 l01ora3 kernel: sdbp: Write Protect is off
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbp: drive cache: write through
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbp: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:31 l01ora3 kernel: sdbp: Write Protect is off
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbp: drive cache: write through
Aug 25 22:49:31 l01ora3 kernel:  sdbp: sdbp1
Aug 25 22:49:31 l01ora3 kernel: sd 3:0:1:7: Attached scsi disk sdbp
Aug 25 22:49:31 l01ora3 kernel: sd 3:0:1:7: Attached scsi generic sg75 type 0
Aug 25 22:49:31 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:31 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbq: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:31 l01ora3 kernel: sdbq: Write Protect is off
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbq: drive cache: write through
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbq: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:31 l01ora3 kernel: sdbq: Write Protect is off
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbq: drive cache: write through
Aug 25 22:49:31 l01ora3 kernel:  sdbq: sdbq1
Aug 25 22:49:31 l01ora3 kernel: sd 3:0:1:8: Attached scsi disk sdbq
Aug 25 22:49:31 l01ora3 kernel: sd 3:0:1:8: Attached scsi generic sg76 type 0
Aug 25 22:49:31 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:31 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbr: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:31 l01ora3 kernel: sdbr: Write Protect is off
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbr: drive cache: write through
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbr: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:31 l01ora3 kernel: sdbr: Write Protect is off
Aug 25 22:49:31 l01ora3 kernel: SCSI device sdbr: drive cache: write through
Aug 25 22:49:31 l01ora3 kernel:  sdbr: sdbr1
Aug 25 22:49:31 l01ora3 kernel: sd 3:0:1:9: Attached scsi disk sdbr
Aug 25 22:49:31 l01ora3 kernel: sd 3:0:1:9: Attached scsi generic sg77 type 0
Aug 25 22:49:32 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:32 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbs: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:32 l01ora3 kernel: sdbs: Write Protect is off
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbs: drive cache: write through
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbs: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:32 l01ora3 kernel: sdbs: Write Protect is off
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbs: drive cache: write through
Aug 25 22:49:32 l01ora3 kernel:  sdbs: sdbs1
Aug 25 22:49:32 l01ora3 kernel: sd 3:0:1:10: Attached scsi disk sdbs
Aug 25 22:49:32 l01ora3 kernel: sd 3:0:1:10: Attached scsi generic sg78 type 0
Aug 25 22:49:32 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:32 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbt: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:32 l01ora3 kernel: sdbt: Write Protect is off
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbt: drive cache: write through
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbt: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:32 l01ora3 kernel: sdbt: Write Protect is off
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbt: drive cache: write through
Aug 25 22:49:32 l01ora3 kernel:  sdbt: sdbt1
Aug 25 22:49:32 l01ora3 kernel: sd 3:0:1:11: Attached scsi disk sdbt
Aug 25 22:49:32 l01ora3 kernel: sd 3:0:1:11: Attached scsi generic sg79 type 0
Aug 25 22:49:32 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:32 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbu: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:32 l01ora3 kernel: sdbu: Write Protect is off
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbu: drive cache: write through
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbu: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:32 l01ora3 kernel: sdbu: Write Protect is off
Aug 25 22:49:32 l01ora3 kernel: SCSI device sdbu: drive cache: write through
Aug 25 22:49:32 l01ora3 kernel:  sdbu: sdbu1
Aug 25 22:49:32 l01ora3 kernel: sd 3:0:1:12: Attached scsi disk sdbu
Aug 25 22:49:32 l01ora3 kernel: sd 3:0:1:12: Attached scsi generic sg80 type 0
Aug 25 22:49:32 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV-LUN-0     Rev: 10.2
Aug 25 22:49:32 l01ora3 kernel:   Type:   RAID                               ANSI SCSI revision: 05
Aug 25 22:49:32 l01ora3 kernel: scsi 3:0:2:0: Attached scsi generic sg81 type 12
Aug 25 22:49:32 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:32 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbv: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:49:33 l01ora3 kernel: sdbv: Write Protect is off
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbv: drive cache: write through
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbv: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:49:33 l01ora3 kernel: sdbv: Write Protect is off
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbv: drive cache: write through
Aug 25 22:49:33 l01ora3 kernel:  sdbv: sdbv1
Aug 25 22:49:33 l01ora3 kernel: sd 3:0:2:1: Attached scsi disk sdbv
Aug 25 22:49:33 l01ora3 kernel: sd 3:0:2:1: Attached scsi generic sg82 type 0
Aug 25 22:49:33 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:33 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbw: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:49:33 l01ora3 kernel: sdbw: Write Protect is off
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbw: drive cache: write through
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbw: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:49:33 l01ora3 kernel: sdbw: Write Protect is off
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbw: drive cache: write through
Aug 25 22:49:33 l01ora3 kernel:  sdbw: sdbw1
Aug 25 22:49:33 l01ora3 kernel: sd 3:0:2:2: Attached scsi disk sdbw
Aug 25 22:49:33 l01ora3 kernel: sd 3:0:2:2: Attached scsi generic sg83 type 0
Aug 25 22:49:33 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:33 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbx: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:33 l01ora3 kernel: sdbx: Write Protect is off
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbx: drive cache: write through
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbx: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:33 l01ora3 kernel: sdbx: Write Protect is off
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdbx: drive cache: write through
Aug 25 22:49:33 l01ora3 kernel:  sdbx: sdbx1
Aug 25 22:49:33 l01ora3 kernel: sd 3:0:2:3: Attached scsi disk sdbx
Aug 25 22:49:33 l01ora3 kernel: sd 3:0:2:3: Attached scsi generic sg84 type 0
Aug 25 22:49:33 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:33 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdby: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:33 l01ora3 kernel: sdby: Write Protect is off
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdby: drive cache: write through
Aug 25 22:49:33 l01ora3 kernel: SCSI device sdby: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:34 l01ora3 kernel: sdby: Write Protect is off
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdby: drive cache: write through
Aug 25 22:49:34 l01ora3 kernel:  sdby: sdby1
Aug 25 22:49:34 l01ora3 kernel: sd 3:0:2:4: Attached scsi disk sdby
Aug 25 22:49:34 l01ora3 kernel: sd 3:0:2:4: Attached scsi generic sg85 type 0
Aug 25 22:49:34 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:34 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdbz: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:49:34 l01ora3 kernel: sdbz: Write Protect is off
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdbz: drive cache: write through
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdbz: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:49:34 l01ora3 kernel: sdbz: Write Protect is off
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdbz: drive cache: write through
Aug 25 22:49:34 l01ora3 kernel:  sdbz: sdbz1
Aug 25 22:49:34 l01ora3 kernel: sd 3:0:2:5: Attached scsi disk sdbz
Aug 25 22:49:34 l01ora3 kernel: sd 3:0:2:5: Attached scsi generic sg86 type 0
Aug 25 22:49:34 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:34 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdca: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:49:34 l01ora3 kernel: sdca: Write Protect is off
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdca: drive cache: write through
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdca: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:49:34 l01ora3 kernel: sdca: Write Protect is off
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdca: drive cache: write through
Aug 25 22:49:34 l01ora3 kernel:  sdca: sdca1
Aug 25 22:49:34 l01ora3 kernel: sd 3:0:2:6: Attached scsi disk sdca
Aug 25 22:49:34 l01ora3 kernel: sd 3:0:2:6: Attached scsi generic sg87 type 0
Aug 25 22:49:34 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:34 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdcb: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:34 l01ora3 kernel: sdcb: Write Protect is off
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdcb: drive cache: write through
Aug 25 22:49:34 l01ora3 kernel: SCSI device sdcb: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:34 l01ora3 kernel: sdcb: Write Protect is off
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdcb: drive cache: write through
Aug 25 22:49:35 l01ora3 kernel:  sdcb: sdcb1
Aug 25 22:49:35 l01ora3 kernel: sd 3:0:2:7: Attached scsi disk sdcb
Aug 25 22:49:35 l01ora3 kernel: sd 3:0:2:7: Attached scsi generic sg88 type 0
Aug 25 22:49:35 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:35 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdcc: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:35 l01ora3 kernel: sdcc: Write Protect is off
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdcc: drive cache: write through
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdcc: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:35 l01ora3 kernel: sdcc: Write Protect is off
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdcc: drive cache: write through
Aug 25 22:49:35 l01ora3 kernel:  sdcc: sdcc1
Aug 25 22:49:35 l01ora3 kernel: sd 3:0:2:8: Attached scsi disk sdcc
Aug 25 22:49:35 l01ora3 kernel: sd 3:0:2:8: Attached scsi generic sg89 type 0
Aug 25 22:49:35 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:35 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdcd: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:35 l01ora3 kernel: sdcd: Write Protect is off
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdcd: drive cache: write through
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdcd: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:35 l01ora3 kernel: sdcd: Write Protect is off
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdcd: drive cache: write through
Aug 25 22:49:35 l01ora3 kernel:  sdcd: sdcd1
Aug 25 22:49:35 l01ora3 kernel: sd 3:0:2:9: Attached scsi disk sdcd
Aug 25 22:49:35 l01ora3 kernel: sd 3:0:2:9: Attached scsi generic sg90 type 0
Aug 25 22:49:35 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:35 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdce: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:35 l01ora3 kernel: sdce: Write Protect is off
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdce: drive cache: write through
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdce: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:35 l01ora3 kernel: sdce: Write Protect is off
Aug 25 22:49:35 l01ora3 kernel: SCSI device sdce: drive cache: write through
Aug 25 22:49:35 l01ora3 kernel:  sdce: sdce1
Aug 25 22:49:36 l01ora3 kernel: sd 3:0:2:10: Attached scsi disk sdce
Aug 25 22:49:36 l01ora3 kernel: sd 3:0:2:10: Attached scsi generic sg91 type 0
Aug 25 22:49:36 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:36 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdcf: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:36 l01ora3 kernel: sdcf: Write Protect is off
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdcf: drive cache: write through
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdcf: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:36 l01ora3 kernel: sdcf: Write Protect is off
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdcf: drive cache: write through
Aug 25 22:49:36 l01ora3 kernel:  sdcf: sdcf1
Aug 25 22:49:36 l01ora3 kernel: sd 3:0:2:11: Attached scsi disk sdcf
Aug 25 22:49:36 l01ora3 kernel: sd 3:0:2:11: Attached scsi generic sg92 type 0
Aug 25 22:49:36 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:36 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdcg: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:36 l01ora3 kernel: sdcg: Write Protect is off
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdcg: drive cache: write through
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdcg: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:36 l01ora3 kernel: sdcg: Write Protect is off
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdcg: drive cache: write through
Aug 25 22:49:36 l01ora3 kernel:  sdcg: sdcg1
Aug 25 22:49:36 l01ora3 kernel: sd 3:0:2:12: Attached scsi disk sdcg
Aug 25 22:49:36 l01ora3 kernel: sd 3:0:2:12: Attached scsi generic sg93 type 0
Aug 25 22:49:36 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV-LUN-0     Rev: 10.2
Aug 25 22:49:36 l01ora3 kernel:   Type:   RAID                               ANSI SCSI revision: 05
Aug 25 22:49:36 l01ora3 kernel: scsi 3:0:3:0: Attached scsi generic sg94 type 12
Aug 25 22:49:36 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:36 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdch: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:49:36 l01ora3 kernel: sdch: Write Protect is off
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdch: drive cache: write through
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdch: 2013265920 512-byte hdwr sectors (1030792 MB)
Aug 25 22:49:36 l01ora3 kernel: sdch: Write Protect is off
Aug 25 22:49:36 l01ora3 kernel: SCSI device sdch: drive cache: write through
Aug 25 22:49:37 l01ora3 kernel:  sdch: sdch1
Aug 25 22:49:37 l01ora3 kernel: sd 3:0:3:1: Attached scsi disk sdch
Aug 25 22:49:37 l01ora3 kernel: sd 3:0:3:1: Attached scsi generic sg95 type 0
Aug 25 22:49:37 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:37 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdci: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:49:37 l01ora3 kernel: sdci: Write Protect is off
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdci: drive cache: write through
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdci: 4026531840 512-byte hdwr sectors (2061584 MB)
Aug 25 22:49:37 l01ora3 kernel: sdci: Write Protect is off
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdci: drive cache: write through
Aug 25 22:49:37 l01ora3 kernel:  sdci: sdci1
Aug 25 22:49:37 l01ora3 kernel: sd 3:0:3:2: Attached scsi disk sdci
Aug 25 22:49:37 l01ora3 kernel: sd 3:0:3:2: Attached scsi generic sg96 type 0
Aug 25 22:49:37 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:37 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdcj: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:37 l01ora3 kernel: sdcj: Write Protect is off
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdcj: drive cache: write through
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdcj: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:37 l01ora3 kernel: sdcj: Write Protect is off
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdcj: drive cache: write through
Aug 25 22:49:37 l01ora3 kernel:  sdcj: sdcj1
Aug 25 22:49:37 l01ora3 kernel: sd 3:0:3:3: Attached scsi disk sdcj
Aug 25 22:49:37 l01ora3 kernel: sd 3:0:3:3: Attached scsi generic sg97 type 0
Aug 25 22:49:37 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:37 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdck: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:37 l01ora3 kernel: sdck: Write Protect is off
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdck: drive cache: write through
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdck: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:37 l01ora3 kernel: sdck: Write Protect is off
Aug 25 22:49:37 l01ora3 kernel: SCSI device sdck: drive cache: write through
Aug 25 22:49:37 l01ora3 kernel:  sdck: sdck1
Aug 25 22:49:37 l01ora3 kernel: sd 3:0:3:4: Attached scsi disk sdck
Aug 25 22:49:38 l01ora3 kernel: sd 3:0:3:4: Attached scsi generic sg98 type 0
Aug 25 22:49:38 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:38 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcl: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:49:38 l01ora3 kernel: sdcl: Write Protect is off
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcl: drive cache: write through
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcl: 201326592 512-byte hdwr sectors (103079 MB)
Aug 25 22:49:38 l01ora3 kernel: sdcl: Write Protect is off
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcl: drive cache: write through
Aug 25 22:49:38 l01ora3 kernel:  sdcl: sdcl1
Aug 25 22:49:38 l01ora3 kernel: sd 3:0:3:5: Attached scsi disk sdcl
Aug 25 22:49:38 l01ora3 kernel: sd 3:0:3:5: Attached scsi generic sg99 type 0
Aug 25 22:49:38 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:38 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcm: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:49:38 l01ora3 kernel: sdcm: Write Protect is off
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcm: drive cache: write through
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcm: 1006632960 512-byte hdwr sectors (515396 MB)
Aug 25 22:49:38 l01ora3 kernel: sdcm: Write Protect is off
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcm: drive cache: write through
Aug 25 22:49:38 l01ora3 kernel:  sdcm: sdcm1
Aug 25 22:49:38 l01ora3 kernel: sd 3:0:3:6: Attached scsi disk sdcm
Aug 25 22:49:38 l01ora3 kernel: sd 3:0:3:6: Attached scsi generic sg100 type 0
Aug 25 22:49:38 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:38 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcn: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:38 l01ora3 kernel: sdcn: Write Protect is off
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcn: drive cache: write through
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcn: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:38 l01ora3 kernel: sdcn: Write Protect is off
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdcn: drive cache: write through
Aug 25 22:49:38 l01ora3 kernel:  sdcn: sdcn1
Aug 25 22:49:38 l01ora3 kernel: sd 3:0:3:7: Attached scsi disk sdcn
Aug 25 22:49:38 l01ora3 kernel: sd 3:0:3:7: Attached scsi generic sg101 type 0
Aug 25 22:49:38 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:38 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:38 l01ora3 kernel: SCSI device sdco: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:38 l01ora3 kernel: sdco: Write Protect is off
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdco: drive cache: write through
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdco: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:39 l01ora3 kernel: sdco: Write Protect is off
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdco: drive cache: write through
Aug 25 22:49:39 l01ora3 kernel:  sdco: sdco1
Aug 25 22:49:39 l01ora3 kernel: sd 3:0:3:8: Attached scsi disk sdco
Aug 25 22:49:39 l01ora3 kernel: sd 3:0:3:8: Attached scsi generic sg102 type 0
Aug 25 22:49:39 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:39 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcp: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:39 l01ora3 kernel: sdcp: Write Protect is off
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcp: drive cache: write through
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcp: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:39 l01ora3 kernel: sdcp: Write Protect is off
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcp: drive cache: write through
Aug 25 22:49:39 l01ora3 kernel:  sdcp: sdcp1
Aug 25 22:49:39 l01ora3 kernel: sd 3:0:3:9: Attached scsi disk sdcp
Aug 25 22:49:39 l01ora3 kernel: sd 3:0:3:9: Attached scsi generic sg103 type 0
Aug 25 22:49:39 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:39 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcq: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:39 l01ora3 kernel: sdcq: Write Protect is off
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcq: drive cache: write through
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcq: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:39 l01ora3 kernel: sdcq: Write Protect is off
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcq: drive cache: write through
Aug 25 22:49:39 l01ora3 kernel:  sdcq: sdcq1
Aug 25 22:49:39 l01ora3 kernel: sd 3:0:3:10: Attached scsi disk sdcq
Aug 25 22:49:39 l01ora3 kernel: sd 3:0:3:10: Attached scsi generic sg104 type 0
Aug 25 22:49:39 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:39 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcr: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:39 l01ora3 kernel: sdcr: Write Protect is off
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcr: drive cache: write through
Aug 25 22:49:39 l01ora3 kernel: SCSI device sdcr: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:39 l01ora3 kernel: sdcr: Write Protect is off
Aug 25 22:49:40 l01ora3 kernel: SCSI device sdcr: drive cache: write through
Aug 25 22:49:40 l01ora3 kernel:  sdcr: sdcr1
Aug 25 22:49:40 l01ora3 kernel: sd 3:0:3:11: Attached scsi disk sdcr
Aug 25 22:49:40 l01ora3 kernel: sd 3:0:3:11: Attached scsi generic sg105 type 0
Aug 25 22:49:40 l01ora3 kernel:   Vendor: IBM       Model: 2810XIV           Rev: 10.2
Aug 25 22:49:40 l01ora3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Aug 25 22:49:40 l01ora3 kernel: SCSI device sdcs: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:40 l01ora3 kernel: sdcs: Write Protect is off
Aug 25 22:49:40 l01ora3 kernel: SCSI device sdcs: drive cache: write through
Aug 25 22:49:40 l01ora3 kernel: SCSI device sdcs: 33554432 512-byte hdwr sectors (17180 MB)
Aug 25 22:49:40 l01ora3 kernel: sdcs: Write Protect is off
Aug 25 22:49:40 l01ora3 kernel: SCSI device sdcs: drive cache: write through
Aug 25 22:49:40 l01ora3 kernel:  sdcs: sdcs1
Aug 25 22:49:40 l01ora3 kernel: sd 3:0:3:12: Attached scsi disk sdcs
Aug 25 22:49:40 l01ora3 kernel: sd 3:0:3:12: Attached scsi generic sg106 type 0
Aug 25 22:53:34 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:53:34 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:53:34 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 236 seconds.
Aug 25 22:57:29 l01ora3 dhclient: DHCPREQUEST on usb0 to 169.254.95.118 port 67
Aug 25 22:57:30 l01ora3 dhclient: DHCPACK from 169.254.95.118
Aug 25 22:57:30 l01ora3 dhclient: bound to 169.254.95.120 -- renewal in 266 seconds.
Aug 25 22:58:16 l01ora3 kernel: qla2xxx 0000:24:00.0: LOOP DOWN detected (2 0 0).
Aug 25 22:58:46 l01ora3 kernel:  rport-3:0-14: blocked FC remote port time out: saving binding
Aug 25 22:58:46 l01ora3 kernel:  rport-3:0-15: blocked FC remote port time out: saving binding
Aug 25 22:58:46 l01ora3 kernel:  rport-3:0-16: blocked FC remote port time out: saving binding
Aug 25 22:58:46 l01ora3 kernel:  rport-3:0-17: blocked FC remote port time out: saving binding
Aug 25 22:58:46 l01ora3 kernel: sd 3:0:0:1: SCSI error: return code = 0x00010000
Aug 25 22:58:46 l01ora3 kernel: end_request: I/O error, dev sdax, sector 2367
Aug 25 22:58:46 l01ora3 kernel: device-mapper: multipath: Failing path 67:16.
Aug 25 22:58:46 l01ora3 kernel: sd 3:0:0:3: SCSI error: return code = 0x00010000
Aug 25 22:58:46 l01ora3 kernel: end_request: I/O error, dev sdaz, sector 2367
Aug 25 22:58:46 l01ora3 kernel: device-mapper: multipath: Failing path 67:48.
Aug 25 22:58:46 l01ora3 kernel: sd 3:0:0:5: SCSI error: return code = 0x00010000
Aug 25 22:58:46 l01ora3 kernel: end_request: I/O error, dev sdbb, sector 2367
Aug 25 22:58:46 l01ora3 kernel: device-mapper: multipath: Failing path 67:80.
Aug 25 22:58:46 l01ora3 multipathd: 67:16: mark as failed 
Aug 25 22:58:46 l01ora3 kernel: sd 3:0:0:6: SCSI error: return code = 0x00010000
Aug 25 22:58:46 l01ora3 multipathd: mpathe: remaining active paths: 7 
Aug 25 22:58:46 l01ora3 kernel: end_request: I/O error, dev sdbc, sector 2367
Aug 25 22:58:46 l01ora3 multipathd: 67:48: mark as failed 
Aug 25 22:58:46 l01ora3 kernel: device-mapper: multipath: Failing path 67:96.
Aug 25 22:58:46 l01ora3 multipathd: mpathc: remaining active paths: 7 
Aug 25 22:58:46 l01ora3 kernel: sd 3:0:0:6: SCSI error: return code = 0x00010000
Aug 25 22:58:46 l01ora3 multipathd: 67:176: mark as failed 
Aug 25 22:58:46 l01ora3 kernel: end_request: I/O error, dev sdbc, sector 2080935
Aug 25 22:58:46 l01ora3 multipathd: mpathl: remaining active paths: 7 
Aug 25 22:58:46 l01ora3 kernel: sd 3:0:0:8: SCSI error: return code = 0x00010000
Aug 25 22:58:46 l01ora3 multipathd: 67:80: mark as failed 
Aug 25 22:58:46 l01ora3 kernel: end_request: I/O error, dev sdbe, sector 2367
Aug 25 22:58:46 l01ora3 multipathd: mpatha: remaining active paths: 7 
Aug 25 22:58:46 l01ora3 kernel: device-mapper: multipath: Failing path 67:128.
Aug 25 22:58:46 l01ora3 multipathd: 67:96: mark as failed 
Aug 25 22:58:46 l01ora3 kernel: sd 3:0:0:11: SCSI error: return code = 0x00010000
Aug 25 22:58:46 l01ora3 multipathd: mpathf: remaining active paths: 7 
Aug 25 22:58:46 l01ora3 kernel: end_request: I/O error, dev sdbh, sector 2247
Aug 25 22:58:46 l01ora3 multipathd: 68:224: mark as failed 
Aug 25 22:58:46 l01ora3 kernel: device-mapper: multipath: Failing path 67:176.
Aug 25 22:58:47 l01ora3 multipathd: mpathf: remaining active paths: 6 
Aug 25 22:58:47 l01ora3 kernel: sd 3:0:0:12: SCSI error: return code = 0x00010000
Aug 25 22:58:47 l01ora3 multipathd: 69:160: mark as failed 
Aug 25 22:58:47 l01ora3 kernel: end_request: I/O error, dev sdbi, sector 2247
Aug 25 22:58:47 l01ora3 multipathd: mpathf: remaining active paths: 5 
Aug 25 22:58:47 l01ora3 kernel: device-mapper: multipath: Failing path 67:192.
Aug 25 22:58:47 l01ora3 multipathd: 67:192: mark as failed 
Aug 25 22:58:47 l01ora3 kernel: sd 3:0:1:7: SCSI error: return code = 0x00010000
Aug 25 22:58:47 l01ora3 multipathd: mpathk: remaining active paths: 7 
Aug 25 22:58:47 l01ora3 kernel: end_request: I/O error, dev sdbp, sector 2367
Aug 25 22:58:47 l01ora3 multipathd: 69:64: mark as failed 
Aug 25 22:58:47 l01ora3 kernel: device-mapper: multipath: Failing path 68:48.
Aug 25 22:58:47 l01ora3 multipathd: mpathk: remaining active paths: 6 
Aug 25 22:58:47 l01ora3 kernel: sd 3:0:3:9: SCSI error: return code = 0x00010000
Aug 25 22:58:47 l01ora3 multipathd: 68:48: mark as failed 
Aug 25 22:58:47 l01ora3 kernel: end_request: I/O error, dev sdcp, sector 2247
Aug 25 22:58:47 l01ora3 multipathd: mpathh: remaining active paths: 7 
Aug 25 22:58:47 l01ora3 kernel: device-mapper: multipath: Failing path 69:208.
Aug 25 22:58:47 l01ora3 multipathd: 67:112: mark as failed 
Aug 25 22:58:47 l01ora3 kernel: sd 3:0:3:9: SCSI error: return code = 0x00010000
Aug 25 22:58:47 l01ora3 multipathd: mpathh: remaining active paths: 6 
Aug 25 22:58:47 l01ora3 kernel: end_request: I/O error, dev sdcp, sector 282863
Aug 25 22:58:47 l01ora3 multipathd: 68:240: mark as failed 
Aug 25 22:58:47 l01ora3 kernel: sd 3:0:3:10: SCSI error: return code = 0x00010000
Aug 25 22:58:47 l01ora3 multipathd: mpathh: remaining active paths: 5 
Aug 25 22:58:47 l01ora3 kernel: end_request: I/O error, dev sdcq, sector 2247
Aug 25 22:58:47 l01ora3 multipathd: 69:208: mark as failed 
Aug 25 22:58:47 l01ora3 kernel: device-mapper: multipath: Failing path 69:224.
Aug 25 22:58:47 l01ora3 multipathd: mpathi: remaining active paths: 7 
Aug 25 22:58:47 l01ora3 kernel: sd 3:0:0:6: SCSI error: return code = 0x00010000
Aug 25 22:58:47 l01ora3 multipathd: 68:80: mark as failed 
Aug 25 22:58:47 l01ora3 kernel: end_request: I/O error, dev sdbc, sector 2097296
Aug 25 22:58:47 l01ora3 multipathd: mpathi: remaining active paths: 6 
Aug 25 22:58:47 l01ora3 kernel: sd 3:0:0:6: SCSI error: return code = 0x00010000
Aug 25 22:58:47 l01ora3 multipathd: 69:16: mark as failed 
Aug 25 22:58:47 l01ora3 kernel: end_request: I/O error, dev sdbc, sector 2097808
Aug 25 22:58:48 l01ora3 multipathd: mpathi: remaining active paths: 5 
Aug 25 22:58:48 l01ora3 kernel: sd 3:0:3:9: SCSI error: return code = 0x00010000
Aug 25 22:58:48 l01ora3 multipathd: 69:224: mark as failed 
Aug 25 22:58:48 l01ora3 kernel: end_request: I/O error, dev sdcp, sector 1032280
Aug 25 22:58:48 l01ora3 multipathd: mpathj: remaining active paths: 7 
Aug 25 22:58:48 l01ora3 kernel: sd 3:0:3:9: SCSI error: return code = 0x00010000
Aug 25 22:58:48 l01ora3 multipathd: 67:160: mark as failed 
Aug 25 22:58:48 l01ora3 kernel: end_request: I/O error, dev sdcp, sector 1032792
Aug 25 22:58:48 l01ora3 multipathd: mpathj: remaining active paths: 6 
Aug 25 22:58:48 l01ora3 kernel: sd 3:0:2:6: SCSI error: return code = 0x00010000
Aug 25 22:58:48 l01ora3 multipathd: 69:32: mark as failed 
Aug 25 22:58:48 l01ora3 kernel: end_request: I/O error, dev sdca, sector 2367
Aug 25 22:58:48 l01ora3 multipathd: mpathj: remaining active paths: 5 
Aug 25 22:58:48 l01ora3 kernel: device-mapper: multipath: Failing path 68:224.
Aug 25 22:58:48 l01ora3 multipathd: 67:64: mark as failed 
Aug 25 22:58:48 l01ora3 kernel: sd 3:0:2:6: SCSI error: return code = 0x00010000
Aug 25 22:58:48 l01ora3 multipathd: mpathb: remaining active paths: 7 
Aug 25 22:58:48 l01ora3 kernel: end_request: I/O error, dev sdca, sector 2097296
Aug 25 22:58:48 l01ora3 multipathd: 68:192: mark as failed 
Aug 25 22:58:48 l01ora3 kernel: sd 3:0:1:9: SCSI error: return code = 0x00010000
Aug 25 22:58:48 l01ora3 multipathd: mpathb: remaining active paths: 6 
Aug 25 22:58:48 l01ora3 kernel: end_request: I/O error, dev sdbr, sector 2247
Aug 25 22:58:48 l01ora3 multipathd: 69:128: mark as failed 
Aug 25 22:58:48 l01ora3 kernel: device-mapper: multipath: Failing path 68:80.
Aug 25 22:58:48 l01ora3 multipathd: mpathb: remaining active paths: 5 
Aug 25 22:58:48 l01ora3 kernel: sd 3:0:1:9: SCSI error: return code = 0x00010000
Aug 25 22:58:48 l01ora3 multipathd: 68:160: mark as failed 
Aug 25 22:58:48 l01ora3 kernel: end_request: I/O error, dev sdbr, sector 1032280
Aug 25 22:58:48 l01ora3 multipathd: mpathd: remaining active paths: 7 
Aug 25 22:58:48 l01ora3 kernel: sd 3:0:3:6: SCSI error: return code = 0x00010000
Aug 25 22:58:48 l01ora3 multipathd: 67:224: mark as failed 
Aug 25 22:58:48 l01ora3 kernel: end_request: I/O error, dev sdcm, sector 2367
Aug 25 22:58:48 l01ora3 multipathd: mpathd: remaining active paths: 6 
Aug 25 22:58:48 l01ora3 kernel: device-mapper: multipath: Failing path 69:160.
Aug 25 22:58:48 l01ora3 multipathd: 69:96: mark as failed 
Aug 25 22:58:48 l01ora3 kernel: sd 3:0:1:9: SCSI error: return code = 0x00010000
Aug 25 22:58:48 l01ora3 multipathd: mpathd: remaining active paths: 5 
Aug 25 22:58:48 l01ora3 kernel: end_request: I/O error, dev sdbr, sector 1032792
Aug 25 22:58:49 l01ora3 multipathd: 68:208: mark as failed 
Aug 25 22:58:49 l01ora3 kernel: sd 3:0:3:6: SCSI error: return code = 0x00010000
Aug 25 22:58:49 l01ora3 multipathd: mpatha: remaining active paths: 6 
Aug 25 22:58:49 l01ora3 kernel: end_request: I/O error, dev sdcm, sector 2097296
Aug 25 22:58:49 l01ora3 multipathd: 68:16: mark as failed 
Aug 25 22:58:49 l01ora3 kernel: sd 3:0:3:6: SCSI error: return code = 0x00010000
Aug 25 22:58:49 l01ora3 multipathd: mpatha: remaining active paths: 5 
Aug 25 22:58:49 l01ora3 kernel: end_request: I/O error, dev sdcm, sector 2097808
Aug 25 22:58:49 l01ora3 multipathd: 69:144: mark as failed 
Aug 25 22:58:49 l01ora3 kernel: sd 3:0:2:9: SCSI error: return code = 0x00010000
Aug 25 22:58:49 l01ora3 multipathd: mpatha: remaining active paths: 4 
Aug 25 22:58:49 l01ora3 kernel: end_request: I/O error, dev sdcd, sector 2247
Aug 25 22:58:49 l01ora3 multipathd: 67:128: mark as failed 
Aug 25 22:58:49 l01ora3 kernel: device-mapper: multipath: Failing path 69:16.
Aug 25 22:58:49 l01ora3 multipathd: mpathg: remaining active paths: 7 
Aug 25 22:58:49 l01ora3 kernel: sd 3:0:2:9: SCSI error: return code = 0x00010000
Aug 25 22:58:49 l01ora3 multipathd: 69:192: mark as failed 
Aug 25 22:58:49 l01ora3 kernel: end_request: I/O error, dev sdcd, sector 1032280
Aug 25 22:58:49 l01ora3 multipathd: mpathg: remaining active paths: 6 
Aug 25 22:58:49 l01ora3 kernel: sd 3:0:2:9: SCSI error: return code = 0x00010000
Aug 25 22:58:49 l01ora3 multipathd: 68:64: mark as failed 
Aug 25 22:58:49 l01ora3 kernel: end_request: I/O error, dev sdcd, sector 1032792
Aug 25 22:58:49 l01ora3 multipathd: mpathg: remaining active paths: 5 
Aug 25 22:58:49 l01ora3 kernel: sd 3:0:0:4: SCSI error: return code = 0x00010000
Aug 25 22:58:49 l01ora3 multipathd: 69:0: mark as failed 
Aug 25 22:58:49 l01ora3 kernel: end_request: I/O error, dev sdba, sector 2367
Aug 25 22:58:49 l01ora3 multipathd: mpathg: remaining active paths: 4 
Aug 25 22:58:49 l01ora3 kernel: device-mapper: multipath: Failing path 67:64.
Aug 25 22:58:49 l01ora3 multipathd: 68:128: mark as failed 
Aug 25 22:58:49 l01ora3 kernel: sd 3:0:2:2: SCSI error: return code = 0x00010000
Aug 25 22:58:49 l01ora3 multipathd: mpathk: remaining active paths: 5 
Aug 25 22:58:49 l01ora3 kernel: end_request: I/O error, dev sdbw, sector 506347999
Aug 25 22:58:49 l01ora3 multipathd: 70:0: mark as failed 
Aug 25 22:58:49 l01ora3 kernel: device-mapper: multipath: Failing path 68:160.
Aug 25 22:58:50 l01ora3 multipathd: mpathk: remaining active paths: 4 
Aug 25 22:58:50 l01ora3 kernel: sd 3:0:2:2: SCSI error: return code = 0x00010000
Aug 25 22:58:50 l01ora3 multipathd: 69:176: mark as failed 
Aug 25 22:58:50 l01ora3 kernel: end_request: I/O error, dev sdbw, sector 9429535
Aug 25 22:58:50 l01ora3 multipathd: mpathh: remaining active paths: 4 
Aug 25 22:58:50 l01ora3 kernel: sd 3:0:2:2: SCSI error: return code = 0x00010000
Aug 25 22:58:50 l01ora3 multipathd: dm-13: add map (uevent) 
Aug 25 22:58:50 l01ora3 kernel: end_request: I/O error, dev sdbw, sector 2367
Aug 25 22:58:50 l01ora3 multipathd: dm-13: devmap already registered 
Aug 25 22:58:50 l01ora3 kernel: sd 3:0:2:2: SCSI error: return code = 0x00010000
Aug 25 22:58:50 l01ora3 multipathd: 68:96: mark as failed 
Aug 25 22:58:50 l01ora3 kernel: end_request: I/O error, dev sdbw, sector 9330267
Aug 25 22:58:50 l01ora3 multipathd: mpathj: remaining active paths: 4 
Aug 25 22:58:50 l01ora3 kernel: sd 3:0:1:9: SCSI error: return code = 0x00010000
Aug 25 22:58:50 l01ora3 multipathd: 68:0: mark as failed 
Aug 25 22:58:50 l01ora3 kernel: end_request: I/O error, dev sdbr, sector 282863
Aug 25 22:58:50 l01ora3 multipathd: mpathb: remaining active paths: 4 
Aug 25 22:58:50 l01ora3 kernel: sd 3:0:1:1: SCSI error: return code = 0x00010000
Aug 25 22:58:50 l01ora3 multipathd: dm-6: add map (uevent) 
Aug 25 22:58:50 l01ora3 kernel: end_request: I/O error, dev sdbj, sector 2367
Aug 25 22:58:50 l01ora3 multipathd: dm-6: devmap already registered 
Aug 25 22:58:50 l01ora3 kernel: device-mapper: multipath: Failing path 67:208.
Aug 25 22:58:50 l01ora3 multipathd: dm-9: add map (uevent) 
Aug 25 22:58:50 l01ora3 kernel: sd 3:0:1:3: SCSI error: return code = 0x00010000
Aug 25 22:58:50 l01ora3 multipathd: dm-9: devmap already registered 
Aug 25 22:58:50 l01ora3 kernel: end_request: I/O error, dev sdbl, sector 2367
Aug 25 22:58:50 l01ora3 multipathd: dm-2: add map (uevent) 
Aug 25 22:58:50 l01ora3 kernel: device-mapper: multipath: Failing path 67:240.
Aug 25 22:58:50 l01ora3 multipathd: dm-2: devmap already registered 
Aug 25 22:58:50 l01ora3 kernel: sd 3:0:2:5: SCSI error: return code = 0x00010000
Aug 25 22:58:50 l01ora3 multipathd: dm-7: add map (uevent) 
Aug 25 22:58:50 l01ora3 kernel: end_request: I/O error, dev sdbz, sector 2367
Aug 25 22:58:50 l01ora3 multipathd: dm-7: devmap already registered 
Aug 25 22:58:50 l01ora3 kernel: device-mapper: multipath: Failing path 68:208.
Aug 25 22:58:50 l01ora3 multipathd: dm-10: add map (uevent) 
Aug 25 22:58:50 l01ora3 kernel: sd 3:0:3:8: SCSI error: return code = 0x00010000
Aug 25 22:58:50 l01ora3 multipathd: dm-10: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: end_request: I/O error, dev sdco, sector 2367
Aug 25 22:58:51 l01ora3 multipathd: dm-5: add map (uevent) 
Aug 25 22:58:51 l01ora3 kernel: device-mapper: multipath: Failing path 69:192.
Aug 25 22:58:51 l01ora3 multipathd: dm-5: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: sd 3:0:1:11: SCSI error: return code = 0x00010000
Aug 25 22:58:51 l01ora3 multipathd: dm-4: add map (uevent) 
Aug 25 22:58:51 l01ora3 kernel: end_request: I/O error, dev sdbt, sector 2247
Aug 25 22:58:51 l01ora3 multipathd: dm-4: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: device-mapper: multipath: Failing path 68:112.
Aug 25 22:58:51 l01ora3 multipathd: dm-12: add map (uevent) 
Aug 25 22:58:51 l01ora3 kernel: sd 3:0:2:12: SCSI error: return code = 0x00010000
Aug 25 22:58:51 l01ora3 multipathd: dm-12: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: end_request: I/O error, dev sdcg, sector 2247
Aug 25 22:58:51 l01ora3 multipathd: dm-9: add map (uevent) 
Aug 25 22:58:51 l01ora3 kernel: device-mapper: multipath: Failing path 69:64.
Aug 25 22:58:51 l01ora3 multipathd: dm-9: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: sd 3:0:0:7: SCSI error: return code = 0x00010000
Aug 25 22:58:51 l01ora3 multipathd: dm-7: add map (uevent) 
Aug 25 22:58:51 l01ora3 kernel: end_request: I/O error, dev sdbd, sector 2367
Aug 25 22:58:51 l01ora3 multipathd: dm-7: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: device-mapper: multipath: Failing path 67:112.
Aug 25 22:58:51 l01ora3 multipathd: dm-2: add map (uevent) 
Aug 25 22:58:51 l01ora3 kernel: sd 3:0:0:10: SCSI error: return code = 0x00010000
Aug 25 22:58:51 l01ora3 multipathd: dm-2: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: end_request: I/O error, dev sdbg, sector 2247
Aug 25 22:58:51 l01ora3 multipathd: dm-13: add map (uevent) 
Aug 25 22:58:51 l01ora3 kernel: device-mapper: multipath: Failing path 67:160.
Aug 25 22:58:51 l01ora3 multipathd: dm-13: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: sd 3:0:2:4: SCSI error: return code = 0x00010000
Aug 25 22:58:51 l01ora3 multipathd: dm-6: add map (uevent) 
Aug 25 22:58:51 l01ora3 kernel: end_request: I/O error, dev sdby, sector 2367
Aug 25 22:58:51 l01ora3 multipathd: dm-6: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: device-mapper: multipath: Failing path 68:192.
Aug 25 22:58:51 l01ora3 multipathd: dm-5: add map (uevent) 
Aug 25 22:58:51 l01ora3 kernel: sd 3:0:3:1: SCSI error: return code = 0x00010000
Aug 25 22:58:51 l01ora3 multipathd: dm-5: devmap already registered 
Aug 25 22:58:51 l01ora3 kernel: end_request: I/O error, dev sdch, sector 2367
Aug 25 22:58:51 l01ora3 multipathd: dm-4: add map (uevent) 
Aug 25 22:58:52 l01ora3 kernel: device-mapper: multipath: Failing path 69:80.
Aug 25 22:58:52 l01ora3 multipathd: dm-4: devmap already registered 
Aug 25 22:58:52 l01ora3 kernel: sd 3:0:3:3: SCSI error: return code = 0x00010000
Aug 25 22:58:52 l01ora3 multipathd: dm-10: add map (uevent) 
Aug 25 22:58:52 l01ora3 kernel: end_request: I/O error, dev sdcj, sector 2367
Aug 25 22:58:52 l01ora3 multipathd: dm-10: devmap already registered 
Aug 25 22:58:52 l01ora3 kernel: device-mapper: multipath: Failing path 69:112.
Aug 25 22:58:52 l01ora3 multipathd: dm-2: add map (uevent) 
Aug 25 22:58:52 l01ora3 kernel: sd 3:0:1:5: SCSI error: return code = 0x00010000
Aug 25 22:58:52 l01ora3 multipathd: dm-2: devmap already registered 
Aug 25 22:58:52 l01ora3 kernel: end_request: I/O error, dev sdbn, sector 2367
Aug 25 22:58:52 l01ora3 multipathd: dm-9: add map (uevent) 
Aug 25 22:58:52 l01ora3 kernel: device-mapper: multipath: Failing path 68:16.
Aug 25 22:58:52 l01ora3 multipathd: dm-9: devmap already registered 
Aug 25 22:58:52 l01ora3 kernel: sd 3:0:1:8: SCSI error: return code = 0x00010000
Aug 25 22:58:52 l01ora3 multipathd: dm-13: add map (uevent) 
Aug 25 22:58:52 l01ora3 kernel: end_request: I/O error, dev sdbq, sector 2367
Aug 25 22:58:52 l01ora3 multipathd: dm-13: devmap already registered 
Aug 25 22:58:52 l01ora3 kernel: device-mapper: multipath: Failing path 68:64.
Aug 25 22:58:52 l01ora3 multipathd: dm-7: add map (uevent) 
Aug 25 22:58:52 l01ora3 kernel: sd 3:0:2:11: SCSI error: return code = 0x00010000
Aug 25 22:58:52 l01ora3 multipathd: dm-7: devmap already registered 
Aug 25 22:58:52 l01ora3 kernel: end_request: I/O error, dev sdcf, sector 2247
Aug 25 22:58:52 l01ora3 multipathd: dm-12: add map (uevent) 
Aug 25 22:58:52 l01ora3 kernel: device-mapper: multipath: Failing path 69:48.
Aug 25 22:58:52 l01ora3 multipathd: dm-12: devmap already registered 
Aug 25 22:58:52 l01ora3 kernel: sd 3:0:1:12: SCSI error: return code = 0x00010000
Aug 25 22:58:52 l01ora3 multipathd: dm-6: add map (uevent) 
Aug 25 22:58:52 l01ora3 kernel: end_request: I/O error, dev sdbu, sector 2247
Aug 25 22:58:52 l01ora3 multipathd: dm-6: devmap already registered 
Aug 25 22:58:52 l01ora3 kernel: device-mapper: multipath: Failing path 68:128.
Aug 25 22:58:52 l01ora3 multipathd: dm-5: add map (uevent) 
Aug 25 22:58:52 l01ora3 kernel: sd 3:0:2:7: SCSI error: return code = 0x00010000
Aug 25 22:58:52 l01ora3 multipathd: dm-5: devmap already registered 
Aug 25 22:58:52 l01ora3 kernel: end_request: I/O error, dev sdcb, sector 2367
Aug 25 22:58:53 l01ora3 multipathd: dm-4: add map (uevent) 
Aug 25 22:58:53 l01ora3 kernel: device-mapper: multipath: Failing path 68:240.
Aug 25 22:58:53 l01ora3 multipathd: dm-4: devmap already registered 
Aug 25 22:58:53 l01ora3 kernel: sd 3:0:2:10: SCSI error: return code = 0x00010000
Aug 25 22:58:53 l01ora3 multipathd: dm-10: add map (uevent) 
Aug 25 22:58:53 l01ora3 kernel: end_request: I/O error, dev sdce, sector 2247
Aug 25 22:58:53 l01ora3 multipathd: dm-10: devmap already registered 
Aug 25 22:58:53 l01ora3 kernel: device-mapper: multipath: Failing path 69:32.
Aug 25 22:58:53 l01ora3 multipathd: dm-13: add map (uevent) 
Aug 25 22:58:53 l01ora3 kernel: sd 3:0:1:2: SCSI error: return code = 0x00010000
Aug 25 22:58:53 l01ora3 multipathd: dm-13: devmap already registered 
Aug 25 22:58:53 l01ora3 kernel: end_request: I/O error, dev sdbk, sector 2367
Aug 25 22:58:53 l01ora3 multipathd: dm-7: add map (uevent) 
Aug 25 22:58:53 l01ora3 kernel: device-mapper: multipath: Failing path 67:224.
Aug 25 22:58:53 l01ora3 multipathd: dm-7: devmap already registered 
Aug 25 22:58:53 l01ora3 kernel: sd 3:0:1:2: SCSI error: return code = 0x00010000
Aug 25 22:58:53 l01ora3 multipathd: dm-9: add map (uevent) 
Aug 25 22:58:53 l01ora3 kernel: end_request: I/O error, dev sdbk, sector 9330267
Aug 25 22:58:53 l01ora3 multipathd: dm-9: devmap already registered 
Aug 25 22:58:53 l01ora3 kernel: sd 3:0:1:2: SCSI error: return code = 0x00010000
Aug 25 22:58:53 l01ora3 multipathd: dm-6: add map (uevent) 
Aug 25 22:58:53 l01ora3 kernel: end_request: I/O error, dev sdbk, sector 9429535
Aug 25 22:58:53 l01ora3 multipathd: dm-6: devmap already registered 
Aug 25 22:58:53 l01ora3 kernel: sd 3:0:1:2: SCSI error: return code = 0x00010000
Aug 25 22:58:53 l01ora3 multipathd: dm-12: add map (uevent) 
Aug 25 22:58:53 l01ora3 kernel: end_request: I/O error, dev sdbk, sector 506347999
Aug 25 22:58:53 l01ora3 multipathd: dm-12: devmap already registered 
Aug 25 22:58:53 l01ora3 kernel: sd 3:0:2:6: SCSI error: return code = 0x00010000
Aug 25 22:58:53 l01ora3 multipathd: dm-4: add map (uevent) 
Aug 25 22:58:53 l01ora3 kernel: end_request: I/O error, dev sdca, sector 2080935
Aug 25 22:58:53 l01ora3 multipathd: dm-4: devmap already registered 
Aug 25 22:58:53 l01ora3 kernel: sd 3:0:3:4: SCSI error: return code = 0x00010000
Aug 25 22:58:53 l01ora3 multipathd: dm-10: add map (uevent) 
Aug 25 22:58:53 l01ora3 kernel: end_request: I/O error, dev sdck, sector 2367
Aug 25 22:58:54 l01ora3 multipathd: dm-10: devmap already registered 
Aug 25 22:58:54 l01ora3 kernel: device-mapper: multipath: Failing path 69:128.
Aug 25 22:58:54 l01ora3 multipathd: dm-11: add map (uevent) 
Aug 25 22:58:54 l01ora3 kernel: sd 3:0:2:1: SCSI error: return code = 0x00010000
Aug 25 22:58:54 l01ora3 multipathd: dm-11: devmap already registered 
Aug 25 22:58:54 l01ora3 kernel: end_request: I/O error, dev sdbv, sector 2367
Aug 25 22:58:54 l01ora3 multipathd: dm-8: add map (uevent) 
Aug 25 22:58:54 l01ora3 kernel: device-mapper: multipath: Failing path 68:144.
Aug 25 22:58:54 l01ora3 multipathd: dm-8: devmap already registered 
Aug 25 22:58:54 l01ora3 kernel: sd 3:0:2:3: SCSI error: return code = 0x00010000
Aug 25 22:58:54 l01ora3 multipathd: dm-3: add map (uevent) 
Aug 25 22:58:54 l01ora3 kernel: end_request: I/O error, dev sdbx, sector 2367
Aug 25 22:58:54 l01ora3 multipathd: dm-3: devmap already registered 
Aug 25 22:58:54 l01ora3 kernel: device-mapper: multipath: Failing path 68:176.
Aug 25 22:58:54 l01ora3 multipathd: dm-11: add map (uevent) 
Aug 25 22:58:54 l01ora3 kernel: sd 3:0:3:5: SCSI error: return code = 0x00010000
Aug 25 22:58:54 l01ora3 multipathd: dm-11: devmap already registered 
Aug 25 22:58:54 l01ora3 kernel: end_request: I/O error, dev sdcl, sector 2367
Aug 25 22:58:54 l01ora3 multipathd: dm-3: add map (uevent) 
Aug 25 22:58:54 l01ora3 kernel: device-mapper: multipath: Failing path 69:144.
Aug 25 22:58:54 l01ora3 multipathd: dm-3: devmap already registered 
Aug 25 22:58:54 l01ora3 kernel: sd 3:0:2:8: SCSI error: return code = 0x00010000
Aug 25 22:58:54 l01ora3 multipathd: dm-11: add map (uevent) 
Aug 25 22:58:54 l01ora3 kernel: end_request: I/O error, dev sdcc, sector 2367
Aug 25 22:58:54 l01ora3 multipathd: dm-11: devmap already registered 
Aug 25 22:58:54 l01ora3 kernel: device-mapper: multipath: Failing path 69:0.
Aug 25 22:58:54 l01ora3 multipathd: dm-3: add map (uevent) 
Aug 25 22:58:54 l01ora3 kernel: sd 3:0:3:11: SCSI error: return code = 0x00010000
Aug 25 22:58:54 l01ora3 multipathd: dm-3: devmap already registered 
Aug 25 22:58:54 l01ora3 kernel: end_request: I/O error, dev sdcr, sector 2247
Aug 25 22:58:54 l01ora3 multipathd: dm-11: add map (uevent) 
Aug 25 22:58:55 l01ora3 kernel: device-mapper: multipath: Failing path 69:240.
Aug 25 22:58:55 l01ora3 multipathd: dm-11: devmap already registered 
Aug 25 22:58:55 l01ora3 kernel: sd 3:0:3:12: SCSI error: return code = 0x00010000
Aug 25 22:58:55 l01ora3 multipathd: dm-3: add map (uevent) 
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdcs, sector 2247
Aug 25 22:58:55 l01ora3 multipathd: dm-3: devmap already registered 
Aug 25 22:58:55 l01ora3 kernel: device-mapper: multipath: Failing path 70:0.
Aug 25 22:58:55 l01ora3 multipathd: dm-2: add map (uevent) 
Aug 25 22:58:55 l01ora3 kernel: sd 3:0:3:7: SCSI error: return code = 0x00010000
Aug 25 22:58:55 l01ora3 multipathd: dm-2: devmap already registered 
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdcn, sector 2367
Aug 25 22:58:55 l01ora3 multipathd: dm-8: add map (uevent) 
Aug 25 22:58:55 l01ora3 kernel: device-mapper: multipath: Failing path 69:176.
Aug 25 22:58:55 l01ora3 multipathd: dm-8: devmap already registered 
Aug 25 22:58:55 l01ora3 kernel: sd 3:0:1:10: SCSI error: return code = 0x00010000
Aug 25 22:58:55 l01ora3 multipathd: dm-8: add map (uevent) 
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdbs, sector 2247
Aug 25 22:58:55 l01ora3 multipathd: dm-8: devmap already registered 
Aug 25 22:58:55 l01ora3 kernel: device-mapper: multipath: Failing path 68:96.
Aug 25 22:58:55 l01ora3 multipathd: mpathg: sdbe - tur checker reports path is down 
Aug 25 22:58:55 l01ora3 kernel: sd 3:0:1:2: SCSI error: return code = 0x00010000
Aug 25 22:58:55 l01ora3 multipathd: mpathc: sdaz - tur checker reports path is down 
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdbk, sector 5103799
Aug 25 22:58:55 l01ora3 multipathd: mpathe: sdax - tur checker reports path is down 
Aug 25 22:58:55 l01ora3 kernel: sd 3:0:1:2: SCSI error: return code = 0x00010000
Aug 25 22:58:55 l01ora3 multipathd: mpathi: sdbf - tur checker reports path is down 
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdbk, sector 6193375
Aug 25 22:58:55 l01ora3 multipathd: checker failed path 67:144 in map mpathi 
Aug 25 22:58:55 l01ora3 kernel: sd 3:0:3:2: SCSI error: return code = 0x00010000
Aug 25 22:58:55 l01ora3 multipathd: mpathi: remaining active paths: 4 
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdci, sector 2367
Aug 25 22:58:55 l01ora3 multipathd: mpathe: sdbj - tur checker reports path is down 
Aug 25 22:58:55 l01ora3 kernel: device-mapper: multipath: Failing path 69:96.
Aug 25 22:58:55 l01ora3 multipathd: checker failed path 67:208 in map mpathe 
Aug 25 22:58:55 l01ora3 kernel: sd 3:0:3:2: SCSI error: return code = 0x00010000
Aug 25 22:58:55 l01ora3 multipathd: mpathe: remaining active paths: 6 
Aug 25 22:58:56 l01ora3 kernel: end_request: I/O error, dev sdci, sector 9330267
Aug 25 22:58:56 l01ora3 multipathd: mpathh: sdbp - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 kernel: sd 3:0:3:2: SCSI error: return code = 0x00010000
Aug 25 22:58:56 l01ora3 multipathd: mpathf: sdbo - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 kernel: end_request: I/O error, dev sdci, sector 9429535
Aug 25 22:58:56 l01ora3 multipathd: checker failed path 68:32 in map mpathf 
Aug 25 22:58:56 l01ora3 kernel: sd 3:0:3:2: SCSI error: return code = 0x00010000
Aug 25 22:58:56 l01ora3 multipathd: mpathf: remaining active paths: 4 
Aug 25 22:58:56 l01ora3 kernel: end_request: I/O error, dev sdci, sector 506347999
Aug 25 22:58:56 l01ora3 multipathd: mpathf: sdbc - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 kernel: sd 3:0:1:4: SCSI error: return code = 0x00010000
Aug 25 22:58:56 l01ora3 multipathd: mpathk: sdbi - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 kernel: end_request: I/O error, dev sdbm, sector 2367
Aug 25 22:58:56 l01ora3 multipathd: mpathh: sdbd - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 kernel: device-mapper: multipath: Failing path 68:0.
Aug 25 22:58:56 l01ora3 multipathd: mpathd: sday - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 kernel: device-mapper: multipath: Failing path 67:144.
Aug 25 22:58:56 l01ora3 multipathd: checker failed path 67:32 in map mpathd 
Aug 25 22:58:56 l01ora3 kernel: device-mapper: multipath: Failing path 68:32.
Aug 25 22:58:56 l01ora3 multipathd: mpathd: remaining active paths: 4 
Aug 25 22:58:56 l01ora3 kernel: device-mapper: multipath: Failing path 67:32.
Aug 25 22:58:56 l01ora3 multipathd: mpathc: sdbl - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: checker failed path 67:240 in map mpathc 
Aug 25 22:58:56 l01ora3 multipathd: mpathc: remaining active paths: 6 
Aug 25 22:58:56 l01ora3 multipathd: mpatha: sdbb - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: mpathj: sdcq - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: mpathd: sdbw - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: mpathb: sdba - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: mpathj: sdbg - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: mpathh: sdcb - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: mpathl: sdbh - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: mpathd: sdbk - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: mpathj: sdce - tur checker reports path is down 
Aug 25 22:58:56 l01ora3 multipathd: mpatha: sdbz - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathf: sdca - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathk: sdcg - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpatha: sdbn - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathl: sdbt - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: checker failed path 68:112 in map mpathl 
Aug 25 22:58:57 l01ora3 multipathd: mpathl: remaining active paths: 6 
Aug 25 22:58:57 l01ora3 multipathd: mpathj: sdbs - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathi: sdcp - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathb: sdby - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathc: sdcj - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: checker failed path 69:112 in map mpathc 
Aug 25 22:58:57 l01ora3 multipathd: mpathc: remaining active paths: 5 
Aug 25 22:58:57 l01ora3 multipathd: mpathi: sdbr - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpatha: sdcl - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathh: sdcn - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathb: sdck - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathd: sdci - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathl: sdcf - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: checker failed path 69:48 in map mpathl 
Aug 25 22:58:57 l01ora3 multipathd: mpathl: remaining active paths: 5 
Aug 25 22:58:57 l01ora3 multipathd: mpathk: sdbu - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathg: sdco - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathi: sdcd - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathe: sdch - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: checker failed path 69:80 in map mpathe 
Aug 25 22:58:57 l01ora3 multipathd: mpathe: remaining active paths: 5 
Aug 25 22:58:57 l01ora3 multipathd: mpathg: sdbq - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathe: sdbv - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: checker failed path 68:144 in map mpathe 
Aug 25 22:58:57 l01ora3 multipathd: mpathe: remaining active paths: 4 
Aug 25 22:58:57 l01ora3 multipathd: mpathl: sdcr - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: checker failed path 69:240 in map mpathl 
Aug 25 22:58:57 l01ora3 multipathd: mpathl: remaining active paths: 4 
Aug 25 22:58:57 l01ora3 multipathd: mpathg: sdcc - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathf: sdcm - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathk: sdcs - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathb: sdbm - tur checker reports path is down 
Aug 25 22:58:57 l01ora3 multipathd: mpathc: sdbx - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: checker failed path 68:176 in map mpathc 
Aug 25 22:58:58 l01ora3 multipathd: mpathc: remaining active paths: 4 
Aug 25 22:58:58 l01ora3 multipathd: dm-8: add map (uevent) 
Aug 25 22:58:58 l01ora3 multipathd: dm-8: devmap already registered 
Aug 25 22:58:58 l01ora3 multipathd: dm-12: add map (uevent) 
Aug 25 22:58:58 l01ora3 multipathd: dm-12: devmap already registered 
Aug 25 22:58:58 l01ora3 multipathd: dm-5: add map (uevent) 
Aug 25 22:58:58 l01ora3 multipathd: dm-5: devmap already registered 
Aug 25 22:58:58 l01ora3 multipathd: mpathg: sdbe - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathc: sdaz - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathe: sdax - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathi: sdbf - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathe: sdbj - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathh: sdbp - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathf: sdbo - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathf: sdbc - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathk: sdbi - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathh: sdbd - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathd: sday - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathc: sdbl - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpatha: sdbb - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathj: sdcq - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathd: sdbw - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathb: sdba - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathj: sdbg - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathh: sdcb - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathl: sdbh - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathd: sdbk - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathj: sdce - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpatha: sdbz - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathf: sdca - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathk: sdcg - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpatha: sdbn - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathl: sdbt - tur checker reports path is down 
Aug 25 22:58:58 l01ora3 multipathd: mpathj: sdbs - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathi: sdcp - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathb: sdby - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathc: sdcj - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathi: sdbr - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpatha: sdcl - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathh: sdcn - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathb: sdck - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathd: sdci - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathl: sdcf - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathk: sdbu - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathg: sdco - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathi: sdcd - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathe: sdch - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathg: sdbq - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathe: sdbv - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathl: sdcr - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathg: sdcc - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathf: sdcm - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathk: sdcs - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathb: sdbm - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathc: sdbx - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathg: sdbe - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathc: sdaz - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathe: sdax - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathi: sdbf - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathe: sdbj - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathh: sdbp - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathf: sdbo - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathf: sdbc - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathk: sdbi - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathh: sdbd - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathd: sday - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathc: sdbl - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpatha: sdbb - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathj: sdcq - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathd: sdbw - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathb: sdba - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathj: sdbg - tur checker reports path is down 
Aug 25 22:58:59 l01ora3 multipathd: mpathh: sdcb - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathl: sdbh - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathd: sdbk - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathj: sdce - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpatha: sdbz - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathf: sdca - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathk: sdcg - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpatha: sdbn - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathl: sdbt - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathj: sdbs - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathi: sdcp - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathb: sdby - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathc: sdcj - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathi: sdbr - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpatha: sdcl - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathh: sdcn - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathb: sdck - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathd: sdci - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathl: sdcf - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathk: sdbu - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathg: sdco - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathi: sdcd - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathe: sdch - tur checker reports path is down 
Aug 25 22:59:00 l01ora3 multipathd: mpathg: sdbq - tur checker reports path is down 
D'Hooge Freek | 26 Aug 17:07 2011
Picon

RE: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Marco,

Your system is complaining about lost disks and io paths.
I also see that your multipathing is configured so that it will wait when all paths to a disk are lost
(queue_if_no_path). When that happens, the clusterware will start reporting unaccessible voting
disks, while the other processes will (appear to) hang.

Can you check if you still get errors like: "tur checker reports path is down" or "kernel: end_request: I/O error,...".
If not, check if they start to appear when you put some load on the io subsystem.

Regards,

Freek D'Hooge
Uptime
Oracle Database Administrator
email: freek.dhooge@...
tel +32(0)3 451 23 82
http://www.uptime.be
disclaimer: www.uptime.be/disclaimer
---
From: oracle-l-bounce@...
[mailto:oracle-l-bounce@...] On Behalf Of Marko Sutic
Sent: vrijdag 26 augustus 2011 10:34
To: David Barbour
Cc: oracle-l@...
Subject: Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Hi David,

/var/log/messages is stuffed with various messages and I cannot identify what is important to look for.

I will attach excerpt of log file from the period during import and when failure occurred.

If you notice something odd please let me know.

Regards,
Marko

On Fri, Aug 26, 2011 at 12:14 AM, David Barbour
<david.barbour1@...> wrote:
Anything in /var/log/messages?  

On Thu, Aug 25, 2011 at 5:42 AM, Marko Sutic <marko.sutic@...> wrote:
Freek,

you are correct - heartbeat fatal messages are there due to the missing voting disk.

I have another database up and running on second node and this database is using same ocfs2 volume for Oracle
database files as the first one.
This database is running without any error so I suppose that other OCFS2 volumes were accessible in the
time of the failure.

In this configuration are 3 voting disk files located on 3 different luns and separate OCFS2 volumes. When
failure occurs two of three voting devices hang.

It is also worth to mention that nothing else is running on that node except import.

I simply can't figure out why two of three voting disks hang.

Regards,
Marko

On Thu, Aug 25, 2011 at 11:08 AM, D'Hooge Freek
<Freek.DHooge@...> wrote:
Marco,

I don't know the error timings for the other node, but I think the heartbeat fatal messages are coming after
the first node has terminated due to the missing voting disk.

This would indicate that there is no general problem with the voting disk itself, but that the problem is
specific to the first node.
Either the connection itself or the load or an ocfs2 bug would then be the cause of the error.

Do you know if at the time of the failure the other OCFS2 volumes where still accessible?
Are your voting disks placed on the same luns as your database files or are they on a separate ocfs2 volume?

Regards,

Freek D'Hooge
Uptime
Oracle Database Administrator
email: freek.dhooge@...
tel +32(0)3 451 23 82
http://www.uptime.be
disclaimer: www.uptime.be/disclaimer
---
From: Marko Sutic [mailto:marko.sutic@...]
Sent: donderdag 25 augustus 2011 10:51
To: D'Hooge Freek
Cc: oracle-l@...
Subject: Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Errors messages from another node:

2011-08-25 10:38:33.563
[cssd(18117)]CRS-1612:node l01ora3 (1) at 50% heartbeat fatal, eviction in 14.000 seconds
2011-08-25 10:38:40.558
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 7.010 seconds
2011-08-25 10:38:41.560
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 6.010 seconds
2011-08-25 10:38:45.558
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 2.010 seconds
2011-08-25 10:38:46.560
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 1.010 seconds
2011-08-25 10:38:47.562
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 0.010 seconds
2011-08-25 10:38:47.574
[cssd(18117)]CRS-1607:CSSD evicting node l01ora3. Details in /u01/app/crs/log/l01ora4/cssd/ocssd.log.
2011-08-25 10:39:01.579
[cssd(18117)]CRS-1601:CSSD Reconfiguration complete. Active nodes are l01ora4 .

Regards,
Marko

--
http://www.freelists.org/webpage/oracle-l

Marko Sutic | 26 Aug 18:05 2011
Picon

Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Freek,


there were lots of "tur checker reports path is down" errors.

Also when I filtered /var/log/messages for "I/O error" there were lots of messages like:
...
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdcn, sector 2367
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdbs, sector 2247
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdbk, sector 5103799
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdbk, sector 6193375
Aug 25 22:58:55 l01ora3 kernel: end_request: I/O error, dev sdci, sector 2367
Aug 25 22:58:56 l01ora3 kernel: end_request: I/O error, dev sdci, sector 9330267
...

Sysadmins replaced FC switch and rebooted nodes.
It seems that not all paths were activated or something like that. (I will ask detailed answer next week)

After that action this errors disappeared and now import has not failed after hour/two (it is still active).

Also I've collected OS statistics and asked Oracle support to help me.



Answer from Oracle support:

----------------
Please find my analysis below.



zzz ***Thu Aug 25 22:09:28 CEST 2011
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
6 7 0 59341556 164072 5250912 0 0 107 20 2 32 0 0 99 1 0
0 7 0 59338752 164072 5250904 0 0 0 0 1017 5378 0 0 87 12 0
0 7 0 59338628 164072 5250904 0 0 0 8 1016 5168 0 0 87 13 0

++ From above we can see there might be some disk problems because b>r

++ bi, bo are having high values indicates that there is a problem with I/O or storage.

The vmstat "b" column

Any count in the "b" column of vmstat is indicative
of threads blocked via sema_p in the biowait state.


This indicates that an I/O call has been made and that Solaris
is waiting on a return response.

Note: I/O can be blocked prior to waiting on a return response.
In particular, if the number of commands active on
a device is greater than or equal to the value of
sd_max_throttle, all threads requesting I/O through
that device will block prior to waiting for the I/O.
This will not be reflected in the "b" column count.

++ No heavey CPU usage

++ Nothing wrong i can find in top command

++ sda, sda2 devices have heavry writes for seconds.


Please check with your system admin why I/O issues are there in the system. If you would like to raise a new SR with storage regarding this, that s fine.
----------------



My current vmstat statistics are:

....
zzz ***Fri Aug 26 14:18:39 CEST 2011
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 6  5      0 59010308 222532 5545300    0    0   436    90   23   24  1  0 98  2  0
 0  5      0 59008336 222532 5545332    0    0    17   468 1077 4482  0  0 93  6  0
 0  5      0 59008088 222532 5545356    0    0     2     1 1019 4189  0  0 94  6  0
zzz ***Fri Aug 26 14:18:49 CEST 2011
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 5  5      0 59010852 222540 5545516    0    0   436    90   23   24  1  0 98  2  0
 0  5      0 59007824 222544 5545532    0    0     1   369 1132 3625  0  0 93  6  0
 0  5      0 59007576 222544 5545536    0    0     2     1 1021 3219  0  0 94  6  0
zzz ***Fri Aug 26 14:18:59 CEST 2011
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 5  5      0 59009780 222548 5545652    0    0   436    90   23   24  1  0 98  2  0
 0  5      0 59006268 222548 5545640    0    0     0   449 1034 4368  0  0 93  6  0
 0  5      0 59006376 222548 5545640    0    0    19    50 1035 3300  0  0 94  6  0
zzz ***Fri Aug 26 14:19:09 CEST 2011
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 5  6      0 59000056 222580 5545700    0    0   436    90   23   24  1  0 98  2  0
 0  6      0 59001064 222580 5545720    0    0     0   364 1040 5069  0  0 93  6  0
 0  6      0 59006064 222584 5545676    0    0     1   100 1048 4974  0  0 94  6  0
zzz ***Fri Aug 26 14:19:19 CEST 2011
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 5  0      0 59010232 222668 5545780    0    0   436    90   23   24  1  0 98  2  0
 0  7      0 59005660 222672 5545932    0    0 14337   197 1233 3823  2  1 89  8  0
 1  6      0 59005536 222680 5545984    0    0    28   470 1038 3290  0  0 87 13  0
zzz ***Fri Aug 26 14:19:29 CEST 2011
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 6  6      0 58998056 222680 5546032    0    0   436    90   23   24  1  0 98  2  0
 0  7      0 58995104 222680 5545996    0    0    16   216 1030 4447  0  0 87 13  0
 0  6      0 58995488 222680 5546056    0    0     3   469 1046 4248  0  0 87 13  0
....


I hope that this is OK now.



Thank you for your help.

Regards,
Marko




On Fri, Aug 26, 2011 at 5:07 PM, D'Hooge Freek <Freek.DHooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org> wrote:
Marco,

Your system is complaining about lost disks and io paths.
I also see that your multipathing is configured so that it will wait when all paths to a disk are lost (queue_if_no_path). When that happens, the clusterware will start reporting unaccessible voting disks, while the other processes will (appear to) hang.

Can you check if you still get errors like: "tur checker reports path is down" or "kernel: end_request: I/O error,...".
If not, check if they start to appear when you put some load on the io subsystem.


Regards,


Freek D'Hooge
Uptime
Oracle Database Administrator
email: freek.dhooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org
tel +32(0)3 451 23 82
http://www.uptime.be
disclaimer: www.uptime.be/disclaimer
---
Sent: vrijdag 26 augustus 2011 10:34
To: David Barbour
Cc: oracle-l-uGLqWuYN4qMgsBAKwltoeQ@public.gmane.org
Subject: Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Hi David,

/var/log/messages is stuffed with various messages and I cannot identify what is important to look for.

I will attach excerpt of log file from the period during import and when failure occurred.

If you notice something odd please let me know.

Regards,
Marko


On Fri, Aug 26, 2011 at 12:14 AM, David Barbour <david.barbour1-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Anything in /var/log/messages? 

On Thu, Aug 25, 2011 at 5:42 AM, Marko Sutic <marko.sutic-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Freek,

you are correct - heartbeat fatal messages are there due to the missing voting disk.

I have another database up and running on second node and this database is using same ocfs2 volume for Oracle database files as the first one.
This database is running without any error so I suppose that other OCFS2 volumes were accessible in the time of the failure.

In this configuration are 3 voting disk files located on 3 different luns and separate OCFS2 volumes. When failure occurs two of three voting devices hang.

It is also worth to mention that nothing else is running on that node except import.


I simply can't figure out why two of three voting disks hang.


Regards,
Marko


On Thu, Aug 25, 2011 at 11:08 AM, D'Hooge Freek <Freek.DHooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org> wrote:
Marco,

I don't know the error timings for the other node, but I think the heartbeat fatal messages are coming after the first node has terminated due to the missing voting disk.

This would indicate that there is no general problem with the voting disk itself, but that the problem is specific to the first node.
Either the connection itself or the load or an ocfs2 bug would then be the cause of the error.

Do you know if at the time of the failure the other OCFS2 volumes where still accessible?
Are your voting disks placed on the same luns as your database files or are they on a separate ocfs2 volume?

Regards,


Freek D'Hooge
Uptime
Oracle Database Administrator
email: freek.dhooge-2tvnBG85piuzQB+pC5nmwQ@public.gmane.org
tel +32(0)3 451 23 82
http://www.uptime.be
disclaimer: www.uptime.be/disclaimer
---
From: Marko Sutic [mailto:marko.sutic-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org]
Sent: donderdag 25 augustus 2011 10:51
To: D'Hooge Freek
Cc: oracle-l-uGLqWuYN4qMgsBAKwltoeQ@public.gmane.org
Subject: Re: CRS-1615:voting device hang at 50% fatal, termination in 99620 ms

Errors messages from another node:

2011-08-25 10:38:33.563
[cssd(18117)]CRS-1612:node l01ora3 (1) at 50% heartbeat fatal, eviction in 14.000 seconds
2011-08-25 10:38:40.558
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 7.010 seconds
2011-08-25 10:38:41.560
[cssd(18117)]CRS-1611:node l01ora3 (1) at 75% heartbeat fatal, eviction in 6.010 seconds
2011-08-25 10:38:45.558
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 2.010 seconds
2011-08-25 10:38:46.560
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 1.010 seconds
2011-08-25 10:38:47.562
[cssd(18117)]CRS-1610:node l01ora3 (1) at 90% heartbeat fatal, eviction in 0.010 seconds
2011-08-25 10:38:47.574
[cssd(18117)]CRS-1607:CSSD evicting node l01ora3. Details in /u01/app/crs/log/l01ora4/cssd/ocssd.log.
2011-08-25 10:39:01.579
[cssd(18117)]CRS-1601:CSSD Reconfiguration complete. Active nodes are l01ora4 .


Regards,
Marko








Gmane