Discussing the Gmane hierarchy

Lars Magne Ingebrigtsen | 15 Aug 2012 15:27

Face

Picon

News server switcharoo

Most of the data has now been synced over, so I'm ready to attempt
switching news.gmane.org from dough to ger tonight.

This will probably entail a bit of downtime, but hopefully not a lot.

Mail-to-news handling will probably be halted for at least a few hours,
though.  But the queue should flush out pretty quickly after the new
server is up.

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Lars Magne Ingebrigtsen | 15 Aug 2012 22:53

Face

Picon

Re: News server switcharoo

Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

> Most of the data has now been synced over, so I'm ready to attempt
> switching news.gmane.org from dough to ger tonight.

The servers are now switched, but the DNS probably hasn't propagated
evenly yet.

Things seem to basically work, though...

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Lars Magne Ingebrigtsen | 15 Aug 2012 23:24

Face

Picon

Re: News server switcharoo

Flushing the incoming spool is now complete, so we should be up-to-data
again now.  The web interface has a bit to catch up, though.

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Lars Magne Ingebrigtsen | 15 Aug 2012 23:43

Face

Picon

Re: News server switcharoo

Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

> Flushing the incoming spool is now complete, so we should be up-to-data
> again now.  The web interface has a bit to catch up, though.

That went faster than expected.  It looks like it's caught up already.

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Lars Magne Ingebrigtsen | 15 Aug 2012 23:47

Face

Picon

Re: News server switcharoo

The load went from 12-18 on the old server to 0.2 on the new server.

Hey!  Faster disks helps!

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Jeffrey Scofield | 16 Aug 2012 06:38

Re: News server switcharoo

Lars,

Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

> The load went from 12-18 on the old server to 0.2 on the new server.
>
> Hey!  Faster disks helps!

Thanks very much for keeping Gmane running.  It is super awesome.

Jeffrey

Ted Zlatanov | 16 Aug 2012 17:13

X-Face

Re: News server switcharoo

On Wed, 15 Aug 2012 23:47:14 +0200 Lars Magne Ingebrigtsen <larsi <at> gnus.org> wrote: 

LMI> The load went from 12-18 on the old server to 0.2 on the new server.
LMI> Hey!  Faster disks helps!

Either that, or things are broken ;)

Ted

Lars Magne Ingebrigtsen | 17 Aug 2012 16:48

Face

Picon

Re: News server switcharoo

Ted Zlatanov <tzz <at> lifelogs.com> writes:

> LMI> The load went from 12-18 on the old server to 0.2 on the new server.
> LMI> Hey!  Faster disks helps!
>
> Either that, or things are broken ;)

Heh heh.

Creating new groups didn't work due to me forgetting to copy over the
.gpg directory, but otherwise things seem to be chugging along.

SSDs are nice.  I've been thinking a bit about how I want to implement
the Gwene web interface, and since some kind gentleman donated another
SSD, I've been thinking about not having stuff in memory, but just
maintaining NOV-like files on disk instead.  That is, whenever a new
message arrives, there's a process that re-creates all (or mostly all)
of a file that represents the view of the group in sorted threadly
fashion.

So that when a web user requests page 7 of the group, it'll more or less
just spew out the stuff based on lines (* 7 page-size) to (* 8 page-size).

I think it should be feasible for Gwene on an SSD.

ger:/tmp# du -sh /var/spool/news/overview/
49G     /var/spool/news/overview/

See?  It's nothing.

(Continue reading)

Adam Sjøgren | 19 Aug 2012 01:47

X-Face

Picon

Re: News server switcharoo

On Fri, 17 Aug 2012 16:48:50 +0200, Lars wrote:

> SSDs are nice.

What setup did you end up using, by the way? RAID controller?
RAID-level/configuration? Filesystem(s)?

  Just curious,

    Adam

--

-- 
 "A cat has nine lives, but a bullfrog croaks                 Adam Sjøgren
  every day."                                            asjo <at> koldfront.dk

Lars Magne Ingebrigtsen | 19 Aug 2012 01:56

Face

Picon

Re: News server switcharoo

asjo <at> koldfront.dk (Adam Sjøgren) writes:

> What setup did you end up using, by the way? RAID controller?
> RAID-level/configuration? Filesystem(s)?

I ended up using the Megaraid SATA3 card in RAID5 with five SSDs.   The
file system is ext4.

The load is still 0.2, and some things, like importing mail archives, is
way faster then it used to be.

So it's going quite well, so far.  The next interesting point will be
what happens when one of the SSDs break down.  SSDs have different
failure modes than spinning disks.  Spinning disk usually start
complaining slowly, growing ever more dead.  SSDs have a tendency to
just die, totally and utterly.  When that happens, the RAID card will
hopefully handle that well, but we'll see...

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Adam Sjøgren | 19 Aug 2012 17:30

X-Face

Picon

Re: News server switcharoo

On Sun, 19 Aug 2012 01:56:01 +0200, Lars wrote:

> I ended up using the Megaraid SATA3 card in RAID5 with five SSDs.   The
> file system is ext4.

Ok. I was thinking of moving to SSDs in my home-server, and I'd probably
have to go RAID5 to get enough space/price...

> The load is still 0.2, and some things, like importing mail archives, is
> way faster then it used to be.

Nice.

> So it's going quite well, so far.  The next interesting point will be
> what happens when one of the SSDs break down.  SSDs have different
> failure modes than spinning disks.  Spinning disk usually start
> complaining slowly, growing ever more dead.  SSDs have a tendency to
> just die, totally and utterly.

Yeah, the SSD in my laptop stopped accepting writes and 3 minutes later
it stopped showing up as a SATA device after boot... Stone dead.

> When that happens, the RAID card will hopefully handle that well, but
> we'll see...

It handled yanking a disk as expected, I hope? 

  Best regards,

    Adam

(Continue reading)

Lars Magne Ingebrigtsen | 19 Aug 2012 18:25

Face

Picon

Re: News server switcharoo

asjo <at> koldfront.dk (Adam Sjøgren) writes:

>> When that happens, the RAID card will hopefully handle that well, but
>> we'll see...
>
> It handled yanking a disk as expected, I hope? 

Yes.  

But I think that's a failure mode most controllers handle well, because
it's so easy to test.  However, when the SATA thing is still plugged in,
but responds with nonsense, that's more difficult to test.  

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Adam Sjøgren | 19 Aug 2012 18:28

X-Face

Picon

Re: News server switcharoo

On Sun, 19 Aug 2012 18:25:02 +0200, Lars wrote:

> But I think that's a failure mode most controllers handle well, because
> it's so easy to test.  However, when the SATA thing is still plugged in,
> but responds with nonsense, that's more difficult to test.  

We shall see - hopefully quite a while out in the future 

  Best regards,

    Adam

--

-- 
 "A cat has nine lives, but a bullfrog croaks                 Adam Sjøgren
  every day."                                            asjo <at> koldfront.dk

Duncan | 19 Aug 2012 23:03

Picon

Re: News server switcharoo

Lars Magne Ingebrigtsen posted on Sun, 19 Aug 2012 18:25:02 +0200 as
excerpted:

> asjo <at> koldfront.dk (Adam Sjøgren) writes:
> 
>>> When that happens, the RAID card will hopefully handle that well, but
>>> we'll see...
>>
>> It handled yanking a disk as expected, I hope? 
> 
> Yes.  
> 
> But I think that's a failure mode most controllers handle well, because
> it's so easy to test.  However, when the SATA thing is still plugged in,
> but responds with nonsense, that's more difficult to test.

That's one of the good things about both md and dm raid in Linux.  
There's a fault injector available to test just that sort of thing, 
spewing nonsense! =:^)

Meanwhile, I've never really tried raid-5 here.  I did run raid-6 for 
awhile, then switched to raid-1, but I never did trust the single-
redundancy of raid-5.  It always seemed to me that if I were going to 
bother with anything beyond dual-disk raid-1, a second redundancy was 
next up the list, either in triple-way raid-1, or 4-device raid-6.  I've 
wanted to try raid-10 too, but again, don't want to change it with four 
disks so would go dual-redundancy raid-10 with six devices, and I've just 
never managed the necessary budget priority for it, here.

The other thing about raid-5 on SSDs, is that they tend to be faster at

(Continue reading)

Ted Zlatanov | 20 Aug 2012 05:47

X-Face

Re: News server switcharoo

On Sun, 19 Aug 2012 01:56:01 +0200 Lars Magne Ingebrigtsen <larsi <at> gnus.org> wrote: 

LMI> SSDs have a tendency to just die, totally and utterly.  When that
LMI> happens, the RAID card will hopefully handle that well, but we'll
LMI> see...

What exact make and model SSDs are you using, so any interested parties
know what to donate?

Ted

Lars Magne Ingebrigtsen | 1 Sep 2012 14:56

Face

Picon

Re: News server switcharoo

Ted Zlatanov <tzz <at> lifelogs.com> writes:

> What exact make and model SSDs are you using, so any interested parties
> know what to donate?

I'm using Samsung 830 512GB SSDs in the main RAID.

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Return

Return to gmane.discuss.

Project Web Page

Discussing the Gmane hierarchy

Search Archive

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane