Re: FastRouter WRR and overloaded nodes

> I had an DB issue with one of my apps than caused workers to get
> blocked for many seconds during request handling, cheaper spawned new
> workers, they also get blocked, backlog filled with requests and whole
> node was overloaded. At the end node was marked with death_mark,
> harakiri kicks in and with no new requests it get's healthy again.
>
> 1) FastRouter doesn't seem to use ref/load fields when choosing node
> to forward request to, only wrr field

that's true, ref and load are used only for memory accounting and monitoring.

> OR
> 2) it uses those field but in my case it didin't made any difference
> (but looking at the code I only see wrr and death_mark checks).
>
> If 1) is true could FastRouter be improved to
> a) skip nodes with high load (can't remeber what this field is, backlog
> usage?)
> b) prefer nodes with lower ref, maybe skip node if ref > 0 and there
> is a node with ref == 0 (?)

This would mean not using weighted round-robin, but weighted
least-connection. In the BlastBeat project (the current private
development branch includes uwsgi subscription support) i am using it
(least-connection) and in my tests it result as a better compromise during
spikes.

> c) maybe just do wrr-- and go to next node if current has ref > 0
> and/or load > 0 (?) - but what if all are loaded? hammering the last