swap_handler and error_logger_file_h

When application has sasl_error_file_h handler installed it is possible
to do
gen_event:swap_handler(error_logger, {sasl_report_file_h, swap},{sasl_report_file_h, ["/tmp/foo.log",error]}).
However when I try to do:
gen_event:swap_handler(error_logger, {error_logger_file_h, swap},{error_logger_file_h, "/tmp/foo.log"}).
I get
{error,{error,einval}}.

I do not want to use error_logger:swap_handler({logfile, "/tmp/bar.log"})
because that doesn't delete the old handler (see gen_event:which_handlers(error_logger) )
and I end up having two error_logger_file_h handlers.
Any ideas?

Thanks,
Hubert