headers
Jim J | 20 May 2012 20:24
Picon

Fetchmail, gmail, and a new computer

I'm using fetchmail 6.3.18 (from Ubuntu Oneiric) to pull email from gmail
using POP3. It works fine. The first time I ran it it downloaded all of my
email. Every time I've run it since then it's downloaded only new email.
Gmail is configured to archive the email after it's been accessed via POP3.

Now I have a new computer (fetchmail 6.3.21, Ubuntu Precise). I would like
my new computer to pick up where the old left off. Fetchmail doesn't seem
to want to do this by default. Instead, it wants to download my entire
Gmail account. How do I make my new installation pick up where the other
one left off? Since fetchmail didn't download everything every time, even
after a reboot, it seems logical that there's a file somewhere I could copy
over, but I can't find one.

Thanks,

Jim
Permalink | Reply |
headers
Carlos E. R. | 20 May 2012 21:08
Favicon

Re: Fetchmail, gmail, and a new computer


On 2012-05-20 20:24, Jim J wrote:

> Now I have a new computer (fetchmail 6.3.21, Ubuntu Precise). I would
> like my new computer to pick up where the old left off. Fetchmail
> doesn't seem to want to do this by default. Instead, it wants to
> download my entire Gmail account. How do I make my new installation
> pick up where the other one left off? Since fetchmail didn't download
> everything every time, even after a reboot, it seems logical that
> there's a file somewhere I could copy over, but I can't find one.

.fetchids

Are you using pop or imap? gmail has both, and imap works better.

--

-- 
Cheers / Saludos,

		Carlos E. R.
		(from 11.4 x86_64 "Celadon" at Telcontar)
Permalink | Reply |
headers
Jim J | 20 May 2012 21:22
Picon

Fetchmail, gmail, and a new computer

Thanks. I've actually been looking for that file. I can't find it. I'm
running fetchmail as a daemon from Ubuntu's init script. From what I can
tell it should be in /var/lib/fetchmail but there are no files in there
(using sudo ls -al).

I'm using pop. Honestly I don't remember exactly why just that I couldn't
get IMAP to work.

Thanks,

Jim

On Sunday, May 20, 2012, Carlos E. R. wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 2012-05-20 20:24, Jim J wrote:
>
> > Now I have a new computer (fetchmail 6.3.21, Ubuntu Precise). I would
> > like my new computer to pick up where the old left off. Fetchmail
> > doesn't seem to want to do this by default. Instead, it wants to
> > download my entire Gmail account. How do I make my new installation
> > pick up where the other one left off? Since fetchmail didn't download
> > everything every time, even after a reboot, it seems logical that
> > there's a file somewhere I could copy over, but I can't find one.
>
> .fetchids
>
>
(Continue reading)

Permalink | Reply |
headers
Rob MacGregor | 20 May 2012 21:44
Picon
Gravatar

Re: Fetchmail, gmail, and a new computer

On Sun, May 20, 2012 at 8:22 PM, Jim J <jahosefet <at> gmail.com> wrote:
> Thanks. I've actually been looking for that file. I can't find it. I'm
> running fetchmail as a daemon from Ubuntu's init script. From what I can
> tell it should be in /var/lib/fetchmail but there are no files in there
> (using sudo ls -al).

I would expect it to be in /etc actually. What do the following
commands show (fetchmail needs to be running for the first):

ps -ef | egrep fetchmail
find /etc -name fetchmailrc -print

--

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche
_______________________________________________
fetchmail-users mailing list
fetchmail-users <at> lists.berlios.de
https://lists.berlios.de/mailman/listinfo/fetchmail-users
Permalink | Reply |
headers
Rob MacGregor | 20 May 2012 21:23
Picon
Gravatar

Re: Fetchmail, gmail, and a new computer

On Sun, May 20, 2012 at 7:24 PM, Jim J <jahosefet <at> gmail.com> wrote:
> I'm using fetchmail 6.3.18 (from Ubuntu Oneiric) to pull email from gmail
> using POP3. It works fine. The first time I ran it it downloaded all of my
> email. Every time I've run it since then it's downloaded only new email.
> Gmail is configured to archive the email after it's been accessed via POP3.
>
> Now I have a new computer (fetchmail 6.3.21, Ubuntu Precise). I would like
> my new computer to pick up where the old left off. Fetchmail doesn't seem
> to want to do this by default. Instead, it wants to download my entire
> Gmail account. How do I make my new installation pick up where the other
> one left off? Since fetchmail didn't download everything every time, even
> after a reboot, it seems logical that there's a file somewhere I could copy
> over, but I can't find one.

What does your fetchmail configuration file and command line look like?

If you've configured it to use UIDL then there is a file to track what
has been downloaded. Being able to see the fetchmail configuration
file and the command line will allow us to either tell you where it
is, or make a reasonable guess.

--

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche
_______________________________________________
fetchmail-users mailing list
fetchmail-users <at> lists.berlios.de
(Continue reading)

Permalink | Reply |
headers
Jim J | 20 May 2012 21:45
Picon

Re: Fetchmail, gmail, and a new computer

On Sun, May 20, 2012 at 3:23 PM, Rob MacGregor <rob.macgregor <at> gmail.com>wrote:

> On Sun, May 20, 2012 at 7:24 PM, Jim J <jahosefet <at> gmail.com> wrote:
> > I'm using fetchmail 6.3.18 (from Ubuntu Oneiric) to pull email from gmail
> > using POP3. It works fine. The first time I ran it it downloaded all of
> my
> > email. Every time I've run it since then it's downloaded only new email.
> > Gmail is configured to archive the email after it's been accessed via
> POP3.
> >
> > Now I have a new computer (fetchmail 6.3.21, Ubuntu Precise). I would
> like
> > my new computer to pick up where the old left off. Fetchmail doesn't seem
> > to want to do this by default. Instead, it wants to download my entire
> > Gmail account. How do I make my new installation pick up where the other
> > one left off? Since fetchmail didn't download everything every time, even
> > after a reboot, it seems logical that there's a file somewhere I could
> copy
> > over, but I can't find one.
>
> What does your fetchmail configuration file and command line look like?
>
> If you've configured it to use UIDL then there is a file to track what
> has been downloaded. Being able to see the fetchmail configuration
> file and the command line will allow us to either tell you where it
> is, or make a reasonable guess.
>
> --
>                  Please keep list traffic on the list.
>
(Continue reading)

Permalink | Reply |
headers
Rob MacGregor | 20 May 2012 22:22
Picon
Gravatar

Re: Fetchmail, gmail, and a new computer

On Sun, May 20, 2012 at 8:45 PM, Jim J <jahosefet <at> gmail.com> wrote:

>
> /etc/fetchmailrc:
> set daemon 300
> poll pop.gmail.com port 995 with protocol pop3
>    user 'myusername' there with password 'mypassword' is 'mylocaluser'
> here
>    options ssl
>    mda 'procmail -d %T'

> I have no idea what the invocation line is, I'm using the built in Ubuntu
> init script to do the launching. Best I can guess from reading the init
> script it's something like: "/usr/bin/fetchmail -f /etc/fetchmailrc
> --pidfile /var/run/fetchmail/fetchmail.pid". I also know it's being run as
> user 'fetchmail' which has a home directory at /var/lib/fetchmail.

The first command I asked for the output of would have told you ;)

> I have not explicitly enabled UIDL anywhere that I know of, yet fetchmail
> does what I want on my old machine across reboots.

That could be Google doing something then - neither their POP3 or IMAP
implementations are quite standard. I suspect you'll have to enable
UIDL in your configuration file and live with having to re-download
lots of email.

--

-- 
                 Please keep list traffic on the list.
(Continue reading)

Permalink | Reply |
headers
Jim J | 21 May 2012 23:43
Picon

Re: Fetchmail, gmail, and a new computer

On Sun, May 20, 2012 at 4:22 PM, Rob MacGregor <rob.macgregor <at> gmail.com>wrote:

> On Sun, May 20, 2012 at 8:45 PM, Jim J <jahosefet <at> gmail.com> wrote:
>
> >
> > /etc/fetchmailrc:
> > set daemon 300
> > poll pop.gmail.com port 995 with protocol pop3
> >    user 'myusername' there with password 'mypassword' is 'mylocaluser'
> > here
> >    options ssl
> >    mda 'procmail -d %T'
>
> > I have no idea what the invocation line is, I'm using the built in Ubuntu
> > init script to do the launching. Best I can guess from reading the init
> > script it's something like: "/usr/bin/fetchmail -f /etc/fetchmailrc
> > --pidfile /var/run/fetchmail/fetchmail.pid". I also know it's being run
> as
> > user 'fetchmail' which has a home directory at /var/lib/fetchmail.
>
> The first command I asked for the output of would have told you ;)
>
> > I have not explicitly enabled UIDL anywhere that I know of, yet fetchmail
> > does what I want on my old machine across reboots.
>
> That could be Google doing something then - neither their POP3 or IMAP
> implementations are quite standard. I suspect you'll have to enable
> UIDL in your configuration file and live with having to re-download
> lots of email.
>
(Continue reading)

Permalink | Reply |
headers
Rob MacGregor | 21 May 2012 23:49
Picon
Gravatar

Re: Fetchmail, gmail, and a new computer

On Mon, May 21, 2012 at 10:43 PM, Jim J <jahosefet <at> gmail.com> wrote:
>
> Sorry, had to stop working yesterday.
>
> ps -ef | egrep fetchmail
> 110     3069        1   0  17:34 ?      00:00:00 /usr/bin/fetchmail -f
> /etc/fetchmailrc --pidfile / var/run/fetchmail/fetchmail.pid --syslog

Nothing there suggesting UIDL support - you are running this on the old box?

> find /etc -name fetchmailrc -print
> /etc/fetchmailrc

And the fetchmailrc you provided was from the old box?

> It would be really unfortunate to have to download all my email again. Are
> you telling me that Gmail can detect my particular fetchmail client through
> the NAT on my firewall in order to feed it only new mail? If I do have to
> redownload everything, do you think it would suffice to download it
> somewhere new, grab the UIDL file, and then delete the emails?

I have no idea what Google does or how it behaves. All I can tell you
is that nothing you've posted to this list suggests that you have UIDL
support enabled.

--

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
(Continue reading)

Permalink | Reply |
headers
Jim J | 22 May 2012 00:00
Picon

Re: Fetchmail, gmail, and a new computer

On Monday, May 21, 2012, Rob MacGregor wrote:

> On Mon, May 21, 2012 at 10:43 PM, Jim J <jahosefet <at> gmail.com<javascript:;>>
> wrote:
> >
> > Sorry, had to stop working yesterday.
> >
> > ps -ef | egrep fetchmail
> > 110     3069        1   0  17:34 ?      00:00:00 /usr/bin/fetchmail -f
> > /etc/fetchmailrc --pidfile / var/run/fetchmail/fetchmail.pid --syslog
>
> Nothing there suggesting UIDL support - you are running this on the old
> box?
>
> > find /etc -name fetchmailrc -print
> > /etc/fetchmailrc
>
> And the fetchmailrc you provided was from the old box?
>
> > It would be really unfortunate to have to download all my email again.
> Are
> > you telling me that Gmail can detect my particular fetchmail client
> through
> > the NAT on my firewall in order to feed it only new mail? If I do have to
> > redownload everything, do you think it would suffice to download it
> > somewhere new, grab the UIDL file, and then delete the emails?
>
> I have no idea what Google does or how it behaves. All I can tell you
> is that nothing you've posted to this list suggests that you have UIDL
> support enabled.
(Continue reading)

Permalink | Reply |
headers
Carlos E. R. | 22 May 2012 02:17
Favicon

Re: Fetchmail, gmail, and a new computer


On 2012-05-22 00:00, Jim J wrote:

> Yes, all of this was on the old box. I agree with you on the UIDL.
> Does this mean that fetchmail doesn't generate anything that I can use
> to maintain state across machines? It's some gmail magic then?

I use gmail on two computers, and both are happy; and one of them using
both thunderbird and fetchmail. But I access using _imap_, using pop3
would be a nightmare.

I do not keep a .fetchids file.

Gmail does not delete emails, it keeps track of what you downloaded
somehow, and this index can be reset. There is a configuration place in
the gmail webmail where you can say from what date you want to download
messages.

And it breaks with pop3, do not use it.

--

-- 
Cheers / Saludos,

		Carlos E. R.
		(from 11.4 x86_64 "Celadon" at Telcontar)
Permalink | Reply |
headers
Jim J | 22 May 2012 05:08
Picon

Re: Fetchmail, gmail, and a new computer

On Monday, May 21, 2012, Carlos E. R. wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 2012-05-22 00:00, Jim J wrote:
>
> > Yes, all of this was on the old box. I agree with you on the UIDL.
> > Does this mean that fetchmail doesn't generate anything that I can use
> > to maintain state across machines? It's some gmail magic then?
>
> I use gmail on two computers, and both are happy; and one of them using
> both thunderbird and fetchmail. But I access using _imap_, using pop3
> would be a nightmare.
>
> I do not keep a .fetchids file.
>
> Gmail does not delete emails, it keeps track of what you downloaded
> somehow, and this index can be reset. There is a configuration place in
> the gmail webmail where you can say from what date you want to download
> messages.
>
> And it breaks with pop3, do not use it.
>
> - --
> Cheers / Saludos,
>
>                Carlos E. R.
>                (from 11.4 x86_64 "Celadon" at Telcontar)
> -----BEGIN PGP SIGNATURE-----
(Continue reading)

Permalink | Reply |
headers
Carlos E. R. | 22 May 2012 11:29
Favicon

Re: Fetchmail, gmail, and a new computer


On 2012-05-22 05:08, Jim J wrote:

> Can somebody verify this and/or post an example for me? It looks like
> the UID comes from the Message-ID header field, though some of my
> emails don't have one so I'm not yet sure what to do in that case.

gmail_name <at> gmail.com <at> pop.gmail.com GmailId113300b9c0231221
gmail_name <at> gmail.com <at> pop.gmail.com GmailId113301960baa2027
gmail_name <at> gmail.com <at> pop.gmail.com GmailId113301ca7d33b545

But I insist that you do not need it if you use imap.

--

-- 
Cheers / Saludos,

		Carlos E. R.
		(from 11.4 x86_64 "Celadon" at Telcontar)
Permalink | Reply |
headers
Jerry | 22 May 2012 13:59
Face

Re: Fetchmail, gmail, and a new computer

On Mon, 21 May 2012 23:08:56 -0400
Jim J articulated:

>On Monday, May 21, 2012, Carlos E. R. wrote:
>> On 2012-05-22 00:00, Jim J wrote:
>>
>> > Yes, all of this was on the old box. I agree with you on the UIDL.
>> > Does this mean that fetchmail doesn't generate anything that I can
>> > use to maintain state across machines? It's some gmail magic then?
>>
>> I use gmail on two computers, and both are happy; and one of them
>> using both thunderbird and fetchmail. But I access using _imap_,
>> using pop3 would be a nightmare.
>>
>> I do not keep a .fetchids file.
>>
>> Gmail does not delete emails, it keeps track of what you downloaded
>> somehow, and this index can be reset. There is a configuration place
>> in the gmail webmail where you can say from what date you want to
>> download messages.
>>
>> And it breaks with pop3, do not use it.
>>
>Well, this is all terribly disappointing. I wish I'd known about the
>UIDL flag to begin with. I think I'm going to try to generate a
>fetchids file from my maildir, but I'm not entirely sure what it looks
>like. From the source it looks like it might be a text file with lines
>something like:
>
><remote username> <at> pop.gmail.com <UID>
(Continue reading)

Permalink | Reply |
headers
grarpamp | 23 May 2012 07:16
Picon

Re: Fetchmail, gmail, and a new computer

> I still cannot fathom why people use GMail.

I share this woe. I'm only on it due to legacy and
am slowly searching for one with minimally:

*  Free
*  IMAPS/POP3S + with proper delete semantics (unlike gmail)
*  SMTPS + allow setting GECOS field to whatever (single word nickname)
*  No source IP in headers of sent mail

Anyways, I use fetchmail a lot with not many problems, ever :)
Just some usage issues that a script should solve.
Permalink | Reply |
headers
Jim J | 24 May 2012 01:57
Picon

Re: Fetchmail, gmail, and a new computer

On Wednesday, May 23, 2012, grarpamp wrote:

> > I still cannot fathom why people use GMail.
>
> I share this woe. I'm only on it due to legacy and
> am slowly searching for one with minimally:
>
> *  Free
> *  IMAPS/POP3S + with proper delete semantics (unlike gmail)
> *  SMTPS + allow setting GECOS field to whatever (single word nickname)
> *  No source IP in headers of sent mail
>
> Anyways, I use fetchmail a lot with not many problems, ever :)
> Just some usage issues that a script should solve.
> _______________________________________________
> fetchmail-users mailing list
> fetchmail-users <at> lists.berlios.de <javascript:;>
> https://lists.berlios.de/mailman/listinfo/fetchmail-users
>

All,

Thanks for all the help. I ended up writing most of the script to generate
my fetchids file when I noticed that things were behaving properly! I have
no idea why I thought the setup was broken, I must have misinterpreted
something in the logs. *sigh* I hate it when that happens.

Thanks again.

Jim
(Continue reading)

Permalink | Reply |

Gmane