headers
Sebastian Spaeth | 19 Sep 2011 14:39
Picon

New folder creation on REMOTE (DO READ)

Hot on the heels of my temp. maintainership, I take the chance to push a
revised branch that implements the automatic creation of remote folders.

We implement a sanity check that will stop if you are using nametrans
rules and they do not lead to the same name when applied back and forth.

If you don't use nametrans, everything is cool and you can stop reading here.

If you use nametrans on the remote repository, you will need a nametrans
rule on the local repository that reverses the name.

So if REMOTE nametrans = lambda f: "INBOX." + f
you need a LOCAL nametrans rule that does
nametrans = lambda f: re.sub("^INBOX.", "", f)

If you don't get it right, OfflineImap will perform a sanity check and
complain that the back and forth resolution does not lead to identical
names. and refuse to create the folder on the remote repository.

This feature will probably require some more documentation extensions
and typical use cases, which I will add in the near future.

With this feature, I consider the next stable version feature complete,
and will freeze feature work, and stabilize the 'next' branch so we can
release a new stable soonish.

Do let me know if and how things work (or break), the change should be
rather safe even if it requires one more nametrans setting.

Sebastian
(Continue reading)

Permalink | Reply |
headers
Vincent Beffara | 19 Sep 2011 14:57
Picon
Picon
Gravatar

Re: New folder creation on REMOTE (DO READ)


Hi,

SS> Do let me know if and how things work (or break), the change should
SS> be rather safe even if it requires one more nametrans setting.

It does not work very well :-( My setup is like this:

[Repository here]

type = IMAP
folderfilter = lambda f: not re.search('^_', f)
nametrans = lambda f: re.sub ('^INBOX/INBOX$', 'INBOX', 'INBOX/' + f)

[Repository ensl_imap]

type = IMAP
nametrans = lambda f: re.sub ('^INBOX/', '', f)

and I get an error on a folder that exists on both sides (and was synced
perfectly well before the change, it is not a new folder):

ERROR: Exceptions occurred during the run!
 ERROR: Creating folder enseignement on repository here
  Folder 'enseignement'[here] could not be created.
  Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists:
                                             enseignement'])
 ERROR: Folder 'enseignement'[here] could not be created.
 Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists:
                                            enseignement'])
(Continue reading)

Permalink | Reply |
headers
Vincent Beffara | 19 Sep 2011 15:02
Picon
Picon
Gravatar

Re: New folder creation on REMOTE (DO READ)


VB> ERROR: Exceptions occurred during the run!
VB>  ERROR: Creating folder enseignement on repository here
VB>   Folder 'enseignement'[here] could not be created.  Server
VB>   responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists:
VB>                                              enseignement'])
VB>  ERROR: Folder 'enseignement'[here] could not be created.  Server
VB>  responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists:
VB>                                             enseignement'])

BTW, switching back to revision 6b2ec95, the error becomes a warning,
the sync proceeds as usual, but there was no warning before ... All goes
well when I remove the local nametrans from the config file.

     /v

--

-- 
|                 |   UMPA - ENS Lyon   | Mél: vbeffara <at> ens-lyon.fr |
| Vincent Beffara |  46 allée d'Italie  | Tél: (+33) 4 72 72 85 25  |
|                 | 69364 Lyon Cedex 07 | Fax: (+33) 4 72 72 84 80  |

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap
Permalink | Reply |
headers
Sebastian Spaeth | 19 Sep 2011 21:11
Picon

Re: New folder creation on REMOTE (DO READ)

On Mon, 19 Sep 2011 14:57:31 +0200, Vincent Beffara wrote:

Hi Vincent, thanks for being my guinea pig. Sorry to hear it did not
work out well. I went once again through the setup with pen and paper,
making notes when to use what and I believe I got it right now.

> nametrans = lambda f: re.sub ('^INBOX/INBOX$', 'INBOX', 'INBOX/' + f)
> nametrans = lambda f: re.sub ('^INBOX/', '', f)

Mmh, your REMOTE repo here' adds an INBOX/ prefix and your LOCAL repo
removes it. 
Your nametrans rules do not do the reverse if you have a
local folder that does not start with INBOX. If every folder does, you
should be fine indeed.

I just pushed what I believe to be the fix.

>   Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists:
>                                              enseignement'])

I was able to reproduce this and to fix this, I think. Can you try
again? In the very worst case, you'll get a few folders to delete but no
data loss.

Thanks, if this is fixed, it is time for rc3 :-)

Sebastian

_______________________________________________
(Continue reading)

Permalink | Reply |
headers
John Wiegley | 19 Sep 2011 20:59
Picon
Gravatar

Re: New folder creation on REMOTE (DO READ)

>>>>> Sebastian Spaeth <Sebastian <at> SSpaeth.de> writes:

> Hot on the heels of my temp. maintainership, I take the chance to push a
> revised branch that implements the automatic creation of remote folders.

Hi Sebastian,

I have my Maildir under Git, and so I put 'git' in my folderfilter so that
offlineimap doesn't sync it.  However, your new remote folder creation code
just created a "git" label in my Gmail account, even though it's not going to
sync that folder.

I should think it would only create remote folders for folders that will be
sync'd.  I happen to keep several "local-only" mailboxes, for archival
purposes.

Thanks,
  John

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap
Permalink | Reply |
headers
Sebastian Spaeth | 19 Sep 2011 21:30
Picon

Re: New folder creation on REMOTE (DO READ)

On Mon, 19 Sep 2011 13:59:23 -0500, John Wiegley <jwiegley <at> gmail.com> wrote:
> Hi Sebastian,
> 
> I have my Maildir under Git, and so I put 'git' in my folderfilter so that
> offlineimap doesn't sync it.  However, your new remote folder creation code
> just created a "git" label in my Gmail account, even though it's not going to
> sync that folder.
> 
> I should think it would only create remote folders for folders that will be
> sync'd.  I happen to keep several "local-only" mailboxes, for archival
> purposes.

Hi John,

it decided whether to create a "LOCAL" folder on "REMOTE" depending if a
folderfilter setting in the *local* repository applies to the local folder.

Should we check for all new local folders if the corresponding folder would be
filtered out using the remote folderfilters too before deciding whether
to create one? That would actually make sense, so I made it so. Can you
delete the label and retry? Besides erronous folders, there is no
increased danger of data loss.

Thanks,
Sebastian

commit 1c58ebe3484fa72e541acba14ff5498df63c77a7
Author: Sebastian Spaeth <Sebastian <at> SSpaeth.de>

Do not create folder on REMOTE if it would fall in REMOTE's folderfilter
(Continue reading)

Permalink | Reply |
headers
John Wiegley | 19 Sep 2011 22:05
Picon
Gravatar

Re: New folder creation on REMOTE (DO READ)

>>>>> Sebastian Spaeth <Sebastian <at> SSpaeth.de> writes:

> commit 1c58ebe3484fa72e541acba14ff5498df63c77a7
> Author: Sebastian Spaeth <Sebastian <at> SSpaeth.de>

> Do not create folder on REMOTE if it would fall in REMOTE's folderfilter

That fixed it, thanks!

John

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap
Permalink | Reply |
headers
Sebastian Spaeth | 20 Sep 2011 18:27
Picon

Re: New folder creation on REMOTE (DO READ)

On Mon, 19 Sep 2011 15:05:51 -0500, John Wiegley <jwiegley <at> gmail.com> wrote:
> >>>>> Sebastian Spaeth <Sebastian <at> SSpaeth.de> writes:
> 
> > commit 1c58ebe3484fa72e541acba14ff5498df63c77a7
> > Author: Sebastian Spaeth <Sebastian <at> SSpaeth.de>
> 
> > Do not create folder on REMOTE if it would fall in REMOTE's folderfilter
> 
> That fixed it, thanks!

OK, thanks good to know.

Sebastian

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap
Permalink | Reply |
headers
Leon Bogaert | 20 Sep 2011 12:57
Picon
Favicon
Gravatar

Re: New folder creation on REMOTE (DO READ)

On Mon, 2011-09-19 at 14:39 +0200, Sebastian Spaeth wrote:
> Hot on the heels of my temp. maintainership, I take the chance to push a
> revised branch that implements the automatic creation of remote folders.

Awesome! I wish I had this branch this weekend. I rearranged my local
Maildir and thunderbird is still syncing the changes to remote...

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap
Permalink | Reply |
headers
Vincent Beffara | 20 Sep 2011 14:59
Picon
Picon
Gravatar

Re: New folder creation on REMOTE (DO READ)


Hi again,

I tried a few things, and none seemed to fix my problem ... To sum it
up:

- I am using IMAP <-> IMAP

- remote is Cyrus IMAP at work, folder names like INBOX, INBOX/math,
  INBOX/something etc.

- local is Dovecot, folder names like INBOX, math, something

So, I have a remote nametrans that does re.sub('INBOX/',''), and all
goes well, up to the time when I upgrade to revision fd6261a, which
complains like this:

AssertionError: newdstname bogus equals newsrcname bogus bogus bogus

Given the code in Base.py, that seems to be expected, so I move to
revision b92e453 which is the next one. Now it complains that it cannot
create folder 'bogus' on the remote side. Again this is expected, it
does not conform to the remote name pattern. So I add a nametrans on the
local side, adding 'INBOX/' in front of the names.

That fixes it, but it now tries to recreate existing folders on the
local side - and that leads to an ERROR (whereas it was just a WARNING
in previous versions).

Upgrading to the current 'next' branch, having a nametrans on the local
(Continue reading)

Permalink | Reply |
headers
Sebastian Spaeth | 20 Sep 2011 18:57
Picon

Re: New folder creation on REMOTE (DO READ)

On Tue, 20 Sep 2011 14:59:17 +0200, Vincent Beffara:
> 
> Hi again,
> 
> I tried a few things, and none seemed to fix my problem ... To sum it
> up:
> 
> - I am using IMAP <-> IMAP
> 
> - remote is Cyrus IMAP at work, folder names like INBOX, INBOX/math,
>   INBOX/something etc.
> 
> - local is Dovecot, folder names like INBOX, math, something
> 
> So, I have a remote nametrans that does re.sub('INBOX/',''), and all
> goes well, up to the time when I upgrade to revision fd6261a, which
> complains like this:
> 
> AssertionError: newdstname bogus equals newsrcname bogus bogus bogus

This is a leftover debug statement and it is a good thing. It means your
back-and-forth nametrans resolve to the same name :-). It also means you
are on the next branch on a commit older than
b92e453f0ea7c26c5eceebfb45083338564281e6 where this debug statement was
removed.

(it slipped in accidentally, I usually don't include stuff like this in
the code, sorry)

> Given the code in Base.py, that seems to be expected, so I move to
(Continue reading)

Permalink | Reply |
headers
Vincent Beffara | 20 Sep 2011 19:05
Picon
Picon
Gravatar

Re: New folder creation on REMOTE (DO READ)


Hi,

>> Upgrading to the current 'next' branch, having a nametrans on the
>> local repository or not changes nothing, it fails to create 'bogus'
>> on the remote end in all cases.

SS> Are you saying that you do see these warnings on the current next
SS> HEAD?  And you still see that "bogus" already exists warning?

On the current 'next' branch (revision 1c58ebe), with two nametrans
validating the previous check, I get this:

ERROR: Folder 'bogus'[ensl_imap] could not be created. Server responded:
('NO', ['Permission denied'])

'bogus' exists locally, not on ensl_imap which is the remote
end. Passing 'bogus' through localrepo's nametrans gives INBOX/bogus
which should be allowed for creation on remote, but indeed creating just
'bogus' there is forbidden.

Seeing as the nametrans for the local repo was not called I tried
removing it altogether from the .offlineimaprc and got exactly the same
error message.

So, not the same "already exists" warnings, no, rather a symptom that
apparently the remote folder creation code is not using the local
nametrans, if I get it right.

HTH,
(Continue reading)

Permalink | Reply |
headers
Sebastian Spaeth | 20 Sep 2011 22:41
Picon

Re: New folder creation on REMOTE (DO READ)

On Tue, 20 Sep 2011 19:05:32 +0200, Vincent Beffara:
> SS> Are you saying that you do see these warnings on the current next
> SS> HEAD?  And you still see that "bogus" already exists warning?

> So, not the same "already exists" warnings, no, rather a symptom that
> apparently the remote folder creation code is not using the local
> nametrans, if I get it right.

This was exactly spot on. I had fixed the sanity check to apply the
nametrans, but we attempted to create the foldername without applying
the local repository nametrans rule. Thanks for testing this. I tried it
out and it created INBOX/bogus for me on the REMOTE using your nametrans
rules.

pushed [db28cb7] Fix another visiblename() glitch in determining the foldername to be created

Let me know if this is finally working, so I can push out another RC.

Sebastian

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap
Permalink | Reply |
headers
Vincent Beffara | 20 Sep 2011 23:53
Picon
Picon
Gravatar

Re: New folder creation on REMOTE (DO READ)


SS> This was exactly spot on. I had fixed the sanity check to apply the
SS> nametrans, but we attempted to create the foldername without
SS> applying the local repository nametrans rule. Thanks for testing
SS> this. I tried it out and it created INBOX/bogus for me on the REMOTE
SS> using your nametrans rules.

SS> pushed [db28cb7] Fix another visiblename() glitch in determining the
SS> foldername to be created

SS> Let me know if this is finally working, so I can push out another
SS> RC.

Fixed ! Thanks,

      /v

--

-- 
|                 |   UMPA - ENS Lyon   | Mél: vbeffara <at> ens-lyon.fr |
| Vincent Beffara |  46 allée d'Italie  | Tél: (+33) 4 72 72 85 25  |
|                 | 69364 Lyon Cedex 07 | Fax: (+33) 4 72 72 84 80  |

_______________________________________________
OfflineIMAP-project mailing list
OfflineIMAP-project <at> lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project

OfflineIMAP homepage: http://software.complete.org/offlineimap
Permalink | Reply |

Gmane