Discuss usage/design issues for SQLAlchemy, a Python SQL toolkit

Chris Withers | 3 Oct 2011 12:11

packages, modules and double imports - oh my!

Hi All,

The attached package gives that smallest possible example of problems 
I'm hitting with some SQLAlchemy declarative classes.

In short, I want to be able to do:

python -m pack.module and have if the __name__=='__main__' block spit 
out the SQL to create the tables necessary for the modules in that class...

So, using the attached package to demonstrate, the first problem I get is:

cwithers <at> cwlin:~> python -m pack.module
Traceback (most recent call last):
    File "/usr/lib64/python2.6/runpy.py", line 121, in _run_module_as_main
      "__main__", fname, loader, pkg_name)
    File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
      exec code in run_globals
    File "/home/cwithers/pack/module.py", line 3, in <module>
      class MyClass(object):
    File "pack/__init__.py", line 7, in __init__
      raise Exception('%r registered twice' % self.__name__)
Exception: 'MyClass' registered twice

Looks like I have a pack.module.MyClass and a __main__.MyClass, which 
feels like a bug to me...

Ho hum, lets try something different:

cwithers <at> cwlin:~> python pack/module.py

(Continue reading)

headers

Chris Rebert | 3 Oct 2011 12:22

Re: packages, modules and double imports - oh my!

On Mon, Oct 3, 2011 at 3:11 AM, Chris Withers <chris <at> simplistix.co.uk> wrote:
> Hi All,
>
> The attached package gives that smallest possible example of problems I'm
> hitting with some SQLAlchemy declarative classes.
>
> In short, I want to be able to do:
>
> python -m pack.module and have if the __name__=='__main__' block spit out
> the SQL to create the tables necessary for the modules in that class...
<snip>
> cwithers <at> cwlin:~> python -m pack
> /usr/bin/python: pack is a package and cannot be directly executed
>
> That's annoying, why isn't pack/__init__.py's __name__=='__main__' block
> executed?
>
> cwithers <at> cwlin:~> python pack
> /usr/bin/python: can't find '__main__.py' in 'pack'
>
> Wha? First I've ever heard of __main__.py... where's that documented?

http://docs.python.org/library/runpy.html :
"The runpy module['s] main use is to implement the -m command line switch"
"If the supplied module name refers to a package rather than a normal
module, then that package is imported and the __main__ submodule
within that package is then executed and the resulting module globals
dictionary returned."

(`runpy` having appeared in your first Traceback. Definitely seems

(Continue reading)

headers

Chris Withers | 3 Oct 2011 17:25

Re: packages, modules and double imports - oh my!

On 03/10/2011 11:22, Chris Rebert wrote:
> http://docs.python.org/library/runpy.html :
> "The runpy module['s] main use is to implement the -m command line switch"
> "If the supplied module name refers to a package rather than a normal
> module, then that package is imported and the __main__ submodule
> within that package is then executed and the resulting module globals
> dictionary returned."

Interesting, but doesn't help with the bizarre double-exec of module.py 
when it happens to be __main__...

cheers,

Chris

--

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

headers

Dave Angel | 3 Oct 2011 20:54

Re: Re: packages, modules and double imports - oh my!

On 01/-10/-28163 02:59 PM, Chris Withers wrote:
> On 03/10/2011 11:22, Chris Rebert wrote:
>> http://docs.python.org/library/runpy.html :
>> "The runpy module['s] main use is to implement the -m command line 
>> switch"
>> "If the supplied module name refers to a package rather than a normal
>> module, then that package is imported and the __main__ submodule
>> within that package is then executed and the resulting module globals
>> dictionary returned."
>
> Interesting, but doesn't help with the bizarre double-exec of 
> module.py when it happens to be __main__...
>
> cheers,
>
> Chris
>
Sounds to me like you have a circular dependency.  It's never a good 
idea, and especially if you have a circular dependency on the script 
itself, it'll get loaded twice.  Once it comes in as __main__, and the 
other time by its filename.  So redesign the module to eliminate the 
circulars.  If a.py imports b.py, then b.py should not import a.py, 
directly or indirectly.

You are lucky you got an explicit error;  We've seen other people get 
very obscure bugs when two instances of some module variables were in 
existence.

DaveA

(Continue reading)

headers

Michael Bayer | 3 Oct 2011 16:15

Re: packages, modules and double imports - oh my!

ive no idea what __main__.py is either ?   where's the SQLAlchemy exceptions here ?  (keeping in mind i havent
yet gone through all the steps to download a file...untar it...figure out what you're trying to do...)  

On Oct 3, 2011, at 6:11 AM, Chris Withers wrote:

> Hi All,
> 
> The attached package gives that smallest possible example of problems I'm hitting with some SQLAlchemy
declarative classes.
> 
> In short, I want to be able to do:
> 
> python -m pack.module and have if the __name__=='__main__' block spit out the SQL to create the tables
necessary for the modules in that class...
> 
> So, using the attached package to demonstrate, the first problem I get is:
> 
> cwithers <at> cwlin:~> python -m pack.module
> Traceback (most recent call last):
>   File "/usr/lib64/python2.6/runpy.py", line 121, in _run_module_as_main
>     "__main__", fname, loader, pkg_name)
>   File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
>     exec code in run_globals
>   File "/home/cwithers/pack/module.py", line 3, in <module>
>     class MyClass(object):
>   File "pack/__init__.py", line 7, in __init__
>     raise Exception('%r registered twice' % self.__name__)
> Exception: 'MyClass' registered twice
> 
> Looks like I have a pack.module.MyClass and a __main__.MyClass, which feels like a bug to me...

(Continue reading)

headers

Chris Withers | 3 Oct 2011 17:24

Re: packages, modules and double imports - oh my!

On 03/10/2011 15:15, Michael Bayer wrote:
> ive no idea what __main__.py is either ?   where's the SQLAlchemy exceptions here ?  (keeping in mind i havent
yet gone through all the steps to download a file...untar it...figure out what you're trying to do...)

The attached .tgz simplifies the real process, since this is a Python 
issue not a SA one...

The SA bit is that Declarative barfs 'cos the class is processed twice...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

--

-- 
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To post to this group, send email to sqlalchemy <at> googlegroups.com.
To unsubscribe from this group, send email to sqlalchemy+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.

headers

Simon King | 3 Oct 2011 18:15

Re: packages, modules and double imports - oh my!

On Mon, Oct 3, 2011 at 4:24 PM, Chris Withers <chris <at> simplistix.co.uk> wrote:

On 03/10/2011 15:15, Michael Bayer wrote:
ive no idea what __main__.py is either ? where's the SQLAlchemy exceptions here ? (keeping in mind i havent yet gone through all the steps to download a file...untar it...figure out what you're trying to do...)

The attached .tgz simplifies the real process, since this is a Python issue not a SA one...

The SA bit is that Declarative barfs 'cos the class is processed twice...

Chris

I haven't run the code, but this sounds like the import gotcha described at http://effbot.org/zone/import-confusion.htm#using-modules-as-scripts. As far as python is concerned these are 2 separate modules, and I don't think it's Declarative's job to detect and ignore the second occurrence.

If I were you, I would try to move the "if __name__ == '__main__'" part into a separate module that imports the first. This can then be run with "python -m mypackage.main".

This isn't so helpful if you were intending to have a number of modules that could be run independently. Perhaps you could live with something like "python -m mypackage.main mod1 mod2 mod3"?

Hope that helps,

Simon

PS. __main__.py and other behaviour are mentioned on http://docs.python.org/using/cmdline.html.

--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To post to this group, send email to sqlalchemy <at> googlegroups.com.
To unsubscribe from this group, send email to sqlalchemy+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.