Ross Berteig | 4 Sep 23:49

Re: LuaRocks with Windows/Visual Studio

Sometime on 9/4/2008, "Mark Meijer" <meijer78 <at> gmail.com> wrote:
 >2008/9/3 Ross Berteig <Ross <at> cheshireeng.com>:
 >> Unfortunately, once you get into an open system where it is
 >> expected that other components just plug in, then everything
 >> that loads within a single process must comply with the same
 >> standards.
 >
 >I'm assuming that standard for LuaRocks on Windows is MSVCRT80.DLL,
 >for the provided interpreter (LuaBinaries) as well as any 
precompiled
 >dll's in the default repository (is this true?). However:

That is the case at the moment. Of course, at some point
MSVCR80.DLL will be out of date. VS2008 uses MSVCR90.DLL by
default, for instance, and the next version of VS will likely
include a yet newer CRT. Scuri builds LuaBinaries for a
disturbingly large number of compilers and runtime libraries on
Windows alone, but LfW has standardized on MSVCR80.DLL
(technically the SxS assembly known as Microsoft.VC80.CRT) as the
preferred runtime for their package.

Any library that is intended to just work with LfW must be
buildable against that CRT. Any binary distribution of such a
library must be built against that CRT.

 >> the off-the-shelf MinGW with -lmsvcrt80 gets you applications and
 >> DLLs that are *mostly* free of dependency on MSVCRT.DLL. Getting
 >> them completely free requires more effort.
 >
 >What I understand from this, is that any rocks I install (compile)
(Continue reading)

RJP Computing | 5 Sep 04:19
Gravatar

Re: LuaRocks with Windows/Visual Studio

On Thu, Sep 4, 2008 at 5:49 PM, Ross Berteig <Ross <at> cheshireeng.com> wrote:
That is the case at the moment. Of course, at some point
MSVCR80.DLL will be out of date. VS2008 uses MSVCR90.DLL by
default, for instance, and the next version of VS will likely
include a yet newer CRT. Scuri builds LuaBinaries for a
disturbingly large number of compilers and runtime libraries on
Windows alone, but LfW has standardized on MSVCR80.DLL
(technically the SxS assembly known as Microsoft.VC80.CRT) as the
preferred runtime for their package.

Any library that is intended to just work with LfW must be
buildable against that CRT. Any binary distribution of such a
library must be built against that CRT.

This is 100% correct. Thanks for clarifying
.
>> the off-the-shelf MinGW with -lmsvcrt80 gets you applications and
>> DLLs that are *mostly* free of dependency on MSVCRT.DLL. Getting
>> them completely free requires more effort.
>
>What I understand from this, is that any rocks I install (compile)
>using MinGW will either depend on an old VC6 runtime (MSVCRT.DLL), or
>on *both* the old and the new one (MSVCRT80.DLL), depending on
>whether or not by whatever magic the -lmsvcrt80 option is used.

It is clear that the MinGW team was trying to make -lmsvcrt80 be
sufficient to eliminate dependency on the old CRT. Unfortunately,
they missed a couple of things, and if you step on one of those
your program still links but is now dependent on both CRT
versions in an unhealthy way.

Under the covers, MSVCR80.DLL itself is linked against
MSVCRT.DLL, but that dependency is presumably healthy because it
came from MS that way, and MSVCRT.DLL is a system component
installed in C:\Windows\System32. Of course, since exactly one
function (named _getdrives()) is imported implicitly from the old
runtime, it is completely possible that this is due to an
oversight and has never been corrected because it doesn't
actually cause problems.

Steve Donovan (IIRC) identified a list of 19 CRT functions that
have their names incorrectly documented in MSVCR80.DLL, and hence
they are mis-wrapped in libmsvcr80.a. Using any of those
functions will produce a dependency on MSVCRT.DLL, and in some
cases will cause the implementations of malloc() and free() to be
mixed between the two CRT versions which cannot be good even if
it appears to work.

In my projects, I follow Steve's list of functions to alias with
macros at compile time, and use a link line that ends with:

       $(LUADLL) -lgcc -lmsvcr80 $(MSVCR80)

Where LUADLL is a fully qualified name of lua.5.1.dll and MSVCR80
is the fully qualified name of MSVCR80.DLL. Since I use LfW, I
find both of those DLLs in its installed tree, based off the
LUA_DEV environment variable it defined. The result is a
dependency tree that has two implicit loads of MSVCR80.DLL, and
only loads MSVCRT.DLL due to MSVCR80's need. If I leave out
libgcc.a or otherwise move it after either reference to MSVCR80,
then I find unhealthy dependencies on MSVCRT.DLL.

Your mileage will vary, so I strongly recommend checking DLL
dependency carefully before releasing a binary.

How come you need  -lmsvcr80 and $(MSVCR80)? shouldn't $(MSVCR80) be enough? Can you explain a bit more?

I suppose I ought to go put this into the wiki page... (see
http://lua-users.org/wiki/BuildingModules).

Please add this. I would like to study this. I would prefer to use the GCC toolchain.


>And if I use
>Visual Studio, many rocks will not compile at all. So.... stop me if
>I'm missing something, but basically my choices are:

LuaRocks is still young. Personally, my hope is that Lua for
Windows will adopt LuaRocks on the Windows platform. Even then,
it is going to take some effort to make sure that binary
compatible rocks are available so that the end users do not need
to recompile rocks themselves. That will take all the pressure
off the end users, and put it on the maintainers of the LuaRocks
repository.

However, builders of rocks still have to deal with the issues you
are worried about, and so far LuaRocks is not the whole answer.

>1: Use Visual Studio and not a lot of rocks....

At the moment (ca. v1.0), LuaRocks sort of assumes that the
Windows platform implies that you have VS installed and
available. If you get it installed and working, then just using
VS for your own project work should work out ok.

Yes this dependency should not be forced on Windows. The rocks that are available should be built and hosted as binaries that link to the VC8 CRT. Thoughts?
 
<snip>

Ryan has said that LuaRocks is in his plans for Lua for
Windows... We all just need to gently encourage that because the
combination will be ideal. The end user gets an easily installed
and complete Lua, and then can extend it at will from the
community of modules, which then keep themselves up to date.

If you want LuaRocks to be integrated faster I would really like help. I would like help getting binary releases of the rocks in the repository. If we can make this effort I will really step up the priority on testing and figuring out how to make it work in LfW. Thoughts? Volenteers?
--
Regards,
Ryan
Hisham | 5 Sep 04:55

Re: LuaRocks with Windows/Visual Studio

On Thu, Sep 4, 2008 at 11:19 PM, RJP Computing <rjpcomputing <at> gmail.com> wrote:
> On Thu, Sep 4, 2008 at 5:49 PM, Ross Berteig <Ross <at> cheshireeng.com> wrote:
>> >1: Use Visual Studio and not a lot of rocks....
>>
>> At the moment (ca. v1.0), LuaRocks sort of assumes that the
>> Windows platform implies that you have VS installed and
>> available. If you get it installed and working, then just using
>> VS for your own project work should work out ok.
>
> Yes this dependency should not be forced on Windows. The rocks that are
> available should be built and hosted as binaries that link to the VC8 CRT.
> Thoughts?

Agreed -- the ones we have there were contributed by the Kepler team
in order to have Kepler fully installable on Windows without requiring
a compiler toolchain in the system. I did try some rocks on Windows
for testing LuaRocks but I never tried to go building them all.
Perhaps a good first step in improving the selection of available
binary rocks on Windows would be to collect a list of which rocks
currently build and which don't.

>> <snip>
>>
>> Ryan has said that LuaRocks is in his plans for Lua for
>> Windows... We all just need to gently encourage that because the
>> combination will be ideal. The end user gets an easily installed
>> and complete Lua, and then can extend it at will from the
>> community of modules, which then keep themselves up to date.
>
> If you want LuaRocks to be integrated faster I would really like help. I
> would like help getting binary releases of the rocks in the repository. If
> we can make this effort I will really step up the priority on testing and
> figuring out how to make it work in LfW. Thoughts? Volenteers?

Yes, getting the binaries in the rocks repo would be the best first
step. Another thing to do is to check if all desired modules do
currently have rockspecs in the repo at all.

(Maybe it would be best to continue this discussion at the luarocks
mailing list.)

-- Hisham

Ignacio Burgueño | 5 Sep 15:24

Re: LuaRocks with Windows/Visual Studio

RJP Computing wrote:
> On Thu, Sep 4, 2008 at 5:49 PM, Ross Berteig <Ross <at> cheshireeng.com 
> <mailto:Ross <at> cheshireeng.com>> wrote:
> 
> Yes this dependency should not be forced on Windows. The rocks that are 
> available should be built and hosted as binaries that link to the VC8 
> CRT. Thoughts?

May I ask why VC8 instead of VC9 ? Is it because MinGW lacks support for 
VC9 (I haven't checked, just guessing).

Just asking because a large part of our codebase uses VC6 and we're 
considering migrating to VS2008 eventually. So no VC8 for me, sadly. 
Also, as someone said, there will probably be a MSVCRT10 :D Seems like 
there's no long term solution.

Regards,
Ignacio

RJP Computing | 5 Sep 16:13
Gravatar

Re: LuaRocks with Windows/Visual Studio



On Fri, Sep 5, 2008 at 9:24 AM, Ignacio Burgueño <ignaciob <at> inconcertcc.com> wrote:
RJP Computing wrote:

On Thu, Sep 4, 2008 at 5:49 PM, Ross Berteig <Ross <at> cheshireeng.com <mailto:Ross <at> cheshireeng.com>> wrote:

Yes this dependency should not be forced on Windows. The rocks that are available should be built and hosted as binaries that link to the VC8 CRT. Thoughts?

May I ask why VC8 instead of VC9 ? Is it because MinGW lacks support for VC9 (I haven't checked, just guessing).

Just asking because a large part of our codebase uses VC6 and we're considering migrating to VS2008 eventually. So no VC8 for me, sadly. Also, as someone said, there will probably be a MSVCRT10 :D Seems like there's no long term solution.

Well it is really because the majority of the modules already were compiled for VC8 so we decided not to rebuild them. The project is more about packaging then building. We are waiting for LuaRocks to take that role. :)
--
Regards,
Ryan
RJP Computing
Thomas Harning | 5 Sep 19:38
Gravatar

Re: LuaRocks with Windows/Visual Studio

Has anyone thought of writing a proxy DLL of sorts for the MSVCRT?   
One could one that meets some least-common-denominator and everyone  
links against it.

It could also take the role of providing a compatibility interface to  
permit newer MSVCRTs being used w/ their fixes/etc, but providing all  
the same APIs.
Tackling this would definitely need some studying of the APIs exposed  
by each MSVCRT..... one thing I don't know is if certain compiler  
constructions by MinGW for example (easiest target I think) hook  
directly into the MSVCRT...........

This could be the way to kill this MSVCRT dumbness once and for all  
(well, testing new versions of MSVCRT w/ your app would be  
preferable)....

On Sep 5, 2008, at 10:13 AM, RJP Computing wrote:

>
>
> On Fri, Sep 5, 2008 at 9:24 AM, Ignacio Burgueño <ignaciob <at> inconcertcc.com 
> > wrote:
> RJP Computing wrote:
>
> On Thu, Sep 4, 2008 at 5:49 PM, Ross Berteig <Ross <at> cheshireeng.com <mailto:Ross <at> cheshireeng.com 
> >> wrote:
>
> Yes this dependency should not be forced on Windows. The rocks that  
> are available should be built and hosted as binaries that link to  
> the VC8 CRT. Thoughts?
>
> May I ask why VC8 instead of VC9 ? Is it because MinGW lacks support  
> for VC9 (I haven't checked, just guessing).
>
> Just asking because a large part of our codebase uses VC6 and we're  
> considering migrating to VS2008 eventually. So no VC8 for me, sadly.  
> Also, as someone said, there will probably be a MSVCRT10 :D Seems  
> like there's no long term solution.
>
> Well it is really because the majority of the modules already were  
> compiled for VC8 so we decided not to rebuild them. The project is  
> more about packaging then building. We are waiting for LuaRocks to  
> take that role. :)
> -- 
> Regards,
> Ryan
> RJP Computing

Paul Moore | 5 Sep 21:49

Re: LuaRocks with Windows/Visual Studio

2008/9/5 Thomas Harning <harningt <at> gmail.com>:
> Has anyone thought of writing a proxy DLL of sorts for the MSVCRT?  One
> could one that meets some least-common-denominator and everyone links
> against it.

I'm not sure how that would work. The main program determines the CRT
in use - only that CRT gets its "process startup" initialisation code
(if any) run. So embedded lua and any modules it uses has to either
use that CRT, or use a CRT which doesn't need startup code (and be
careful not to mix CRTs in any way). The problem is that not all
applications use the CRT (a real-life example: JPSoft's 4NT/Take
Command does not use the CRT at all, and so my Lua plugin cannot use a
CRT which needs initialisation code to run).

Based on my experimentation, MSVCRT80 crashes if it isn't "initialised
properly" - which I guess means it has crucial startup code. I guess
the same is likely true of MSVCRT90.

The original MSVCRT doesn't crash. This either means it doesn't have
any required startup code, or that I haven't found the problem yet :-)

Ultimately, this basically means that Lua can't actually rely on
having any fully-initialised CRT available. But for all practical
purposes, if you either (1) link with the CRT used by the main
application, or (2) use MSVCRT, you don't seem to hit issues.
Unfortunately, later MS compilers have no option to link with the
older CRT, so that there is no universal solution.

> This could be the way to kill this MSVCRT dumbness once and for all (well,
> testing new versions of MSVCRT w/ your app would be preferable)....

There has been talk of writing a library of precisely the CRT
functions Lua needs, and distributing that. For functions like the
various string and maths functions, there is clearly no dependency on
static data, so no initialisation issues can arise. Whether memory
allocation or IO can be done this way is a little less obvious.

Paul.


Gmane