Juan Jose Garcia-Ripoll | 26 Jun 22:36 2013
Picon

ASDF's MAKE-BUILD breaks ECL's DLL target

I have real trouble deciphering the new make-build code but I have come to the conclusion that the :DLL target has been eliminated.

The current :DLL target claims to "Link together all the dynamic library used by this system into a single one." which is not what ECL's :DLL target was doing.

ECL's :DLL/:SHARED-LIBRARY target did the same as :STATIC-LIBRARY, which is to link all FASL files into a single, linkable file, but in the shared library form.

The elimination of this feature and the replacement with a possibly empty behavior (I see no opeartion associated to DLL at all), breaks ECL's users code and expectations, having had already some bug reports for that.

The worse thing is that I no longer seem to understand ASDF's internal's logic well enough to provide any reliable kind of fix, but what I attached seems to do the job just as well. In this patch, since the DLL operation does nothing on any other platform, I took the liberty to condition it on ECL/MKCL

Juanjo

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://quinfog.iff.csic.es
Attachment (asdf.patch): application/octet-stream, 3205 bytes
Faré | 26 Jun 23:57 2013
Picon

Re: ASDF's MAKE-BUILD breaks ECL's DLL target

Dear Juanjo,

sorry for the breakage and thanks for the patch.

On Wed, Jun 26, 2013 at 4:36 PM, Juan Jose Garcia-Ripoll
<jj.garcia.ripoll@...> wrote:
> I have real trouble deciphering the new make-build code but I have come to
> the conclusion that the :DLL target has been eliminated.
>
> The current :DLL target claims to "Link together all the dynamic library
> used by this system into a single one." which is not what ECL's :DLL target
> was doing.
>
> ECL's :DLL/:SHARED-LIBRARY target did the same as :STATIC-LIBRARY, which is
> to link all FASL files into a single, linkable file, but in the shared
> library form.
>
> The elimination of this feature and the replacement with a possibly empty
> behavior (I see no opeartion associated to DLL at all), breaks ECL's users
> code and expectations, having had already some bug reports for that.
>
There was never the intent to break the :dll target,
but we still don't have a test case for it in the test suite,
so the feature is fragile.
Can you add a #+ecl test for it in test/test-bundle.script?
I'm not sure exactly how it's supposed to work, so I prefer you do it.

I tried to integrate your patch, with modifications, in 3.0.1.12.
It looks like it works to me. Can you test again?

Then I tried to compile the latest ECL from git
and now it's all broken for me (on Ubuntu Linux amd64).

> The worse thing is that I no longer seem to understand ASDF's internal's
> logic well enough to provide any reliable kind of fix, but what I attached
> seems to do the job just as well. In this patch, since the DLL operation
> does nothing on any other platform, I took the liberty to condition it on
> ECL/MKCL
>
I'm not sure what there is to understand or not understand.
ASDF internals are actually much cleaner than they used to be:
these days they actually make sense.

The main magic methods for bundles are:
  (defmethod component-depends-on ((o bundle-compile-op) (c system))
  (defmethod input-files ((o bundle-compile-op) (c system))
  (defmethod perform ((o bundle-compile-op) (c system))
The component-depends-on finds all the components that will have to be linked,
each with a lib-op or compile-op operation,
depending on the bundle being monolithic or not.
The input-files extracts the list of files to link from these dependencies
and crucially depends on all these being direct dependencies.
The perform does the magic linking.

I kept the defclass on all implementations, because it's simpler than
to add a bunch of #+(or ecl mkcl) everywhere, or to add more special cases
to the symbol definition test in test/test-utilities.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Whatever says the law, it is only ever forbidden but to get caught. — Faré

Faré | 27 Jun 22:55 2013
Picon

Re: ASDF's MAKE-BUILD breaks ECL's DLL target

I'd like to add a test in test-bundle.script.
I tried this:

  (operate 'dll-op :test-asdf/bundle-2)
  (si:load-foreign-module (first (output-files 'dll-op :test-asdf/bundle-2)))

  (ffi:def-function "init_dll_BUNDLE_2" () :returning :void)
  (init-dll-BUNDLE-2)

But the function call fails, because it the test script is
interpreted, not compiled.
How do I call the initialization function from the interpreter?

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
It is not recognized in the full amplitude of the word that all freedom is
essentially self-liberation — that I can have only so much freedom as I
procure for myself by my ownness. — Max Stirner

Juan Jose Garcia-Ripoll | 27 Jun 23:35 2013
Picon

Re: ASDF's MAKE-BUILD breaks ECL's DLL target


On Thu, Jun 27, 2013 at 10:55 PM, Faré <fahree-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
I'd like to add a test in test-bundle.script.
I tried this:

  (operate 'dll-op :test-asdf/bundle-2)
  (si:load-foreign-module (first (output-files 'dll-op :test-asdf/bundle-2)))

  (ffi:def-function "init_dll_BUNDLE_2" () :returning :void)
  (init-dll-BUNDLE-2)

But the function call fails, because it the test script is
interpreted, not compiled.
How do I call the initialization function from the interpreter?

The initialization function cannot be directly called, so I would rather recommend the following solution. First, file2.lisp is changed as below. Then, instead of calling a function, we verify that the library is available by serching for a symbol

(si::find-foreign-symbol "sample_function" "name-of-dll" :pointer-void 0)

---- file2.lisp ----
#+ecl
(ffi:clines "
extern int sample_function();

int sample_function()
{
return 0;
}
")



--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://quinfog.iff.csic.es

Faré | 27 Jun 23:59 2013
Picon

Re: ASDF's MAKE-BUILD breaks ECL's DLL target

>> How do I call the initialization function from the interpreter?
>
> The initialization function cannot be directly called, so I would rather
> recommend the following solution. First, file2.lisp is changed as below.
> Then, instead of calling a function, we verify that the library is available
> by serching for a symbol
>
> (si::find-foreign-symbol "sample_function" "name-of-dll" :pointer-void 0)
>
> ---- file2.lisp ----
> #+ecl
> (ffi:clines "
> extern int sample_function();
>
> int sample_function()
> {
> return 0;
> }
> ")
>
OK. Well, I added a test for dll-op, so in the future, this
functionality will hopefully keep working. I didn't add a test for
monolithic-dll-op, though. Sigh.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
My opinions may have changed, but not the fact that I am right.

Juan Jose Garcia-Ripoll | 28 Jun 00:09 2013
Picon

Re: ASDF's MAKE-BUILD breaks ECL's DLL target

On Thu, Jun 27, 2013 at 11:59 PM, Faré <fahree-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
OK. Well, I added a test for dll-op, so in the future, this
functionality will hopefully keep working. I didn't add a test for
monolithic-dll-op, though. Sigh.

Thanks, Faré. Your help is very much appreciated. 

Juanjo

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://quinfog.iff.csic.es

Gmane