headers
JC Chu | 1 Jul 2012 17:39
Picon
Favicon
Gravatar

Custom operators for non-record types

In Delphi mode, each custom operator has to be associated with a
record type, and must take or return such a record.  It seems that the
OBJFPC mode doesn’t have this restriction; for example, the Math unit
exports a custom ** on Int64, which is a simple type.

However, I’ve been unable to define operators other than ** for
non-record types, even if they do not conflict with the default
interpretation.  Examples include OPERATOR MOD (x, m: Double): Double
and OPERATOR AND (x, y: Tristate): Tristate, where Tristate is an
enumeration type.

Is this a bug, or is there any explanation for this?

Thanks.

--

-- 
Best regards,
JC Chu
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
Sven Barth | 1 Jul 2012 21:41

Re: Custom operators for non-record types

On 01.07.2012 17:39, JC Chu wrote:
> In Delphi mode, each custom operator has to be associated with a
> record type, and must take or return such a record.  It seems that the
> OBJFPC mode doesn’t have this restriction; for example, the Math unit
> exports a custom ** on Int64, which is a simple type.

Global operator declarations were forbidden for mode Delphi as Delphi 
does not support them either (you can use them in Delphi modes though 
even if they were declared in non-Delphi modes). Global operators 
existed in FPC before Delphi introduced their operators in records concept.

> However, I’ve been unable to define operators other than ** for
> non-record types, even if they do not conflict with the default
> interpretation.  Examples include OPERATOR MOD (x, m: Double): Double
> and OPERATOR AND (x, y: Tristate): Tristate, where Tristate is an
> enumeration type.
>
> Is this a bug, or is there any explanation for this?

Some parameter combinations are not allowed, because certain inline 
operators are already in place. I don't know their exact rulings, but 
ordinals (Integer values, floating point values), enums and sets are a 
bit restricted regarding overload (especially if it is a binary operator 
where left and right is of such a type).

It's best you simply try whether it's supported and if not you need to 
look for a different method... (or ask on the list why it's not allowed)

Regards,
Sven
(Continue reading)

Permalink | Reply |
headers
JC Chu | 2 Jul 2012 04:46
Picon
Favicon
Gravatar

Re: Custom operators for non-record types

Do you have some inside information on the extra limitations regarding
ordinal, floating point, and set types?

As I understand, the only limitation on operator “overloading” should
be that the operator do not have a default interpretation for the
operands–result type combination.

The modulus operator, for example, is certainly not defined by default
as (Double, Double) → Double; but when I try to define it, the
compiler says it’s an “[i]mpossible operator overload”.  The same goes
for logical operators on enumeration types.  (Although each
enumeration element has an underlying numerical value, it requires the
use of Ord() or an explicit cast to convert it into a numerical type
accepted by bitwise logical operators.)

Maybe I should start preparing for a bug report?

On Mon, Jul 2, 2012 at 3:41 AM, Sven Barth <pascaldragon@...> wrote:
> Some parameter combinations are not allowed, because certain inline
> operators are already in place. I don't know their exact rulings, but
> ordinals (Integer values, floating point values), enums and sets are a bit
> restricted regarding overload (especially if it is a binary operator where
> left and right is of such a type).

--

-- 
Best regards,
JC Chu
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
(Continue reading)

Permalink | Reply |
headers
Sven Barth | 2 Jul 2012 09:54

Re: Custom operators for non-record types

Am 02.07.2012 04:46, schrieb JC Chu:
> Do you have some inside information on the extra limitations regarding
> ordinal, floating point, and set types?
>
> As I understand, the only limitation on operator “overloading” should
> be that the operator do not have a default interpretation for the
> operands–result type combination.
>
> The modulus operator, for example, is certainly not defined by default
> as (Double, Double) → Double; but when I try to define it, the
> compiler says it’s an “[i]mpossible operator overload”.  The same goes
> for logical operators on enumeration types.  (Although each
> enumeration element has an underlying numerical value, it requires the
> use of Ord() or an explicit cast to convert it into a numerical type
> accepted by bitwise logical operators.)

The best source of information is the compiler source itself. The whole 
logic that decides whether an operator is overloadable or not is located 
in htypechk.pas, isoperatoracceptable and more imporantly also 
isbinaryoperatoroverloadable. From what I see everything is forbidden 
except what is written in there as an exception (and the ** operator 
which is always allowed).

> Maybe I should start preparing for a bug report?

Maybe there were reasons to not allow them... You can of course prepare 
a bug report or wait till one of the other developers who are more 
experienced in FPC's overloading speak up.

Regards,
(Continue reading)

Permalink | Reply |
headers
JC Chu | 2 Jul 2012 13:37
Picon
Favicon
Gravatar

Re: Custom operators for non-record types

Anyway, here’s a patch for reviewing…

On Mon, Jul 2, 2012 at 3:54 PM, Sven Barth <pascaldragon@...> wrote:
> Maybe there were reasons to not allow them... You can of course prepare a
> bug report or wait till one of the other developers who are more experienced
> in FPC's overloading speak up.
>

--

-- 
Best regards,
JC Chu
Attachment (htypechk.pas.patch): application/octet-stream, 5667 bytes
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
JC Chu | 2 Jul 2012 16:50
Picon
Favicon
Gravatar

Re: Custom operators for non-record types

Apologies for the previous patch which I didn’t test carefully.

I’ve made a revised version which fixes the over-relaxation problem
that led to a number of default operand type combinations to be
treated as overloadable.

◦  internal_check() now handles enumeration, set, and floating-point types.
◦  Unnecessary restrictions on enumerations have been removed.
◦  Restrictions on the string-as-the-first-operand case have been relaxed.

There may be other valid cases not covered by this fix.

I’ve filed a bug report for the patch at
<http://bugs.freepascal.org/view.php?id=22359>.

On Mon, Jul 2, 2012 at 7:37 PM, JC Chu <jcchu@...> wrote:
> Anyway, here’s a patch for reviewing…

--

-- 
Best regards,
JC Chu
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
Sven Barth | 2 Jul 2012 21:35

Re: Custom operators for non-record types

Am 02.07.2012 16:51 schrieb "JC Chu" <jcchu <at> acm.org>:
> ◦  Restrictions on the string-as-the-first-operand case have been relaxed.

I don't know whether you realized that already, but the isbinaryoperatoroverloadable function is called twice: once with the order given by the user and once with the order reversed as at least regarding the possibility of overloading the binary operators are commutative. So the checks for the first operand being a string indirectly also apply to the right one being a string.

Regards,
Sven

_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
JC Chu | 3 Jul 2012 02:51
Picon
Favicon
Gravatar

Re: Custom operators for non-record types

isbinaryoperatoroverloadable() is called once.  It first calls
internal_check() with the default (LHS, RHS) order.  internal_check()
accepts or rejects the pair only if it recognizes the pair.  Only if
(LHS, RHS) is not recognized, will internal_check() be called again
with the order reversed.  If neither (LHS, RHS) nor (RHS, LHS) is
recognized, isbinaryoperatoroverloadable() defaults to False.
(Besides, commutativity is a semantic property which relies on the
user implementation and cannot be deduced at compile time.)

Rejection rules are used to prevent conflicts with the default
interpretation.  The original implementation, however, rejects
overloads based solely on the operand type combination, not taking
into consideration the operator being used.  For the
string-as-the-first-operand case, this resulted in, say, * as a
function on (string, PChar) being rejected, which is legal.  What I
did with it is restrict the operand to +, =, <>, <, <=, >, and >=
(and, of course, removing the limits on the second operand being an
enumeration).

For some operators, only one-sided protection is necessary.  For
example, IN must not be overloaded as a function on (<enumeration>,
<set>).

For some operators, the set of accepted operand type pairs should be
symmetric, and thus so should its complement, the set of rejected
pairs.  If acceptance and rejection rules are inconsistent, rejection
becomes sensitive to the order of operands.  For example, I didn’t
change pointer-as-the-first-operand case to allow for (PChar, string)
overloads, so * as a function on (PChar, string), despite legal, still
gets rejected.

That is to say, the rules in internal_check() are not yet exhaustive.

On Tue, Jul 3, 2012 at 3:35 AM, Sven Barth <pascaldragon@...> wrote:
> Am 02.07.2012 16:51 schrieb "JC Chu" <jcchu@...>:
>
>
>> ◦  Restrictions on the string-as-the-first-operand case have been relaxed.
>
> I don't know whether you realized that already, but the
> isbinaryoperatoroverloadable function is called twice: once with the order
> given by the user and once with the order reversed as at least regarding the
> possibility of overloading the binary operators are commutative. So the
> checks for the first operand being a string indirectly also apply to the
> right one being a string.
>
> Regards,
> Sven
>
>
> _______________________________________________
> fpc-pascal maillist  -  fpc-pascal@...
> http://lists.freepascal.org/mailman/listinfo/fpc-pascal

--

-- 
Best regards,
JC Chu
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
Sven Barth | 3 Jul 2012 09:51

Re: Custom operators for non-record types

Am 03.07.2012 02:51, schrieb JC Chu:
> isbinaryoperatoroverloadable() is called once.  It first calls
> internal_check() with the default (LHS, RHS) order.  internal_check()
> accepts or rejects the pair only if it recognizes the pair.  Only if
> (LHS, RHS) is not recognized, will internal_check() be called again
> with the order reversed.  If neither (LHS, RHS) nor (RHS, LHS) is
> recognized, isbinaryoperatoroverloadable() defaults to False.

Yes, I mean internal_check. Sorry. I hadn't the source available when I 
wrote you yesterday.

> (Besides, commutativity is a semantic property which relies on the
> user implementation and cannot be deduced at compile time.)

For commutativity regarding the implementation you are right, but I was 
talking about commutativity regarding the possibility to overload. E.g. 
you should be able to overload "and" no matter whether it's argument 
types are "(String, LongInt)" or "(LongInt, String)".

> Rejection rules are used to prevent conflicts with the default
> interpretation.  The original implementation, however, rejects
> overloads based solely on the operand type combination, not taking
> into consideration the operator being used.  For the
> string-as-the-first-operand case, this resulted in, say, * as a
> function on (string, PChar) being rejected, which is legal.  What I
> did with it is restrict the operand to +, =,<>,<,<=,>, and>=
> (and, of course, removing the limits on the second operand being an
> enumeration).
 >
> For some operators, only one-sided protection is necessary.  For
> example, IN must not be overloaded as a function on (<enumeration>,
> <set>).
>

Agreed so far. :)

> For some operators, the set of accepted operand type pairs should be
> symmetric, and thus so should its complement, the set of rejected
> pairs.  If acceptance and rejection rules are inconsistent, rejection
> becomes sensitive to the order of operands.  For example, I didn’t
> change pointer-as-the-first-operand case to allow for (PChar, string)
> overloads, so * as a function on (PChar, string), despite legal, still
> gets rejected.
>
> That is to say, the rules in internal_check() are not yet exhaustive.

So in the end "* (PChar, String)" should be allowed as well out of 
consistency, shouldn't it?

Nevertheless for pointers one needs to be careful, as I don't know 
exactly how far internal pointer arithmetic operators are available 
(AFAIK at least "+" and maybe "-" exist).

Regards,
Sven
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
JC Chu | 3 Jul 2012 12:21
Picon
Favicon
Gravatar

Re: Custom operators for non-record types

Yes, it should.  The bottomline is that cases treated as overloadable
must not be defined by the default interpretation, because the
overloaded interpretation will always take precedence.  This is
probably why the ld.typ = orddef case is not handled by either
isbinaryoperatoroverloadable() or its unary counterpart.  (But it’s
really not true that ordinal types always make trouble—think of * as
(Char, Integer) → string, for example.)

Speaking of unary operators…  I did not change
isunaryoperatoroverloadable() to allow NOT to be defined on Double and
enumeration types.  I think NOT should be allowed on all types except
ordinals (which include Booleans).

Our terminology seems a bit inconsistent, too, don’t you think?

PChar and PWideChar correspond to pointerdef, but behaves like
strings---namely they can be manipulated and compared using operators
+, =, <>, <, <=, >, and >= with another string-like entity (- doesn’t
seem to belong there).  The original implementation for the ld.typ =
pointerdef case already guarantees that both the LHS and the RHS are
string-like, so it remains only to add a limit on the operator so that
only those mentioned above are forbidden.

The thing that worries me is that I’m not sure if the default
interpretation depends on the compilation mode being used.  If that is
the case maybe the rules will become even more complicated.

On Tue, Jul 3, 2012 at 3:51 PM, Sven Barth <pascaldragon@...> wrote:
> So in the end "* (PChar, String)" should be allowed as well out of
> consistency, shouldn't it?
>
> Nevertheless for pointers one needs to be careful, as I don't know exactly
> how far internal pointer arithmetic operators are available (AFAIK at least
> "+" and maybe "-" exist).

--

-- 
Best regards,
JC Chu
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
Jonas Maebe | 2 Jul 2012 16:51
Picon
Favicon

Re: Custom operators for non-record types


JC Chu wrote on Mon, 02 Jul 2012:

> On Mon, Jul 2, 2012 at 3:54 PM, Sven Barth  
> <pascaldragon@...> wrote:
>> Maybe there were reasons to not allow them... You can of course prepare a
>> bug report or wait till one of the other developers who are more experienced
>> in FPC's overloading speak up.
>
> Anyway, here’s a patch for reviewing…

Going to a situation where overloading is allowed by default is not  
good. It's too easy to miss something (because there is no formal  
specification of the language anywhere), and then in subsequent  
releases you may have to have to break existing code because you have  
to disable certain overloads again.

By explicitly allowing particular operation overloads and forbidding  
the rest, the risk of having to break backward compatibility later on  
is much smaller (at least if every exception is carefully considered).

Jonas
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
JC Chu | 3 Jul 2012 02:49
Picon
Favicon
Gravatar

Re: Custom operators for non-record types

Yes.  I made that error in my first patch.  The new compiler wasn’t
even able to compile itself…

I think someone should start making a complete chart for acceptance
and rejection rules.

On Mon, Jul 2, 2012 at 10:51 PM, Jonas Maebe <jonas.maebe@...> wrote:
> Going to a situation where overloading is allowed by default is not good.
> It's too easy to miss something (because there is no formal specification of
> the language anywhere), and then in subsequent releases you may have to have
> to break existing code because you have to disable certain overloads again.

--

-- 
Best regards,
JC Chu
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
Sven Barth | 3 Jul 2012 09:55

Re: Custom operators for non-record types

Am 03.07.2012 02:49, schrieb JC Chu:
> Yes.  I made that error in my first patch.  The new compiler wasn’t
> even able to compile itself…

I don't know whether you know this, but:
* testing the compiler compilation (as a first step) is best done using 
"make cycle" in the compiler subdirectory (optionally "make fullcycle" 
if you changed something were a subset of the supported platforms is 
affected)
* if the compiler can be compiled you should do a "make clean all 
TEST_FPC=path/to/your/newly/compiled/ppcXXX" in the "tests" directory 
and compare with a testrun of the same version without the modifications 
(it's sufficient to copy the "log" or "faillist" file from the 
"tests/output/yourcpu-youros" directory for comparison)

Regards,
Sven

_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
JC Chu | 3 Jul 2012 12:22
Picon
Favicon
Gravatar

Re: Custom operators for non-record types

On Tue, Jul 3, 2012 at 3:55 PM, Sven Barth <pascaldragon@...> wrote:
> I don't know whether you know this, but:
> * testing the compiler compilation (as a first step) is best done using
> "make cycle" in the compiler subdirectory (optionally "make fullcycle" if
> you changed something were a subset of the supported platforms is affected)

I did “make compiler” in $(source)\compiler and copied the resulting
executable to my scratch directory.  No wonder I didn’t find out the
error.  (ಠ_ಠ)

> * if the compiler can be compiled you should do a "make clean all
> TEST_FPC=path/to/your/newly/compiled/ppcXXX" in the "tests" directory and
> compare with a testrun of the same version without the modifications (it's
> sufficient to copy the "log" or "faillist" file from the
> "tests/output/yourcpu-youros" directory for comparison)

Could you elaborate on that?  Do I need to run the test suite with the
old compiler first?

--

-- 
Best regards,
JC Chu
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
Sven Barth | 3 Jul 2012 15:00

Re: Custom operators for non-record types

Am 03.07.2012 12:23 schrieb "JC Chu" <jcchu-HInyCGIudOg@public.gmane.org>:
>
> On Tue, Jul 3, 2012 at 3:55 PM, Sven Barth <pascaldragon-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote:
> > I don't know whether you know this, but:
> > * testing the compiler compilation (as a first step) is best done using
> > "make cycle" in the compiler subdirectory (optionally "make fullcycle" if
> > you changed something were a subset of the supported platforms is affected)
>
> I did “make compiler” in $(source)\compiler and copied the resulting
> executable to my scratch directory.  No wonder I didn’t find out the
> error.  (ಠ_ಠ)

Now you know a better way :)

> > * if the compiler can be compiled you should do a "make clean all
> > TEST_FPC=path/to/your/newly/compiled/ppcXXX" in the "tests" directory and
> > compare with a testrun of the same version without the modifications (it's
> > sufficient to copy the "log" or "faillist" file from the
> > "tests/output/yourcpu-youros" directory for comparison)
>
> Could you elaborate on that?  Do I need to run the test suite with the
> old compiler first?

* first do a "make cycle" on a unmodified checkout, then run the testsuite.
* save the "log" and/or "faillog" file
* do your modifications (might involve some "make cycle" executions until you're satisfied with the results)
* run the testsuite again and compare the new "log" or "faillist" file against the backuped counterpart
* if there are no explainable changes in succeeding/failing tests: publish your patch
* otherwise correct your changes so that you don't introduce regressions and repeat at point 4

Regards,
Sven

_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
Jonas Maebe | 3 Jul 2012 15:28
Picon
Favicon

Re: Custom operators for non-record types


Sven Barth wrote on Tue, 03 Jul 2012:

> Am 03.07.2012 12:23 schrieb "JC Chu" <jcchu@...>:
>>
>> Could you elaborate on that?  Do I need to run the test suite with the
>> old compiler first?
>
> * first do a "make cycle" on a unmodified checkout, then run the testsuite.

While "make cycle" is useful as a first test, for a complete testsuite  
run you also need the packages. So before running the testsuite, run a  
"make all" in the top level FPC directory (possibly preceded by an "rm  
build*" in case make says that nothing needs to be done for "all").

Also, if you do this regularly and have a multicore, you can speed up  
things significantly by using (replace "x" with the number of cores  
you have, or the number of cores+1 if you have a slow hard drive or  
are performing a cold build)
a) for the fpc make all: make all -j x FPMAKEOPT="-T x"
b) for the testsuite: make full TEST_OPT="-O2" -j x

(the testsuite "full" target is equivalent to "clean all digest", but  
written in a way that makes all dependencies explicit so that parallel  
building works correctly)

Jonas
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
JC Chu | 4 Jul 2012 15:20
Picon
Favicon
Gravatar

Re: Custom operators for non-record types

Thank you both for the tips.

I have just finished the patch and tested it.  The patched version
gives identical test results as the trunk.

The patch refines operator overloading acceptance and rejection rules
for both unary and binary operators.

The new rules for unary operators allow NOT to be defined on
enumeration and floating-point types.

The new rules for binary operators should allow for all operator
overloads that do not conflict with the default interpretation (DI).*
The rules were designed based on a classification of operators and
operand types according to their roles in the DI, and achieved a high
accuracy in the first trial.

*Note: An exception is that the operator IN cannot be overloaded with
the signature (ordinal, set), where ordinal is any type covered by
TOrdDef, even though not all such typs have a corresponding set type.
This is unlikely to become a real obstacle, however, and can be
relaxed very easily.

The patch will soon be added to the bug report at
<http://bugs.freepascal.org/view.php?id=22359>.

On Tue, Jul 3, 2012 at 9:28 PM, Jonas Maebe <jonas.maebe@...> wrote:
>
> Sven Barth wrote on Tue, 03 Jul 2012:
>
>> Am 03.07.2012 12:23 schrieb "JC Chu" <jcchu@...>:
>>>
>>>
>>> Could you elaborate on that?  Do I need to run the test suite with the
>>> old compiler first?
>>
>>
>> * first do a "make cycle" on a unmodified checkout, then run the
>> testsuite.
>
>
> While "make cycle" is useful as a first test, for a complete testsuite run
> you also need the packages. So before running the testsuite, run a "make
> all" in the top level FPC directory (possibly preceded by an "rm build*" in
> case make says that nothing needs to be done for "all").
>
> Also, if you do this regularly and have a multicore, you can speed up things
> significantly by using (replace "x" with the number of cores you have, or
> the number of cores+1 if you have a slow hard drive or are performing a cold
> build)
> a) for the fpc make all: make all -j x FPMAKEOPT="-T x"
> b) for the testsuite: make full TEST_OPT="-O2" -j x
>
> (the testsuite "full" target is equivalent to "clean all digest", but
> written in a way that makes all dependencies explicit so that parallel
> building works correctly)
>
>
> Jonas
>
> _______________________________________________
> fpc-pascal maillist  -  fpc-pascal@...
> http://lists.freepascal.org/mailman/listinfo/fpc-pascal

--

-- 
Best regards,
JC Chu
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |
headers
Jonas Maebe | 2 Jul 2012 13:39
Picon
Favicon

Re: Custom operators for non-record types


Sven Barth wrote on Mon, 02 Jul 2012:

> Maybe there were reasons to not allow them... You can of course  
> prepare a bug report or wait till one of the other developers who  
> are more experienced in FPC's overloading speak up.

I think most of operator overloading support was implemented by Peter,  
who no longer has time for FPC nowadays. I don't know whether anyone  
ever made a thorough analysis of what can be safely allowed and what  
not, and to what extent the current limitations are for implementation  
resp. semantic reasons.

Jonas
_______________________________________________
fpc-pascal maillist  -  fpc-pascal@...
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Permalink | Reply |

Gmane