General mailing list for SimpleORM (an open source Java O/R mapping framework) users

anthony | 25 Nov 2010 00:23

Thoughts from ADO

I was reading through ADO.Net recently, and two ideas from it might be good to add to SimpleOrm.

The first is DataSet.GetChanges, which produces a dataset which only has changed records.  It does a copy,
but we could easily do a PurgeCleanRecords.  The point being that if a dataset is returned from a remote
middle tier, we could purge out any unchanged records for efficiency.  Minor point but easy to implement.

The second is AcceptChanges.  What this does is explicitly mark a record non-dirty, and copies fields to the
optimistic values.  It would be called by SessionJDBC after reading or updating records.  Might be a
slightly cleaner approach to our current handing of dirtyness.

(SimpleOrm was conceived independently of ADO.Net, but there are similarities and I am always happy to
pinch good ideas.)

Regards,

Anthony

Dr Anthony Berglas, anthony <at> berglas.org       Mobile: +61 4 4838 8874
Just because it is possible to push twigs along the ground with ones nose
does not necessarily mean that is the best way to collect firewood.

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/SimpleORM/

<*> Your email settings:
    Individual Email | Traditional

(Continue reading)

headers

Franck Routier | 25 Nov 2010 18:42

Re: Thoughts from ADO

Hi Anthony,

> The first is DataSet.GetChanges, which produces a dataset which only
> has changed records. It does a copy, but we could easily do a
> PurgeCleanRecords. The point being that if a dataset is returned from
> a remote middle tier, we could purge out any unchanged records for
> efficiency.

Do you mean if the dataset has to travel on the wire or between several
JVMs ?

> 
> The second is AcceptChanges. What this does is explicitly mark a
> record non-dirty, and copies fields to the optimistic values. It would
> be called by SessionJDBC after reading or updating records. Might be a
> slightly cleaner approach to our current handing of dirtyness.

Couldn't you elaborate on this, I don't get the point ?...

While we are on new ideas, I'm missing a feature in current Simpleorm,
related to data validation.
Currently we can valide a field, or a record.
But there is no easy way to validate coherence between different
records.
For example I can handle the fact that a field must be a date, or a date
after 01/01/2000, with SValidator. I can check on the record level that
the beginning date is before the end date of the record, with
onValidateRecord(). But I have no way to check that two record won't
have overlapping periods before committing... (well, no way native to
Simpleorm).

(Continue reading)

headers

Noel Grandin | 26 Nov 2010 09:33

Re: Thoughts from ADO

Franck Routier wrote:

While we are on new ideas, I'm missing a feature in current Simpleorm,
related to data validation.
Currently we can valide a field, or a record.
But there is no easy way to validate coherence between different
records.
For example I can handle the fact that a field must be a date, or a date
after 01/01/2000, with SValidator. I can check on the record level that
the beginning date is before the end date of the record, with
onValidateRecord(). But I have no way to check that two record won't
have overlapping periods before committing... (well, no way native to
Simpleorm).

I think there are some things you are just going to have to handle at a higher level, and this is one of them.
I've seen frameworks where this is possible, but the downside is that performance becomes very non-obvious, because the framework has to query the database in order to perform cross-record validation.

On the other hand, you could always override doValidateRecord() and do your extra validation there?

-- Noel

Disclaimer: http://www.peralex.com/disclaimer.html

__._,_.___

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___

headers

Franck Routier | 26 Nov 2010 10:00

Re: Thoughts from ADO

Hi,

Le vendredi 26 novembre 2010 à 10:33 +0200, Noel Grandin a écrit :

> I think there are some things you are just going to have to handle at
> a higher level, and this is one of them. 

Yes, I agree we want to keep Simpleorm simple... but...
That's what I do right now; that is in my DAO layer, I have a method
checkCustomConstraints that get called before a dataset is persisted.

My problem is that a dataset can have every kind of records. So I must
make sure custom constraints are called for each record type in the
dataset... (before Sorm3, I was handling lists of records of the same
type, so it was must easier).

> I've seen frameworks where this is possible, but the downside is that
> performance becomes very non-obvious, because the framework has to
> query the database in order to perform cross-record validation.

> On the other hand, you could always override doValidateRecord() and do
> your extra validation there?
> 
Well, doing it in doValidateRecord can be tricky, as it is called while
the session has begun (on flush). So you'll have to get the the current
thread's session and use queryNoFlush to get extra records from
database, then check against still unflushed records in the dataset...
Well, I could experiment with it...

...

Franck

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/SimpleORM/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/SimpleORM/join
    (Yahoo! ID required)

<*> To change settings via email:
    SimpleORM-digest <at> yahoogroups.com 
    SimpleORM-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    SimpleORM-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

headers

Noel Grandin | 26 Nov 2010 13:00

Re: Thoughts from ADO

Maybe this is one of those places where SimpleORM should provide a "pre-commit" hook that you could override to implement custom stuff like this.

Or you could create your own sub-class of SSession and override the commit() method, and do your validation there.

-- Noel.

Franck Routier wrote:

Hi,

Le vendredi 26 novembre 2010 à 10:33 +0200, Noel Grandin a écrit :

> I think there are some things you are just going to have to handle at
> a higher level, and this is one of them.

Yes, I agree we want to keep Simpleorm simple... but...
That's what I do right now; that is in my DAO layer, I have a method
checkCustomConstraints that get called before a dataset is persisted.

My problem is that a dataset can have every kind of records. So I must
make sure custom constraints are called for each record type in the
dataset... (before Sorm3, I was handling lists of records of the same
type, so it was must easier).

> I've seen frameworks where this is possible, but the downside is that
> performance becomes very non-obvious, because the framework has to
> query the database in order to perform cross-record validation.

> On the other hand, you could always override doValidateRecord() and do
> your extra validation there?
>
Well, doing it in doValidateRecord can be tricky, as it is called while
the session has begun (on flush). So you'll have to get the the current
thread's session and use queryNoFlush to get extra records from
database, then check against still unflushed records in the dataset...
Well, I could experiment with it...

...

Franck

Disclaimer: http://www.peralex.com/disclaimer.html

__._,_.___

__,_._,___

headers

anthony | 17 Dec 2010 04:44

Re: Thoughts from ADO

Responses below...

At 03:42 AM 26/11/2010, Franck Routier wrote:
>  
>
>Hi Anthony,
>
>> The first is DataSet.GetChanges, which produces a dataset which only
>> has changed records. It does a copy, but we could easily do a
>> PurgeCleanRecords. The point being that if a dataset is returned from
>> a remote middle tier, we could purge out any unchanged records for
>> efficiency.
>
>Do you mean if the dataset has to travel on the wire or between several
>JVMs ?

Yes.  Well, between two JVMs.

>> 
>> The second is AcceptChanges. What this does is explicitly mark a
>> record non-dirty, and copies fields to the optimistic values. It would
>> be called by SessionJDBC after reading or updating records. Might be a
>> slightly cleaner approach to our current handing of dirtyness.
>
>Couldn't you elaborate on this, I don't get the point ?...

This is a minor point, but in our current implementation, retrieveRecord calls defineInitialValue which
is not quite clean, and does not reset dirtyness.  I was wondering if it would be better to follow ADO here and
just call AcceptChanges (not a good name) after setting the fields normally.

Also, setting setDirty(false) does not update optimistic values properly.

So one way to clean this up is to replace setDirty(false) and defineInitialValue with just one
AcceptChanges, which clears dirty and flushes optimistic values.

Primary key only records still require thought.

>While we are on new ideas, I'm missing a feature in current Simpleorm,
>related to data validation.
>Currently we can valide a field, or a record.
>But there is no easy way to validate coherence between different
>records.
>For example I can handle the fact that a field must be a date, or a date
>after 01/01/2000, with SValidator. I can check on the record level that
>the beginning date is before the end date of the record, with
>onValidateRecord(). But I have no way to check that two record won't
>have overlapping periods before committing... (well, no way native to
>Simpleorm).

A per record pre-flush hook would be a good idea and easy to implement.  

There should probably be a DataSet.DoValidateAllRecords (ie all) and thus a RecordInstance and DataSet
OnValidateAllRecords.  Ie. be able to validate independently from flushing.  Only dirty records
validated.  flush calls dataset.DoValidateAllRecords of course.

(One other idea from ADO is they allow record.BeginEdit/EndEdits to be applied around record changes. 
Then EndEdit would call onValidateRecord, whereas we only do that at flush time.  But I wonder if that is
really useful.)

>I don't really know how this would be possible, or how other frameworks
>address this issue, but it would certainly be interesting to have.
>
>Any comment ?
>
>Regards,
>
>Franck
>
>

Dr Anthony Berglas, anthony <at> berglas.org       Mobile: +61 4 4838 8874
Just because it is possible to push twigs along the ground with ones nose
does not necessarily mean that is the best way to collect firewood.

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/SimpleORM/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/SimpleORM/join
    (Yahoo! ID required)

<*> To change settings via email:
    SimpleORM-digest <at> yahoogroups.com 
    SimpleORM-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    SimpleORM-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

headers

anthony | 10 Jan 2011 08:50

Re: Thoughts from ADO

At 03:42 AM 26/11/2010, you wrote:
>  
>
>Hi Anthony,
>
>> The first is DataSet.GetChanges, which produces a dataset which only
>> has changed records. It does a copy, but we could easily do a
>> PurgeCleanRecords. The point being that if a dataset is returned from
>> a remote middle tier, we could purge out any unchanged records for
>> efficiency.
>
>Do you mean if the dataset has to travel on the wire or between several
>JVMs ?

Yes, which I thought was one of your use cases.

>> 
>> The second is AcceptChanges. What this does is explicitly mark a
>> record non-dirty, and copies fields to the optimistic values. It would
>> be called by SessionJDBC after reading or updating records. Might be a
>> slightly cleaner approach to our current handing of dirtyness.
>
>Couldn't you elaborate on this, I don't get the point ?...

Mainly just that our dirtyness is a little dirty at the moment.  SessionJdbc knows too much about the
internal state of datasets.  

>While we are on new ideas, I'm missing a feature in current Simpleorm,
>related to data validation.
>Currently we can valide a field, or a record.
>But there is no easy way to validate coherence between different
>records.
>For example I can handle the fact that a field must be a date, or a date
>after 01/01/2000, with SValidator. I can check on the record level that
>the beginning date is before the end date of the record, with
>onValidateRecord(). But I have no way to check that two record won't
>have overlapping periods before committing... (well, no way native to
>Simpleorm).

So maybe an onValidateFlush hook?  Could be on both SRecordMeta and SSession. 
SRecordMeta.onValidateFlush called for all dirty records in the order in the update list.  Then
SSession.onValidateFlush.  And only then the records are actually flushed so that exceptions will not
produce inconsistent state.  

Easy to do, and indeed a gap.

Hibernate etc. do not really do validation in normal usage.  It is all supposed to be just dumb fields in
POJOs.  There are ways to listen for events, but they are not really designed for application logic -- you
need to reflect into the object being processed.

>I don't really know how this would be possible, or how other frameworks
>address this issue, but it would certainly be interesting to have.
>
>Any comment ?
>
>Regards,
>
>Franck
>
>

Dr Anthony Berglas, anthony <at> berglas.org       Mobile: +61 4 4838 8874
Just because it is possible to push twigs along the ground with ones nose
does not necessarily mean that is the best way to collect firewood.

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/SimpleORM/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/SimpleORM/join
    (Yahoo! ID required)

<*> To change settings via email:
    SimpleORM-digest <at> yahoogroups.com 
    SimpleORM-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    SimpleORM-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/