General mailing list for SimpleORM (an open source Java O/R mapping framework) users

Noel Grandin | 3 Feb 2012 07:59

[patch] add CLOB datatype [1 Attachment]

<*>[Attachment(s) from Noel Grandin included below]

Hi

The attached file adds the CLOB datatype.

Regards, Noel Grandin

Disclaimer: http://www.peralex.com/disclaimer.html

<*>Attachment(s) from Noel Grandin:

<*> 1 of 1 File(s)
http://groups.yahoo.com/group/SimpleORM/attachments/folder/838970561/item/list 
  <*> SFieldClob.java

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/SimpleORM/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/SimpleORM/join
    (Yahoo! ID required)

(Continue reading)

headers

Noel Grandin | 16 Apr 2012 14:46

Re: [patch] add CLOB datatype

Ping?
Anybody here?

On 2012-02-03 08:59, Noel Grandin wrote:

Hi

The attached file adds the CLOB datatype.

Regards, Noel Grandin

Disclaimer: http://www.peralex.com/disclaimer.html

Disclaimer: http://www.peralex.com/disclaimer.html

__._,_.___

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___

headers

anthony berglas | 23 Apr 2012 07:58

Re: [patch] add CLOB datatype

Hello Noel,

Thanks for the submission and sorry about the tardy response, have been busy. Will try to review and add this sooner than later.

Franck, do you have time to do this?

Anthony

On Mon, Apr 16, 2012 at 10:46 PM, Noel Grandin <noel <at> peralex.com> wrote:

Ping?
Anybody here?

On 2012-02-03 08:59, Noel Grandin wrote:

Hi

The attached file adds the CLOB datatype.

Regards, Noel Grandin

Disclaimer: http://www.peralex.com/disclaimer.html

Disclaimer: http://www.peralex.com/disclaimer.html

--

Dr Anthony Berglas, anthony <at> berglas.org Mobile: +61 4 4838 8874
Just because it is possible to push twigs along the ground with ones nose
does not necessarily mean that that is the best way to collect firewood.

__._,_.___

__,_._,___

headers

Franck Routier | 24 Apr 2012 09:12

Re: [patch] add CLOB datatype

Hi Noël and Anthony,

I've been looking at your code, and I was wondering why Clobs wouldn't be comparable... What is the reason behind that ?
Typically, Postgresql type for Clobs (TEXT) would just compare fine, as well as java representation of a Clob (a String).
Is there any Oracle specificity around that, or simply a size problem ??

Thanks,
Franck

Le 23/04/2012 07:58, anthony berglas a écrit :

Hello Noel,

Thanks for the submission and sorry about the tardy response, have been busy. Will try to review and add this sooner than later.

Franck, do you have time to do this?

Anthony

On Mon, Apr 16, 2012 at 10:46 PM, Noel Grandin <noel <at> peralex.com> wrote:

Ping?
Anybody here?

On 2012-02-03 08:59, Noel Grandin wrote:

Hi

The attached file adds the CLOB datatype.

Regards, Noel Grandin

Disclaimer: http://www.peralex.com/disclaimer.html

Disclaimer: http://www.peralex.com/disclaimer.html

--

Dr Anthony Berglas, anthony <at> berglas.org Mobile: +61 4 4838 8874
Just because it is possible to push twigs along the ground with ones nose
does not necessarily mean that that is the best way to collect firewood.

__._,_.___

__,_._,___

headers

Noel Grandin | 24 Apr 2012 11:12

Re: [patch] add CLOB datatype

No good reason, just that it would be a little surprising performance wise - could trigger a very long fetch from the server that is likely to be uncached.

But I don't have a principled objection to it.

On 2012-04-24 09:12, Franck Routier wrote:

Hi Noël and Anthony,

I've been looking at your code, and I was wondering why Clobs wouldn't be comparable... What is the reason behind that ?
Typically, Postgresql type for Clobs (TEXT) would just compare fine, as well as java representation of a Clob (a String).
Is there any Oracle specificity around that, or simply a size problem ??

Thanks,
Franck

Disclaimer: http://www.peralex.com/disclaimer.html

__._,_.___

__,_._,___

headers

Franck Routier | 24 Apr 2012 12:18

Re: [patch] add CLOB datatype

Ok.
But, then maybe we should give the user (I mean the developper using Simpleorm) an access to the Clob object, instead of fetching its value on queryFieldValue ?
And maybe add a custom method to SFieldClob that would effectively fetch the value, but not on queryFieldValue ?

Another point is, did this code work ? I'm asking because the break condition in queryFieldValue seems erroneous, as reader.read(buf) will return -1 on end of file, not 0. ?

Franck

Le 24/04/2012 11:12, Noel Grandin a écrit :

No good reason, just that it would be a little surprising performance wise - could trigger a very long fetch from the server that is likely to be uncached.

But I don't have a principled objection to it.

On 2012-04-24 09:12, Franck Routier wrote:

Hi Noël and Anthony,

I've been looking at your code, and I was wondering why Clobs wouldn't be comparable... What is the reason behind that ?
Typically, Postgresql type for Clobs (TEXT) would just compare fine, as well as java representation of a Clob (a String).
Is there any Oracle specificity around that, or simply a size problem ??

Thanks,
Franck

Disclaimer: http://www.peralex.com/disclaimer.html

__._,_.___

__,_._,___

headers

Noel Grandin | 26 Apr 2012 11:41

Re: [patch] add CLOB datatype

You know, you are right, that loop does look dodgy.
To be honest, my code only briefly used CLOB's and then I moved onto something else, so that code is no longer in use here.

On 2012-04-24 12:18, Franck Routier wrote:

Ok.
But, then maybe we should give the user (I mean the developper using Simpleorm) an access to the Clob object, instead of fetching its value on queryFieldValue ?
And maybe add a custom method to SFieldClob that would effectively fetch the value, but not on queryFieldValue ?

Another point is, did this code work ? I'm asking because the break condition in queryFieldValue seems erroneous, as reader.read(buf) will return -1 on end of file, not 0. ?

Franck

Le 24/04/2012 11:12, Noel Grandin a écrit :
No good reason, just that it would be a little surprising performance wise - could trigger a very long fetch from the server that is likely to be uncached.

But I don't have a principled objection to it.

On 2012-04-24 09:12, Franck Routier wrote:

Hi Noël and Anthony,

I've been looking at your code, and I was wondering why Clobs wouldn't be comparable... What is the reason behind that ?
Typically, Postgresql type for Clobs (TEXT) would just compare fine, as well as java representation of a Clob (a String).
Is there any Oracle specificity around that, or simply a size problem ??

Thanks,
Franck

Disclaimer: http://www.peralex.com/disclaimer.html

Disclaimer: http://www.peralex.com/disclaimer.html

__._,_.___

__,_._,___

headers

Franck Routier | 26 Apr 2012 14:55

Re: [patch] add CLOB datatype

Ok. So I'll fix this.

What I have done is :
1) allow user to choose a maxAutoRetrieveSize, to (eventually) avoid getting all content when building the SRecordInstance
2) tweak Prostgresql, CachéDB and Mysql driver (to get the right sql type, eg TEXT or LONGVARCHAR)
3) add a test in DataTypeTest

Notice :
1) Postgresql does not really support Clobs.
This would map to TEXT fields, but the streaming part is not implemented, and the whole text will be in the resultset anyway.
2) I was not able to find a way to allow interaction with the Clob object.
Maybe with another SFieldXXXX that would have an internal type java.sql.Clob (but then you could only interact while within the transaction as I understand it).
Anyway, it's probably not very useful for now.

I have added a test case, but I have no Oracle database to test it now. Anyone willing to test is welcome.

Regards,
Franck

Le 26/04/2012 11:41, Noel Grandin a écrit :

You know, you are right, that loop does look dodgy.
To be honest, my code only briefly used CLOB's and then I moved onto something else, so that code is no longer in use here.

On 2012-04-24 12:18, Franck Routier wrote:
Ok.
But, then maybe we should give the user (I mean the developper using Simpleorm) an access to the Clob object, instead of fetching its value on queryFieldValue ?
And maybe add a custom method to SFieldClob that would effectively fetch the value, but not on queryFieldValue ?

Another point is, did this code work ? I'm asking because the break condition in queryFieldValue seems erroneous, as reader.read(buf) will return -1 on end of file, not 0. ?

Franck

Le 24/04/2012 11:12, Noel Grandin a écrit :
No good reason, just that it would be a little surprising performance wise - could trigger a very long fetch from the server that is likely to be uncached.

But I don't have a principled objection to it.

On 2012-04-24 09:12, Franck Routier wrote:

Hi Noël and Anthony,

I've been looking at your code, and I was wondering why Clobs wouldn't be comparable... What is the reason behind that ?
Typically, Postgresql type for Clobs (TEXT) would just compare fine, as well as java representation of a Clob (a String).
Is there any Oracle specificity around that, or simply a size problem ??

Thanks,
Franck

Disclaimer: http://www.peralex.com/disclaimer.html

Disclaimer: http://www.peralex.com/disclaimer.html

__._,_.___

__,_._,___

headers

anthony berglas | 27 Apr 2012 06:39

Re: [patch] add CLOB datatype

Hello Noel,

Thanks for providing the patch, but we really do need a decent test case for all changes. (Not sure what HSQL does with Clobs, but if not supported then the test case should run properly on at least one database!).

As it is, Franck will have spent more time on it than the original authorship, for a feature that no one is currently requiring. Remember that people can always use raw SQL for odd cases, and a goal is to keep SimpleOrm simple. And B/Clobs have extra issues that we should be able to retrieve them as a single string/byte array, or as a stream.

As to PostgreSql, there is a fair chance that accessing an ordinary text field as a Clob should work, remember that the Java type need not be the same as the SQL type in JDBC.

Anthony

On Thu, Apr 26, 2012 at 10:55 PM, Franck Routier <franck.routier <at> axege.com> wrote:

Ok. So I'll fix this.

What I have done is :
1) allow user to choose a maxAutoRetrieveSize, to (eventually) avoid getting all content when building the SRecordInstance
2) tweak Prostgresql, CachéDB and Mysql driver (to get the right sql type, eg TEXT or LONGVARCHAR)
3) add a test in DataTypeTest

Notice :
1) Postgresql does not really support Clobs.
This would map to TEXT fields, but the streaming part is not implemented, and the whole text will be in the resultset anyway.
2) I was not able to find a way to allow interaction with the Clob object.
Maybe with another SFieldXXXX that would have an internal type java.sql.Clob (but then you could only interact while within the transaction as I understand it).
Anyway, it's probably not very useful for now.

I have added a test case, but I have no Oracle database to test it now. Anyone willing to test is welcome.

Regards,
Franck

Le 26/04/2012 11:41, Noel Grandin a écrit :

You know, you are right, that loop does look dodgy.
To be honest, my code only briefly used CLOB's and then I moved onto something else, so that code is no longer in use here.

On 2012-04-24 12:18, Franck Routier wrote:
Ok.
But, then maybe we should give the user (I mean the developper using Simpleorm) an access to the Clob object, instead of fetching its value on queryFieldValue ?
And maybe add a custom method to SFieldClob that would effectively fetch the value, but not on queryFieldValue ?

Another point is, did this code work ? I'm asking because the break condition in queryFieldValue seems erroneous, as reader.read(buf) will return -1 on end of file, not 0. ?

Franck

Le 24/04/2012 11:12, Noel Grandin a écrit :
No good reason, just that it would be a little surprising performance wise - could trigger a very long fetch from the server that is likely to be uncached.

But I don't have a principled objection to it.

On 2012-04-24 09:12, Franck Routier wrote:

Hi Noël and Anthony,

I've been looking at your code, and I was wondering why Clobs wouldn't be comparable... What is the reason behind that ?
Typically, Postgresql type for Clobs (TEXT) would just compare fine, as well as java representation of a Clob (a String).
Is there any Oracle specificity around that, or simply a size problem ??

Thanks,
Franck

Disclaimer: http://www.peralex.com/disclaimer.html

Disclaimer: http://www.peralex.com/disclaimer.html

Dr Anthony Berglas, anthony <at> berglas.org Mobile: +61 4 4838 8874
Just because it is possible to push twigs along the ground with ones nose
does not necessarily mean that that is the best way to collect firewood.

__._,_.___

__,_._,___

headers

Franck Routier | 27 Apr 2012 12:08

Re: [patch] add CLOB datatype

Hi,

just to make it clear, the test in DataTypeTest passes on Postgresql.
It does not bring any real benefit, as TEXT field are retrieved entirely by Postgresql jdbc driver in the ResultSet anyway, but it will allow using 'TEXT' field just fine.

But I did not test any other database engine, especially Oracle.

Regards,
Franck

Le 27/04/2012 06:39, anthony berglas a écrit :

Hello Noel,

Thanks for providing the patch, but we really do need a decent test case for all changes. (Not sure what HSQL does with Clobs, but if not supported then the test case should run properly on at least one database!).

As it is, Franck will have spent more time on it than the original authorship, for a feature that no one is currently requiring. Remember that people can always use raw SQL for odd cases, and a goal is to keep SimpleOrm simple. And B/Clobs have extra issues that we should be able to retrieve them as a single string/byte array, or as a stream.

As to PostgreSql, there is a fair chance that accessing an ordinary text field as a Clob should work, remember that the Java type need not be the same as the SQL type in JDBC.

Anthony

On Thu, Apr 26, 2012 at 10:55 PM, Franck Routier <franck.routier <at> axege.com> wrote:

Ok. So I'll fix this.

What I have done is :
1) allow user to choose a maxAutoRetrieveSize, to (eventually) avoid getting all content when building the SRecordInstance
2) tweak Prostgresql, CachéDB and Mysql driver (to get the right sql type, eg TEXT or LONGVARCHAR)
3) add a test in DataTypeTest

Notice :
1) Postgresql does not really support Clobs.
This would map to TEXT fields, but the streaming part is not implemented, and the whole text will be in the resultset anyway.
2) I was not able to find a way to allow interaction with the Clob object.
Maybe with another SFieldXXXX that would have an internal type java.sql.Clob (but then you could only interact while within the transaction as I understand it).
Anyway, it's probably not very useful for now.

I have added a test case, but I have no Oracle database to test it now. Anyone willing to test is welcome.

Regards,
Franck

Le 26/04/2012 11:41, Noel Grandin a écrit :

You know, you are right, that loop does look dodgy.
To be honest, my code only briefly used CLOB's and then I moved onto something else, so that code is no longer in use here.

On 2012-04-24 12:18, Franck Routier wrote:
Ok.
But, then maybe we should give the user (I mean the developper using Simpleorm) an access to the Clob object, instead of fetching its value on queryFieldValue ?
And maybe add a custom method to SFieldClob that would effectively fetch the value, but not on queryFieldValue ?

Another point is, did this code work ? I'm asking because the break condition in queryFieldValue seems erroneous, as reader.read(buf) will return -1 on end of file, not 0. ?

Franck

Le 24/04/2012 11:12, Noel Grandin a écrit :
No good reason, just that it would be a little surprising performance wise - could trigger a very long fetch from the server that is likely to be uncached.

But I don't have a principled objection to it.

On 2012-04-24 09:12, Franck Routier wrote:

Hi Noël and Anthony,

I've been looking at your code, and I was wondering why Clobs wouldn't be comparable... What is the reason behind that ?
Typically, Postgresql type for Clobs (TEXT) would just compare fine, as well as java representation of a Clob (a String).
Is there any Oracle specificity around that, or simply a size problem ??

Thanks,
Franck

Disclaimer: http://www.peralex.com/disclaimer.html

Disclaimer: http://www.peralex.com/disclaimer.html

--

Dr Anthony Berglas, anthony <at> berglas.org Mobile: +61 4 4838 8874
Just because it is possible to push twigs along the ground with ones nose
does not necessarily mean that that is the best way to collect firewood.

__._,_.___

__,_._,___