In Pike 8.1 I checked in a few new native Sql types. This allows for faster, easier and lossless manipulation of dates/times/intervals. It also facilitates binary transfer to and from the database.
I currently implemented all this for the pgsql driver. The new code still needs documentation. Reviews welcome.
In Pike 8.1 I checked in a few new native Sql types. This allows for faster, easier and lossless manipulation of dates/times/intervals. It also facilitates binary transfer to and from the database.
Ok.
I currently implemented all this for the pgsql driver. The new code still needs documentation. Reviews welcome.
Did you take a look at the similar support in Sql.odbc?
/grubba
Henrik Grubbstr?m (Lysator) @ Pike (-) developers forum wrote:
In Pike 8.1 I checked in a few new native Sql types. This allows for faster, easier and lossless manipulation of dates/times/intervals. It also facilitates binary transfer to and from the database.
Ok.
I currently implemented all this for the pgsql driver. The new code still needs documentation. Reviews welcome.
Did you take a look at the similar support in Sql.odbc?
Actually, I did. But that approach has some drawbacks: - The TOD type stores as separate values for hour/minute/second/nanos. This would require conversion calculations for every value arriving from the database, and conversion calculations for every value going to the database. That sounds like undesirable overhead. - Then there is timestamp_factory(), which uses Calendar. It produces something, and I'm sure it's correct, but it's very far from anything that can go in and out the database without a lot of conversions. - I'm not sure what scale_numeric() does in that list.
On the positive side: - The user_defined_factory() might be nice to use, it allows for dynamic creation and handling of user defined types. But this is more a convenience for user defined types, than that it is efficient to handle SQL-native types with this.
I do notice that I used microseconds, whereas nanoseconds are being used here. Since some databases handle up to nanosecond resolution, I'll change the generic interface to accept nanoseconds instead of microseconds.
After some consideration, I decided to bite the bullet, and propose introducing native Pike datatypes (inspired by the SQL standard):
Time Timestamp Date Interval TimeTZ Inet Range
As well as some new supporting Vals: Val.nan Val.posinfty Val.neginfty
See my most recent commit on 8.1.
The proposed types are lightweight, fast (because I recently fixed mktime) and capable of holding the SQL equivalents; but can, of course, be used outside of the SQL context.
I added some minimal docs, but I don't see all of those appear. There seems to be something special with __builtin docs? Or am I simply doing something silly?
Is this spot agreeable for where the types should live? If not, any other spot that would seem more appropriate? I'd like to avoid putting them down under Sql somewhere, because they're more generic (and useful) than that.
As for the code itself: I tried moving it into separate .pike files, but that didn't go down well because of cyclic references; maybe it's solvable, but this seemed like the only way to get it working for now.
Also, some of the create() variants insist on generating warnings. I tried very hard to get rid of those, but it seems like that is impossible; so, presuming that the code is correct as is, the warnings probably need to be fixed in the compiler.
Math.nan and Math.inf (with -Math.inf working) already exist.
Mirar @ Pike developers forum wrote:
Math.nan and Math.inf (with -Math.inf working) already exist.
Good point. In theory they're different, but maybe they can be used for both. I'll see if I can reuse them.
After some consideration, I decided to bite the bullet, and propose introducing native Pike datatypes (inspired by the SQL standard):
Time Timestamp Date Interval TimeTZ Inet Range
Where are they in the name space?
Code quality request: no nested ?:-expressions. Functions like this ought to be a series of if-return statements.
protected int(0..1) `<(mixed that) { return intp(that) ? nsecs < [int]that * NANOSECONDS : floatp(that) ? nsecs < [float]that * NANOSECONDS : objectp(that) && nsecs < ([object]that)->nsecs && !zero_type(([object]that)->nsecs); }
Martin Nilsson (Coppermist) @ Pike (-) developers forum wrote:
Time Timestamp Date Interval TimeTZ Inet Range
Where are they in the name space?
They are created in __builtin: e.g. __builtin.Timestamp etc. They are made available in the root through a constant linking Timestamp to __builtin.Timestamp. It should allow you to write simple code like:
Timestamp my_new_timestamp = Timestamp();
Code quality request: no nested ?:-expressions. Functions like this ought to be a series of if-return statements.
protected int(0..1) `<(mixed that) { return intp(that) ? nsecs < [int]that * NANOSECONDS : floatp(that) ? nsecs < [float]that * NANOSECONDS : objectp(that) && nsecs < ([object]that)->nsecs && !zero_type(([object]that)->nsecs);
Code readability is relative. I actually find this code (if formatted pleasantly) easier to parse than multiple if() constructs.
But I'll change it, if it's easier for the majority to have if() constructs instead.
You're not alone thinking that ?: construct is easier to read.. :)
Weirdos.
Martin Nilsson (Coppermist) @ Pike (-) developers forum wrote:
Time Timestamp Date Interval TimeTZ Inet Range
Where are they in the name space?
They are created in __builtin: e.g. __builtin.Timestamp etc. They are made available in the root through a constant linking Timestamp to __builtin.Timestamp. It should allow you to write simple code like:
Timestamp my_new_timestamp = Timestamp();
Those are way to generic to be made global without a real discussion. If they have been checked into 8.1 then expect them to be backed out as soon as I need to do a dist.
Peter Bortas @ Pike developers forum wrote:
They are created in __builtin: e.g. __builtin.Timestamp etc. They are made available in the root through a constant linking Timestamp to __builtin.Timestamp. It should allow you to write simple code like:
Timestamp my_new_timestamp = Timestamp();
Those are way to generic to be made global without a real discussion. If they have been checked into 8.1 then expect them to be backed out as soon as I need to do a dist.
Well, they currently are not referenced from within the rest of Pike, except through their __builtin.* references. The global namespace usage will only be used from userspace. So if we decide to move them elsewhere, that's fine. It's just that I myself couldn't come up with anything better than simply a global type; anything else seemed silly from a user standpoint.
Those are way to generic to be made global without a real discussion. If they have been checked into 8.1 then expect them to be backed out as soon as I need to do a dist.
Well, they currently are not referenced from within the rest of Pike, except through their __builtin.* references. The global namespace usage will only be used from userspace. So if we decide to move them elsewhere, that's fine. It's just that I myself couldn't come up with anything better than simply a global type; anything else seemed silly from a user standpoint.
There are other sets of basic types. Standards.BSON.Timestamp, Standards.ASN1.Types.UTC etc. Serializing and deserializing data is a recurring theme, so we should probably spend more time thinking on it on a higher level.
Right now I would prefer to not compile and put that code in the global name space every time Pike is started. Perhaps put it in Val for now.
Martin Nilsson (Coppermist) @ Pike (-) developers forum wrote:
Well, they currently are not referenced from within the rest of Pike, except through their __builtin.* references.
The global namespace usage will only be used from userspace. So if we decide to move them elsewhere, that's fine. It's just that I myself couldn't come up with anything better than simply a global type; anything else seemed silly from a user standpoint.
:-).
There are other sets of basic types. Standards.BSON.Timestamp, Standards.ASN1.Types.UTC etc. Serializing and deserializing data is a recurring theme, so we should probably spend more time thinking on it on a higher level.
Yes, please. I know and have seen some of those; then again, a lot are hidden in various places in Pike. Consider this my first attempt to collect these more or less generically occurring types in a central place from where they can be easily reused by all libraries *and* endusers.
Right now I would prefer to not compile and put that code in the global name space every time Pike is started. Perhaps put it in Val for now.
Forgive my ignorance, but if they are linked in through the defined constants in the global module.pmod; is it then just as heavy during startup as if the whole objects would have been defined in there themselves?
Or is that lightweight and will not incur a large runtime penalty (memory or time-wise) until one of the object types are accessed?
Stephen R. van den Berg wrote:
Martin Nilsson (Coppermist) @ Pike (-) developers forum wrote:
Right now I would prefer to not compile and put that code in the global name space every time Pike is started. Perhaps put it in Val for now.
Another option would be to make two (or more) versions of these types: - One very basic version without operators in a easily accessible global spot. That should be very small codewise (the objects themselves only store one to max three integers) and even more lightweight to compile. - Another more advanced version of these types can inherit the basic version and then add operators.
For the SQL subsystem, the basic versions would be fine, we merely need a way to store and retrieve the values. Not even _encode/_decode is needed. Having a cast-to-string that is SQL parseable is a plus, but even that could be stripped and moved into an SQL private inherit of these types.
pike-devel@lists.lysator.liu.se
-
Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum -
Martin Nilsson (Coppermist) ＠ Pike (-) developers forum
-
Mirar ＠ Pike developers forum
-
Peter Bortas ＠ Pike developers forum
-
Stephen R. van den Berg