And I feel it's not worth the effort to try to do anything fancy with "secure deletion", mlock etc.
mlock is pretty useless as it only works for root, and one don't want to encourage people to run their processes as root "for security reasons". (I'm sure not everybody agrees with that).
Clearing strings and other memory when they're deallocated is reasonable, if you want to pay the extra cycles. Only question is how configurable that should be. A flag to make a particular string not appear in backtraces (or more generally, by sprintf("%O", s)) seems a little obscure, but it might be useful. Anything more complex than that? I don't think that's a good idea.
The reason why I repeat "System.Memory" is that if the Crypto (and similarly used modules like for instance Stdio) could use generic string objects, all that extra payload as well as functionality could be put in nice secure objects, separate from anything else in the Pike core and then not mess anything else up. Also, these objects are unique and destructive, not shared like strings.
Ponder:
non-crypto part: crypto part:
string s="apelsin"; string key=secure_my_string("apelsin"); ... ... werror("s=%O\n",s);
In this case, if the shared string "apelsin" is flagged, it will be obvious in the non-crypto part that it's a key or password somewhere.
/ Mirar
Previous text:
2003-01-29 09:05: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
And I feel it's not worth the effort to try to do anything fancy with "secure deletion", mlock etc.
mlock is pretty useless as it only works for root, and one don't want to encourage people to run their processes as root "for security reasons". (I'm sure not everybody agrees with that).
Clearing strings and other memory when they're deallocated is reasonable, if you want to pay the extra cycles. Only question is how configurable that should be. A flag to make a particular string not appear in backtraces (or more generally, by sprintf("%O", s)) seems a little obscure, but it might be useful. Anything more complex than that? I don't think that's a good idea.
/ Niels Möller ()
A flag to make a particular string not appear in backtraces (or more generally, by sprintf("%O", s)) seems a little obscure, but it might be useful.
I actually find that the most usefull suggestion yet. Passwords in backtraces is one of my bigger fears securitywise.
/ Peter Bortas
Previous text:
2003-01-29 09:05: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
And I feel it's not worth the effort to try to do anything fancy with "secure deletion", mlock etc.
mlock is pretty useless as it only works for root, and one don't want to encourage people to run their processes as root "for security reasons". (I'm sure not everybody agrees with that).
Clearing strings and other memory when they're deallocated is reasonable, if you want to pay the extra cycles. Only question is how configurable that should be. A flag to make a particular string not appear in backtraces (or more generally, by sprintf("%O", s)) seems a little obscure, but it might be useful. Anything more complex than that? I don't think that's a good idea.
/ Niels Möller ()
For a webserver or other similar server, having a stringtype that does not appear in backtraces is very useful. It would save a lot of special code in roxen, designed to filter out passwords, as an example.
/ Per Hedbor ()
Previous text:
2003-01-29 09:05: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
And I feel it's not worth the effort to try to do anything fancy with "secure deletion", mlock etc.
mlock is pretty useless as it only works for root, and one don't want to encourage people to run their processes as root "for security reasons". (I'm sure not everybody agrees with that).
Clearing strings and other memory when they're deallocated is reasonable, if you want to pay the extra cycles. Only question is how configurable that should be. A flag to make a particular string not appear in backtraces (or more generally, by sprintf("%O", s)) seems a little obscure, but it might be useful. Anything more complex than that? I don't think that's a good idea.
/ Niels Möller ()
That was in principle what I meant with the SecretString class. I don't know if you really meant to make it possible to flag a normal string, but if you do then I think an object implementing string behavior is much better in just about every way.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-29 09:05: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
And I feel it's not worth the effort to try to do anything fancy with "secure deletion", mlock etc.
mlock is pretty useless as it only works for root, and one don't want to encourage people to run their processes as root "for security reasons". (I'm sure not everybody agrees with that).
Clearing strings and other memory when they're deallocated is reasonable, if you want to pay the extra cycles. Only question is how configurable that should be. A flag to make a particular string not appear in backtraces (or more generally, by sprintf("%O", s)) seems a little obscure, but it might be useful. Anything more complex than that? I don't think that's a good idea.
/ Niels Möller ()
I think an object implementing string behavior is much better in just about every way.
Why??? To me, there's a big advantage of having *one* single string type. If the builtin strings aren't good enough and pople start coding their own "SecretString" and "FooString" and "BarString", then we'll get the same mess as C++, where the common cry of war is "Death to all strings but basic_string<>!!!"
When adding support for arbitrary string-like objects to all C modules (in particular the I/O and the Crypto code), I fear you get enough added complexity that you end up with more security bugs than you started with.
So if strings need more features, add them to the vanilla string type, and to functions operating on them. Reasonable improvement suggestions so far are
1. A per-string flag to hide a string in %O output.
2. A global runtime flag that clears all memory at deallocation time.
/ Niels Möller ()
Previous text:
2003-01-30 02:02: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
That was in principle what I meant with the SecretString class. I don't know if you really meant to make it possible to flag a normal string, but if you do then I think an object implementing string behavior is much better in just about every way.
/ Martin Stjernholm, Roxen IS
For starters, the basic strings are shared. They don't have any identity in the way objects have and there thus isn't any place to put a flag on them that will propagate in a sane way. Introducing it would add a level of indirection that affects all strings, not only those that uses the flag. It'd also be incompatible with the current strings and we'd have to find seek out every place in the C code that compares strings for equality with svalue1->u.string == svalue2->u.string.
The compatibility issue aside, I think it's designwise ugly to clutter up the default string implementation with features that are needed only rarely, especially if it adds complexity when it isn't used. Objects impersonating strings is a fairly elegant way to avoid that, and I don't see any serious problems handling such objects (speaking from some experience of the Locale.DeferredLocale class which implements localized strings that choose language when they are printed).
Objects are also useful since they provide better typing, e.g. functions like crypt() could only accept a SecretString as the password and thus force the user to not make the mistake of using a normal string for it which is likely to get shown in cleartext in a backtrace. Granted, that can be fixed for builtin strings too by introducing a new builtin type, perhaps "secret_string" or "string(secret)", for them.
/.../ If the builtin strings aren't good enough and pople start coding their own "SecretString" and "FooString" and "BarString", then we'll get the same mess as C++, where the common cry of war is "Death to all strings but basic_string<>!!!"
I suspect this has more to do with C++ strong typing and exceedingly complex operator overloading semantics. What problems are there that we'd get in Pike too?
When adding support for arbitrary string-like objects to all C modules (in particular the I/O and the Crypto code), I fear you get enough added complexity that you end up with more security bugs than you started with.
In what way? The SecretString class appears to be a self-contained data type, and it's fairly easy to reason about the operations it implements.
- A global runtime flag that clears all memory at deallocation
time.
That would cause quite a lot of unnecessary overhead since a program typically handles a lot more strings that don't need it. But of course, given that one adds a flag field to strings (which probably will never happen considering the implementation difficulty), it's not hard to a flag for that too on a per-string basis.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-30 11:24: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
I think an object implementing string behavior is much better in just about every way.
Why??? To me, there's a big advantage of having *one* single string type. If the builtin strings aren't good enough and pople start coding their own "SecretString" and "FooString" and "BarString", then we'll get the same mess as C++, where the common cry of war is "Death to all strings but basic_string<>!!!"
When adding support for arbitrary string-like objects to all C modules (in particular the I/O and the Crypto code), I fear you get enough added complexity that you end up with more security bugs than you started with.
So if strings need more features, add them to the vanilla string type, and to functions operating on them. Reasonable improvement suggestions so far are
A per-string flag to hide a string in %O output.
A global runtime flag that clears all memory at deallocation time.
/ Niels Möller ()
When I said flag, I meant a _shared_ flag. If I set the flag on "foo", all strings "foo" anywhere in the program would be treated as secret by %O. One needs a function to set the flag, but normal programs should never try to clear it.
As for objects, a C module that wants to operate on strings need a way to dig out a char * from the internal representation. You don't want to support more than one internal representation. Typical examples may be Stdio.file->write and Crypt.md5()->update (both may well be invoked with sensitive information as input).
For the general case, the C code would have to allocate a new char (or wchar_t, or whatever) array, probably on the stack, and then copy the contents of the general string object character by character using the `[] operator. You can't do it the easy way by casting the object to a normal pike string, as that defeats the entire purpose of SecretString.
And in the paranoid case, it should of course clear its copy again before it's deallocated. That's one of the uglier aspects: You'd like to put the knowledge that a string should be wiped from memory after it's used in the SecretString class, but still every single C module that handles strings will need to know about that.
/ Niels Möller ()
Previous text:
2003-01-30 14:54: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
For starters, the basic strings are shared. They don't have any identity in the way objects have and there thus isn't any place to put a flag on them that will propagate in a sane way. Introducing it would add a level of indirection that affects all strings, not only those that uses the flag. It'd also be incompatible with the current strings and we'd have to find seek out every place in the C code that compares strings for equality with svalue1->u.string == svalue2->u.string.
The compatibility issue aside, I think it's designwise ugly to clutter up the default string implementation with features that are needed only rarely, especially if it adds complexity when it isn't used. Objects impersonating strings is a fairly elegant way to avoid that, and I don't see any serious problems handling such objects (speaking from some experience of the Locale.DeferredLocale class which implements localized strings that choose language when they are printed).
Objects are also useful since they provide better typing, e.g. functions like crypt() could only accept a SecretString as the password and thus force the user to not make the mistake of using a normal string for it which is likely to get shown in cleartext in a backtrace. Granted, that can be fixed for builtin strings too by introducing a new builtin type, perhaps "secret_string" or "string(secret)", for them.
/.../ If the builtin strings aren't good enough and pople start coding their own "SecretString" and "FooString" and "BarString", then we'll get the same mess as C++, where the common cry of war is "Death to all strings but basic_string<>!!!"
I suspect this has more to do with C++ strong typing and exceedingly complex operator overloading semantics. What problems are there that we'd get in Pike too?
When adding support for arbitrary string-like objects to all C modules (in particular the I/O and the Crypto code), I fear you get enough added complexity that you end up with more security bugs than you started with.
In what way? The SecretString class appears to be a self-contained data type, and it's fairly easy to reason about the operations it implements.
- A global runtime flag that clears all memory at deallocation
time.
That would cause quite a lot of unnecessary overhead since a program typically handles a lot more strings that don't need it. But of course, given that one adds a flag field to strings (which probably will never happen considering the implementation difficulty), it's not hard to a flag for that too on a per-string basis.
/ Martin Stjernholm, Roxen IS
When I said flag, I meant a _shared_ flag. If I set the flag on "foo", all strings "foo" anywhere in the program would be treated as secret by %O. One needs a function to set the flag, but normal programs should never try to clear it.
If a function is supposed to print "foo" (a string which is not secret) and suddenly starts printing "CENSORED", then this leaks the information that another part of the program has a secret string which also happens to be "foo". If there is only one part of the program that sets the flag, and it sets it on keys, they you will have exposed the fact that the key is "foo". That's pretty bad, woudln't you say?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2003-01-30 15:40: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
When I said flag, I meant a _shared_ flag. If I set the flag on "foo", all strings "foo" anywhere in the program would be treated as secret by %O. One needs a function to set the flag, but normal programs should never try to clear it.
As for objects, a C module that wants to operate on strings need a way to dig out a char * from the internal representation. You don't want to support more than one internal representation. Typical examples may be Stdio.file->write and Crypt.md5()->update (both may well be invoked with sensitive information as input).
For the general case, the C code would have to allocate a new char (or wchar_t, or whatever) array, probably on the stack, and then copy the contents of the general string object character by character using the `[] operator. You can't do it the easy way by casting the object to a normal pike string, as that defeats the entire purpose of SecretString.
And in the paranoid case, it should of course clear its copy again before it's deallocated. That's one of the uglier aspects: You'd like to put the knowledge that a string should be wiped from memory after it's used in the SecretString class, but still every single C module that handles strings will need to know about that.
/ Niels Möller ()
No, I don't consider information "leaks" within a process to be a security problem (unless you use the pike security system).
And write("foo") would not be affected, only write("%O", "foo");
/ Niels Möller ()
Previous text:
2003-01-30 15:44: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
When I said flag, I meant a _shared_ flag. If I set the flag on "foo", all strings "foo" anywhere in the program would be treated as secret by %O. One needs a function to set the flag, but normal programs should never try to clear it.
If a function is supposed to print "foo" (a string which is not secret) and suddenly starts printing "CENSORED", then this leaks the information that another part of the program has a secret string which also happens to be "foo". If there is only one part of the program that sets the flag, and it sets it on keys, they you will have exposed the fact that the key is "foo". That's pretty bad, woudln't you say?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
That sounds like a really bad solution.
/ Martin Nilsson (Åskblod)
Previous text:
2003-01-30 15:51: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
No, I don't consider information "leaks" within a process to be a security problem (unless you use the pike security system).
And write("foo") would not be affected, only write("%O", "foo");
/ Niels Möller ()
They are rather serious when you try to be friendly to the developer, and actually show backtraces, since they then tend to be externalized from the process.
/ Per Hedbor ()
Previous text:
2003-01-30 15:51: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
No, I don't consider information "leaks" within a process to be a security problem (unless you use the pike security system).
And write("foo") would not be affected, only write("%O", "foo");
/ Niels Möller ()
Well. Then I don't see any simple way to automatically censor certain strings. Is there any? If you can't do it in a reasonable simple way, I don't think it's worth the effort.
/ Niels Möller ()
Previous text:
2003-01-30 15:54: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
They are rather serious when you try to be friendly to the developer, and actually show backtraces, since they then tend to be externalized from the process.
/ Per Hedbor ()
Just wrap it in a "secrecy object", and overload all operators generating new strings to wrap their results in secrecy objects as well.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2003-01-30 16:00: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Well. Then I don't see any simple way to automatically censor certain strings. Is there any? If you can't do it in a reasonable simple way, I don't think it's worth the effort.
/ Niels Möller ()
Oh yes. Make a SecretString class, inheriting from String, but overloading _sprintf and a few other methods.
/ Per Hedbor ()
Previous text:
2003-01-30 16:00: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Well. Then I don't see any simple way to automatically censor certain strings. Is there any? If you can't do it in a reasonable simple way, I don't think it's worth the effort.
/ Niels Möller ()
But it's still not worth the effort, is it?
/ Martin Nilsson (Åskblod)
Previous text:
2003-01-30 16:02: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Oh yes. Make a SecretString class, inheriting from String, but overloading _sprintf and a few other methods.
/ Per Hedbor ()
I think so. I am actually a bit tempted to improve Roxen with exactly such a thing.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-30 16:03: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
But it's still not worth the effort, is it?
/ Martin Nilsson (Åskblod)
Why not? I really think it is.
And you don't really need to be able to pass *String objects around to each and every C-function that want a string.
But it would be quite useful if, say, read and write at least handled them. And the low-level crypto methods.
Doing so is very easy if my suggestion about the internal structure is used.
All functions using get_all_args and .cmod:s could easily be fixed to handle them without any changes to the actual functions (at least not any major changes).
/ Per Hedbor ()
Previous text:
2003-01-30 16:03: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
But it's still not worth the effort, is it?
/ Martin Nilsson (Åskblod)
It is. I have at least 20 applications that would have use of that.
/ Peter Bortas
Previous text:
2003-01-30 16:03: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
But it's still not worth the effort, is it?
/ Martin Nilsson (Åskblod)
I can accept that if you make sure that C modules need not know the difference between String and pike_string.
/ Niels Möller ()
Previous text:
2003-01-30 16:02: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Oh yes. Make a SecretString class, inheriting from String, but overloading _sprintf and a few other methods.
/ Per Hedbor ()
No, I don't consider information "leaks" within a process to be a security problem (unless you use the pike security system).
They are not within the process if they are printed to a publicly avaiable channel. Take for example a webserver, which uses sprintf("%O") to print forms variables back to the requester. Safe enough. But when someone requests page.html?blurk=foo he gets back "blurk=CENSORED". Now the webserver has inadvertedly and unnecessarily informed the external person that it considers "foo" to be a secret (obvisouly for a completely different reason than that it was sent to it in a forms variable, presumably it's used as some kind of password or key).
And write("foo") would not be affected, only write("%O", "foo");
So?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2003-01-30 15:51: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
No, I don't consider information "leaks" within a process to be a security problem (unless you use the pike security system).
And write("foo") would not be affected, only write("%O", "foo");
/ Niels Möller ()
Well, that's no leak, because everybody already knows that in every webserver "foo" is used as a password somewhere by somebody. ;-)
You do have a point, even if it's not a terribly efficient way to perform a dictionary attack.
/ Niels Möller ()
Previous text:
2003-01-30 15:58: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
No, I don't consider information "leaks" within a process to be a security problem (unless you use the pike security system).
They are not within the process if they are printed to a publicly avaiable channel. Take for example a webserver, which uses sprintf("%O") to print forms variables back to the requester. Safe enough. But when someone requests page.html?blurk=foo he gets back "blurk=CENSORED". Now the webserver has inadvertedly and unnecessarily informed the external person that it considers "foo" to be a secret (obvisouly for a completely different reason than that it was sent to it in a forms variable, presumably it's used as some kind of password or key).
And write("foo") would not be affected, only write("%O", "foo");
So?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
You probably already realize this, but "foo" would of course in practice be some other string, like the name of the admins daughter or something else which is likely to be used both as a secret and occur in non-secret contexts as well.
I did not intend it as a means for "dictionary attack", but as a way of _accidentaly_ gaining this information. "Opportunity makes the thief", no?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2003-01-30 16:02: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Well, that's no leak, because everybody already knows that in every webserver "foo" is used as a password somewhere by somebody. ;-)
You do have a point, even if it's not a terribly efficient way to perform a dictionary attack.
/ Niels Möller ()
For the general case, the C code would have to allocate a new char (or wchar_t, or whatever) array, probably on the stack, and then copy the contents of the general string object character by character using the `[] operator. You can't do it the easy way by casting the object to a normal pike string, as that defeats the entire purpose of SecretString.
Not at all nessesary if the SecretString inherits from, say, String, which in turn is implemented in C, with a storage identical to a struct pike_string, only not shared.
/ Per Hedbor ()
Previous text:
2003-01-30 15:40: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
When I said flag, I meant a _shared_ flag. If I set the flag on "foo", all strings "foo" anywhere in the program would be treated as secret by %O. One needs a function to set the flag, but normal programs should never try to clear it.
As for objects, a C module that wants to operate on strings need a way to dig out a char * from the internal representation. You don't want to support more than one internal representation. Typical examples may be Stdio.file->write and Crypt.md5()->update (both may well be invoked with sensitive information as input).
For the general case, the C code would have to allocate a new char (or wchar_t, or whatever) array, probably on the stack, and then copy the contents of the general string object character by character using the `[] operator. You can't do it the easy way by casting the object to a normal pike string, as that defeats the entire purpose of SecretString.
And in the paranoid case, it should of course clear its copy again before it's deallocated. That's one of the uglier aspects: You'd like to put the knowledge that a string should be wiped from memory after it's used in the SecretString class, but still every single C module that handles strings will need to know about that.
/ Niels Möller ()
struct generic_string *s; switch( Pike_sp[-1].type ) { case PIKE_T_OBJECT: if( FIRST_INHERIT( Pike_sp[-1].u.object ) == string_program ) s = (struct generic_string*)Pike_sp[-1].u.object->storage; break; case PIKE_T_STRING: s = (struct generic_string *)Pike_sp[-1].u.string; }
/ Per Hedbor ()
Previous text:
2003-01-30 15:45: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
For the general case, the C code would have to allocate a new char (or wchar_t, or whatever) array, probably on the stack, and then copy the contents of the general string object character by character using the `[] operator. You can't do it the easy way by casting the object to a normal pike string, as that defeats the entire purpose of SecretString.
Not at all nessesary if the SecretString inherits from, say, String, which in turn is implemented in C, with a storage identical to a struct pike_string, only not shared.
/ Per Hedbor ()
Making pike's builtin string type a true inheritable program would be nice. I mean, that's improving the one true string type, and I'm only arguing against new alternative, independent, string classes. If you understand what I mean.
But I think it's best to consider making string (or other pike builtin types) inheritable classes as an orthogonal issue.
/ Niels Möller ()
Previous text:
2003-01-30 15:45: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
For the general case, the C code would have to allocate a new char (or wchar_t, or whatever) array, probably on the stack, and then copy the contents of the general string object character by character using the `[] operator. You can't do it the easy way by casting the object to a normal pike string, as that defeats the entire purpose of SecretString.
Not at all nessesary if the SecretString inherits from, say, String, which in turn is implemented in C, with a storage identical to a struct pike_string, only not shared.
/ Per Hedbor ()
I did not really suggest that, only that the new string type has an internal structure identical to a pike_string struct, thus making it quite easy to fix the functions that need to handle them.
/ Per Hedbor ()
Previous text:
2003-01-30 15:57: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Making pike's builtin string type a true inheritable program would be nice. I mean, that's improving the one true string type, and I'm only arguing against new alternative, independent, string classes. If you understand what I mean.
But I think it's best to consider making string (or other pike builtin types) inheritable classes as an orthogonal issue.
/ Niels Möller ()
You need a single set of functions in the Pike module API that can handle all kinds of strings, builtin or custom. An individual C module should not need to know.
/ Niels Möller ()
Previous text:
2003-01-30 16:00: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
I did not really suggest that, only that the new string type has an internal structure identical to a pike_string struct, thus making it quite easy to fix the functions that need to handle them.
/ Per Hedbor ()
Why not? They already have to know about bignums.
/ Per Hedbor ()
Previous text:
2003-01-30 16:04: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
You need a single set of functions in the Pike module API that can handle all kinds of strings, builtin or custom. An individual C module should not need to know.
/ Niels Möller ()
I am not suggesting that the SecretString should work everywhere.
And nor am I suggesting that it should not be possible to cast it to a string.
/ Per Hedbor ()
Previous text:
2003-01-30 16:07: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Why not? They already have to know about bignums.
/ Per Hedbor ()
For most other uses for special string classes, it's an acceptable fallback to simply cast them to the builtin string, one could even have functions like get_all_args to that automatically. But if you do that for SecretString, it gets quite pointless.
/ Niels Möller ()
Previous text:
2003-01-30 16:07: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
I am not suggesting that the SecretString should work everywhere.
And nor am I suggesting that it should not be possible to cast it to a string.
/ Per Hedbor ()
Far from pointless, I think, since it still solves the by far more serious problem with accidental printouts in backtraces etc. But a really good solution should address this too, and that's why I wrote PCHARP and not struct pike_string in 9670457. (PCHARP comes from global.h and is a char * together with a size shift.)
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-30 16:14: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
For most other uses for special string classes, it's an acceptable fallback to simply cast them to the builtin string, one could even have functions like get_all_args to that automatically. But if you do that for SecretString, it gets quite pointless.
/ Niels Möller ()
You should get a pointer, a size shift, *and* a length, I hope?
/ Niels Möller ()
Previous text:
2003-01-30 16:24: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Far from pointless, I think, since it still solves the by far more serious problem with accidental printouts in backtraces etc. But a really good solution should address this too, and that's why I wrote PCHARP and not struct pike_string in 9670457. (PCHARP comes from global.h and is a char * together with a size shift.)
/ Martin Stjernholm, Roxen IS
Ah, yes of course. My mistake. Then we could either reuse struct pike_string (without requiring that it points to an interned string), or add a new struct for this (which might be better to avoid mistakes with mixups between interned and uninterned pike_strings).
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-30 16:28: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
You should get a pointer, a size shift, *and* a length, I hope?
/ Niels Möller ()
You could even reuse the struct pike_string, but you have to look out for places that push it, or do anything with refs.
/ Per Hedbor ()
Previous text:
2003-01-30 17:19: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Thus my 'struct generic_string *'
/ Per Hedbor ()
If the string stuff is reorganized, would it make sense to have some base class that implements a non-shared (perhaps even mutable) string, and have the vanilla pike string type inherit that and do the sharing stuff?
/ Niels Möller ()
Previous text:
2003-01-30 17:20: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
You could even reuse the struct pike_string, but you have to look out for places that push it, or do anything with refs.
/ Per Hedbor ()
If the builtin strings internally should be treated as objects then we're talking about a much bigger change in terms of necessary code change and compatibility (consider e.g. the use of runtime types and the svalue struct).
Although it isn't designwise as beautiful, I think it's both simpler and more efficient to just tag objects that want to interface as strings, e.g. through inheriting a magic String class which would act mainly as a flag.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-30 18:07: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
If the string stuff is reorganized, would it make sense to have some base class that implements a non-shared (perhaps even mutable) string, and have the vanilla pike string type inherit that and do the sharing stuff?
/ Niels Möller ()
One interesting trick if one wants to objectify all values and at the same time have good performance, is to make some classes "top-wired". Every class can inherit at most one top-wired class, directly or indirectly.
For each instance, there's a field in the header that says which top-wired class (if any) that is inherited by the object's class. The data associated with that class is always stored first in the instance data area, at a fix offset.
Thus, if strings are top-wired, C code can operate on any subclass
if (o->top_wired == GENERIC_STRING_TYPE) { struct generic_string *s = o->data; ... } else error("Expected string"); /* Or generic code using method calls * instead of accessing the internal representation */
which shouldn't be slower than
if (o->type == PIKE_T_STRING) { ... }
/ Niels Möller ()
Previous text:
2003-01-30 18:18: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
If the builtin strings internally should be treated as objects then we're talking about a much bigger change in terms of necessary code change and compatibility (consider e.g. the use of runtime types and the svalue struct).
Although it isn't designwise as beautiful, I think it's both simpler and more efficient to just tag objects that want to interface as strings, e.g. through inheriting a magic String class which would act mainly as a flag.
/ Martin Stjernholm, Roxen IS
True, but that doesn't solve quite the same problem since it wouldn't make it possible to implement string-like objects that can't store the string data that way. E.g. deferred locale strings need to get a function called to retrieve the right value. It'd be nice to have both variants.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-30 18:31: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
One interesting trick if one wants to objectify all values and at the same time have good performance, is to make some classes "top-wired". Every class can inherit at most one top-wired class, directly or indirectly.
For each instance, there's a field in the header that says which top-wired class (if any) that is inherited by the object's class. The data associated with that class is always stored first in the instance data area, at a fix offset.
Thus, if strings are top-wired, C code can operate on any subclass
if (o->top_wired == GENERIC_STRING_TYPE) { struct generic_string *s = o->data; ... } else error("Expected string"); /* Or generic code using method calls * instead of accessing the internal representation */
which shouldn't be slower than
if (o->type == PIKE_T_STRING) { ... }
/ Niels Möller ()
Not really, no. "real" strings will not be objects.
/ Per Hedbor ()
Previous text:
2003-01-30 18:07: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
If the string stuff is reorganized, would it make sense to have some base class that implements a non-shared (perhaps even mutable) string, and have the vanilla pike string type inherit that and do the sharing stuff?
/ Niels Möller ()
Well, it reduces the problem, but doesn't *solve* it. Consider a C module calling a pike function that raises an exception.
/ Niels Möller ()
Previous text:
2003-01-30 16:24: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Far from pointless, I think, since it still solves the by far more serious problem with accidental printouts in backtraces etc. But a really good solution should address this too, and that's why I wrote PCHARP and not struct pike_string in 9670457. (PCHARP comes from global.h and is a char * together with a size shift.)
/ Martin Stjernholm, Roxen IS
That's not a problem if the "cast" is inside the C function since the value on the stack still would be a SecretString. But you're correct if the pike code would have to look like write((string) secret).
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-30 16:30: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Well, it reduces the problem, but doesn't *solve* it. Consider a C module calling a pike function that raises an exception.
/ Niels Möller ()
And there's also a lot of functions that complains about that it isn't an integer when they get a bignum.
A more abstracted interface to convert whatever is on the stack to the data types the functions want would be a really good thing. Not that it would completely solve the bignum case, but improving the error from "not an integer" to "integer too big (> MAXINT)" or somesuch would still be worthwhile.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-30 16:07: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
Why not? They already have to know about bignums.
/ Per Hedbor ()
As for objects, a C module that wants to operate on strings need a way to dig out a char * from the internal representation. You don't want to support more than one internal representation. Typical examples may be Stdio.file->write and Crypt.md5()->update (both may well be invoked with sensitive information as input).
Yes, the API needs to be improved in this regard. Every object that wants to behave like a string should have a magic constant or inherit, and there should be a standardized C level function get_string() that gives a PCHARP from all such objects and builtin strings.
Something like that is useful anyway to e.g. handle Locale.DeferredLocale better; I would definitely think it'd be foolish to implement deferred locale driven strings directly in the builtin string data type.
And in the paranoid case, it should of course clear its copy again before it's deallocated. That's one of the uglier aspects: You'd like to put the knowledge that a string should be wiped from memory after it's used in the SecretString class, but still every single C module that handles strings will need to know about that.
Yes, this is a problem, but it's the same one in either case. Is it even possible to ensure that buffers in various libraries are cleared in a timely way? Anyway, clearing the memory used by the string implementation itself is still a lot better than nothing.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-30 15:40: Subject: Re: OpenSSL wrapper vs Pike's SSL (Was: Bz2)
When I said flag, I meant a _shared_ flag. If I set the flag on "foo", all strings "foo" anywhere in the program would be treated as secret by %O. One needs a function to set the flag, but normal programs should never try to clear it.
As for objects, a C module that wants to operate on strings need a way to dig out a char * from the internal representation. You don't want to support more than one internal representation. Typical examples may be Stdio.file->write and Crypt.md5()->update (both may well be invoked with sensitive information as input).
For the general case, the C code would have to allocate a new char (or wchar_t, or whatever) array, probably on the stack, and then copy the contents of the general string object character by character using the `[] operator. You can't do it the easy way by casting the object to a normal pike string, as that defeats the entire purpose of SecretString.
And in the paranoid case, it should of course clear its copy again before it's deallocated. That's one of the uglier aspects: You'd like to put the knowledge that a string should be wiped from memory after it's used in the SecretString class, but still every single C module that handles strings will need to know about that.
/ Niels Möller ()
pike-devel@lists.lysator.liu.se
-
Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) ＠ Pike (-) developers forum -
Martin Nilsson (�skblod) ＠ Pike (-) developers forum
-
Martin Stjernholm, Roxen IS ＠ Pike developers forum
-
Mirar ＠ Pike developers forum
-
Niels M�ller () ＠ Pike (-) developers forum
-
Per Hedbor () ＠ Pike (-) developers forum
-
Peter Bortas ＠ Pike developers forum