Hi list,
OK, so I thought it was time to create a mailing list about the Wotsap .wot updates. I took the liberty to subscribe those I beleive can be interested: Patrick, Henk and Thomas. If you know more people that may be interested, ask them to subscribe at http://lists.lysator.liu.se/mailman/listinfo/wotsap-updates . -users and -dev lists are on their way too. I will get info up on the web pages soon.
Anyway, what triggers the creation of the mailing list today is the upgrade to .wot fileformat 0.2. The main purpose of the new file format is to include information about signature types, and a small change was made at the same time to make the signature list somewhat easier to parse with low-level languages.
A specification for the new file format can be found at
http://www.lysator.liu.se/~jc/wotsap/wotfileformat-0.2.txt
If you find bugs or typos, please tell me.
Due to some test runs, the new file version was used for 2004-11-05.wot. I have now renamed that file and repointed the latest.wot symlink to 2004-11-04.wot. The 0.2 file is however great as an example file, and is now available at http://www.lysator.liu.se/~jc/wotsap/wots/2004-11-05.fileversion0.2.wot .
With the specification, the example file and the mailing list you should be able to implement support for the new file format everywhere needed, I hope. I'd like to suggest that we keep updates in the 0.1 format up to and including 2004-11-30.wot, and let 2004-12-01.wot be in the 0.2 format. Is this OK with everyone?
To summarize the changes:
- The signatures file is changed. It now includes a signature type in the 4 most significant bits, and instead of lists being INDEX INDEX INDEX 0xFFFFFFFF they are now 4 type+INDEX type+INDEX type+INDEX type+INDEX
- A debug file is included in the .wot file.
- GnuPG instead of pksclient is now used to parse and validate the keys. This is needed to get the signature type, but also removes invalid keys and gives us the primary UID instead of just a random(?) one. Because some invalid keys and signatures are removed, the size of the web of trust will shrink on 2004-12-01.
This is meant to be a discussion list, not an announce list. Discuss :)
Jörgen
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi all
First of all: Thank you Jorgen for the mailing list.
interested: Patrick, Henk and Thomas.
Hi to you Henk and Thomas.
Sorry for the trouble I generated by switching to the new .wot fileformat without announcing it. I did not think on the nice scripts depending on the .wot file.
Now the old script is reactivated.
format up to and including 2004-11-30.wot, and let 2004-12-01.wot be in the 0.2 format. Is this OK with everyone?
I have written that date into my agenda so I hope not to overlook it then ;-)
Cheers, Patrick
- -- PGP Vertrauensnetz: http://www.rubin.ch/pgp/weboftrust.de.html
Hi,
Am So, den 07.11.2004 schrieb Jorgen Cederlof um 16:04:
- GnuPG instead of pksclient is now used to parse and validate the
keys. This is needed to get the signature type, but also removes invalid keys and gives us the primary UID instead of just a random(?) one. Because some invalid keys and signatures are removed, the size of the web of trust will shrink on 2004-12-01.
I like the changes. Are the signatures also checked before adding them to the wot?
Thomas
On Sun, Nov 07, 2004 at 17:16:25 +0100, Thomas Butter wrote:
Am So, den 07.11.2004 schrieb Jorgen Cederlof um 16:04:
- GnuPG instead of pksclient is now used to parse and validate the
keys. This is needed to get the signature type, but also removes invalid keys and gives us the primary UID instead of just a random(?) one. Because some invalid keys and signatures are removed, the size of the web of trust will shrink on 2004-12-01.
I like the changes. Are the signatures also checked before adding them to the wot?
Yes. pksclient is used to get a key when given a KeyID. Then GnuPG is asked to get the name of the key and the KeyIDs of the keys signing this key. All those keys are concatenated into a keyring and GnuPG is now asked to validate all signatures. The result of that validation (minus keys not in the strongly connected set) is what finally gets into the .wot.
Regards, Jörgen
On Sun, 7 Nov 2004, Jorgen Cederlof wrote:
Date: Sun, 7 Nov 2004 16:04:00 +0100 From: Jorgen Cederlof jc@lysator.liu.se To: wotsap-updates@lists.lysator.liu.se Subject: [Wotsap-updates] Welcome and new .wot file format Sender: wotsap-updates-admin@lists.lysator.liu.se
Hi list,
http://www.lysator.liu.se/~jc/wotsap/wotfileformat-0.2.txt
If you find bugs or typos, please tell me.
May I suggest that the wot version is encoded into the signatures file. I suggest to use the first 4 bytes in the 'signature' files to denote
0 w 1 o 3 t 4 version-index, for example 0 -> 0.1 1 -> 0.2 2 -> 0.2.1 3 -> 0.3
This way the file is 'self contained' and the software can find out how to interpret the file, without resort to the WOTSAP files. If we run out of numbers, the scheme can be extended.
I assume no 0.1 'signatures file starts with 'wot0'.
Any other scheme is ok, as long as the version can be found in the wot file. The first 16 bytes could just be the plain text version id, for instance.
----------------------
Is it the intention do drop revoked keys form the strong set ? Is it the intention do drop expired keys form the strong set ?
Just curious:
Why keep keys/uids that aren't self signed ? Doesn't Gnupg just drop them ? In general, I think the strong set should be as clean as possible. It makes easier to do analysis and it is an incentive for people to keep there keys in order.
Jörgen
regards
Henk Penning
---------------------------------------------------------------- _ Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ _ Dept of Computer Science, Utrecht University T +31 30 253 4106 / _/ \ Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 _/ _/ http://www.cs.uu.nl/staff/henkp.html M penning@cs.uu.nl _/
On Sun, Nov 07, 2004 at 19:32:23 +0100, Henk P. Penning wrote:
On Sun, 7 Nov 2004, Jorgen Cederlof wrote:
http://www.lysator.liu.se/~jc/wotsap/wotfileformat-0.2.txt
If you find bugs or typos, please tell me.
May I suggest that the wot version is encoded into the signatures file. [...] This way the file is 'self contained' and the software can find out how to interpret the file, without resort to the WOTSAP files.
I had never intended the signatures part of a .wot file to be used on its own, and I can think of very few reasons to do so. Since the file format version is encoded in the WOTVERSION part, I see no reason to also include it in the signatures part. Or am I misunderstanding you?
Is it the intention do drop revoked keys form the strong set ? Is it the intention do drop expired keys form the strong set ?
Yes, that is the intention.
Why keep keys/uids that aren't self signed ? Doesn't Gnupg just drop them ?
Such keys are not supposed to be included. Did you find a key like that in the example 0.2 file?
In general, I think the strong set should be as clean as possible. It makes easier to do analysis and it is an incentive for people to keep there keys in order.
I agree fully.
Regards, Jörgen
On Sun, 7 Nov 2004, Jorgen Cederlof wrote:
Date: Sun, 7 Nov 2004 20:14:13 +0100 From: Jorgen Cederlof jc@lysator.liu.se To: Henk P. Penning henkp@cs.uu.nl Cc: wotsap-updates@lists.lysator.liu.se Subject: Re: [Wotsap-updates] Welcome and new .wot file format Sender: wotsap-updates-admin@lists.lysator.liu.se
On Sun, Nov 07, 2004 at 19:32:23 +0100, Henk P. Penning wrote:
On Sun, 7 Nov 2004, Jorgen Cederlof wrote:
http://www.lysator.liu.se/~jc/wotsap/wotfileformat-0.2.txt
If you find bugs or typos, please tell me.
May I suggest that the wot version is encoded into the signatures file. [...] This way the file is 'self contained' and the software can find out how to interpret the file, without resort to the WOTSAP files.
I had never intended the signatures part of a .wot file to be used on its own, and I can think of very few reasons to do so. Since the file format version is encoded in the WOTVERSION part, I see no reason to also include it in the signatures part. Or am I misunderstanding you?
For my 'strong set analysis' I only use (and keep) the signatures. The analysis is 'anonymous', so to speak. It is just not handy to keep the WOTVERSION around just to be able to interpret the signatures file. Note that many types of files put type+version stuff in the file.
Such keys are not supposed to be included. Did you find a key like that in the example 0.2 file?
The 0.2 description says:
The 4-bit signature type is interpreted as:
The two least significant bits are the cert check level (0-3) of the signature on the primary user ID, or the highest cert check level if the primary user ID is not signed. Bit 2 is set if and only if the primary user ID is signed. Bit 3 is reserved and set to zero.
I must confess I don't quite understand the implications, but it seemed that "unsigned uid's" introduce extra complications that could be avoided if such keys/uids/sigs are discarded in the strong set.
Jörgen
HPP
---------------------------------------------------------------- _ Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ _ Dept of Computer Science, Utrecht University T +31 30 253 4106 / _/ \ Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 _/ _/ http://www.cs.uu.nl/staff/henkp.html M penning@cs.uu.nl _/
On Sun, Nov 07, 2004 at 20:35:21 +0100, Henk P. Penning wrote:
On Sun, 7 Nov 2004, Jorgen Cederlof wrote:
On Sun, Nov 07, 2004 at 19:32:23 +0100, Henk P. Penning wrote:
On Sun, 7 Nov 2004, Jorgen Cederlof wrote:
http://www.lysator.liu.se/~jc/wotsap/wotfileformat-0.2.txt
If you find bugs or typos, please tell me.
May I suggest that the wot version is encoded into the signatures file. [...] This way the file is 'self contained' and the software can find out how to interpret the file, without resort to the WOTSAP files.
I had never intended the signatures part of a .wot file to be used on its own, and I can think of very few reasons to do so. Since the file format version is encoded in the WOTVERSION part, I see no reason to also include it in the signatures part. Or am I misunderstanding you?
For my 'strong set analysis' I only use (and keep) the signatures. The analysis is 'anonymous', so to speak. It is just not handy to keep the WOTVERSION around just to be able to interpret the signatures file. Note that many types of files put type+version stuff in the file.
I'd like to make a small analogy. I like anologies. Imagine someone is distributing .png (or .png-like if .png doesn't work exactly like this) images. The images consists of a palette and a compressed bitmap that assigns a color in the palette to each pixel. The image file also contains some meta information, including comments and version information. Each file is self-contained and contains version information and everything else you need. Now, someone uses these images but does not care about what colors are used, so he rips out just the compressed bitmap. Unfortunately, the bitmap is compressed differently in different versions of the file format, and he chose to only rip the compressed bitmap, not the version information. He loses.
In other words: The .wot file is self-contained. It was never designed, and should not be designed, for the possibility that someone would use a small chunk of information from it out of context. Don't let the fact that the file is in fact a compressed ar archive fool you - that's just a way to be lazy/effective and avoid designing, documenting and implementing a new way to store several chunks of information in a single file. So, if you want to get rid of the names and key IDs (to save disk space?), you might create a new compressed archive without those files, you could save the concatenation of WOTVERSION and signatures, or any of a million other solutions. If you feel the need to distribute files like that (although I can't figure out why) I'd recommend following the .wot specification and omitting the files you don't want to include.
Such keys are not supposed to be included. Did you find a key like that in the example 0.2 file?
The 0.2 description says:
The 4-bit signature type is interpreted as: The two least significant bits are the cert check level (0-3) of the signature on the primary user ID, or the highest cert check level if the primary user ID is not signed. Bit 2 is set if and only if the primary user ID is signed. Bit 3 is reserved and set to zero.
I must confess I don't quite understand the implications, but it seemed that "unsigned uid's" introduce extra complications that could be avoided if such keys/uids/sigs are discarded in the strong set.
Ah, now I understand. I was a bit unclear. I'm not talking about self signatures there. I rewrote that section and hope it is clearer now? I put the new version on http://www.lysator.liu.se/~jc/wotsap/wotfileformat-0.2.txt . The changes are below.
Jörgen
--- wotfileformat.txt 7 Nov 2004 14:44:24 -0000 1.8 +++ wotfileformat.txt 7 Nov 2004 22:57:07 -0000 1.9 @@ -25,9 +25,10 @@
2. File format
-A wot file is a bzip2-compressed ar-archive. All texts are coded using -UTF-8, and all integers are in network byte order. The archive -contains these files, in this order: +A wot file consists of the data chunks with names and content listed +below. The chunks are stored, in the same order as below, in a +bzip2-compressed ar-archive. All texts are coded using UTF-8, and all +integers are in network byte order.
"README":
@@ -42,15 +43,16 @@ "names"
One string specifying the name of the primary user ID of each key. - Each name is followed by a newline. The order of the keys is random - and has no meaning, except that the same order is used in all lists - in a specific wot file. The orders in two different wot files are - generally not the same. + Each name is followed by a newline. The inclusion of a key here + implies that it is a valid key which is part of the web of trust. + The order of the keys is random and has no meaning, except that the + same order is used in all lists in a specific wot file. The orders + in two different wot files are generally not the same.
"keys"
Four bytes specifying the key ID of each key. The keys are in the - same order as in the "names" file. + same order as in the "names" chunk.
"signatures"
@@ -63,11 +65,20 @@
The 4-bit signature type is interpreted as:
- The two least significant bits are the cert check level (0-3) of - the signature on the primary user ID, or the highest cert check - level if the primary user ID is not signed. Bit 2 is set if and - only if the primary user ID is signed. Bit 3 is reserved and set - to zero. + If the signing key has signed the primary user ID and one or more + other IDs, all signatures except the one on the primary user ID + are ignored. + + If and only if the signing key has signed the primary user ID, bit + 2 is set and the two least significant bits are set to the cert + check level (0-3) of that signature. + + If the primary user ID is not signed, one or more other user IDs + are. Bit 2 is not set, but the two least significant bits are set + to the highest cert check level of those signatures. + + Bit 3 is reserved and might be given a meaning in future 0.2.x + versions. Set to zero when writing and ignore when reading.
"debug" (optional)
@@ -76,9 +87,9 @@
4. Future extensions
-Future versions will, if possible, only add new files to the archive. +Future versions will, if possible, only add new chunks to the archive. Such versions will have version numbers 0.2.x. Current implementations -must be able to read these files, by ignoring the extra files. If +must be able to read these files, by ignoring the extra chunks. If incompatible changes are introduced, the version number will change to 0.3.
@@ -86,15 +97,16 @@
Version 0.2 - 2004-11-07 * Some clarifications - * Easier parsing of signatures file - signatures file in 0.1 contained: + * Easier parsing of signatures chunk + signatures chunk in 0.1 contained: "For each key, a list specifying the keys that has signed this key. The list elements are 4-byte indices into the above - lists. The lists are in the same order as in the above files. + lists. The lists are in the same order as in the above chunks. The lists are separated by 0xFFFFFFFF." * Specify if primary UID is signed * Specify cert check level * New (optional) debug file + * Renamed internal files as "chunks" to avoid confusion Available at http://www.lysator.liu.se/~jc/wotsap/wotfileformat-0.2.txt with detached signature at
wotsap-updates@lists.lysator.liu.se
-
Henk P. Penning -
Jorgen Cederlof -
Patrick Feisthammel -
Thomas Butter