Code Librarian reads raw cvs repositories, and the version running at the pike site is sadly outdated nowadays.

Current state of the art Code Librarian flourishes at Opera Software ASA, where it's replaced (and somehow in the process adopted the name of) a legacy hack called cvsdoc. I think there has been talk about packaging it for an (inofficial) release, possibly in time for the conference.

Most of the reason Code Librarian exists is because cvs (and not yet covered svn) really stink for the kind of visualization Code Librarian provides, in a way I think git doesn't, though. (Not that it wouldn't be nice with git coverage, of course; we're creatures of habit, after all.)

(Not that I've really tried the git query tools yet; my X11 is broken, and I haven't found any with a native mac backend.)

i also wonder why they have been deleted from the repository itself instead of using cvs to remove them, which should have placed them into the attic.

For each new minor we copy the repository and clean things up. We don't care about being able to check out a compilable 7.6.66 from the 7.7 repository, but we do care about the file content history. So files are not ofter so much deleted as they are moved manually inside the repository. Fiddling with CVS commands has not been a priority, because CVS can't handle the history we wanted any way.

On Thu, Oct 11, 2007 at 03:05:01PM +0000, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum wrote:

There is nothing magical with the "trunk" or "branches" directories in Subversion.

of course not, but it seems a bit odd to have branches in /branches/ and in /Pike/ and then also non branches in /Pike/

And those conventions didn't really fit the development model of Pike up until now

could you elaborate on that please? which part of the development model needs a specific layout for branches?

then all the Attics were removed.

that's something i am wondering about. why not just leave them? well, no matter, they are gone...

generally it was possible to pinpoint between which two regular commits that such an operation should be inserted. It did take a lot of detective work though, mind you.

i can believe that. i'll take a closer look at that then and see what i can make of it. hopefully i can import from svn one branch at a time and then merge them like i did with the cvs import. i'll give that a go.

See above. The files were removed using cvs remove, and then the attic was cleaned out after the split. The ,v file containing the remove operation still exists in the attic in the repository of the previous Pike version.

no, that is the confusing thing, some files that do exist in a newer version were removed from older versions.

Modifications to the repository between splits consists mostly of moves/renames since CVS provides no useable facility to do that.

detecting moves and renames was actually rather trivial. git does automatic rename detection helping me discover non-obvious moves...

greetings, martin.

how is that part of the development model? they are still branches, wether that information is stored in the repository or not.

what effect does it have on development if the repository stores them as branches? especially in svn where branches are just a convention anyways.

how would following the svn convention change the development model?

the only thing that would change is the path you get when checking out all branches at once. you could check out one branch at a time and have a layout any way you like.

what difference does it make?

greetings, martin.

both those drawings represent the same kind of tree. only that one tree has branches of branches, while the other only has one level of branches:

you could draw the tree like this:

--branch / ---0.5 ---0.6 ---7.0 ---7.2 ---7.4 ---7.6 / / / / / / --------------------------7.1--------7.3------7.5--------------7.7

hence, devel could be seen as the trunk, from which stable releases are branched off.

i don't care about the svn convention, it can be done any way you like. in git the whole issue becomes moot anyways. and the git hirtoey tree looks more like the second example and not like the svn one.

i'd just put all the branches (main and sub ones) into the same directory (whatever the name) and put the nonbranches somewhere else. and if you don't like the trunk convention then don't use it.

what confused me is that there are branches in two places, and non-branches mixed with branches in Pike.

however i can now also see the point of having main branches and sub branches in seperate directories.

in the end however the whole thing for me only matters as it affects how much work i need to do when importing the repo into git. as such i am just trying to understand what's going on here.

greetings, martin.

In a subversion based Pike repository, that would probably have been achieved through renaming a 7.<odd> tree 7.<odd+2>, retaining history while not cluttering up the Pike/ directory.

This is more or less what happens in my svn import. At a split point, 7.<odd+1> and 7.<odd+2> are both created as copies (with history) of 7.<odd>, and 7.<odd> is deleted. This is eqivalent to a rename and a copy, but there is no distinction in which one is the copy and which one is the rename. Therefore there is no "branch" and "nonbranch" distinction between the two either.

no, extra_tests, nt-tools, ulpc, and ulpc.old (although ulpc could be considered a branch)

I don't exactly remember the ancestry for each of these right now, but at least nt-tools comes from inside the devline of Pike itself. There was a split like this (IIRC)

Pike/7.0 (with NT subdir) / Pike/0.7 (with NT subdir) - Pike/7.1 (without NT subdir) \ Pike/nt-tools

What makes 7.0 a branch and nt-tools a non-branch?

my lack of knowledge of the history? :-) i did notice that delete if the NT directory, and i was wondering where that stuff went. well, now i know, thanks :-)

I could give you the control files for my conversion script as they contain this kind of information, but they are probably not so easy for someone other than me to read... :-) If you base your conversion on the svn repository instead you should be able to pick it up from there instead. It's a little outdated by now I'm afraid though.

The directory /pike/home/marcus/cvs2svn-1.3.1 (world-readable) contains my scripts and control files. The conversion is done in multiple steps, because there were different classes of problems which needed different approches to solve. The first step joins the dumps of the different cvs repositories into one merged dump, weeding out duplicate commits (and also verifying that they are indeed identical). This is controlled by the file "specs", which specifies the times when the repositories where copied, and which destructive modifications where done to the dev repository afterwards. The next pass moves the destructive operations forward in time from the split point to where they actually happened, again weeding out any duplicate commits on the way, and checking that they are identical. It also handles copies, and removes some bogus tags and branches. This is controlled by "specs2". Finally there is a "specs3" which handles the ulpc and ulpc.old stuff, which wasn't originally included.

Unfortunalely, this doesn't cover all the bases. There is also some weirdness in the SSL module (IIRC it was originally maintained in a separate repository, but has common tags with the main repo) which were easier to fix by simply stomping on the ,v files a bit. This is done by the sslfix script. (Please don't run this on pelix unless you know exactly what you are doing.) Also, the merger.pike script has some fudge for special situations that cropped up and which didn't lend themselves to any general solutions. This was typically cases where the sanity check would fail to acknowledge that a commit (typically the creation of a tag) was identical in two repositories, but an ocular inspection showed that the differences were immaterial so long as the commit from the older repository was used (which is the MO of the script anyway).

The file /home/marcus/pike_svn.dump.gz on pelix is the data that was imported into svn.lysator.liu.se, and represent the latest run of my conversion script. Dumping https://svn.lysator.liu.se/repos/pike/ will not give you anything newer/better.

But it is OK to use it for experimenting if you wish. The SVN server on Lysator is not very loaded at all.

On Fri, Oct 12, 2007 at 02:55:01PM +0000, Johan Sundstr�m (Achtung Liebe!) @ Pike (-) developers forum wrote:

I may be wrong here, but I think the stuff on pelix is some playground or similar and that Marcus' imported Pike repository resides here: https://svn.lysator.liu.se/repos/pike/

marcus initially announced a dump available at: pelix:/pike/home/marcus/pike_svn.dump.gzi

and then two months later peter announced that this dump would be available on the above url. (search for the thread with "subversion" in the subject line)

I think the way to achieve this kind of thing with subversion is to use its export-to-xml feature and work your magic on that before doing an import-from-xml. I haven't tried, though, so that could be a rumour.

git-svn works directly on the live svn repository. (so does git-cvsimport) it works bidirectional, that is, after importing it can then push commits back into the svn repository. (there is also a git-cvs-exportcommit tool which can commit to cvs, have not tried that yet though. being able to commit back to cvs/svn is one of the goals however, so that i (and anyone else who likes) may use git locally regardless of what the main repository is running)

greetings, martin.

git has support for a number of other revision control systems too. and i hope some of those other systems will also be able to support different systems. (most of them seem to support svn)

the trend that i believe this should head into is that all revision control systems (well at least the distributed ones) can interoperate with each other, so that anyone may choose the tool they like best, and in the end there may even be a common revision control network protocol in some standard form, that allows transfering changesets back and forth. like we have it with smtp and mail servers and clients. (it's a dream, but who knows :-)

greetings, martin.

for the git-svn tool it would be most convenient if all branches lived in the same directory. i'll attempt to move them around. (if you have a quick hint how i can achieve that, i'd appreciate it)

There is a tool called "svndumpfilter" which is part of the regular svn distribution and which can be used to rename stuff in a dumpfile.

On Fri, Oct 12, 2007 at 02:00:01PM +0000, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum wrote:

Yes, but notice that the name of your "bottom line" changes, rather than being "trunk" all the time. So it still is not compatible with the svn convention.

right, don't worry about the svn convention. it doesn't really concern me.

Mainly I just want the stuff in branches/ out of the way, since it isn't relevant any more.

yes, that makes a lot of sense.

...

what confused me is that there are branches in two places, and non-branches mixed with branches in Pike.

And I don't see that, because I consider everything in Pike/ to be non-branches.

ok, we are just using different terminology here. as far as i am aware of 'tag' and 'branch' are terms common to most revision control systems. and i use the terms in that sense. (in svn tags appear to be just another convention, whereas in cvs and git tags are really just markers of a given state. (what happens if you commit a change to a tag in svn? will that change the tag?)

greetings, martin.

ok, we are just using different terminology here.

Yes, sometimes even the same person. :-)

as far as i am aware of 'tag' and 'branch' are terms common to most revision control systems. and i use the terms in that sense.

In CVS, "tag" and "branch" have specific meanings, related to metadata in the ,v file. And the things in "tags" and "branches" in my svn repository happen to match exactly the things which have this metadata in CVS. So no mismatch thus far.

In subversion, "tag" and "branch" are not part of the infrastructure. So "branch", by convention, means "something which was copied from the trunk/ to the branches/ directory". Well, we don't have a trunk/, so this convention can't be applied.

When you say that things in Pike are mixed of branches and nonbranches, you don't seem to use the term branch in either the CVS sense (having a CVS version with an odd number of digits) or the svn sense (residing in the "branches/" directory). Maybe you are using it in the git sense, but I don't know what that is.

(in svn tags appear to be just another convention, whereas in cvs and git tags are really just markers of a given state. (what happens if you commit a change to a tag in svn? will that change the tag?)

Well, if you commit a change to a directory under tags, that directory changes (on head). There's nothing special about this directory. Whether you consider this to be "changing the tag" is up to you, as the tag concept does not exist inside svn itself.

I also tend to think that "branches" are not nearly as convenient as having a repository per version. It's much more visual at the very least.

On Fri, 12 Oct 2007, Johan Sundström (Achtung Liebe!) @ Pike (-) developers forum wrote:

Those are really the same model, except the Pike way calls trunk 7.odd for increasing odd numbers (and rarely, new majors), over time. I, for one, think the signal value of that is enough to merit sticking to the same directory structure in svn, as svn doesn't care either way. That way, you can tell apart a checked-out "trunk of 2003" from a "trunk of 2007", at a glance, just through a quick peek at the directory name (and swiftly do away with the old after a check for uncommitted code).

!DSPAM:470f77a681275209328925!

How would the commit have to look for git-svn to consider it a tag? Since I post-process the result of cvs2svn anyway I could make modifications to how these commits are represented.

i assume in svn you create a tag by copying the tree, but then the cvs2svn importscript seems to create another commit on top of that, making the tag effectively a branch.

Can you give me an example of two such commits (by revision number or otherwise)? As far as I have seen, the tag is always created with a single commit. For example, the tag 'Release-0-2-internal' is created in revision 238, and there are no other checkins which touch the path tags/Release-0-2-internal.

Say what? 19906 modifies version.h. It has nothing to do with the tag. The tag is created in 19907, and _only_ in 19907.

ok, maybe this is a svn problem. for git (and i believe for cvs), a tag marks a specific commit and does not make up a commit in itself.

Well, in CVS, a tag does not mark a specific commit, but N specific commits, where N is the number of files in the project. Those commits may lie on different branches, be made on completely different dates, etc. Marking a specific commit in subversion would not be as flexible, so that's why they didn't implement any such tags, and instead rely on the regular copy mechanism to commit stuff into /tags. That way, you can mix and match versions by creating a commit to your liking.

subdirectories in the attic? how did that happen? :-) did you send those changes upstream?

i suppose a diff to the original 1.3.1 should make them easy to pick out and maybe forward port to 2.0...

greetings, martin.

the point would be to allow to use your changes with the cvs2git feature that is contained in cvs2svn now.

greetings, martin.

detecting moves and renames was actually rather trivial. git does automatic rename detection helping me discover non-obvious moves...

What did it say about README and README-CVS, which are based on the same ,v file but has co-existed at the same time?

means, the ,v file was copied?

Of course. Just as ,v files have been renamed when a file should be renamed. In CVS, this is the only way do it without breaking annotate etc.

well in that case the complete history would have been copied and the content would appear double from the first checkin.

Which means you didn't detect it at all. In my svn import, the copy (with history) is properly represented at the appropriate time, and there are no double checkins.

i did 'reconstruct' the missing files by importing them seperately and then merging the result. i see the pike77added, etc, branches.

greetings, martin.