Henrik Grubbstr?m (Lysator) @ Pike (-) developers forum wrote:

...

How do I solve the namespace problem, I'd like to unite them under master module IO?

The module build system actually supports installing modules in other places than the module root, it's just that most don't. Take a look at eg modules/Parser/configure.in.

...

Renaming the USB module to _USB, then inheriting the module under lib/modules/IO.pmod again might work, but gets messy with autodoc, since I'd like to keep the documentation in the cmod file itself.

*Now* they tell me :-). ...Scurries away moving _PGsql.PGsql to Sql.PGsql...

Martin Stjernholm wrote:

"Stephen R. van den Berg" srb@cuci.nl wrote:

...

Most logical would be under a module named IO, so it would become IO.USB.devices and IO.OneWire.devices or somesuch.

IO sounds like a container module for all things i/o, but in reality there's already a whole lot of i/o elsewhere (Stdio, Protocols, ...). Do you have any coherent vision for what this new container would contain and not contain?

I considered adding it to Protocols. Then figured that Protocols mostly contains IP based protocols.

Then I thought about what these things are about, and decided that the (in my mind) logical group I'm thinking about all has to do with *hardware* I/O.

So I'm currently implementing USB and OneWire, but I could imagine that other stuff that belongs in this category would be things like: I2C, SPI, 1-Wire, USB, JTAG, MIDI, PC keyboard, UART

(Personally I favor a very flat namespace - the top level is afterall the most convenient one, so let's use it. ;) I've never understood the benefit of Standards.pmod, in particular.)

Well, I can understand both camps. Anyone else got opinions?

Henrik Grubbstr?m (Lysator) @ Pike (-) developers forum wrote:

...

Martin Stjernholm wrote:

...

"Stephen R. van den Berg" srb@cuci.nl wrote:

Then I thought about what these things are about, and decided that the (in my mind) logical group I'm thinking about all has to do with *hardware* I/O.

Hmm... Maybe the top-level module should be named HW or Hardware then?

Well, that is too broad a term, I think. It does prove a point though: that it is apparently not obvious what hierarchy to put modules/classes in.

...

So I'm currently implementing USB and OneWire, but I could imagine that other stuff that belongs in this category would be things like: I2C, SPI, 1-Wire, USB, JTAG, MIDI, PC keyboard, UART

...

...

(Personally I favor a very flat namespace - the top level is afterall the most convenient one, so let's use it. ;) I've never understood the benefit of Standards.pmod, in particular.)

...

Well, I can understand both camps. Anyone else got opinions?

Well, I'd prefer not having them at the top-level since they're mostly somewhat unlikely to be used by a random Pike program, and it makes sense to group them together.

That by itself, is not a very strong argument (I think). The most obvious real reason one would want to group them under a hierarchical module name would be to make the list of modules in the autodoc section not too long and too cluttered with modulenames one would be unlikely to use.

I'd say, if, in essence, it's this, then we should solve this by fixing the presentation of the documentation, to allow a topic oriented grouping (next to the linear listing we have now) which should make things easier to find, without actually moving things into submodules.

For the compiler itself, having every module at top level is not a problem at all. Naming conflicts between modules are unlikely as well, and if there would arise any, the module writer(s) are more than capable to think up a logical separation (possibly moving one into a submodule, or simply renaming it) to solve the issue at hand.

This would mean that preferably, modules should become firstclass citizens (i.e. Standards, Protocols and Parser come to mind).

I have to admit that I find it rather tedious and it does *not* improve readability whenever I have to address some library module using submodule addressing. One tends to get rather long names this way, and it takes up a lot of line real-estate; and usually it doesn't add to clarity, since the only name one would be interested in is the last modulename, not any of the prefixes.

Case in point: for the USB module, I have some subclasses called: USB.devices, USB.Iterator, USB.device, USB.transfer

It makes sense to have these four classes, and it makes sense to have them as subclasses from the USB module; it does not really add anything to have to address them as: IO.USB.devices, IO.USB.Iterator, IO.USB.device, IO.USB.transfer

It would clutter the code, but not provide more information to the reader.

I think it is good with containers that tells you the context for a module. Some modules are misplaced, some containers are misnamed, but I prefer that over having ZPRLQX et.al. popping up in the top name space with no information on if it is a file format, a data type, an image format etc.

Well, you will have to create the namespaces in documentation then, otherwise you don't know which of the top level modules you are looking for or which you don't have to look at.

And the most obvious way to do that is to put the module in a container so you don't have to look at the documentation to at least know what to avoid... :)

What you're describing is categorization. It's not the same thing as an hierarchy - using a hierarchy for categorization limits you to one category only, which is often insufficient. E.g. XML is both a standard and a format that can be parsed, and we can't put it both in Parser.pmod and Standards.pmod (or in an equally relevant Formatter.pmod).

But a worse side-effect from using container modules to improve the documentation is that the names actually used in the code also have to carry around these categories. True, there's the import directive to help in such cases, but it brings its own set of problems.

another thought: the documentation can hide a flat namespace.

on the filesystem the flat namespace could be broken up too by introducing several subdirectories (instead of /modules/Parser.pmod/ it could be just /modules/parsers/) which are added to the search path for modules.

so you get a flat namespace that the user does not see. the only remaining place where the user will see that namespace is when hitting tab in hilfe, or any other tool that lists all the available modules.

greetings, martin.

---

From: Martin Bähr mbaehr@email.archlab.tuwien.ac.at To: Stephen R. van den Berg srb@cuci.nl Cc: "Henrik Grubbstr�m (Lysator) @ Pike (-) developers forum" 10353@lyskom.lysator.liu.se; pike-devel@lists.lysator.liu.se Sent: Saturday, June 9, 2012 12:01 PM Subject: Re: USB.devices OneWire.devices, how to unite it into an IO module?

btw: what's the difference between Pango and GTK2.Pango?

greetings, martin.

I used to have the code in GTK2 module, but the pre_install has been commented out of Makefile.in, and the .pmod files aren't there either.

As a second datapoint, I've created a few modules (not all public, though), that I've placed in Public.IO, and my thinking pretty closely aligns with what Stephen has indicated: modules for operating with non-software I/O devices. Another datapoint from outside Pike: Apple's IOKit is all about device drivers, so I think it's not an uncommon line of thought that I/O typically involves a software-hardware interface in some way.

Though, also keep in mind that not everyone is going to be able to use libusb, so while I don't advocate over-classifying things, it might be somewhat ill-considered to use a generic (USB) name for something that's actually quite implementation specific, unless you were also planning on offering a generic interface to multiple USB-IO backends. Consider this case-in-point: file event notification apis are generally OS-specific, so we have System.Inotify and (soon) System.FSEvents which will hopefully plug into a more universal interface that works no matter what OS you're running.

Bill

On Jun 9, 2012, at 11:16 AM, Stephen R. van den Berg wrote:

Henrik Grubbstr?m (Lysator) @ Pike (-) developers forum wrote:

...

...

Martin Stjernholm wrote:

...

"Stephen R. van den Berg" srb@cuci.nl wrote:

Then I thought about what these things are about, and decided that the (in my mind) logical group I'm thinking about all has to do with *hardware* I/O.

...

Hmm... Maybe the top-level module should be named HW or Hardware then?

Well, that is too broad a term, I think. It does prove a point though: that it is apparently not obvious what hierarchy to put modules/classes in.

...

...

So I'm currently implementing USB and OneWire, but I could imagine that other stuff that belongs in this category would be things like: I2C, SPI, 1-Wire, USB, JTAG, MIDI, PC keyboard, UART

...

...

...

(Personally I favor a very flat namespace - the top level is afterall the most convenient one, so let's use it. ;) I've never understood the benefit of Standards.pmod, in particular.)

...

...

Well, I can understand both camps. Anyone else got opinions?

...

Well, I'd prefer not having them at the top-level since they're mostly somewhat unlikely to be used by a random Pike program, and it makes sense to group them together.

On Jun 9, 2012, at 12:58 PM, Stephen R. van den Berg wrote:

I briefly checked, and as far as I can see, libusb-1.0 is the *only* usb library/interface being offered which is cross-OS. This does not mean that in the future, something else might arise which offers the same, but I'd consider it extremely unlikely. So, we could name it libUSB or similar, but for brevity, USB sounded better.

I wasn't necessarily suggesting I had a better idea, or even really disagree with using USB, and agree that libUSB is kind of an ugly name, but calling it USB suggests that it may be canonical, when there are a number of (pretty important) holes in its functionality that someone else might need to fill with an overlapping bit of code: HID devices, as an obvious example, and something I've been dabbling with in my spare time.

AFAICS, libusb is actively being ported and/or offered on several OSes. But if you know of competing offerings and/or know that libusb has shortcomings which make it likely that some would not want to use it, please speak up.

Well, I was thinking in terms of someone creating a native module for certain functionality, they'd then be forced to pick a different name (example: the various JSON, JSON2, JSON3 modules floating around).

What would be nice if there were a way to have providers that can be registered and then used (much like the Sql.Provider namespace). If the USB module consisted solely of classes and didn't have any module.pmod magic, one could use join-nodes to blend them together, but then they'd need to be in different module roots.

I'm definitely not arguing in favor of a Java-style namespace, but rather advocating that modules going into the non-Public root be designed with consideration to the possibility that there might be overlapping functionality down the road. It's certainly possible that nothing comes out of it, but at least it was thought about.

On Sun, Jun 10, 2012 at 05:16:13PM -0400, H. William Welliver III wrote:

How about whatever.USB.Generic? That way, someone could write more specific driver modules such as whatever.USB.HID, for the libhid library, as an example. Even if it were IO.USB.Generic.Devices(), there's still plenty of precedent for a namespace that deep (Protocols.HTTP.Server.Port|Request, for example). And still way better than a lot of Java code! :)

is the problem that if USB is a cmod then it is not easy to add stuff without messing around with that code?

so you are suggesting to have the top level a pmod directory instead?

not sure i understand the problem, please elaborate.

greetings, martin.

In an ideal world, whatever.pmod/USB.pmod would be a directory. That directory could contain pike classes, pods or .so files. If it were whatever.pmod/USB.so, there's no way to mix-in other modules into whatever.USB. If whatever.pmod/USB.so were moved to _USB.so, you could have whatever.pmod/USB.pmod/module.pmod that inherited _USB, and then other modules could be placed along-side that. I think in that situation pike would convert whatever.USB into a join-node, but my experience is that there can be unexpected results from that (there's a fair amount of magic there, imho).

That, combined with my sense that a library-glue module almost never expands beyond being a library-glue (which seems reasonable to me, really) leads me to my position that glue modules should always be leaf nodes in the module tree rather than branch nodes.

Of course, that's just my 2 cents.

Bill

On Jun 10, 2012, at 10:01 PM, Martin Bähr wrote:

On Sun, Jun 10, 2012 at 05:16:13PM -0400, H. William Welliver III wrote:

...

How about whatever.USB.Generic? That way, someone could write more specific driver modules such as whatever.USB.HID, for the libhid library, as an example. Even if it were IO.USB.Generic.Devices(), there's still plenty of precedent for a namespace that deep (Protocols.HTTP.Server.Port|Request, for example). And still way better than a lot of Java code! :)

is the problem that if USB is a cmod then it is not easy to add stuff without messing around with that code?

so you are suggesting to have the top level a pmod directory instead?

not sure i understand the problem, please elaborate.

greetings, martin.

On Jun 11, 2012, at 4:09 AM, Stephen R. van den Berg wrote:

Forgive my ignorance here (perhaps), but as far as I can determine (without actually having written a driver that accesses a HID over USB), the libusb-1.0 interface offered is so generic that it can be used for any USB device, including HIDs.

No, there are per-OS issues surrounding the way HID drivers work, and it's a bit of a mess. libusb doesn't attempt to solve/deal with them, hence the other library(ies). I don't know if there are other limitations beyond HID devices, most of my interests lie in that area.

...

What would be nice if there were a way to have providers that can be registered and then used (much like the Sql.Provider namespace). If the USB module consisted solely of classes and didn't have any module.pmod magic, one could use join-nodes to blend them together, but then they'd need to be in different module roots.

I'd say there is nothing stopping you from adding USB.HID() or other convenience methods/classes that help you in dealing with HIDs across USB. No need for fancy registering.

If a module exists as USB.so, then there's no way for another module to get mixed in because it can't be physically installed in a sub-location. If USB.pmod were a directory that contained a mix of pike and c level modules, that's a different story, but because classes in c-level modules are housed within a container singleton, it means that the classes in your module couldn't simultaneously follow that pattern _and_ appear in the root of the USB tree. As has been pointed out, it's possible to have the c-level code be in a place holder module (like _USB) and then inherited into the right place via a pike module.pmod, but it still means that any supplementary modules would end up in a module.pmod + dirnode, which sometimes (frequently?) results in odd behavior.

That's the origin of my suggestion that a library glue module not take up the root of a potentially diverse module tree.

Bill

...

/.../ As has been pointed out, it's possible to have the c-level code be in a place holder module (like _USB) and then inherited into the right place via a pike module.pmod,

That would perhaps be a good thing to do.

Yes, and I guess that's pretty standard, in fact.

What kind of odd behavior is that? I'm not aware of any specific problems of late with the dirnode stuff. Well, with the exception that inheriting them won't do what you want, but I don't think that's a serious problem since inheriting modules is quite uncommon.

Well, I don't have an example off the top of my head, though I do know that I've seen a number of situations where you are able to look over the edge and see all of the scaffolding, which has required some effort to work around.

If libusb is as generic as srb says (I haven't looked at it myself), then I think it'd be ok to grab the name "USB" for it. It doesn't seem to get too messy if people start to extend it with more advanced functionality, and it's fairly unlikely that a contender will actually materialize in the foreseeable future. Unless perhaps you have concrete plans to develop some more abstract usb interface?

I wasn't necessarily thinking of something more abstract, though it might be useful to consider other options in the design, which would theoretically make things more complete:

libhid, which does for hid devices what libusb does for non-hid devices openusb, which adds support for solaris as well as hotplugging, which are two pretty nice features.

My opinion here is also based on my general scepticism towards container modules - reserving USB.pmod as a container for all things usb-related isn't something I particularly fancy. So that only leaves the possibility of a "properly abstract" module that would fit better, and whether it cannot be joined with srb's low-level stuff in a decent way.

Well, I'm not as opposed to a certain, reasonable, amount of organization, I guess I share skepticism in a way, though mostly around positioning of a library glue module as something potentially more comprehensive or general.

Bill

"Stephen R. van den Berg" srb@cuci.nl wrote:

Then I thought about what these things are about, and decided that the (in my mind) logical group I'm thinking about all has to do with *hardware* I/O.

Maybe Peripherals, Interfaces, or Bus, then? Although I still believe the distinction between that and Protocols could potentially get blurred.

So I'm currently implementing USB and OneWire, but I could imagine that other stuff that belongs in this category would be things like: I2C, SPI, 1-Wire, USB, JTAG, MIDI, PC keyboard, UART

Technically precise names like these are perfectly good by themselves - the risk of an accidental name conflict with something else is minimal. Adding any hierarchical structure in such cases doesn't help. It can only make it more difficult to find the module since one would have to figure out how the implementor has thought, which in the case of Pike with its rather weak central control means it probably isn't even coherent, since different module writers have had different ideas.

Wikipedia is a good example of a very large flat namespace. Considering its size I think it shows well how relatively seldom name conflicts occur in practice, and how easy it is to find the right definition for a specific term. It would definitely be a whole lot more difficult if they had tried to impose some kind of hierarchical classification system. In particular if there could only be a single hierarchy, as would be the case with pmod directories.

So that just leaves the problem that the top level list could grow long. I think that's non-issue in practice, considering the tiny amount of development in Pike, but if it would become a problem it could be handled with some search tools.