I'd like to solicit some design feedback for a new feature that I'd like to contribute. I apologize for the length of this message, but hopefully it contains all of the details needed to get the conversation started.

First, some quick background:

Over the past few years I've written a few modules[1] that provide access to a number of technologies that are either part of Darwin or one of Apple's Darwin-based OSes (OSX, iOS, etc). A common feature that keeps occurring is their reliance on an active run loop. Normally, this is either NSRunloop or CFRunLoop (they are distinct APIs, but they share a common set of event sources.) In the past, I've resorted to adding a persistent call-out to the default back end, which is similar to the approach taken by the GTK2 module. This is normally good enough for sitatuions where events are relatively infrequent, such as GUI-input work.

However, it's not always suitable for higher interrupt volumes and it also has the side effect of presenting a continual, though low, load on the CPU. This isn't a huge problem, but it's unclean and if one were to increase the frequency of the poll-out to the CF/NS run loop, the load becomes higher, and thus harder to ignore.

A Solution:

After doing some research on better ways to approach the problem, I discovered that CF/NS run loops use kqueue at their core. As a result, a kqueue can be added to the run loop and polled for events. That's convenient, because pike uses kqueue on Darwin in its default backend, Pike.PollDeviceBackend. Swapping out the direct call to kevent() with one that calls the run loop results in "native" Pike I/O working the way it should, plus Darwin layered functionality working properly as well (and low CPU utilization, too).

I think that this is a useful new feature to add to Pike, but I've got some concerns about how best to include it. Right now, I've created a branch called hww3/cfrunloop_backend[2] that includes the functionality and swaps out the existing implementation under Darwin. That is, of course, the simplest approach, but I think is probably not the best idea:

Pros:

- Uses kqueue() at its core, so the existing backend will fire events in the same way.

- Allows a number of technologies to function normally when used from with Pike processes, without the use of hacks.

Cons:

- There is exactly one Run Loop per thread, which, if used universally, could present issues because this means behavior would differ from other Backends.

- There is slightly more overhead per trip through the backend due to events being single-shot. This means that the kqueue has to be re-added to the runloop each loop. I don't think it's a major overhead, it still exists.

Alternate Implementations:

A second approach would be to provide an additional backend implementation, say Pike.CFBackend, that's available on Darwin based systems. I see one or two problems with this, though:

- There would need to be a way to activiate this selectively at runtime, something like set_default_backend(Pike.CFBackend()), which probably doesn't exist right now, right?

- Because 90% of the implementation would be shared with PollDeviceBackend, I think it would involve a fair amount of code duplication inside of backend.cmod, which I think would make maintenance more difficult.

A third (and right now, I believe the best) approach would be to add one or two methods to Pike.Backend that could be implemented by PollDeviceBackend on Darwin and be used to activate this alternate mode.

For example:

int(0..1) has_client_mode()

which would return true if there were a client optimization (I suggest client because CFRunLoop would be more useful for client apps than traditional server based apps, though that's certainly not an either/or situaiton) and:

void enable_client_mode(int(0..1))

that would trigger calls to CFRunLoopRunInMode() instead of kevent() directly.

Input Sought:

I think either of the two alternative approaches would be just fine, assuming that the concerns could be addressed. I'm just not sure which would be best from a design and implementation standpoint (I sort of think the last option would be simplest for _me_ to get working, but if someone has desire to help tackle it, I'd be game to play along).

The upshot of allowing the proposed backend changes for other folks using Pike are that:

a) SDL and GL becomes usable out of the box on OSX b) I am prepared to contribute a few pieces of code, namely: - FSEvents module - Updated Filesystem.Monitor that uses FSEvents on Darwin and inotify on Linux to eliminate polling - DNS_SD querying support on OSX - ObjectiveC bridge (possibly)

Hopefully we can have a bit of discussion about the options and approaches and come up with a solution that works without being terribly inconvenient. Please do drop me a note if anything about this proposal is unclear.

Best,

Bill

[1] Public.ObjectiveC, Public.System.FSEvents, Public.IO.IOWarror, additions to DNS-SD; SDL and GL on OSX also share this requirement. [2] http://hg.welliver.org/pike/changesets, not yet pushed to the pike master repo.