I’ve noticed that when using Filesystem.Monitor on a system with inotify there is the possibility of monitors being created on paths that don’t exist.
Specifically, if a directory is created and deleted in very rapid succession, the inotify events are processed after the directory is already gone. The resulting Monitors hang around forever (with their associated polls). It doesn’t appear that any events get triggered for this situation, though I can’t be absolutely sure because I’m not watching for directory changes. My concern is that these phantom monitors can build up over time.
Does it make sense to check for the existence of something before monitoring it? I didn’t want to assume that was desirable because I suppose it’s possible someone might want to watch with the expectation that it will exist later… (maybe?)
Bill
Unless the action is atomic, there is still no guarantee that the path will exist between the time you check and when you set the monitor.
Sent from Yahoo Mail on Android
On Sat, Mar 17, 2018 at 12:28 PM, H. William Welliver IIIwilliam@welliver.org wrote: I’ve noticed that when using Filesystem.Monitor on a system with inotify there is the possibility of monitors being created on paths that don’t exist.
Specifically, if a directory is created and deleted in very rapid succession, the inotify events are processed after the directory is already gone. The resulting Monitors hang around forever (with their associated polls). It doesn’t appear that any events get triggered for this situation, though I can’t be absolutely sure because I’m not watching for directory changes. My concern is that these phantom monitors can build up over time.
Does it make sense to check for the existence of something before monitoring it? I didn’t want to assume that was desirable because I suppose it’s possible someone might want to watch with the expectation that it will exist later… (maybe?)
Bill
Yes, that’s true, but (I think) the problem here is that the monitor is being created without there being a valid stat object, so the stat comparison never changes. I think that if it was at least initially populated with some stat, subsequent checks would detect it.
On Mar 17, 2018, at 2:20 PM, Lance Dillon riffraff169@yahoo.com wrote:
Unless the action is atomic, there is still no guarantee that the path will exist between the time you check and when you set the monitor.
Sent from Yahoo Mail on Android https://overview.mail.yahoo.com/mobile/?.src=Android On Sat, Mar 17, 2018 at 12:28 PM, H. William Welliver III william@welliver.org wrote: I’ve noticed that when using Filesystem.Monitor on a system with inotify there is the possibility of monitors being created on paths that don’t exist.
Specifically, if a directory is created and deleted in very rapid succession, the inotify events are processed after the directory is already gone. The resulting Monitors hang around forever (with their associated polls). It doesn’t appear that any events get triggered for this situation, though I can’t be absolutely sure because I’m not watching for directory changes. My concern is that these phantom monitors can build up over time.
Does it make sense to check for the existence of something before monitoring it? I didn’t want to assume that was desirable because I suppose it’s possible someone might want to watch with the expectation that it will exist later… (maybe?)
Bill
I guess what I was looking for was an answer to whether or not it is a design feature to be able to create monitors on folders that don't exist. I was planning to ensure that new monitors have valid stat objects on creation. That way, if the directory is gone by the time the monitor attempts to create a monitor object for the folder, it will throw an exception and not end up hanging around.
If, however it's intended that a monitor could exist for something that doesn't exist yet, I don't think that my plan can be used to mitigate the problem. I think then that the only real solution would be to filter out non-existent paths from within the inotify callback.
Any thoughts?
March 17, 2018 2:56 PM, "H. William Welliver III" <william@welliver.org (mailto:%22H.%20William%20Welliver%20III%22%20william@welliver.org)> wrote: Yes, that’s true, but (I think) the problem here is that the monitor is being created without there being a valid stat object, so the stat comparison never changes. I think that if it was at least initially populated with some stat, subsequent checks would detect it. On Mar 17, 2018, at 2:20 PM, Lance Dillon <riffraff169@yahoo.com (mailto:riffraff169@yahoo.com)> wrote: Unless the action is atomic, there is still no guarantee that the path will exist between the time you check and when you set the monitor. Sent from Yahoo Mail on Android (https://overview.mail.yahoo.com/mobile/?.src=Android) On Sat, Mar 17, 2018 at 12:28 PM, H. William Welliver III <william@welliver.org (mailto:william@welliver.org)> wrote: I’ve noticed that when using Filesystem.Monitor on a system with inotify there is the possibility of monitors being created on paths that don’t exist. Specifically, if a directory is created and deleted in very rapid succession, the inotify events are processed after the directory is already gone. The resulting Monitors hang around forever (with their associated polls). It doesn’t appear that any events get triggered for this situation, though I can’t be absolutely sure because I’m not watching for directory changes. My concern is that these phantom monitors can build up over time. Does it make sense to check for the existence of something before monitoring it? I didn’t want to assume that was desirable because I suppose it’s possible someone might want to watch with the expectation that it will exist later… (maybe?) Bill
Per the man page, A successful call to inotify_add_watch() returns a unique watch descriptor for this inotify instance, for the filesystem object (inode) that corresponds to pathname.That implies that if the inode doesn't exist yet, then the watch doesn't work. I'm pretty sure it won't keep checking for it to start existing, then grab the inode. So, I think filtering out watches for things that don't exist yet is probably the right answer.
On Tuesday, March 20, 2018, 2:31:50 PM EDT, H. William Welliver III william@welliver.org wrote:
I guess what I was looking for was an answer to whether or not it is a design feature to be able to create monitors on folders that don't exist. I was planning to ensure that new monitors have valid stat objects on creation. That way, if the directory is gone by the time the monitor attempts to create a monitor object for the folder, it will throw an exception and not end up hanging around.
If, however it's intended that a monitor could exist for something that doesn't exist yet, I don't think that my plan can be used to mitigate the problem. I think then that the only real solution would be to filter out non-existent paths from within the inotify callback.
Any thoughts?
March 17, 2018 2:56 PM, "H. William Welliver III" william@welliver.org wrote:
Yes, that’s true, but (I think) the problem here is that the monitor is being created without there being a valid stat object, so the stat comparison never changes. I think that if it was at least initially populated with some stat, subsequent checks would detect it. On Mar 17, 2018, at 2:20 PM, Lance Dillon riffraff169@yahoo.com wrote: Unless the action is atomic, there is still no guarantee that the path will exist between the time you check and when you set the monitor. Sent from Yahoo Mail on Android On Sat, Mar 17, 2018 at 12:28 PM, H. William Welliver III william@welliver.org wrote: I’ve noticed that when using Filesystem.Monitor on a system with inotify there is the possibility of monitors being created on paths that don’t exist. Specifically, if a directory is created and deleted in very rapid succession, the inotify events are processed after the directory is already gone. The resulting Monitors hang around forever (with their associated polls). It doesn’t appear that any events get triggered for this situation, though I can’t be absolutely sure because I’m not watching for directory changes. My concern is that these phantom monitors can build up over time. Does it make sense to check for the existence of something before monitoring it? I didn’t want to assume that was desirable because I suppose it’s possible someone might want to watch with the expectation that it will exist later… (maybe?) Bill
I guess what I was looking for was an answer to whether or not it is a design feature to be able to create monitors on folders that don't exist. I was planning to ensure that new monitors have valid stat objects on creation. That way, if the directory is gone by the time the monitor attempts to create a monitor object for the folder, it will throw an exception and not end up hanging around.
If, however it's intended that a monitor could exist for something that doesn't exist yet, I don't think that my plan can be used to mitigate the problem. I think then that the only real solution would be to filter out non-existent paths from within the inotify callback.
Filesystem.Monitor is supposed to be able to monitor not yet existing paths. No such monitors should however have MF_AUTO or MF_HARD.
/grubba
pike-devel@lists.lysator.liu.se