POSIX file watching doesn't get notified when files are transfered using VOS

[sefffal]

Low priority bug / feature request:

I created a script using Julia's watch_folder function to trigger an action when a file is added to a given directory. Testing using linux commands on the science platform e.g. tough text.txt triggers events, but transferring files using vcp does not. The files appear immediately using ls but no event is generated.

[andamian]

What is the feature request/bug here? Is the vcp command triggered?

[sefffal]

Brian asked me to open this.

When we copy files using vcp to arc, scripts watching that folder for changes are not triggered. The request is for vcp to notify on file changes using the standard system on linux (inotify I believe, see man page https://man7.org/linux/man-pages/man7/inotify.7.html)

[andamian]

Still don't understand what is missing in vcp. If I understand correctly, you copy a file to arc using vcp but inotify is not triggered. Is the directory under watched mounted to arc? Does it work when you upload the file via the Web GUI? Am I misunderstanding the problem here? If so, please provide more details. Thanks

[andamian]

@brianmajor - can you comment more on this?

[sefffal]

Yes, it is mounted to arc.
I am a user running a notebook on the science platform. Arc is mounted and I can access files under /arc/projects/...

I wrote a script that watches for changes to a directory in projects. If I run touch test.txt on the science platform, this change is picked up by the script (I presume through inotify, but this is beyond me to test).

If I instead copy a file to that directory remotely using e.g. vcp the file gets added silently and the script is not triggered.

I had a use case for this functionality last weekend while observing. As data was transferred from the telescope to our projects directory, the script would have picked up the new files and triggered a live analysis of the data.

This is not essential functionality and we were able to work around it, but I reported it in case inotify was already expected to work.

[andamian]

This is not a vcp issue for sure as vcp has no control over that storage FS. It could be a problem with the arc VOSpace service (which Brian is more familiar with) or with the sshfs mounting. Have you tried to manually copy a file into the directory that's being monitored but using the mount point instead of vcp. If that works then it's a problem with arc and the issue should be opened in a different repo (I can assist you with that). Let me know how it works.

[brianmajor]

Yes, I guess since it's a remote upload (rather than a local download) using vcp then it probably is an issue in cavern, not the tool. I'll move this over to vos.git/cavern. Thanks for reporting.

[pdowler]

wild guess: when you put a file, it initially gets created as root and then chown-ed to the user. Does the notify mechanism care about who owns the file when it is created? or somehow not get triggered by root?

I don't think I could find an atomic "create+chown" functionality in the java nio library, but I could dig deeper or probably fake it out by creating the file in another location, chown, then atomic move to the right location in the directory. That would have some side benefits of not leaving a file in an intermediate and possibly broken state if something fails.

[pdowler]

Is this still an issue?

When refactoring some of the code, there was a change in server behaviour where the (empty) file is created and chowned during the transfer (same as before) negotiation. The change is that durting the upload the file is now opened in append mode rather than overwriten (and chown-ed again).

Maybe the notify doesn't get triggered because the crating/writing process is run as root (cavern is a "file server" and thus has to run as root and chown).

Maybe the notify mechanism needs to watch for file modification and will trigger because of that subtle difference?