Improved ls #764
Merged
Improved ls #764
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a new broadcast RPC that requests all servers to return metadata for the files they own. This is just the initial commit and RPC is not fully functional yet.
Continuing the work on the metaget_all broadcast RPC. It's now functional, but there's still a handful of "ToDo" comments that need to be addressed.
Fix up the error handling and remove TODO comments from process_metaget_bcast_rpc().
Implement proper error handling for some of the functions related to the metaget_all_bcast RPC. Also fix a few bugs found during testing.
We weren't deleting the child_attr_list array after we were done with it.
Contributor
Author
|
As is typical of my pull requests, there are a handful of comments marked as "TODO" that describe questions I have about the proper way to accomplish something. In particular, I'd like some input regarding the comments on lines 182 and 262 of unifyfs_group_rpc.c. |
Move the tree-walk code out of the process_metaget_bcast_rpc function and into its own function in unifyfs_inode.c Two reasons: 1) we want to keep all the inode manipulation code in one place for convenience and 2) it'll make it easier to re-use that code.
We decided "Exit" was too generic for a label, so now it's called "Exit_Invoke_BMA".
Change unifyfs_invoke_broadcast_metaget_all() to also walk the local tree and include file metadata with the results returned from the RPC call.
Fix a case where we could incorrectly return ENOMEM even though the malloc() succeeded. Also cleaned up several comments.
MichaelBrim
approved these changes
May 22, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implements a new broadcast RPC that commands servers to reply with the file attribute data for all the files
Description
The new RPC is named UNIFYFS_SERVER_BCAST_RPC_METAGET. As stated above, this sends a broadcast to all servers to reply with the file attribute structs for all files they manage.
Note that this is the first (and so far, only) broadcast RPC that requires the recipients to return a significant amount of data. (Prior to this, they only data returned from a broadcast RPC was an error code if something went wrong.) This has required a fair amount of new code for gathering the results from individual children, merging them with those from the parent and propagating all of that back up the broadcast tree. See
merge_metaget_all_bcast_outputsin unifyfs_group_rpc.cMotivation and Context
This is needed to implement 'ls' like functionality. The current unifyfs-ls tool can only list files known by the server handling the ls request. With this change in place, the tool can now list files from all the server processes.
How Has This Been Tested?
Existing unit tests work (
srun -N1 -n1 make check)Further testing was done with a handful of utilities to write files and then trigger the RPC.
Test environment: one Ubuntu 20.04 workstation with 3 VM's also running Ubuntu 20.04. Slurm and OpenMPI are configured on the host and VM's.
Types of changes
Checklist: