Using NFS mounted volumes for MacOSX

[devalih]

To improve the performance issues found with both dev.sync.up and dev.up.

NFS is the best method we've found so far for the heavy Open edX devstack on MacOSX.

To use it, follow new README instructions on MacOS

[openedx-webhooks]

Thanks for the pull request, @devalih! I've created OSPR-3987 to keep track of it in JIRA. JIRA is a place for product owners to prioritize feature reviews by the engineering development teams.

Feel free to add as much of the following information to the ticket:

• supporting documentation
• edx-code email threads
• timeline information ("this must be merged by XX date", and why that is)
• partner information ("this is a course on edx.org")
• any other information that can help Product understand the context for the PR

All technical communication about the code itself will still be done via the GitHub pull request interface. As a reminder, our process documentation is here.

[natabene]

@devalih Thank you for your contribution. @jmbowman Can you give this a look when you have a chance?

[jmbowman]

Looks potentially useful, just have some questions and comments. I see this is now the latest suggestion in docker/for-mac#1592 , would be interesting to see how much of an impact it has.

[jmbowman]

Did you encounter any problems with an NFS equivalent of docker-compose-themes.yml, or just not need it? It would be nice to have that to keep functionality comparable to the default dev.up.

Also, you might find some of the changes introduced in https://github.com/edx/devstack/pull/463 useful, for starting just one service and its dependencies at a time (make dev.up.lms, etc.)

[devalih]

I'm using a different theming strategy in my devstack, please feel free to add, change to my PR to align it with yours.

[OmarIthawi]

Thanks @devalih for taking on this effort! It's been super annoying to run devstack on mac so far. I've added a couple of notes.

[OmarIthawi]

Another quick review.

[devalih]

Thank you @OmarIthawi

[OmarIthawi]

@jmbowman I'd love to know your thoughts about this PR, whether it looks like something that can be merged or refactored in the next couple of weeks.

[jmbowman]

Sure, this can be merged as is; we need to make a few minor updates to match changes made since the PR was created, but we can do that in a separate PR later (since this doesn't change any existing behavior). I've created https://openedx.atlassian.net/browse/BOM-1194 to track that work.

[openedx-webhooks]

@devalih 🎉 Your pull request was merged!

Please take a moment to answer a two question survey so we can improve your experience in the future.

[robrap]

@OmarIthawi @devalih: We are considering deprecating this NFS functionality because:

1. No one here at edX has been able to show any improvements using NFS.
2. People have issues with devstack from time-to-time that seem to go away when they stop using NFS.
3. It adds complexity with no observable benefit.
4. Although marked as optional, the current docs lead people on Mac to choose NFS.

Before I initiate deprecation, I was wondering if either of you could shed light on the improvements you are seeing.

• Do you have any benchmarks that show what cases are improved, and by how much?
• Do you know if the improvements still hold, or if non-NFS performance has caught up, and that is why no one seems to be able to tell the difference?
• Any other thoughts about the proposed deprecation?

Thank you!

[OmarIthawi]

Thanks @robrap.

Summary: Please feel free to disable and remove this feature.

Here's why:

• I'm not a MacOSX user so I can't provide good feedback on the actual benefits of NFS.
• At Appsembler we don't use devstack on our Engineer's MacOSX since it was very slow and we often run devstack on GCP via https://github.com/appsembler/sultan. I'm a Linux user, so this isn't a problem for me, but we've seen horrible performance issues that wastes days of work and sometimes weeks because devstack isn't able to start due to the following error: getcwd error on volume mount docker/for-mac#1509
• I worked with Ali to add NFS support on MacOSX but they won't be using devstack Lilac+ at all. So it's completely safe to deprecated. I'll double check with Ali just to be safe.

I don't know if someone else is using NFS, so the DEPR process can be used just in case.

[robrap]

Thanks @OmarIthawi. This is useful context. Please confirm when you hear back from Ali.

@jmbowman: It looks like you had run into the getcwd issue, but I haven't seen much noise around this. Thoughts? Do you think it might be fixed?

[jmbowman]

Summarizing from the DEPR working group discussion on this: the original osxfs Docker back end was replaced by gRPC-FUSE in Docker Desktop Community 2.4.0.0 (released on 2021-09-30), which largely eliminates the need for NFS mounts or other previous file synchronization hacks.