Changes View performance problems ~1400 items

[cblgh]

Here are some notes I took after running it for the first time on a big project, with Unity 5.5.1f1:

Commits aren't applied when you have > 2k items and you select the folder only (no commit is made, and nothing is seen in the history tab)
Workaround: select the items in the folder, one by one, and it will work

To make more commits in the changes tab, you have to switch from it to the history tab and back again, and the changes tab will update both the view and unlock the commit button

Changes tab is incredibly unresponsive (latency of 2-3 seconds between every action) when > 2k items changed
note: Sluggishness disappeared around <1400 items

And finally, a question I couldn't find an answer to is you are supposed to clone a project, a regular git clone fails due to needing to run some kind of git-lfs command, of which I'm not sure what to run.

Error during regular clone:

C:\project_name>git clone https://github.com/cblgh/project_name.git
Cloning into 'project_name'...
remote: Counting objects: 1778, done.
remote: Compressing objects: 100% (1016/1016), done.
remote: Total 1778 (delta 786), reused 1746 (delta 754), pack-reused 0Receiving objects:  96% (1707/1778), 628.0
Receiving objects: 100% (1778/1778), 1.07 MiB | 1.11 MiB/s, done.
Resolving deltas: 100% (786/786), done.
Checking connectivity... done.
Downloading Assets/ADG_Textures/ground_vol1/ground1/ground1_Ambient_Occlusion.tga (1.00 MB)
Error downloading object: Assets/ADG_Textures/ground_vol1/ground1/ground1_Ambient_Occlusion.tga (08da51f84430900bc7856459ec5e942e228f408d6de991b0eb13c03fc4ebd241)

Errors logged to C:\project_name\.git\lfs\objects\logs\20170424T185315.7141229.log
Use `git lfs logs last` to view the log.
error: external filter git-lfs smudge -- %f failed 2
error: external filter git-lfs smudge -- %f failed
fatal: Assets/ADG_Textures/ground_vol1/ground1/ground1_Ambient_Occlusion.tga: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

With the contents in the git-lfs log file being:

git-lfs/2.0.2 (GitHub; windows amd64; go 1.8; git 85e2aec4)
git version 2.8.1.windows.1

$ git-lfs.exe smudge -- Assets/ADG_Textures/ground_vol1/ground1/ground1_Ambient_Occlusion.tga
Error downloading object: Assets/ADG_Textures/ground_vol1/ground1/ground1_Ambient_Occlusion.tga (08da51f84430900bc7856459ec5e942e228f408d6de991b0eb13c03fc4ebd241)

Smudge error: Error downloading Assets/ADG_Textures/ground_vol1/ground1/ground1_Ambient_Occlusion.tga (08da51f84430900bc7856459ec5e942e228f408d6de991b0eb13c03fc4ebd241): batch response: This repository is over its data quota. Purchase more data packs to restore access.
github.com/git-lfs/git-lfs/errors.newWrappedError
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:166
github.com/git-lfs/git-lfs/errors.NewSmudgeError
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/errors/types.go:252
github.com/git-lfs/git-lfs/lfs.PointerSmudge
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/lfs/pointer_smudge.go:68
github.com/git-lfs/git-lfs/lfs.(*Pointer).Smudge
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/lfs/pointer.go:64
github.com/git-lfs/git-lfs/commands.smudge
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:63
github.com/git-lfs/git-lfs/commands.smudgeCommand
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:91
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:477
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:551
github.com/git-lfs/git-lfs/commands.Run
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/src/github.com/git-lfs/git-lfs/commands/run.go:68
main.main
	C:/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:35
runtime.main
	C:/Go/src/runtime/proc.go:185
runtime.goexit
	C:/Go/src/runtime/asm_amd64.s:2197

ENV:
LocalWorkingDir=C:\project_name\domkyrkan
LocalGitDir=C:\project_name\domkyrkan\.git
LocalGitStorageDir=C:\project_name\.git
LocalMediaDir=C:\project_name\.git\lfs\objects
LocalReferenceDir=
TempDir=C:\project_name\.git\lfs\tmp
ConcurrentTransfers=3
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic
UploadTransfers=basic
GIT_DIR=.git
GIT_LFS_PATH=C:\Program Files\Git LFS

As a final aside, looks like I went over the limit quite a bit as I found this in my email:

Git LFS has been disabled on your personal account cblgh because you’ve exceeded your data plan by at least 150%. Please purchase additional data packs to cover your bandwidth and storage usage:

  https://github.com/account/billing/data/upgrade

Current usage as of 24 Apr 2017 04:38PM UTC:

  Bandwidth: 0.0 GB / 1 GB (0%)
  Storage: 3.21 GB / 1 GB (321%)

I guess that's that for experimenting with git-lfs in the meantime! Hope the above details are of some use though.

[shana]

Thanks for the report, this is really useful! Clearly we still have a lot of work to do on the whole commit workflow.

Regarding the cloning problem, this is what looks like happened, let me know if it matches or not with what you did:

• You have an existing project that's around 3.21GB
• You initialized the repository with GitHub for Unity, committed all the files, and pushed
• The LFS free limit is 1GB of storage and 1GB of bandwidth, so the upload exhausted those limits
• When you cloned, the LFS download failed because of the limits being used up, so you got the clone error

Does that sound right? Is your project around that size, or is it larger?

To get things working again, you'll need to add a data pack to your account (they're 50GB packs, so that should take care of your limits).

In relation to bandwidth and data packs: When you need to do multiple clones of large repositories, you'll likely need to add an extra data pack just for that time to cover the extra bandwidth required (you can add and remove data packs as needed).

FWIW, the fastest way to clone a repository is with git lfs clone directly, which will download things in parallel (instead of one by one like git does). This wouldn't have helped since LFS was already disabled due to the limits, but just for future reference when cloning large repos :)

[cblgh]

Glad it could be of assistance! It sounds like that's what happened, and yeah the project's around that size.

I was just doing some research because it would help a bunch in collabs, but I think the current state of things would be too much for the modeller I'm working with at the moment.

Thanks for the tips & info!

[shana]

Thanks for the feedback! We're building up to a beta soon, and some features are still missing in the whole workflow, notably the ability to clone a repository directly from Unity, as well as publish an existing project (creating a repository on the server and pushing things). Once those are in (as well as a lot of bug fixes), hopefully that will make it easier to start using it.

[shana]

What do you think would help your modeller, any particular parts or missing features that could make it easier to use?

[cblgh]

Sounds like the beta would do it; basically as long as I could set up the project, and help my modeller with making a github account I'm confident I could teach them the basic git workflow of "pull, commit, push", But any troubles in that basic loop would render the workflow broken, and diminish the use of the plugin.

As I see it, anyway!

[shana]

Nod nod, makes sense. We'll work at getting this workflow smoother!

[StanleyGoldman]

Changing up the title of this issue

[StanleyGoldman]

@GeorgeZZZZZ sent us this video of the experience.

https://www.youtube.com/watch?v=utQ2wLaOT08

[thorade]

You might want to mention the git LFS free quota limits in the readme of this repository and/or link to the github help pages.
BTW: Do I have to install anything for LFS? Or is that bundled in here?

[StanleyGoldman]

Hey @thorade

That's a good point, I'll make a not to add that in the readme soon.

If you are on Windows the plugin uses it's own version of git and git-lfs.
If are are not on Windows, you need to have these things installed and in your PATH.