#34 all files in subfolders not under repository control

hello all,

i used "gin get Marci/myrepo" to create my repository on my desktop. Uploading files in this folder works great as well. My problem arises when i copy (for example) my Data-folders into this repository. "gin ls" reports all these files as "??" - so as not being under repository control - and thus "gin upload" will not do anything.

How do i add these folders and files to the control of the repository. I can not find a direct command for it. Or did i do a mistake in setting up the repository?

im sorry for this very basic question. It seems like im missing something here and i just cant figure it out.

best regards, Marcel

Hello Marcel, by default gin upload only considers files that are already tracked by gin. Using gin commit you should be able to add also files in subfolders to the list of tracked files. For all possible parameter options see gin commit --help. Alternatively, you can also directly specify files upon upload, see gin upload --help.

Hello Marcel,

Sorry if some of these are obvious, but it's best to check the common issues first:

Do you perhaps have a .gitignore in the repository that filters your data filetypes or some pattern that matches their names?

Do you get any output when you try to do gin commit <data folder name> ?

What about gin annex add <data folder name> ?

Do the untracked files appear in git status?

thank you so much for the fast reply sprenger and Achilleas.

i did in fact accidentally copy the .gitignore file as well.... but removing it did not help.

'gin commit' actually worked! it added the untracked files to "OK" files! unfortunately it seems to stop continuing/responding after around 1364 files for some reason. which is is bit of a problem with 80,000 files

'git annex add' worked great! After a long time it seemed to stop responding at one point too, but i think i made it work now.

thank you again for the help!

Becoming unresponsive when adding thousands of files is something I've seen before. As far as I can tell, the root cause is in git-annex, though calling it directly can sometimes be a bit better as you discovered, so the gin-cli can also use some improvement there.

The only workaround for now is to see if you can add the files in smaller batches, but I understand that might take a while too or it might be too tedious.

I'm going to close this issue now since the original problem was resolved, but if there are any further issues or if the checking in of files keeps hanging let me know (perhaps in a separate issue). It might help find some other cause.