Local GIN client - Usage Guide

Setup
Basic Workflow
- The workflow
- General usage
Selective sync Workflows
- Selective download
- Selective upload
Multi-user Workflows
- Handling conflicts
File locking
Using git and git-annex directly

Setup

The proper setup of a local GIN client is important and can be confusing at first. This is why we wrote an extra GIN client setup guide. Here, you can find all the information on how to setup GIN locally.

The process looks something like this:

Register an account on the GIN Server.
Download and install the GIN Client distribution for your operating system.
Sign in with the client using gin login. The command will prompt you for a username and password. If entered correctly, you will be greeted with a confirmation message.
Check whether everything is working properly by typing gin --version.

Whether you are an expert user or a beginner, our list of all GIN commands with a short description of their purpose and functionality might become useful at some point.

If you want to quickly look up some quick short commands for accomplishing common tasks, have a look at the GIN client recipes page.

Basic Workflow (only using GIN)

The Workflow

Open a terminal (Unix) or run the GIN client (Windows) and type gin --version to make sure it is properly installed and working.
Create a new repository on the GIN server and a local workspace. This can be done in two ways: via the gin.g-node.org website or via the previously downloaded/installed GIN client (See Setup above).
- On the website:
  1. Sign into the GIN Server.
  2. Create a new repository using the "+" on the top right.
  3. Run
```
   gin get <user name>/<repository name>
```
    in the terminal on your local computer.
  For example, if your username is "ABC" and you have created a repository named "XYZ", the command would be gin get ABC/XYZ. Note that the command will fail with an error if a directory with your repository name already exists locally.
- Using the GIN client:
  1. Run
```
   gin create <repository name>
```
    in the terminal on your local computer.
Be aware: The path you have selected when executing gin get or gin create will be the path or location where your repository will be created. E.g. The path [user]:~/Desktop$ will create the repository on the Desktop. If you desire to change the location of your GIN repositories, use bash commands like cd, pwd, ls. See here for more. The result will look similar to this:
With either method, the repository is created on the server under your username and directly copied to the local machine into a directory with the same name as the repository (<repository name>).
Copy new files into the newly created directory via Drag & Drop, Copy & Paste etc.
In the GIN client (terminal) window, navigate into the newly created local workspace by typing cd <repository name>. So far it should look like this:
Upload the new files using
```
 gin upload .
```
Note the period at the end of the command. This command will commit your changes. In other words, it will detect the new files in the directory, add them to the repository, and start uploading to the GIN server. Every time you perform a gin upload . the changes are saved and uploaded and a checkpoint is made of your data.

The . that follows the upload command specifies that you want to upload every file in the current directory and below. You can instead upload individual files or directories by listing them on the command line. For example:
```
gin upload file1.data recordings/recording1.h5
```
This will upload changes made to two files: file1.data and recording1.h5, where the latter is in the recordings directory.

Note that upload here doesn't only mean sending new files and changes to the server. This command sends all changes made in the directory to the server, including deletions, renames, etc. Therefore, if you delete files from the directory on your computer and perform a gin upload, the deletion will also be sent and the file will be removed from the server as well. Such changes can be synchronized without uploading any new files by not specifying any files or directories.
```
gin upload
```
Check the files in the local workspace along with their sync status using gin ls. You can also check that the new files have been uploaded by logging into the GIN Server and clicking on the name of the newly created repository.
IMPORTANT: If the repository was created using GIN client v1.6 or below (check gin --version) all large files (default >10MB) are automatically gin-annexed and are now locked. This means that you can read them but cannot copy nor edit them. For copying or editing a specific file, unlock it using gin unlock <filename>. To unlock all files at once, use gin unlock . (Note the period at the end). For more, take a look at the advanced workflow section: for large file sizes below. See also the Note below.
Edit files and add new files.
Once you are done editing, you may repeat the workflow from step 4 above to add new files.

Note: Starting with GIN client version 1.7, all newly created (or downloaded) repositories set up the local clone in a new mode where added files remain in unlocked mode unless explicitly locked and committed. More details about locking and unlocking are available on the Command Overview page (see the sections for the lock and unlock commands). See also the File locking section below.

General usage

Beyond the main workflow described above, the following commands are generally useful at any time when working with GIN.

List repositories on the server

You can see a list of the repositories that you own using:

gin repos

To see repositories shared with you by other users (i.e., repositories on which you are a collaborator) use:

gin repos --shared

The combined output of the two previous commands (repositories that you own and are collaborating on) can be retrieved using:

gin repos --all

Additionally, you can use the same command to see a list of repositories owned by any given user that are accessible by you.

For instance:

gin repos ABC

will list all repositories owned by the user ABC that you can download. This includes public repositories owned by ABC as well as repositories that ABC has chosen to share with you.

Fetch any repository updates from the server

If changes are made to your data elsewhere, for example on another computer (assuming they were uploaded to the server), or from another user that you share your data with, you can download these changes by typing the download command from within the repository.

gin download

This command will only download changes made to the repository (file deletions, renames, etc.) but any new files are downloaded as placeholders. Placeholder files are empty files that represent files uploaded to the repository but do not hold any of the data. This is useful for downloading the contents of larger files on demand without downloading the entire repository.

If you would like to download all the data contained in a repository, you can do so using the --content flag.

gin download --content

This will synchronize the local directory with all changes made on the server and download the content of all files.

Checking the status of your files

While working with GIN repositories you will often need to check the status of your local files with respect to the remote repository. Files can be in one of a number of states:

Synced
No local content
Type changed (locked or unlocked)
Locally modified (unsaved)
Locally modified (not uploaded)
Untracked

To determine the status of files you can use the ls command:

gin ls

This will display all files under the current directory separated by their status. You can also see a full listing of files prefixed by an abbreviated status code.

gin ls --short

As with many other commands, the status of individual files can be determined by specifying the files on the command line.

gin ls file1.data recordings/recording1.h5

Selective sync Workflows

Selective download

When new data has become available or existing files have been changed on the GIN server, a selected subset of the changes can be downloaded to the local workspace.

Download a summary of the changes on the GIN server using gin download. IMPORTANT: This does not download any data. New files and files changed on the GIN server are considered to be "unsynced".
Use gin ls to check the sync status of the files in the repository.
Use gin get-content <file name> to download the data of a specific file.

Selective upload

When new data has been added to or existing files changed in the local workspace, a selected subset of the changes can be uploaded to the GIN server.

Use gin ls to check the sync status of the files in the repository.
Use gin upload <file name> to only upload the specified new file or changes to the specified existing file.

Multi-user Workflows

Collaboration is one of the most important reasons to use GIN. The following workflow outlines one way in which two users can work together in the same repository, without creating conflicts.

As a general rule, the best way to avoid creating conflicts is to avoid having multiple users work on the same file at the same time (e.g., from different locations). There are several ways to accomplish this:

Communication between collaborators separation of tasks: in many cases, work can be distributed (either across time, or across files) such that no two users need to work on the same file simultaneously.
Visiting the repository web page (at https://gin.g-node.org): keeping an eye on the repository's last update time and checking which files have changed since the last download.
Habitually downloading the newest versions of all files: it is good to develop the habit of downloading the newest version before editing a file (using gin download).

In this example, we will refer to two users, Owner and Collaborator. Owner will be the user that created the repository, whereas Collaborator is a user that has been given read & write access to the repository. Each command block mentions which user performs each command. At the end of each step is a side-by-side comparison of the state of the local directory for each user.

Creating the repository

Owner creates the repository as described in the Basic Workflow above.

Owner:
```
gin create shareddata "Data for collaboration with Collaborator"
```
Adding Collaborator to repository

On the website, Owner can give Write access to Collaborator by navigating to the repository's settings page (in our example, the address would be https://gin.g-node.org/Owner/shareddata/settings), clicking on Collaboration, and entering Collaborator into the Add New Collaborator field. The access level can be changed between Read (can download from the repository, but not upload), Write (can download and upload), and Admin (can download, upload, and change repository settings).
Owner adds some data to the repository (by copying files into the shareddata directory—not shown) and uploads them.

Owner:
```
cd shareddata
gin upload .
```

Collaborator clones the repository on their workstation and downloads all the data files.

Collaborator:

gin get Owner/shareddata
cd shareddata
gin get-content .

The output of gin ls for each user shows they both have a datafile with its contents:

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        datafile.h5                                      datafile.h5

Collaborator writes some data analysis scripts, generates some stats based on the original data, and adds them to the repository.

Collaborator:

gin upload runstats.py
python runstats.py
gin upload runstats.py stats/

(Note: The second line, python runstats.py, is simply there to demonstrate that the user is running a script that generates files. The contents or actions of the script are not relevant.)

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        datafile.h5                                      datafile.h5
                                                         runstats.py
                                                         stats/stat_data.h5

Owner downloads the stats and writes a plotting script to generate some figures based on the stats. After generating the figures, Owner uploads them to the remote repository.

Owner:

gin download --content
python plotfigures.py
gin upload plotfigures.py figures/

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        datafile.h5                                      datafile.h5
        figures/plot_A.png                               runstats.py
        figures/plot_B.png                               stats/stat_data.h5
        plotfigures.py
        runstats.py
        stats/stat_data.h5

Collaborator downloads the plotting script and figures, discovers a bug in the runstats.py script. Collaborator fixes the bug, reruns the entire analysis and plotting pipeline, and uploads the new versions.

Collaborator:
```
gin download --content
```
The file or files in the stats/ directory are data files, so it's likely they are big and were added to the annex. Collaborator needs to specify the --content flag when downloading to download all the data.

Collaborator:
```
python runstats.py
```
runstats.py here is the fixed version of the script.
```
python plotfigures.py
gin upload runstats.py stats/ figures/
```
The plotfigures.py file was not changed, so there's no need to upload it.

Owner doesn't need the original data on the local hard drive anymore, but wants to see the updated figures. Owner removes the content of all the data files and downloads the new figures.

Owner:

gin remove-content .
gin download
gin get-content figures/

Note that the get-content line may be unnecessary if the figures are not very large. If the figures are not in the annex and the command is run, it will have no effect.

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        figures/plot_A.png                               datafile.h5
        figures/plot_B.png                               figures/plot_A.png
        plotfigures.py                                   figures/plot_B.png
        runstats.py                                      plotfigures.py
                                                         runstats.py
No local content:                                        stats/stat_data.h5

        datafile.h5
        stats/stat_data.h5

Collaborator wants to tweak the figures a little bit. Since the figures are generated from the data in the stats/ directory, the original data are not needed. Collaborator removes the original data, but keeps the stats/ files around to redraw some of the figures.

Collaborator:

gin remove-content original_data.h5
python plotfigures.py
gin upload plotfigures.py figures/

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        figures/plot_A.png                               figures/plot_A.png
        figures/plot_B.png                               figures/plot_B.png
        plotfigures.py                                   figures/plot_C.png
        runstats.py                                      plotfigures.py
                                                         runstats.py
No local content:                                        stats/stat_data.h5

        datafile.h5                              No local content:
        stats/stat_data.h5
                                                         datafile.h5

Owner and Collaborator can continue to upload and download files to the repository. After Step 9, the content of the original_data.h5 file exists only on the server, but both Owner and Collaborator can retrieve it at any time and can even download it from the GIN website.

Handling conflicts

The above scenario paints a rather ideal view of what a multi-user workflow might look like. In reality, it is not uncommon for two or more users to edit a file simultaneously. This results in a situation where two file versions exist with up-to-date changes, but neither of which is the true latest version. Issues arise because, given two versions of a file that were updated by two different people separately, there is no general way to determine which the latest version should be: it could be either, or it could be a combination of both. As a result, there is often no automated way to determine what to do, so the client will instead inform the user of the conflict and it becomes the task of the user to decide which solution is the right one.

Note that simultaneously doesn't necessarily imply that file edits were made at the exact same time, but rather that two or more users edited a file and added the changes to the repository before either was uploaded to the server.

In some situations, git (and gin by extension) can merge changes made to a file by multiple users. This is possible for plain text files such as source code. For large files, a conflict will create copies of a file to avoid data loss.

Our scenario continues from the above Multi-User Workflow tutorial. In step 9, Collaborator had changes the plotfigures.py script, generated new figures (or updated the existing ones) and uploaded these changes to the server. The following steps demonstrated the different ways in which users can find themselves in conflicting situations and how each can be resolved.

Owner adds a file with notes to the repository and tries to upload the file.

Owner:

gin upload Notes.txt

The upload fails with the following error:

upload failed: changes were made on the server that have not been downloaded; run 'gin download' to update local copies

The reason for this is that Collaborator has uploaded changes to the server that Owner has not yet downloaded. The fact that the changes made by both users are on different files is irrelevant. The important part is that the repository on the Owner's computer was in a different state than the repository on the server before Owner tried to perform an upload.

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        figures/plot_A.png                               figures/plot_A.png
        figures/plot_B.png                               figures/plot_B.png
        plotfigures.py                                   figures/plot_C.png
        runstats.py                                      plotfigures.py
                                                         runstats.py
No local content:                                        stats/stat_data.h5

        datafile.h5                              No local content:
        stats/stat_data.h5
                                                         datafile.h5
Locally modified (not uploaded):

        Notes.txt

The error message in the previous step tells Owner all they need to know to resolve this situation.

Owner:

gin download

In this case, the resolution is simple since there are no file conflicts. In other words, the changes made at the two locations—the server and the Owner's computer—are in different files (Collaborator changed figures/ and uploaded those changes to the server while Owner added Notes.txt) and merging the two states is straightforward and can be performed automatically.

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        figures/plot_A.png                               figures/plot_A.png
        figures/plot_B.png                               figures/plot_B.png
        figures/plot_C.png                               figures/plot_C.png
        plotfigures.py                                   plotfigures.py
        runstats.py                                      runstats.py
                                                         stats/stat_data.h5
No local content:
                                                 No local content:
        datafile.h5
        stats/stat_data.h5                               datafile.h5

Locally modified (not uploaded):

        Notes.txt

Now Owner can upload the new file.

Owner:

gin upload Notes.txt

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        Notes.txt                                        figures/plot_A.png
        figures/plot_A.png                               figures/plot_B.png
        figures/plot_B.png                               figures/plot_C.png
        figures/plot_C.png                               plotfigures.py
        plotfigures.py                                   runstats.py
        runstats.py                                      stats/stat_data.h5

No local content:                                No local content:

        datafile.h5                                      datafile.h5
        stats/stat_data.h5

While working in the repository, Collaborator also likes to keep a file with notes called Notes.txt, but since this file contains personal notes, it was never added to the repository. This creates an issue when Collaborator attempts to download the newest changes to the repository.

Collaborator:

gin download

The download fails with the following error:

download failed: local modified or untracked files would be overwritten by download:
  Notes.txt

The client will refuse to download the Notes.txt file added to the repository by Owner because this would overwrite the personal notes file of Collaborator.

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        Notes.txt                                        figures/plot_A.png
        figures/plot_A.png                               figures/plot_B.png
        figures/plot_B.png                               figures/plot_C.png
        figures/plot_C.png                               plotfigures.py
        plotfigures.py                                   runstats.py
        runstats.py                                      stats/stat_data.h5

No local content:                                No local content:

        datafile.h5                                      datafile.h5
        stats/stat_data.h5
                                                 Untracked:

                                                         Notes.txt

Collaborator needs to move or rename the personal notes file before downloading the latest changes.

Collaborator:

mv Notes.txt PersonalNotes.txt
gin download

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        Notes.txt                                        Notes.txt
        figures/plot_A.png                               figures/plot_A.png
        figures/plot_B.png                               figures/plot_B.png
        figures/plot_C.png                               figures/plot_C.png
        plotfigures.py                                   plotfigures.py
        runstats.py                                      runstats.py
                                                         stats/stat_data.h5
No local content:
                                                 No local content:
        datafile.h5
        stats/stat_data.h5                               datafile.h5

                                                 Untracked:

                                                         PersonalNotes.txt

Both Owner and Collaborator decide to add a list of relevant articles to the repository called references.txt. Owner adds the file first and uploads, then Collaborator adds the same file (same file name) with different contents and tries to upload.

Owner:

gin upload references.txt

Collaborator:

gin upload references.txt

Collaborator's upload stops with the following error (the same error we saw earlier):

upload failed: changes were made on the server that have not been downloaded; run 'gin download' to update local copies

However, this time, when Collaborator attempts to download the changes from the server, the download fails with the following error:

download failed: files changed locally and remotely and cannot be automatically merged (merge conflict):
  references.txt

Unlike the first case we examined, where there were no file conflicts and the two states could be merged automatically, this case requires manual intervention.

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        Notes.txt                                        Notes.txt
        figures/plot_A.png                               figures/plot_A.png
        figures/plot_B.png                               figures/plot_B.png
        figures/plot_C.png                               figures/plot_C.png
        plotfigures.py                                   plotfigures.py
        references.txt                                   runstats.py
        runstats.py                                      stats/stat_data.h5

No local content:                                No local content:

        datafile.h5                                      datafile.h5
        stats/stat_data.h5
                                                 Locally modified (not uploaded):

                                                         references.txt

                                                 Untracked:

                                                         PersonalNotes.txt

The solution is similar to the case where the Notes.txt file was going to be overwritten, however there is one minor difference: Since Collaborator already tried to upload a different version of references.txt, the file has already been recorded into the repository. Therefore, the file needs to be moved, removed, or renamed, and the change must be recorded before continuing.

Collaborator:

mv references.txt myrefs.txt
gin commit references.txt
gin download

Now Collaborator can copy the contents of myrefs.txt into Owner's references.txt and upload the combined file.

Collaborator:

cat myrefs.txt >> references.txt
gin upload references.txt

(Note: The first line, cat myrefs..., is a common way to append one file to another in Unix shells. The result is that the contents of myrefs.txt is added to the end of references.txt. The specifics of the merging of the two files is not relevant to this tutorial. Any kind of merger of the two files would be valid.)

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        Notes.txt                                        Notes.txt
        figures/plot_A.png                               figures/plot_A.png
        figures/plot_B.png                               figures/plot_B.png
        figures/plot_C.png                               figures/plot_C.png
        plotfigures.py                                   plotfigures.py
        references.txt                                   references.txt
        runstats.py                                      runstats.py
                                                         stats/stat_data.h5
No local content:
                                                 No local content:
        datafile.h5
        stats/stat_data.h5                               datafile.h5

                                                 Untracked:

                                                         PersonalNotes.txt
                                                         myrefs.txt

The same scenario as the previous one works out a little differently in the case of large files (i.e., files that are recorded in the annex). In this case, both Owner and Collaborator decide to run some new analysis that generates a new data file stats/ called new_stats.h5. The analysis done by each user is not the same, therefore, as in the case with references.txt above, both users now have a file named stats/new_stats.h5 but the contents are different. Collaborator uploads this new file to the repository first, then Owner:

Collaborator:

gin upload stats/

Owner:

gin upload stats/

As we've come to expect, this produces the upload error we've seen before:

upload failed: changes were made on the server that have not been downloaded; run 'gin download' to update local copies

Owner therefore runs the download command as instructed.

Owner:

gin download

Unlike the case with references.txt, the conflict is automatically (partially) resolved by the underlying functionality (namely, git-annex). The following message informs the user of this resolution:

files changed locally and remotely. Both files have been kept:
  stats/new_stats.h5

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        Notes.txt                                        Notes.txt
        figures/plot_A.png                               figures/plot_A.png
        figures/plot_B.png                               figures/plot_B.png
        figures/plot_C.png                               figures/plot_C.png
        plotfigures.py                                   plotfigures.py
        references.txt                                   references.txt
        runstats.py                                      runstats.py
                                                         stats/new_stats.h5
No local content:                                        stats/stat_data.h5

        datafile.h5                              No local content:
        stats/stat_data.h5
                                                         datafile.h5
Locally modified (not uploaded):
                                                 Untracked:
        stats/new_stats.variant-be88.h5
        stats/new_stats.variant-cc5f.h5                  PersonalNotes.txt
                                                         myrefs.txt

Owner can check the status of the files in the repository now, or just the stats/ directory where the conflict occurred.

Owner:
```
gin ls stats/
```
The output should be something that looks like the following:
```
Locally modified (not uploaded):
     stats/new_stats.variant-be88.h5
     stats/new_stats.variant-cc5f.h5
```
(the 4 characters that follow .variant- will differ each time).

As mentioned at the beginning of this section and demonstrated in the previous scenarios, the conflict should, in the end, be resolved by manual intervention. Therefore, Owner needs to decide which of the two files should be kept (if only one is to be kept), if the data should be merge into a single file, or if both should be kept separately. For the purposes of this tutorial, we will assume that both files will be kept under different names.

Owner:

mv stats/new_stats.variant-be88.h5 stats/owner_analysis_results.h5
mv stats/new_stats.variant-cc5f.h5 stats/collab_analysis_results.h5
gin upload stats/

Note that, in the end, the collab_analysis_results.h5 is only the placeholder file for the new_stats.h5 file created by Collaborator. When Owner renames the file, it is not necessary to have the content downloaded locally. When Collaborator downloads the changes in the future, the file will be renamed while still pointing to the same content as before.

[Owner] gin ls                                   [Collaborator] gin ls

Synced:                                          Synced:

        Notes.txt                                        Notes.txt
        figures/plot_A.png                               figures/plot_A.png
        figures/plot_B.png                               figures/plot_B.png
        figures/plot_C.png                               figures/plot_C.png
        plotfigures.py                                   plotfigures.py
        references.txt                                   references.txt
        runstats.py                                      runstats.py
        stats/owner_analysis_results.h5                  stats/new_stats.h5
                                                         stats/stat_data.h5
No local content:
                                                 No local content:
        datafile.h5
        stats/collab_analysis_results.h5                  datafile.h5
        stats/stat_data.h5
                                                 Untracked:

                                                         PersonalNotes.txt
                                                         myrefs.txt

Important note: In all cases where a gin upload command fails with an error, the client first records the data before attempting to upload. This data remains as part of the history of the repository, even if it is later deleted. This includes the two original and conflicting versions of references.txt (before merging) as well as the two original and conflicting versions of new_stats.h5. It does not include the Notes.txt file owned by Collaborator (later renamed to PersonalNotes.txt), because this file was never recorded into the repository. In the end, even if one of the files is deleted, it is still accessible through the repository's history with the use of the gin version command. A tutorial for the gin version command is coming soon, but for now, you can read about it in the built-in help by typing gin help version and you can see some examples of its usage in the Versioning section of the GIN client recipes page.

File locking

Files in a GIN repository have a lock state; they can be either locked or unlocked. The lock state of a file persists in the repository. In other words, if a file is locked, committed, uploaded, and then downloaded to another computer, the downloaded file will also be locked.

When a file is added to a repository, it is copied into the git-annex object store (.git/annex/objects) with a name that represents its content and a separate file is kept in its place in the working directory. The lock state determines the nature of the file that remains in the working directory.

Locked files cannot be edited. On Unix-like systems (Linux and macOS), the file in the working directory is replaced by a symbolic link to the annexed content (the file in .git/annex/objects). This allows the user to read the file without modifying it. On Windows, the file in the working directory simply contains plain text referring to the content. This is less useful since reading the file doesn't provide users with the content of the original file. It does however allow manipulation of the file in other ways: renaming, moving, copying, and deleting the reference file will work as expected.

Unlocked files on the other hand can be edited. The file in the working directory is a copy of the original file. A disadvantage of this mode is that it doubles the disk usage of files checked into the repository. The advantage is that users can modify files and revert those modifications to a previous commit.

The gin lock and gin unlock commands switch the state for one or more files. After changing the state, the gin commit command should be used to record the state change.

General information and command usage can also be found using the built-in help function of the client:

gin help lock
gin help unlock
gin help commit

When to use each

If a workflow requires editing large files and keeping snapshots of the progress, files need to be kept in the unlocked state. This will increase storage usage with every commit/snapshot of the file.

If large files are added to the repository for long term storage, archiving, or only to be read, and if the filesystem supports symbolic links, locked files save the user the extra storage of keeping two copies of the file.

Changing the default

Since version 1.7 of the client, files are added to a repository in the unlocked state by default. This can be changed by running:

git config annex.addunlocked false

After this command is executed, all new files added to the repository will be automatically locked. This is a per-repository option and will not affect other existing or new repositories.

To return to the default option, run:

git config annex.addunlocked true

Using git and git-annex directly

GIN repositories are git repositories with git-annex support. Users familiar with git and git-annex can use them directly to manage their data with the GIN server. However, mixing gin commands with git and git-annex workflows is not recommended, unless the user is familiar with the way the gin client sets up a repository.

The client sets up certain configuration options when initialising a repository. Some of these options are for convenience and consistency when working on multiple platforms. If some of these options are not set up as expected, the client may behave unexpectedly.

Below we list these options, the reasons for setting them, and any risks involved with changing them:

core.quotepath git configuration option disabled.
- When this option is not disabled, git prints escape sequences for files with unicode characters in their name. This can break gin commands internally that fail to match filenames with the names from the output of git commands.
core.symlinks git configuration option disabled on Windows.
- On newer Windows versions, if developer mode is enabled, this can default to being enabled. This works with git commands, but git-annex doesn't work properly with Windows-style symbolic links, so it needs to be disabled for git-annex to work properly.
annex.backends configuration option should be set to MD5 and/or WORM.
- The default git-annex backend (hashing algorithm) is SHA256. This can produce very long filenames which causes issues on Windows if long pathnames are not supported (by default, they are unsupported). This option does not only affect the local repository. Any files added to a repository when SHA256 is enabled will create git-annex content objects with very long filenames, which may fail to download on Windows, making the repository unusable for Windows users.
annex.addunlocked option enabled.
- This makes files added to the repository's annex stay unlocked by default. Keeping this disabled will not cause any issues, but it will require unlocking annexed files in order to edit them. Locked files are also not readable on Windows, since symlinks are required for working with locked files and don't work as expected.
Annex initialised in version 8 mode.
- Version 8 repositories enable support for the addunlocked option above.

Local GIN client - Usage Guide

Contents

Setup

Basic Workflow (only using GIN)

The Workflow

General usage

List repositories on the server

Fetch any repository updates from the server

Checking the status of your files

Selective sync Workflows

Selective download

Selective upload

Multi-user Workflows

Owner:

Owner:

Collaborator:

Collaborator:

Owner:

Collaborator:

Collaborator:

Owner:

Collaborator:

Handling conflicts

Owner:

Owner:

Owner:

Collaborator:

Collaborator:

Owner:

Collaborator:

Collaborator:

Collaborator:

Collaborator:

Owner:

Owner:

Owner:

Owner:

File locking

When to use each

Changing the default

Using git and git-annex directly