#67 Web interface upload broken

Closed
opened 9 months ago by davidhaslacher · 4 comments

When attempting to upload a small number (< 100) of small (< 2 GB) data files using the web interface, I am getting an error: "Invalid csrf token".

When attempting to upload a small number (< 100) of small (< 2 GB) data files using the web interface, I am getting an error: "Invalid csrf token".
David Haslacher commented 9 months ago
Poster

Now I received "504 Gateway Time-out" when uploading a really small number (< 10) files. However, they were successfully pushed to the repo, and now appear there!

Now I received "504 Gateway Time-out" when uploading a really small number (< 10) files. However, they were successfully pushed to the repo, and now appear there!
Thomas Wachtler commented 9 months ago
Owner

The csrf token for the upload form expires after one day. Maybe you left the page open for too long before you uploaded. Refreshing the page will recreate the token.

The https protocol is not particularly suited for bulk uploads, this is why there are limits on the web upload. There is always a chance of a timeout or connection loss. For uploading multiple files it is highly recommended to use the GIN client (https://gin.g-node.org/G-Node/Info/wiki/GIN+CLI+Usage+Tutorial), which is not only more robust but also more convenient.

The csrf token for the upload form expires after one day. Maybe you left the page open for too long before you uploaded. Refreshing the page will recreate the token. The https protocol is not particularly suited for bulk uploads, this is why there are limits on the web upload. There is always a chance of a timeout or connection loss. For uploading multiple files it is highly recommended to use the GIN client (https://gin.g-node.org/G-Node/Info/wiki/GIN+CLI+Usage+Tutorial), which is not only more robust but also more convenient.
David Haslacher commented 9 months ago
Poster

Understood, thank you.

I had wanted to avoid the GIN client because I wanted to avoid needing to make a local copy of the dataset on my computer. The dataset is quite large, and I have it on an external drive. Therefore, I wanted to upload it directly to the repository from there. Setting the checkout or mountpoint directory to be on the external drive apparently doesn't work.

Understood, thank you. I had wanted to avoid the GIN client because I wanted to avoid needing to make a local copy of the dataset on my computer. The dataset is quite large, and I have it on an external drive. Therefore, I wanted to upload it directly to the repository from there. Setting the checkout or mountpoint directory to be on the external drive apparently doesn't work.
Thomas Wachtler commented 9 months ago
Owner

If the root directory of your dataset is writable it should be possible to turn it into a gin repository, no matter whether it is on an external drive.

On your local computer you could then have a lightweight copy of it that has everything except the contents of the large (annexed) files.

If the root directory of your dataset is writable it should be possible to turn it into a gin repository, no matter whether it is on an external drive. On your local computer you could then have a lightweight copy of it that has everything except the contents of the large (annexed) files.
Sign in to join this conversation.
No Milestone
No assignee
2 Participants
Loading...
Cancel
Save
There is no content yet.