This PR reduces the size of the files stored in git-annex by reducing the recording time of large files. This concerns the following formats and files:
@sprenger Can you also add https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/src/master/neuroscope/test1/test1.dat to the list? 37 MB instead of < 10MB, super simple binary blob format and the # of frames is not encoded in the header .xml
Progress update: The neuroshare and tdt files can not be easily shortened, as the first ones has no specifications and is read via a dll and the second ones has an interdependency between tsq and tev files.
I will try to replace these with smaller files from other datasets.
Update:
Neuroshare has been replaced by another file with much smaller size.
Progress update: The neuroshare and tdt files can not be easily shortened, as the first ones has no specifications and is read via a dll and the second ones has an interdependency between `tsq` and `tev` files.
I will try to replace these with smaller files from other datasets.
Update:
Neuroshare has been replaced by another file with much smaller size.
@samuelgarcia, I moved the remaining files to be reduced to a new issue (#83), so we can merge this PR now. Currently this reduces the size of the repository to about 1.3GB to 900MB.
@samuelgarcia, I moved the remaining files to be reduced to a new issue (https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/issues/83), so we can merge this PR now. Currently this reduces the size of the repository to about 1.3GB to 900MB.
I ran the Neo testsuite locally with the new set of files and everything seems to work when using https://github.com/NeuralEnsemble/python-neo/pull/1122.
Could you merge https://github.com/NeuralEnsemble/python-neo/pull/1122 and then this PR here?
Hi @samuelgarcia, thanks for merging the linked neo PR. Can you review and merge this one next? Not all files in the list have been reduced, but I think we should merge this rather sooner than later. All other files have been listed in (#83).
Hi @samuelgarcia, thanks for merging the linked neo PR. Can you review and merge this one next? Not all files in the list have been reduced, but I think we should merge this rather sooner than later. All other files have been listed in (#83).
This PR reduces the size of the files stored in git-annex by reducing the recording time of large files. This concerns the following formats and files:
Update
The current changes reduce the repo size from 1.3Gb to 1.1Gb More potential files for shortening
Update
On the Neo side this PR depends on https://github.com/NeuralEnsemble/python-neo/pull/1122
@sprenger Can you also add https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/src/master/neuroscope/test1/test1.dat to the list? 37 MB instead of < 10MB, super simple binary blob format and the # of frames is not encoded in the header .xml
Progress update: The neuroshare and tdt files can not be easily shortened, as the first ones has no specifications and is read via a dll and the second ones has an interdependency between
tsq
andtev
files. I will try to replace these with smaller files from other datasets.Update: Neuroshare has been replaced by another file with much smaller size.
@samuelgarcia, I moved the remaining files to be reduced to a new issue (#83), so we can merge this PR now. Currently this reduces the size of the repository to about 1.3GB to 900MB.
I ran the Neo testsuite locally with the new set of files and everything seems to work when using https://github.com/NeuralEnsemble/python-neo/pull/1122. Could you merge https://github.com/NeuralEnsemble/python-neo/pull/1122 and then this PR here?
Hi @samuelgarcia, thanks for merging the linked neo PR. Can you review and merge this one next? Not all files in the list have been reduced, but I think we should merge this rather sooner than later. All other files have been listed in (#83).