|
@@ -20,24 +20,40 @@
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- " 1. 変数にメタデータ情報を入力し、入力後セルを実行する"
|
|
|
+ "1. 変数にメタデータ情報を入力し、入力後セルを実行する"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "code",
|
|
|
- "execution_count": 10,
|
|
|
+ "execution_count": 16,
|
|
|
"metadata": {},
|
|
|
"outputs": [],
|
|
|
"source": [
|
|
|
- "Experiment_date = \"2021/12/15\" \n",
|
|
|
- "Experimenter = \"yamada taro\""
|
|
|
+ "import json\n",
|
|
|
+ "meta_data = {\n",
|
|
|
+ " \"experiment_date\": \"2021/12/16\",\n",
|
|
|
+ " \"experimenter\": \"tanaka taro\"\n",
|
|
|
+ "}"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- " 2. meta_data.jsonに保存する"
|
|
|
+ "2. meta_data.jsonに保存する"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": 20,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "import json\n",
|
|
|
+ "from collections import OrderedDict\n",
|
|
|
+ "\n",
|
|
|
+ "with open('./meta_data.json', 'w') as jf:\n",
|
|
|
+ " json.dump(meta_data, jf, ensure_ascii=False, indent=2)"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -51,23 +67,23 @@
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- " 1. コミットメッセージを変数に入力する"
|
|
|
+ "1. コミットメッセージを変数に入力する"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "code",
|
|
|
- "execution_count": 12,
|
|
|
+ "execution_count": 1,
|
|
|
"metadata": {},
|
|
|
"outputs": [],
|
|
|
"source": [
|
|
|
- "save_message = \"\""
|
|
|
+ "save_message = \" \""
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- " 2. 一部のファイルのみ保存したい場合は、変数に保存したいファイル名を入力する"
|
|
|
+ "2. 途中保存する"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -76,14 +92,73 @@
|
|
|
"metadata": {},
|
|
|
"outputs": [],
|
|
|
"source": [
|
|
|
- "save_files = \"\""
|
|
|
+ "from datalad import api\n",
|
|
|
+ "\n",
|
|
|
+ "api.save(message=save_message)"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "markdown",
|
|
|
+ "metadata": {},
|
|
|
+ "source": [
|
|
|
+ "- ## 実験を終了する\n"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "markdown",
|
|
|
+ "metadata": {},
|
|
|
+ "source": [
|
|
|
+ "1. 実行環境構成を記録する"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": 5,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "%%sh\n",
|
|
|
+ "conda env export > environment.yaml"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": 6,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "%%sh\n",
|
|
|
+ "pip freeze > requirements.txt"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "markdown",
|
|
|
+ "metadata": {},
|
|
|
+ "source": [
|
|
|
+ "2. パイプラインを作成する(検討中)"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "markdown",
|
|
|
+ "metadata": {},
|
|
|
+ "source": [
|
|
|
+ "3. 実験単体の再現性をモニタリングする(コード付帯機能で行うか検討中)"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- " 3. 途中保存する"
|
|
|
+ "4. コミットメッセージを変数に入力し、実験記録の書き戻し準備をする"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "SAVE_MESSAGE = ''"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -94,27 +169,74 @@
|
|
|
"source": [
|
|
|
"from datalad import api\n",
|
|
|
"\n",
|
|
|
- "api.save(message=save_message, path=save_files)"
|
|
|
+ "api.save(message=SAVE_MESSAGE)"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
- "cell_type": "markdown",
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
"metadata": {},
|
|
|
+ "outputs": [],
|
|
|
"source": [
|
|
|
- "- ## 実験を終了する\n"
|
|
|
+ "import os\n",
|
|
|
+ "\n",
|
|
|
+ "os.chdir(\"/home/jovyan\")"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "%%sh\n",
|
|
|
+ "chmod 600 /home/jovyan/.ssh/id_rsa"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "%%sh\n",
|
|
|
+ "mkdir ./tmp\n",
|
|
|
+ "git remote show origin > ./tmp/remote.txt"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "with open('./tmp/remote.txt', 'r') as f:\n",
|
|
|
+ " lines = [s.strip() for s in f.readlines()]\n",
|
|
|
+ " ssh_push_url = lines[2].replace('Push URL: https://github.com/', 'ssh://root@dg02.dg.rcos.nii.ac.jp:3001/')"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "option: 実験メタデータを入力する\n",
|
|
|
- "1. 実行環境構成を記録する\n",
|
|
|
- "2. パイプラインを作成する(検討中)\n",
|
|
|
- "3. 実験単体の再現性をモニタリングする(コード付帯機能で行うか検討中)\n",
|
|
|
- "4. 実験記録の書き戻し準備\n",
|
|
|
"5. 実験記録を書き戻す"
|
|
|
]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "from datalad import api\n",
|
|
|
+ "\n",
|
|
|
+ "try:\n",
|
|
|
+ " api.siblings(action='add', name=SIBLING_NAME, url=SSH_PUSH_URL)\n",
|
|
|
+ "except:\n",
|
|
|
+ " pass\n",
|
|
|
+ "\n",
|
|
|
+ "api.push(to=SIBLING_NAME, result_renderer='default')"
|
|
|
+ ]
|
|
|
}
|
|
|
],
|
|
|
"metadata": {
|