|
@@ -4,9 +4,9 @@
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "# 書き戻しの準備をする\n",
|
|
|
+ "# ワークフロー実行のための準備\n",
|
|
|
"\n",
|
|
|
- "このノートブックでは、ワークフロー実行結果をginリポジトリに書き戻すための準備を行います。 \n",
|
|
|
+ "このノートブックでは、ワークフロー実行のための準備を行います。 \n",
|
|
|
"上から順番にセルを実行してください。"
|
|
|
]
|
|
|
},
|
|
@@ -14,16 +14,17 @@
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "## 0. SSH秘密鍵を.ssh/id_rsaに配置する\n",
|
|
|
+ "## 1. SSH秘密鍵を.ssh/id_rsaに配置する\n",
|
|
|
"\n",
|
|
|
- "プロトタイプではSSH秘密鍵をjupyterhubの.ssh/id_rsa、公開鍵をginリポジトリに登録することでSSH接続するため、初めにSSH秘密鍵を.ssh/id_rsaに配置してください。"
|
|
|
+ "プロトタイプでは、SSH秘密鍵をjupyterhubの.ssh/id_rsaに、公開鍵をginリポジトリに登録することでSSH接続するため、初めにSSH秘密鍵を.ssh/id_rsaに配置してください。 \n",
|
|
|
+ "※.sshフォルダは新規作成してください。"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "## 1. バージョン管理システムdataladを使用するために、このリポジトリをdatalad datasetとして認識させる"
|
|
|
+ "## 2. SSH接続で書き戻しを行うためにssh鍵に権限を与える(ssh鍵は.sshディレクトリの中にid_rsaとして保存する)"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -33,7 +34,41 @@
|
|
|
"outputs": [],
|
|
|
"source": [
|
|
|
"%%sh\n",
|
|
|
- "cd\n",
|
|
|
+ "chmod 600 ~/.ssh/id_rsa"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "markdown",
|
|
|
+ "metadata": {},
|
|
|
+ "source": [
|
|
|
+ "## 3. SSH鍵の利用確認をオフにする"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "%%sh\n",
|
|
|
+ "echo \"host *\\n\\tStrictHostKeyChecking no\\n\\tUserKnownHostsFile /dev/null\" >> ~/.ssh/config"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "markdown",
|
|
|
+ "metadata": {},
|
|
|
+ "source": [
|
|
|
+ "## 4. dataladを使用するため、このリポジトリをdatalad datasetとする"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "%%sh\n",
|
|
|
+ "cd ~/\n",
|
|
|
"datalad create --force"
|
|
|
]
|
|
|
},
|
|
@@ -41,7 +76,7 @@
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "## 2. dataladでワークフロー実行結果をginリポジトリに反映させるために、ginリポジトリをリモートリポジトリとして登録する"
|
|
|
+ "## 5. ワークフロー実行結果をginリポジトリに反映させるため、ginリポジトリをリモートリポジトリに設定する"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -55,6 +90,19 @@
|
|
|
"git remote show origin > ./tmp/remote.txt"
|
|
|
]
|
|
|
},
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "import json\n",
|
|
|
+ "json_open = open('/home/jovyan/param_files/param_for_dataladsiblings.json', 'r')\n",
|
|
|
+ "json_load = json.load(json_open)\n",
|
|
|
+ "http = json_load['http']\n",
|
|
|
+ "ssh = json_load['ssh']"
|
|
|
+ ]
|
|
|
+ },
|
|
|
{
|
|
|
"cell_type": "code",
|
|
|
"execution_count": null,
|
|
@@ -63,7 +111,18 @@
|
|
|
"source": [
|
|
|
"with open('./tmp/remote.txt', 'r') as f:\n",
|
|
|
" lines = [s.strip() for s in f.readlines()]\n",
|
|
|
- " ssh_push_url = lines[2].replace('Push URL: http://dg02.dg.rcos.nii.ac.jp/', 'ssh://root@dg02.dg.rcos.nii.ac.jp:3001/') "
|
|
|
+ " ssh_push_url = lines[2].replace(http, ssh) "
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
+ "source": [
|
|
|
+ "import os\n",
|
|
|
+ "\n",
|
|
|
+ "os.system(\"rm -r ./tmp\")"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -84,7 +143,14 @@
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "## 3. SSH接続で書き戻しを行うためにssh鍵に権限を与える(ssh鍵は.sshディレクトリの中にid_rsaとして保存する)"
|
|
|
+ "## 4. .gitconfigにユーザー名とメールアドレスを登録する"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "markdown",
|
|
|
+ "metadata": {},
|
|
|
+ "source": [
|
|
|
+ " 1. 以下のセルでgitアカウントのユーザー名とメールアドレスを入力してください。"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -93,64 +159,60 @@
|
|
|
"metadata": {},
|
|
|
"outputs": [],
|
|
|
"source": [
|
|
|
- "%%sh\n",
|
|
|
- "cd\n",
|
|
|
- "chmod 600 /home/jovyan/.ssh/id_rsa"
|
|
|
+ "name = input()"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
- "cell_type": "markdown",
|
|
|
+ "cell_type": "code",
|
|
|
+ "execution_count": null,
|
|
|
"metadata": {},
|
|
|
+ "outputs": [],
|
|
|
"source": [
|
|
|
- "## 4. .gitconfigにユーザー名とメールアドレスを登録する"
|
|
|
+ "email = input()"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- " 1. 以下のセルでgitアカウントのユーザー名とメールアドレスを入力してください。"
|
|
|
+ " 2. 入力されたユーザー名とメールアドレスを.gitconfigに登録します。"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "code",
|
|
|
- "execution_count": 1,
|
|
|
- "metadata": {},
|
|
|
- "outputs": [
|
|
|
- {
|
|
|
- "name": "stdout",
|
|
|
- "output_type": "stream",
|
|
|
- "text": [
|
|
|
- "ivis-mizuguchi\n"
|
|
|
- ]
|
|
|
- }
|
|
|
- ],
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
"source": [
|
|
|
- "name = input()"
|
|
|
+ "import os\n",
|
|
|
+ "\n",
|
|
|
+ "os.system(\"cd ~/\")\n",
|
|
|
+ "os.system(\"git config --global user.name \" + name)\n",
|
|
|
+ "os.system(\"git config --global user.email \" + email)"
|
|
|
+ ]
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "cell_type": "markdown",
|
|
|
+ "metadata": {},
|
|
|
+ "source": [
|
|
|
+ "## 5. .local配下をgit管理しないように.gitignoreで設定する"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "code",
|
|
|
- "execution_count": 2,
|
|
|
- "metadata": {},
|
|
|
- "outputs": [
|
|
|
- {
|
|
|
- "name": "stdout",
|
|
|
- "output_type": "stream",
|
|
|
- "text": [
|
|
|
- "rino.mizuguchi@ivis.co.jp\n"
|
|
|
- ]
|
|
|
- }
|
|
|
- ],
|
|
|
+ "execution_count": null,
|
|
|
+ "metadata": {},
|
|
|
+ "outputs": [],
|
|
|
"source": [
|
|
|
- "email = input()"
|
|
|
+ "%%sh\n",
|
|
|
+ "echo \".local/*\" >> ~/.gitignore"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- " 2. 入力されたユーザー名とメールアドレスを.gitconfigに登録する"
|
|
|
+ "## 6. 実験用フォルダ構成を初期化する"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -161,15 +223,15 @@
|
|
|
"source": [
|
|
|
"import os\n",
|
|
|
"\n",
|
|
|
- "os.system(\"git config --global user.name \" + name)\n",
|
|
|
- "os.system(\"git config --global user.email \" + email)"
|
|
|
+ "experiment_folder = \"/home/jovyan/experiments\"\n",
|
|
|
+ "os.makedirs(experiment_folder)"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "## 5. .gitignoreで.local配下をgit管理しない設定にする"
|
|
|
+ "## 7. 必要なファイルをPACKAGEテンプレートに複製する"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -179,15 +241,19 @@
|
|
|
"outputs": [],
|
|
|
"source": [
|
|
|
"%%sh\n",
|
|
|
- "cd\n",
|
|
|
- "echo \".local/*\" >> .gitignore"
|
|
|
+ "cd ~/\n",
|
|
|
+ "mkdir WORKFLOW/PACKAGE/.ssh\n",
|
|
|
+ "cp .ssh/id_rsa WORKFLOW/PACKAGE/.ssh/id_rsa\n",
|
|
|
+ "cp .ssh/config WORKFLOW/PACKAGE/.ssh/config\n",
|
|
|
+ "cp .gitconfig WORKFLOW/PACKAGE/.gitconfig\n",
|
|
|
+ "cp .gitconfig WORKFLOW/PACKAGE/.gitconfig"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
"cell_type": "markdown",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "## 6. SSH鍵の利用確認をオフにする"
|
|
|
+ "## 7. 変更内容を書き戻す"
|
|
|
]
|
|
|
},
|
|
|
{
|
|
@@ -196,9 +262,13 @@
|
|
|
"metadata": {},
|
|
|
"outputs": [],
|
|
|
"source": [
|
|
|
- "%%sh\n",
|
|
|
- "cd\n",
|
|
|
- "echo \"host *\\n\\tStrictHostKeyChecking no\\n\\tUserKnownHostsFile /dev/null\" >> .ssh/config"
|
|
|
+ "import papermill as pm\n",
|
|
|
+ "\n",
|
|
|
+ "pm.execute_notebook(\n",
|
|
|
+ " '../util/datalad_save_push.ipynb',\n",
|
|
|
+ " '-',\n",
|
|
|
+ " parameters = dict(SAVE_MESSAGE = 'ワークフロー実行準備')\n",
|
|
|
+ ")"
|
|
|
]
|
|
|
}
|
|
|
],
|