ファイル追加'

README.md

@@ -1,2 +1,23 @@
-# 0121
+# 0121-20
 
+openpyxl
+openssl
+pip
+wheel
+pygraphvizなし
+
+# データガバナンス機能　研究支援ワークフロー利用ガイド
+
+データガバナンス機能では研究活動をサポートする、様々な機能をワークフローとして提供しています。以下の手順に沿って、ワークフロー機能を利用することができます。
+
+## 初期設定をする
+
+1. ワークフロー機能は、入力いただいたDMP情報を利用します。まだDMP情報の入力がお済みでない場合は、右上の`Add DMP`ボタンからDMPを作成ください。<br>
+◆（仮置き）Add DMPボタンは、DMPの提出先を選ぶことができるプルダウンになっています。提出機関未定の場合は「meti」を選択してください。
+
+2. DMPの作成が終わったら、画面右上に出てくる「Generate maDMP」ボタンをクリックしてください。これにより、maDMP（コンピュータが実行可能なDMP）が生成されます。
+
+3. 「Generate maDMP」ボタンをクリックしてmaDMPを生成したら、以下のボタンをクリックしてmaDMP実行環境へ移ることができます。<br>
+※maDMP作成前は、ボタンをクリックしても遷移できませんのでご注意ください。
+
+maDMP実行環境へ遷移：[![Binder](https://binder.cs.rcos.nii.ac.jp/badge_logo.svg)](https://binder.cs.rcos.nii.ac.jp/v2/git/http%3A%2F%2Fdg02.dg.rcos.nii.ac.jp%2Fivis-mizuguchi%2F0121-20.git/master?filepath=maDMP.ipynb)

WORKFLOW/FLOW/01_preparation_phase/setup_data_analysis_tools.ipynb

@@ -0,0 +1,42 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# データ分析ツールの環境構築を行う\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## ◆◆◆開発メモ◆◆◆\n",
+    "\n",
+    "TBD"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/FLOW/01_preparation_phase/setup_data_cleaning_tools.ipynb

@@ -0,0 +1,39 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# データクリーニングツールの環境構築を行う\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "TBD"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/FLOW/01_preparation_phase/work_with_GakuNinRDM.ipynb

@@ -0,0 +1,65 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# GakuNin RDMと連携する\n",
+    "\n",
+    "GakuNin RDMと連携するためのノートブックです。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## ◆◆◆開発メモ◆◆◆\n",
+    "GakuNin RDMとの連携方法が明確でないためマークダウンのみ。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1.[GakuNin RDM](https://rdm.nii.ac.jp/)に遷移する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2.GakuNin RDMにサインインして新規プロジェクトを作成する\n",
+    "\n",
+    "※既存プロジェクトと連携する場合はこのステップを飛ばしてください。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3.データガバナンス機能連携アドオンを設定する"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/FLOW/02_experimental_phase/launch_an_experiment.ipynb

@@ -0,0 +1,299 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 実験を開始する\n",
+    "\n",
+    "実験パッケージを用意して実験の開始をサポートします。  \n",
+    "上から順番に実行してください。  "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. 実験用リポジトリの作成、リポジトリ名の入力、実験の入力データの設定を行う"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### - 1.1  実験用の空リポジトリを作成する\n",
+    "\n",
+    "以下のリンクに遷移し、データガバナンス機能で実験用の空リポジトリを作成してください。  \n",
+    "※空のリポジトリを作成するために、「選択されたファイルおよびテンプレートでリポジトリを初期化」のチェックを外してください。  \n",
+    " [データガバナンス機能に遷移する](http://dg02.dg.rcos.nii.ac.jp/repo/create)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### - 1.2 作成したリポジトリ名を入力する\n",
+    "\n",
+    "以下のセルを実行してリポジトリ名を入力してください。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "repository_title = input(\"リポジトリ名\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### - 1.3 この実験の入力データを既存の実験の出力データからコピーしたい場合は以下を実行する\n",
+    "\n",
+    "以下のセルを実行して、この実験の入力データとしたい出力データを持つ既存実験のリポジトリをセレクトボックスから選択してください。  \n",
+    "※入力データのコピーが不要であれば、1.3をスキップして2.に進んでください。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from ipywidgets import Select, Button\n",
+    "from IPython.display import clear_output\n",
+    "\n",
+    "# 実験リポジトリ名を取得する\n",
+    "import os\n",
+    "path = '/home/jovyan/experiments'\n",
+    "files = os.listdir(path)\n",
+    "files_dir = [f for f in files if os.path.isdir(os.path.join(path, f)) and f != '.ipynb_checkpoints']\n",
+    "\n",
+    "# セレクトボックスで選択された実験リポジトリ名をex_repoに格納する\n",
+    "def on_click_callback(clicked_button: Button) -> None:\n",
+    "    global ex_repo\n",
+    "    ex_repo = select.value\n",
+    "\n",
+    "button = Button(description='選択完了')\n",
+    "select = Select(options=files_dir)\n",
+    "button.on_click(on_click_callback)\n",
+    "display(select, button)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true
+   },
+   "source": [
+    "## 2. 実験環境を整える"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.1 実験リポジトリのリモートURLを生成する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%cd ~/\n",
+    "tmp = !git remote -v\n",
+    "tmp = tmp[0]\n",
+    "tmp = tmp[tmp.find(\"http\"):]\n",
+    "tmp = tmp.split(\"/\")\n",
+    "tmp = tmp[0] + \"//\" + tmp[2] + \"/\" + tmp[3]\n",
+    "\n",
+    "remote_url = tmp + \"/\" + repository_title + \".git\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.2 空リポジトリ名のフォルダを作成し、実験パッケージをコピーする"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "!mkdir ~/.tmp/\n",
+    "!git init ~/.tmp/$repository_title\n",
+    "# !datalad create ~/.tmp/$repository_title\n",
+    "!cp -r ~/WORKFLOW/PACKAGE/. ~/.tmp/$repository_title/\n",
+    "# ex_repoが存在するなら、既存の実験の出力データをinput_dataにコピーする\n",
+    "try:\n",
+    "    ex_repo\n",
+    "    %cd /home/jovyan/experiments/$ex_repo\n",
+    "    !git pull\n",
+    "    %cd ~/\n",
+    "    !cp -r ~/experiments/$ex_repo/output_data/. ~/.tmp/$repository_title/input_data/\n",
+    "except:\n",
+    "    pass\n",
+    "%cd ~/.tmp/$repository_title\n",
+    "!git add ~/.tmp/$repository_title\n",
+    "!git commit -m '1st commit'\n",
+    "!git remote add $repository_title $remote_url\n",
+    "!git push $repository_title master\n",
+    "# !datalad save --to-git -m \"1st commit\"\n",
+    "# !datalad push"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.3 上記リポジトリをサブデータセットとして設定する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%cd ~/\n",
+    "!git submodule add $remote_url ./experiments/$repository_title"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.4. pipeline.jsonに実験名を追記する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "\n",
+    "with open('/home/jovyan/experiments/pipeline.json', 'r') as f:\n",
+    "    pipeline = json.load(f)\n",
+    "\n",
+    "pipeline.append(repository_title)\n",
+    "\n",
+    "with open('/home/jovyan/experiments/pipeline.json', 'w') as f:\n",
+    "    json.dump(pipeline, f, indent = 4)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.5 このノートブックを保存する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "from IPython.display import display, Javascript\n",
+    "display(Javascript('IPython.notebook.save_checkpoint();'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.6 実行結果をデータガバナンス機能に反映する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%cd ~/\n",
+    "!git add ~/\n",
+    "!git commit -m '実験の作成'\n",
+    "!git push origin master\n",
+    "# !datalad save --to-git -m \"実験の作成\"\n",
+    "# !datalad push"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. 実験リポジトリにアクセスして実験を開始する\n",
+    "\n",
+    "以下のセルを実行した後に出力されるリンクをクリックして実験に移ってください。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import urllib\n",
+    "\n",
+    "print(\n",
+    "    \"https://binder.cs.rcos.nii.ac.jp/v2/git/\" + urllib.parse.quote(remote_url, safe='') + \"/HEAD?filepath=experiment.ipynb\"\n",
+    ")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/FLOW/02_experimental_phase/monitor_repository.ipynb

@@ -0,0 +1,257 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# モニタリングを実施する\n",
+    "\n",
+    "複数の実験の再現性をモニタリングします。  \n",
+    "モニタリングでは、[pipeline.json](../../../experiments.pipeline.json)に設定された順番に実験が再現されます。  \n",
+    "以下のセルを上から実行してください。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## ◆◆◆開発メモ◆◆◆\n",
+    "\n",
+    "pipeline.jsonの説明をどこかで入れる"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1.パイプライン定義ファイルのチェック結果を確認する\n",
+    "\n",
+    "パイプライン定義ファイルとパッケージ群の情報に相違がないか確認します。  \n",
+    "以下のセルを実行し、エラーが起きないことを確認してください。  \n",
+    "エラーが起きた場合は、表示されたリンクに遷移してパイプライン定義ファイルを修正＆エラーが起きなくなるまで再実行を繰り返してください。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import json\n",
+    "import glob\n",
+    "\n",
+    "# 作業ディレクトリに移動\n",
+    "%cd ~/experiments/\n",
+    "\n",
+    "# 実験パイプライン定義ファイルの読み込み\n",
+    "pipeline = json.load(open(\"pipeline.json\"))\n",
+    "print(\"実験パイプライン：\" + str(pipeline))\n",
+    "\n",
+    "# 実験パッケージ群の読み込み\n",
+    "files = glob.glob(\"./*/\")\n",
+    "\n",
+    "experiments = []\n",
+    "for file in files:\n",
+    "    experiments += [os.path.basename(file[:-1])]\n",
+    "print(\"実験パッケージ　：\" + str(experiments))\n",
+    "\n",
+    "# 元のディレクトリに移動\n",
+    "%cd -\n",
+    "\n",
+    "# 過不足の確認\n",
+    "if set(pipeline) != set(experiments):\n",
+    "    link = \"https://jupyter.cs.rcos.nii.ac.jp\" + os.environ[\"JUPYTERHUB_SERVICE_PREFIX\"] + \"edit/experiments/pipeline.json\"\n",
+    "    raise Exception(\"実験パイプラインと実験パッケージ間で相違があります\\n\\n以下のURLに遷移して修正してください\\n\" + link)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true
+   },
+   "source": [
+    "## 2.実験パイプラインを実行する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.1 実験パイプラインの定義情報を取得する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "\n",
+    "# 作業ディレクトリに移動\n",
+    "%cd ~/experiments/\n",
+    "\n",
+    "# 実験パイプライン定義ファイルの読み込み\n",
+    "pipeline = json.load(open(\"pipeline.json\"))\n",
+    "print(\"実験パイプライン：\" + str(pipeline))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.2 実験パッケージを最新化する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "for experiment in pipeline:\n",
+    "    %cd $experiment\n",
+    "    !git pull\n",
+    "    %cd ../"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.3 実験パイプラインを実行する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%cd ~/experiments/\n",
+    "\n",
+    "import papermill\n",
+    "\n",
+    "outputs = []\n",
+    "\n",
+    "for experiment in pipeline:\n",
+    "    print(\"\\n============================================================ \" + experiment)\n",
+    "\n",
+    "    print(\"\\n# 出力ディレクトリの初期化\")\n",
+    "    cmd = \"rm -rf \" + experiment + \"/output_data/*\"\n",
+    "    print(\"$ \" + cmd)\n",
+    "    !$cmd\n",
+    "    \n",
+    "    print(\"\\n前実験の出力データを当実験の入力データとして複製\")\n",
+    "    for output in outputs:\n",
+    "        cmd = \"cp -rf \" + output + \"/output_data/* \" + experiment + \"/input_data/\" \n",
+    "        print(\"$ \" + cmd)\n",
+    "        !$cmd\n",
+    "\n",
+    "    print(\"\\n実験再現のための環境構築\")\n",
+    "    cmd = \"pip install -r \" + experiment + \"/requirements.txt\"\n",
+    "    print(\"$ \" + cmd)\n",
+    "    !$cmd\n",
+    "\n",
+    "    print(\"\\n実験ソースコードの実行\")\n",
+    "    cmd = experiment + \"/source/\"\n",
+    "    print(\"$ cd \" + cmd)\n",
+    "    %cd $cmd\n",
+    "\n",
+    "    papermill.execute_notebook(\"main.ipynb\", \"log.ipynb\")\n",
+    "    \n",
+    "    cmd = \"-\"\n",
+    "    print(\"$ cd \" + cmd)\n",
+    "    %cd $cmd\n",
+    "\n",
+    "    print(\"\\n次実験に備えて本実験の出力データを登録\")\n",
+    "    outputs += [experiment]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.4 このノートブックを保存する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "from IPython.display import display, Javascript\n",
+    "display(Javascript('IPython.notebook.save_checkpoint();'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.5 実行結果をデータガバナンス機能に反映する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%cd ~/\n",
+    "!git add ~/\n",
+    "!git commit -m 'モニタリング実施'\n",
+    "\n",
+    "# import papermill as pm\n",
+    "# \n",
+    "# pm.execute_notebook(\n",
+    "#     '../util/datalad_save_push.ipynb',\n",
+    "#     '-',\n",
+    "#     parameters = dict(SAVE_MESSAGE = 'パイプライン構築', PATH = [未定])\n",
+    "# )"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/FLOW/03_after_research_phase/publish.ipynb

@@ -0,0 +1,54 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 研究成果を公開する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## ◆◆◆開発メモ◆◆◆\n",
+    "後期フェーズについては不明確な部分が多いためプロトタイプではTBDとする"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## キュレーション機能と連携する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 公開基盤と連携する"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/FLOW/README.md

@@ -0,0 +1,17 @@
+# FLOWS
+
+## FLOWSでは分野ごとのフロー群を整理する
+
+## ◆◆◆開発メモ◆◆◆
+
+現時点では、プロトタイプを動かすために以下の作業が必要なのでprepare_for_datalad_push.ipynbで作業を定義している。
+
+- .gitconfigにユーザー名とメールアドレスを登録する
+- jupyterhubでid_rsa（秘密鍵）を.ssh下に配置し、ginリポジトリにペアの公開鍵を登録する
+- .gitignoreで.local以下のファイルをgit管理から外す
+- .ssh/configで以下を指定
+    host *
+            StrictHostKeyChecking no
+            UserKnownHostsFile=/dev/null
+
+

WORKFLOW/FLOW/make_FLOW.ipynb

@@ -0,0 +1,138 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# フローを作成する\n",
+    "\n",
+    "研究の準備、実施、公開までの研究者のワークフローを支援するためのフローを作成し、フローに遷移します。  \n",
+    "以下のセルを上から順番に実行してください。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true
+   },
+   "source": [
+    "## 1.FLOWを作成する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.1 FROWの関連図を作成する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "# フォントをインストールする\n",
+    "%cd ~/\n",
+    "!mkdir ~/.fonts\n",
+    "%cd ~/.fonts\n",
+    "!wget http://archive.ubuntu.com/ubuntu/pool/universe/f/fonts-ipafont/fonts-ipafont-gothic_00303-18ubuntu1_all.deb\n",
+    "!dpkg-deb -x fonts-ipafont-gothic_00303-18ubuntu1_all.deb ~/.fonts\n",
+    "!cp ~/.fonts/usr/share/fonts/opentype/ipafont-gothic/ipag.ttf ~/.fonts/ipag.ttf\n",
+    "\n",
+    "# 関連図を作成する\n",
+    "%run /home/jovyan/WORKFLOW/scripts/nb_utils.py\n",
+    "setup_diag()\n",
+    "filename=generate_svg_diag(diag='/home/jovyan/WORKFLOW/images/notebooks.diag')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.2 FLOW.mdを作成する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "from IPython.display import Markdown\n",
+    "%cd /home/jovyan/WORKFLOW\n",
+    "%run scripts/nb_utils.py\n",
+    "%cd FLOW\n",
+    "md_list = [\n",
+    "    '## フロー支援機能の使い方',\n",
+    "    '「ワークフロー実行準備」を最初に実行したあと、研究準備フェーズから実験期フェーズ、実験終了後フェーズの順に進んでください。<br>各フェーズでは、リンクを押下して、表示される画面の手順に沿って実行することでワークフローを実施できます。  ',\n",
+    "    '以下で流れを確認してから、ワークフロー実施に移ってください。'\n",
+    "    '[![関連図](../images/notebooks.svg)](../images/notebooks.svg)',\n",
+    "    '## 0. ワークフローを実行するための準備をする',\n",
+    "    '次のフェーズに進む前に必ず実行する必要があります。<br>ワークフロー機能を使うのが初めての場合は、「FLOWの立ち上げ毎に実行」、「初回立ち上げ時のみ実行」の順にで実行してください。<br>2回目以降は、ワークフロー機能を実行する前に「FLOWの立ち上げ毎に実行」を実行してください。',\n",
+    "    str(notebooks_toc('util')),\n",
+    "    '## 1. 研究準備フェーズ',\n",
+    "    '研究準備フェーズでは実験前の準備をサポートします。',\n",
+    "    str(notebooks_toc('01_preparation_phase')),\n",
+    "    '## 2. 実験期フェーズ',\n",
+    "    '実験期フェーズでは実験の開始から、複数の実験の再現性の確認などをサポートします。<br>まずは、「実験を開始する」から実験を作成してください。実験の終了後、複数の実験の再現性を確認したいタイミングで「実験パイプラインをチェックする」「実験パイプラインを実行する」を実行してください。',\n",
+    "    str(notebooks_toc('02_experimental_phase')),\n",
+    "    '## 3 実験終了後フェーズ',\n",
+    "    '実験終了後フェーズでは研究成果の公開をサポートします。',\n",
+    "    str(notebooks_toc('03_after_research_phase'))\n",
+    "]\n",
+    "%cd ~/\n",
+    "with open('WORKFLOW/FLOW/FLOW.md', 'w') as f:\n",
+    "    f.write('\\n'.join(md_list))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2.作成したFLOWに遷移する\n",
+    "\n",
+    "以下のセルを実行すると表示されるリンクをクリックしてFLOWに遷移してください。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "print(\"https://jupyter.cs.rcos.nii.ac.jp\" + os.environ[\"JUPYTERHUB_SERVICE_PREFIX\"] + \"notebooks/WORKFLOW/FLOW/FLOW.md\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/FLOW/param_files/param_for_dataladsiblings.json

@@ -0,0 +1,4 @@
+{
+    "http":"Push  URL: http://dg02.dg.rcos.nii.ac.jp/",
+    "ssh":"ssh://root@dg02.dg.rcos.nii.ac.jp:3001/"
+}

WORKFLOW/FLOW/util/README.md

@@ -0,0 +1 @@
+どのフェーズでも必要なセクションをモジュール化する

WORKFLOW/FLOW/util/datalad_save_push.ipynb

@@ -0,0 +1,105 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# ワークフロー実行結果を書き戻す\n",
+    "\n",
+    "このノートブックでは、変更内容をGINリポジトリに書き戻します。  \n",
+    "※各セクションのセルでpapermillによって自動実行されるノートブックなので手作業での実施は非推奨です。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": [
+     "parameters"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "SIBLING_NAME = \"gin\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "※PATHが指定されない場合は/home/jovyan/をPATHとする"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": [
+     "parameters"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "PATH = \"/home/jovyan\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1.書き戻しの準備を行う"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from datalad import api\n",
+    "\n",
+    "api.save(message=SAVE_MESSAGE, path=PATH, recursive=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2.変更内容を書き戻す"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from datalad import api\n",
+    "\n",
+    "api.push(to=SIBLING_NAME, result_renderer='default', path=PATH, recursive=True)"
+   ]
+  }
+ ],
+ "metadata": {
+  "celltoolbar": "Tags",
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/FLOW/util/prepare_for_workflow.ipynb

@@ -0,0 +1,188 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 初回FLOW立ち上げ時に必要な準備を行う\n",
+    "\n",
+    "ワークフロー実行のための準備を行います。FLOWの立ち上げの初回のみ、「FLOWの立ち上げ毎に実行」の後に実行してください。  \n",
+    "上から順番に実行してください。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true
+   },
+   "source": [
+    "## 1.ワークフロー実行準備を行う"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.1  Dataladを使用するためにこのリポジトリをデータセットとする"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%%sh\n",
+    "cd ~/\n",
+    "datalad create --force"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.2 Git管理対象外ファイルを.gitignoreで設定する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%%sh\n",
+    "\n",
+    "echo \"\n",
+    "/.cache/\n",
+    "/.conda/\n",
+    "/.config/\n",
+    "/.ipython/\n",
+    "/.local/\n",
+    "/.tmp/\n",
+    "/.bashrc\n",
+    "/.bash_logout\n",
+    "/.profile\n",
+    "/.netrc\n",
+    ".ipynb_checkpoints/\n",
+    "\" > ~/.gitignore"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.3 実験用フォルダを作成し、その中にpipeline.jsonを用意する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%%sh\n",
+    "mkdir ~/experiments\n",
+    "echo \"[]\" > ~/experiments/pipeline.json"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.4 必要なファイルをPACKAGEテンプレートに追加する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "!cd ~/\n",
+    "!cp .gitconfig WORKFLOW/PACKAGE/.gitconfig\n",
+    "!cp .gitignore WORKFLOW/PACKAGE/.gitignore\n",
+    "!echo -e \"channels:\\n  - conda-forge\\n  - defaults\\ndependencies:\\n  -datalad\" >> /home/jovyan/WORKFLOW/PACKAGE/environment.yml"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.5 このノートブックを保存する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "from IPython.display import display, Javascript\n",
+    "display(Javascript('IPython.notebook.save_checkpoint();'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.6 実行結果をデータガバナンス機能に反映する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%cd ~/\n",
+    "!datalad save --to-git -m \"save\"\n",
+    "!datalad push"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/FLOW/util/prepare_for_workflow_every_time.ipynb

@@ -0,0 +1,126 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "4b6b7c2a",
+   "metadata": {},
+   "source": [
+    "# FLOW起動毎に必要な準備を行う\n",
+    "\n",
+    "FLOWの立ち上げ毎に、ワークフロー実行のために必要なGINのユーザー情報を保持します。  \n",
+    "上から順番に実行してください。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5db3e526",
+   "metadata": {},
+   "source": [
+    "## ◆◆◆開発メモ◆◆◆\n",
+    "ゆくゆくはSSOに対応し、コンテナ立ち上げ毎のユーザー名、パスワード、メールアドレスの入力を避けたい"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "396fd7ef",
+   "metadata": {},
+   "source": [
+    "## 1.ユーザー情報を情報を入力する\n",
+    "\n",
+    "以下のセルを実行してデータガバナンス機能のユーザー名、パスワード、メールアドレスを入力してください。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b8192cc3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from IPython.display import clear_output\n",
+    "import getpass\n",
+    "name = input(\"ユーザー名：\")\n",
+    "password = getpass.getpass(\"パスワード：\")\n",
+    "email = input(\"メールアドレス：\")\n",
+    "clear_output()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fee70e65",
+   "metadata": {
+    "heading_collapsed": true
+   },
+   "source": [
+    "## 2.ユーザー情報を登録する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b722abea",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.1 .netrcにユーザー情報を追加する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8c77d50b",
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "!echo -e \"machine dg02.dg.rcos.nii.ac.jp\\nlogin $name\\npassword $password\" > ~/.netrc"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "60c49da3",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.2 .gitconfigにユーザー情報を追加する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c8a66562",
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "!cd ~/\n",
+    "!git config --global user.name $name\n",
+    "!git config --global user.email $email"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

WORKFLOW/PACKAGE/SECTIONS/enter_metadata.ipynb

@@ -0,0 +1,194 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 実験メタデータを入力する\n",
+    "\n",
+    "実験日や実験者などのメタデータを実験記録に追加します。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. メタデータを入力する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "1. 以下のセルを実行して、実験日を入力してください。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from ipywidgets import Text, Button\n",
+    "from IPython.display import clear_output\n",
+    "import datetime\n",
+    "\n",
+    "d_today = datetime.date.today()\n",
+    "def on_click_callback(clicked_button: Button) -> None:\n",
+    "    global experiment_date\n",
+    "    experiment_date = text.value\n",
+    "    clear_output()\n",
+    "    print(\"登録しました：\", experiment_date)\n",
+    "\n",
+    "# テキストボックス\n",
+    "text = Text(\n",
+    "    value=str(d_today),\n",
+    "    description='実験日：'\n",
+    ")\n",
+    "button = Button(description='入力完了')\n",
+    "button.on_click(on_click_callback)\n",
+    "display(text, button)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "2. 以下のセルを実行して、実験者を入力してください。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from ipywidgets import Text, Button\n",
+    "from IPython.display import clear_output\n",
+    "import datetime\n",
+    "\n",
+    "def on_click_callback(clicked_button: Button) -> None:\n",
+    "    global experimenter\n",
+    "    experimenter = text.value\n",
+    "    clear_output()\n",
+    "    print(\"登録しました：\", experimenter)\n",
+    "\n",
+    "# テキストボックス\n",
+    "text = Text(\n",
+    "    description='実験者：'\n",
+    ")\n",
+    "button = Button(description='入力完了')\n",
+    "button.on_click(on_click_callback)\n",
+    "display(text, button)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true
+   },
+   "source": [
+    "## 2.メタ情報をファイルに保存する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.1 メタ情報をmeta_data.jsonに書き込む"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "from collections import OrderedDict\n",
+    "\n",
+    "meta_data = {\n",
+    "    \"experiment_date\": experiment_date,\n",
+    "    \"experimenter\": experimenter\n",
+    "}\n",
+    "with open('../meta_data.json', 'w') as jf:\n",
+    "    json.dump(meta_data, jf, ensure_ascii=False, indent=2)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.2 このノートブックを保存する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "from IPython.display import display, Javascript\n",
+    "display(Javascript('IPython.notebook.save_checkpoint();'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 2.3 メタ情報ファイルとこのノートブックを保存する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%cd ~/\n",
+    "!git add './meta_data.json'\n",
+    "!git commit -m 'メタデータ入力'\n",
+    "# import papermill as pm\n",
+    "# \n",
+    "# pm.execute_notebook(\n",
+    "#     './util/datalad_save_push.ipynb',\n",
+    "#     '-',\n",
+    "#     parameters = dict(SAVE_MESSAGE = 'メタデータ入力', PATH = '/home/jovyan/meta_data.json')\n",
+    "# )"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/PACKAGE/SECTIONS/finish.ipynb

@@ -0,0 +1,117 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 実験を終了する\n",
+    "\n",
+    "実験記録をデータガバナンス機能に保存して、実験を終了します。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true
+   },
+   "source": [
+    "## 1.実験記録を保存し実験終了する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.1 実行環境構成を記録する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%%sh\n",
+    "conda env export > ~/environment.yml\n",
+    "pip freeze > ~/requirements.txt"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.2 このノートブックを保存する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "from IPython.display import display, Javascript\n",
+    "display(Javascript('IPython.notebook.save_checkpoint();'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### - 1.3 実験記録をデータガバナンス機能に保存する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "%cd ~/\n",
+    "!git add ~/\n",
+    "!git commit -m '実験終了'\n",
+    "!git push origin master\n",
+    "\n",
+    "# import papermill as pm\n",
+    "# \n",
+    "# pm.execute_notebook(\n",
+    "#     './util/datalad_save_push.ipynb',\n",
+    "#     '-',\n",
+    "#     parameters = dict(SAVE_MESSAGE = save_message)\n",
+    "# )"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/PACKAGE/SECTIONS/save.ipynb

@@ -0,0 +1,72 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 実験を途中保存する\n",
+    "\n",
+    "これは、実験終了前に実験を途中で保存するためのノートブックです。  \n",
+    "途中保存ではGINリポジトリへの書き戻しは行われませんが、変更履歴が残ります。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "1. コミットメッセージを変数に入力する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "save_message = input()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "2. 途中保存する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%cd ~/\n",
+    "!git add ~/\n",
+    "!git commit -m $save_message\n",
+    "\n",
+    "# from datalad import api\n",
+    "# api.save(message=save_message)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/PACKAGE/SECTIONS/util/datalad_save_push.ipynb

@@ -0,0 +1,105 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# ワークフロー実行結果を書き戻す\n",
+    "\n",
+    "このノートブックでは、変更内容をginリポジトリに書き戻します。  \n",
+    "※各セクションのセルでpapermillによって自動実行されるノートブックなので手作業での実施は非推奨です。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": [
+     "parameters"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "SIBLING_NAME = \"gin\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "※PATHが指定されない場合は/home/jovyan/をPATHとする"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": [
+     "parameters"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "PATH = \"/home/jovyan\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. 書き戻しの準備を行う"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from datalad import api\n",
+    "\n",
+    "api.save(message=SAVE_MESSAGE, path=PATH, recursive=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. 変更内容を書き戻す"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from datalad import api\n",
+    "\n",
+    "api.push(to=SIBLING_NAME, result_renderer='default', path=PATH, recursive=True)"
+   ]
+  }
+ ],
+ "metadata": {
+  "celltoolbar": "Tags",
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/PACKAGE/SECTIONS/util/prepare_for_workflow_every_time.ipynb

@@ -0,0 +1,102 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "4719bc46",
+   "metadata": {},
+   "source": [
+    "# 必要な準備を行う\n",
+    "\n",
+    "ワークフロー実行のために必要なGINのユーザー情報を保持します。  \n",
+    "上から順番に実行してください。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0c1b8c2d",
+   "metadata": {},
+   "source": [
+    "## ◆◆◆開発メモ◆◆◆\n",
+    "ゆくゆくはSSOに対応し、コンテナ立ち上げ毎のユーザー名、パスワード、メールアドレスの入力を避けたい"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3d968f7c",
+   "metadata": {},
+   "source": [
+    "## 1.ユーザー情報を情報を入力する\n",
+    "\n",
+    "以下のセルを実行してデータガバナンス機能のユーザー名、パスワード、メールアドレスを入力してください。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "178dfe28",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from IPython.display import clear_output\n",
+    "import getpass\n",
+    "name = input(\"ユーザー名：\")\n",
+    "password = getpass.getpass(\"パスワード：\")\n",
+    "email = input(\"メールアドレス：\")\n",
+    "clear_output()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e6bf534d",
+   "metadata": {
+    "heading_collapsed": true
+   },
+   "source": [
+    "## 2.ユーザー情報を登録する"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f7b22b8a",
+   "metadata": {
+    "hidden": true
+   },
+   "source": [
+    "### .netrcにユーザー情報を追加する"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d00ece15",
+   "metadata": {
+    "hidden": true
+   },
+   "outputs": [],
+   "source": [
+    "!echo -e \"machine dg02.dg.rcos.nii.ac.jp\\nlogin $name\\npassword $password\" > ~/.netrc"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

WORKFLOW/PACKAGE/environment.yml

@@ -0,0 +1,10 @@
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  -datalad
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  -datalad

WORKFLOW/PACKAGE/output_data/.gitkeep

@@ -0,0 +1 @@
+

WORKFLOW/PACKAGE/source/main.ipynb

@@ -0,0 +1,32 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

WORKFLOW/README.md

@@ -0,0 +1,3 @@
+# データガバナンス機能　ワークフロー
+
+[WORKFLOW/FLOWS/base_FLOW.ipynb](WORKFLOW/FLOWS/base_FLOW.ipynb)

WORKFLOW/images/notebooks.diag

@@ -0,0 +1,88 @@
+blockdiag {
+    node_width = 180
+    node_height = 120
+
+    group {
+        orientation = portrait;
+        shape = line;
+        style = none;
+
+        group {
+            shape = line;
+            style = none;
+            orientation = portrait;
+            group {
+                orientation = portrait;
+                label = "0. ワークフロー準備";
+                color = "#ffeed9";
+                "FLOW起動毎に必要な準備を行う";
+                "初回FLOW立ち上げ時に必要な準備を行う";
+            }
+        }
+
+        group {
+            shape = line;
+            style = none;
+            orientation = portrait;
+            group {
+                orientation = portrait;
+                label = "1. 研究準備フェーズ";
+                color = "#fcdcb1";
+                "データ分析ツールの環境構築を行う";
+                "データクリーニングツールの環境構築を行う";
+                "GakuNin RDMと連携する";
+            }
+        }
+        
+        group {
+            shape = line;
+            style = none;
+            orientation = portrait;
+            group {
+                orientation = portrait;
+                label = "2. 実験期フェーズ";
+                color = "#ffd6a1";
+                "実験を開始する";
+                "リポジトリ構成をモニタリングする";
+            }
+        }
+        group {
+            shape = line;
+            style = none;
+            orientation = portrait;
+            group {
+                orientation = portrait;
+                label = "3. 実験終了後フェーズ";
+                color = "#fcb14e";
+                "研究成果を公開する";
+            }
+        }
+        group {
+            shape = line;
+            style = none;
+            orientation = portrait;
+            group {
+                orientation = portrait;
+                label = "4. 研究終了後";
+                color = "#fcb14e";
+                "研究を終了する";
+            }
+        }
+    }
+
+    "FLOW起動毎に必要な準備を行う" -> "初回FLOW立ち上げ時に必要な準備を行う";
+    "初回FLOW立ち上げ時に必要な準備を行う" -> "データクリーニングツールの環境構築を行う";
+    "初回FLOW立ち上げ時に必要な準備を行う" -> "データ分析ツールの環境構築を行う";
+    "初回FLOW立ち上げ時に必要な準備を行う" -> "GakuNin RDMと連携する";
+
+    "データクリーニングツールの環境構築を行う" -> "実験を開始する";
+    "データ分析ツールの環境構築を行う" -> "実験を開始する";
+    "GakuNin RDMと連携する" -> "実験を開始する";
+    
+    "実験を開始する" -> "実験を開始する";
+    "実験を開始する" -> "リポジトリ構成をモニタリングする";
+
+    "リポジトリ構成をモニタリングする" -> "研究成果を公開する";
+    
+    "研究成果を公開する" -> "研究を終了する";
+}

WORKFLOW/scripts/nb_utils.py

@@ -0,0 +1,291 @@
+# -*- coding: utf-8 -*-
+import os
+import re
+import sys
+import shutil
+from subprocess import run, CalledProcessError
+from tempfile import TemporaryDirectory
+from pathlib import Path
+from lxml import etree
+from nbformat import read, NO_CONVERT
+from itertools import chain, zip_longest
+from jinja2 import Template
+from datetime import datetime
+
+title_font_size = 11
+item_font_size = 9
+head_margin = 3
+text_margin = 2
+
+SVG_TEXT = '{http://www.w3.org/2000/svg}text'
+SVG_RECT = '{http://www.w3.org/2000/svg}rect'
+
+
+def parse_headers(nb_path):
+    nb = read(str(nb_path), as_version=NO_CONVERT)
+
+    # Notebookのセルからmarkdownの部分を取り出し、行ごとのリストにする
+    lines = [
+        line.strip()
+        for line in chain.from_iterable(
+            cell['source'].split('\n')
+            for cell in nb.cells
+            if cell['cell_type'] == 'markdown'
+        )
+        if len(line.strip()) > 0 and not line.startswith('---')
+    ]
+
+    # h1, h2 の行とその次行の最初の１文を取り出す
+    headers = [
+        (' '.join(line0.split()[1:]),
+         line1.split("。")[0] if line1 is not None else '')
+        for (line0, line1) in zip_longest(lines, lines[1:])
+        if line0.startswith('# ') or line0.startswith('## ')
+    ]
+    # 最初の見出しはtitle, 残りはheadersとして返す
+    return {
+        'title': {
+            'text': _to_title_text(nb_path, headers[0][0]),
+            'summary': headers[0][1],
+        },
+        'headers': [
+            {
+                'text': text,
+                'summary': (
+                    summary if not re.match(r'(?:#|!\[)', summary) else ''),
+            }
+            for (text, summary) in headers[1:]
+        ],
+    }
+
+def _to_title_text(nb_path, text):
+    no = nb_path.name.split('-')[0]
+    title = text if not text.startswith('About:') else text[6:]
+    return f'{title}'
+
+def _get_notebook_headers(nb_dir):
+    return dict([
+        (nb.name, parse_headers(nb))
+        for nb in nb_dir.glob("*.ipynb")
+    ])
+
+def notebooks_toc(nb_dir):
+    nb_headers = sorted(
+        _get_notebook_headers(Path(nb_dir)).items(),
+        key=lambda x: x[0])
+    
+    return "\n".join(chain.from_iterable([
+        [
+            f'* [{headers["title"]["text"]}]({nb_dir}/{str(nb)})'
+        ] + list(chain.from_iterable([
+            [
+                f'    - {header["text"]}',
+                (f'      - {header["summary"]}'
+                 if len(header["summary"]) > 0 else ''),
+            ]
+            for header in headers['headers']
+        ]))
+        for nb, headers in nb_headers
+    ]))
+
+import json
+
+JSON = ""
+def load_json(PATH):
+    with open(PATH) as f:
+        JSON = json.load(f)
+        return JSON
+
+
+def setup_diag():
+    setup_blockdiag()
+    setup_lxml()
+
+def setup_lxml():
+    if not check_lxml():
+        install_lxml()
+    return check_lxml()
+
+
+def check_lxml():
+    try:
+        import lxml
+        return True
+    except ModuleNotFoundError:
+        return False
+
+
+def install_lxml():
+    run('pip install -q --user lxml', shell=True)
+    setup_python_path()
+
+
+def setup_blockdiag():
+    if not check_blockdiag():
+        install_blockdiag()
+    return check_blockdiag()
+
+def check_blockdiag():
+    try:
+        run('blockdiag -h', shell=True, check=True)
+        return True
+    except CalledProcessError:
+        return False
+
+def install_blockdiag():
+    run('pip install -q --user blockdiag', shell=True)
+    paths = os.environ['PATH'].split(':')
+    local_bin = str(Path('~/.local/bin').expanduser())
+    if local_bin not in paths:
+        paths.append(local_bin)
+        os.environ['PATH'] = ':'.join(paths)
+    if not check_blockdiag():
+        install_blockdiag()
+
+def generate_svg_diag(
+        output='/home/jovyan/WORKFLOW/images/notebooks.svg',
+        diag='images/notebooks.diag',
+        nb_dir='/home/jovyan/FLOW',
+        font='/home/jovyan/.fonts/ipag.ttf',
+):
+    with TemporaryDirectory() as workdir:
+        skeleton = Path(workdir) / 'skeleton.svg'
+        _generate_skeleton(skeleton, Path(diag), Path(font))
+        _embed_detail_information(Path(output), skeleton, Path(nb_dir))
+        return output
+
+def _generate_skeleton(output, diag, font):
+    run(['blockdiag', '-f', font, '-Tsvg', '-o', output, diag], check=True)
+
+def setup_python_path():
+    ver = sys.version_info
+    lib_path = f'~/.local/lib/python{ver.major}.{ver.minor}/site-packages'
+    lib_path = str(Path(lib_path).expanduser())
+    if lib_path not in sys.path:
+        sys.path.append(lib_path)
+
+def _embed_detail_information(output, skeleton, nb_dir):
+    # Notebookのヘッダ取得
+    nb_headers = _get_notebook_headers(nb_dir)
+
+    # 雛形の読み込み
+    tree = etree.parse(str(skeleton))
+
+    # 雛形をNotebook情報で置き換え
+    for elem in list(tree.findall(SVG_TEXT)):
+        if _is_target_rect(elem, nb_headers.keys()):
+            nb_name = _find_matching_notebook(nb_headers.keys(), elem.text)
+            _embed_info_in_one_rect(elem, nb_headers, nb_dir, nb_name)
+
+    # SVGの保存
+    output.parent.mkdir(parents=True, exist_ok=True)
+    with output.open(mode='wb') as f:
+        f.write(etree.tostring(tree, method='xml', pretty_print=True))
+
+def _is_target_rect(elem, notebooks):
+    return (
+        elem.getprevious() is not None and
+        elem.getprevious().tag == SVG_RECT and
+        len(elem.text) > 0 and
+        _find_matching_notebook(notebooks, elem.text) is not None)
+
+def _find_matching_notebook(notebooks, prefix):
+    nb_prefix = prefix if prefix.find(':') < 0 else prefix.split(':')[0]
+    for nb in notebooks:
+        if nb.startswith(nb_prefix):
+            return nb
+
+def _embed_info_in_one_rect(elem, nb_headers, nb_dir, nb_name):
+    headers = nb_headers[nb_name]
+    nb_file = nb_dir / nb_name
+    rect_elem = elem.getprevious()
+    rect = (
+        (int(rect_elem.attrib['x']), int(rect_elem.attrib['y'])),
+        (int(rect_elem.attrib['width']), int(rect_elem.attrib['height'])))
+    childpos = elem.getparent().index(elem)
+    parent_elem = elem.getparent()
+    remove_texts(elem)
+    title = headers['title']['text']
+    if elem.text.find(':') >= 0:
+        title = title + ' - ' + elem.text.split(':')[1]
+    line_num = insert_title(parent_elem, childpos, rect, title, str(nb_file))
+    insert_headers(parent_elem, childpos, rect, headers['headers'], line_num)
+
+def remove_texts(elem):
+    old_text = elem
+    while old_text is not None:
+        if (old_text.getnext() is not None and
+                old_text.getnext().tag == SVG_TEXT):
+            next_text = old_text.getnext()
+        else:
+            next_text = None
+        old_text.getparent().remove(old_text)
+        old_text = next_text
+
+def insert_title(parent_elem, childpos, rect, title, link):
+    height_title = (
+        text_margin + (title_font_size + text_margin) * 2 + head_margin * 2)
+    lines = split_title(title)
+    if len(lines) == 2:
+        text_elem = create_text(rect, title_font_size, font_weight='bold')
+        text_elem.text = lines[0]
+        text_elem.attrib['y'] = str(
+                rect[0][1] + head_margin + text_margin + title_font_size)
+        text_elems = [text_elem]
+
+        text_elem = create_text(rect, title_font_size, font_weight='bold')
+        text_elem.text = lines[1]
+        text_elem.attrib['y'] = str(
+                rect[0][1] + height_title - text_margin - head_margin)
+        text_elems.append(text_elem)
+    else:
+        text_elem = create_text(rect, title_font_size, font_weight='bold')
+        text_elem.text = title
+        text_elem.attrib['y'] = str(
+                rect[0][1] + height_title // 2 + title_font_size // 2)
+        text_elems = [text_elem]
+
+    parent_elem.insert(childpos, create_anchor(text_elems, link))
+    return len(lines)
+
+def insert_headers(parent_elem, childpos, rect, headers, title_lines):
+    offset_y = (
+        text_margin +
+        (title_font_size + text_margin) * (title_lines + 1) +
+        head_margin * 2 + text_margin)
+    for i, header in enumerate(headers):
+        text_elem = create_text(rect, item_font_size)
+        text_elem.text = header['text']
+        text_elem.attrib['y'] = str(
+                rect[0][1] + offset_y + (item_font_size + text_margin) * i +
+                item_font_size)
+        parent_elem.insert(childpos, text_elem)
+
+def split_title(title):
+    if u'：' in title:
+        return [title[:title.index(u'：') + 1], title[title.index(u'：') + 1:]]
+    elif len(title) >= 15:
+        words = re.split(r'([-(（])', title, 1)
+        ret = words[0:1] + [''.join(x) for x in zip(words[1::2], words[2::2])]
+        return [re.sub(r'^--', '- ', x) for x in ret]
+    else:
+        return [title]
+
+def create_text(rect, font_size, font_weight='normal', font_style='normal'):
+    text_elem = etree.Element(SVG_TEXT)
+    text_elem.attrib['fill'] = 'rgb(0,0,0)'
+    text_elem.attrib['font-family'] = 'sans-serif'
+    text_elem.attrib['font-size'] = str(font_size)
+    text_elem.attrib['font-style'] = font_style
+    text_elem.attrib['font-weight'] = font_weight
+    text_elem.attrib['font-anchor'] = 'middle'
+    text_elem.attrib['x'] = str(rect[0][0] + text_margin)
+    text_elem.attrib['width'] = str(rect[1][0] - text_margin * 2)
+    return text_elem
+
+def create_anchor(elems, link):
+    a_elem = etree.Element('a')
+    a_elem.attrib['{http://www.w3.org/1999/xlink}href'] = link
+    for elem in elems:
+        a_elem.append(elem)
+    return a_elem

dmp.json

@@ -0,0 +1,37 @@
+{
+  "schema": "meti",
+  "field": "basic",
+  "dmpType": "New",
+  "agreementTitle": "The Data Management Plan",
+  "agreementDate": "2021-09-20",
+  "submitDate": "2021-10-01",
+  "corporateName": "The Corporate",
+  "researches": [
+    {
+      "index": 1,
+      "title": "The Research Data",
+      "description": "This is description.",
+      "manager": "John Doe",
+      "dataType": "My Data",
+      "releaseLevel": 4,
+      "concealReason": "nothing",
+      "concealPeriod": "",
+      "acquirer": "John Lab",
+      "acquireMethod": "by download link",
+      "remarks": ""
+    },
+    {
+      "index": 2,
+      "title": "The Research Data2",
+      "description": "This is description.",
+      "manager": "Jumpei Kuwata",
+      "dataType": "My Data",
+      "releaseLevel": 4,
+      "concealReason": "nothing",
+      "concealPeriod": "",
+      "acquirer": "IVIS Lab",
+      "acquireMethod": "by download link",
+      "remarks": ""
+    }
+  ]
+}

environment.yml

@@ -0,0 +1,11 @@
+name: Binder environment
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  - datalad
+  - pip:
+    - ploomber==0.12.8
+    - ploomber-scaffold==0.2.4
+    - lxml
+    - blockdiag

maDMP.ipynb

@@ -0,0 +1,89 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# ようこそ\n",
+    "\n",
+    "maDMPへようこそ。ここでは、DMPに入力いただいた研究分野等情報に基づき、ワークフローを最適化します。  \n",
+    "以下のセルを実行することで、ワークフローを生成します。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "# DMP情報\n",
+    "field = 'basic'"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "%sh\n",
+    "# ワークフローテンプレート取得\n",
+    "# 検証のため、参照先をivis-mizuguchi/workflow-templateの\n",
+    "# develop/workflow-templateブランチに変更中\n",
+    "# git clone https://github.com/ivis-kuwata/workflow-template.git /home/jovyan/WORKFLOW\n",
+    "git clone -b develop/workflow-template https://github.com/ivis-mizuguchi/workflow-template.git /home/jovyan/WORKFLOW\n",
+    "rm -rf /home/jovyan/WORKFLOW/.git"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "# dmp.jsonに\"fields\"プロパティがある想定\n",
+    "import os\n",
+    "import glob\n",
+    "import shutil\n",
+    "\n",
+    "path_flows = os.path.join('WORKFLOW', 'FLOW')\n",
+    "\n",
+    "templates = glob.glob(os.path.join(path_flows, '**'), recursive=True)\n",
+    "\n",
+    "# 選択外の分野のセクション群を削除\n",
+    "for tmpl in templates:\n",
+    "    file = os.path.basename(tmpl)\n",
+    "    if not os.path.isdir(tmpl) and os.path.splitext(file)[1] == '.ipynb':\n",
+    "        if 'base_' not in file and field not in file:\n",
+    "            os.remove(tmpl)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}