.. _run5:

Cleaning up
-----------

Just now you realize that you need to fit both logos onto the same slide.
"Ah, damn, I might then really need to have them 400 by 400 pixel to fit",
you think. "Good that I know how to not run into the permission denied errors anymore!"

Therefore, we need to do the :command:`datalad run` command yet again - we wanted to have
the image in 400x400 px size. "Now this definitely will be the last time I'm running this",
you think.

.. runrecord:: _examples/DL-101-112-101
   :language: console
   :workdir: dl-101/DataLad-101
   :emphasize-lines: 5

   $ datalad run -m "Resize logo for slides" \
   --input "recordings/longnow/.datalad/feed_metadata/logo_interval.jpg" \
   --output "recordings/interval_logo_small.jpg" \
   "convert -resize 400x400 recordings/longnow/.datalad/feed_metadata/logo_interval.jpg recordings/interval_logo_small.jpg"

Oh for f**** sake... run is "impossible"?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Weird. After the initial annoyance about yet another error message faded,
and you read on,
DataLad informs that a "clean dataset" is required.
Run a :command:`datalad status` to see what is meant by this:

.. runrecord:: _examples/DL-101-112-102
   :language: console
   :workdir: dl-101/DataLad-101

   $ datalad status

Ah right. We forgot to save the notes we added, and thus there are
unsaved modifications present in ``DataLad-101``.
But why is this a problem?

By default, at the end of a :command:`datalad run` is a :command:`datalad save`.
Remember the section on :ref:`populate`: A general :command:`datalad save` without
a path specification will save *all* of the modified or untracked
contents to the dataset.

Therefore, in order to not mix any changes in the dataset that are unrelated
to the command plugged into :command:`datalad run`, by default it will only run
on a clean dataset with no changes or untracked files present.

There are two ways to get around this error message:
The obvious -- and recommended -- one is to save the modifications,
and run the command in a clean dataset.
We will try this way with the ``logo_interval.jpg``.
It would look like this:
First, save the changes,


.. runrecord:: _examples/DL-101-112-103
   :language: console
   :workdir: dl-101/DataLad-101

   $ datalad save -m "add additional notes on run options"

and then try again:

.. runrecord:: _examples/DL-101-112-104
   :language: console
   :workdir: dl-101/DataLad-101

   $ datalad run -m "Resize logo for slides" \
   --input "recordings/longnow/.datalad/feed_metadata/logo_interval.jpg" \
   --output "recordings/interval_logo_small.jpg" \
   "convert -resize 400x400 recordings/longnow/.datalad/feed_metadata/logo_interval.jpg recordings/interval_logo_small.jpg"

Note how in this execution of :command:`datalad run`, output unlocking was actually
necessary and DataLad provides a summary of this action in its output.

Add a quick addition to your notes about this way of cleaning up prior
to a :command:`datalad run`:

.. runrecord:: _examples/DL-101-112-105
   :language: console
   :workdir: dl-101/DataLad-101

   $ cat << EOT >> notes.txt
   Important! If the dataset is not "clean" (a datalad status output is empty),
   datalad run will not work - you will have to save modifications present in your
   dataset.
   EOT


A way of executing a :command:`datalad run` *despite* an "unclean" dataset,
though, is to add the ``--explicit`` flag to :command:`datalad run`.
We will try this flag with the remaining ``logo_salt.jpg``. Note that
we have an "unclean dataset" again because of the
addition to the note.


.. runrecord:: _examples/DL-101-112-106
   :language: console
   :workdir: dl-101/DataLad-101

   $ datalad run -m "Resize logo for slides" \
   --input "recordings/longnow/.datalad/feed_metadata/logo_salt.jpg" \
   --output "recordings/salt_logo_small.jpg" \
   --explicit \
   "convert -resize 400x400 recordings/longnow/.datalad/feed_metadata/logo_salt.jpg recordings/salt_logo_small.jpg"

With this flag, DataLad considers the specification of inputs and outputs to be "explicit".
It doesn't warn if the repository is dirty, but importantly, it
**only** saves modifications to the *listed outputs* (which is a problem in the
vast amount of cases where one does not exactly know which outputs are produced).

A :command:`datalad status` will show that your previously modified ``notes.txt``
is still modified:

.. runrecord:: _examples/DL-101-112-110
   :language: console
   :workdir: dl-101/DataLad-101

   $ datalad status

Add an additional note on the ``--explicit`` flag, and finally save your changes to ``notes.txt``.

.. runrecord:: _examples/DL-101-112-107
   :language: console
   :workdir: dl-101/DataLad-101

   $ cat << EOT >> notes.txt
   A suboptimal alternative is the --explicit flag,
   used to record only those changes done
   to the files listed with --output flags.

   EOT

.. runrecord:: _examples/DL-101-112-108
   :language: console
   :workdir: dl-101/DataLad-101

   $ datalad save -m "add note on clean datasets"

To conclude this section on :command:`datalad run`, take a look at the last :command:`datalad run`
commit to see a :term:`run record` with more content:

.. runrecord:: _examples/DL-101-112-109
   :language: console
   :workdir: dl-101/DataLad-101
   :lines: 1, 24-50
   :emphasize-lines: 11, 15-17, 18-20

   $ git log -p -2

.. todo::

   - how to escape special characters