You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Go to file
Jakub Valenta 2c6d712b7b Version 0.4.0 2 years ago
automatic_diary cli: Improve help message 2 years ago
.ackrc ackrc: Add 5 years ago
.gitignore gitignore,Pipfile: Minor updates 5 years ago
COPYING Add COPYING and NOTICE 5 years ago
Makefile icalendar: Make compatible with ics 0.8.0-dev 2 years ago
NOTICE NOTICE: Update copyright 3 years ago icalendar: Make compatible with ics 0.8.0-dev 2 years ago
automatic-diary automatic-diary: Fix passing args 2 years ago
automatic-diary-visualize automatic-diary: Fix passing args 2 years ago
automatic_diary_screenshot.png screenshot: Take it from Epiphany 5 years ago
config-sample.json README: Describe all providers 5 years ago
poetry.lock pyproject: Use poetry instead of pipenv 2 years ago
pyproject.toml Version 0.4.0 2 years ago
setup.cfg Update all dependencies and fix typing 3 years ago Version 0.4.0 2 years ago
tox.ini pyproject: Use poetry instead of pipenv 2 years ago

Automatic Diary

Automatic Diary is a script that creates one timeline from various digital sources describing your life.

Automatic Diary screenshot

On the screenshot above, you can see: calendar events (blue), completed todo list items (yellow), software development work (gray), sport activity (green), and films watched (red).


Automatic Diary can be useful to those who:

(a) have bad memory,
(b) want to remember what they did,
(c) but don't have the time to write a real diary.

What is collected

  • Calendar events (via CalDAV service and iCalendar file reading)
  • Emails sent and received (via Maildir reading)
  • Sport activity (via spreadsheet parsing)
  • Facebook and Twitter posts (via Facebook/Twitter data archive parsing)
  • Software development work (via Git repository log)
  • Completed todo list items (via todo.txt parsing)
  • Films watched (via ČSFD website parsing)
  • Custom diary notes (via plain-text and Org-mode files parsing)

What is not collected

Automatic Diary could be extended to collect also the following data but right now it is considered low priority:

  • Custom notes and ideas
  • Articles and blog posts written
  • Phone calls and text messages (SMS)
  • Social media private messages and messaging apps
  • Browser history
  • Music listened (via
  • OpenStreetMap contributions
  • Money transfers
  • News headlines
  • Weather



$ brew install python
$ pip install poetry
$ make setup

Arch Linux

# pacman -S python-poetry libsecret
$ make setup

Other systems

Install these dependencies manually:

  • Python >= 3.7
  • poetry
  • libsecret (Linux keyring) -- required by the caldav provider

Then run:

$ make setup


Before you run Automatic Diary, you need to configure all the providers (sources from which the data for your timeline will be read). The following providers are supported:

All providers are configured using a single config.json file. Use config-sample.json as a template for your own configuration.


  • Input: CalDAV server

  • Output: Names and locations of calendar events

  • Configuration:

        "url": "<server url>",
        "username": "<server authentication username>",
        "password_key": "<server authentication password -- libsecret key>",
        "password_val": "<server authentication password -- libsecret value>",
        "cache_dir": "<cache directory path>"


  • Input: User profile on ČSFD (film database website, something like IMDB)

  • Output: Titles of films rated

  • Configuration:

        "profile_url": "< profile url>",
        "cache_dir": "<cache directory path>"


  • Input: CSV spreadsheet (.csv) file

  • Output: Rows formatted using a template

  • Configuration

        "path": "<csv file path>",
        "date_source": "{{<column name>}}",
        "date_format": "<strptime date format>",
        "text_source": "<template string in the Mustache format>"


  • Input: Downloaded Facebook archive

  • Output: Texts of statuses

  • Configuration

        "path": "<path to wall.htm or timeline.htm>",
        "username": "<facebook username>"


  • Input: Directory with checked-out Git repositories and an author name

  • Output: Commit messages by the author from all repositories

  • Configuration

    "config": {
        "base_path": "<path to directory - will be searched recursively for git repos>",
        "author": "<author name>"


  • Input: Calendar events stored offline in iCalendar (.ics) files

  • Output: Names and locations of calendar events

  • Configuration:

        "paths": [
            "<path to an .ics file>",

    Not that events from all the listed .ics files will be merged -- duplicate events removed.


  • Input: Emails stored offline in the Maildir format

  • Output: Subjects of emails

  • Configuration:

        "received_pathname": "<glob pathname of directories with received emails>",
        "sent_pathname": "<glob pathname of directories with sent emails>"


  • Input: Emacs Org-mode (.org) file in format:

      * <2019-01-17 Thu>
      Lorem ipsum
      * <2019-01-18 Fri>
      spam spam
  • Output: Example:

    2019-01-17,Lorem ipsum foo.
    2019-01-18,spam spam
  • Configuration:

        "path": "<path to the .org file>"


  • Input: Emacs Org-mode (.org) file in format:

      - Lorem ipsum foo. <2019-01-17 Thu>
      - bar <2019-01-18 Fri 11:30>
  • Output: Example:

    2019-01-17,Lorem ipsum foo.
  • Configuration:

        "path": "<path to the .org file>"


  • Input: Todo.txt completed tasks file (done.txt)

  • Output: Texts of completed tasks

  • Configuration

        "path": "<done.txt file path>"


  • Input: Downloaded Twitter archive

  • Output: Texts of tweets

  • Configuration

        "path": "<path to twitter archive directory>"


  • Input: Plain text (.txt) file in format:

    2015-12-02 St
    2015-12-03 Čt
  • Output: Example:

    2015-12-02,Lorem: Ipsum: Hierarchy
    2015-12-02,Lorem: Ipsum: Spam
  • Configuration

        "path": "<path to the .txt file>"


Generating CSV

The basic output of Automatic Diary is a CSV file. Generate it by running:

$ ./automatic-diary <config path> <output csv path>


$ ./automatic-diary ~/.config/automatic-diary/config.json ~/Desktop/automatic_diary.csv

The CSV output is in format:


Example CSV output:

2019-01-23T15:31:54-08:00,git,human-activities,model: Log exceptions while scanning
2019-01-24T09:00:00+01:00,caldav,,DHL Packstation
2019-01-25,todotxt,done.txt,Opravit Ondrovi kolo

See the help for all command line options:

$ ./automatic-diary --help


The output CSV file can also be rendered as an HTML document which looks kind of like a calendar. See the screenshot above. Generate this HTML document by running:

$ ./automatic-diary-visualize <csv path> <output html path>


$ ./automatic-diary-visualize ~/Desktop/automatic_diary.csv ~/Desktop/automatic_diary.html

See the help for all command line options:

$ ./automatic-diary-visualize --help


Testing and linting

$ make test
$ make lint


$ make help


Feel free to remix this project under the terms of the GNU General Public License version 3 or later. See COPYING and NOTICE.