You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Jakub Valenta a2b1534d02 cli: Catch errors in providers 1 week ago
automatic_diary cli: Catch errors in providers 1 week ago
.ackrc ackrc: Add 10 months ago
.gitignore gitignore,Pipfile: Minor updates 9 months ago
COPYING Add COPYING and NOTICE 8 months ago
Makefile Don't change working dir when running from outside 5 months ago
NOTICE Add COPYING and NOTICE 8 months ago
Pipfile orgmodelist: Add provider 5 months ago
Pipfile.lock orgmodelist: Add provider 5 months ago orgmodelist: Add provider 5 months ago
automatic-diary Don't change working dir when running from outside 5 months ago
automatic-diary-visualize automatic-diary-visualize: Dont change working dir 4 months ago
automatic_diary_screenshot.png screenshot: Take it from Epiphany 8 months ago
config-sample.json README: Describe all providers 8 months ago
setup.cfg Rename to automatic-diary (from automatic-journal) 8 months ago visualize: Add convenient script to run it 8 months ago
tox.ini tox,setup: Fix testrunners 8 months ago

Automatic Diary

Automatic Diary is a script that creates one timeline from various digital sources describing your life.

Automatic Diary screenshot

On the screenshot above, you can see: calendar events (blue), completed todo list items (yellow), software development work (gray), sport activity (green), and films watched (red).


Automatic Diary can be useful to those who:

(a) have bad memory,\ (b) want to remember what they did,\ © but don’t have the time to write a real diary.

What is collected

  • [X] Calendar events (via CalDAV service and iCalendar file reading)
  • [X] Emails sent and received (via Maildir reading)
  • [X] Sport activity (via spreadsheet parsing)
  • [X] Facebook and Twitter posts (via Facebook/Twitter data archive parsing)
  • [X] Software development work (via Git repository log)
  • [X] Completed todo list items (via todo.txt parsing)
  • [X] Films watched (via ČSFD website parsing)
  • [X] Custom diary notes (via plain-text and Org-mode files parsing)

What is not collected

Automatic Diary could be extended to collect also the following data but right now it is considered low priority:

  • Custom notes and ideas
  • Articles and blog posts written
  • Phone calls and text messages (SMS)
  • Social media private messages and messaging apps
  • Browser history
  • Music listened (via
  • OpenStreetMap contributions
  • Money transfers
  • News headlines
  • Weather



$ brew install python
$ pip install pipenv
$ make setup

Arch Linux

# pacman -S pipenv libsecret
$ make setup

Other systems

Install these dependencies manually:

  • Python 3.7
  • pipenv
  • libsecret (Linux keyring) -- required by the caldav provider

Then run:

$ make setup


Before you run Automatic Diary, you need to configure all the providers (sources from which the data for your timeline will be read). The following providers are supported:

All providers are configured using a single config.json file. Use config-sample.json as a template for your own configuration.


  • Input: CalDAV server

  • Output: Names and locations of calendar events

  • Configuration:

        "url": "<server url>",
        "username": "<server authentication username>",
        "password_key": "<server authentication password -- libsecret key>",
        "password_val": "<server authentication password -- libsecret value>",
        "cache_dir": "<cache directory path>"


  • Input: User profile on ČSFD (film database website, something like IMDB)

  • Output: Titles of films rated

  • Configuration:

        "profile_url": "< profile url>",
        "cache_dir": "<cache directory path>"


  • Input: CSV spreadsheet (.csv) file

  • Output: Rows formatted using a template

  • Configuration

        "path": "<csv file path>",
        "date_source": "{{<column name>}}",
        "date_format": "<strptime date format>",
        "text_source": "<template string in the Mustache format>"


  • Input: Downloaded Facebook archive

  • Output: Texts of statuses

  • Configuration

        "path": "<path to wall.htm or timeline.htm>",
        "username": "<facebook username>"


  • Input: Directory with checked-out Git repositories and an author name

  • Output: Commit messages by the author from all repositories

  • Configuration

    "config": {
        "base_path": "<path to directory - will be searched recursively for git repos>",
        "author": "<author name>"


  • Input: Calendar events stored offline in iCalendar (.ics) files

  • Output: Names and locations of calendar events

  • Configuration:

        "paths": [
            "<path to an .ics file>",

    Not that events from all the listed .ics files will be merged -- duplicate events removed.


  • Input: Emails stored offline in the Maildir format

  • Output: Subjects of emails

  • Configuration:

        "received_pathname": "<glob pathname of directories with received emails>",
        "sent_pathname": "<glob pathname of directories with sent emails>"


  • Input: Emacs Org-mode (.org) file in format:

    * <2019-01-17 Thu>
    Lorem ipsum
    * <2019-01-18 Fri>
    spam spam
  • Output: Example:

    2019-01-17,Lorem ipsum foo.
    2019-01-18,spam spam
  • Configuration:

        "path": "<path to the .org file>"


  • Input: Emacs Org-mode (.org) file in format:

    - Lorem ipsum foo. <2019-01-17 Thu>
    - bar <2019-01-18 Fri 11:30>
  • Output: Example:

    2019-01-17,Lorem ipsum foo.
  • Configuration:

        "path": "<path to the .org file>"


  • Input: Todo.txt completed tasks file (done.txt)

  • Output: Texts of completed tasks

  • Configuration

        "path": "<done.txt file path>"


  • Input: Downloaded Twitter archive

  • Output: Texts of tweets

  • Configuration

        "path": "<path to twitter archive directory>"


  • Input: Plain text (.txt) file in format:

    2015-12-02 St
    2015-12-03 Čt
  • Output: Example:

    2015-12-02,Lorem: Ipsum: Hierarchy
    2015-12-02,Lorem: Ipsum: Spam
  • Configuration

        "path": "<path to the .txt file>"


Generating CSV

The basic output of Automatic Diary is a CSV file. Generate it by running:

./automatic-diary <config path> <output csv path>


./automatic-diary ~/.config/automatic-diary/config.json ~/Desktop/automatic_diary.csv

The CSV output is in format:


Example CSV output:

2019-01-23T15:31:54-08:00,git,human-activities,model: Log exceptions while scanning
2019-01-24T09:00:00+01:00,caldav,,DHL Packstation
2019-01-25,todotxt,done.txt,Opravit Ondrovi kolo

See the help for all command line options:

./automatic-diary --help


The output CSV file can also be rendered as an HTML document which looks kind of like a calendar. See the screenshot above. Generate this HTML document by running:

./automatic-diary-visualize <csv path> <output html path>


./automatic-diary-visualize ~/Desktop/automatic_diary.csv ~/Desktop/automatic_diary.html

See the help for all command line options:

./automatic-diary-visualize --help



make setup-dev

Testing and linting

make test
make lint


make help