RLiterate 2

RLiterate is a graphical tool for doing literate programming. Literate programming is a style of programming in which you don't write source code directly. Instead you write a document which has code snippets interspersed with regular text. It shifts the mindset of the programmer from writing code to writing a document in which the program is presented in small pieces.

This book describes RLiterate, its history, and also includes the complete implementation of the second version.

Showcase

TODO: Show an example how literate programming is done in RLiterate.

Concepts

TODO: Explain concepts and features in greater detail.

Why literate programming?

To me, the central idea in literate programming is that you should write programs for other humans. Only secondary for the machine. I find code easier to understand if I can understand it in isolated pieces. One example where it is difficult to isolate a piece without literate programming is test code. Usually the test code is located in one file, and the implementation in another. To fully understand the piece it is useful to both read the tests and the implementation. To do this you have to find this related information in two unrelated files. I wrote about this problem in 2013 in Related things are not kept together. Literate programming allows you to present the test and the implementation in the same place, yet have the different pieces written to different files. The compiler might require that they are in separate files, but with literate programming, you care first about the other human that will read our code, and only second about the compiler.

Another argument for literate programming is to express the "why". Why is this code here? Timothy Daly talks about it in his talk Literate Programming in the Large. He also argues that programmers must change the mindset from wring a program to writing a book. Some more can be read in Example of Literate Programming in HTML.

Some more resources about literate programming:

Implementation

This chapter contains the complete implementation of the second version of RLiterate.

Files

RLiterate is written in a single Python file that looks like this:

  1. rliterate2.py
#!/usr/bin/env python3

from collections import namedtuple, defaultdict, OrderedDict
from operator import add, sub, mul, floordiv
import base64
import contextlib
import cProfile
import io
import json
import math
import os
import pstats
import sys
import textwrap
import time
import uuid

from pygments.token import Token as TokenType
from pygments.token import string_to_tokentype
import pygments.lexers
import pygments.token
import wx

<<globals>>

<<decorators>>

<<functions>>

<<base base classes>>

<<base classes>>

<<classes>>

if __name__ == "__main__":
    main()

There is also a corresponding test file:

  1. test_rliterate2.py
from unittest.mock import Mock, call

import rliterate2

The remainder of this chapter will fill in the sections in those files to create the complete implementation.

Architecture overview

TODO: Introduce GUI framework, document model, and main frame.

Pattern for passing props

The architecture of the GUI language is to have a top level widget and a function that generates props for it like this:

frame MainFrame %layout_rows {
    ...
}

def main_frame_props(...):
    return {
        ...
    }

Let's say that the main frame has two sub components. It would probably look like this:

frame MainFrame %layout_rows {
    Foo {
        #foo
        %align[EXPAND]
    }
    Bar {
        #bar
        %align[EXPAND]
    }
}

def main_frame_props(...):
    return {
        "foo": foo_props(...),
        "bar": bar_props(...),
    }

def foo_props(...):
    return {
        ...
    }

def bar_props(...):
    return {
        ...
    }

Now let's say that there is a common prop in foo and bar called baz. Should foo_props and bar_props change to the following?

def foo_props(...):
    return {
        "baz": ...,
        ...
    }

def bar_props(...):
    return {
        "baz": ...,
        ...
    }

Or should the common prop be promoted to a main frame prop and passed in the GUI code like this?

frame MainFrame %layout_rows {
    Foo {
        #foo
        baz = #baz
        %align[EXPAND]
    }
    Bar {
        #bar
        baz = #baz
        %align[EXPAND]
    }
}

def main_frame_props(...):
    return {
        "foo": foo_props(...),
        "bar": bar_props(...),
        "baz": ...,
    }

def foo_props(...):
    return {
        ...
    }

def bar_props(...):
    return {
        ...
    }

Both ways are supported by the GUI language.

From a performance perspective, it probably doesn't matter too much. If the common prop is used deep in the widget hierarchy, it needs to be passed at each level if it's passed in the GUI code. If it's inserted in the prop function it can be inserted just where it needs to be.

The pattern with one prop function per widget is quite straight forward. If that is the way to go, perhaps the GUI language can be restricted to only allow that. Not sure if that is a good idea. But I like this pattern, and I will try to follow it.

The problem I encountered with this pattern was that when the column width changed, props for code had to be generated again and code had to be highlighted again. That shouldn't be necessary. But because caching only happened at the paragraph level, and the column width is part of the arguments to paragraph prop, it resulted in a cache miss. The solution is probably to cache deeper down as well.

Main function

  1. rliterate2.py
  2. functions 
def main():
    args = parse_args()
    start_app(
        MainFrame,
        create_props(
            main_frame_props,
            Document(args["path"])
        )
    )
  1. rliterate2.py
  2. functions 
def parse_args():
    args = {
        "path": None,
    }
    script = sys.argv[0]
    rest = sys.argv[1:]
    if len(rest) != 1:
        usage(script)
    args["path"] = rest[0]
    return args
  1. rliterate2.py
  2. functions 
def usage(script):
    sys.exit(f"usage: {script} <path>")

For reference: start_app, create_props.

Main frame

  1. rliterate2.py
  2. classes 
frame MainFrame %layout_rows {
    Toolbar {
        #toolbar
        %align[EXPAND]
    }
    ToolbarDivider {
        #toolbar_divider
        %align[EXPAND]
    }
    MainArea {
        #main_area
        %align[EXPAND]
        %proportion[1]
    }
}
  1. rliterate2.py
  2. functions 
def main_frame_props(document):
    return {
        "title": format_title(
            document.get(["path"])
        ),
        "toolbar": toolbar_props(
            document
        ),
        "toolbar_divider": toolbar_divider_props(
            document
        ),
        "main_area": main_area_props(
            document
        ),
    }
  1. rliterate2.py
  2. functions 
@cache()
def format_title(path):
    return "{} ({}) - RLiterate 2".format(
        os.path.basename(path),
        os.path.abspath(os.path.dirname(path))
    )

Toolbar

  1. rliterate2.py
  2. classes 
panel Toolbar %layout_columns {
    %space[#margin]
    ToolbarButton {
        icon    = "quit"
        %margin[#margin,TOP|BOTTOM]
    }
    ToolbarButton {
        icon    = "settings"
        @button = #actions.rotate_theme()
        %margin[#margin,TOP|BOTTOM]
    }
    %space[#margin]
    if (#text_fragment_selection) {
        Panel {
            #bar
            %align[EXPAND]
            %margin[#margin,TOP|BOTTOM]
        }
        %space[#margin]
        ToolbarButton {
            icon    = "bold"
            @button = self._add_text()
            %margin[#margin,TOP|BOTTOM]
        }
        %space[#margin]
    }
}

<<Toolbar>>
  1. rliterate2.py
  2. classes
  3. Toolbar 
def _add_text(self):
    selection = self.prop(["selection"])
    self.prop(["actions", "modify_paragraph"])(
        selection.value["paragraph_id"],
        selection.value["path"],
        lambda fragments: [{"type": "text", "text": "Hej!"}]+fragments,
        selection.update_value(dict(selection.value, **{
            "start": [0, 0],
            "end": [0, 3],
            "cursor_at_start": False,
        }))
    )
  1. rliterate2.py
  2. functions 
def toolbar_props(document):
    toolbar_theme = document.get(["theme", "toolbar"])
    selection = document.get(["selection"])
    if (selection.value and
        selection.value.get("what", None) == "text_fragments"):
        text_fragment_selection = True
    else:
        text_fragment_selection = False
    return {
        "background": toolbar_theme["background"],
        "margin": toolbar_theme["margin"],
        "bar": {
            "background": document.get(["theme", "toolbar_divider", "color"]),
            "size": (1, -1),
        },
        "text_fragment_selection": text_fragment_selection,
        "selection": selection,
        "actions": document.actions,
    }

Toolbar divider

  1. rliterate2.py
  2. classes 
panel ToolbarDivider %layout_rows {
}
  1. rliterate2.py
  2. functions 
def toolbar_divider_props(document):
    toolbar_divider_theme = document.get(["theme", "toolbar_divider"])
    return {
        "background": toolbar_divider_theme["color"],
        "min_size": (
            -1,
            toolbar_divider_theme["thickness"]
        ),
    }

Main area

  1. rliterate2.py
  2. classes 
panel MainArea %layout_columns {
    TableOfContents[toc] {
        #toc
        %align[EXPAND]
    }
    TableOfContentsDivider {
        #toc_divider
        @drag = self._on_toc_divider_drag(event)
        %align[EXPAND]
    }
    Workspace {
        #workspace
        %align[EXPAND]
        %proportion[1]
    }
}

<<MainArea>>
  1. rliterate2.py
  2. classes
  3. MainArea 
def _on_toc_divider_drag(self, event):
    if event.initial:
        toc = self.get_widget("toc")
        self._start_width = toc.get_width()
    else:
        self.prop(["actions", "set_toc_width"])(
            self._start_width + event.dx
        )
  1. rliterate2.py
  2. functions 
def main_area_props(document):
    return {
        "actions": document.actions,
        "toc": toc_props(
            document
        ),
        "toc_divider": toc_divider_props(
            document
        ),
        "workspace": workspace_props(
            document
        ),
    }

Table of contents

  1. rliterate2.py
  2. classes 
panel TableOfContents %layout_rows {
    if (#has_valid_hoisted_page) {
        Button {
            label   = "unhoist"
            @button = #actions.set_hoisted_page(None)
            %margin[#margin,ALL]
            %align[EXPAND]
        }
    }
    TableOfContentsScrollArea {
        #scroll_area
        %align[EXPAND]
        %proportion[1]
    }
}
  1. rliterate2.py
  2. functions 
def toc_props(document):
    return {
        "background": document.get(
            ["theme", "toc", "background"]
        ),
        "min_size": (
            max(50, document.get(["toc", "width"])),
            -1
        ),
        "has_valid_hoisted_page": is_valid_hoisted_page(
            document,
            document.get(["toc", "hoisted_page"]),
        ),
        "margin": 1 + document.get(
            ["theme", "toc", "row_margin"]
        ),
        "actions": document.actions,
        "scroll_area": toc_scroll_area_props(
            document
        ),
    }
  1. rliterate2.py
  2. functions 
def is_valid_hoisted_page(document, page_id):
    try:
        page = document.get_page(page_id)
        root_page = document.get_page()
        if page["id"] != root_page["id"]:
            return True
    except PageNotFound:
        pass
    return False
Scroll area
  1. rliterate2.py
  2. classes 
scroll TableOfContentsScrollArea %layout_rows {
    drop_target = "move_page"
    loop (#rows cache_limit=#rows_cache_limit) {
        TableOfContentsRow[rows] {
            $
            __reuse = $id
            __cache = True
            %align[EXPAND]
        }
    }
}

<<TableOfContentsScrollArea>>
  1. rliterate2.py
  2. classes
  3. TableOfContentsScrollArea 
_last_drop_row = None

def on_drag_drop_over(self, x, y):
    self._hide()
    drop_point = self._get_drop_point(x, y)
    if drop_point is not None:
        self._last_drop_row = self._get_drop_row(drop_point)
    if self._last_drop_row is not None:
        valid = self.prop(["actions", "can_move_page"])(
            self.prop(["dragged_page"]),
            drop_point.target_page,
            drop_point.target_index
        )
        self._last_drop_row.show_drop_line(
            self._calculate_indent(drop_point.level),
            valid=valid
        )
        return valid
    return False

def on_drag_drop_leave(self):
    self._hide()

def on_drag_drop_data(self, x, y, page_info):
    drop_point = self._get_drop_point(x, y)
    if drop_point is not None:
        self.prop(["actions", "move_page"])(
            self.prop(["dragged_page"]),
            drop_point.target_page,
            drop_point.target_index
        )

def _hide(self):
    if self._last_drop_row is not None:
        self._last_drop_row.hide_drop_line()

def _get_drop_point(self, x, y):
    lines = defaultdict(list)
    for drop_point in self.prop(["drop_points"]):
        lines[
            self._y_distance_to(
                self._get_drop_row(drop_point),
                y
            )
        ].append(drop_point)
    if lines:
        columns = {}
        for drop_point in lines[min(lines.keys())]:
            columns[self._x_distance_to(drop_point, x)] = drop_point
        return columns[min(columns.keys())]

def _get_drop_row(self, drop_point):
    return self.get_widget("rows", drop_point.row_index)

def _y_distance_to(self, row, y):
    span_y_center = row.get_y() + row.get_drop_line_y_offset()
    return int(abs(span_y_center - y))

def _x_distance_to(self, drop_point, x):
    return int(abs(self._calculate_indent(drop_point.level + 1.5) - x))

def _calculate_indent(self, level):
    return (
        (2 * self.prop(["row_margin"])) +
        (level + 1) * self.prop(["indent_size"])
    )
  1. rliterate2.py
  2. functions 
def toc_scroll_area_props(document):
    return dict(generate_rows_and_drop_points(document), **{
        "rows_cache_limit": document.count_pages() - 1,
        "row_margin": document.get(
            ["theme", "toc", "row_margin"]
        ),
        "indent_size": document.get(
            ["theme", "toc", "indent_size"]
        ),
        "dragged_page": document.get(
            ["toc", "dragged_page"]
        ),
        "actions": document.actions,
    })
  1. rliterate2.py
  2. functions 
def generate_rows_and_drop_points(document):
    try:
        root_page = document.get_page(
            document.get(["toc", "hoisted_page"])
        )
    except PageNotFound:
        root_page = document.get_page(None)
    return generate_rows_and_drop_points_page(
        root_page,
        toc_row_common_props(document),
        document.get(["toc", "collapsed"]),
        document.get(["toc", "dragged_page"]),
        document.get_open_pages(),
        0,
        False,
        0
    )
  1. rliterate2.py
  2. functions 
@cache(limit=1000, key_path=[0, "id"])
def generate_rows_and_drop_points_page(
    page,
    row_common_props,
    collapsed,
    dragged_page,
    open_pages,
    level,
    is_dragged,
    row_offset
):
    rows = []
    drop_points = []
    is_collapsed = page["id"] in collapsed
    is_dragged = is_dragged or page["id"] == dragged_page
    rows.append(toc_row_props(
        row_common_props,
        page,
        level,
        is_dragged,
        is_collapsed,
        open_pages
    ))
    if is_collapsed:
        target_index = len(page["children"])
    else:
        target_index = 0
    drop_points.append(TableOfContentsDropPoint(
        row_index=row_offset+len(rows)-1,
        target_index=target_index,
        target_page=page["id"],
        level=level+1
    ))
    if not is_collapsed:
        for target_index, child in enumerate(page["children"]):
            sub_result = generate_rows_and_drop_points_page(
                child,
                row_common_props,
                collapsed,
                dragged_page,
                open_pages,
                level+1,
                is_dragged,
                row_offset+len(rows)
            )
            rows.extend(sub_result["rows"])
            drop_points.extend(sub_result["drop_points"])
            drop_points.append(TableOfContentsDropPoint(
                row_index=row_offset+len(rows)-1,
                target_index=target_index+1,
                target_page=page["id"],
                level=level+1
            ))
    return {
        "rows": rows,
        "drop_points": drop_points,
    }
  1. rliterate2.py
  2. classes 
TableOfContentsDropPoint = namedtuple("TableOfContentsDropPoint", [
    "row_index",
    "target_index",
    "target_page",
    "level",
])
Row
  1. rliterate2.py
  2. classes 
panel TableOfContentsRow %layout_rows {
    TableOfContentsTitle[title] {
        #title
        @click       = #actions.open_page(#id)
        @drag        = self._on_drag(event)
        @right_click = self._on_right_click(event)
        @hover       = self._set_background(event.mouse_inside)
        %align[EXPAND]
    }
    TableOfContentsDropLine[drop_line] {
        #drop_line
        %align[EXPAND]
    }
}

<<TableOfContentsRow>>
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow 
def _on_drag(self, event):
    if not event.initial:
        self.prop(["actions", "set_dragged_page"])(
            self.prop(["id"])
        )
        try:
            event.initiate_drag_drop("move_page", {})
        finally:
            self.prop(["actions", "set_dragged_page"])(
                None
            )
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow 
def _on_right_click(self, event):
    event.show_context_menu([
        ("Hoist", self.prop(["level"]) > 0, lambda:
            self.prop(["actions", "set_hoisted_page"])(
                self.prop(["id"])
            )
        ),
    ])
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow 
def _set_background(self, hover):
    if hover:
        self.get_widget("title").update_props({
            "background": self.prop(["hover_background"]),
        })
    else:
        self.get_widget("title").update_props({
            "background": None,
        })
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow 
def get_drop_line_y_offset(self):
    drop_line = self.get_widget("drop_line")
    return drop_line.get_y() + drop_line.get_height() / 2
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow 
def show_drop_line(self, indent, valid):
    self.get_widget("drop_line").update_props({
        "active": True,
        "valid": valid,
        "indent": indent
    })
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow 
def hide_drop_line(self):
    self.get_widget("drop_line").update_props({
        "active": False,
    })
  1. rliterate2.py
  2. functions 
def toc_row_common_props(document):
    return {
        "row_margin": document.get(
            ["theme", "toc", "row_margin"]
        ),
        "indent_size": document.get(
            ["theme", "toc", "indent_size"]
        ),
        "foreground": document.get(
            ["theme", "toc", "foreground"]
        ),
        "hover_background": document.get(
            ["theme", "toc", "hover_background"]
        ),
        "divider_thickness": document.get(
            ["theme", "toc", "divider_thickness"]
        ),
        "dragdrop_color": document.get(
            ["theme", "dragdrop_color"]
        ),
        "dragdrop_invalid_color": document.get(
            ["theme", "dragdrop_invalid_color"]
        ),
        "placeholder_color": document.get(
            ["theme", "toc", "placeholder_color"]
        ),
        "font": document.get(
            ["theme", "toc", "font"]
        ),
        "actions": document.actions,
    }
  1. rliterate2.py
  2. functions 
def toc_row_props(
    row_common_props,
    page,
    level,
    is_dragged,
    is_collapsed,
    open_pages
):
    return {
        "id": page["id"],
        "hover_background": row_common_props["hover_background"],
        "actions": row_common_props["actions"],
        "level": level,
        "title": toc_row_title_props(
            row_common_props,
            page,
            level,
            is_dragged,
            is_collapsed,
            open_pages
        ),
        "drop_line": toc_row_drop_line_props(
            row_common_props
        ),
    }
Title
  1. rliterate2.py
  2. classes 
panel TableOfContentsTitle %layout_columns {
    %space[add(#row_margin mul(#level #indent_size))]
    if (#has_children) {
        ExpandCollapse {
            cursor       = "hand"
            size         = #indent_size
            collapsed    = #collapsed
            @click       = #actions.toggle_collapsed(#id)
            @drag        = None
            @right_click = None
            %align[EXPAND]
        }
    } else {
        %space[#indent_size]
    }
    Text {
        #text_props
        %align[EXPAND]
        %margin[#row_margin,ALL]
    }
}
  1. rliterate2.py
  2. functions 
def toc_row_title_props(
    row_common_props,
    page,
    level,
    is_dragged,
    is_collapsed,
    open_pages
):
    if page["title"]:
        text = page["title"]
        if is_dragged:
            color = row_common_props["dragdrop_invalid_color"]
        else:
            color = row_common_props["foreground"]
    else:
        text = "Enter title..."
        color = row_common_props["placeholder_color"]
    return dict(row_common_props, **{
        "id": page["id"],
        "text_props": TextPropsBuilder(**dict(row_common_props["font"],
            bold=page["id"] in open_pages,
            color=color
        )).text(text).get(),
        "level": level,
        "has_children": bool(page["children"]),
        "collapsed": is_collapsed,
        "dragged": is_dragged,
    })
Drop line
  1. rliterate2.py
  2. classes 
panel TableOfContentsDropLine %layout_columns {
    %space[#indent]
    Panel {
        background = self._get_color(#active #valid)
        %align[EXPAND]
        %proportion[1]
    }
}

<<TableOfContentsDropLine>>
  1. rliterate2.py
  2. classes
  3. TableOfContentsDropLine 
def _get_color(self, active, valid):
    if active:
        if valid:
            return self.prop(["color"])
        else:
            return self.prop(["invalid_color"])
    else:
        return None
  1. rliterate2.py
  2. functions 
def toc_row_drop_line_props(row_common_props):
    return {
        "indent": 0,
        "active": False,
        "valid": True,
        "min_size": (-1, row_common_props["divider_thickness"]),
        "color": row_common_props["dragdrop_color"],
        "invalid_color": row_common_props["dragdrop_invalid_color"],
    }

Table of contents divider

  1. rliterate2.py
  2. classes 
panel TableOfContentsDivider %layout_columns {
}
  1. rliterate2.py
  2. functions 
def toc_divider_props(document):
    toc_divider_theme = document.get(["theme", "toc_divider"])
    return {
        "background": toc_divider_theme["color"],
        "min_size": (
            toc_divider_theme["thickness"],
            -1
        ),
        "cursor": "size_horizontal",
    }

Workspace

  1. rliterate2.py
  2. classes 
hscroll Workspace %layout_columns {
    %space[#margin]
    loop (#columns) {
        Column {
            $
            %align[EXPAND]
        }
        Panel {
            #divider_panel
            @drag = self._on_divider_drag(event)
            %align[EXPAND]
        }
    }
}

<<Workspace>>
  1. rliterate2.py
  2. classes
  3. Workspace 
def _on_divider_drag(self, event):
    if event.initial:
        self._initial_width = self.prop(["page_body_width"])
    else:
        self.prop(["actions", "set_page_body_width"])(
            max(50, self._initial_width + event.dx)
        )
  1. rliterate2.py
  2. functions 
def workspace_props(document):
    return {
        "background": document.get(
            ["theme", "workspace", "background"]
        ),
        "margin": document.get(
            ["theme", "workspace", "margin"]
        ),
        "page_body_width": document.get(
            ["workspace", "page_body_width"]
        ),
        "actions": document.actions,
        "columns": columns_props(
            document
        ),
        "divider_panel": {
            "cursor": "size_horizontal",
            "min_size": (document.get(["theme", "workspace", "margin"]), -1)
        }
    }
  1. rliterate2.py
  2. functions 
@profile_sub("columns_props")
def columns_props(document):
    return [
        column_props(
            document,
            column,
            document.get(["selection"]).add("workspace", "column", index)
        )
        for index, column
        in enumerate(document.get(["workspace", "columns"]))
    ]
Column
  1. rliterate2.py
  2. classes 
vscroll Column %layout_rows {
    %space[#margin]
    loop (#column) {
        Page {
            $
            %align[EXPAND]
        }
        %space[#margin]
    }
}
  1. rliterate2.py
  2. functions 
def column_props(document, column, selection):
    column_prop = []
    index = 0
    for page_id in column:
        try:
            column_prop.append(page_props(
                document,
                document.get_page(page_id),
                selection.add("page", index, page_id)
            ))
            index += 1
        except PageNotFound:
            pass
    return {
        "min_size": (
            document.get(["workspace", "page_body_width"]) +
            2*document.get(["theme", "page", "margin"]) +
            document.get(["theme", "page", "border", "size"]),
            -1
        ),
        "margin": document.get(
            ["theme", "workspace", "margin"]
        ),
        "column": column_prop,
    }
Page
  1. rliterate2.py
  2. classes 
panel Page %layout_rows {
    PageTopRow {
        #
        %align[EXPAND]
        %proportion[1]
    }
    PageBottomBorder {
        #border
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. classes 
panel PageTopRow %layout_columns {
    PageBody {
        #body
        %align[EXPAND]
        %proportion[1]
    }
    PageRightBorder {
        #border
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. classes 
panel PageBody %layout_rows {
    Title {
        #title
        %margin[#margin,ALL]
        %align[EXPAND]
    }
    loop (#paragraphs) {
        $widget {
            $
            %margin[#margin,LEFT|BOTTOM|RIGHT]
            %align[EXPAND]
        }
    }
}
  1. rliterate2.py
  2. classes 
panel PageRightBorder %layout_rows {
    %space[#size]
    Panel {
        min_size   = makeTuple(#size -1)
        background = #color
        %align[EXPAND]
        %proportion[1]
    }
}
  1. rliterate2.py
  2. classes 
panel PageBottomBorder %layout_columns {
    %space[#size]
    Panel {
        min_size   = makeTuple(-1 #size)
        background = #color
        %align[EXPAND]
        %proportion[1]
    }
}
  1. rliterate2.py
  2. functions 
def page_props(document, page, selection):
    page_theme = document.get(["theme", "page"])
    return {
        "body": {
            "id": page["id"],
            "title": page_title_props(
                page,
                document.get(["workspace", "page_body_width"]),
                page_theme,
                selection.add("title"),
                document.actions
            ),
            "paragraphs": paragraphs_props(
                document,
                page["paragraphs"],
                selection.add("paragraphs")
            ),
            "background": page_theme["background"],
            "margin": page_theme["margin"],
        },
        "border": page_theme["border"],
    }
Title
  1. rliterate2.py
  2. classes 
panel Title %layout_rows {
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions 
@cache()
def page_title_props(page, page_body_width, page_theme, selection, actions):
    input_handler = TitleInputHandler(page, page_theme, selection, actions)
    return {
        "text_edit_props": {
            "text_props": dict(
                input_handler.text_props,
                break_at_word=True,
                line_height=page_theme["line_height"]
            ),
            "max_width": page_body_width,
            "selection": selection,
            "selection_color": page_theme["selection_border"],
            "input_handler": input_handler,
            "actions": actions,
        },
    }
  1. rliterate2.py
  2. classes 
class TitleInputHandler(StringInputHandler):

    def __init__(self, page, page_theme, selection, actions):
        self.page = page
        self.page_theme = page_theme
        self.selection = selection
        self.actions = actions
        StringInputHandler.__init__(
            self,
            self.page["title"],
            self.selection.value_here
        )

    def create_selection_value(self, values):
        return dict(values, what="page_title", page_id=self.page["id"])

    def build(self):
        builder = TextPropsBuilder(
            **self.page_theme["title_font"],
            selection_color=self.page_theme["selection_color"],
            cursor_color=self.page_theme["cursor_color"]
        )
        self.build_with_selection_value(builder, self.selection_value)
        return builder

    def build_with_selection_value(self, builder, selection_value):
        if self.data:
            if selection_value is not None:
                builder.selection_start(selection_value["start"])
                builder.selection_end(selection_value["end"])
                if self.selection.active:
                    if selection_value["cursor_at_start"]:
                        builder.cursor(selection_value["start"], main=True)
                    else:
                        builder.cursor(selection_value["end"], main=True)
            builder.text(self.data, index_increment=0)
        else:
            if self.selection.active:
                if selection_value is not None:
                    builder.cursor(main=True)
            builder.text("Enter title...", color=self.page_theme["placeholder_color"], index_constant=0)
        return builder.get()

    def save(self, new_title, new_selection_value):
        self.actions["edit_page"](
            self.page["id"],
            {
                "title": new_title,
            },
            self.selection.create(new_selection_value)
        )
Paragraphs
  1. rliterate2.py
  2. functions 
def paragraphs_props(document, paragraphs, selection):
    return [
        paragraph_props(
            dict(
                paragraph,
                meta=build_paragraph_meta(document, paragraph)
            ),
            document.get(["theme", "page"]),
            document.get(["workspace", "page_body_width"]),
            selection.add(index, paragraph["id"]),
            document.actions
        )
        for index, paragraph in enumerate(paragraphs)
    ]
  1. rliterate2.py
  2. functions 
def build_paragraph_meta(document, paragraph):
    meta = {
        "variables": {},
        "page_titles": {},
    }
    if paragraph["type"] in ["text", "quote", "image"]:
        build_text_fragments_meta(meta, document, paragraph["fragments"])
    if paragraph["type"] == "list":
        build_list_meta(meta, document, paragraph["children"])
    return meta

def build_list_meta(meta, document, children):
    for child in children:
        build_text_fragments_meta(meta, document, child["fragments"])
        build_list_meta(meta, document, child["children"])

def build_text_fragments_meta(meta, document, fragments):
    for fragment in fragments:
        if fragment["type"] == "variable":
            meta["variables"][fragment["id"]] = document.get_variable(fragment["id"])
        elif fragment["type"] == "reference":
            meta["page_titles"][fragment["page_id"]] = document.get_page(fragment["page_id"])["title"]
  1. rliterate2.py
  2. functions 
@cache(limit=1000, key_path=[0, "id"])
def paragraph_props(paragraph, page_theme, body_width, selection, actions):
    BUILDERS = {
        "text": text_paragraph_props,
        "quote": quote_paragraph_props,
        "list": list_paragraph_props,
        "code": code_paragraph_props,
        "image": image_paragraph_props,
    }
    return BUILDERS.get(
        paragraph["type"],
        unknown_paragraph_props
    )(paragraph, page_theme, body_width, selection, actions)
Text
  1. rliterate2.py
  2. classes 
panel TextParagraph %layout_rows {
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions 
@profile_sub("text_paragraph_props")
def text_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": TextParagraph,
        "text_edit_props": text_fragments_to_text_edit_props(
            paragraph,
            ["fragments"],
            selection,
            page_theme,
            actions,
            max_width=body_width,
        ),
    }
Quote
  1. rliterate2.py
  2. classes 
panel QuoteParagraph %layout_columns {
    %space[#indent_size]
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions 
@profile_sub("quote_paragraph_props")
def quote_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": QuoteParagraph,
        "text_edit_props": text_fragments_to_text_edit_props(
            paragraph,
            ["fragments"],
            selection,
            page_theme,
            actions,
            max_width=body_width-page_theme["indent_size"],
        ),
        "indent_size": page_theme["indent_size"],
    }
List
  1. rliterate2.py
  2. classes 
panel ListParagraph %layout_rows {
    loop (#rows) {
        ListRow {
            $
            %align[EXPAND]
        }
    }
}
  1. rliterate2.py
  2. classes 
panel ListRow %layout_columns {
    %space[mul(#level #indent)]
    Text {
        #bullet_props
    }
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions 
@profile_sub("list_paragraph_props")
def list_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": ListParagraph,
        "rows": list_item_rows_props(
            paragraph,
            paragraph["children"],
            paragraph["child_type"],
            page_theme,
            body_width,
            actions,
            selection
        ),
    }
  1. rliterate2.py
  2. functions 
def list_item_rows_props(paragraph, children, child_type, page_theme, body_width, actions, selection, path=[], level=0):
    rows = []
    for index, child in enumerate(children):
        rows.append(list_item_row_props(
            paragraph,
            child_type,
            index,
            child,
            page_theme,
            body_width,
            actions,
            selection.add(index, "fragments"),
            path+[index],
            level
        ))
        rows.extend(list_item_rows_props(
            paragraph,
            child["children"],
            child["child_type"],
            page_theme,
            body_width,
            actions,
            selection.add(index),
            path+[index, "children"],
            level+1
        ))
    return rows

def list_item_row_props(paragraph, child_type, index, child, page_theme, body_width, actions, selection, path, level):
    return {
        "level": level,
        "indent": page_theme["indent_size"],
        "bullet_props": dict(
            TextPropsBuilder(
                **page_theme["text_font"]
            ).text(_get_bullet_text(child_type, index)).get(),
            max_width=page_theme["indent_size"],
            line_height=page_theme["line_height"]
        ),
        "text_edit_props": text_fragments_to_text_edit_props(
            paragraph,
            ["children"]+path+["fragments"],
            selection,
            page_theme,
            actions,
            max_width=body_width-(level+1)*page_theme["indent_size"],
            line_height=page_theme["line_height"]
        ),
    }

def _get_bullet_text(list_type, index):
    if list_type == "ordered":
        return "{}. ".format(index + 1)
    else:
        return u"\u2022 "
Code
  1. rliterate2.py
  2. classes 
panel CodeParagraph %layout_rows {
    CodeParagraphHeader {
        #header
        %align[EXPAND]
    }
    CodeParagraphBody {
        #body
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions 
@profile_sub("code_paragraph_props")
def code_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": CodeParagraph,
        "header": code_paragraph_header_props(
            paragraph,
            page_theme,
            body_width
        ),
        "body": code_paragraph_body_props(
            paragraph,
            page_theme,
            body_width
        ),
    }
Header
  1. rliterate2.py
  2. classes 
panel CodeParagraphHeader %layout_rows {
    Text {
        #text_props
        %align[EXPAND]
        %margin[#margin,ALL]
    }
}
  1. rliterate2.py
  2. functions 
def code_paragraph_header_props(paragraph, page_theme, body_width):
    return {
        "background": page_theme["code"]["header_background"],
        "margin": page_theme["code"]["margin"],
        "text_props": dict(code_paragraph_header_path_props(
            paragraph["filepath"],
            paragraph["chunkpath"],
            page_theme["code_font"]
        ), max_width=body_width-2*page_theme["code"]["margin"]),
    }
  1. rliterate2.py
  2. functions 
@cache()
def code_paragraph_header_path_props(filepath, chunkpath, font):
    builder = TextPropsBuilder(**font)
    for index, x in enumerate(filepath):
        if index > 0:
            builder.text("/")
        builder.text(x)
    if filepath and chunkpath:
        builder.text(" ")
    for index, x in enumerate(chunkpath):
        if index > 0:
            builder.text("/")
        builder.text(x)
    return builder.get()
Body
  1. rliterate2.py
  2. classes 
panel CodeParagraphBody %layout_rows {
    Text {
        #text_props
        %align[EXPAND]
        %margin[#margin,ALL]
    }
}
  1. rliterate2.py
  2. functions 
def code_paragraph_body_props(paragraph, page_theme, body_width):
    return {
        "background": page_theme["code"]["body_background"],
        "margin": page_theme["code"]["margin"],
        "text_props": dict(
            code_paragraph_body_text_props(
                paragraph,
                page_theme
            ),
            max_width=body_width-2*page_theme["code"]["margin"]
        ),
    }
  1. rliterate2.py
  2. functions 
@cache(limit=100, key_path=[0, "id"])
def code_paragraph_body_text_props(paragraph, page_theme):
    builder = TextPropsBuilder(**page_theme["code_font"])
    for fragment in apply_token_styles(
        code_body_fragments_props(
            paragraph["fragments"],
            code_pygments_lexer(
                paragraph.get("language", ""),
                paragraph["filepath"][-1] if paragraph["filepath"] else "",
            )
        ),
        page_theme["token_styles"]
    ):
        builder.text(**fragment)
    return builder.get()
  1. rliterate2.py
  2. functions 
@profile_sub("apply_token_styles")
def apply_token_styles(fragments, token_styles):
    styles = build_style_dict(token_styles)
    def style_fragment(fragment):
        if "token_type" in fragment:
            token_type = fragment["token_type"]
            while token_type not in styles:
                token_type = token_type.parent
            style = styles[token_type]
            new_fragment = dict(fragment)
            new_fragment.update(style)
            return new_fragment
        else:
            return fragment
    return [
        style_fragment(fragment)
        for fragment in fragments
    ]
  1. rliterate2.py
  2. functions 
def build_style_dict(theme_style):
    styles = {}
    for name, value in theme_style.items():
        styles[string_to_tokentype(name)] = value
    return styles
  1. rliterate2.py
  2. functions 
def code_body_fragments_props(fragments, pygments_lexer):
    code_chunk = CodeChunk()
    for fragment in fragments:
        if fragment["type"] == "code":
            code_chunk.add(fragment["text"])
        elif fragment["type"] == "chunk":
            code_chunk.add(
                "{}<<{}>>\n".format(
                    fragment["prefix"],
                    "/".join(fragment["path"])
                ),
                {"token_type": TokenType.Comment.Preproc}
            )
    return code_chunk.tokenize(pygments_lexer)
  1. rliterate2.py
  2. functions 
def code_pygments_lexer(language, filename):
    try:
        if language:
            return pygments.lexers.get_lexer_by_name(
                language,
                stripnl=False
            )
        else:
            return pygments.lexers.get_lexer_for_filename(
                filename,
                stripnl=False
            )
    except:
        return pygments.lexers.TextLexer(stripnl=False)
Image
  1. rliterate2.py
  2. classes 
panel ImageParagraph %layout_rows {
    Image {
        #image
        %align[CENTER]
    }
    ImageText {
        #image_text
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. classes 
panel ImageText %layout_columns {
    %space[#indent]
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
    %space[#indent]
}
  1. rliterate2.py
  2. functions 
@profile_sub("image_paragraph_props")
def image_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": ImageParagraph,
        "image": {
            "base64_image": paragraph.get("image_base64", None),
            "width": body_width,
        },
        "image_text": {
            "indent": page_theme["indent_size"],
            "text_edit_props": text_fragments_to_text_edit_props(
                paragraph,
                ["fragments"],
                selection,
                page_theme,
                actions,
                align="center",
                max_width=body_width-2*page_theme["indent_size"],
            ),
        },
    }
Unknown paragraph
  1. rliterate2.py
  2. classes 
panel UnknownParagraph %layout_rows {
    Text {
        #text_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions 
def unknown_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": UnknownParagraph,
        "text_props": dict(
            TextPropsBuilder(**page_theme["code_font"]).text(
                "Unknown paragraph type '{}'.".format(paragraph["type"])
            ).get(),
            max_width=body_width
        ),
    }

Reusable widgets

TextEdit

The Text widget has support for rendering text. It has support for styling text as well as showing selections and cursors. This edit widget adds generic editing capabilities for data that has been projected to text.

Implementing a text editor has two parts: first the data has to be projected as text, and then an input handler needs to respond to events and change the data/selection accordingly.

  1. rliterate2.py
  2. classes 
panel TextEdit %layout_rows {
    selection_box = self._box(#selection #selection_color)
    Text[text] {
        #text_props
        max_width  = sub(#max_width mul(2 self._margin()))
        immediate  = self._immediate(#selection)
        focus      = self._focus(#selection)
        cursor     = self._get_cursor(#selection)
        @click     = self._on_click(event #selection)
        @left_down = self._on_left_down(event #selection)
        @drag      = self._on_drag(event #selection)
        @key       = self._on_key(event #selection)
        @focus     = self._on_focus(#selection)
        @unfocus   = self._on_unfocus(#selection)
        %align[EXPAND]
        %margin[self._margin(),ALL]
    }
}

<<TextEdit>>
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _box(self, selection, selection_color):
    return {
        "width": 1 if selection.value_here else 0,
        "color": selection_color,
    }
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _immediate(self, selection):
    return selection.value_here
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _focus(self, selection):
    if selection.value_here:
        return selection.stamp
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _margin(self):
    width = self.prop(["selection_box", "width"])
    if width > 0:
        return width + 2
    else:
        return 0
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _on_click(self, event, selection):
    if selection.value_here:
        return
    index = self._get_index(event.x, event.y)
    if index is not None:
        self.prop(["actions", "set_selection"])(

            selection.create(
                self.prop(["input_handler"]).create_selection_value({
                    "start": index,
                    "end": index,
                    "cursor_at_start": True,
                })
            )
        )
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _on_left_down(self, event, selection):
    if not selection.value_here:
        return
    index = self._get_index(event.x, event.y)
    if index is not None:
        self.prop(["actions", "set_selection"])(
            selection.create(
                self.prop(["input_handler"]).create_selection_value({

                    "start": index,
                    "end": index,
                    "cursor_at_start": True,
                })
            )
        )
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _on_drag(self, event, selection):
    if not selection.value_here:
        return
    if event.initial:
        self._initial_index = self._get_index(event.x, event.y)
    else:
        new_index = self._get_index(event.x, event.y)
        if self._initial_index is not None and new_index is not None:
            self.prop(["actions", "set_selection"])(
                selection.create(
                    self.prop(["input_handler"]).create_selection_value({
                        "start": min(self._initial_index, new_index),
                        "end": max(self._initial_index, new_index),
                        "cursor_at_start": new_index <= self._initial_index,
                    })
                )
            )
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _get_index(self, x, y):
    character, right_side = self.get_widget("text").get_closest_character_with_side(
        x,
        y
    )
    if character is not None:
        if right_side:
            return character.get("index_right", None)
        else:
            return character.get("index_left", None)
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _on_key(self, event, selection):
    if selection.value_here:
        self.prop(["input_handler"]).handle_key(
            event,
            self.get_widget("text")
        )
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _get_cursor(self, selection):
    if selection.value_here:
        return "beam"
    else:
        return None
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _on_focus(self, selection):
    self.prop(["actions", "activate_selection"])(selection.trail)
  1. rliterate2.py
  2. classes
  3. TextEdit 
def _on_unfocus(self, selection):
    self.prop(["actions", "deactivate_selection"])(selection.trail)
TextPropsBuilder
  1. rliterate2.py
  2. classes 
class TextPropsBuilder(object):

    def __init__(self, **styles):
        self._characters = []
        self._cursors = []
        self._selections = []
        self._base_style = dict(styles)
        self._main_cursor_char_index = None

    def get(self):
        return {
            "characters": self._characters,
            "cursors": self._cursors,
            "selections": self._selections,
            "base_style": self._base_style,
        }

    def get_main_cursor_char_index(self):
        return self._main_cursor_char_index

    def text(self, text, **kwargs):
        fragment = {}
        for field in TextStyle._fields:
            if field in kwargs:
                fragment[field] = kwargs[field]
        index_prefix = kwargs.get("index_prefix", None)
        if index_prefix is None:
            create_index = lambda x: x
        else:
            create_index = lambda x: index_prefix + [x]
        index_increment = kwargs.get("index_increment", None)
        index_constant = kwargs.get("index_constant", None)
        for index, character in enumerate(text):
            x = dict(fragment, text=character)
            if index_increment is not None:
                x["index_left"] = create_index(
                    index_increment + index
                )
                x["index_right"] = create_index(
                    index_increment + index + 1
                )
            if index_constant is not None:
                x["index_left"] = x["index_right"] = create_index(
                    index_constant
                )
            self._characters.append(x)
        return self

    def selection_start(self, offset=0, color=None):
        self._selections.append((
            self._index(offset),
            self._index(offset),
            color
        ))

    def selection_end(self, offset=0):
        last_selection = self._selections[-1]
        self._selections[-1] = (
            last_selection[0],
            self._index(offset),
            last_selection[2]
        )

    def cursor(self, offset=0, color=None, main=False):
        index = self._index(offset)
        self._cursors.append((index, color))
        if main:
            self._main_cursor_char_index = index

    def _index(self, offset):
        return len(self._characters) + offset
StringInputHandler
  1. rliterate2.py
  2. base classes 
class StringInputHandler(object):

    def __init__(self, data, selection_value):
        self.data = data
        self.selection_value = selection_value
        builder = self.build()
        self.text_props = builder.get()
        self.main_cursor_char_index = builder.get_main_cursor_char_index()

    @property
    def start(self):
        return self.selection_value["start"]

    @property
    def end(self):
        return self.selection_value["end"]

    @property
    def cursor_at_start(self):
        return self.selection_value["cursor_at_start"]

    @property
    def cursor(self):
        if self.cursor_at_start:
            return self.start
        else:
            return self.end

    @cursor.setter
    def cursor(self, position):
        self.selection_value = self.create_selection_value({
            "start": position,
            "end": position,
            "cursor_at_start": True,
        })

    @property
    def has_selection(self):
        return self.start != self.end

    def replace(self, text):
        self.data = self.data[:self.start] + text + self.data[self.end:]
        position = self.start + len(text)
        self.selection_value = self.create_selection_value({
            "start": position,
            "end": position,
            "cursor_at_start": True,
        })

    def handle_key(self, key_event, text):
        print(key_event)
        if key_event.key == "\x08": # Backspace
            if self.has_selection:
                self.replace("")
            else:
                self.selection_value = self.create_selection_value({
                    "start": self._next_cursor(self._cursors_left(text)),
                    "end": self.start,
                    "cursor_at_start": True,
                })
                self.replace("")
        elif key_event.key == "\x00": # Del (and many others)
            if self.has_selection:
                self.replace("")
            else:
                self.selection_value = self.create_selection_value({
                    "start": self.start,
                    "end": self._next_cursor(self._cursors_right(text)),
                    "cursor_at_start": False,
                })
                self.replace("")
        elif key_event.key == "\x02": # Ctrl-B
            self.cursor = self._next_cursor(self._cursors_left(text))
        elif key_event.key == "\x06": # Ctrl-F
            self.cursor = self._next_cursor(self._cursors_right(text))
        else:
            self.replace(key_event.key)
        self.save(self.data, self.selection_value)

    def _next_cursor(self, cursors):
        for cursor in cursors:
            if self._cursor_differs(cursor):
                return cursor
        return self.cursor

    def _cursor_differs(self, cursor):
        if cursor == self.cursor:
            return False
        builder = TextPropsBuilder()
        self.build_with_selection_value(builder, self.create_selection_value({
            "start": cursor,
            "end": cursor,
            "cursor_at_start": True,
        }))
        return builder.get_main_cursor_char_index() != self.main_cursor_char_index

    def _cursors_left(self, text):
        for char in text.char_iterator(self.main_cursor_char_index-1, -1):
            yield char.get("index_right")
            yield char.get("index_left")

    def _cursors_right(self, text):
        for char in text.char_iterator(self.main_cursor_char_index, 1):
            yield char.get("index_left")
            yield char.get("index_right")
Text fragments projection
  1. rliterate2.py
  2. functions 
def text_fragments_to_text_edit_props(paragraph, path, selection, page_theme, actions, align="left", **kwargs):
    input_handler = TextFragmentsInputHandler(
        paragraph,
        path,
        selection,
        actions["modify_paragraph"],
        page_theme
    )
    return {
        "text_props": dict(
            input_handler.text_props,
            break_at_word=True,
            line_height=page_theme["line_height"],
            align=align
        ),
        "selection": selection,
        "selection_color": page_theme["selection_border"],
        "input_handler": input_handler,
        "actions": actions,
        **kwargs,
    }
  1. rliterate2.py
  2. classes 
class TextFragmentsInputHandler(StringInputHandler):

    def __init__(self, paragraph, path, selection, modify_paragraph, page_theme):
        self.paragraph = paragraph
        self.path = path
        self.selection = selection
        self.modify_paragraph = modify_paragraph
        self.page_theme = page_theme
        data = self.paragraph
        for part in path:
            data = data[part]
        StringInputHandler.__init__(
            self,
            data,
            self.selection.value_here
        )

    def save(self, fragments, selection_value):
        self.modify_paragraph(
            self.paragraph["id"],
            self.path,
            lambda value: fragments,
            self.selection.create(selection_value)
        )

    def create_selection_value(self, values):
        return dict(
            values,
            what="text_fragments",
            paragraph_id=self.paragraph["id"],
            path=self.path
        )

    def build(self):
        builder = TextPropsBuilder(
            selection_color=self.page_theme["selection_color"],
            cursor_color=self.page_theme["cursor_color"],
            **self.page_theme["text_font"]
        )
        self.build_with_selection_value(builder, self.selection_value)
        return builder

    def build_with_selection_value(self, builder, selection_value):
        for index, fragment in enumerate(self.data):
            params = {}
            params["index_prefix"] = [index]
            start = None
            end = None
            cursor_at_start = True
            placeholder = False
            if selection_value is not None:
                cursor_at_start = selection_value["cursor_at_start"]
                if selection_value["start"][0] == index:
                    start = selection_value["start"][1]
                if selection_value["end"][0] == index:
                    end = selection_value["end"][1]
            if fragment["type"] == "text":
                params.update(
                    self.page_theme["token_styles"]["RLiterate.Text"]
                )
                text = fragment["text"]
            elif fragment["type"] == "strong":
                params.update(
                    self.page_theme["token_styles"]["RLiterate.Strong"]
                )
                text = fragment["text"]
            elif fragment["type"] == "emphasis":
                params.update(
                    self.page_theme["token_styles"]["RLiterate.Emphasis"]
                )
                text = fragment["text"]
            elif fragment["type"] == "code":
                params.update(
                    self.page_theme["token_styles"]["RLiterate.Code"]
                )
                text = fragment["text"]
            elif fragment["type"] == "variable":
                params.update(
                    self.page_theme["token_styles"]["RLiterate.Variable"]
                )
                text = self.paragraph["meta"]["variables"][fragment["id"]]
            elif fragment["type"] == "reference":
                params.update(
                    self.page_theme["token_styles"]["RLiterate.Reference"]
                )
                if fragment["text"]:
                    text = fragment["text"]
                    placeholder = False
                else:
                    text = self.paragraph["meta"]["page_titles"][fragment["page_id"]]
                    placeholder = True
            elif fragment["type"] == "link":
                params.update(
                    self.page_theme["token_styles"]["RLiterate.Link"]
                )
                if fragment["text"]:
                    text = fragment["text"]
                    placeholder = False
                else:
                    text = fragment["url"]
                    placeholder = True
            self.build_text(
                text,
                start,
                end,
                cursor_at_start,
                params,
                builder,
                placeholder
            )

    def build_text(self, text, start, end, cursor_at_start, params, builder, placeholder):
        if text:
            if placeholder and self.selection is not None:
                text = "<{}>".format(text)
                params["color"] = self.page_theme["placeholder_color"]
        else:
            placeholder = True
            text = "<enter text>"
            params["color"] = self.page_theme["placeholder_color"]
        if placeholder:
            params["index_constant"] = 0
        else:
            params["index_increment"] = 0
        if start is not None:
            if placeholder:
                builder.selection_start(0)
                if cursor_at_start and self.selection.active:
                    builder.cursor(1, main=True)
            else:
                builder.selection_start(start)
                if cursor_at_start and self.selection.active:
                    builder.cursor(start, main=True)
        if end is not None:
            if placeholder:
                builder.selection_end(len(text))
                if not cursor_at_start and self.selection.active:
                    builder.cursor(1, main=True)
            else:
                builder.selection_end(end)
                if not cursor_at_start and self.selection.active:
                    builder.cursor(end, main=True)
        builder.text(text, **params)

    def replace(self, text):
        before = self.data[:self.start[0]]
        left = self.data[self.start[0]]
        right = self.data[self.end[0]]
        after = self.data[self.end[0]+1:]
        if left is right:
            middle = [
                im_modify(
                    left,
                    ["text"],
                    lambda value: value[:self.start[1]] + text + value[self.end[1]:]
                ),
            ]
            position = [self.start[0], self.start[1]+len(text)]
        elif self.cursor_at_start:
            middle = [
                im_modify(
                    left,
                    ["text"],
                    lambda value: value[:self.start[1]] + text
                ),
                im_modify(
                    right,
                    ["text"],
                    lambda value: value[self.end[1]:]
                ),
            ]
            if not middle[1]["text"]:
                middle.pop(1)
            position = [self.start[0], self.start[1]+len(text)]
        else:
            middle = [
                im_modify(
                    left,
                    ["text"],
                    lambda value: value[:self.start[1]]
                ),
                im_modify(
                    right,
                    ["text"],
                    lambda value: text + value[self.end[1]:]
                ),
            ]
            if not middle[0]["text"]:
                middle.pop(0)
                position = [self.end[0]-1, len(text)]
            else:
                position = [self.end[0], len(text)]
        self.data = before + middle + after
        self.selection_value = self.create_selection_value({
            "start": position,
            "end": position,
            "cursor_at_start": True,
        })

    def handle_key(self, key_event, text):
        StringInputHandler.handle_key(self, key_event, text)

Document

  1. rliterate2.py
  2. classes 
class Document(Immutable):

    ROOT_PAGE_PATH = ["doc", "root_page"]

    def __init__(self, path):
        Immutable.__init__(self, {
            "path": path,
            "doc": load_document_from_file(path),
            "selection": Selection.empty(),
            <<Document/init>>
        })
        self._build_page_index()
        self.actions = {
            "can_move_page": self.can_move_page,
            "edit_page": self.edit_page,
            "modify_paragraph": self.modify_paragraph,
            "deactivate_selection": self.deactivate_selection,
            "move_page": self.move_page,
            "open_page": self.open_page,
            "rotate_theme": self.rotate,
            "set_dragged_page": self.set_dragged_page,
            "set_hoisted_page": self.set_hoisted_page,
            "set_page_body_width": self.set_page_body_width,
            "set_selection": self.set_selection,
            "set_toc_width": self.set_toc_width,
            "activate_selection": self.activate_selection,
            "toggle_collapsed": self.toggle_collapsed,
        }

    <<Document>>
  1. rliterate2.py
  2. functions 
def load_document_from_file(path):
    if os.path.exists(path):
        return load_json_from_file(path)
    else:
        return create_new_document()
  1. rliterate2.py
  2. functions 
def load_json_from_file(path):
    with open(path) as f:
        return json.load(f)
  1. rliterate2.py
  2. functions 
def create_new_document():
    return {
        "root_page": create_new_page(),
        "variables": {},
    }

Page index

  1. rliterate2.py
  2. classes
  3. Document 
def _build_page_index(self):
    def build(page, path, parent, index):
        page_meta = PageMeta(page["id"], path, parent, index)
        page_index[page["id"]] = page_meta
        for index, child in enumerate(page["children"]):
            build(child, path+["children", index], page_meta, index)
    page_index = {}
    build(self.get(self.ROOT_PAGE_PATH), self.ROOT_PAGE_PATH, None, 0)
    self._page_index = page_index
  1. rliterate2.py
  2. classes
  3. Document 
def _get_page_meta(self, page_id):
    if page_id not in self._page_index:
        raise PageNotFound()
    return self._page_index[page_id]
  1. rliterate2.py
  2. classes 
class PageNotFound(Exception):
    pass
  1. rliterate2.py
  2. classes 
class PageMeta(object):

    def __init__(self, id, path, parent, index):
        self.id = id
        self.path = path
        self.parent = parent
        self.index = index

Pages

  1. rliterate2.py
  2. functions 
def create_new_page():
    return {
        "id": genid(),
        "title": "New page...",
        "children": [],
        "paragraphs": [],
    }
  1. rliterate2.py
  2. classes
  3. Document 
def get_page(self, page_id=None):
    if page_id is None:
        return self.get(self.ROOT_PAGE_PATH)
    else:
        return self.get(self._get_page_meta(page_id).path)
  1. rliterate2.py
  2. classes
  3. Document 
def count_pages(self):
    return len(self._page_index)
  1. rliterate2.py
  2. classes
  3. Document 
def move_page(self, source_id, target_id, target_index):
    try:
        self._move_page(
            self._get_page_meta(source_id),
            self._get_page_meta(target_id),
            target_index
        )
    except PageNotFound:
        pass
  1. rliterate2.py
  2. classes
  3. Document 
def can_move_page(self, source_id, target_id, target_index):
    try:
        return self._can_move_page(
            self._get_page_meta(source_id),
            self._get_page_meta(target_id),
            target_index
        )
    except PageNotFound:
        return False
  1. rliterate2.py
  2. classes
  3. Document 
def _move_page(self, source_meta, target_meta, target_index):
    if not self._can_move_page(source_meta, target_meta, target_index):
        return
    source_page = self.get(source_meta.path)
    operation_insert = (
        target_meta.path + ["children"],
        lambda children: (
            children[:target_index] +
            [source_page] +
            children[target_index:]
        )
    )
    operation_remove = (
        source_meta.parent.path + ["children"],
        lambda children: (
            children[:source_meta.index] +
            children[source_meta.index+1:]
        )
    )
    if target_meta.id == source_meta.parent.id:
        insert_first = target_index > source_meta.index
    else:
        insert_first = len(target_meta.path) > len(source_meta.parent.path)
    if insert_first:
        operations = [operation_insert, operation_remove]
    else:
        operations = [operation_remove, operation_insert]
    with self.transaction():
        for path, fn in operations:
            self.modify(path, fn)
        self._build_page_index()
  1. rliterate2.py
  2. classes
  3. Document 
def _can_move_page(self, source_meta, target_meta, target_index):
    page_meta = target_meta
    while page_meta is not None:
        if page_meta.id == source_meta.id:
            return False
        page_meta = page_meta.parent
    if (target_meta.id == source_meta.parent.id and
        target_index in [source_meta.index, source_meta.index+1]):
        return False
    return True
  1. rliterate2.py
  2. classes
  3. Document 
def edit_page(self, source_id, attributes, new_selection):
    try:
        with self.transaction():
            path = self._get_page_meta(source_id).path
            for key, value in attributes.items():
                self.replace(path + [key], value)
            if new_selection is not None:
                self.set_selection(new_selection)
    except PageNotFound:
        pass

Paragraphs

  1. rliterate2.py
  2. classes
  3. Document 
def modify_paragraph(self, source_id, path, fn, new_selection):
    try:
        with self.transaction():
            self.modify(
                self._find_paragraph(source_id)+path,
                fn,
                only_if_differs=True
            )
            self.set_selection(new_selection)
    except ParagraphNotFound:
        pass
  1. rliterate2.py
  2. classes 
class ParagraphNotFound(Exception):
    pass
  1. rliterate2.py
  2. classes
  3. Document 
def _find_paragraph(self, paragraph_id):
    def find_in_page(page, path):
        for index, paragraph in enumerate(page["paragraphs"]):
            if paragraph["id"] == paragraph_id:
                return path + ["paragraphs", index]
        for index, child in enumerate(page["children"]):
            try:
                return find_in_page(child, path + ["children", index])
            except ParagraphNotFound:
                pass
        raise ParagraphNotFound()
    return find_in_page(self.get(self.ROOT_PAGE_PATH), self.ROOT_PAGE_PATH)

Variables

  1. rliterate2.py
  2. classes
  3. Document 
def get_variable(self, variable_id):
    return self.get(["doc", "variables", variable_id])

Id generation

  1. rliterate2.py
  2. functions 
def genid():
    return uuid.uuid4().hex

Code paragraph utilities

  1. rliterate2.py
  2. classes 
class CodeChunk(object):

    def __init__(self):
        self._fragments = []

    def add(self, text, extra={}):
        part = {"text": text}
        part.update(extra)
        self._fragments.append(part)

    def tokenize(self, pygments_lexer):
        self._apply_token_types(
            pygments_lexer.get_tokens(
                self._get_uncolorized_text()
            )
        )
        return self._fragments

    def _get_uncolorized_text(self):
        return "".join(
            part["text"]
            for part in self._fragments
            if "token_type" not in part
        )

    def _apply_token_types(self, pygments_tokens):
        part_index = 0
        for token_type, text in pygments_tokens:
            while "token_type" in self._fragments[part_index]:
                part_index += 1
            while text:
                if len(self._fragments[part_index]["text"]) > len(text):
                    part = self._fragments[part_index]
                    pre = dict(part)
                    pre["text"] = pre["text"][:len(text)]
                    pre["token_type"] = token_type
                    self._fragments[part_index] = pre
                    part_index += 1
                    post = dict(part)
                    post["text"] = post["text"][len(text):]
                    self._fragments.insert(part_index, post)
                    text = ""
                else:
                    part = self._fragments[part_index]
                    part["token_type"] = token_type
                    part_index += 1
                    text = text[len(part["text"]):]

Theme

  1. rliterate2.py
  2. classes
  3. Document 
base00  = "#657b83"
base1   = "#93a1a1"
yellow  = "#b58900"
orange  = "#cb4b16"
red     = "#dc322f"
magenta = "#d33682"
violet  = "#6c71c4"
blue    = "#268bd2"
cyan    = "#2aa198"
green   = "#859900"

DEFAULT_THEME = {
    "toolbar": {
        "margin": 4,
        "background": None,
    },
    "toolbar_divider": {
        "thickness": 1,
        "color": "#aaaaaf",
    },
    "toc": {
        "background": "#ffffff",
        "foreground": "#000000",
        "placeholder_color": "gray",
        "indent_size": 20,
        "row_margin": 2,
        "divider_thickness": 2,
        "hover_background": "#cccccc",
        "font": {
            "size": 10,
        },
    },
    "toc_divider": {
        "thickness": 3,
        "color": "#aaaaaf",
    },
    "workspace": {
        "background": "#cccccc",
        "margin": 12,
    },
    "page": {
        "indent_size": 20,
        "line_height": 1.2,
        "text_font": {
            "size": 10,
        },
        "title_font": {
            "size": 16,
        },
        "code_font": {
            "size": 10,
            "family": "Monospace",
        },
        "border": {
            "size": 2,
            "color": "#aaaaaf",
        },
        "background": "#ffffff",
        "selection_border": "red",
        "selection_color": "red",
        "cursor_color": "red",
        "placeholder_color": "gray",
        "margin": 10,
        "code": {
            "margin": 5,
            "header_background": "#eeeeee",
            "body_background": "#f8f8f8",
        },
        "token_styles": {
            "":                    {"color": base00},
            "Keyword":             {"color": green},
            "Keyword.Constant":    {"color": cyan},
            "Keyword.Declaration": {"color": blue},
            "Keyword.Namespace":   {"color": orange},
            "Name.Builtin":        {"color": red},
            "Name.Builtin.Pseudo": {"color": blue},
            "Name.Class":          {"color": blue},
            "Name.Decorator":      {"color": blue},
            "Name.Entity":         {"color": violet},
            "Name.Exception":      {"color": yellow},
            "Name.Function":       {"color": blue},
            "String":              {"color": cyan},
            "Number":              {"color": cyan},
            "Operator.Word":       {"color": green},
            "Comment":             {"color": base1},
            "Comment.Preproc":     {"color": magenta},
            "RLiterate.Text":      {},
            "RLiterate.Strong":    {"bold": True},
            "RLiterate.Emphasis":  {"italic": True},
            "RLiterate.Code":      {"family": "Monospace"},
            "RLiterate.Variable":  {"italic": True, "family": "Monospace"},
            "RLiterate.Reference": {"italic": True, "color": blue},
            "RLiterate.Link":      {"underlined": True, "color": blue},
        },
    },
    "dragdrop_color": "#ff6400",
    "dragdrop_invalid_color": "#cccccc",
}

ALTERNATIVE_THEME = {
    "toolbar": {
        "margin": 4,
        "background": "#dcd6c6",
    },
    "toolbar_divider": {
        "thickness": 2,
        "color": "#b0ab9e",
    },
    "toc": {
        "background": "#fdf6e3",
        "foreground": "#657b83",
        "placeholder_color": "gray",
        "indent_size": 22,
        "row_margin": 3,
        "divider_thickness": 3,
        "hover_background": "#d0cabb",
        "font": {
            "size": 12,
        },
    },
    "toc_divider": {
        "thickness": 5,
        "color": "#b0ab9e",
    },
    "workspace": {
        "background": "#d0cabb",
        "margin": 18,
    },
    "page": {
        "indent_size": 30,
        "line_height": 1.3,
        "text_font": {
            "size": 12,
        },
        "title_font": {
            "size": 18,
        },
        "code_font": {
            "size": 12,
            "family": "Monospace",
        },
        "border": {
            "size": 3,
            "color": "#b0ab9e",
        },
        "background": "#fdf6e3",
        "selection_border": "blue",
        "selection_color": "blue",
        "cursor_color": "blue",
        "placeholder_color": "gray",
        "margin": 14,
        "code": {
            "margin": 7,
            "header_background": "#eae4d2",
            "body_background": "#f3ecdb",
        },
        "token_styles": {
            "":                    {"color": base00},
            "Keyword":             {"color": green},
            "Keyword.Constant":    {"color": cyan},
            "Keyword.Declaration": {"color": blue},
            "Keyword.Namespace":   {"color": orange},
            "Name.Builtin":        {"color": red},
            "Name.Builtin.Pseudo": {"color": blue},
            "Name.Class":          {"color": blue},
            "Name.Decorator":      {"color": blue},
            "Name.Entity":         {"color": violet},
            "Name.Exception":      {"color": yellow},
            "Name.Function":       {"color": blue},
            "String":              {"color": cyan},
            "Number":              {"color": cyan},
            "Operator.Word":       {"color": green},
            "Comment":             {"color": base1},
            "Comment.Preproc":     {"color": magenta},
            "RLiterate.Text":      {},
            "RLiterate.Strong":    {"bold": True},
            "RLiterate.Emphasis":  {"italic": True},
            "RLiterate.Code":      {"family": "Monospace"},
            "RLiterate.Variable":  {"italic": True, "family": "Monospace"},
            "RLiterate.Reference": {"italic": True, "color": blue},
            "RLiterate.Link":      {"underlined": True, "color": blue},
       },
    },
    "dragdrop_color": "#dc322f",
    "dragdrop_invalid_color": "#cccccc",
}
  1. rliterate2.py
  2. classes
  3. Document
  4. init 
"theme": self.DEFAULT_THEME,
  1. rliterate2.py
  2. classes
  3. Document 
def rotate(self):
    if self.get(["theme"]) is self.ALTERNATIVE_THEME:
        self.replace(["theme"], self.DEFAULT_THEME)
    else:
        self.replace(["theme"], self.ALTERNATIVE_THEME)

Session

  1. rliterate2.py
  2. classes
  3. Document
  4. init 
"toc": {
    "width": 230,
    "collapsed": [],
    "hoisted_page": None,
    "dragged_page": None,
},
"workspace": {
    "page_body_width": 300,
    "columns": [
        [
            "cf689824aa3641828343eba2b5fbde9f",
            "ef8200090225487eab4ae35d8910ba8e",
            "97827e5f0096482a9a4eadf0ce07764f"
        ],
        [
            "e6a157bbac8842a2b8c625bfa9255159",
            "813ec304685345a19b1688074000d296",
            "004bc5a29bc94eeb95f4f6a56bd48729",
            "b987445070e84067ba90e71695763f72"
        ]
    ],
},
  1. rliterate2.py
  2. classes
  3. Document 
def open_page(self, page_id):
    self.replace(["workspace", "columns"], [[page_id]])

def get_open_pages(self):
    pages = set()
    for column in self.get(["workspace", "columns"]):
        for page in column:
            pages.add(page)
    return pages

def set_hoisted_page(self, page_id):
    self.replace(["toc", "hoisted_page"], page_id)

def set_dragged_page(self, page_id):
    self.replace(["toc", "dragged_page"], page_id)

def set_toc_width(self, width):
    self.replace(["toc", "width"], width)

def set_page_body_width(self, width):
    self.replace(["workspace", "page_body_width"], width)

def toggle_collapsed(self, page_id):
    def toggle(collapsed):
        if page_id in collapsed:
            return [x for x in collapsed if x != page_id]
        else:
            return collapsed + [page_id]
    self.modify(["toc", "collapsed"], toggle)

Selection

  1. rliterate2.py
  2. classes
  3. Document 
def set_selection(self, selection):
    self.replace(["selection"], selection)

def activate_selection(self, widget_path):
    self._set_selection_active(widget_path, True)

def deactivate_selection(self, widget_path):
    self._set_selection_active(widget_path, False)

def _set_selection_active(self, widget_path, active):
    current_selection = self.get(["selection"])
    if (current_selection.widget_path == widget_path and
        current_selection.active != active):
        self.modify(
            ["selection"],
            lambda x: x._replace(active=active)
        )
  1. rliterate2.py
  2. classes 
class Selection(namedtuple("Selection", ["trail", "value", "widget_path", "active", "stamp"])):

    @staticmethod
    def empty():
        return Selection(trail=[], value=[], widget_path=[], active=False, stamp=genid())

    def add(self, *args):
        return self._replace(trail=self.trail+list(args))

    def create(self, value):
        return Selection(
            trail=[],
            value=value,
            widget_path=self.trail,
            active=True,
            stamp=genid()
        )

    def update_value(self, new_value):
        return self._replace(stamp=genid(), value=new_value)

    @property
    def value_here(self):
        if self.trail == self.widget_path:
            return self.value

GUI framework

Tutorial

Widgets are written in a custom language. Here is an example frame:

frame ExampleFrame %layout_rows {
    Panel {
        background = #color
        %align[EXPAND]
    }
    Panel {
        background = "red"
        %align[EXPAND]
    }
}

This gets compiled into the following Python class with the same name:

class ExampleFrame(Frame):

    def _get_local_props(self):
        return {
        }

    def _create_sizer(self):
        return wx.BoxSizer(wx.VERTICAL)

    def _create_widgets(self):
        pass
        props = {}
        sizer = {"flag": 0, "border": 0, "proportion": 0}
        name = None
        handlers = {}
        props['background'] = self.prop(['color'])
        sizer["flag"] |= wx.EXPAND
        self._create_widget(Panel, props, sizer, handlers, name)
        props = {}
        sizer = {"flag": 0, "border": 0, "proportion": 0}
        name = None
        handlers = {}
        props['background'] = 'red'
        sizer["flag"] |= wx.EXPAND
        self._create_widget(Panel, props, sizer, handlers, name)

All widgets get something called props. It is a Python dictionary with arbitrary values that can be used to construct the GUI. color in the example is a prop.

A GUI application is started by calling start_app and passing a frame class and a props object. A props object is created by calling create_props with a function that creates the props from a list of dependencies. As soon as one of the dependencies change, the function is invoked again to recompute the props and rebuild the GUI.

Compiler

  1. rlgui
  2. rlgui.py
#!/usr/bin/env python2

from collections import defaultdict
import sys

<<rlmeta support library>>

<<grammars>>

<<support functions>>

if __name__ == "__main__":
    parser = GuiParser()
    codegenerator = WxCodeGenerator()
    try:
        sys.stdout.write(
            codegenerator.run("ast", parser.run("widget", sys.stdin.read()))
        )
    except _MatchError as e:
        sys.exit(e.describe())
  1. rlgui
  2. rlgui.py
  3. rlmeta support library 
# Placeholder to generate RLMeta support library

Support library

Widget mixin
  1. rliterate2.py
  2. base classes 
class WidgetMixin(object):

    def __init__(self, parent, handlers, props):
        self._pending_props = None
        self._parent = parent
        self._props = {}
        self._builtin_props = {}
        self._event_handlers = {}
        self._setup_gui()
        self.update_event_handlers(handlers)
        self.update_props(props)

    def update_event_handlers(self, handlers):
        for name, fn in handlers.items():
            self.register_event_handler(name, fn)

    @profile_sub("register event")
    def register_event_handler(self, name, fn):
        self._event_handlers[name] = profile(f"on_{name}")(profile_sub(f"on_{name}")(fn))

    def call_event_handler(self, name, event, propagate=False):
        if self.has_event_handler(name):
            self._event_handlers[name](event)
        elif self._parent is not None and propagate:
            self._parent.call_event_handler(name, event, True)

    def has_event_handler(self, name):
        return name in self._event_handlers

    def _setup_gui(self):
        pass

    def prop_with_default(self, path, default):
        try:
            return self.prop(path)
        except (KeyError, IndexError):
            return default

    def prop(self, path):
        value = self._props
        for part in path:
            value = value[part]
        return value

    def update_props(self, props):
        if self._pending_props is None:
            self._pending_props = props
            if self._update_props(props):
                self._update_gui()
            pending_props = self._pending_props
            self._pending_props = None
            if pending_props is not props:
                self.update_props(pending_props)
        else:
            self._pending_props = props

    def _update_props(self, props):
        self._changed_props = []
        for p in [lambda: props, self._get_local_props]:
            for key, value in p().items():
                if self._prop_differs(key, value):
                    self._props[key] = value
                    self._changed_props.append(key)
        return len(self._changed_props) > 0

    def prop_changed(self, name):
        return (name in self._changed_props)

    def _get_local_props(self):
        return {}

    def _prop_differs(self, key, value):
        if key not in self._props:
            return True
        prop = self._props[key]
        if prop is value:
            return False
        return prop != value

    def _update_gui(self):
        for name in self._changed_props:
            if name in self._builtin_props:
                self._builtin_props[name](self._props[name])

    @profile_sub("register builtin")
    def _register_builtin(self, name, fn):
        self._builtin_props[name] = profile_sub(f"builtin {name}")(fn)
Events
  1. rliterate2.py
  2. classes 
DragEvent = namedtuple("DragEvent", "initial,x,y,dx,dy,initiate_drag_drop")
  1. rliterate2.py
  2. classes 
SliderEvent = namedtuple("SliderEvent", "value")
  1. rliterate2.py
  2. classes 
HoverEvent = namedtuple("HoverEvent", "mouse_inside")
  1. rliterate2.py
  2. classes 
MouseEvent = namedtuple("MouseEvent", "x,y,show_context_menu")
  1. rliterate2.py
  2. classes 
KeyEvent = namedtuple("KeyEvent", "key")
Props
  1. rliterate2.py
  2. functions 
def create_props(fn, *args):
    fn = profile_sub("create props")(fn)
    immutable = Immutable(fn(*args))
    for arg in args:
        if isinstance(arg, Immutable):
            arg.listen(lambda: immutable.replace([], fn(*args)))
    return immutable
Misc
  1. rliterate2.py
  2. functions 
def makeTuple(*args):
    return tuple(args)
Profiling

Limitation: It is not possible to profile recursive functions.

  1. rliterate2.py
  2. globals 
PROFILING_TIMES = defaultdict(list)
PROFILING_ENABLED = os.environ.get("RLITERATE_PROFILE", "") != ""
  1. rliterate2.py
  2. decorators 
def profile_sub(text):
    def wrap(fn):
        def fn_with_timing(*args, **kwargs):
            t1 = time.perf_counter()
            value = fn(*args, **kwargs)
            t2 = time.perf_counter()
            PROFILING_TIMES[text].append(t2-t1)
            return value
        if PROFILING_ENABLED:
            return fn_with_timing
        else:
            return fn
    return wrap
  1. rliterate2.py
  2. decorators 
def profile(text):
    def wrap(fn):
        def fn_with_cprofile(*args, **kwargs):
            pr = cProfile.Profile()
            pr.enable()
            value = fn(*args, **kwargs)
            pr.disable()
            s = io.StringIO()
            ps = pstats.Stats(pr, stream=s).sort_stats("tottime")
            ps.print_stats(10)
            profile_print_summary(text, s.getvalue())
            PROFILING_TIMES.clear()
            return value
        if PROFILING_ENABLED:
            return fn_with_cprofile
        else:
            return fn
    return wrap
  1. rliterate2.py
  2. functions 
def profile_print_summary(text, cprofile_out):
    text_width = 0
    for name, times in PROFILING_TIMES.items():
        text_width = max(text_width, len(f"{name} ({len(times)})"))
    print(f"=== {text} {'='*60}")
    print(f"{textwrap.indent(cprofile_out.strip(), '    ')}")
    print(f"--- {text} {'-'*60}")
    for name, times in sorted(PROFILING_TIMES.items(), key=lambda x: sum(x[1])):
        time = sum(times)*1000
        if time > 10:
            color = "\033[31m"
        elif time > 5:
            color = "\033[33m"
        else:
            color = "\033[0m"
        print("    {}{} = {:.3f}ms{}".format(
            color,
            f"{name} ({len(times)})".ljust(text_width),
            time,
            "\033[0m"
        ))
Immutable
  1. rliterate2.py
  2. base base classes 
class Immutable(object):

    def __init__(self, value):
        self._listeners = []
        self._value = value
        self._transaction_counter = 0
        self._notify_pending = False

    def listen(self, listener):
        self._listeners.append(listener)

    @profile_sub("get")
    def get(self, path=[]):
        value = self._value
        for part in path:
            value = value[part]
        return value

    def replace(self, path, value):
        self.modify(
            path,
            lambda old_value: value,
            only_if_differs=True
        )

    def modify(self, path, fn, only_if_differs=False):
        try:
            self._value = im_modify(
                self._value,
                path,
                self._create_modify_fn(fn, only_if_differs)
            )
            self._notify()
        except ValuesEqualError:
            pass

    def _create_modify_fn(self, fn, only_if_differs):
        if only_if_differs:
            def modify_fn(old_value):
                new_value = fn(old_value)
                if new_value == old_value:
                    raise ValuesEqualError()
                return new_value
            return modify_fn
        else:
            return fn

    @contextlib.contextmanager
    def transaction(self):
        self._transaction_counter += 1
        original_value = self._value
        try:
            yield
        except:
            self._value = original_value
            raise
        finally:
            self._transaction_counter -= 1
            if self._notify_pending:
                self._notify()

    def _notify(self):
        if self._transaction_counter == 0:
            for listener in self._listeners:
                listener()
            self._notify_pending = False
        else:
            self._notify_pending = True
  1. rliterate2.py
  2. classes 
class ValuesEqualError(Exception):
    pass
  1. rliterate2.py
  2. functions 
@profile_sub("im_modify")
def im_modify(obj, path, fn):
    def inner(obj, path):
        if path:
            new_child = inner(obj[path[0]], path[1:])
            if isinstance(obj, list):
                new_obj = list(obj)
            elif isinstance(obj, dict):
                new_obj = dict(obj)
            else:
                raise ValueError("unknown type")
            new_obj[path[0]] = new_child
            return new_obj
        else:
            return fn(obj)
    return inner(obj, path)
Cache
  1. rliterate2.py
  2. decorators 
def cache(limit=1, key_path=[]):
    entries = OrderedDict()
    def wrap(fn):
        def fn_with_cache(*args):
            entry_key = _cache_get_key(key_path, args)
            if entry_key not in entries:
                entries[entry_key] = ([], None)
            entry_args, entry_value = entries[entry_key]
            if _cache_differ(entry_args, args):
                entry_value = fn(*args)
                entries[entry_key] = (list(args), entry_value)
                if len(entries) > limit:
                    entries.popitem(last=False)
            return entry_value
        return fn_with_cache
    return wrap

def _cache_get_key(path, args):
    if path:
        for part in path:
            args = args[part]
        return args
    return None

def _cache_differ(one, two):
    if len(one) != len(two):
        return True
    for index in range(len(one)):
        if one[index] is not two[index] and one[index] != two[index]:
            return True
    return False

Front end

Parser
  1. rlgui
  2. rlgui.py
  3. grammars 
GuiParser {
  widget          =
    container:container WS NAME:name layout:layout
    WS '{' widgetBody:body WS '}' ' '* '\n'? .*:verbatim -> [
      "widget"
      name
      container
      layout
      extract(body "prop")
      extract(body "instance")
      join(verbatim)
    ]
  container       =
    WS (
      | 'frame'
      | 'panel'
      | 'scroll'
      | 'vscroll'
      | 'hscroll'
    ):x WB                                      -> ["container" x]
  layout          =
    WS (
      | '%layout_rows'
      | '%layout_columns'
    ):x WB                                      -> ["layout" x]
  widgetBody      =
    (
      | instance
      | prop
    )*:xs                                       -> partition(xs)
  instance        =
    | WS 'loop' WS '(' expr:x WS loopOption*:ys WS ')'
      WS '{' instanceInner*:zs WS '}'           -> ["instance" "%loop" x ys zs]
    | WS 'if' WS '(' expr:x WS ')'
      WS '{' instanceInner*:ys WS '}'
      WS 'else' WS '{' instanceInner*:zs WS '}' -> ["instance" "%if" x ys zs]
    | WS 'if' WS '(' expr:x WS ')'
      WS '{' instanceInner*:ys WS '}'           -> ["instance" "%if" x ys]
    | instanceInner
  loopOption =
    | WS 'cache_limit' WS '=' WS expr:x         -> ["loopOption" "cache_limit" x]
  instanceInner   =
    | WS '%space' WB WS '[' expr:x WS ']'       -> ["instance" "%space" x]
    | WS expr:x instanceName:ys WS
      '{' instanceBody*:zs WS '}'               -> ["instance" x [~ys ~zs]]
  instanceName =
    | WS '[' WS NAME:x WS ']'                   -> [["instanceName" x]]
    |                                           -> []
  instanceBody    =
    | layoutAttribute
    | loopvarExplode
    | propExplode
    | propAssign
    | handler
  layoutAttribute =
    | WS '%proportion[' number:x WS ']'         -> ["layoutAttribute" "proportion" x]
    | WS '%align[' WS 'EXPAND' WS ']'           -> ["layoutAttribute" "align" "expand"]
    | WS '%align[' WS 'CENTER' WS ']'           -> ["layoutAttribute" "align" "center"]
    | WS '%margin[' expr:x ',' margins:y WS ']' -> ["layoutAttribute" "margin" x ~y]
  margins         = margin:x ('|' margin)*:xs   -> [x ~xs]
  margin          = (
    | 'TOP'
    | 'BOTTOM'
    | 'LEFT'
    | 'RIGHT'
    | 'ALL'
  ):x                                           -> ["margin" x]
  propRef         = propChain:x                 -> ["propRef" x]
  propExplode     = propChain:x                 -> ["propExplode" x]
  propChain       =
    | WS '#' NAME:x ('.' NAME)*:xs              -> ["propChain" x ~xs]
    | WS '#' WB                                 -> ["propChain"]
  propAssign      = WS NAME:x WS '=' expr:y     -> ["propAssign" x y]
  handler         = WS '@' NAME:x WS '=' expr:y -> ["handler" x y]
  expr            = expr1:x ('.' expr1)*:xs     -> ["chain" x ~xs]
  expr1           = (string | call | number | propRef | identifier | loopvarRef | bool | none)
  string          = WS STRING:x                 -> ["string" x]
  bool            =
    | WS 'True' WB                              -> ["bool" bool(" ")]
    | WS 'False' WB                             -> ["bool" bool("")]
  none            = WS 'None' WB                -> ["none"]
  call            =
    (identifier|propRef):x '(' expr*:xs WS ')'  -> ["call" x ~xs]
  number          = WS NUMBER:x                 -> ["number" x]
  identifier      = WS NAME:x                   -> ["identifier" x]
  loopvarRef      = loopvarChain:x              -> ["loopvarRef" x]
  loopvarExplode  = loopvarChain:x              -> ["loopvarExplode" x]
  loopvarChain    =
    | WS '$' NAME:x ('.' NAME)*:xs              -> ["loopvarChain" x ~xs]
    | WS '$' WB                                 -> ["loopvarChain"]
  prop            = WS NAME:x WS '=' expr:y     -> ["prop" x y]
  <<GuiParser>>
}
  1. rlgui
  2. rlgui.py
  3. grammars
  4. GuiParser 
NUMBER    = ('-' | -> ""):x DIGIT:y DIGIT*:ys -> int(join([x y ~ys]))
DIGIT     = '0'-'9'
NAME      = NAMESTART:x NAMECHAR*:xs          -> join([x ~xs])
NAMESTART = 'a'-'z' | 'A'-'Z' | '_'
NAMECHAR  = NAMESTART | '0'-'9'
STRING    = '"' (!'"' .)*:xs '"'              -> join(xs)
WS        = (' ' | '\n')*
WB        = !NAMECHAR
  1. rlgui
  2. rlgui.py
  3. support functions 
def join(items, sep=""):
    return sep.join(items)
  1. rlgui
  2. rlgui.py
  3. support functions 
def partition(values):
    by_type = defaultdict(list)
    for x in values:
        by_type[x[0]].append(x)
    return by_type
  1. rlgui
  2. rlgui.py
  3. support functions 
def extract(by_type, name):
    return by_type[name]
Pygments plugin
  1. rlgui
  2. pygments
  3. rlgui_lexer
  4. __init__.py
from pygments.lexer import RegexLexer
from pygments.token import *

class RLGuiLexer(RegexLexer):

    name = 'RLGui'
    aliases = ['rlgui']
    filenames = ['*.rlgui']

    tokens = {
        'root': [
            (r'"', String, "string"),
            (r'[=]', Name.Builtin),
            (r'\b(vscroll|hscroll|scroll|frame|panel|loop|if|else|True|False|None|cache_limit)\b', Keyword),
            (r'#(\w+([.]\w+)*)?', Name.Class),
            (r'[$]\w*', Name.Builtin),
            (r'@\w+', Name.Exception),
            (r'%\w+', Comment),
            (r'.', Text),
        ],
        'string': [
            (r'[^"\\]+', String),
            (r'"', String, "#pop"),
        ],
    }
  1. rlgui
  2. pygments
  3. setup.py
from setuptools import setup

setup(
    name='rlgui_lexer',
    version='0.1',
    packages=['rlgui_lexer'],
    entry_points={
        'pygments.lexers': ['rlgui_lexer=rlgui_lexer:RLGuiLexer'],
    },
    zip_safe=False,
)
  1. rlgui
  2. pygments
  3. install.sh
#!/bin/sh

set -e

pip install --upgrade --user .

pip3 install --upgrade --user .

Wx backend

Code generator
  1. rlgui
  2. rlgui.py
  3. grammars 
WxCodeGenerator {
  ast = [%:x] -> x
  astItems =
    | ast:x astItem*:xs -> { x xs }
    |                   -> {}
  astItem = ast:x -> { ", " x }
  widget = .:name ast:container ast:sizer [ast*:props] [ast*:inst] .:verbatim -> {
    "class " name "(" container "):\n\n" >
      "def _get_local_props(self):\n" >
        "return {\n" >
          props
        < "}\n\n"
      <
      "def _create_sizer(self):\n" >
        "return " sizer "\n\n"
      <
      "def _create_widgets(self):\n" >
        "pass\n"
        inst
      <
      verbatim
    <
  }
  instance =
    | "%space" ast:x -> {
      "self._create_space(" x ")\n"
    }
    | ast:class [ast*:xs] -> {
      "props = {}\n"
      "sizer = {\"flag\": 0, \"border\": 0, \"proportion\": 0}\n"
      "name = None\n"
      "handlers = {}\n"
      xs
      "self._create_widget(" class ", props, sizer, handlers, name)\n"
    }
    | "%loop" ast:x [ast*:ys] [ast*:zs] -> {
      "def loop_fn(loopvar):\n" >
        "pass\n"
        zs
      <
      "loop_options = {}\n"
      ys
      "with self._loop(**loop_options):\n" >
        "for loopvar in " x ":\n" >
          "loop_fn(loopvar)\n"
        <
      <
    }
    | "%if" ast:x ifTrue:y ifFalse:z -> {
      "if_condition = " x "\n"
      y
      z
    }
  instanceName =
    | py:x -> { "name = " x "\n" }
    |      -> {}
  loopOption = py:x ast:y -> { "loop_options[" x "] = " y "\n" }
  ifTrue = [ast*:xs] -> {
    "def loop_fn(loopvar):\n" >
      "pass\n"
      xs
    <
    "with self._loop():\n" >
      "for loopvar in ([None] if (if_condition) else []):\n" >
        "loop_fn(loopvar)\n"
      <
    <
  }
  ifFalse =
    | [ast*:xs] -> {
        "def loop_fn(loopvar):\n" >
          "pass\n"
          xs
        <
        "with self._loop():\n" >
          "for loopvar in ([None] if (not if_condition) else []):\n" >
            "loop_fn(loopvar)\n"
          <
        <
      }
    | -> {}
  layoutAttribute =
    | "proportion" ast:x      -> { "sizer[\"proportion\"] = " x "\n" }
    | "align" "expand"        -> { "sizer[\"flag\"] |= wx.EXPAND\n" }
    | "align" "center"        -> { "sizer[\"flag\"] |= wx.ALIGN_CENTER\n" }
    | "margin" ast:x ast*:ys  -> { "sizer[\"border\"] = " x "\n" ys }
  margin =
    | "TOP"    -> "sizer[\"flag\"] |= wx.TOP\n"
    | "BOTTOM" -> "sizer[\"flag\"] |= wx.BOTTOM\n"
    | "LEFT"   -> "sizer[\"flag\"] |= wx.LEFT\n"
    | "RIGHT"  -> "sizer[\"flag\"] |= wx.RIGHT\n"
    | "ALL"    -> "sizer[\"flag\"] |= wx.ALL\n"
  prop = py:name ast:default -> {
    name ": " default ",\n"
  }
  chain = ast:x chainAst*:xs -> { x xs }
  chainAst = ast:x -> { "." x }
  propRef = ast:x -> { "self.prop(" x ")" }
  propAssign = py:name ast:value -> { "props[" name "] = " value "\n" }
  propExplode = ast:x -> { "props.update(self.prop(" x "))\n" }
  propChain = .*:xs -> repr(xs)
  loopvarRef = ast:x -> { "loopvar" x }
  loopvarChain = loopvar*:xs -> { xs }
  loopvar = py:x -> { "[" x "]" }
  loopvarExplode = ast:x -> { "props.update(loopvar" x ")\n" }
  call = ast:x astItems:y -> { x "(" y ")" }
  handler = py:name ast:y -> { "handlers[" name "] = lambda event: " y "\n" }
  layout =
    | "%layout_rows"    -> "wx.BoxSizer(wx.VERTICAL)"
    | "%layout_columns" -> "wx.BoxSizer(wx.HORIZONTAL)"
  container =
    | "frame"   -> "Frame"
    | "panel"   -> "Panel"
    | "scroll"  -> "Scroll"
    | "vscroll" -> "VScroll"
    | "hscroll" -> "HScroll"
  string = py
  bool = py
  none = -> "None"
  number = py
  identifier = .
  py = .:x -> repr(x)
}
Widgets
  1. rliterate2.py
  2. base classes 
class WxWidgetMixin(WidgetMixin):

    def _setup_gui(self):
        WidgetMixin._setup_gui(self)
        self._default_cursor = self.GetCursor()
        self._setup_wx_events()
        self._register_builtin("background", self._set_background)
        self._register_builtin("min_size", self._set_min_size)
        self._register_builtin("drop_target", self._set_drop_target)
        self._register_builtin("focus", self._set_focus)
        self._register_builtin("cursor", lambda value:
            self.SetCursor({
                "size_horizontal": wx.Cursor(wx.CURSOR_SIZEWE),
                "hand": wx.Cursor(wx.CURSOR_HAND),
                "beam": wx.Cursor(wx.CURSOR_IBEAM),
                None: self._default_cursor,
            }.get(value, wx.Cursor(wx.CURSOR_QUESTION_ARROW)))
        )
        self._event_map = {
            "left_down": [
                (wx.EVT_LEFT_DOWN, self._on_wx_left_down),
            ],
            "drag": [
                (wx.EVT_LEFT_DOWN, self._on_wx_left_down),
                (wx.EVT_LEFT_UP, self._on_wx_left_up),
                (wx.EVT_MOTION, self._on_wx_motion),
            ],
            "click": [
                (wx.EVT_LEFT_UP, self._on_wx_left_up),
            ],
            "right_click": [
                (wx.EVT_RIGHT_UP, self._on_wx_right_up),
            ],
            "hover": [
                (wx.EVT_ENTER_WINDOW, self._on_wx_enter_window),
                (wx.EVT_LEAVE_WINDOW, self._on_wx_leave_window),
            ],
            "key": [
                (wx.EVT_CHAR, self._on_wx_char),
            ],
            "focus": [
                (wx.EVT_SET_FOCUS, self._on_wx_set_focus),
            ],
            "unfocus": [
                (wx.EVT_KILL_FOCUS, self._on_wx_kill_focus),
            ],
        }
        <<WxWidgetMixin/_setup_gui>>

    def _set_background(self, color):
        self.SetBackgroundColour(color)
        self._request_refresh()

    def _set_min_size(self, size):
        self.SetMinSize(size)
        self._request_refresh(layout=True)

    def _setup_wx_events(self):
        self._wx_event_handlers = set()
        self._wx_down_pos = None

    def _update_gui(self):
        WidgetMixin._update_gui(self)
        for name in ["drag", "hover", "click", "right_click"]:
            if self._parent is not None and self._parent.has_event_handler(name):
                self._register_wx_events(name)

    def update_event_handlers(self, handlers):
        WidgetMixin.update_event_handlers(self, handlers)
        for name in handlers:
            self._register_wx_events(name)

    def _register_wx_events(self, name):
        if name not in self._wx_event_handlers:
            self._wx_event_handlers.add(name)
            for event_id, handler in self._event_map.get(name, []):
                self.Bind(event_id, handler)

    def _on_wx_left_down(self, wx_event):
        self._wx_down_pos = self.ClientToScreen(wx_event.Position)
        self._call_mouse_event_handler(wx_event, "left_down")
        self.call_event_handler("drag", DragEvent(
            True,
            self._wx_down_pos.x,
            self._wx_down_pos.y,
            0,
            0,
            self.initiate_drag_drop
        ), propagate=True)

    def _on_wx_left_up(self, wx_event):
        if self.HitTest(wx_event.Position) == wx.HT_WINDOW_INSIDE:
            self._call_mouse_event_handler(wx_event, "click")
        self._wx_down_pos = None

    def _on_wx_right_up(self, wx_event):
        if self.HitTest(wx_event.Position) == wx.HT_WINDOW_INSIDE:
            self._call_mouse_event_handler(wx_event, "right_click")

    def _call_mouse_event_handler(self, wx_event, name):
        screen_pos = self.ClientToScreen(wx_event.Position)
        self.call_event_handler(
            name,
            MouseEvent(screen_pos.x, screen_pos.y, self._show_context_menu),
            propagate=True
        )

    def _on_wx_motion(self, wx_event):
        if self._wx_down_pos is not None:
            new_pos = self.ClientToScreen(wx_event.Position)
            dx = new_pos.x-self._wx_down_pos.x
            dy = new_pos.y-self._wx_down_pos.y
            self.call_event_handler("drag", DragEvent(
                False,
                new_pos.x,
                new_pos.y,
                dx,
                dy,
                self.initiate_drag_drop
            ), propagate=True)

    def _on_wx_enter_window(self, wx_event):
        self.call_event_handler("hover", HoverEvent(True), propagate=True)

    def _on_wx_leave_window(self, wx_event):
        self._hover_leave()

    def initiate_drag_drop(self, kind, data):
        self._wx_down_pos = None
        self._hover_leave()
        obj = wx.CustomDataObject(f"rliterate/{kind}")
        obj.SetData(json.dumps(data).encode("utf-8"))
        drag_source = wx.DropSource(self)
        drag_source.SetData(obj)
        result = drag_source.DoDragDrop(wx.Drag_DefaultMove)

    def _hover_leave(self):
        self.call_event_handler("hover", HoverEvent(False), propagate=True)

    def _set_drop_target(self, kind):
        self.SetDropTarget(RLiterateDropTarget(self, kind))

    def _set_focus(self, focus):
        if focus:
            self.SetFocus()

    def _on_wx_char(self, wx_event):
        self.call_event_handler(
            "key",
            KeyEvent(chr(wx_event.GetUnicodeKey())),
            propagate=False
        )

    def _on_wx_set_focus(self, wx_event):
        self.call_event_handler(
            "focus",
            None,
            propagate=False
        )
        wx_event.Skip()

    def _on_wx_kill_focus(self, wx_event):
        self.call_event_handler(
            "unfocus",
            None,
            propagate=False
        )
        wx_event.Skip()

    def on_drag_drop_over(self, x, y):
        pass

    def on_drag_drop_leave(self):
        pass

    def on_drag_drop_data(self, x, y, data):
        pass

    def get_y(self):
        return self.Position.y

    def get_height(self):
        return self.Size.height

    def get_width(self):
        return self.Size.width

    def _show_context_menu(self, items):
        def create_handler(fn):
            return lambda event: fn()
        menu = wx.Menu()
        for item in items:
            if item is None:
                menu.AppendSeparator()
            else:
                text, enabled, fn = item
                menu_item = menu.Append(wx.NewId(), text)
                menu_item.Enable(enabled)
                menu.Bind(
                    wx.EVT_MENU,
                    create_handler(fn),
                    menu_item
                )
        self.PopupMenu(menu)
        menu.Destroy()
        self._hover_leave()

    <<WxWidgetMixin>>
  1. rliterate2.py
  2. base classes 
class RLiterateDropTarget(wx.DropTarget):

    def __init__(self, widget, kind):
        wx.DropTarget.__init__(self)
        self.widget = widget
        self.data = wx.CustomDataObject(f"rliterate/{kind}")
        self.DataObject = self.data

    def OnDragOver(self, x, y, defResult):
        if self.widget.on_drag_drop_over(x, y):
            if defResult == wx.DragMove:
                return wx.DragMove
        return wx.DragNone

    def OnData(self, x, y, defResult):
        if (defResult == wx.DragMove and
            self.GetData()):
            self.widget.on_drag_drop_data(
                x,
                y,
                json.loads(self.data.GetData().tobytes().decode("utf-8"))
            )
        return defResult

    def OnLeave(self):
        self.widget.on_drag_drop_leave()
Toolbar button
  1. rliterate2.py
  2. base classes 
class ToolbarButton(wx.BitmapButton, WxWidgetMixin):

    def __init__(self, wx_parent, *args):
        wx.BitmapButton.__init__(self, wx_parent, style=wx.NO_BORDER)
        WxWidgetMixin.__init__(self, *args)

    def _setup_gui(self):
        WxWidgetMixin._setup_gui(self)
        self._register_builtin("icon", self._set_icon)
        self._event_map["button"] = [(wx.EVT_BUTTON, self._on_wx_button)]

    def _set_icon(self, value):
        self.SetBitmap(wx.ArtProvider.GetBitmap(
            {
                "add": wx.ART_ADD_BOOKMARK,
                "back": wx.ART_GO_BACK,
                "forward": wx.ART_GO_FORWARD,
                "undo": wx.ART_UNDO,
                "redo": wx.ART_REDO,
                "quit": wx.ART_QUIT,
                "save": wx.ART_FILE_SAVE,
                "settings": wx.ART_HELP_SETTINGS,
                "bold": "gtk-bold",
            }.get(value, wx.ART_QUESTION),
            wx.ART_BUTTON,
            (24, 24)
        ))
        self._request_refresh(layout=True)

    def _on_wx_button(self, wx_event):
        self.call_event_handler("button", None)
Button
  1. rliterate2.py
  2. base classes 
class Button(wx.Button, WxWidgetMixin):

    def __init__(self, wx_parent, *args):
        wx.Button.__init__(self, wx_parent)
        WxWidgetMixin.__init__(self, *args)

    def _setup_gui(self):
        WxWidgetMixin._setup_gui(self)
        self._register_builtin("label", self._set_label)
        self._event_map["button"] = [(wx.EVT_BUTTON, self._on_wx_button)]

    def _set_label(self, label):
        self.SetLabel(label)
        self._request_refresh(layout=True)

    def _on_wx_button(self, wx_event):
        self.call_event_handler("button", None)
Slider
  1. rliterate2.py
  2. base classes 
class Slider(wx.Slider, WxWidgetMixin):

    def __init__(self, wx_parent, *args):
        wx.Slider.__init__(self, wx_parent)
        WxWidgetMixin.__init__(self, *args)

    def _setup_gui(self):
        WxWidgetMixin._setup_gui(self)
        self._register_builtin("min", self.SetMin)
        self._register_builtin("max", self.SetMax)

    def register_event_handler(self, name, fn):
        WxWidgetMixin.register_event_handler(self, name, fn)
        if name == "slider":
            self._bind_wx_event(wx.EVT_SLIDER, self._on_wx_slider)

    def _on_wx_slider(self, wx_event):
        self._call_event_handler("slider", SliderEvent(self.Value))
Image
  1. rliterate2.py
  2. base classes 
class Image(wx.StaticBitmap, WxWidgetMixin):

    def __init__(self, wx_parent, *args):
        wx.StaticBitmap.__init__(self, wx_parent)
        WxWidgetMixin.__init__(self, *args)

    def _update_gui(self):
        WxWidgetMixin._update_gui(self)
        reset = False
        if self.prop_changed("base64_image"):
            self._image = base64_to_image(self.prop(["base64_image"]))
            self._scaled_image = self._image
            reset = True
        if reset or self.prop_changed("width"):
            self._scaled_image = fit_image(self._image, self.prop(["width"]))
            reset = True
        if reset:
            self.SetBitmap(self._scaled_image.ConvertToBitmap())
            self._request_refresh(layout=True)
  1. rliterate2.py
  2. functions 
def base64_to_image(data):
    try:
        return wx.Image(
            io.BytesIO(base64.b64decode(data)),
            wx.BITMAP_TYPE_ANY
        )
    except:
        return wx.ArtProvider.GetBitmap(
            wx.ART_MISSING_IMAGE,
            wx.ART_BUTTON,
            (64, 64)
        ).ConvertToImage()
  1. rliterate2.py
  2. functions 
def fit_image(image, width):
    if image.Width <= width:
        return image
    factor = float(width) / image.Width
    return image.Scale(
        int(image.Width*factor),
        int(image.Height*factor),
        wx.IMAGE_QUALITY_HIGH
    )
Expand/Collapse
  1. rliterate2.py
  2. base classes 
class ExpandCollapse(wx.Panel, WxWidgetMixin):

    def __init__(self, wx_parent, *args):
        wx.Panel.__init__(self, wx_parent)
        WxWidgetMixin.__init__(self, *args)
        self.Bind(wx.EVT_PAINT, self._on_paint)

    def _get_local_props(self):
        return {
            "min_size": (self.prop(["size"]), -1),
        }

    def _update_gui(self):
        WxWidgetMixin._update_gui(self)
        self._request_refresh()

    def _on_paint(self, event):
        dc = wx.GCDC(wx.PaintDC(self))
        render = wx.RendererNative.Get()
        (w, h) = self.Size
        render.DrawTreeItemButton(
            self,
            dc,
            (
                0,
                (h-self.prop(["size"]))/2,
                self.prop(["size"])-1,
                self.prop(["size"])-1
            ),
            flags=0 if self.prop(["collapsed"]) else wx.CONTROL_EXPANDED
        )
Text
  1. rliterate2.py
  2. classes 
class Text(wx.Panel, WxWidgetMixin):

    def __init__(self, wx_parent, *args):
        wx.Panel.__init__(self, wx_parent)
        WxWidgetMixin.__init__(self, *args)
        self.Bind(wx.EVT_PAINT, self._on_paint)
        self.Bind(wx.EVT_TIMER, self._on_timer)
        self.Bind(wx.EVT_WINDOW_DESTROY, self._on_window_destroy)

    def _setup_gui(self):
        WxWidgetMixin._setup_gui(self)
        self._timer = wx.Timer(self)
        self._show_cursors = True
        self._cursor_positions = []

    def _update_gui(self):
        WxWidgetMixin._update_gui(self)
        old_min_size = self.GetMinSize()
        did_measure = False
        if (self.prop_changed("base_style") or
            self.prop_changed("characters")):
            self._measure(
                self.prop_with_default(["characters"], [])
            )
            did_measure = True
        did_reflow = False
        if (did_measure or
            self.prop_changed("max_width") or
            self.prop_changed("break_at_word") or
            self.prop_changed("line_height") or
            self.prop_changed("align")):
            self._reflow(
                self.prop_with_default(["max_width"], None),
                self.prop_with_default(["break_at_word"], False),
                self.prop_with_default(["line_height"], 1),
                self.prop_with_default(["align"], "left")
            )
            did_reflow = True
        need_refresh = did_reflow
        if did_reflow or self.prop_changed("cursors"):
            self._show_cursors = True
            self._calculate_cursor_positions(
                self.prop_with_default(["cursors"], [])
            )
            if self.prop(["cursors"]):
                self._timer.Start(400)
            else:
                self._timer.Stop()
            need_refresh = True
        if did_reflow or self.prop_changed("selections"):
            self._calculate_selection_rects(
                self.prop_with_default(["selections"], [])
            )
            need_refresh = True
        if need_refresh:
            self._request_refresh(
                layout=self.GetMinSize() != old_min_size,
                immediate=(
                    self.prop_with_default(["immediate"], False) and
                    self.prop_changed("characters")
                )
            )

    <<Text>>
Style
  1. rliterate2.py
  2. classes 
TextStyle = namedtuple("TextStyle", "size,family,color,bold,italic,underlined")
  1. rliterate2.py
  2. classes
  3. Text 
def _get_style(self, character):
    style = {
        "size": 10,
        "family": None,
        "color": "#000000",
        "bold": False,
        "italic": False,
        "underlined": False,
    }
    base_style = self.prop_with_default(["base_style"], {})
    for field in TextStyle._fields:
        if field in base_style:
            style[field] = base_style[field]
    for field in TextStyle._fields:
        if field in character:
            style[field] = character[field]
    return TextStyle(**style)

def _apply_style(self, style, dc):
    dc.SetTextForeground(style.color)
    font_info = wx.FontInfo(style.size)
    if style.family == "Monospace":
        font_info.Family(wx.FONTFAMILY_TELETYPE)
    if style.bold:
        font_info = font_info.Bold()
    if style.italic:
        font_info = font_info.Italic()
    if style.underlined:
        font_info = font_info.Underlined()
    dc.SetFont(wx.Font(font_info))
Measure

The measured characters augments the characters with a size.

  1. rliterate2.py
  2. classes
  3. Text 
size_map_cache = {}

@profile_sub("text measure")
def _measure(self, characters):
    dc = wx.MemoryDC()
    dc.SelectObject(wx.Bitmap(1, 1))
    self._measured_characters = []
    for character in characters:
        style = self._get_style(character)
        entry = (style, character["text"])
        if entry not in self.size_map_cache:
            self._apply_style(style, dc)
            if character["text"] in ["\n", "\r"]:
                _, line_height = dc.GetTextExtent("I")
                self.size_map_cache[entry] = wx.Size(0, line_height)
            else:
                self.size_map_cache[entry] = dc.GetTextExtent(character["text"])
        self._measured_characters.append((
            character,
            self.size_map_cache[entry]
        ))
Reflow
  1. rliterate2.py
  2. classes
  3. Text 
pair_size_map_cache = {}

@profile_sub("text reflow")
def _reflow(self, max_width, break_at_word, line_height, align):
    self._draw_fragments_by_style = defaultdict(lambda: ([], []))
    self._characters_bounding_rect = []
    self._characters_by_line = []

    # Setup DC
    dc = wx.MemoryDC()
    dc.SelectObject(wx.Bitmap(1, 1))

    # Split into lines
    lines = []
    index = 0
    y = 0
    max_w = 0
    while index < len(self._measured_characters):
        line = self._extract_line(index, max_width, break_at_word)
        w, h = self._layout_line(dc, line, line_height, y, max_width, align)
        max_w = max(max_w, w)
        y += h
        index += len(line)
    self.SetMinSize((max_w if max_width is None else max_width, y))

def _extract_line(self, index, max_width, break_at_word):
    x = 0
    line = []
    while index < len(self._measured_characters):
        character, size = self._measured_characters[index]
        if max_width is not None and x + size.Width > max_width and line:
            if break_at_word:
                index = self._find_word_break_point(line, character["text"])
                if index is not None:
                    return line[:index+1]
            return line
        line.append((character, size))
        if character["text"] == "\n":
            return line
        x += size.Width
        index += 1
    return line

def _find_word_break_point(self, line, next_character):
    index = len(line) - 1
    while index >= 0:
        current_character = line[index][0]["text"]
        if current_character == " " and next_character != " ":
            return index
        next_character = current_character
        index -= 1

def _layout_line(self, dc, line, line_height, y, max_width, align):
    # Calculate total height
    max_h = 0
    for character, size in line:
        max_h = max(max_h, size.Height)
    height = int(round(max_h * line_height))
    height_offset = int(round((height - max_h) / 2))

    # Bla bla
    characters_by_style = []
    characters = []
    style = None
    for character, size in line:
        this_style = self._get_style(character)
        if not characters or this_style == style:
            characters.append(character)
        else:
            characters_by_style.append((style, characters))
            characters = [character]
        style = this_style
    if characters:
        characters_by_style.append((style, characters))

    # Hmm
    total_width = 0
    characters_by_style_wiht_text_widths = []
    for style, characters in characters_by_style:
        text = "".join(
            character["text"]
            for character
            in characters
        )
        widths = [0, self.size_map_cache[(style, text[0])].Width]
        index = 0
        while len(widths) <= len(text):
            first_width = self.size_map_cache[(style, text[index])].Width
            pair = text[index:index+2]
            key = (style, pair)
            if key in self.pair_size_map_cache:
                pair_width = self.pair_size_map_cache[key]
            else:
                self._apply_style(style, dc)
                pair_width = dc.GetTextExtent(pair).Width
                self.pair_size_map_cache[key] = pair_width
            widths.append(widths[-1]+pair_width-first_width)
            index += 1
        characters_by_style_wiht_text_widths.append((
            style,
            characters,
            text,
            widths,
        ))
        total_width += widths[-1]

    characters_in_line = []
    if max_width is not None and align == "center":
        x = int((max_width - total_width) / 2)
    else:
        x = 0
    for style, characters, text, widths in characters_by_style_wiht_text_widths:
        texts, positions = self._draw_fragments_by_style[style]
        texts.append(text)
        positions.append((x, y+height_offset))

        for index, character in enumerate(characters):
            characters_in_line.append((
                character,
                wx.Rect(
                    x+widths[index],
                    y,
                    widths[index+1]-widths[index],
                    height,
                ),
            ))
        x += widths[-1]

    self._characters_bounding_rect.extend(characters_in_line)
    self._characters_by_line.append((y, height, characters_in_line))

    return (x, height)
Cursors
  1. rliterate2.py
  2. classes
  3. Text 
def _calculate_cursor_positions(self, cursors):
    self._cursor_positions = []
    for cursor in cursors:
        self._cursor_positions.append(self._calculate_cursor_position(cursor))

def _calculate_cursor_position(self, cursor):
    cursor, color = cursor
    if color is None:
        color = self.prop_with_default(["base_style", "cursor_color"], "black")
    if cursor >= len(self._characters_bounding_rect):
        rect = self._characters_bounding_rect[-1][1]
        return (rect.Right, rect.Top, rect.Height, color)
    elif cursor < 0:
        cursor = 0
    rect = self._characters_bounding_rect[cursor][1]
    return (rect.X, rect.Y, rect.Height, color)
Selections
  1. rliterate2.py
  2. classes
  3. Text 
def _calculate_selection_rects(self, selections):
    self._selection_rects = []
    self._selection_pens = []
    self._selection_brushes = []
    for start, end, color in selections:
        self._selection_rects.extend([
            rect
            for character, rect
            in self._characters_bounding_rect[start:end]
        ])
        if color is None:
            color = self.prop_with_default(["base_style", "selection_color"], "black")
        color = wx.Colour(color)
        color = wx.Colour(color.Red(), color.Green(), color.Blue(), 100)
        while len(self._selection_pens) < len(self._selection_rects):
            self._selection_pens.append(wx.Pen(color, width=0))
        while len(self._selection_brushes) < len(self._selection_rects):
            self._selection_brushes.append(wx.Brush(color))
Paint
  1. rliterate2.py
  2. classes
  3. Text 
def _on_paint(self, wx_event):
    dc = wx.PaintDC(self)
    for style, items in self._draw_fragments_by_style.items():
        self._apply_style(style, dc)
        dc.DrawTextList(*items)
    if self._show_cursors:
        for x, y, height, color in self._cursor_positions:
            dc.SetPen(wx.Pen(color, width=2))
            dc.DrawLines([
                (x, y),
                (x, y+height),
            ])
    dc.DrawRectangleList(
        self._selection_rects,
        self._selection_pens,
        self._selection_brushes
    )
    if WX_DEBUG_TEXT:
        dc.DrawRectangleList(
            [x[1] for x in self._characters_bounding_rect],
            [wx.Pen("blue", 1) for x in self._characters_bounding_rect],
            [wx.Brush() for x in self._characters_bounding_rect],
        )
  1. rliterate2.py
  2. globals 
WX_DEBUG_TEXT = os.environ.get("WX_DEBUG_TEXT", "") != ""
Timer
  1. rliterate2.py
  2. classes
  3. Text 
def _on_timer(self, wx_event):
    self._show_cursors = not self._show_cursors
    self._request_refresh()
  1. rliterate2.py
  2. classes
  3. Text 
def _on_window_destroy(self, event):
    self._timer.Stop()
Queries
  1. rliterate2.py
  2. classes
  3. Text 
def get_closest_character_with_side(self, x, y):
    if not self._characters_bounding_rect:
        return (None, False)
    x, y = self.ScreenToClient(x, y)
    return self._get_closest_character_with_side_in_line(
        self._get_closest_line(y),
        x
    )

def _get_closest_line(self, y):
    for (line_y, line_h, line_characters) in self._characters_by_line:
        if y < line_y or (y >= line_y and y <= line_y + line_h):
            return line_characters
    return line_characters

def _get_closest_character_with_side_in_line(self, line, x):
    for character, rect in line:
        if x < rect.Left or (x >= rect.Left and x <= rect.Right):
            return (character, x > (rect.Left+rect.Width/2))
    return (character, True)
  1. rliterate2.py
  2. classes
  3. Text 
def char_iterator(self, index, offset):
    while index >= 0 and index < len(self._characters_bounding_rect):
        yield self._characters_bounding_rect[index][0]
        index += offset
Containers
  1. rliterate2.py
  2. base classes 
class WxContainerWidgetMixin(WxWidgetMixin):

    def _setup_gui(self):
        WxWidgetMixin._setup_gui(self)
        self._setup_layout()
        self._children = []
        self._inside_loop = False
        self._sizer_index = 0
        <<WxContainerWidgetMixin/_setup_gui>>

    def _setup_layout(self):
        self.Sizer = self._sizer = self._create_sizer()
        self._wx_parent = self

    def _update_children(self):
        self._sizer_changed = False
        old_sizer_index = self._sizer_index
        self._sizer_index = 0
        self._child_index = 0
        self._names = defaultdict(list)
        self._create_widgets()
        return old_sizer_index != self._sizer_index or self._sizer_changed

    def _create_sizer(self):
        return wx.BoxSizer(wx.HORIZONTAL)

    def _create_widgets(self):
        pass

    @contextlib.contextmanager
    def _loop(self, cache_limit=-1):
        if self._child_index >= len(self._children):
            self._children.append([])
        old_children = self._children
        next_index = self._child_index + 1
        self._children = self._children[self._child_index]
        self._child_index = 0
        self._inside_loop = True
        try:
            yield
        finally:
            self._clear_leftovers(cache_limit=cache_limit)
            self._children = old_children
            self._child_index = next_index
            self._inside_loop = False

    def _clear_leftovers(self, cache_limit):
        child_index = self._child_index
        sizer_index = self._sizer_index
        num_cached = 0
        while child_index < len(self._children):
            widget, sizer_item = self._children[child_index]
            if (widget is not None and
                widget.prop_with_default(["__cache"], False) and
                (cache_limit < 0 or num_cached < cache_limit)):
                sizer_item.Show(False)
                child_index += 1
                sizer_index += 1
                num_cached += 1
            else:
                if widget is None:
                    self._sizer.Remove(sizer_index)
                else:
                    widget.Destroy()
                self._children.pop(child_index)

    def _create_widget(self, widget_cls, props, sizer, handlers, name):
        if not self._inside_loop:
            def re_use_condition(widget):
                return True
        elif "__reuse" in props:
            def re_use_condition(widget):
                return (
                    type(widget) is widget_cls and
                    widget.prop(["__reuse"]) == props["__reuse"]
                )
        else:
            def re_use_condition(widget):
                return type(widget) is widget_cls
        re_use_offset = self._reuse(re_use_condition)
        if re_use_offset == 0:
            widget, sizer_item = self._children[self._child_index]
            widget.update_event_handlers(handlers)
            widget.update_props(props)
            if sizer_item.Border != sizer["border"]:
                sizer_item.SetBorder(sizer["border"])
                self._sizer_changed = True
            if sizer_item.Proportion != sizer["proportion"]:
                sizer_item.SetProportion(sizer["proportion"])
                self._sizer_changed = True
        else:
            if re_use_offset is None:
                widget = widget_cls(self._wx_parent, self, handlers, props)
            else:
                widget = self._children.pop(self._child_index+re_use_offset)[0]
                widget.update_event_handlers(handlers)
                widget.update_props(props)
                self._sizer.Detach(self._sizer_index+re_use_offset)
            sizer_item = self._insert_sizer(self._sizer_index, widget, **sizer)
            self._children.insert(self._child_index, (widget, sizer_item))
            self._sizer_changed = True
        sizer_item.Show(True)
        if name is not None:
            self._names[name].append(widget)
        self._sizer_index += 1
        self._child_index += 1

    def _create_space(self, thickness):
        if (self._child_index < len(self._children) and
            self._children[self._child_index][0] is None):
            new_min_size = self._get_space_size(thickness)
            if self._children[self._child_index][1].MinSize != new_min_size:
                self._children[self._child_index][1].SetMinSize(
                    new_min_size
                )
                self._sizer_changed = True
        else:
            self._children.insert(self._child_index, (None, self._insert_sizer(
                self._sizer_index,
                self._get_space_size(thickness)
            )))
            self._sizer_changed = True
        self._sizer_index += 1
        self._child_index += 1

    def _reuse(self, condition):
        index = 0
        while (self._child_index+index) < len(self._children):
            widget = self._children[self._child_index+index][0]
            if widget is not None and condition(widget):
                return index
            else:
                index += 1
        return None

    @profile_sub("insert sizer")
    def _insert_sizer(self, *args, **kwargs):
        return self._sizer.Insert(*args, **kwargs)

    def _get_space_size(self, size):
        if self._sizer.Orientation == wx.HORIZONTAL:
            return (size, 1)
        else:
            return (1, size)

    def get_widget(self, name, index=0):
        return self._names[name][index]

    <<WxContainerWidgetMixin>>
Frame
  1. rliterate2.py
  2. base classes 
class Frame(wx.Frame, WxContainerWidgetMixin):

    def __init__(self, wx_parent, *args):
        wx.Frame.__init__(self, wx_parent)
        WxContainerWidgetMixin.__init__(self, *args)

    def _setup_gui(self):
        WxContainerWidgetMixin._setup_gui(self)
        self._register_builtin("title", self.SetTitle)

    def _setup_layout(self):
        self._wx_parent = wx.Panel(self)
        self._wx_parent.Sizer = self._sizer = self._create_sizer()
        self.Sizer = wx.BoxSizer(wx.HORIZONTAL)
        self.Sizer.Add(self._wx_parent, flag=wx.EXPAND, proportion=1)
Panel
  1. rliterate2.py
  2. base classes 
class Panel(wx.Panel, WxContainerWidgetMixin):

    def __init__(self, wx_parent, *args):
        wx.Panel.__init__(self, wx_parent)
        WxContainerWidgetMixin.__init__(self, *args)
        self.Bind(wx.EVT_PAINT, self._on_paint)

    def _on_paint(self, wx_event):
        box = self.prop_with_default(["selection_box"], {})
        if box and box["width"] > 0:
            dc = wx.PaintDC(self)
            dc.SetPen(wx.Pen(
                box["color"],
                width=box["width"],
                style=wx.PENSTYLE_SOLID
            ))
            dc.SetBrush(wx.Brush(
                wx.Colour(),
                wx.BRUSHSTYLE_TRANSPARENT
            ))
            dc.DrawRoundedRectangle((0, 0), self.GetSize(), int(box["width"]*2))
Scroll
  1. rliterate2.py
  2. base classes 
class CompactScrolledWindow(wx.ScrolledWindow):

    MIN_WIDTH = 200
    MIN_HEIGHT = 200

    def __init__(self, parent, style=0, size=wx.DefaultSize, step=100):
        w, h = size
        size = (max(w, self.MIN_WIDTH), max(h, self.MIN_HEIGHT))
        wx.ScrolledWindow.__init__(self, parent, style=style, size=size)
        self.Size = size
        self._style = style
        if style == wx.HSCROLL:
            self.SetScrollRate(1, 0)
        elif style == wx.VSCROLL:
            self.SetScrollRate(0, 1)
        else:
            self.SetScrollRate(1, 1)
        self.step = step
        self.Bind(wx.EVT_MOUSEWHEEL, self._on_mousewheel)

    def _on_mousewheel(self, event):
        if self._style == wx.HSCROLL and not wx.GetKeyState(wx.WXK_SHIFT):
            self._forward_scroll_event(event)
        if self._style == wx.VSCROLL and wx.GetKeyState(wx.WXK_SHIFT):
            self._forward_scroll_event(event)
        else:
            x, y = self.GetViewStart()
            delta = event.GetWheelRotation() / event.GetWheelDelta()
            if wx.GetKeyState(wx.WXK_SHIFT):
                pos = self._calc_scroll_pos_hscroll(x, y, delta)
            else:
                pos = self._calc_scroll_pos_vscroll(x, y, delta)
            self.Scroll(*pos)

    def _forward_scroll_event(self, event):
        parent = self.Parent
        while parent:
            if isinstance(parent, CompactScrolledWindow):
                parent._on_mousewheel(event)
                return
            parent = parent.Parent

    def _calc_scroll_pos_hscroll(self, x, y, delta):
        return (x-delta*self.step, y)

    def _calc_scroll_pos_vscroll(self, x, y, delta):
        return (x, y-delta*self.step)

    def Layout(self):
        wx.ScrolledWindow.Layout(self)
        self.FitInside()
  1. rliterate2.py
  2. base classes 
class VScroll(CompactScrolledWindow, WxContainerWidgetMixin):

    def __init__(self, wx_parent, *args):
        CompactScrolledWindow.__init__(self, wx_parent, wx.VSCROLL)
        WxContainerWidgetMixin.__init__(self, *args)
  1. rliterate2.py
  2. base classes 
class HScroll(CompactScrolledWindow, WxContainerWidgetMixin):

    def __init__(self, wx_parent, *args):
        CompactScrolledWindow.__init__(self, wx_parent, wx.HSCROLL)
        WxContainerWidgetMixin.__init__(self, *args)
  1. rliterate2.py
  2. base classes 
class Scroll(CompactScrolledWindow, WxContainerWidgetMixin):

    def __init__(self, wx_parent, *args):
        CompactScrolledWindow.__init__(self, wx_parent)
        WxContainerWidgetMixin.__init__(self, *args)
Refreshing widgets

When a widget changes, the screen must be refreshed. But it is not always enough to refresh the portion of the screen that the widget covers. If the widget changes size, other widgets might have to be drawn at different locations. This section deals with correctly and efficiently refreshing the screen when a widget changes.

When a widget changes, it should call _request_refresh. It is a thin wrapper that creates a RefreshRequests object and passes it to handle_refresh_requests. Optional keyword arguments are used to inform what has changed. layout should be true if the size of the widget has changed so that parent widgets need to be refreshed. layout_me should be true if the number of visible child widgets have changed so that this widget need to be refreshed. immediate should be true if this refresh should be done before other refresh requests.

  1. rliterate2.py
  2. base classes
  3. WxWidgetMixin 
def _request_refresh(self, layout=False, layout_me=False, immediate=False):
    self.handle_refresh_requests(RefreshRequests(
        {
            "widget": self._find_refresh_widget(
                layout,
                layout_me
            ),
            "layout": layout or layout_me,
            "immediate": immediate,
        },
    ))
  1. rliterate2.py
  2. base classes
  3. WxWidgetMixin 
def _find_refresh_widget(self, layout, layout_me):
    if layout_me:
        return self._find_layout_root(self)
    elif layout:
        return self._find_layout_root(self.Parent)
    else:
        return self
  1. rliterate2.py
  2. base classes
  3. WxWidgetMixin 
def _find_layout_root(self, widget):
    while widget.Parent is not None:
        if isinstance(widget, wx.ScrolledWindow) or widget.IsTopLevel():
            break
        widget = widget.Parent
    return widget

Refresh requests are always handled at the top level, so if a parent exists, refresh requests are passed to it.

  1. rliterate2.py
  2. base classes
  3. WxWidgetMixin 
def handle_refresh_requests(self, refresh_requests):
    if self._parent is None:
        self._handle_refresh_requests(refresh_requests)
    else:
        self._parent.handle_refresh_requests(refresh_requests)

Handling refresh requests means first handling all immediate requests. If any, the handling of others are delayed.

  1. rliterate2.py
  2. base classes
  3. WxWidgetMixin 
def _handle_refresh_requests(self, refresh_requests):
    if any(refresh_requests.process(self._handle_immediate_refresh_request)):
        if self._later_timer:
            self._later_timer.Start(300)
        else:
            self._later_timer = wx.CallLater(300, self._handle_delayed_refresh_requests)
    else:
        self._handle_delayed_refresh_requests()
  1. rliterate2.py
  2. base classes
  3. WxWidgetMixin 
def _handle_immediate_refresh_request(self, request):
    if request["immediate"]:
        self._handle_refresh_request(request)
        return True
    else:
        self._delayed_requests.add(request)
        return False
  1. rliterate2.py
  2. base classes
  3. WxWidgetMixin
  4. _setup_gui 
self._delayed_requests = RefreshRequests()
self._later_timer = None
  1. rliterate2.py
  2. base classes
  3. WxWidgetMixin 
def _handle_delayed_refresh_requests(self):
    self._delayed_requests.process(self._handle_refresh_request)
    self._later_timer = None

Handling a refresh requests means that Layout and Refresh is called on the given widget.

  1. rliterate2.py
  2. base classes
  3. WxWidgetMixin 
def _handle_refresh_request(self, request):
    widget = request["widget"]
    if WX_DEBUG_REFRESH:
        print("Refresh layout={!r:<5} widget={}".format(
            request["layout"],
            widget.__class__.__name__
        ))
    if request["layout"]:
        widget.Layout()
    widget.Refresh()
  1. rliterate2.py
  2. globals 
WX_DEBUG_REFRESH = os.environ.get("WX_DEBUG_REFRESH", "") != ""

Container widgets need extra handling. While child widgets are being processed, all refresh requests need to be captured, and when the container is finished, all captured refresh requests should be processed.

  1. rliterate2.py
  2. base classes
  3. WxContainerWidgetMixin
  4. _setup_gui 
self._captured_requests = None
self._prune_filter = None
  1. rliterate2.py
  2. base classes
  3. WxContainerWidgetMixin 
def handle_refresh_requests(self, requests):
    if self._captured_requests is None:
        WxWidgetMixin.handle_refresh_requests(self, requests)
    else:
        if self._prune_filter is not None:
            requests.prune(**self._prune_filter)
        self._captured_requests.take_from(requests)
  1. rliterate2.py
  2. base classes
  3. WxContainerWidgetMixin 
def _update_gui(self):
    self._captured_requests = RefreshRequests()
    try:
        self._updagte_gui_with_capture_active()
    finally:
        captured = self._captured_requests
        self._captured_requests = None
        self.handle_refresh_requests(captured)
  1. rliterate2.py
  2. base classes
  3. WxContainerWidgetMixin 
def _updagte_gui_with_capture_active(self):
    self._prune_filter = None
    WxWidgetMixin._update_gui(self)
    if self._captured_requests.has(
        layout=True,
        immediate=False
    ):
        self._captured_requests.prune(
            layout=False,
            immediate=False
        )
        self._prune_filter = {
            "immediate": False,
        }
    elif self._captured_requests.has(
        immediate=False
    ):
        self._prune_filter = {
            "layout": False,
            "immediate": False,
        }
    if self._update_children():
        self._prune_filter = None
        self._captured_requests.prune(
            immediate=False
        )
        self._request_refresh(layout_me=True)

RefreshRequests is an object to simplify working with refresh requests.

  1. rliterate2.py
  2. classes 
class RefreshRequests(object):

    def __init__(self, *initial_requests):
        self._requests = list(initial_requests)

    def take_from(self, requests):
        requests.process(self.add)

    def add(self, request):
        if request not in self._requests:
            self._requests.append(request)

    def process(self, fn):
        result = [
            fn(x)
            for x
            in self._requests
        ]
        self._requests = []
        return result

    def has(self, **attrs):
        return any(
            self._match(x, **attrs)
            for x
            in self._requests
        )

    def prune(self, **attrs):
        index = 0
        while index < len(self._requests):
            if self._match(self._requests[index], **attrs):
                self._requests.pop(index)
            else:
                index += 1

    def _match(self, request, **attrs):
        for key, value in attrs.items():
            if request[key] != value:
                return False
        return True
Start app
  1. rliterate2.py
  2. functions 
def start_app(frame_cls, props):
    @profile_sub("render")
    def update(props):
        frame.update_props(props)
    @profile("show frame")
    @profile_sub("show frame")
    def show_frame():
        props.listen(lambda: update(props.get()))
        frame = frame_cls(None, None, {}, props.get())
        frame.Show()
        return frame
    app = wx.App()
    frame = show_frame()
    if WX_DEBUG_FOCUS:
        def onTimer(evt):
            print("Focused window: {}".format(wx.Window.FindFocus()))
        frame.Bind(wx.EVT_TIMER, onTimer)
        focus_timer = wx.Timer(frame)
        focus_timer.Start(1000)
    app.MainLoop()
  1. rliterate2.py
  2. globals 
WX_DEBUG_FOCUS = os.environ.get("WX_DEBUG_FOCUS", "") != ""

History

This chapter documents the evolution and development of RLiterate.

The idea (Dec 2017)

I started to think about what would become RLiterate when I read the paper Active Essays on the Web. In it they talk about embedding code in documents that the reader can interact with:

An Active Essay combines a written essay, program fragments, and the resulting live simulations into a single cohesive narrative.

They also mention literate programming as having a related goal of combining literature and programming.

At the time I was working on a program that I thought would be nice to write in this way. I wanted to write an article about the program and have the code for the program embedded in the article. I could have used a literate programming tool for this, but the interactive aspect of active essays made me think that a tool would be much more powerful if the document could be edited "live", similar to WYSIWYG editors. Literate programming tools I were aware of worked by editing plain text files with a special syntax for code and documentation blocks, thus lacking the interactive aspect.

So the tool I envisoned was one that would allow literate programming and also allow more interaction with the content.

The prototype (Jan-Mar 2018)

So I decided to build a prototype to learn what such a tool might be like.

First I came up with a document model where pages were organized in a hierarchy. Each page had paragraphs that could be of different types. This idea was stolen from Smallest Federated Wiki. The code paragraph would allow for literate programming, while the text paragraph would be for prose. I also envisioned other paragraph types that would allow for more interaction. Perhaps one paragraph type could be Graphviz code, and when edited, a generated graph would appear instead of the code.

After coming up with a document model, I implement a GUI that would allow editing such documents. This GUI had to be first class as it would be the primary way author (and read) documents.

The first version of the GUI was not first class though. I started to experiment with a workspace for showing pages.

First GUI prototype of workspace.

Then I contintued with a table of contents. At this point it was based on a tree control present in wxPython.

First GUI prototype of table of contents. Workspace pages also have titles.

All the data so far had been static and hard coded. I started to flesh out the document model and implement GUI controls for manipulating the document. Here is the add button on pages.

Add button that creates the factory paragraph.

Drag and drop was implemented fairly early. I wanted good visual feedback. This was part of the first class idea.

Drag and drop with red divider line indicating drop point.

Drag and drop with good visual feedback was hard to achieve with the tree control from wxWidgets, so at this point I rewrote the table of contents widget as a custom widget.

Custom table of contents that allows drag and drop and custom rendering.

I added more operations for adding and deleting pages.

Context menu that allows adding and deleting pages.

Then finally I added a second paragraph type: the code paragraph type that would enable literate programming.

Code paragraphs showing literate programming in action.

At this point I had all the functionality in place for writing documents and embedding code in them. I imported all the code into an RLiterate document (previously it was written as a single Python file) and started extracting pieces and adding prose to explain the program. This was a tremendously rewarding experience. RLiterate was now bootstrapped.

Improving the prototype (Jul-Sep 2018)

Dogfeeding. I imporoved RLiterate inside RLiterate.

As I continued to improve the RLiterate document, I noticed features I was lacking. I added them to the program and iterated.

ProjecturED was an influence for introducing variables.

When researching ideas for RLiterate, I stumbled across ProjecturED. It is similar to RLiterate in the sense that it is an editor for more structured documents. Not just text. The most interesting aspect for me was that a variable name exists in one place, but can be rendered in multiple. So a rename is really simple. With RLiterate, you have to do a search and replace. But with ProjecturED you just change the name and it replicates everywhere. This is an attractive feature and is made possible by the different document model.

Writing "the program" (Sep-Dec 2018)

I started writing the article and noticed other features I was missing.

Direct manipulation (Dec 2018-Feb 2019)

Started work on getting rid of modes.

Inspired by Alan Kay and Lary Tesler.

Successful experiment with no-modes editing of titles in dec 2018.

RLiterate works in modes: view and edit. This is in conflict with "no modes". ProjecturED is more direct. It can show multiple projections of the same data structure, but full direct editing is available in both projections. RLiterate can show two projections (view and edit) but only supports edits in one.

Support for more direct editing might be desierable. But I'm a Vim user, and I think modes are powerful. I need to investigate this area more. Can direct manipulation feel as "precise" as different modes? How can automatic save work if there is no explicit "edit ends"?

PDF by Lary Tesler about modes: A Personal History of Modeless Text Editing and Cut/Copy-Paste .

If modes must be used, it should be visually clearly indicated that you are in a mode.

On WYSIWYG by Alan Kay.

Rewrite (Oct 2019)

I had been using RLiterate to write blog posts and really liked its features. However, it had some performance issues that made it annoying to use as well as some recurring bugs. I concluded that in order to fix these issues and also continue to improve RLiterate, a complete rewrite was necessary.

I decided to try doing a rewrite with the following initial goals:

I also thought that the features in the second version should be roughly the same as in the first version.

Proof of concept (Oct 2019)

One of the goals with the rewrite was to improve performance. One thing that made the first version of RLiterate slow was that the GUI was re-rendered even though nothing had changed. To solve this, I though using an architecture inspired by React might work.

React works by passing props, which is a JSON document, to a component that then renders the GUI from that data. The props is immutable which makes it easy for the component to see if it is exactly the same as in the previous render or if something changed. The first version of RLiterate had a similar architecture, but the props passed to the component was a Python class which had methods to get the different parts of the data. So even if nothing had changed, the component had to call those methods and re-render because it could not know if something changed. With an immutable Python dictionary as props instead, comparing to values would be easy.

So I decided to try this architecture out. The first thing I implemented was the main layout of RLiterate where the table of contents could be resized by dragging.

Resizing table of contents.

It worked as follows:

{
  'title': 'example.rliterate (/home/rick) - RLiterate 2',
  'toolbar': {
    'margin': 4
  },
  'toolbar_divider': {
    'color': '#aaaaaf',
    'thickness': 2
  }
  'main_area': {
    'toc': {
      'background': '#ffeeff',
      'set_width': <bound method ...>,
      'width': 229
    },
    'toc_divider': {
      'color': '#aaaaff',
      'thickness': 3
    },
    'workspace': {}
  },
}

Since performance was the main focus with this proof of concept, I added profiling early to see what parts of a render took time. I concluded that if a render is done in less that 10ms (100 frames per second) then that is good enough. 20ms (50 frames per second) starts to feel significantly slower.

From a performance perspective, this architecture seemed to work pretty well. So this gave me confidence to move forward with it and see if it would scale for what RLiterate needs.

This proof of concept also improved RLiterate by allowing the table of contents to be resized, something which was not possible in the first version.

Another goal with the rewrite was to clean up old cruft. One such thing which was on the TODO list for the first version was to completely separate wx GUI code from the RLiterate code. RLiterate had a GUI framework so that it did not depend directly on the underlying GUI framework. In the first version this was not 100% completed. RLiterate code still had a direct dependency on wxPython. In the second version this was fixed. Since it was started from scratch, this was easier to just ensure.

Fleshing out table of contens (Oct-Nov 2019)

Figure out how the document should interact with the GUI architecture. Do it by fleshing out table of contents.

First attempt at rendering table of contents.

Fleshing out structure.

Proper styling.

Experiment with theme switcher to see how it works throughout.

Table of contents can be modified by drag and drop.

And pages can be hoisted.

{
  'title': 'rliterate2.rliterate (/home/rick/rliterate2-history) - RLiterate 2',
  'toolbar': {
    'background': None,
    'margin': 4,
    'rotate_theme': <bound method ...>
  },
  'toolbar_divider': {'background': '#aaaaaf', 'min_size': (-1, 1)}
  'main_area': {
    'toc': {
      'background': '#ffffff',
      'has_valid_hoisted_page': False,
      'min_size': (230, -1),
      'row_margin': 2,
      'scroll_area': {
        'can_move_page': <bound method ...>,
        'dragged_page': None,
        'drop_points': [
          TableOfContentsDropPoint(row_index=0, target_index=0, target_page='46533f3be0674d1cae08ff281edbc37d', level=1),
          TableOfContentsDropPoint(row_index=1, target_index=0, target_page='165a1189e30f4ce5b22001ea8091aa9c', level=2),
          ...
        ],
        'move_page': <bound method ...>,
        'row_extra': {
          'divider_thickness': 2,
          'dragdrop_color': '#ff6400',
          'dragdrop_invalid_color': '#cccccc',
          'foreground': '#000000',
          'hover_background': '#cccccc',
          'indent_size': 20,
          'row_margin': 2,
          'set_dragged_page': <bound method ...>,
          'set_hoisted_page': <bound method ...>,
          'toggle_collapsed': <bound method ...>
        },
        'rows': [
          {
            'collapsed': False,
            'dragged': False,
            'has_children': True,
            'id': '46533f3be0674d1cae08ff281edbc37d',
            'level': 0,
            'title': 'RLiterate 2'
          },
          {
            'collapsed': False,
            'dragged': False,
            'has_children': False,
            'id': '165a1189e30f4ce5b22001ea8091aa9c',
            'level': 1,
            'title': 'File'
          },
          ...
        ],
        'total_num_pages': 52
      },
      'set_hoisted_page': <bound method ...>
    },
    'set_toc_width': <bound method ...>,
    'toc_divider': {
      'background': '#aaaaaf',
      'min_size': (3, -1)
    },
    'workspace': {
      'background': '#cccccc'
    }
  },
}

Had to introduce caching in GUI layer to make expand/collapse fast.

Fleshing out workspace (Nov 2019, Jan 2020)

The document can not yet be saved. All changes done in table of contents are in memory. Since performance is the main goal with this rewrite, it is more interesting to flesh out the workspace than to implement save. That way we can get feedback faster if the new GUI architecture will work throughout.

Can resize columns with drag.

Proper styling of page border.

Hard code some pages to display and render their titles.

First take at rendering some code.

Fleshing out syntax highlighting.

Had to introduce caching in props generation for paragraphs to make unrelated prop changes fast. At this point, generating props for a code paragraph includes syntax highlighting which is slow.

Add support for more paragraph types. Here image and list is shown.

Made clicking a title append a dot character to it to see how fast it re-rendered. The workspace re-rendered in under 10ms. The table of contents took some extra time though, so probably more caching can be done there.

I had now fleshed out enough of the workspace to be confident that the new architecture would work well for what RLiterate needed and also be fast enough.

Completing rewrite (Jan 2020-??)

I started by adding direct manipulation of titles. That meant introducing a selection, fleshing out more events, tweaking interface of text widget.

Title with a selection.

Paragraph text with a selection.

Added smarted caching of TOC so that generating TOC props would be faster.

Implemented smarter refresh of widgets to redraw only edit parts first. Measured with Typometer.

Experiment with local toolbar to show details about the current selection.

Flesh out more direct manipulation of text.

Ideas for new features

Review TODO from first version.

Figure out rendering order of files automatically so that rlgui.py is rendered before rliterate2.py since the later depends on the former.

Support for plugins/script buttons in documents. In the style of Leo. For example, I would like a button in the toolbar that says "Publish" for blog posts.

Support altering a code paragraph in a later page to allow incremental tutorial style writing. (As snaptoken's kilo).

Support examples that can be scrubbed. Sort of interactive TDD test code.

Better support for moving/renaming code chunks. For example moving chunks "foo/bar" to "foo/blah/bar" requires manual editing of all chunks.

Elastic tabstops?

Renaming files does not remove the old file from disk.

Generated from dc4ea84e5ef15b496df66440eecd2239fae9130c.

What is Rickard working on and thinking about right now?

Every month I write a newsletter about just that. You will get updates about my current projects and thoughts about programming, and also get a chance to hit reply and interact with me. Subscribe to it below.