A Newsletter Domain-Specific-Language with Org-Mode
Table of Contents
Attention Conservation Notice
An attempt at creating the building blocks necessary for a simple newsletter generating domain-specific-language in org-mode. Not very useful unless you are building a newsletter of sufficient complexity (or are sufficiently stubborn to turn everything into plain text instead of using a GUI).
Preamble
I have been writing a quarterly newsletter for the past couple of quarters. To be more communal, I made the mistake of using $DOMINANT_OS_DISTRIBUTOR
's $OFFICE_SOFTWARE
newsletter maker $ONE_WHO_PUBLISHES
since the folks I write the newsletter with don't code.
Difficulties with $ONE_WHO_PUBLISHES
The program provides an area with which to drag and drop different elements. In (my) practice this amounted to dragging squares all over the place, many of which overlap and obscure each other. When using more than 3+ replicas of a single component style, making changes became cumbersome. An update on one component would mean having to manually update the others. Since all the components share a single workspace, changing one item causes shifts in the placement of other items. With 10+ matching components, this became very time consuming. I'd estimate that I spent more time fiddling with components, moving things out of the way, and moving them back than I did making/adding content. The resulting newsletter ended up being both time-consuming to make and inconsistent in presentation.
A new hope
Being the libre-loving code-monkey that I am, my first instinct was to:
- never do that again
- write out something that can be version-controlled and will be consistent
- escape the clutches of a software ecosystem that may not be relevant in \(n\) years (as technology release velocity increases, \(n \to 0\)) in favor of plain-text and open formats
Goals for the End Product
- It needs to be usable by non-programmers
- The build system should not require any special skills or arcane terminal usage
- The language itself should be pretty straightforward
- extending and adding functionality should be simple
Final Result (Visual)
Final Result (Code + Stylesheet)
This is the code that is written by the newsletter writer
Click for Newsletter Code
#+BEGIN_PROPERTIES #+TITLE: @@html:<p class="title-p"><span class="title-left">Q3</span><span class="title-right">Newsletter Update</span></p>@@ #+SUBTITLE: @@html:<p class="subtitle-p"><span class="vol-issue-left">Vol. 1, Issue 3</span><span class="vol-issue-right">30Sep2022</span></p><hr>@@ #+DATE: <2022-07-17 Sun> #+OPTIONS: num:nil author:nil date:nil html-postamble:nil #+HTML_HEAD: <link rel="stylesheet" type="text/css" href="newsletter_styles.css"/> #+MACRO: split-sentence @@html: <span class="date-str-date"><code>$1</code></span> | <span class="date-str-str">$2</span><br>@@ #+MACRO: h3 @@html: <span class="in-column-h3">$1</span><br>@@ #+MACRO: hr #+HTML: <hr> #+FOOTER: #+END_PROPERTIES * Welcome! {{{hr}}} *Here is some text welcoming you!* Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum. * Our Main News Story {{{hr}}} #+begin_banner ./images/nscrop.png #+end_banner Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam mattis orci ultrices enim gravida, vitae venenatis libero faucibus. Donec at pulvinar sem, ut lobortis elit. Mauris eget pretium nunc, pretium consequat tellus. Ut justo augue, iaculis id efficitur ullamcorper, suscipit sit amet sapien. In pulvinar id mauris vel viverra. Duis vitae nunc consequat, iaculis eros id, cursus nisi. Cras vitae sem metus. Duis lobortis porta dui, vel volutpat turpis pretium vitae. Cras aliquet lectus eu felis lacinia euismod. Maecenas eros velit, pellentesque lacinia ipsum id, imperdiet cursus leo. Nullam pharetra blandit quam, ut ornare risus gravida quis. Nullam hendrerit et erat et faucibus. Aenean vitae sagittis mi. Nunc ornare congue fringilla. Aliquam luctus nisi bibendum orci dignissim, et consectetur tortor dapibus. Vivamus lacinia accumsan pellentesque. Fusce nisi neque, auctor eu tellus in, ullamcorper aliquet ipsum. Duis dui nisi, aliquet consectetur tincidunt non, ultricies id metus. Maecenas tristique laoreet egestas. Nunc lacinia, tortor id sagittis scelerisque, mi est viverra quam, at porttitor justo lorem vel ipsum. Praesent aliquet enim magna. Sed luctus neque nibh, vel egestas eros ultrices in. Nulla efficitur accumsan nisi, et venenatis velit fringilla at. Nam vitae magna id neque elementum pellentesque ut eu lacus. Vivamus sapien libero, elementum aliquam leo id, luctus pellentesque augue. Donec ac ligula sit amet lacus consectetur hendrerit. Aenean vitae tempor nulla, at dapibus tortor. In eu congue quam. Nulla facilisi. Nam massa metus, suscipit a cursus et, posuere sit amet mauris. Sed id mi sem. Phasellus placerat augue nisl, sit amet eleifend elit semper ut. Maecenas eget risus ultricies, feugiat diam et, auctor est. Integer maximus orci vitae lectus commodo scelerisque. Fusce a elementum mauris. Maecenas in cursus ipsum. Pellentesque accumsan tempor sem hendrerit auctor. Mauris placerat est volutpat sapien luctus congue. Quisque pharetra mauris quis purus cursus dapibus. Maecenas molestie vehicula lacus aliquet gravida. Morbi euismod lacinia enim, at volutpat ex lacinia eget. Quisque in ligula ut metus mattis luctus. Praesent ut vehicula mi. Vestibulum velit odio, convallis eu suscipit blandit, convallis quis ex. Nullam sagittis est vitae diam tristique imperdiet. Fusce vel urna auctor, feugiat libero vel, bibendum lectus. Curabitur aliquam dolor non faucibus porta. * A look at some cats {{{hr}}} #+begin_center *hello, world!* All of the cat images in this were generated by https://thiscatdoesnotexist.com/ #+end_center #+begin_twocol *Fusce nisi neque*, auctor eu tellus in, /ullamcorper aliquet ipsum/. _Duis dui nisi_, aliquet consectetur =tincidunt non=, ultricies id metus. Maecenas tristique laoreet egestas. Nunc lacinia, tortor id sagittis scelerisque, mi est viverra quam, at porttitor justo lorem vel ipsum. Praesent aliquet enim magna. Sed luctus neque nibh, vel egestas eros ultrices in. Nulla efficitur accumsan nisi, et venenatis velit fringilla at. Nam vitae magna id neque elementum pellentesque ut eu lacus. Vivamus sapien libero, elementum aliquam leo id, luctus pellentesque augue. Donec ac ligula sit amet lacus consectetur hendrerit. Aenean vitae tempor nulla, at dapibus tortor. In eu congue quam. Maecenas molestie vehicula lacus aliquet gravida. Morbi euismod lacinia enim, at volutpat ex lacinia eget. Quisque in ligula ut metus mattis luctus. ./images/c1.jpeg #+end_twocol #+begin_twocol ./images/c2.jpeg *Mauris efficitur nisi ut ex efficitur, id ultrices felis dignissim?* Nam fringilla arcu eu mi accumsan, in rhoncus velit auctor. *Aenean eget lacus dapibus, venenatis arcu sed, egestas nisl?* Cras quis elit porta, imperdiet turpis ac, luctus dui. *Sed non ligula porta purus sodales accumsan?* Integer vestibulum tortor sit amet maximus facilisis. *Vestibulum pulvinar urna a varius lacinia?* Nullam porta dolor quis condimentum aliquet. #+end_twocol ** Pet Trio! {{{hr}}} #+begin_threecol #+CAPTION: Choco ./images/c3.jpeg #+CAPTION: Latte ./images/c4.jpeg #+CAPTION: Chip ./images/c5.jpeg #+end_threecol * Events {{{hr}}} ** Fitness | Event Name | Date | Notes | Emoji | |-----------------------------+------------------+---------------+-------| | Doing Yoga with Goats | <2022-07-18 Mon> | very fun! | 😺 | | Weightlifting with Gorillas | <2022-07-25 Mon> | very intense | 😸 | | Calisthenics with Ants | <2022-08-18 Thu> | not even fair | 😹 | ** Non-Fitness | Event Name | Date | Notes | Emoji | |----------------------------+------------------+------------+-------| | Sitting Around with Sloths | <2022-07-26 Tue> | *yawn* | 😻 | | Not Doing Much with Bats | <2022-07-27 Wed> | snooze | 😼 | | Cat Naps with Dogs | <2022-07-28 Thu> | good times | 🙀 | * Celebrations {{{hr}}} #+begin_twocol {{{h3(Set of Chaotic Maps 1)}}} {{{split-sentence(10 coolness, 3-Cells CNN System)}}} {{{split-sentence(11 coolness, 2D Lorenz System)}}} {{{split-sentence(12 coolness, 2D Rational Chaotic Map)}}} {{{split-sentence(13 coolness, ACT Chaotic Attractor)}}} {{{split-sentence(14 coolness, Aizawa Chaotic Attractor)}}} {{{split-sentence(15 coolness, Arneodo Chaotic System)}}} {{{h3(Set of Chaotic Maps 2)}}} {{{split-sentence(16 coolness, Arnold's Cat Map)}}} {{{split-sentence(17 coolness, Baker's Map)}}} {{{split-sentence(18 coolness, Basin Chaotic Map)}}} {{{split-sentence(19 coolness, Beta Chaotic Map)}}} {{{split-sentence(20 coolness, Bogdanov Map)}}} {{{split-sentence(21 coolness, Brusselator)}}} {{{h3(Set of Chaotic Maps 3)}}} {{{split-sentence(22 coolness, Burke-Shaw Chaotic Attractor)}}} {{{split-sentence(23 coolness, Chen Chaotic Attractor)}}} {{{split-sentence(24 coolness, Chen-Celikovsky System)}}} {{{split-sentence(25 coolness, Chen-LU System)}}} {{{split-sentence(26 coolness, Chen-Lee System)}}} {{{split-sentence(27 coolness, Chossat-Golubitsky Symmetry Map)}}} #+CAPTION: This is artwork made by a GAN ./images/a1.jpeg #+end_twocol #+begin_footer This newsletter was built with *Org-Mode* #+end_footer
This is the stylesheet that isn't touched by the newsletter writer
Click for Template Stylesheet
@import url(https://fonts.googleapis.com/css?family=Montserrat:400,500,80); :root { --c1: #F2F1E8; --c2: #A8DADC; --c3: #050533; --c4: #1D3557; --c5: #E34234; --c6: #403D39; } body { width: 100vw; background-color: var(--c1); color: var(--c6); line-height: 1.4; font-size: 18px; font-family: 'Montserrat', sans-serif; } h1, h2 { color: var(--c5); font-weight: bold; } h3 { color: var(--c3); } p { padding: 20px; } .twocol { background-color: var(--c2); column-rule: dotted var(--c5); column-rule-width: thin; column-count: 2; padding-top: 25px; padding-bottom: 25px; margin-top: 10px; margin-bottom: 10px; border-radius: 10px; } .twocol img { max-width: 100%; max-height: 100%; border-radius: 10px; } .threecol { background-color: var(--c2); column-rule: dotted var(--c5); column-rule-width: thin; column-count: 3; padding-top: 25px; padding-bottom: 25px; border-radius: 10px; } .threecol img { max-width: 100%; max-height: 100%; border-radius: 10px; } .banner { padding: 0px; margin: 0px; } .banner img { width: auto; padding: 0px; margin: 0px; } .banner .figure { padding: 0; } .banner .figure p { padding: 0; } .timestamp { font-size: 18px; color: var(--c5); font-weight: thin; } .top-banner { float: right; } .title-left { float: left; color: var(--c2); font-size: 64px; font-family: 'Montserrat', sans-serif; font-weight: bold; } .title-right { float: right; color: var(--c5); font-size: 64px; font-family: 'Montserrat', sans-serif; font-weight: bold; } .vol-issue-left { float: left; color: var(--c3); font-weight: thin; font-family: 'Montserrat', sans-serif; } .vol-issue-right { float: right; color: var(--c3); font-weight: thin; font-family: 'Montserrat', sans-serif; } .title-p { padding-top: 0px; padding-bottom: 1px; } .subtitle-p { padding: 8px; } hr { border-top: 1px dotted; border-bottom: 0px; border-color: var(--c5); } a { color: var(--c3); text-decoration: none; } a:hover { color: var(--c4); } .date-str-date { font-weight: thin; color: var(--c5) } .date-str-str { color: var(--c4) } .footer { float: right; } .in-column-h1 { font-weight: bold; font-size: 36px; } .in-column-h2 { font-weight: bold; font-size: 28px; } .in-column-h3 { font-weight: bold; font-size: 20px; } .figure-number { display: none; } table { border-collapse: separate; border-spacing: 0; } table tr th, table tr td { border-right: 1px solid #bbb; border-bottom: 1px solid #bbb; box-shadow: 2px 2px 1px #e5dfcc; } table tr th:first-child, table tr td:first-child { border-left: 1px solid #bbb; } table tr th { border-top: 1px solid #bbb; } /* top-left border-radius */ table tr:first-child th:first-child { border-top-left-radius: 0px; } /* top-right border-radius */ table tr:first-child th:last-child { border-top-right-radius: 0px; } /* bottom-left border-radius */ table tr:last-child td:first-child { border-bottom-left-radius: 6px; } /* bottom-right border-radius */ table tr:last-child td:last-child { border-bottom-right-radius: 6px; } th, td { padding: 8px 20px; } th { background: var(--c2); color: var(--c4); } td { background: var(--c1); }
The Secret Sauce
Below you will find an overview of the different components and the related bits of org-markdown-language created. Before we get into that jazz, let's zoom out and ask:
What is the secret sauce? How does it all work?
3 big things allow this to work:
- Org-mode has a ton of functionality already built into it, including different types of markup and export options
- Special blocks can be created on the fly, giving the user a container object to interact with instead of HTML/CSS/Javascript. These can then be styled with CSS
- If functionality needs to be added to a component, org lets the user create macros that expand at export time. These can inject HTML, CSS, or emacs-lisp into the export
Org-mode functionality
Org comes with a lot of functionality out of the box. You'd be well served to look at the org syntax specification
The most important part for the end-user (of the newsletter generator) is to know the very basics:
# Headers * This is a heading ** This is a subheading *** etc etc # Lists 1. item 1 2. item 2 3. item 3 - item 5 - item 6 - item 7 # Inline styles *bold* /italic/ _underline_ =monospace= ~code~ +strike-through+
Special Blocks
For every component I want to add to the newsletter, I simply make a corresponding org-mode block like so:
#+begin_component-name some content here #+end_component-name
At export time, this turns into:
<div class="component-name"> <p>some content here</p> </div>
Here is the relevant section from the org-manual:
When special blocks do not have a corresponding HTML5 element, the HTML exporter reverts to standard translation (see org-html-html5-elements). For example,
#+BEGIN_lederhosen
exports to <div class="lederhosen">
Since org export turns things it doesn't recognize as HTML tags into a div with a class, I can then style the class with CSS in the style.css
file.
Extra details
This also works particularly nice when you use a tag that is found in html5. As an example,
#+BEGIN_aside Lorem ipsum #+END_aside
exports to:
<aside> <p>Lorem ipsum</p> </aside>
Functionality with Macros
If I want to add functionality to something, I can use a macro that creates HTML tags (or css, elisp, etc) to implement the functionality.
Macros allow org to replace text during export. Here is an example from the org-manual:
{{{poem(red,blue)}}}
becomes
Rose is red, violet's blue. Life's ordered: Org assists you.
It even allows the use of emacs-lisp snippets, but I haven't needed to use that functionality here (yet).
For the newsletter I use the following 3 macros (defined in the BEGIN_PROPERTIES
block):
This syntax:
@@html: stuff here@@
allows org to directly place html into the exported output (instead of running it through its preprocessor).
These are all used throughout the newsletter using the following syntax:
# example {{{something(arg1, arg2)}}} <h1>Write arg1 Write arg2 # actual usage {{{split-sentence(left hand side, right hand side)}}} {{{h1(sentence here)}}} {{{hr}}}
where something
is the macro name and the args are replacing the $1
and $2
anchors.
Build Tool
The build tool is very simple. The idea is the following:
- Write the newsletter as newsletter.org
- Build the newsletter with emacs through an executable script
This way the end user only has to do 3 things to get going:
- Write the file in emacs
- Build the file by clicking a shortcut
- Inspect the results in the browser
On Windows
Since emacs is very portable on Windows
(you can download a self-contained program in a folder here) each user can get access to it without much trouble.
To create an executable
Then the build tool can be put together using a little bit of elisp and powershell
Here is the file create-html.el
#!/home/neptune/.guix-profile/bin/emacs --script (progn (find-file "newsletter.org") (org-html-export-to-html) (kill-buffer)) (browse-url--browser "newsletter.html")
Then the user can make the file executable and run the command
./create-html.el
Additional Notes
Why HTML? Why not pdf?
While there are ways of creating PDF content with org-mode, getting all the styling and spacing to line up seemed like an extra hurdle for little gain. I find it much easier to parse and tinker with HTML pages than PDF pages.
Components Overview
table of contents stylized
Syntax
This comes for free with org-export to html. As a default org export also creates unique anchor points for each of the headers and subheaders. This can be customized to use the verbatim text as an anchor point as well. In our case, it doesn't matter too much what the anchor wording is.
If you want to get more readable link anchors, you can use the following emacs lisp snippet that I took from this excellent blog post:
(defun my/ensure-headline-ids (&rest _) "All non-alphanumeric characters are cleverly replaced with ‘-’. If multiple trees end-up with the same id property, issue a message and undo any property insertion thus far. " (interactive) (let ((ids)) (org-map-entries (lambda () (org-with-point-at (point) (let ((id (org-entry-get nil "CUSTOM_ID"))) (unless id (thread-last (nth 4 (org-heading-components)) (s-replace-regexp "[^[:alnum:]']" "-") (s-replace-regexp "-+" "-") (s-chop-prefix "-") (s-chop-suffix "-") (setq id)) (if (not (member id ids)) (push id ids) (message-box "Oh no, a repeated id!\n\n\t%s" id) (undo) (setq quit-flag t)) (org-entry-put nil "CUSTOM_ID" id)))))))) ;; Whenever html & md export happens, ensure we have headline ids. (advice-add 'org-html-export-to-html :before 'my/ensure-headline-ids) (advice-add 'org-md-export-to-markdown :before 'my/ensure-headline-ids)
prefix | sentence
Syntax
This is a simple macro that expands to html at export time.
This provides syntax like the following:
{{{split-sentence(first argument, second argument)}}}
which expands to the following at export time
<span class="date-str-date"><code>first argument</code></span> | <span class="date-str-str">second argument</span><br>
We can add a class to style it as well:
.date-str-date { font-weight: thin; color: var(--c5) } .date-str-str { color: var(--c4) }
left-image
Syntax
This creates a div with the class id twocol
. Where the magic happens is in the css file. With css, I can split the div into n columns and style the intersecting border:
.twocol { background-color: var(--c2); column-rule: dotted var(--c5); column-rule-width: thin; column-count: 2; padding-top: 25px; padding-bottom: 25px; margin-top: 10px; margin-bottom: 10px; border-radius: 10px; } .twocol img { max-width: 100%; max-height: 100%; border-radius: 10px; }
Then, when I enter text into a twocol block, it will split the content into the first set of content and the second set.
#+begin_twocol an image goes here some text accompanying it goes here #+end_twocol
This is also generalized in the right image and triple-image components
right-image
Syntax
This is the same thing as the left image component, except we add the image to the block second:
#+begin_twocol some text accompanying the image goes here an image goes here #+end_twocol
triple-image
Syntax
This is the same as left-image, except we use css to create 3 images.
.threecol { background-color: var(--c2); column-rule: dotted var(--c5); column-rule-width: thin; column-count: 3; padding-top: 25px; padding-bottom: 25px; border-radius: 10px; } .threecol img { max-width: 100%; max-height: 100%; border-radius: 10px; }
This could also house content like image | text | image
or any 3-tuple of image and text inputs. In the future, if needed, I could add any n-tuple.
headers inside of columns
I wanted to keep the functionality sufficiently separated from the component itself, i.e. don't hardcode the header transformation into the underlying component. To do this, I used an org-macro (see The Secret Sauce > Functionality with Macros
) and allow the user to invoke it when desired.
Syntax
{{{h3(Set of Chaotic Maps 1)}}}
image captions
This comes for free with org-mode. Simply add the #+CAPTION
tag above an image
Syntax
This is [[https://thisartworkdoesnotexist.com/][artwork]] made by a GAN ,./images/a1.jpeg
tables
Tables are super easy to use in org-mode. I usually just write \(n\) | characters in a row and |- like so for a 3-column table:
|||| |-
and then hit <tab>
and the table is created like so:
| | | | |---+---+---| | | | |