Emacs on Lucid Manager

Why Use Emacs

Peter Prevos — Fri, 07 May 2021 00:00:00 +0000

Most people believe that the first step in becoming more productive is finding the perfect suite of software tools to manage the vast amounts of information we consume daily. I also used to hop from application to application to perform what I needed—jumping from the action list to my schedule, onward to the word processor and spreadsheet, PDF reader and so on. Wouldn't it be nice if there was one program that could help you with almost all your tasks?

Welcome to the Emacs computing system, the Swiss army chainsaw of productivity.

This article explains why you should use Emacs for writing and undertake almost every task in your writing project, from ideation to publication in one application. This article is part of the Emacs Writing Studio starter kit, which explains how to use Emacs to write and publish articles, books, theatre or film scripts and websites.

Emacs Writing Studio

A comprehensive guide for writers seeking to streamline their workflow using Emacs. The book covers everything from organising ideas and writing distraction-free to publishing in multiple formats. It’s perfect for both beginners and experienced Emacs users, offering practical tips and a tailored configuration to enhance your writing process.

Payhip e-readers Paperback LeanPub

You can find the most recent configuration files on GitHub.

What is Emacs?

The official tagline of GNU Emacs is that it is an "extensible self-documenting text editor". However, these words barely do justice to Emacs because they focus on its original purpose as a software development tool.

Emacs is a multi-purpose computing environment that can help you manage your information, track projects, write and publish articles, books, websites and any other text-based activity. Emacs is a text processor on steroids. The first version of Emacs was released forty years ago, which might seem obsolete software. However, a vibrant community of developers continually improves the system.

Emacs is not a productivity hack; it is a productivity hacking system.

The term text editor requires clarification because it is more akin to developing software. Emacs is not merely a text editor but a text processor, which is more than just a word processor. Editing is the last step in the process of writing an article or book. Software developers refer to the whole process as editing.

When you use a computer, you more than likely write a memo in a word processor, then switch to a spreadsheet to create a graph, which you copy and paste into your memo. Next, you open your email client to send the message to your colleague. The last step is to close the action in your productivity tool. After this hard work, you might play a game and open Tetris. When using these pieces of software, you have to learn different skills for each program. You have to use the software the way the developers intended you to use it.

Working with Emacs is a different experience. You write the memo in Emacs, create the graph with Emacs, email the result and close the action with Emacs. Finally, you play the game, you guessed it, within Emacs. If you use Emacs, you only need to learn one piece of software, and you can configure and extend it to how you prefer to use it, not how a programmer wants you to use the software.

At first instance, Emacs looks more like an angry fruit salad than a modern piece of software. The default interface looks dreary, but don't let these simple looks fool you. Under the hood, Emacs is a modern and robust computing environment that is fully configurable to your liking.

Why use Emacs?

Emacs is a malleable software platform, which means that you are free to change and enhance how it works. This malleability ensures that Emacs can perform any task that you can undertake with a keyboard. Emacs is, as such, the Swiss Army Chainsaw of productivity.

The first principle of malleable software is that it is easy to change. You can build applications using the Emacs LISP language (Elisp). This task might sound daunting, but it is about the possibility. Writing code is optional because most Emacs users share what they have developed through thousands of external packages you can freely install.

Emacs Writing Studio makes it easy for you to get started as this configuration has almost everything you need to write prose.

Users can configure almost everything in the system with little knowledge of Elisp. This knowledge requirement might seem like a hurdle, but learning how to use it will give you virtually unlimited power over how you use your computer. Software should adjust to the user, not vice versa, as is the case with most modern packages.

The advantage of this approach is that you have complete freedom in using this software. You can instruct it to do almost anything you like and configure it to your specific needs. The disadvantage is that it requires a different computing approach than contemporary software. Using Emacs throws you back to the original intent of using a computer and genuine user-friendliness.

Are you ready to change the way you use your computer? To paraphrase a famous scene from The Matrix:

You take the blue Microsoft pill—the story ends, and everything stays the same. You take the purple Emacs pill—you stay in Wonderland, and I show you how deep the rabbit hole goes.

Redefining user-friendliness

Emacs' lack of a slick graphical interface might discourage new users. Unfortunately, most software developers confuse user-friendliness with a smooth design and using a mouse. However, the graphical mouse-driven approach is not user-friendly because your dominant hand constantly moves between the mouse and the keyboard. Graphically-driven software is a gilded cage. It might be pleasant, but it is still a cage.

Emacs is a plain text editor, which means that there is limited markup, such as images, bold and italics. Plain text is not the same as plain English; it relates to how the information is stored on disk. Plain text is the opposite of rich text, which includes hidden definitions for font sizes, colours and other text and formatting attributes.

Plain text is not formatted and most commonly has a .txt extension. There are, however, many other plain text formats, such as HTML, Markdown, LaTeX and Emacs Org Mode. Windows users might be familiar with the venerable Notepad software (which is, by the way, almost as old as Emacs, but with only limited functionality).

Plain text is intelligible across computer systems, so you never have to worry about locking your writing into a proprietary format and being locked-in by a software package. The internet runs on plain text files, which is highly unlikely to change in the long term.

Text modes can display ‘graphics’. In primary school in the 1970s our teacher showed us some art printed with a computer. The art consisted of a series of keyboard characters that resemble a picture, such as this cute stegosaurus below, borrowed from the ASCII Art Website.

                         .       .
                        / `.   .’ \
                .---.  <    > <    >  .---.
                |    \  \ - ~ ~ - /  /    |
                 ~-..-~             ~-..-~
             \~~~\.'                    `./~~~/
   .-~~^-.    \__/                        \__/
 _/  O    \     /               /       \  \
(_____,    `._.'               |         }  \/~~~/
 `----.          /       }     |        /    \__/
       `-.      |       /      |       /      `. ,~~|
           ~-.__|      /_ - ~ ^|      /- _      `..-'   f: f:
                |     /        |     /     ~-.     `-. _||_||_
                |_____|        |_____|         ~ - . _ _ _ _ _>

The main benefit of using plain text over rich text is that it provides a distraction-free writing environment. In word processors, the screen looks like a printed page. While this might be aesthetically pleasing, it distracts the writer from creating content and instead fiddle with the design.

Only a tiny part of written text is printed on paper, so the What You See is What You Get (WYSIWYG) approach does not make much sense in the digital age. Plain text writing follows a different philosophy and uses the What You See Is What You Mean (WYSIWYM) approach.

What You See is What You Mean

Graphical interfaces simulate the physical world by making objects on the screen look like pieces of paper and folders on a desk. You point, click and drag documents into folders; documents appear as they would on paper and when done, they go into the rubbish bin. Graphical interfaces are a magic trick that makes you believe you are doing something physical. This approach might be convenient, but it moves the user away from understanding how a computer works.

The WYSIWYG approach distracts the mind from the content and lures the user into fiddling with style instead of writing text. Office workers around the globe waste a lot of time trying to format or typeset documents. Following the Emacs way will help you become more productive by not worrying about the document’s design until you complete the text. As I write this book, it only takes a few keystrokes to convert the text into a fully formatted ebook or web page, ready for distribution.

As I write this website, I don’t see what it will look like in published form. In Emacs, I only see text, images and some instructions for the computer on what the final product should look like (the What You Mean part of WYSIWYM). This approach lets me focus on writing text instead of worrying about the end product.

The image below shows what this looks like in action. The left side shows this article while writing it. The right side shows the result after creating the content. The colours on the screen indicate the function of the text, such as headers, italic text, or metadata, instead of the actual colour. The fonts are uniform, and Org mode hides images to allow me to focus on the text. Org mode can show the images inside the text with a simple keystroke. The image and the text remain separate files. I rarely view a live preview of the eventual output, as that negates the benefits of writing in plain text.

The WYSIWYM approach: Left: this article as I am writing it in Emacs. Right: The exported result in Firefox.

Principles of writing plain text

When I export this document to a web page or any other format, a template and/or style sheet are attached to the text, which define the design of the published work. Another advantage of this approach is that your text can be easily exported to multiple formats.

In my workflow, the Hugo software converts the text in Emacs to a usable website using my custom template and a CSS style sheet. When I convert a text to PDF, the LaTeX software attaches a document class to the text to define the layout and typography. Each export format has its own method to convert plain text to published content.

To publish any text your text will have four elements:

Content: The text itself.
Semantics: Context (markers for headings, rich text, tables, images and so on).
Template: Translates the structure of the document
Style sheet: The design of the publication.

In a WYSIWYG word processor, all these elements are embedded in one file, but only the text is visible with the semantics and style sheet applied to show the printed result. In plain text, the content and semantics are directly visible, depending on the text format. Please note that TXT files don’t contain any semantics; they are the simplest form of plain text. The table below shows how italic text is signified in four common plain text formats.

Format	Italic semantics
HTML	`<i>Italic Text</i>`
LaTeX	`\emph{Italic Text}`
Markdown	`Italic Text`
Org mode	`/Italic Text/`

Italic text in common plain text formats.

The style sheet exists separate from the text. This means that you can easily export your document to various formats with fine control over the end result. Org mode has a versatile export engine that allows you to convert your writings to a website, ebook or physical book with just a few keystrokes and some configuration.

These are only some example of using plain text for writing projects. Many other markup and markdown formats are available to turn your thoughts into a published work. For example, LilyPond is a plain text music notation format and Fountain is a format to write screenplays.

The Emacs learning curve

The second hurdle is the steep learning curve. To make Emacs work for you, you will need to learn the basics principles and learn some of the associated add-on packages. As Emacs is old software, some default settings and terminology differ from contemporary graphical software. For example, opening a file is 'visiting a file'. Pasting a text is' yanking'; cutting it is the same as killing. Emacs terminology is certainly more poetic than the current handicraft terms, such as cutting and pasting. Closing Emacs itself is called killing.

The steeper the learning curve, the bigger the reward.

The key to learning Emacs is to not get overwhelmed by the virtually infinite configuration options but to only learn those bits of functionality that you need to do what you need to do. Even without any configuration, you can do a lot with Emacs.

Learning some Elisp enables you to configure the software to your wishes. While that might sound daunting, you can simply copy and paste (kill and yank) examples from the Internet. Even better, Emacs Writing Studio is a starter kit to shorten this learning curve as it optimises Emacs for the writing workflow.

The articles on this website guide you through what you need to know for Emacs to become your favourite writing tool. You can also download and use the complete configuration file from these articles to get you started.

Advantages of using Emacs

In summary, these are some of the significant advantages of using Emacs to create written content:

One piece of software to undertake most of your computing activities makes you more productive because you only need to master one system.
You store all your information in plain text files. You will never have any problems with compatibility due to esoteric file formats.
You can modify almost everything in the software to suit your workflow.
Emacs runs on all major operating systems: All Linux flavours, Windows, Chrome OS, and macOS.
Emacs is open source and supported by a large community willing to help.

Limitations of Emacs

After singing the praises of this multi-functional editor, you would almost think that Emacs is the omnipotent god of software. Some people even have established the Church of Emacs as a mock religion to express their admiration.

Emacs can display images and integrate them with text. Still, it has limited functionality to create or modify graphical files. If you need to create or edit pictures, consider using GIMP (GNU Image Manipulation Program). Video content is currently not supported and cannot be easily integrated with text other than hyperlinks to a file or web page.

The focus on text limits working in Emacs to serial thinking to the exclusion of visual thinking. I like to scribble little diagrams in my notebook to help me solve problems. Emacs is limited to text. But you can import your doodles as images.

The second disadvantage is that Emacs does not have a fully operational web browser. There are packages to surf the web within Emacs, but they have limited functionality.

Emacs has the risk of becoming a productivity sink. But remember that just because you can configure everything does not mean you should. Don't spend too much time worrying about the minutia of your workflow. Spend this time inside your workflow and create new things. Most productivity hacks do not affect your output because you write with your mind, not the keyboard.

The best way to learn Emacs is one step at a time. Start with a simple configuration and then build on this starting point once you need further enhancements.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

Emacs is a malleable system, and everybody has personal preferences on how to undertake a task and configure Emacs. Any article on how to use Emacs is thus opinionated. If you have a different way of doing things, please complete the contact form to send me an email or contact me on social media.

The next article discusses getting started with Emacs and introduces the basic principles.

Getting Started with Emacs

Peter Prevos — Fri, 07 May 2021 00:00:00 +0000

Emacs has a reputation for being hard to use. This article crushes that myth and describes getting started with Emacs without any configuration. Once you understand the basic principles, you will soon play the software like a virtuoso plays the piano.

Emacs will take some getting used to because the first version of the software was released almost forty years ago. Although Emacs is one of the oldest continuously developed software packages, it has evolved into a versatile contemporary computing system. But, just like biological organisms have vestigial features that originated in the distant past, Emacs also has some quirky features and terminology that stem from its long history. This article guides you through the basics of using Emacs to get you started by explaining the principles.

This article is an abridged chapter of Emacs Writing Studio (EWS) — a configuration, website and book that explains how to use Emacs to undertake research and write and publish articles, books, and websites. Emacs Writing Studio is available as an e-book with expanded content from your favourite retailer.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Installing Emacs

The first point of call is to install Emacs on your computer. The Emacs computing environment works on almost all operating systems, but best integrates with GNU / Linux. You can install it on your Linux distribution with the relevant package manager. If you use another OS, follow the links below:

The EWS configuration is independent of your operating system.

This video by Marcus Birkenkrahe provides a succinct overview of installing Emacs on Windows or MacOS.

Emacs is also an interface to various other free software packages for such tasks as checking spelling, read PDF files or typesetting and formatting (e)books. The EWS GitHub repository contains a list of these programs and their function.

Getting Started with Emacs

Now that you have cleared the first hurdle, it is time to open Emacs and look around. When you first start Emacs, you see a splash screen with links to help files. Click on the link to read the tutorial, or press the q button to remove the screen and keep reading this article.

The Emacs splash screen (Emacs 29.1).

Pressing q is the standard method to close (kill) read-only screens in Emacs. When the splash screen closes, you enter the ’scratch buffer’, which you can use for temporary notes. Emacs does not save the content of the scratch buffer, so don’t start writing your dissertation just yet.

Emacs has a mouse-driven menu system, with drop-down menus on the top of the screen. Most users ignore this convenience and rely on keyboard-driven instructions. There is no real agreement on whether using a keyboard or a mouse is most efficient. What is clear is that regularly moving your hands between the keyboard and the mouse can be annoying and impede your workflow. When you misspell a word in most software, you move your hand from the keyboard to the mouse, click on the offending word and select the desired spelling. In Emacs, you use one keystroke and the misspelled word automatically changes to the most likely correct version, and you keep on writing.

One advantage of the menu system is that it helps discovering functionality in Emacs, but you don't need a mouse for this. Just press F10 and use the arrow keys to find what you need.

Emacs is mostly written in the Emacs Lisp (Elisp) language, which means that every action runs an Elisp function. Most functions are only for internal processes, but the user can call some functions (the commands) interactively. So every command is a function, but not every function is a command.

The default way to activate commands is by typing ALT-x and then the name of the command and the enter key. For example type ALT-x tetris ENTER to play tetris. Don't get too distracted, just press q three times to exit the game.

Typing the full function name every time is too much work for us Emacs users who seek ultimate efficiency. The minibuffer completion system helps you to find the command you seek. In standard Emacs, that would be ALT-x tet TAB ENTER. But that method is not on the efficiency frontier. To meet this demand, Emacs uses keyboard shortcuts.

Working with the keyboard

Emacs is a keyboard-driven system. You can use the mouse, but you;ll find that there is no need to remove your hand from the keyboard. The standard computer keyboard has five types of keys: alphanumeric / punctuation, editing, function, escape key, and modifier keys.

Alphanumeric and punctuation keys activate the self-insert function, which adds text to the computer’s memory and displays it on the screen. The editing keys, such as arrow keys, page up and down, delete, backspace, enter and tab, do what it says on their label. These are the only keys you need to writing prose, but we like to do more than insert text.

Function and multimedia keys perform specified tasks. For example, pressing F10 in Emacs show the menu bar. Multimedia keys activate predefined tasks, such as increasing the screen brightness or playing music.

The escape key is the most potent key on the board. Like Dorothy’s Ruby Slippers in the Wizard of Oz, this key gets you out of trouble by clicking it three times.

These keys cover about 100 options, but more are needed to manage the thousands of possible commands you might want your computer to perform. The modifier keys on a modern PC or Apple keyboard are shift, control, alt/option and windows/command. Some programmable keyboards might have additional modifier keys, such as Fn. Modifier keys have no effect when pressed by themselves. As the name suggests, these keys modify other keys when pressed simultaneously. In Emacs vocabulary, these are called chords.

Emacs documentation uses a special notation for key chords. Some of the Emacs terminology for modifier keys stems from a time when the current standard keyboard layout did not yet exist. What we now call the Alt key used to be the Meta key. What was the Super key in the olden days is now the Windows key on PC keyboards. The image below shows an old Space-Cadet keyboard with now non-existing modifier keys.

Symbolics LM-2 Lisp machine space-cadet keyboard.

Emacs documentation uses abbreviations, such as C-a, which stands for the Control and a keys, or M-a, which stands for Alt-a. Each modifier key has its own letter, as shown in the table below. The dash indicates that the keys are pressed while keeping the modifier key down. You can combine modifier keys, occasionally leading to awkward combinations, such as C-M-S a (Control, Alt and Shift a), which requires the nimble fingers of a sleight-of-hand artist to execute smoothly. The shift modifier is often not mentioned in Emacs notation, C-M A is the same as C-M-S a.

The most critical chord is C-g (keyboard-quit), which cancels a partially typed command or one still running. This command can quit running functions, unlike the triple escape key.

The Windows key on PC keyboards is mapped to the old Super key. Your operating system uses Chords with this key, so Emacs does not use this modifier by default. Lastly, the Hyper key is a leftover from the old keyboards.

Modifier	Example	Function
Shift	`S-8`	`*` sign on US keyboard
Control	`C-e`	End of line
Alt	`M-d`	Delete (kill) word
Windows	`s-`	Used by the operating system
Hyper	`H-`	Not mapped to regular keys

These modifier keys provide thousands of possible key chords to instruct Emacs. But wait, there is more. Emacs also uses prefix keys. When you press these, the system will wait for further input. For example, C-x C-f means that you first press Control and x and then Control and f, the default sequence for finding a file to open with the find-file function. The standard prefix keys are:

C-x: Mostly used for built-in Emacs commands
C-c: Mostly used by packages
C-h: Help functions
M-x: Execute commands

This means that the standard Common User Access keyboard shortcuts for cutting text (Ctrl+x) does not work in Emacs.

If you want to find out which command relates to a specific shortcut, then use C-h k and enter the key sequence to see what it is bound to for the mode you currently use. If you want to read the Emacs tutorial, the use C-h t.

Two more prefix keys need mentioning. Lots of commands have alternative states, which means that they can do things in different ways. You activate an alternative state by adding C-u before the regular key-chord.

Lastly, adding a number after Control or Alt repeats the next keystroke. For example M-80 * adds eighty asterisks to your text.

You can change how a keyboard behaves at three levels. Some high-end keyboards are programmable and let you define the output of each key. You could map the right control key as the Hyper key so you can use it in key chords. At the second level, your operating system interprets the input from the keyboard. In Windows, s-E opens the file explorer, but you can erase this binding so that it becomes available in Emacs. Each operating system has its own methods to change keyboard maps (keymaps). Some experienced Emacs users remap the caps lock key to act as the control key to make it easier to use.

Last but not least, you can define keys within Emacs itself. Some people don’t like the Emacs defaults because it requires frequent use of the ALT and Control keys. These people suggest that repetitive use of these keys causes a strain injury, the dreaded’ Emacs pinky’. Several packages, such as Evil Mode or Viper mode, exist within the Emacs ecosystem that changes the default keybindings to a different model. Emacs Writing Studio follows the standard conventions.

You might be dazzled by this detailed description of how Emacs uses the keyboard. Don’t worry about it for now. You will gradually understand its intricacies and eventually drive the system like a virtuoso.

Modes

Emacs is a versatile tool that accomplishes the different tasks it is capable of through the use of major and minor modes.

A major mode determines the main functionality for an open buffer. Each buffer has at least one major mode and each major mode has its own functionality and key bindings. The articles on this website mainly describe how to use Org mode.

A major mode is like opening an app within the Emacs environment. For example, Org mode provides a task management system and publication tools. Artist mode is a quirky tool in Emacs to create plain text drawings with the mouse and keyboard. Other common plain text writing major modes are plain text, Markdown and Fountain. All major modes provide specialise functions for working with these text formats. All major modes share the same underlying Emacs functionality, such as copying and pasting (killing and yanking).

Minor modes provide additional functionality, such as spell-checking, text completion or displaying line numbers. A minor mode is an auxiliary program that enhances the functionality of a major mode. While each buffer has only one major mode, a buffer can have one or more active minor modes. A minor mode can also apply to the whole Emacs session.

Each mode defines how a buffer displays text. For example, each heading level in Org mode has a different colour (depending on your theme). The different colours and fonts help finding your way around the text. However, how the text looks on the screen differs from how it will look when you print it or export it to a PDF of another format. This is a What you See is What You Mean approach, which this article explains in more detail.

Emacs automatically selects the relevant major mode using the extension of the file and displays it in the mode line below each buffer. Minor modes have to be enabled explicitly on top of a major mode.

The available keyboard shortcuts depend on the major and minor modes that are active at the time. Each mode can have its own keymap. Some keymaps are global and apply to the whole of Emacs, other maps are specific to a mode. Some shortcuts remain the same for all modes (such as M-u, which converts a word to uppercase), unless a mode overrides this binding. Packages can change or add shortcuts, depending on the required functionality. So a shortcut like C c c-c is used by different modes for different actions, depending on the context it is used.

If you are lost and like to see all possible keyboard shortcuts for the active buffer (screen), call C-h b to view a list of all available shortcuts. Note that some shortcuts can relate to a specific mode and will thus only work when the mode is active.

Frames and Windows

When you open Emacs, the software runs within a frame. In most operating systems, a frame is called a window. To confuse matters further, you can divide an Emacs frame into windows. You can also open multiple frames on a desktop, for example, one on each monitor.

By default, a frame has one window. You can duplicate the current window horizontally or vertically by pressing C-x 2 or C-x 3 (split-window-below and split-window-right). The C-x 0 shortcut (delete-window) removes your current window, and C-x 1 removes all other windows (delete-other-windows), so you work in the full frame again.

When splitting a window, the same buffer can appear on both sides, each with point at a different location in the text. Using M-x follow-mode flows the text so it becomes like a two-column document. This function toggles the follow function as a minor mode, so you can switch it on and off with the same function.

To move between windows, use the C-x o shortcut (other-window).

The bottom of each window has a mode line, and the bottom of the frame contains the mini buffer. The mode line provides information, such as the open file name, current line number and other helpful information. The mini buffer is where Emacs provides feedback or seeks your input.

Emacs frame with three windows.

Buffers

A buffer holds the text that is displayed in a window. Buffers can also contain a user interface or output from functions. The name of the buffer or its associated file is shown in the status bar at the bottom of the window. Special buffers, such as *Messages* start with an asterisk. This buffer shows the output of any commands you run.

You can have multiple buffers open at the same time so that you can easily switch between buffers. Emacs is highly stable and some Emacs users have hundreds of open buffers as they rarely need to restart the program. The C-x b shortcut (switch-to-buffer) lets you select another buffer to move to.

With the C-x left and C-x right key sequences (previous-buffer and next-buffer), you can move between buffers in chronological activation order.

Files

Most buffers, except those surrounded by an asterisk, are linked to a file. Like office software, you are working on the version in memory (the buffer), and the previous version is on disk (the file). Emacs works precisely the same way.

Opening files in Emacs is called visiting a file. You first need to find it with the find-file function (C-x C-f) to visit a file. Emacs will open the file and display the contents in the buffer. Emacs will create a new file when you type a name that does not yet exist. If you open a directory, Emacs will show the contents of that directory in the file manager.

Emacs will ask you to nominate a file or folder in the mini buffer. When you hit the TAB button twice, all the available files and folders appear in the mini buffer. To complete the file name, start typing the filename and hit TAB again.

After you complete your edits, C-x C-s (save-buffer) saves your buffer. To save all open buffers, press C-x s (save-some-buffers) and follow the prompts in the minibuffer at the bottom of the screen. To save a buffer under a new name, you can use C-x C-w.

Keystroke	Function	Description
`C-x C-f`	`find-file`	Find (open) a file
`C-x C-s`	`save-buffer`	Save the current buffer to its file
`C-x s`	`save-some-buffers`	Save all open buffers (with confirmation)
`C-x C-w`	`write-file`	Write current buffer to a file (Save as)

Exiting Emacs

After your first practice round, it is time to close (kill) Emacs. The C-x C-c kills the Emacs session, checking for unsaved buffers first (save-buffers-kill-terminal).

Learn more

This article only discussed the basic principles of using Emacs. If you like to know more, then I encourage you to follow the built-in Emacs tutorial, which you activate with C-h t.

There are many great YouTube videos that explain how to use Emacs. The video below by David Williamson provides a detailed introduction into how to use Emacs. The blog posts and YouTube videos by Protesilaos Stavrou also provide useful insights.

The Absolute Beginner's Guide to Emacs (System Crafters).

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

Emacs is a malleable system, so everybody will have their personal preferences. Any article on how to use Emacs is thus opinionated. If you have a different way of doing things, please complete the contact form to send me an email or contact me on social media.

You can do a lot with Emacs without any configuration, but you will need to configure the system to make it behave how you want it to. The next article introduces the principles of configuring Emacs to convert it into your own writing studio.

Configure Emacs Writing Studio

Peter Prevos — Fri, 07 May 2021 00:00:00 +0000

Commercial software is usually good at doing one thing really well. You write documents in LibreOffice Writer or Google Docs. You create presentation slides in PowerPoint or Keynote and manage tasks with Trello or Todoist. If you are lucky, the developers let you change the configuration to customise the software’s behaviour and optimise your workflow. In most cases, however, you are bound by the choices the developers have made for you. Linking the information that is held in these programs can be problematic. Emacs does not have such limitations. You do almost all tasks in one program, but you have to configure it to make it behave the way you want.

This article explains how to configure Emacs to transform it into a personalised productivity suite for authors. Please note that this configuration assumes that you are using the latest version of Emacs, which at the time of writing is 29.4.

This article is part of Emacs Writing Studio, a book that explains how to use Emacs to undertake research and write and publish articles, books, and websites. Emacs Writing Studio is also available as an e-book from your favourite retailer.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Configuring Emacs

A vanilla version of Emacs has a lot of functionality out of the box and you could easily write a book without any configuration. But to truly leverage its capabilities, Emacs needs external packages and additional configuration.

Some Emacs users use pre-configured systems, such as Doom Emacs, or Spacemacs, or one of the other starter kits. While these configurations are helpful, they sometimes provide ‘everything but the kitchen sink’.

On the other side of the spectrum, you can configure your system from scratch, which can become a productivity sink as you wade your way through highly technical instructions written by experts.

Emacs Writing Studio (EWS) seeks the golden mean by providing a minimal configuration. The main objective of EWS is to get you started writing in Emacs and generate sufficient curiosity for you to further develop your skills and keep improving your configuration.

The Initialisation File

When you start Emacs, it loads the initialisation file, or init file in short. This file contains Lisp code that configures Emacs and loads additional packages.

The first time you start Emacs, it will create the configuration directory. The init file lives in this location, which will also contain the packages and other files needed to personalise your system.

Emacs looks for a file called .emacs, .emacs.el or init.el. The dot in front of the file means that it is hidden from view to prevent cluttering your directories. The location of the configuration folder depends on your operating system and Emacs version. The Emacs help functionality can tell you where to store the init.el file.

Type C-h v (describe-variable), and type user-emacs-directory and Enter (or user-e TAB) in the minibuffer at the bottom of the screen. The help buffer that now appears provides the correct folder name, for example "Its value is "~/.config/emacs/".

You can now visit (open or create) the init.el file in your Emacs directory and copy and write your configuration. Your init file will become active after you restart Emacs (use M-x restart-emacs).

You can download the complete EWS configuration file from GitHub and save it in the user directory. Copy the init.el and ews.el files to your user directory. The first file contains the Emacs configuration and the second file contains bespoke EWS functions.

Please note that when you start Emacs with the Emacs Writing Studio configuration for the first time, Emacs will download several packages from their online repositories and compile them. This process might generate scary-looking text, but you can ignore it, as it only appears when you install new packages.

Emacs Lisp

While using commercial software is like renting a fully furnished house, using Emacs is more like buying a new empty house. Your new digital home needs some paint and new carpets, which means you need to configure Emacs to make it your digital home.

Commercial software provides graphical menus to define how the software operates. Like the examples below, you might tick a box, select an item in a list, or enter a value in a text box to configure the program to your wishes.

Inhibit startup message

Initial scratch message

Cursor Type

Typical graphical configuration screen.

Being a plain text program, Emacs doesn't use a graphical interface but a programming language called Lisp. The code below is equivalent to the graphical user interface shown above.

  (setq-default inhibit-startup-message t
                initial-scratch-message "Hello world"
                cursor-type 'bar)

Although the text-based method is superficially more complex than ticking boxes, writing text and selecting a drop-down list, it is far more potent than a graphical interface. Writing code to configure your computer may not seem user-friendly initially because we are used to the graphical interface. However, once you learn how to write simple Emacs Lisp programs, you will find that Emacs is the most user-friendly system because it gives you so much power over your computer. But with that power comes great responsibility and a steep learning curve.

Using Emacs Lisp is the epitome of ease of use. You control how your computer behaves instead of the computer controlling your behaviour.

A Lisp program consists of expressions, which are statements enclosed in parentheses. Each expression starts with the name of a function (setq-default), followed by some parameters.

The setq-default function is commonly used to set the default value of a parameter. For example, (setq-default inhibit-startup-message t) has the same effect as putting a checkmark in the inhibit-startup-message box, while inhibit-startup-message nil is the same as removing the checkmark in that box. The expression in this example determines whether Emacs displays a startup message when you open it for the first time.

You don’t have to learn to program Lisp to configure Emacs. The examples on this site provide sensible defaults to turn vanilla Emacs into your personal writing studio. Each article offers a thoroughly documented configuration. Many Emacs users share their configuration with the community so that you can copy their code. The copy-and-paste method is a shortcut to learning more about Emacs Lisp because you can change the settings and examine the results of your changes.

The Emacs Writing Studio Configuration

Checking external software requirements

Emacs is not just a text editor but a Lisp computing environment that can interface with command line tools. For Emacs Wring Studio to function fully, it needs to have access to a bunch of external programs, which are listed in the relevant articles.

A bespoke Emacs Writing Studio function (ews-missing-executables) tests wether the relevant external executable files are available. This function displays an error message into the *Messages* buffer if it cannot find one or more of the external tools. Emacs will still work, but some functionality might be unavailable.

Each article in the Emacs Writing Studio series explains which external software is required an how to obtain it. The core functionality of Emacs will work fine without these, but some specialised tasks require assistance from this software.

soffice (LibreOffice): View and create office documents
zip: Unpack ePub documents
pdftotext (poppler-utils): Convert PDF to text
djvu (DjVuLibre): View DjVu files
curl: Reading RSS feeds
dot (GraphViz): Create note network diagrams
Convert image files
- convert (ImageMagick)
- gm (GraphicsMagick)
latex (TexLive, MacTex or MikTeX): Preview LaTex and export Org to PDF
- divpng: Part of LaTeX
hunspell: Spellcheck. Also requires a hunspell dictionary
Search inside files
- grep
- ripgrep: Faster alternative for grep
View PDF files
- gs (GhostScript): View PDF files
- mutool (MuPDF): View PDF files
Play music
- mpg321, ogg123 (vorbis-tools), mplayer, mpv, vlc

Loading Packages

The Emacs base system has extensive functionality, but you can extend its capability with external packages. Many people write packages in Emacs Lisp to improve or enhance what the system can do. You can also download Lisp files directly from GitHub or other sources. Developers of these packages often share their work through public package repositories, which are websites that let you easily download and install packages. The two most important ones are:

ELPA: GNU Emacs Lisp Package Archive — the official package archive, enabled by default.
MELPA: Milkypostman’s Emacs Lisp Package Archive — Unofficial archive.

The next blob of Elisp (Emacs Lisp) code adds the MELPA archive to the packages archive list. The ELPA archive is installed by default. This code appends the configuration (add-to-list) and does not override the default. Just ensure you add a trailing slash (at the end of the URL), otherwise, Emacs gets confused when downloading the package index.

  (use-package package
    :config
    (add-to-list 'package-archives
                 '("melpa" . "https://melpa.org/packages/"))
    (package-initialize))

You can explore the list of packages with the list-packages function. This function has no keyboard shortcut, so first press M-x and type the function name, followed by the Enter key.

Configuring Packages

John Wiggle has developed the use-package software, which simplifies configuring packages. Computer scientists call this approach ‘syntactic sugar’ because it makes the syntax of a configuration file a bit easier to read and write. Most configurations on this website use this method to install and configure Emacs extensions.

  (use-package use-package
    :custom
    (use-package-always-ensure t)
    (package-native-compile t)
    (warning-minimum-level :error))

This is the basic setup required to run any other configurations in this series of articles.

Keeping your packages up-to-date

Packages are constantly updated by their developers. To ensure you get the latest version, you can use the package-upgrade-all function, which has no default keybinding. This might seem a strange name for this function, upgrade-all-packages would seem a bit more linguistically sound. However, the convention for naming Emacs Lisp functions is that the first word is the package name, which is package. So, for example, all Emacs Writing Studio functions start with ews-.

Changing keyboard shortcuts

The previous article explained how Emacs uses the keyboard and the use of shortcuts to run functions. Emacs comes with a lot of default key bindings, all of which you can change or you can add new ones for functions you need to use regularly that don't have a default shortcut.

For example, adding this line to your configuration lets you play Tetris whenever you hit F8. So this line associates the keyboard shortcut (kbd "<f8>") in the global keymap with the tetris function.

(keymap-set global-map "<f8>" #'tetris)

EWS deploys the use-package configuration, which has a slightly different convention. You can as such change any of the keyboard shortcuts in this EWS configuration to your preference by simply changing the bit between quotation marks. Please note that function keys and other special keys are between <> to prevent them being interpreted as individual keys.

  (use-package emacs
    :bind ("<f8>" . tetris))

Emacs Writing Studio configuration

Besides crafting your own configuration or using a starter kit, Emacs has a customisation menu that lets you configure Emacs without writing code. This system writes the required Emacs Lisp code to the init file.

Emacs Wring Studio uses two separate init files to prevent Emacs from changing your hand-crafted config. The configuration below changes the automated configuration file’s name to custom.el and evaluates it when it exists.

  (setq-default custom-file
                (expand-file-name "custom.el" user-emacs-directory))
  (when (file-exists-p custom-file)
    (load custom-file))

The first lines of code instruct Emacs to use a separate file for the internal configuration system (custom.el). The expand-file-name expression adds the filename to the location of the Emacs user directory. This method ensures the code works irrespective of the Emacs user directory location.

The last two lines load this file when it exists (file-exists-p). As this is the first action in the configuration, your personal init file will override any settings in the custom file.

Emacs Writing Studio variables

For convenience, the Emacs Writing Studio defines some variables to configure the system.

You can change these, and any other variable in Emacs, variables with the customize-variable function, search for ews, and pick the variable you seek to change. Alternatively, use customize-group and search for ews to see all variables. You can then change the entry in the text field to your preference and hit the Apply and save button. Emacs saves the new value(s) to the custom.el file and they will be loaded next time you start Emacs.

Emacs customisation screen.

The Emacs customisation functionality is a low-code option to configure your system and the Emacs Writing Studio is configured so that the variables set with this method will override the default configuration presented in these pages. Alternatively you can of course hack your init file directly.

Emacs Writing Studio also includes a handful of bespoke convenience functions, which are discussed in their relevant pages. All bespoke configuration is stored in the ews.el file on GitHub.

Is Emacs a Productivity Sink?

Configuring Emacs can be a daunting task and become a productivity sink because of the ease by which it can be done. Within the Emacs community, a discussion emerges every now and then about the need to make so many changes to the vanilla software to get it to behave how you like it.

A commonly asked question is why can't Emacs have 'sensible default' values? This may seem like a reasonable question, but what is sensible to one user is an annoyance for another. Emacs developers are as such quite conservative when it comes to changing the default settings.

The Emacs developers cannot cater to every personal preference, so they provide a skeleton system you must enhance to suit your workflow. Emacs Lisp is like a box of Lego. You can play with it out of the box using vanilla Emacs, but it is much more fun when you create your own toys. Working on your Emacs configuration is like building with Lego and eventually you configure Emacs to perfectly meet your needs.

Emacs configuration and Lego.

It can be tempting to constantly fine-tune your configuration. Wielding the power to create a bespoke Emacs system is a temptation that can lead to fake productivity, which is one of the three forms of procrastination:

Nihilistic procrastination: Watching TV or playing computer games.
Sophisticated procrastination: Fake productivity, e.g. Emacs hacking, switching productivity tools, taking notes for volume instead of quality.
Productive procrastination: Daydreaming.

Tinkering with your Emacs configuration is not as bad as nihilistic procrastination but can become a form of fake productivity. The productivity gains from fine-tuning your Emacs to cut out a few keystrokes can take more time than you will save with your new workflow. Writing is more than the number of words you can type into your buffer. Writing is as much a contemplative act as it is about productivity. Don’t get sucked into trying as many packages as possible. It will lead to what Emacs aficionados call Dot Emacs bankruptcy, which occurs when your init file is so enormous and disorganised that it becomes unmanageable.

The best way to learn how to configure your Emacs system is to search the web for examples and copy them to your init file. Initially, the code might look gibberish, but its logic will become more apparent as you do this more often. Emacs has an extensive help system that you can use to explore functions or variables you like to know more about. Use C-h o (describe-symbol) to search for any function or variable.

Source: xbd.com

Hopefully, the Emacs Writing Studio configuration will help you reduce the time it takes to configure Emacs to suit your preferred workflow.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The next article explains the principles of writing prose in Emacs.

Writing Prose with Emacs

Peter Prevos — Sun, 10 Dec 2023 00:00:00 +0000

Emacs is known as an extensible text editor, but this is a confusing description to authors. Computer programmers refer to writing code as editing, while for an author, the term editing refers to the last stage in the writing process. A text editor is a tool to write text for computers to read. However, Emacs can be a tool for editing code and writing for humans.

This article explains the basics of how to write prose using Emacs as an advanced typewriter. This article is part of Emacs Writing Studio, which explains how to use Emacs to undertake research and write and publish articles, books, and websites.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Writing modes

Emacs is like a Swiss Army knife in that it can undertake several different functions through so-called major modes.

An Emacs major mode controls how a buffer (the currently open file) behaves. A text file could be a movie script, a to-do list, the next great novel, computer code, or however you express your creativity. In commercial software, each file could be a different file type that you can only read in specialised software. Emacs automatically recognises the type of file you open and modifies its behaviour accordingly.

In Emacs, text mode is for simple text files. Fountain mode helps you write movie or theatre scripts. Org mode excels at managing your to-do lists and creative writing. Many other modes with specialised functions are available in the package repositories.

Each buffer can only have one major mode operating at a time, but it can have several minor modes. A minor mode provides optional extras, such as Flyspell, for on-the-fly spell checking.

Org mode is arguably the most versatile major mode in Emacs. A large portion of these articles describe how to use this package.

The remainder of this article focuses on the principles that apply to all text modes.

Basic principles

The foundational major mode for writing prose is text-mode. Emacs documentation describes it as the mode for writing text for humans to read, in contrast to prog-mode, which is for writing code that computers read. Text mode forms the foundation for all other prose formats. This means that all these modes use the same basic functionality for writing.

The basic principle of writing in Emacs is easy: you just start. However, some differences exist between standard word processors and the Emacs way. Once you get used to the convenience of writing in Emacs, you never want to use other software again, as you will miss the extensive editing functionality to jump around your text with ease.

Moving around in a buffer

You can move the cursor with arrow keys and other standard navigation keys. Emacs documentation sometimes refers to the cursor as a point; technically, the cursor is the character displayed on the screen (a line or a box), and a point indicates where the next typed character will appear. In most situations, the terms cursor and point are used interchangeably.

In addition to the standard methods to move around a buffer, Emacs provides additional functionality to help you navigate your project. For example, C-p (previous-line) does the same as the up key (see table below). Some people prefer these keys so their hands can stay in the default position for fast touch-typing. However, writing is more about thinking than maximising keystrokes per minute, but feel free to try them out.

Direction	Function	Key
Left	`left-char`	`C-b`, `<left>`
Right	`right-char`	`C-f`, `<right>`
Up	`previous-line`	`C-p`, `<up>`
Down	`next-line`	`C-n`, `<down>`
Previous word	`backwar-word`	`M-b`, `C-<left>`
Next word	`forward-word`	`M-f`, `C-<right>`
Scroll down	`scroll-down-command`	`C-v`, `<PageDown`
Scroll up	`scroll-up-command`	`M-v`, `<PageUp>`
Start of line	`move-beginning-of-line`	`C-a`, `<home>`
End of line	`move-end-of-line`	`C-e`, `<end>`
Start of buffer	`beginning-of-buffer`	`M-<`, `C-<home>`
End of buffer	`end-of-buffer`	`M->`, `C-<end>`

Moving around a buffer in Emacs.

Making a mark

You will undoubtedly have the experience of moving from one part of a document to another and then like to jump back but lose your place, so you search through the document to get back to where you left off.

In Emacs, you can save the cursor's current position with C-space, which marks the current location of the point. You can now move to another part of your document, edit or read what you need, and jump back with C-u C-space.

Focusing

Getting lost in a sea of words on your screen is easy. Some simple keystrokes can help you to focus your eyes quickly. Keying C-l (recenter-top-bottom) moves the line that your cursor is on to the centre of the screen.

If you repeat this keystroke, the cursor will move to the top of the screen. If you do this three times in a row, the cursor moves to the bottom of the screen.

Copying and pasting text the Emacs way

The most apparent difference between Emacs and typical word processors is that it does not use the standard Common User Access (CUA) keys standard for selecting, copying, cutting and pasting text used in modern applications. But the trade-off for the minor inconvenience of learning a new method to edit text is that the Emacs way is much more versatile than other software.

Selecting text

With the shift and arrow keys, you can select text in almost all major modes. Because the shift and arrow keys in Org mode are reserved for editing lists, you must use a more advanced way to select a text block. You first set a mark with C-space and then move to the end of the section to select your text, which is highlighted.

To select a complete paragraph, use the M-h key. Repeatedly pressing M-h will select subsequent sections. Using C-x h will select all text in the buffer.

Once the text is marked, you can act on this block by deleting, copying or moving it.

Copy and paste text

In modern computing language, copying and pasting are handicraft analogues for moving text from one place to another. Emacs terminology is more evocative. Copying a text is the same as saving it to the 'kill-ring' and yanking a text retrieves it from that seemingly bleak location.

While the clipboard in most systems only retains the last entry, the kill ring provides access to a history of killing actions. In other words, Emacs stores a history of all text you copy and cut from a buffer to the kill ring. The length of this history is stored in kill-ring-max, which is 60 entries by default. Once the kill ring is full, the oldest item vanishes. Emacs erases the content of the kill ring when you exit the program. Use the following keyboard shortcuts to copy and move the text to and from the kill ring:

Shortcut	Function	Description
`M-w`	`kill-ring-save`	Copy a selection to the kill ring
`C-W`	`kill-region`	Move a selection to the kill ring
`C-y`	`yank`	Paste the most recent entry in the kill ring to the buffer
`M-y`	`yank-pop`	Replace the previously yanked text with the next kill ring entry

Copying and pasting in Emacs.

The kill* commands copy or move text to the kill ring and the system clipboard. The yank* commands copy an entry from the kill ring to the current buffer. The yank-pop (M-y) command cycles through the contents of the kill ring so you can access the history. When you use advanced completion systems, you see a list of all available previous selections in the minibuffer. Use the arrow keys or search to pick what you like to yank into the active buffer.

If you prefer using the familiar keyboard shortcuts, enable cua-mode for convenience. With this mode, you can use C-z, C-x, C-c, and C-v to undo, cut, copy, and paste in addition to the standard Emacs bindings. The C-x and C-c keys only activate when you mark a region. You can enable this mode by adding (cua-mode t) to your init file. This mode is not enabled in the Emacs Writing Studio configuration.

Search and replace

The most common search method in Emacs is incremental. An incremental search (C-s) begins searching as soon as you type the first character of the search string. As you type in the search term, Emacs shows you where the term is found. When you have typed enough characters to identify the place you want, you can terminate the search explicitly with ENTER or C-g.

The C-s shortcut searches incrementally from the cursor. You cycle through the search results by repeatedly pressing C-s. Using C-r searches the text before the cursor.

Emacs saves search terms in the search ring. Typing C-s C-s will show the previous search term. Using C-p and C-n lets you scroll through previous search terms.

To search and replace text in a buffer, use M % and follow the prompts in the mini buffer.

Correcting mistakes

The most convenient aspect of writing on an electric screen is that it is easy to change your mind or correct a mistake without having to resort to correction fluids or other archaic methods. The following basic editing commands are available to modify text. Please note that commands that start with kill store the deleted text on the kill ring.

Action	Function	Key
Delete character after point	`delete-char`	`C-d`, `<delete>`
Delete character before point	`delete-backward-char`	`<backspace>`
Delete the next word	`kill-word`	`M-d`, `C-<delete>`
Delete to the end of the line	`kill-line`	`C-k`

Emacs deletion commands.

Besides removing unwanted characters, you can also swap them with a series of transposing commands. When you accidentally reverse two letters in a word, you can switch their order with the transpose-char command with the cursor between them (C-t). Swapping words is quickly done with the transpose-words (M-t) command.

Undo the Emacs way

The Emacs undo command is mapped to C-/. If you need to undo the step, use C-? (undo-redo). Each consecutive command will undo earlier changes. Any command other than an undo command breaks the sequence.

Although each editing command usually makes a separate entry in the undo records, simple commands (single key press) may be grouped together.

The Emacs undo system recovers any past text version. In contrast, the standard undo/redo system can lose past edits as soon as you redo. Getting lost in a sequence of undo and redo commands is easy. The Undo Tree package by Toby Cubitt helps you keep track of your changes by visualising them as a tree. Use the ~ undo-tree-visualise ~ (C-x u) command to walk through previous versions of your text.

Example of an undo tree.

The current buffer will change as you work your way through its history with the arrow keys where x marks the spot of the selected step. Use q to select the chosen edit and continue writing. The C-q chord jumps out of the undo tree without making changes.

  (use-package undo-tree
    :config
    (global-undo-tree-mode)
    :custom
    (undo-tree-auto-save-history nil))

There is also a nuclear option. You can discard all the changes since the buffer was last visited or saved with M-x revert-buffer, which reloads the file from the disk.

Configure text modes

The original purpose of Emacs is to edit computer code, so by default, the editor does not wrap lines at the end of the screen. While this behaviour is great for developing software, it isn't very pleasant for writing prose. The visual-line-mode minor mode enables line-wrapping, which is common in word processors. You can run the function separately for each buffer, but ideally, you link (hook) it to all text modes.

The second change is that when you select (mark) a section of text and start typing, Emacs will not replace the selected text by default. To enable Emacs to overwrite selected text, you set the delete-selection-mode variable, as below.

Another sensible default for authors is to ensure that Emacs lets you use the page-up and page-down buttons to go all the way to the first and last line in the buffer.

The last line better integrates the system clipboard with the Emacs kill ring.

  ;; Sensible line-breaking
  (add-hook 'text-mode-hook 'visual-line-mode)

  ;; Overwrite selected text
  (delete-selection-mode t)

  ;; Scroll to the first and last line of the buffer
  (setq-default scroll-error-top-bottom t)

  ;; Copy the system clipboard to the kill ring
  (setq save-interprogram-paste-before-kill t)

Checking spelling

The Emacs minor mode Flyspell provides an interface to the Hunspell spell-checking software. To enable spell-checking, you thus first need to ensure that Hunspell is available on your computer. Hunspell is easy to install on Linux computers but is also available for Windows and Chromebook.

The configuration below enables Flyspell for all text modes and sets Hunspell as the default checking program. The M-F7 key checks the spelling for all words in the buffer, and F7 checks the word on which the cursor is standing.

A more productive method is to use the flyspell-auto-correct-previous-word (C-;) function. This function provides the most likely correction of the first spelling error before the cursor. Emacs will show the list of possible corrections in the mini buffer. Repeatedly pressing C-; will cycle through the options until you return to the original. This function prevents you from having to jump to your spelling mistakes.

  ;; Spell-checking
  ;; Requires Hunspell
  (use-package flyspell
    :custom
    (ispell-program-name "hunspell")
    (ispell-default-dictionary "en_AU")
    :hook (text-mode . flyspell-mode)
    :bind (("M-<f7>" . flyspell-buffer)))

  (use-package flyspell-correct
    :after (flyspell)
    :bind (("C-;" . flyspell-auto-correct-previous-word)
           ("<f7>" . flyspell-correct-wrapper)))

You must modify the configuration to set your preferred dictionary as the default. The available English dictionaries in Hunspell are:

en_US (American)
en_CA (Canadian)
en_GB (British)
en_AU (Australian)

Hunspell supports a lot of other dictionaries, so you can install and choose your own when English is not your preferred language.

If you are multilingual, you can set a different language for each buffer by adding a file variable to the relevant buffer. Ass the text below as the last lines in your Org mode file, where you replace nederlands (Dutch) with your preferred language:

,# Local Variables:
,# ispell-local-dictionary: nederlands
,# End:

Emacs evaluates this line and activates the dictionary when you run M-x normal-mode and the next time you open the file. This dictionary only applies to the relevant file; all other buffers remain in the default language.

Counting words

To count the words in a buffer, or part thereof, use the count-word-region-function, which is bound to M-=.

The result appears in the minibuffer. When you select a portion of text, this function will count only the selection. Adding the C-u, prefix counts the words for the hole buffer (C-u M-=).

For Org mode buffers, the ews-org-count-words command counts the words in each section and adds the word count as a property. Refer to Distration-Free Writing with Org mode for details.

Text Completion

Emacs developers have also written many completion systems to help you write text. These packages fall into two categories: minibuffer completion and expansion systems for prose and code.

Expanding text in human languages comes in two forms. You either start typing a word, and Emacs predicts what should go next, or you write a predefined abbreviation that expands into a word, a paragraph, a table or whatever else you define it as.

Predictive text is one of these technologies that can be both a blessing and a curse. Use it cautiously because it can hinder your writing when used too eagerly.

Company mode is a versatile package that can help you with completing long words. Its primary purpose is to assist developers with writing code, but it can also help you complete words.

Company mode provides suggested completions in a dropdown menu whenever you type a defined number of characters. It derives these options from all open text buffers. Use the arrow keys and enter to select your option, or ignore the menu and keep typing.

This configuration only activates when entering at least four characters and only shows words of eight or more letters. The wraparound option lets you cycle through the completions menu with the down and up arrow keys.

  ;; Auto completion
  (use-package company
    :custom
    (company-minimum-prefix-length 4)
    (company-dabbrev-minimum-length 8)
    (company-selection-wrap-around t)
    :init
    (global-company-mode))

When using variable pitch fonts, the dropdown menu for text completion gets distorted because the letters are not all the same size. The company-posframe package by Tumashu solves this problem. A post frame is a floating child frame that can pop up anywhere on the screen, usually to show additional information.

  ;; Required for variable pitch
  (use-package company-posframe
    :config
    (company-posframe-mode 1))

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The next article dives into getting productive with Emacs by explaining how to improve the Emacs user interface.

Improving the Emacs User Interface

Peter Prevos — Fri, 25 Aug 2023 00:00:00 +0000

A minimalist interface means that your Emacs becomes a place of rest and contemplation away from the cacophony of contemporary software, filled with buttons and functionality you don't need. Emacs is an ugly duckling that you can configure into a beautiful swan. This article further explains how to improve the Emacs user interface. These settings are obviously opinionated, so this article is, more than anything else, an example of how to configure Emacs to your personal preferences.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Improving the Emacs user interface

The default Emacs user interface is a hybrid between keyboard and mouse-driven operations. While using the mouse might seem convenient, a keyboard-driven system is more efficient because you don't have to move your hand that much and switch contexts when finding icons.

Emacs Writing Studio is a keyboard-centric configuration and removed any distractions from the Emacs. The following code snippet modifies the Emacs interface to remove these distractions. The first line inhibits the splash or startup screen. If you want to see it again for nostalgia, use C-h C-a (about-emacs).

Emacs has the paternalistic habbit of asking for confirmation with either y / n or with yes / no, depending on the situation. The basic philosophy is that typing the whole word prevents accidentally killing unsaved buffers. While this functionality is a nice gesture, it can be annoying and confusing. The last line sets all yes / no confirmations to the one-letter abbreviation. The use-short-answers variable defines this behaviour.

The following three lines remove the tool, menu and scroll bars because we don't need these in a keyboard-driven system. These lines don't set a variable but disable the various minor modes that control these graphical elements.

This configuration also removes the menu bar. While the menus are a useful tool to discover functionality, there is no need to constantly keep it on the screen. You can access it by pressing F10 (menu-bar-open) and selecting menu options with the arrow keys and ENTER. You exit the menu with C-g (keyboard-quit). But after using Emacs for a while, you'll quickly build muscle memory and revert to keyboard shortcuts.

If you like to see the effect of each line of code, place your cursor at the end of each line and press C-x C-e to evaluate the expression (eval-last-sexp). If the cursor is inside the parenthesis, then use C-M-x (eval-defun). To evaluate all code in a buffer, run M-x eval-buffer.

  ;; LOOK AND FEEL

  ;; Keyboard-centric user interface
  (setq-default inhibit-startup-message t
                use-short-answers t)
  (tool-bar-mode -1)
  (menu-bar-mode -1)
  (scroll-bar-mode -1)

Themes

A theme is a set of instructions describing the colours of headings and other text parts. Colours in a text editor play a different role than in a word processor. Colours in Emacs are semantic, meaning they indicate the type of text, not how it looks when published. A heading might have a different colour than the text or metadata, which helps you find your way through the document.

Emacs does this through font-lock-mode. Font locking assigns faces to (or 'fontifies' in Emacs speak) various parts of your text using logical rules. Typing M-x font-lock-mode toggles between your buffer's themed and plain versions. Try this to see the effect of themes.

Two basic classes of themes exist for text editors: light and dark. Light backgrounds, common with most modern word processing software, can cause asthenopia (eye strain) after you stare at the screen for a while. Many text editor users, therefore, prefer dark themes. Light themes are, however, useful when you work in a brightly lit room.

The code snippet below installs and activates the most recent version of Prot's Modus themes. The Modus themes each have four versions. The modus-operandi theme is the primary light theme, while modus-vivendi is its dark counterpart.

The primary Modus themes maximise contrast between background and foreground following the Web Content Accessibility Guidelines (WCAG). The Modus themes comply with the triple-A standard of the WCAG, which specifies contrast ratio between background and foreground of 7:1. This high contrast ratio is legible for people with moderately low vision. Prot has also optimised versions for red-green and blue-yellow colour blindness (deuteranopia and tritanopia).

Emacs Writing Studio uses the tinted versions as default. These versions have a slightly lower contrast ratio than the default and are as such suitable for people with normal vision. The Modus themes do not prescribe any keyboard shortcuts, so I have defined some for EWS. The C-c w m shortcut toggles between the light and dark side (insert Star Wars pun here). Using C-c w M provides a selection menu of all Modus themes. Emacs Writing Studio uses C-c w as its default keybinding for its specific functionality, where the w is a mnemonic for writing and m/M for Modus themes.

  ;; Modus Themes
  (use-package modus-themes
    :custom
    (modus-themes-italic-constructs t)
    (modus-themes-bold-constructs t)
    (modus-themes-mixed-fonts t)
    (modus-themes-headings '((1 . (1.5))
                             (2 . (1.3))
                             (t . (1.1))))
    (modus-themes-to-toggle
     '(modus-operandi-tinted modus-vivendi-tinted))
    :bind
    (("C-c w m" . modus-themes-toggle)
     ("C-c w M" . modus-themes-select))
    :init
    (load-theme 'modus-operandi-tinted :no-confirm))

Let's unpack what this piece of code means. The use-package declaration has three sections.

The :custom section does the same as the setq-default function described previously. This configuration adds more variety to the theme, such as italic text, bold text, mixed fonts, and headings are rendered a bit larger than the normal text. This configuration uses the *-tinted versions of the Modus themes (medium contrast).

The :bind section binds the relevant keys to the function that toggles between light and dark Modus themes.

Lastly, the :init section defines the functions run when the package is loaded. In this case, we load the tinted version of the dark Modus theme without confirmation.

If you like your configuration to default to the high-contrast versions of the theme or one of the two colour blindness-safe versions, then customise the modus-themes-to-toggle variable to override this default and change the init version to the one you like Emacs to start with.

Other Themes

Several themes are part of Vanilla Emacs, which you can customise with the customize-themes function. When you click the Save Theme Settings function, the theme will be loaded next time you start Emacs. You will have to remove the Modus themes code to ensure that your other choice is loaded. When the theme is used for the first time, Emacs might ask you to confirm that it can run Elisp code.

Emacs users have developed a ragtag collection of themes. You can browse the Emacs Themes Gallery to pick your favourite. If the theme is available in the ELPA or MELPA package repositories, change the package and theme name in the code below to match the name of your favourite theme.

  (use-package dracula-theme
    :init
    (load-theme 'dracula))

Setting the fonts

The default font in Emacs is a fixed-pitch (mono-spaced) font designed for writing code. In a fixed-pitch font, all characters have the same width. An i or an m will use the same amount of space, just like the traditional typewriters. This type of letter is ideal when writing code because it helps to align the text. Mono-spaced fonts are also necessary to create fantastic ASCII art, such as the dinosaur shown in another article.

A variable-pitch font is easier on the eye when writing prose. Not all characters have the same width in a variable-pitch font, as is common in natural writing. Ideally, we want the best of both worlds and configure Emacs to use the best font category for the best situation. For example, Emacs can define a different font for certain parts of the text, for individual buffers, or for a major mode.

The configuration below uses Alex Branham's mixed-pitch package, which assigns fixed and variable pitch fonts to the relevant elements in the buffer. For example, tables in Org mode become fixed pitch so that all columns are the same size. The use-package call hooks the mixed-pitch mode to all text modes, including Org mode.

  ;; Set default, fixed and variable pitch fonts
  (use-package mixed-pitch
    :hook
    (text-mode . mixed-pitch-mode))

The Emacs Writing Studio configuration does not specify any particular fonts and use your system's defaults. You can configure your favourite fonts, provided they are available on your computer. You need to define three font variable:

default: The default face settings when nothing else is specified (a fixed-pitch font).
fixed-pitch: The font used for computer code .
variable-pitch: The settings for prose.

In Emacs, a 'face' is a collection of attributes for displaying text. It defines the font, foreground colour, background colour, optional underlining, etc. Various face attributes are available for configuration. The ones we use here are:

font: The name of the font
height: The font height as an integer in units of 1/10 point.

The settings of the last two categories override the default. The nil in the set-face-attribute function sets the parameters for all Emacs frames.

You can also use the customisation menu by evoking customize-face and select default, fixed-pich and variable-pitch and enter the font name in the Font Family box. Click Apply and Save for each font. This action saves the font settings to the custom.el file, which Emacs evaluates at the end of the startup sequence. See Configure Emacs for more information about the initialisation process.

Please note that anything you configure here will override the theme settings, so only customise font family and size.

To see which fonts are available, you can set the fonts in a graphical window, run menu-set-font. When you use the GUI to set the default font, the change is immediate but transient. Use menu-bar-options-save to save your default font and size to the custom.el file.

Alternatively, add the following three lines, with your fonts of choice to your init.el file.

  (set-face-attribute 'default nil :font "DejaVu Sans Mono" :height 130)
  (set-face-attribute 'fixed-pitch nil :font "DejaVu Sans Mono")
  (set-face-attribute 'variable-pitch nil :font "DejaVu Sans")

This video by Prot explains the principles of setting fonts in Emacs beautifully.

#+caption:Emacs: customize fonts (generic method and 'fontaine' package).

Tooltips

Emacs supports tooltips when the mouse hovers over when there is a pause in mouse movement over parts of text in a window, the mode line, or some other part of the Emacs frame such as a tool bar button or menu item.

The layout and typography of these tooltips is in most cases controlled by the window manager. To disable this behaviour, customise the variable use-system-tooltips to nil.

With this option you can finetune the appearance of the tooltip by customising `tooltip-frame-parameters` and change the font with for example: (set-face-attribute 'tooltip nil :font "Monospace-16").

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio (EWS) book. this book explains everything you need to know to research, write and publish articles and books with Emacs in a no-code environment.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The next article dives into enhancing the minibuffer completion system to make it easier to find files, buffers and other information.

Enhancing Minibuffer Completion

Peter Prevos — Sat, 03 Jul 2021 00:00:00 +0000

Even with the advent of speech-to-text software, the keyboard is still the most common method to convert thoughts to text. While computers might one day even read our minds, there is something to be said about using your fingers to do the talking. Who would want their’ ums’ and’ ahs’ written down, or their uncensored stream of consciousness committed to text? Writing is as much about thinking and crafting a stream of words as it is about maximising keystrokes per minute.

Completion systems are like predictive text on a mobile phone. You start typing some characters, and the computer lets you complete your choice. Emacs has an extendable completion system that helps you complete long words, find files, remember function names and other menial tasks. This article explains the basic minibuffer completion engine and introduces some packages extending this functionality.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Emacs Completion Systems

Emacs has three types of completion systems:

Minibuffer completion assists with picking choices in the minibuffer, such as function names, files and other selections.
Keychord completion: Systems to help with keyboard shortcuts.
Text completion helps you complete words you type in the buffer (see Writing Prose with Emacs).

Minibuffer completion

The minibuffer is where you find files, call functions and other vital aspects. The minibuffer completion system aims to make it easier to find what you need by providing a search mechanism that provides a list of possible options.

The standard minibuffer Emacs completion system focuses on entering functions, filenames, buffer names and any other selection process in the minibuffer. When typing a partial function or file name, you can hit the TAB key. Emacs will display completion candidates in the minibuffer.

For example, to evaluate the visual-line-mode function to change how Emacs wraps paragraphs, you type M-x visu and TAB. If you hit TAB after each letter, you’ll notice that Emacs narrows the completion candidates as you get closer to your desired selection.

This principle also works with variable names and filenames. As you start looking for a filename, use the TAB key to prevent having to type the whole filename. The TAB key is your secret weapon to help you remember and discover functions, variables, file names, buffer names and other selection candidates.

The minibuffer completion system is highly configurable, and several packages extend this functionality. The approach in this configuration uses a stack of connected packages that provide a seamless experience. This approach showcases the extensibility of Emacs as a modular computing environment.

Vertico

The Vertico package enhances minibuffer completion. This extension uses incremental search, meaning the list of completion candidates is shortened to match your entry as soon as you type a letter.

When, for example, opening a file with C-c C-f, you can start typing any part of the filename to locate the file you seek. Use C-backspace keys to move to a higher directory.

The savehist package remembers your selections and saves your minibuffer history when exiting Emacs. This package ensures that your most popular choices remain on top for further convenience.

  ;; MINIBUFFER COMPLETION

  ;; Enable vertico
  (use-package vertico
    :init
    (vertico-mode)
    :custom
    (vertico-sort-function 'vertico-sort-history-alpha))

  ;; Persist history over Emacs restarts.
  (use-package savehist
    :init
    (savehist-mode))

Orderless

To further enhance our ability to find completion candidates, the orderless package matches patterns, irrespective of the order they are typed in.

For example, typing use pack TAB provides the same results as pack use TAB. The basic configuration sets some variables for integration with the Vertico completion package.

  (use-package orderless
    :custom
    (completion-styles '(orderless basic))
    (completion-category-defaults nil)
    (completion-category-overrides
     '((file (styles partial-completion)))))

Marginalia

Emacs is a self-documenting computing environment, which means that every function and variable includes a text describing what it does.

The marginala package displays these texts next to your completion candidates. When you now type M-x, you will see a list of functions and a brief description of what they do, making it even easier to find the function you need.

  ;; Enable richer annotations using the Marginalia package
  (use-package marginalia
    :init
    (marginalia-mode))

Minibuffer completion with Vertico, Orderless and Marginalia.

This video by Emacs guru Prot explains minibuffer completion in great detail.

Emacs: modern minibuffer packages (vertico, consult etc.).

Help with keyboard shortcuts

Completion shortens the amount of text you must type and is terrific for discovering new functionality you did not yet realise existed. However, we usually don’t type function names but use keyboard shortcuts.

The Marginalia package shows the relevant keyboard shortcut when displaying the completion candidates to help you discover these shortcuts.

Remembering which keyboard shortcut you need in your next action takes some effort. The which-key package is not so much a completion system but a great help when trying to remember which keyboard shortcut to use. This package provides a minor mode that displays the key bindings following the currently-entered incomplete command (a prefix) in a popup.

Many keyboard shortcuts have multiple parts, such as C-x C-f. The which-key package shows a popup menu that lists all the available options. When, for example, you press C-x, the menu will list all follow-up keys and the function they are bound to. Where it says prefix in the popup, this means that there is a deeper level.

If the number of shortcuts is too large to fit in the popup window, then you can move to the next page with C-h n and the previous page with C-h p. Just typing C-h inside the Which-Key popup displays some additional options to navigate the list of key bindings.

The configuration below displays the popup at the bottom of the screen after you type the partial keyboard shortcut.

  ;; Improve keyboard shortcut discoverability
  (use-package which-key
    :config
    (which-key-mode)
    (which-key-setup-side-window-right))

Which-Key popup window for C-x-w (Emacs Writing Studio).

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The next article discusses the Emacs Writing Studio workflow.

Emacs Workflow for Authors

Peter Prevos — Tue, 27 Dec 2022 00:00:00 +0000

Emacs is the Swiss Army Chainsaw of productivity tools. This venerable software tool can help you undertake almost every task in the writing process, from idea generation to writing and publishing. Experienced users say they "live inside Emacs" as virtually all computing activities can be done in this software. This article describes a possible Emacs workflow for writers.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

The Emacs Workflow

The process of writing can be chaotic as it involves many iterative cycles. But an orderly pattern emerges when we step back from our daily tasks. We read literature, develop new ideas, work on these ideas and publish the results. Even though reality is never as linear as this list suggests, it is a helpful guide to organise an Emacs configuration.

The Emacs Writing Studio configuration follows a five-step workflow:

Inspiration: Reading, listening and watching.
Ideation: Cultivating ideas (taking notes).
Production: Writing and editing.
Publication: Export Org mode to PDF, ePub or a website.
Administration: Manage your projects.
Communication: Tell others about your work and exchange ideas.

The basic principle of this workflow is that the author collects information from literature, the web, email and so on (inspiration), which they process in a note-taking system (ideation). These notes are the central repository of information and can be linked to a bibliography. These notes form the foundation of the writing process (production). When the work is finally completed, the author publishes it in its final format (publication). Elated with the final result, the author communicates her new work through email or social media (communication). At the end of a long work day, we must also do some Administration to keep our systems in good shape, which is the bonus step.

The image below summarises the packages that Emacs Writing Studio uses to convert ideas into reality.

Emacs Writing Studio workflow.

Inspiration

Ideas don't pop into your mind out of thin air. Our thoughts, plans and inspirations depend on what we read, hear or watch.

Emacs has extensive facilities to read any plain text format imaginable. Emacs can also display PDF files, ebooks, and images out of the box. Listening to a podcast or watching a video is impossible within an Emacs frame. Still, the system does integrate with external multimedia applications.

You can also maintain a bibliography in Emacs with BibTeX files and link these to electronic files for reading inside Emacs or with another program. The Citar package by Bruce D'Arcus provides a fast and efficient interface to your bibliography.

The Emacs internet browser is called eww (Emacs Web Wowser). This is not a fully-featured browser but a tool that displays the website you like to read to a plain text file. While this approach creates readable websites without distracting crud, it cannot render more complex internet functionality.

Elfeed is an extensible web feed reader for Emacs, supporting both Atom and RSS feeds. You can use this package to follow your favourite websites. If you like to keep up to date on Emacs articles on this website, then use this feed.

Emacs can also interface with multimedia players and, as such, be configured to become a complete music and video platform.

The following articles on this website discuss how to configure Emacs to read literature or listen to music or podcasts:

Ingesting so many new ideas is only worthwhile if you keep a record of your new-found inspirations. Hence, maintaining research notes is essential to facilitate the ideation process.

Emacs Writing Studio: Inspiration (Read e-books, manage a bibliography, surf the web and play music).

Ideation

My personal ideation process starts with a physical notebook. This might be a strange thing to admit as somebody who professes admiration for the electronic virtues of Emacs. Physical notebooks have some advantages in the creative process. Firstly, you can use it everywhere. The only exception might be the shower, which is paradoxically where we get our best ideas. Using a notebook is also a slower process than using a keyboard. Writing with a pen forces you to think a bit more deeply about what you are writing. Apparently, writing on paper triggers different neural pathways than writing electronically.

My physical notes on the Emacs workflow.

Once ideas have germinated in my notebook, some of these will find a place in my electronic system inside Emacs. As part of my weekly action list, I review my physical notes and transfer the ideas that require archiving to my digital system.

Emacs is an ideal tool for storing notes in plain text. Several packages are available to manage your digital brain. The Emacs Writing Studio revolves around the Denote package by Protesilaos (Prot) Stavrou.

I don't follow any specific, such as Zettelklasten or Bullet Journal, to manage my notes. My collection of notes is a primordial soup of my ideas, categorised using tags and opportunistically linked. Besides extracts from my physical notebooks, Denote collects information I find online, extracts from my email inbox, social media or journal articles and books. Everything worth keeping goes to my Denote collection.

Most note-taking systems contain five types of notes:

Fleeting notes: Quickly take a note while doing something else.
Permanent notes: Contains new ideas, project files and anything else worth keeping.
Bibliographic notes: Notes about a book or article linked to the bibliography.
Journals: Daily or weekly reflections.
Attachments: Read-only files, such as images, PDFs, videos and anything else that supports the content of the written notes.

Fleeting notes

Fleeting notes are collected in the central inbox. The Org mode capture mechanism provides a distraction-free tool to collect and store random ideas for later review.

A fleeting note can be an idea, an action item, a URL to review later or whatever else comes my way that has yet to require my attention.

Regularly reviewing your fleeting notes will turn them into actions to do later or convert them into permanent notes.

Permanent notes

The permanent notes form your second brain. Using the Denote package, your collection of notes is structured using a tagging system, and they link together to connect your ideas and form a network of thoughts.

Denote can help you implement formal systems, such as Zettelkasten, or lets you use a more organic approach.

Bibliographic notes

Literature is the lifeblood of the researchers. Every new thought finds its origin in the ideas of others. Therefore, managing a bibliography is an essential skill for a researcher. Many commercial and free tools are available to help you manage your literature. These tools store a literature list and provide access to notes and attachments.

Bibliographic notes reference one or more bibliographic items, such as a book or an article. A bibliographic note contains a summary or interpretation of the literature you read. For that purpose, I have written the Citar-Denote package, which provides functionality to write bibliographic notes. This package enables a many-to-many relationship between notes and bibliographic items. This means you can add multiple notes per bibliographic item or one note for more than one piece of literature.

Journals

A journal is your personal diary where you can reflect on the past day or week. The Denote package can help you build a journal, whether you like to document your travels or jot down personal reflections.

Emacs Writing Studio: Ideation (Taking fleeting and permanent notes).

Production

Once you have gathered your thoughts, it is time to start writing. Emacs developers have published a lot of utilities to assist with the writing process, such as auto-completion, checking spelling and grammar, dictionaries, thesaurus and other tools indispensable to a writer.

Emacs Org mode is ideal for writing articles and books or developing websites. Another popular plain text format for writing is Markdown, which does not have the same flexibility as Org mode. Fountain mode is another plain text format designed to write scripts for theatre, film or a YouTube video.

The Writing proce with Emacs article discusses how to write and edit text in Emacs. The Distraction-Free Writing article discusses how to mange writing projects with Org mode.

Publication

The glorious moment has arrived when you can publish the fruits of your labour. Emacs Org mode has powerful capabilities to export the text to various formats, most importantly PDF for physical books, ePub for ebooks and HTML for websites. The article on Publishing Articles and Books with Org Mode Export discusses creating ebooks in detail.

PDF format

Org mode exports files to PDF through the LaTeX document preparation system popular with technical authors and publishers. Emacs will export your Org files to LaTeX format, which then exports your file to a beautifully designed document. Many journals accept LaTeX submissions, so you can also export your Org mode to a .tex or PDF file to send to the editor.

ePub format

Org mode has built-in capabilities to export a file to HTML. The ox-epub package by Mark Meyer leverages this functionality to produce a book in epub format.

Word Processor

The Org mode export functionality has built-in capabilities to export to the standard open document format, which is compatible with all commercial word processors.

Websites

Org mode can not only export to individual HTML files, but it can also publish them to a website. I use the Hugo software to convert my collection or Org mode files into my websites. Hugo provides extensive capabilities to use templates for your websites, including shortcodes to dynamically insert HTML in your pages beyond the current capabilities of Org Mode.

The article Create Websites with org Mode and Hugo discusses how to create a website, such as the one you are reading now.

Communication

Emacs can also connect you to the world to communicate with other writers and your readers.

The software has built-in capabilities to read and write emails and several external packages, such as mu4e and notmuch are available to make sense of your email. Using Emacs for email is a very efficient way to grind through your daily dose of electronic mail. Other packages let you connect to IRC, Reddit, Mastodon and other communication protocols.

Emacs Writing Studio does not contain any configuration for communication packages.

Administration

Working through a writing project is a fantastic journey in creative expression, but there is also some overhead in managing your projects.

Emacs interfaces with other GNU software to help you manage your files using the powerful directory editor (dired). Managing your files is a breeze with this package. You can also use Emacs to manage your photograph collection with the built-in image-dired package.

Lastly, working on a big project means tracking many tasks. Org mode has a fully functional task management system to help you keep track of your projects. You can implement your personal workflow or use a Getting Things Done (GTD) approach.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The next article discusses how to use Emacs to read ebooks and other electronic documents.

Reading eBooks with Emacs

Peter Prevos — Sat, 02 Sep 2023 00:00:00 +0000

The first step in the Emacs Writing Studio workflow is ingestion (reading, listening and watching) new content. Reading ebooks with Emacs is straightforward as it provides functionality to read several popular types of electronic documents, such as PDF and ePub. The software can also browse the web with an unobtrusive plain text browser.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Emacs Writing Studio video demonstration of reading e-books, managing a bibliography, reading websites and listening to music.

Reading Documents

Emacs is excels at working with plain text in almost every imaginable format. However, it needs assistance from external software to display other document types, such as PDF or office formats. The diagram below shows the various external software packages necessary to render pages from the supported file formats. When you start the Emacs Writing Studio configuration, you will receive a warning if any of the required packages is unavailable on your system.

Document conversion in Doc-View.

ePub files

An ePub file (Electronic Publication) is a widely used open standard format for digital books, magazines, and other written content. The content can adapt to different screen sizes, making it ideal for e-readers, tablets, and other devices.

An ePub file is a website in a box. The content and the layout are stored in a ZIP file. You can see the raw content of an ePub file with an archiving utility. You need to install the unzip package to view the content.

The nov package by Vasilij Schneidermann lets you read ePub books inside Emacs. The code after the :init line associates files with the .epub extension to this package.

  ;; Read ePub files
  (use-package nov
    :init
    (add-to-list 'auto-mode-alist '("\\.epub\\'" . nov-mode)))

You can now open ebooks like any other Emacs file. Use the arrow and page-up / page-down keys to navigate each chapter. Several keyboard shortcuts are available to navigate the book:

t: go Table of contents
n and p or [ and ]: Next or previous chapter
q: Quit the ebook reader
?: Help file with list of other keyboard shortcuts

To increase or decrease the text, use the C-x C-+ and C-c C-- shortcuts. To reset the length of the lines, press g to re-render the document.

The ebooks are opened as a read-only file. You can copy and paste text into your bibliographic notes. Opening the file as an archive with the a key shows the document's structure. From here, you can also copy images from the book with the C keyboard shortcut. For more information about using files in Emacs, read manage files with Emacs.

PDF files

The Portable Document Format (PDF) is a versatile format developed by Adobe in the early 1990s. This format presents documents consistently, regardless of the software, hardware, or operating system used to view them. This format is codified in an international standard and has become the de facto standard for electronic literature.

The layout of PDF documents is fixed, in contrast with other electronic formats, which can adjust to the screen size. A PDF file is usually designed for printing and, as such, follows a traditional physical layout and typography.

Emacs can display PDF files out of the box with the DocView major mode. However, you will need some external software for DocView to work. GhostScript or MUPDF convert PDF files to images and provides them to DocView. This approach works because Emacs has built-in functionality to display and manipulate images.

This conversion means that DocView can be sluggish with large documents and will warn you. The large-file-warning-threshold variable sets the threshold for these warnings, which is 10 Mb by default. This value is a bit low for modern systems and in Emacs Writing Studio, this threshold is 50 Mb.

Use the arrow and page-up / page-down keys to turn pages. Several keyboard shortcuts are available to navigate the file:

P: Zoom to the full page
W: Fit document width to window
H: Fit height to window
+ / -: Zoom in and out
M-g g: Jump to page
M-< / M->: Jump to first page or last page
k: Kill (close) the file
?: Help file

To enable searching in PDF and other supported document formats, you must have the pdftotext software installed, which is part of the poppler package, a PDF utility named after a Futurama episode.

You can search within the document with doc-view-search (bound to C-s), which starts a search, creates a list of all matching pages and shows how many pages that contain the search query were found. After completing the search, you can jump to the next page containing a match with an additional C-s. DocView does not highlight the searched term, but pressing C-t (doc-view-show-tooltip) shows the search results for this page in a tooltip.

Poppler also allows you to view a PDF file as a plain text file with the C-c t shortcut (doc-view-open-text). This option makes it easier to search and copy relevant text. Return to the graphical view of the text by pressing C-c C-c twice.

We need some configuration to optimise how DocView displays pages. This code enhances the resolution of the page images to 300 DPI.

If you use Emacs 29.1 or higher and have the MuPDF software available, then DocView will convert PDF pages to SVG files instead of PNG files, which provides a crisper image and better for zooming in closely.

Lastly, this configuration also moves the threshold for warning against opening large files to 50Mb ($50 \times 2^{20}$ bytes).

  ;; Doc-View
  (use-package doc-view
    :custom
    (doc-view-resolution 300)
    (doc-view-mupdf-use-svg t)
    (large-file-warning-threshold (* 50 (expt 2 20))))

This article on the Emacs Notes website also provides a detailed explanation of how to configure and use DocView and enhances the dropdown menu functionality.

DjVu Books

The DjVu format is designed primarily to store scanned documents. DocView can read DjVu files when DjVuLibre is available on your system.

Office Suite Documents

The built-in DocView package can read Microsoft Office and LibreOffice files (text documents and presentations), but you first need to install the LibreOffice software.

When opening an office document, Emacs invokes LibreOffice to convert the file to a PDF, from where the abovementioned functionality is available.

There is currently a confirmed bug in Org mode (version 9.6.6) that overrides the associations between LibreOffice and Doc View mode. The code below is a workaround to reinstate the desired behaviour and associates the various file extensions with Doc View. The bug is slotted to be resolved in version 9.7.

  ;; Reading LibreOffice files
  (use-package ox-odt
    :ensure nil
    ;; Correct settings 
    :config
    (add-to-list 'auto-mode-alist '("\\.\\(?:OD[CFIGPST]\\|od[cfigpst]\\)\\'" . doc-view-mode-maybe)))

Reading websites

Emacs has a built-in web browser called eww (The Emacs Web Wowser). This package provides a simple, no-frills experience focusing on readability by displaying a website as a plain text file. It can display images but does not render any CSS for formatting or JavaScript.

A wowser is somebody with strong moral views for temperance and abstinence. So perhaps the plain text approach to browsing is a moral stance on the current World Wide Web and its security and privacy issues.

You can open a URL or search the web with the command M-x eww. If the input doesn't look like a URL or domain name, Emacs searches the web with the DuckDuckGo search engine.

Use the arrow and page-up / page-down keys to navigate the page. Several keyboard shortcuts are available to navigate the webpage:

<, >: Beginning and end of the page
R: Readable format (only display the main text)
M-I: Toggle images
H: Browsing history
&: Open the page in the external browser
G: New search or website
q: Quit the window
?: Help file with list of other keyboard shortcuts

When opening a link to a website from inside Emacs, it will open the default browser on your operating system instead of eww. If you prefer to use eww as the default, add this to your configuration file: (setq-default browse-url-browser-function 'eww-browse-url).

Many blogs and podcasts use RSS feeds to notify their readers of new content. The Elfeed package lets you follow your favourite web content using these feeds.

Reading eBooks with Emacs

This chapter focused on how to read documents in non-text formats. Reading these formats requires external software, in sumarry, to read all these formats you need to install:

Ghostscript (gs) or MuPDF (mutool): Reading PDF files
Poppler (pdftotext): Converting PDF files to text
LibreOffice (soffice): Reading and exporting to office documents
unzip: Reading ePub ebooks
DjVuLibre (ddjvu): Reading DjVu ebooks

The Emacs Writing Studio configuration issues a warning when any of these packages is unavailable. Emacs will work normally, but some functionality might be unavailable.

Emacs can also publish documents in most of these formats, starting with an Org mode file and export this to the desired format. Emacs Writing Studio discusses:

Publishing Articles and Books with Org Mode Export (PDF, office formats and ePub)
Create websites with Emacs and Hugo

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The next article discusses how to read RSS feeds to follow your favourite blogs and podcasts.

Read RSS and Atom feeds with Emacs and Elfeed

Peter Prevos — Thu, 17 Dec 2020 00:00:00 +0000

Finding interesting content on the internet can be like sifting your way through a piles of garbage to discover something valuable. Social media can be fun and engaging, but the cacophony of irrelevant content driven by dark algorithms is disheartening. RSS and Atom let you subscribe to the websites and blogs you enjoy, without having to subscribe via email. A feed is an XML file on a web server with recent articles, either the complete text or an excerpt. This article describes reading RSS and Atom files with Org mode and the Elfeed package.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

RSS and Atom Feeds

RSS is an elegant mechanism to consume content because you only get to see those blogs or podcasts that you subscribe to. When you use RSS, no algorithm decides what you can and cannot see. Subscribing to RSS feeds is anonymous, so you will not be spammed with email funnels trying to sell you stuff or services. Some websites have multiple feeds, so readers can subscribe to specific topics.

Atom feeds are a newer feed format that clarifies some of the ambiguities in RSS. Both feeds are a form of XML, and you need an aggregator to display the content. Unfortunately, RSS and Atom feeds have lost importance due to the dominance of social media. However, the technology is still alive and used in most websites. Browsers no longer point them out, and websites rarely prominently link to them like they used to, but the feeds still exist.

Some websites that offer feeds use the RSS logo. When this is not the case, you can still find the feed. Over 40% of websites use WordPress. In these websites, you can find the feed by adding feed to the end of the URL. If all else fails, you can find the feed by looking at the page source by right-clicking on the page. Search for rss-xml, copy the URL in the href specification and Bob's your uncle. For example, the RSS feed for Emacs articles on this website is:

https://lucidmanager.org/tags/emacs/index.xml

Elfeed

The Elfeed Emacs package manages and reads your favourite RSS feeds. You can list your favourite feeds and categorise them. The Elfeed browser helps you to navigate through your collection of unread articles or podcasts.

The custom section sets the location of the downloaded content to your Emacs configuration folder instead of your home folder. You can remove this line or use a different directory. The second part in the custom section instructs Elfeed to open posts in a separate window. The last line sets the keyboard shortcut to start Elfeed to C-c w e.

  ;; Configure Elfeed
  (use-package elfeed
    :custom
    (elfeed-db-directory
     (expand-file-name "elfeed" user-emacs-directory))
     (elfeed-show-entry-switch 'display-buffer)
    :bind
    ("C-c w e" . elfeed))

Elfeed screendump.

You must install the cURL program, which stands for 'Client for URLs'. This program assists with downloading files from the internet. If cURL is unavailable, then Elfeed uses the slower built-in Emacs method to extract data. which only works on Unix-based systems.

Adding RSS Feeds

The basic configuration for Elfeed includes a list of RSS feeds by setting the elfeed-feeds list variable. You can easily access this variable with the customise-variable function.

There is, however, a more convenient way to manage your collection of RSS feeds. The elfeed-org package lets you configure your list of favourite websites in an Org mode file.

The package reads the nominated Org mode file(s) and collects all internet addresses or links in the headers with the :elfeed: tag. You can also attach other tags to any link in a headline to organise your feed by topic.

The example below shows how you can structure your Elfeed Org Mode file. Note that a tag applies to all headings at a lower level, so the :elfeed: tag also applies to the Emacs and news headings. To get started, you can copy and paste this example into an Org File. This sample file is also available on the EWS GitHub repository.

  #+title: Elfeed configuration
  
  * Blogs                                             :elfeed:
  ** Emacs                                            :emacs:
  Anything Emacs related.

  *** https://lucidmanager.org/tags/emacs/index.xml
  *** http://www.reddit.com/r/emacs/.rss
  *** https://www.youtube.com/feeds/videos.xml?channel_id=UCEqYjPJdmEcUVfHmQwJVM9A

  ** News                                             :news:
  *** [[https://www.abc.net.au/news/feed/2942460/rss.xml][ABC Australia]]

You can either use the plain URL or an Org mode hyperlink. A hyperlink in Org mode consists of a nested set of square brackets [[link][description]]. In Org mode, the link looks like a properly formatted hyperlink. You can insert a link with the org-insert-ink function (C-c C-l) and follow the prompts in the minibuffer. You can select between many link types but need not worry about these when adding an internet link, just copy the URL and paste it in with C-y.

The org-webtools package helps inserts fully formatted hyperlinks into Org mode. The org-web-tools-insert-link-for-url function (C-c w w) constructs an Org mode link from a web address in the kill-ring (when you copy a link) and extracts the link title from the website.

The only configuration you need for elfeed-org is to set the name of the Org file(s) you like to use to store your feed links. To add or remove a feed, edit this file and update the feed database with M-x elfeed-update. You can also add text comments, as Elfeed only reads headings.

In Emacs Writing Studio, the location of the Elfeed configuration is stored in the ews-elfeed-config-file variable, defined in a previous article. You can customise this variable to meet your needs or simple replace it with a string, e.g. "~/Documents/elfeed.org".

  ;; Configure Elfeed with org mode
  (use-package elfeed-org
    :config
    (elfeed-org)
    :custom
    (rmh-elfeed-org-files (list ews-elfeed-config-file)))

  ;; Easy insertion of weblinks
  (use-package org-web-tools
    :bind
    (("C-c w w" . org-web-tools-insert-link-for-url)))

When Emacs Writing Studio is first loaded, a special function creates an Elfeed note with the Denote package when such a note does not yet exist. You can then find the Elfeed Denote file and start entering your configuration.

If you don't use the full Emacs Writing Studio configuration, then remote the :after denote line and change the expression after rmh-elfeed-org-files to another name, for example, (list "~/Documents/elfeed.org"). Please note that this variable has to be a list with the filename(s) as strings.

Using Elfeed

You are now ready to read your RSS feeds. The first time you use Elfeed, use the elfeed-update command to establish the database of feed content.

Press C-c w e to start the Elfeed browser, which shows a list with the date and title of each entry, the feed's name and any tags. When you hit enter, Elfeed displays the webpage or a summary with a hyperlink to the web version in another window. You can use the following keystrokes to manage your feed:

G: Fetch feed updates from the servers
s: Update the search filter
c: Clear the search filter
r Mark the entry as read
u: Mark the entry as unread
g: Refresh view of the feed listing (remove unread items)
b: Open the article in the system browser
q: Quit Elfeed

All new entries are tagged as unread by default. The other tags are derived from your list of RSS feeds.

When you remove a feed from your list, all articles that you previously downloaded will remain in the database and will show on your list until they are marked as read.

Elfeed has a powerful search filter that you can use to filter by tag, feed name and dates. The filter can process Regex queries. The Elfeed manual describes the functionality in more detail than is warranted in this article.

Emacs Writing Studio video demonstration of reading e-books, managing a bibliography, reading websites and listening to music.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The next article in this series describes how to listen to music with the Emacs Multimedia System.

Configure EMMS: Emacs MultiMedia System on GNU/Linux

Peter Prevos — Thu, 20 Aug 2020 00:00:00 +0000

Music is a great tool to boost your productivity. Playing the transcendental music of J.S. Bach or Sepultura's screeching heavy metal soothes the soul while writing. Scientists have studied the psychological effect of background music and found that it can improve episodic memory, intelligence, and verbal and visual processing speeds.

Emacs might be known as a text editor, but you can also play background music while you write, or listen to podcasts and videos in your information ingestion process. Emacs has several packages, such as Bongo and EMMS, to help you access your collection of sound files. You might already have a great music player on your computer, but not having to switch applications to play music helps to retain your focus.

EMMS, the Emacs Multimedia System, is a comprehensive music and video player for Emacs. It contains an intuitive browser that can show album covers and metadata. This article explains how to configure EMMS to convert your Emacs system into your personal jukebox.

This article is part of Emacs Writing Studio, a book that explains how to use Emacs to undertake research and write and publish articles, books, and websites. Emacs Writing Studio is available as an e-book from your favourite retailer.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Basic Installation

EMMS does not work out of the box when you install it via GNU ELPA. You will need some configuration and a music player. The minimum configuration for your init file consists of these lines:

  ;; Emacs Multimedia System
  (use-package emms
    :config
    (require 'emms-setup)
    (require 'emms-mpris)
    (emms-all)
    (emms-default-players)
    (emms-mpris-enable)
    :custom
    (emms-browser-covers #'emms-browser-cache-thumbnail-async)
    :bind
    (("C-c w m b" . emms-browser)
     ("C-c w m e" . emms)
     ("C-c w m p" . emms-play-playlist )
     ("<XF86AudioPrev>" . emms-previous)
     ("<XF86AudioNext>" . emms-next)
     ("<XF86AudioPlay>" . emms-pause)))

This configuration initiates EMMS with all its functionality and provides access to default music players. This setup is, however, not enough to fully enjoy your music. EMMS needs access to a music player to create sound and access to software to convert cover images to thumbnails.

To play sounds with Emacs, you must install one of the compatible sound players (ogg123, mpg321, mplayer, mpv or VLC). EMMS uses the convert program of the ImageMagick, a software suite to edit and manipulate digital images. When you run the complete Emacs Writing Studio configuration, the system will check whether one of the players and ImageMagick are available.

The last step to start playing music is to cache the music files. Evaluate the emms-add-directory-tree command to scan your collection. EMMS creates a cache in your Emacs configuration directory.

EMMS can read metadata in music files for Ogg Vorbis, Opus, FLAC and MP3 files. This process might take a while, depending on the size of your collection. You can start playing music when EMMS: All track information loaded shows in the minibuffer.

Play music with EMMS

Phew, we can finally start enjoying the music collection. The basic principle is that you move tracks to the playlist buffer.

The browser is the most convenient way to select the music of your liking. You start the browser with emms-browser (C-c w m b). Loading the browser for the first time in an Emacs session might take a while if you have an extensive music collection.

Screenshot of the EMMS browser.

When in the EMMS browser, use the b key followed by a number to browse by artist, album, genre, year, composer or performer. The browser is collapsed by default, showing only album covers, artist names, or whatever you select as the browsing category. The browser is hierarchical. When, for example, browsing by genre, the hierarchy has four levels Genre > Artist > Album > Track. Use the 1 to collapse and the 2-4 keys to expand the categories at levels 1 to 3.

As is the case in most Emacs applications, press ? for a list of available keystrokes, such as:

r: Jump to a random entry
s: Search by album, artist, title, performer or composer
Enter: Add selection to playlist
C-j: Add selection to playlist and play
W: Lookup entry on Wikipedia

Being an Emacs buffer, standard search commands will also work for visible entries.

When the cursor is on a category, such as an album name or a composer, it will add all tracks belonging to that category. So when the cursor is on an album it will add all tracks. When the cursor is on a single track, it will only add that track to the playlist.

You jump to the playlist with emms (C-c w m e), from where you can manage what you play. Press ? for a list of keyboard shortcuts, some of which are:

n / p: Next or previous track
r: Play a random track.
D: Remove selected track from playlist
C: Clear playlist

Once you have curated a funky playlist, keep it for future reference. You can save a playlist buffer in m3u or pls format for use in other multimedia players using the save-buffer command (C-x s). The emms-play-playlist function (C-c w m p) lets you load and play a saved playlist.

The configuration above also sets the multimedia buttons on your keyboard (play, pause, next, previous). The MPRIS (Media Player Remote Interfacing Specification) extension ensures that these buttons also work when you are playing music with EMMS but are not in Emacs.

EMMS has many advanced features available for advanced control of your sound collection. Some other useful EMMS functions are:

emms-play-directory: Add directory to playlist
emms-play-find: Plays all files in the music directory that match a given search criterion.

This is only the start of using EMMS. The Emacs Multimedia System has a plethora of options you can use to fine-tune your listening experience. Read the EMMS manual with the info browser (C-h i) for detailed information about the various options.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The ingestion process of the Emacs Writing Studio workflow is now complete. The next article in this series starts the ideation process and describes how to take notes with Org mode.

Taking Notes With the Emacs Denote Package

Peter Prevos — Mon, 28 Aug 2023 00:00:00 +0000

Over the years I have tried a cornucopia of digital applications to take notes. Emacs has several note-taking packages that extend basic functionality to help you manage your solidified thoughts. This article explains how to use Org mode and the Denote package by Protesilaos Stavrou.

Denote excels in its simplicity and the ease by which it can be extended. The most exciting aspect of Denote is that it can help you manage any file type, not only notes in Org mode. I use Denote extensively to manage my Org mode and other types of notes, PDF documents, photographs and videos and anything else. Another advantage is that Denote does not necessarily rely on Emacs, you can search and view your notes with any software that can process plain text files.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Emacs Writing Studio: Ideation (Taking fleeting and permanent notes).

Creating Notes

Note-taking seems to have gained popularity in recent years judging by the flood of blog pages and videos that tell you how to best take notes. Whether Luhmann's Zettelkasten or Carroll's Bullet Journal, they all profess to solve all your personal knowledge management challenges. In my personal experience with paper and digital journals for over thirty years, the structure of your notes is not that important. The sincerity and originality of the thoughts contained within your notes has primacy over following some rigid system. My methodology is a primordial soup of thoughts from which my writings emerge. Even Luhmann himself saw his Zettelkasten as a septic tank for ideas. However, we can apply some logic to our workflow.

Not all notes are created equal and the relevant workflow depends on their purpose. Broadly speaking, we can distinguish between six types of notes:

Fleeting notes
Permanent notes
Bibliographic notes
Journal entries
Meta notes
Attachments

Fleeting notes

Fleeting notes are transient thoughts you store for a future review. You might have a sudden insight or remember a task you need to do. You quickly jot it down in your notebook and move on with whatever you were doing.

Physical notebooks or mobile phone applications are good methods to capture fleeting notes. Imagine you are in the middle of writing an article and suddenly you remind myself to put the rubbish bins out. In Org mode you don't open a special note-taking application and break your focus with a complex workflow. Adding a fleeting note is only a few keystrokes away.

When you feel the urge to write a note that is unrelated to the buffer you are currently working in, simply press C-c c and a the Org mode capture screen pops up. You select the relevant template and write your thoughts into the popup buffer and press C-c C-c to save it. If you decide it was not worth storing after all, press C-c C-k to cancel the input. Org mode then returns to where you left off and you happily continue with your task with minimal disturbance.

The Emacs Writing Studio configuration defines two types of fleeting notes:

Textual notes
New tasks

When you create a new fleeting note, Org mode asks you wether you need a new fleeting note or a task. Any entries are stored under the relevant heading and new notes are placed directly under previous entries.

The basic structure of your inbox looks like this:

  #+title: Inbox

  * Notes

  * Tasks

The configuration below binds the keyboard shortcuts for capturing fleeting notes and links (see below on how to use links). The configuration also uses the ews-inbox-file variable to store the filename of you fleeting note location, which is ~/Documents/inbox.org by default. You can of course customise this location to any filename of your choosing (see configure Emacs for details).

  ;; Fleeting notes
  (use-package org
    :after
    denote
    :bind
    (("C-c c" . org-capture)
     ("C-c l" . org-store-link))
    :custom
    (org-default-notes-file ews-inbox-file)
    (org-capture-bookmark nil)
    ;; Capture templates
    (org-capture-templates
     '(("f" "Fleeting note" item
        (file+headline org-default-notes-file "Notes")
        "- %?")
       ("p" "Permanent note" plain
        (file denote-last-path)
        #'denote-org-capture
        :no-save t
        :immediate-finish nil
        :kill-buffer t
        :jump-to-captured t)
       ("t" "New task" entry
        (file+headline org-default-notes-file "Tasks")
        "* TODO %i%?"))))

As you create more and more fleeting notes your inbox steadily fills with random musings and actions. In my personal Getting Things Done setup I promise myself to review these notes once per week and process them. You can use the C-u C-c c shortcut to jump to the inbox for your chosen template.

The weekly process involves reviewing these entries and either:

Undertaking the actions
Move the actions to your calendar
Convert the fleeting note to a permananet note.
Trashing the fleeting note.

Ideally, your Inbox should trend towards zero content.

The Org mode capture templates are a versatile system that allows you to craft templates for different types of notes. You could, for example, create a separate entry for a shopping list that you synchronise with your mobile phone.

You can access the configuration in the capture menu with the C button, which pops up the customisation screen for the org-capture-templates variable. Next click the INS button to add another entry and complete the relevant fields as below and save the new variable. The example below create a shopping list stored in a file in your Dropbox folder. Several mobile apps exist that can read Org mode files, so you can take your list to the shops if you have a means to synchronise the relevant files.

Org capture customisation.

The possibilities for capture templates are extensive and depend on your individual use cases. Explaining the configuration of the Org capture options in detail is outside the scope of this website. Section 10.1 of the Org manual discusses developing capture templates in detail.

Creating notes with Denote

The Denote package is amazingly flexible and can implement whatever note-taking approach you like to use, including a fully-fledged Zettelkasten approach or just a freeform stream of consciousness.

The driving force of the Denote package is its file-naming convention. This approach embeds any metadata in the filename so there is no need for complex nested folder structures to find your way through your jungle of notes.

The Denote file naming convention has five parts (all in lowercase by default), of which only the first part and file extensions are required:

Timestamp in ISO 8601 format with a T separating date and time, e.g. 20230930T160112.
The signature of the note (freeform letters and numbers), staring with a double equals sign, e.g. ==9a12.
Note title separated by dashes (kebab-case), staring with a double dash, e.g. --taking-notes-with-denote.
Keywords separated by an underscore (snake_case), starting with a double underscore, e.g. __emacs_productivity
Filename extension so Emacs recognises your file type.

The default setting is that filenames are in all lowercase. So a full Denote file name for this example looks like this:

20230930T160112==9a12--taking-notes-with-denote__emacs_productivity.org

By default, Denote enforces lowercase characters in its filenames, but you can change his behaviour and control the letter casing of file names.

The timestamp is a unique and immutable identifier that Denote uses for linking notes. The signature indicates a semantic order for notes, such as used in the . Semantic note-taking systems such Zettelkasten method. The keywords define categories the note belongs to. The signature, title and tags are flexible and can change over time without damaging the integrity of the system.

If you like to include signatures for your notes, then you need to use the denote-signature function.

All new notes are stored in the same folder, signified by the denote-directory variable, which in EWS is mapped to ews-notes-directory. There is technically no need to store notes in subdirectories, but you can do so with the denote-subdirectory function.

Using tags is a more versatile method to organise your information than subdirectories. If you use a subdirectory to order files, then part of the meta data for that file disappears as soon as you move or copy the file to another location. Also, a file can only live in one directory, but it can have multiple tags. Modern operating systems can easily manage tens of thousands of files in one directory, so use tags to categorise your information.

Configure the denote-prompts variable to define the default way Denote generates and renames files.

Denote has a minor mode that formats compliant filenames in the directory editor so it is easy to recognise the individual items of the note's metadata. The filenames not only provide metadata for the note itself, they are also a heuristic to make it easy to find notes based on date, signatures, title or keyword.

Extract of Denote files in Dired.

Denote can generate notes in plain text, Markdown or Org mode. Emacs Writing Studio uses the Org mode format because it is most suitable for writing bibliographic notes and managing citations. You can also use the Denote file naming convention for any other file types, for example to organise your photographs or PDF documents. So in Denote we can have notes (txt, md or org) and any other file is an attachment.

Permanent notes

A permanent note is a piece of information you like to retain long term. In my personal workflow, I regularly convert fleeting notes and writings in my paper notebook to permanent notes.

The denote-create-note function, which you activate with C-c w d n creates new notes. It first asks for a title and then for the relevant keywords. You either select a keyword from the completion list in the minibuffer with the TAB key or you enter new ones, separated by a comma.

You can also create a permanent note through the Org capture mechanism with the p option in the capture menu (C-c c).

The title of the note becomes part of the filename. In Org mode, you can also add a subtitle to the note by adding #+subtitle: to the front matter of your note.

Bibliographic notes

A bibliographic note is a special category of permanent notes that link to one or more publications in your bibliography. The citar-denote package provides extended functionality to create and manage bibliographic notes, described in detail in another article.

Journal notes

You can also use Denote for personal reflection and create a journal. Writing a journal with Denote is easy because the identifier for each note indicates the date and time you created each note. Adding a standard tag, such as _journal makes your journal entries easy to distinguish from other notes.

If you like to create a note for an entry in the past, then use the denote-date function (C-c w d d). You enter the date in Year-Month-Day (ISO 8601) notation like 2023-09-06. Optionally you can add a specific time, e.g: 2023-09-6 20:30. If no date and/or time is provided, Denote uses the present date and time. It makes sense to tag each journal entry with a standard keyword, e.g. _journal, or whatever makes sense in your native language.

The Denote manual provides some extra functionality on keeping a journal or diary.

Meta notes

A metanote provides an entry point to other notes about a similar topic. Lets say that you are working on a project to write a paper about the daimonion (prophetic monitor) that spoke to ancient Greek philosopher Socrates. You read the literature and create a bunch of permanent and bibliographic notes that use the _daimonion keyword. When it is time to gather your thoughts into an integrated view, you can create a new note that links to all your relevant notes.

Org mode uses dynamic blocks to achieve this. The denote-org-dblock-insert-links function (C-c w d D) will ask you for a regular expression that matches the notes you like to list. The great thing about dynamic blocks is that you can update them in case new notes are added by hitting C-c C-c twice when the cursor is on the BEGIN or END line.

#+BEGIN: denote-links :regexp "_daimonion"
- Plato: Apology
- Socrates and Plato
- Plato: Crito
#+END:

To change the search criteria you change the text between quotation marks after the :regexp part of the first line.

You could, for example, use this approach to collate your journal notes for a certain month. Using the regular expression ^202309.*_journal will list all your journal entries for September 2023. The regular expression lists filenames that starts with 202309 and include the _journal keyword. The tilde (^) denotes that you are searching at the start of the filename. The .* in the middle of the regular expression indicates that any character (.) can appear multiple times (*). In other words, this regular expression searches for all notes that start with 202309, then some random characters, followed by _journal. The .* expression effectively acts as an AND statement, so we look for a note that starts with 202309 and contains journal. The only restriction is that the two elements have to appear in this order. Using ^202308\|_journal searches for notes that start with 202308 or contain _journal. Regular expressions are a powerful tool for searching but detailed discussion is outside the scope of the Emacs Writing Studio.

The latest version of Denote also lets you transclude the text of other notes inside a note. The denote-org-dblock-insert-text function works much in the same way as the links version, but includes the text of the referenced notes inside the meta note.

The dynamic block option provides a versatile tool to aggregate your thoughts about a topic and link to relevant notes. They provide an easy entry point into your collection of notes and are a great starting point for your writing projects.

Attachments

Denote's reliance on a filename to store metadata allows you to manage files other than the three types specified by Denote, which we can call attachments. An attachment in Denote is any file that is not recognised by Denote as a note, but with a compatible filename. Any file stored in the Denote directory that follows the Denote file naming convention will be recognised as an attachment and can be linked from inside a Denote file.

The denote-file-types variable defines the file types that are considered notes (by default Org mode, Markdown and plain text). You only need to adust this variable if you like to use Denote to create new notes in a non-standard format. No configuration is required to visit and link to attachments.

This means that your digital notes garden can be much more than just text, Markdown or Org files. You can manage your photographs with Denote, store an archive of PDF files, such as bank statements, course certificates, or scans of your paper archive. Extending the Denote in this way converts your list of notes to a complete information management system with easy heuristics to find your documents and linking your them to notes. Extending the Denote in this way converts your list of notes to a complete information management system with great heuristics to enable finding attachments and linking your them to notes.

The use-cases for extending Denote to binary files are numerous. I personally save my photographs and videos in the Denote file format. I also store numerous PDF files, such as documents I scan or receive in this format. You could store literature as attachments instead of in your bibliography, but then you lose the ability to cite them in your writing.

If you like to separate collections of files then you can use silos within Denote. A silos is a group of Denote files that do link to each other. The denote-directory defines the default silo. If you want to link Denote files to each other, then they need to live within the same silo. Alternatively, you can create a symbolic link inside the Denote directory.

One use case would be to store all your tax receipts as PDF files. Getting ready for your tax return is than as easy as using a regular expression filter in Dired to mark all the files you need. Alternatively, you can link them all in an Org mode Denote file with a dynamic block using denote-org-dblock-insert-links.

For example, you tag all your tax records with the _tax keyword and you can then find your 2023 tax files with the ^2023.*_tax regular expression, which creates a list of links.

You can also link to individual attachments inside a Denote file in the same way as you link your notes. It is obviously not possible to link back from an attachment to a note using Denote.

Denote recognises any file in the Denote directory that follows the correct naming convention. So the first step to register your attachment in Denote is giving it a compliant correct name.

The denote-rename-file function (C-c w d r) lets you construct a Denote-compliant file name for attachments from within the dired file manager. This function uses the filename as a default title, which you can of course modify. The user adds relevant keywords and the last modified timestamp for the file becomes the identifier.

However, the creation date on the file system is not always the actual creation date. When working with attachments there are three options for a valid timestamp:

When the digitised document was created
When the electronic file was born
When the electronic file was created on the file system

Using the universal argument C-u instructs denote-rename-file to ask the user to enter a date and/or time for the attachment.

So in Emacs Writing Studio, use C-u C-c d r to rename a file and manually provide a timestamp. You need to enter the timestamp is ISO 8601 format, for example: 2023-11-04 12:32:45. Alternatively, when you use Org mode, you can customise the denote-date-prompt-use-org-read-date variable.

Over the years I have gradually digitised my paper archives. The oldest timestamp in my Denote library is 13700623T120000, which is a scan of a historic document that holds the mortgage of the house in which I was born. The creation date of the document is 20 September 2021. The Denote renaming function uses the file system date, but in this case I need to manually create a timestamp that places the document in the distant past.

The second scenario mostly occurs with photographs. The timestamp on the file system might not be the same as the moment the photograph was taken. So in this situation we need to extract the creation date from the file. Several tools, such as exiftool are available to extract metadata from photographs and PDF files.

Finding Notes and Attachments

When collecting thousands of notes and attachments it is important that you have a way into the system to find the information you need or to make new connections between ideas.

The most straightforward method to find files is by opening a file in your Denote directory. The minibuffer completion system helps you to find what you need.

If, for example, you like to filter to all notes tagged as Emacs, type C-c C-f and move to your notes folder and type _emacs and minibuffer completion narrows the options. If you need a note with Emacs in the title, but not as a tag, then use -emacs. If you type emacs without a prefix then you see all posts with Emacs in the signature, title or as a tag.

When you use the minibuffer completion configuration for the Emacs Writing Studio, you can separate keywords with spaces in any order. You can apply the same approach to title words and signatures. For example, using the search string --denote _ews ==9a shows all notes with a title that starts with Denote (using the double hyphen), include the "ews" keyword and have a signature that starts with "9a".

The consult-notes package by Colin McLear merges the capabilities of Denote and the Daniel Mendler's Consult package to help you find notes without having to remember which directory they live in. This package also provides facilities to search through the content of your notes collection.

To find a note by any part of its filename, use the consult-notes function bound to C-c w f. You can use all methods described in the section on finding notes to narrow your quest for the perfect note. The configuration for Consult-Notes can include your main Denote files and one or more silos, giving you full access to all your files in a convenient interface.

Search for titles, tags and other meta data is great, but sometimes you will want to search through the content of your notes. The consult-notes-search-in-all-notes function (C-c w s) activates the search facilities. For this purpose, the package uses the grep software, which needs to be available on your system. Emacs Writing Studio checks for the presence of external software at startup and leaves a warning in the messages buffer.

Linking Notes

You can link notes and attachments with the denote-link-or-create function (C-c w d i). This function provides a list of all available notes. Using the minibuffer completion system and the heuristics described in the previous section you can find the note you like to link to and hit ENTER.

The link only stores the identifier, e.g. denote:20210208T150244. This means that you are free to change the signature, title or keywords of notes without having to worry about broken links. If you change the identifier timestamp in the filename, then Denote will be unable to find the linked document.

You follow a link in Org mode with the org-open-at-point command, bound to C-c C-o.

You can jump to any linked note with denote-link-find-file (C-c w d l). This function shows all notes linked in this file in the minibuffer so you can select the one you like to jump to. This saves you having to find the location of the link and click on it. To find out which notes link to the one that you are currently reading, use the denote-find-backlink function (C-c w d b). This will, just like the previous function, show a list of all notes that are linked to the current one.

Managing Meta Data

If you need to change the title or signature of a note, you can do so in the front matter inside the file. The denote-rename-file-using-front-matter function (C-c w d R) renames the file based on the information in the file.

To conveniently change keywords with minibuffer completion, use the denote-keyword-add (C-c w d k) and denote-keyword-remove (C-c w d K) functions.

The Denote-Explore package provides some additional functionality to manage Denote files.

Configuring Denote

The following lines of Elisp code configure Denote. This configuration sets the ews-notes-directory variable as the repository for your Denote notes. You can change this location to any one you choose using the customisation menu, or replace this variable with a string.

  ;; Denote
  (use-package denote
    :init
    (require 'denote-org-dblock)
    (denote-rename-buffer-mode 1)
    :custom
    (denote-directory ews-notes-directory)
    :hook
    (dired-mode . denote-dired-mode)
    :custom-face
    (denote-faces-link ((t (:slant italic)))))

  ;; Denote extensions
  (use-package consult-notes
    :commands (consult-notes
               consult-notes-search-in-all-notes)
    :custom
    (consult-notes-file-dir-sources
     `(("Denote" ?d ,ews-notes-directory))))

Learn more about Denote

This article only provides a brief introduction to the Denote package. The extensive manual describes its full functionality in great detail. You can read the manual online, or read it inside Emacs with the info function (C-h R denote).

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The next articles in this series discusses how to explore your Denote collection with some enhanced functionality.

Exploring Your Ideas With the Denote-Explore Package

Peter Prevos — Fri, 08 Sep 2023 00:00:00 +0000

The Denote package by Protesilaos (Prot) Stavrou provides extensive functionality to create, retrieve, manage and link files in plain text, Markdown and Org Mode. The most redeeming qualities of this package are its filename convention and modular simplicity. You can also use the package to access other file types, such as PDFs or multimedia files (attachments). This way, Denote becomes a fully-featured file-management system. You can read more about Denote in the context of Emacs Writing Studio in another article.

The Denote-Explore package came into existence as my collection of Denote files grew. I created some auxiliary functions to manage and explore my burgeoning collection of Denote files, which I eventually packaged together. The package provides four types of commands:

Summary statistics: Count notes, attachments and keywords.
Random walks: Generate new ideas using serendipity.
Janitor: Maintain your denote collection.
Visualisations: Visualise your Denote files as a network.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

The Denote-Explore package is available on MELPA and GitHub.

Emacs Writing Studio (Exploring your Denote digital knowledge garden).

Summary Statistics

This Denote-Explore package distinguishes between Denote files (notes) and attachments. Denote files are either Org Mode, Markdown, or Plain Text. All other files, such as photographs, PDFs, media files, LaTeX, and HTML, are attachments.

After a day of working hard on your digital knowledge garden, you might like to count the notes and attachments in your collection. Two functions provide some basic statistics of your Denote files:

denote-explore-count-notes: Count the number of notes and attachments.
denote-explore-count-keywords: Count the number of distinct Denote keywords.

These functions are informative, but a graph says more than a thousand numbers. The built-in chart.el package by Eric M. Ludlam is a quaint tool for creating bar charts in a plain text buffer. Two commands are available in Denote-Explore to visualise basic statistics:

denote-explore-keywords-barchart: Visualise the top n keywords.
denote-explore-extensions-barchart: Visualise used file extensions.

Example of a bar chart of top-twenty keywords.

Random Walks

Creativity springs from a medley of experiences, emotions, subconscious musings, and connecting random ideas. Introducing random elements into the creative process generates avenues of thought you might not have travelled otherwise. Random walks through your notes can be beneficial when you're stuck in a rut or just like to walk through your files randomly.

A random walk is an arbitrary sequence of events without a defined relationship between the steps. You take a random walk by jumping to a random note, either connected or unconnected to the current buffer.

The Denote-Explore package provides four commands to inject some randomness into your explorations:

denote-explore-random-note: Jump to a random note or attachment.
denote-explore-random-link: Jump to a random linked note (either forward or backwards) or attachments (forward only).
denote-explore-random-keyword: Jump to a random note or attachment with the same selected keyword(s).
denote-explore-random-regex: Jump to a random note that matches a regular expression.

The default state is that these three functions jump to any Denote text file (plain text, Markdown or Org-mode). Adding the universal argument (C-u) includes attachments as candidates for a random jump.

Jumping to a randomly linked file only works when the current buffer is a Denote file. A warning appears when the current buffer is an isolated note (no links or backlinks available).

When jumping to a random file with selected keywords, you can choose between one or more keywords from the current buffer, or you can override the completion options with free text. When the current buffer only has one keyword, the section process is skipped. When the current buffer is not a Denote file, you can choose any available keyword(s) in your Denote collection. The asterisk symbol * selects all keywords in the completion list. Please note that jumping to a random note with multiple keywords only works when the denote-sort-keywords is enabled or when the selected keywords are in the same order as in the target file. The following section explains how you can alphabetise your keywords using the Janitor.

Denote-Explore Janitor

After using Denote for a while, you might need a janitor to keep your collection organised. A janitor is a member of the maintenance and cleaning staff for buildings. Their primary responsibility is to ensure cleanliness, orderliness, and sanitation, so this role is also perfect to apply to your Denote files. The Denote-Explore package provides a series of commands to assist with cleaning, ordering and sanitising your files.

Duplicate notes

The Denote identifier is a unique string constructed of the note's creation date and time in ISO 8601 format (e.g., 2024035T203312). Denote either uses the current date and time when generating a new note or the date and time the file was created on the file system. The Denote package prevents duplicate identifiers when creating a new note.

Besides my Denote-generated notes, I have historical attachments in my collection and manually add identifiers. My 'oldest' Denote file is a mortgage document from 23 June 1370. The creation date and time of the file are irrelevant, as I like to keep my files sorted in historical order, so I manually add identifiers (for example 13700623T120000). I used the same approach when renaming scanned photographs in Denote format. Adding the Denote identifier manually introduces a risk of duplication. Duplicates can also arise when exporting Denote Org mode files, as the exported files have the same file name but a different extension.

The denote-explore-identify-duplicate-notes command lists all duplicate identifiers in a popup buffer. Be careful when changing the identifier of a Denote file, as it can destroy the integrity of your links, so please ensure that the file you rename does not have any links pointing to it. You can use the denote-find-link and denote-find-backlink commands to check a file for links.

Using this command with the universal argument (C-u), looks for duplicated filenames instead of identifiers. This option thus ignores any duplicated identifiers created when exporting Denote Org mode files.

Isolated notes

Denote implements a linking mechanism that connects notes (either Org mode, Markdown, or plain text) to other notes or attachments. This mechanism allows the user to visualise all notes as a network, as explained below. However, some notes don't have any links or backlinks. Depending on your note-taking strategy, you might want all your notes linked to another note.

The denote-explore-isolated-notes function provides a list in the minibuffer of all notes without links or backlinks for you to peruse. You can select any note and add any links. Calling this function with the universal argument C-u includes attachments in the list of lonely files.

Managing Keywords

Denote keywords connect notes with similar content. Keywords should not exist in solitude because a category with only one member is not informative. Single keywords can arise because topics need to be fully developed or due to a typo. The denote-explore-single-keywords command provides a list of file tags that are only used once. The list of single keywords is presented in the minibuffer, from where you can open the relevant note or attachment.

You can also find any notes or attachments that don't have any keywords at all with the denote-explore-zero-keywords. This command presents all notes and attachments without keywords in the minibuffer, so you can open them and consider adding a keyword or leaving them as is.

You can remove or rename keywords with denote-explore-rename-keyword. Select one or more existing keywords from the completion list and enter the new name of the keyword(s). This function renames all chosen keywords to their new version or removes the original keyword from all existing notes when you enter an empty string as the new keyword. This function cycles through all notes and attachments containing one or more of the selected keywords and asks for confirmation before making any changes. The new keyword list is stored alphabetically, and the front matter is synchronised with the file name.

Denote sorts keywords alphabetically when the denote-sort-keywords variable is enabled. Ordering keywords alphabetically makes searching for files more predictable. If you rename files manually, the keywords might not always be in the correct order. The denote-explore-sort-keywords function checks all notes and notifies the user if there are any notes where keywords are not alphabetised. The function warns the user before renaming any files. This function also synchronises the front matter with the file name.

Synchronising Meta Data

Denote stores the metadata for each note in the filename using its ingenious format. Some of this metadata is copied to the front matter of a note, which can lead to differences between the two metadata sources.

The denote-explore-sync-metadata function checks all notes and asks the user to rename any file where these two data sets are mismatched. The front matter data is the source of truth. This function also enforces the alphabetisation of keywords, which assists with finding notes.

Visualise Denote

Emacs is a text processor with limited graphical capabilities. Committing your ideas to text requires a linear way of thinking, as you can only process one word at a time. In my paper journal, I often use diagrams, such as a mind map, rather than a narrative to relate my thoughts. Visual thinking is another way to approach your ideas, and one of the most common methods to visualise interlinked documents is in a network diagram.

Viewing your notes as a network can help discover hitherto unseen connections between your thoughts. Visualising your Denote digital garden can be helpful in your creative process.

Denote Explore does not provide a live environment to view the structure of your Denote collection. The purpose of network visualisation is to analyse the structure of your notes, not to act as an alternative user interface. Live previews of note networks are dopamine traps. While seeing the network of your thoughts develop in front of your eyes is satisfying, it can also become a distraction. The Denote-Explore visualisation tools provide a surgical tool to analyse and restructure your notes.

A network diagram has nodes (vertices) and edges. Each node represents a file in your Denote system, indicated with a circle. Each edge is a link between notes. Denote-Explore provides three types of network diagrams to explore the relationships between your thoughts. The package exports and displays each of these in one of three formats, with SVG files viewed in the browser as the default.

Visualising Denote files as a network.

You create a network with the denote-explore-network command. This command will ask the user to select the type of network to create. Each network type requires additional inputs to zoom in to a defined part of your Denote files.

Community of Notes

A community consists of notes that share part of an ID, name, signature or keyword. The software asks to enter a search term or regular expression. For example, all notes with Emacs as their keyword (_emacs), or all notes with a certain part of a signature, e.g. ==ews01.

The regular expression ==ews01.*_emacs selects all notes with ews01 in their signature and emacs and a keyword. The .* in the middle of the regular expression indicates that any character (.) can appear multiple times (*). In other words, this regular expression searches for all notes that contain ews01, then some random characters, followed by _emacs. The .* expression effectively acts as an AND statement. The only restriction is that the two elements have to appear in this order. Using =ews01\|_journal searches for notes that contain _ews01 OR _journal. Regular expressions are a powerful tool for searching but a detailed discussion is outside the scope of Emacs Writing Studio.

A community graph displays all notes matching the search term and their connections. The algorithm prunes any links to non-matching notes (dotted line in the example).

The denote-explore-network-regex-ignore variable lets you define a regular expression of notes to ignore in your visualisations. Lets assume you create meta notes with long lists of dynamic links and they have the _meta keyword, then you could exclude this set of nodes by setting this variable to "_meta".

Community of Denote files with Emacs keyword.

To generate a community graph, use denote-explore-network, choose community and enter a search string or regular expression.

The denote-explore-network-regenerate command recreates the current graph with the same parameters, which is useful when changing the structure of your notes and you like to see the result visualised.

Note Neighbourhood

The neighbourhood of a note consists of all files linked to it at one or more steps deep. The algorithm selects members of the graph from linked and backlinked notes. This visualisation effectively creates the possible paths you can follow with the denote-explore-random-link function discussed in the Random Walks session above.

Denote neighbourhood of files (depth = 2).

To generate a neighbourhood graph from the current Denote note buffer, use denote-explore-network and enter the graph's depth. The user enters the required depth, and the software searches all notes linked to the current buffer at that depth. When building this graph from a buffer that is not a Denote note, the system also asks to select a source file (A in the diagram). The system issues a warning when you select a note without links or backlinks. You can identify your Denote files without any links with the denote-explore-isolated-notes function.

The denote-explore-network-regenerate command recreates the current graph with the same parameters, which is useful when you want to change the structure of your notes after viewing the first version of the graph.

The complete set of your Denote files is most likely a disconnected Graph, meaning that there is no one path that connects all nodes. Firstly, there will be isolated notes. There will also exist isolated neighbourhoods of notes that connect to each other but not to other files.

A depth of more than three links is usually not informative because the network can become very large and hard to read, or you hit the edges of your island of connected notes.

Keyword Network

The last available method to visualise your Denote collection is to develop a network of keywords. Two keywords are connected when used in the same note. All keywords in a note create a complete network. The union of all complete networks from all files in your Denote collection defines the keywords network. The relationship between two keywords can exist in multiple notes, so the links between keywords are weighted. The line thickness between two keywords indicates the frequency (weight) of their relationship.

When generating this graph, you will need to enter a minimum edge weight for the graph. The graph then will only show those keywords that are at least n times associated with each other. The default is one, which can generate a rather large graph.

While the first two graph types are directed (arrows indicate the direction of links), the keyword network is undirected as these are bidirectional associations between keywords. The diagram below shows a situation with two nodes and three possible keywords and how they combine into a keyword network.

Denote network of keywords example.

Some keywords should be excluded from this graph because they skew the results. For example, when using the Citar Denote package, you might like to exclude the bib keyword from the diagram because it is only used to minimise the search space for bibliographic notes and has no further semantic value. The denote-explore-network-keywords-ignore variable lists keywords ignored in this visualisation.

The denote-explore-network-regenerate command recreates the current graph with the same parameters, which is useful when you are changing your notes' structure.

Network Layout and Presentation

Emacs cannot independently generate graphics and thus relies on external software. This package can use three external mechanisms to create graphs (configurable with denote-explore-network-format), set to GraphViz SVG output by default.

The Denote-Explore network algorithm consists of four steps:

Use the denote-explore-network function to enter the network type and pass on to another function to enter the required parameters.
The code generates a nested association list that holds all relevant metadata for the selected graph:
- Metadata e.g.: (meta (directed . t) (type . "Neighbourhood '20210104T194405' (depth: 2)"))
- Association list of nodes and their degrees, e.g., (((id . "20210104T194405") (name . "Platonic Solids") (keywords "geometry" "esotericism") (type . "org") (degree . 4)) ...). In the context of Denote, the degree of a network node is the unweighted sum of links and backlinks in a note.
- Association list of edges and their weights, e.g., (((source . "20220529T190246") (target . "20201229T143000") (weight . 1)) ...). The weight of an edge indicates the number of times it occurs, which is the number of time two files are linked or the number of times two keywords appear in the same note.
The package encodes the association list to a GraphViz DOT file, JSON file, or GEXF file. The location and name of this file is configurable with the denote-explore-network-directory and denote-explore-network-filename variables.
Relevant external software is called upon to display the result.

The denote-explore-network-graph-formats variable defines the file extension and the relevant functions for encoding and visualisation for each graph format.

GraphViz

GraphViz is an open-source graph visualisation software toolkit, ideal for this task. The Denote-Explore software saves the graph in the DOT language as a .gv file. The GraphViz software converts the DOT code to an SVG file.

You will need to install the GraphViz software to enable this functionality. Denote-Explore will raise an error when trying to create a GraphViz graph without the required external software available.

The configurable denote-explore-network-graphviz-header variable defines the basic settings for GraphViz graphs, such as the layout method and default node and edge settings.

The denote-explore-network-graphviz-filetype variable defines the GraphViz output format. SVG (the default) or PDF provide the best results. The SVG format should be viewed in a web browser to view tooltips of nodes to show their name and other meta data, and to follow hyperlinks. Emacs can display SVG files, but is unable to follow links or show tootltips.

Hovering the mouse cursor over a node provides its name and other meta data. You can open the relevant file by clicking on the node, which works best when using Emacs as a server, if you configure your browser to open Org mode, Markdown and text files with the Emacs client. Links only work in neighbourhood and community graphs. These interactive functions are only available when viewing SVG files in a web browser.

The diameter $D$ of nodes are sized relative to their degree $d$: $D = \sqrt{d + 1}$. Thus, the most referenced note in your system will be the most visible. For nodes with a degree greater than two, the name is displayed outside the node (top left). When generating a neighbourhood, the source node is marked in a contrasting colour. In keyword graphs, the thickness of the edges indicates the number of times two keywords are associated with each other.

The layout of the graph uses the Neato spring model in GraphViz.

This method is ideal for viewing small parts of your network. The network will be hard to read when the number of notes becomes too large. I have not yet figured out how to better scale the output based on the number of nodes in a graph.

Example GraphViz visualisation.

D3.js

D3.js is a JavaScript library for data visualisation. This method provides an aesthetically pleasing and interactive view of your note collection. The Denote-Explore package stores the desired network as a JSON file.

The JavaScript file is generated with the R language as I have not yet mastered JavaScript to write it myself from scratch. R saves the network as an HTML file in the designated folder with the networkD3 R package. Hover over any node to reveal its name.

The colours indicate a statistical grouping based on the connections between nodes. This grouping is calculated with the Walktrap community detection algorithm, which finds communities of nodes by assessing which ones are more connected to each other than to nodes outside the community.

To enable this view, you must install the R language on your system. R will install some required libraries when you run this code for the first time. Any JavaScript developers interested in writing a better solution are cordially invited to submit improvements.

Example D3.js visualisation.

Graph Exchange XML Format

The first two formats provide some analysis of your knowledge network, but there is a lot more you can do with this type of information. While GraphViz and D3 are suitable for analysing parts of your network, this third option is ideal for storing the complete Denote network for further analysis. To do this, use the Community option and enter an empty search string to include all files.

Graph Exchange XML Format ( GEXF) is a language for describing complex network structures. This option saves the network as a GEXF file without opening it in external software.

You can analyse the exported file with Gephi Lite, a free online network analysis tool. The GEXF file only contains the IDs, names and degree of the nodes, and the edges and their weights.

Example Gephi Lite visualisation.

Analysing the Denote Network

A well-trodden trope in network analysis is that all people are linked within six degrees of separation. This may also be the case for your notes, but digging more than three layers deep is not very informative as the network can become large and difficult to review.

It might seem that adding more connections between your notes improves them, but this is not necessarily the case. The extreme case is a complete network where every file links to every other file. This situation lacks any interesting structure and wouldn't be very useful for analyses. So, be mindful of your approach to linking notes and attachments.

Your Denote network is unlikely to be a fully connected graph. In a connected graph, there is a path from any point to any other point. Within the context of Denote, this means that all files have at least one link or backlink. Your network will most likely have isolated nodes (files without any (back) links) and islands of connected notes. The previously discussed denote-explore-isolated-notes function lists all files without any links and backlinks to and from the note in the minibuffer.

The number of links and backlinks in a file (in mathematical terms, edges connected to a node) is the degree of a node. The degree distribution of a network is the probability distribution of these degrees over the whole network. The denote-explore-degree-barchart function uses the built-in chart package to display a simple bar chart of the frequency of degrees. This function might take a few seconds to run, depending on the number of notes in your system. Evaluating this function with the universal argument C-u excludes attachments from the analysis.

Installation and Package Configuration

This package is available through MELPA. The configuration below customises all available variables and binds the interactive functions using the Emacs Writing Studio C-c w prefix key and e for Denote-Explore. You should modify this configuration to suit your needs, as one person's sensible defaults are another person's nightmare.

  (use-package denote-explore
    :bind
    (;; Statistics
     ("C-c w x c" . denote-explore-count-notes)
     ("C-c w x C" . denote-explore-count-keywords)
     ("C-c w x b" . denote-explore-barchart-keywords)
     ("C-c w x e" . denote-explore-barchart-filetypes)
     ;; Random walks
     ("C-c w x r" . denote-explore-random-note)
     ("C-c w x l" . denote-explore-random-link)
     ("C-c w x k" . denote-explore-random-keyword)
     ("C-c w x x" . denote-explore-random-regex)
     ;; Denote Janitor
     ("C-c w x d" . denote-explore-identify-duplicate-notes)
     ("C-c w x z" . denote-explore-zero-keywords)
     ("C-c w x s" . denote-explore-single-keywords)
     ("C-c w x o" . denote-explore-sort-keywords)
     ("C-c w x w" . denote-explore-rename-keyword)
     ;; Visualise denote
     ("C-c w x n" . denote-explore-network)
     ("C-c w x v" . denote-explore-network-regenerate)
     ("C-c w x D" . denote-explore-degree-barchart)))

The most recent manual is available in Emacs which you can access with C-h R denote-explore or by evaluating (info "denote-explore").

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The next article discusses how to manage literature with BibTeX mode.

Manage Literature with Emacs BibTeX Mode

Peter Prevos — Fri, 07 May 2021 00:00:00 +0000

Authors need three resources: imagination, writing skills and the work of other writers. A common advice for authors to improve their writing is to read a lot. Keeping track of all the books and articles you read can be daunting. This article describes how to manage references and bibliographies with Emacs BibTeX mode. The biblio package helps searching internet databases and add literature, and the citar package provides access to your bibliography.

These Emacs packages let you create and manage bibliographies, read electronic documents and take notes, turning Emacs into a fully-featured literature management system.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Getting started with Emacs BibTeX mode

Emacs ships with bibtex-mode, a major mode to create and manage bibliographies. This mode, as the name suggests, uses the BibTeX file format as your default literature database. BibTeX is a plain text format to manage literature and citations and is typically used for LaTeX typesetting, but also perfectly integrates with Org mode. A typical entry for a book will look like this:

@article{stallman_1981_emac,
  title      = {{{EMACS}} the Extensible, Customizable Self-Documenting Display Editor},
  author     = {Stallman, Richard M.},
  year       = 1981,
  journal    = {ACM SIGOA Newsletter},
  volume     = 2,
  number     = {1-2},
  pages      = {147--156},
  doi        = {10.1145/1159890.806466},
  keywords   = {Emacs}
  file       = {computing/stallman-1981-emacs.pdf}
}

Each entry starts with an @-sign and the entry type, followed by a curly brace and a unique citation key. The following lines contain the relevant data about this entry, in this example, the earliest publication about Emacs.

BibTeX can process different types of literature, such as articles and conference papers, each of which has its own field types. BibTeX ignores any field names that are not recognised, which provides opportunities to use the file for other purposes, such as attaching files and keywords.

BibTeX is a plain text file with the .bib extension. You can store one or more bibliography files in a central folder to refer to them from anywhere in Emacs. You can also attach specific BibTeX files to an Org mode file to set a bibliography for a project.

You don’t need to install any packages to get started. Just created an empty file with a .bib extension, and Emacs enables bibtex-mode when you open the file.

Adding new entries

Emacs bibtex-mode uses templates to add new entries. To add a reference, use the bibtex-entry function (C-c C-b). Use the minibuffer completion to select the relevant type. Emacs also provides a shortcut for every kind of literature, which you can see with C-c C-e C-h.

Each entry type has at least one compulsory field. Any optional fields start with OPT. For fields that begin with ALT, such as author or editor, you must complete at least one of them.

Jump from field to field with C-j (bibtex-next-field) and complete all required fields and one of the ALT fields. When you’re done, press C-c twice (bibtex-clean-entry) to check the syntax and remove empty fields. This function also assigns a citation key to the entry using some configurable rules, but you can also set one manually.

To enter names, place the family name first, followed by a comma and the first name or initials. Any additional authors are separated by “and”, e.g. "Hawking, S. and Penrose, R.".

If you copy and paste an author name, or it is automatically inserted using the biblio tool discussed below, the first and family name might be the wrong way around. A nice Emacs hack is to use the org-transpose-words function, bound to M-t. This function swaps the order of two words, for example from "Richard Stallman" to "Stallman Richard".

To align the lines of each entry, you can also use the bibtex-fill-entry function (C-c C-q). The table below summarises the most salient keyboard shortcuts and functions in bibtex-mode.

Shortcut	Function	Description
`C-c C-e a`	`bibtex-article`	Add an article
`C-c C-e b`	`bibtex-book`	Add a book
`C-c d`	`bibtex-empty-field`	Empty the current field
`C-j`	`bibtex-next-field`	Jump to next field
`C-down`	`bibtex-next-entry`	Jump to the next entry
`C-up`	`bibtex-previous-entry`	Jump to the previous entry
`C-c C-c`	`bibtex-clean-entry`	Clean the entry
`C-c C-q`	`bibtex-fill-entry`	Align the fields
-	`bibtex-reformat`	Reformat all entries in region or buffer
-	`bibtex-sort-buffer`	Sort all entries in the buffer by their keys

Overview of keyboard shortcuts to add and edit entries.

Importing existing BibTeX files

You can also link another bibliographic database, such as Zotero, to Emacs. The Better BibTeX package in Zotero can export parts of or your complete bibliography as a BibTeX file every time it changes, keeping both version in sync.

Any existing bib file created with other software might not be fully compatible with bibtex-mode because some high-level features are unavailable in BibTeX mode. The bibtex-convert-alien function checks whether bibtex-mode can parse an ’alien’ BibTeX file. Files generated with Zotero do not seem to require this conversion step.

Fine-tuning BibTeX mode

BibTeX mode works out of the box, but you can fine-tune the variables. This Elisp expression adds optional fields for keywords and files for each entry type. These fields are essential when using the bibliographic files to read electronic documents and take notes. This expression also instructs Emacs to align the equal signs for each entry when you press C-q.

  ;; Managing Bibliographies
  (use-package bibtex
    :custom
    (bibtex-dialect 'BibTeX)
    (bibtex-user-optional-fields
     '(("keywords" "Keywords to describe the entry" "")
       ("file" "Link to a document file." "" )))
    (bibtex-align-at-equal-sign t))

BibTeX is old but stable software that was last updated in 1988 and has, as such, some minor limitations. The BibLaTeX dialect is a newer version. To change bibtex-mode to BibLaTeX, change the bibtex-dialect variable in the configuration to biblatex.

  (bibtex-set-dialect 'biblatex)

Jonathan Le Roux publishes a comprehensive manual on his website that explains some more functionality, such as structuring the citation key, in more detail.

Managing Bibliographies

Several packages are available to explore bibliography files and connect it to notes. These packages achieve four main tasks:

Edit the bibliography
Explore bibliographies (searching, narrowing etc) and insert citations
Access links and attachments for reading
Relate bibliography to notes note file related to the entry

Package	Edit	Explore	Assets	Notes
bibtex-mode (built-in)	X
biblio	X
citar		X	X	X
citar-denote				X
ebib	X	X	X	X
helm-bibtex /ivy-bibtex		X	X	X
org-roam-bibtex				X
zetteldesk-ref				X
citar-org-roam				X
zk-citar				X

This article further explores using biblio and citar to manage bibliographies. The other packages are provided as alternatives.

Using the Biblio package to add new entries

BibTeX mode requires you to type all entries manually, which is inefficient and could easily lead to errors. Clément Pit-Claudel’s biblio package lets you browse and import bibliographic references from a range bibliographic sources: CrossRef, DBLP, arXiv, Dissemin, and doi.org.

The configuration for this package is straightforward, as there are no variables to configure. To get better response times from CrossRef, you need to customise the biblio-crossref-user-email-address variable and include your email address in the string.

  ;; Biblio package for adding BibTeX records and download publications
  (use-package biblio)

To use the package, open the relevant BibTeX file, run M-x biblio-lookup, and select the database and enter a search query. Once the search has completed, a new buffer opens with the results. Select your target with the arrow keys or search in the buffer to select the preferred candidate.

Once you have your target, you can insert the BibTeX record into the buffer from where you called the function with i. Alternatively, you can copy the BibTeX record with c and paste it into place later. You quit the search results with q. To see all possible commands in this buffer, use h or ?.

If you have a DOI (Digital Object Identifier) for a publication, then use the biblio-doi-insert-bibtex function to insert a BibTeX record into the current buffer.

Unfortunately, large corporate publishers still hold the world’s academic knowledge behind lock and key. However, open access and pre-print publications are slowly becoming the norm. The Dissemin website searches for copies of papers in an extensive collection of open repositories and websites.

To use this service with Emacs, evaluate dissemin-lookup to show information about the open access status of a paper using a DOI number. You can also press x when in the search menu for the biblio-lookup function to check for open access version.

Using Citar to Access Bibliographies

Several Emacs packages are available that link bibliography files to notes. The Ebib, Helm-BibTeX and Ivy-BibTeX each use their own completion system. The Citar package uses the standard minibuffer completion mechanism to access your bibliographies. This package provides access to hyperlinks, notes, attachments and the source bibliography file. Citar also integrates with org-cite, Org mode’s citation module.

Within the Org mode system, you can have two types of bibliographies. The global bibliography is a set of BibTeX files available from anywhere within Emacs. You can also link one or more project-specific local BibTeX files to an Org mode file, discussed in the article about bibliographic notes.

The ews-bibtex-directory variable stores the location of all BibTeX files and their associated attachments. This variable is defined in the article on Emacs configuration. By default, this variable is set at ~/Documents/library/. Using this variable in combination with the ews-bibtex-register command will configure Emacs to recognise all BibTeX files in the configured directory.

You store all your BibTeX files and the associated attachments in this location. BibTeX files have to reside in this directory, attachments can be in subdirectories. Any file paths for BibTeX entries start at this path. For example, when the BibTeX entry states: file = {computing/stallman-1981-emacs.pdf}, the attachment is stored at: ~/Documents/library/computing/stallman-1981-emacs.pdf

The EWS configuration creates a new variable named ews-bibtex-file which lists the .bib files in the ews-bibtex-directory that start with a letter or a number (to exclude backup and hidden files). This code is a regular expression, which is a powerful method to find text. If you add another bibliography file, you will have to rerun the configuration code, or restart Emacs to register it.

If you add or remove any bibliography files from the bibliography directory, then use the ews-bibtex-register function to register the new set of bibliographies. You don't need to do this when you add new entries to a file, only when adding, renaming or removing a file.

Citations are further discussed in another article.

  ;; Citar to access bibliographies
  (use-package citar
  :defer t
  :custom
  (citar-bibliography ews-bibtex-files)
  :bind
  (("C-c w b o" . citar-open)))

Open the Citar menu with citar-open (C-c w b o) and select the literature you seek.

Finding literature with Citar is easy. After every keystroke in the Citar menu, the list of options is narrowed to the relevant matches. Select your candidate with the arrow keys and hit Enter. Use the Tab key to select more than one entry.

The first three columns in the menu indicate which entries include a hyperlink (L), one or more attached files (F) and an associated note (N). The configuration for taking bibliographic notes is explained in another article. You can filter the Citar completion menu for entries with notes using :n, showing only entries with an attachment use ":f" and entries that have links with ":l".

After hitting Enter, Citar provides a popup menu in the minibuffer from where you can open any attachments, create a new note or open an existing note and follow any hyperlinks listed in the BibTeX entry.

Example of the Citar menu.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

Emacs is a malleable system, so everybody will have their personal preferences to undertake a task. Any article on how to use Emacs is thus opinionated. If you have a different way of doing things, please complete the contact form to send me an email or contact me on social media.

The next article explains how to create bibliographic notes with the Citar-Denote package.

Bibliographic Notes with the Citar-Denote Emacs Package

Peter Prevos — Fri, 17 Feb 2023 00:00:00 +0000

Taking notes about the articles and books you read is an essential part of intellectual life. Many note-taking systems can connect to a bibliographic database (such as a plain text BibTeX file, or external software such as Zotero). These databases are helpful when adding citations to your writings. Electronic bibliographies also help you to create and find notes related to your literature collection. and provide access to related materials, such as links to online sources and electronic versions of relevant books or articles.

The Citar-Denote Emacs package integrates the Citar bibliography package with the Denote note-taking system. Citar lets you browse and act on bibliographic data in BibTeX, BibLaTeX and JSON-CSL format. Denote is a is a versatile Emacs package for taking notes. Combining these two packages enables linking any note in your Denote collection to one or more bibliographic entries, providing a complete solution for documenting literature reviews. My article about taking notes with Emacs explains how to use Denote.

This article is part of Emacs Writing Studio, a book that explains how to use Emacs to undertake research and write and publish articles, books, and websites. Emacs Writing Studio is available as an e-book from your favourite retailer.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

To learn how to create and maintain a plain text bibliographic database in Emacs, read the article about BibTeX mode. The article about writing in Org mode explains how to add citations. This page explains how to use the Citar-Denote package, which integrates a Citar bibliography with your Denote notes.

Emacs Writing Studio (Create and manage literature notes with Citar-Denote).

Principles

A bibliographic note is a file that refers to one or more pieces of literature in your bibliography. The note and the entry in your literature database are intrinsically linked, making it easy to move between the two. Two types of relationships exist between your notes and your bibliography:

Reference: Relates the whole file to one or more publication (a bibliographic note). Indicated by the reference line in the front matter and the bib keyword.
Citation: A citation inside a note mentions one or more publications. A citation relates a portion of a text to a publication.

The Citar-Denote package maintains a many-to-many relationship between your notes and your bibliographies. Each bibliographic entry can have one or more notes, and each note can reference one or more pieces of literature.

You can create several atomic notes for each publication or write a complete literature review of a collection of references within one note. You could, for example, create a note about each chapter of a book, or write a single literature review note for a collection of journal articles. You can use Citar and Denote which ever way best suits your workflow.

To enable references, we need a stable link between the collection of notes and the bibliography. The most common behaviour for bibliography managers in Emacs is that the filename for the relevant note includes the citation key (e.g. marcuse_1969_essay.org) to link it to the bibliography. However, Denote follows a strict naming convention, so we cannot use this approach to link notes and literature as it might contain characters that Denote disallows in its file names. The various citation system don't have strict naming conventions on citation keys, so we need to find another way to link literature to a note.

In Citar-Denote, the citation key is part of the front matter through the #+reference: keyword. Citar-Denote stores the relationship between citation keys and note files in a cache. The package regenerates this cache after you edit a bibliography file or call any Citar functions. Each bibliographic note uses the configurable bib file tag. This tag reduces the number of Denote files the system needs to track. The front matter for a bibliographic note would look something like this:

  #+title:      Marcuse: An Essay on Liberation
  #+date:       [2022-11-12 Sat 19:23]
  #+filetags:   :bib:culture:marketing:philosophy:
  #+identifier: 20221112T192310
  #+reference:  marcuse_1969_essay

The Citar-Denote package also provides insight into citations used in the Denote directory, which uses the xref system. You can improve the search process by installing the faster ripgrep program and setting xref-search-program to ripgrep.

Create bibliographic notes

Open the Citar interface with C-c w b c (citar-create-note). The letters or icons at the start of the Citar menu indicate whether the entry has a link, note, attachment or is cited in the current buffer. Select the entry you want to create a note for, hit ENTER and follow the prompts.

If a note already exists and you have set the citar-open-always-create-notes variable to t, then you can create additional notes for this entry. Otherwise, Citar will open the exiting note.

Screenshot of Citar with Vertico, indicating notes and links.

Various configuration options are available to fine-tune determine how Emacs creates now bibliographic notes.

Default file type

You can create bibliographic notes for each file type that Denote supports: Org mode, two flavours of Markdown and plain text. Citar-Denote uses the default file type set by the denote-file-type variable, which you can override with citar-denote-file-type to use a different file type for bibliographic notes.

Bibliographic note title

The default name for a new note is the title of the bibliographic entry. You can modify this behaviour with the citar-denote-title-format variable. Four options are available:

title: Extract the title from the bibliographic item (default)
author-year: Use citation format as "author(s) (year)" (or editors when no authors are defined)
author-year-title: Concatenate the first two options
full: Full citation of the entry
Fallback option or nil: The citation key

For example, using this citation: Coppa, Hass, Peck, Burger (2008) Performing Magic on the Western Stage: From the Eighteenth Century to the Present, Palgrave Macmillan.

title: "Performing Magic on the Western Stage: From the Eighteenth Century to the Present".
author-year: "Coppa et al. (2008)".
author-year-title: "Coppa et al. (2008) Performing Magic on the Western Stage: From the Eighteenth Century to the Present".
full: "Coppa, Hass, Peck, Burger (2008) Performing Magic on the Western Stage: From the Eighteenth Century to the Present, Palgrave Macmillan".
nil: "coppa_2008_perf".

Using author-year for entries allows for some further configuration. For entries with more than one author, you can specify the maximum number of names with citar-denote-title-format-authors, which is one by default. When the number of defined authors is more than the number in the citation, "et al." is added to the end. When this variable exceeds the number of defined authors, all authors are listed. The default term between authors is "and", which can be changed by customising citar-denote-title-format-andstr.

For example, when using up to three authors and "&" as connecting term, the title for the example above becomes Or "Coppa, Hass & Peck et al. (2008)"

Bibliographic keywords

The bib file tag is added to every new bibliographic note. This tag minimises the search space when caching notes to accelerate the process. The citar-denote-keyword variable lets you change the tag to something else. For example, if you primay language is Welsh, you might want to change it to lly.

Changing the default requires updating all your existing bibliographic notes, because the package only recognises a single string. The change will come into effect the next time the package is loaded. The Denote-Explore package has a function to rename Denote keywords.

The citar-denote-use-bib-keywords variable lets you include keywords defined in the bibliography as Denote keywords. When set to a non-nil value, Citar-Denote extracts the available keywords from the BibTeX keywords field.

Subdirectory

If you prefer storing your bibliographic notes in a subdirectory of your Denote folder, then set the citar-denote-subdir variable to t. When this variable is non-nil, Denote asks for a subdirectory when creating a new bibliographic note. When this variable contains a string, then any new bibliographic notes are stored in the subdirectory that matches this name.

Signature

When the citar-denote-signature variable is non-nil, Denote will ask for a signature when creating a new bibliographic note. If you enter an empty line for a signature, then the citation key is used.

Please note that Denote changes or removes punctuation marks in the citation key to enforce compliance with its file naming convention. For example, when the citation key is einstein_1905, the signature becomes einstein=2005.

Create notes using templates

Denote allows you to create template for various types of notes. You could, for example, have standard headings for bibliographic notes, other headings for meta notes, or whatever else you might need.

To include templates in new bibliographic notes, set the citar-denote-template to non-nil and define a template.

The example below lets the user choose between a note that has two headings (Abstract and Review), or a plain note without any template content. The extensive Denote manual provides some more detailed information about using templates for all your notes.

  (setq denote-templates
        '((biblio . "* Abstract\n\n* Review")
          (plain . nil)))

Working with existing notes

Once you have some bibliographic notes you will want to access and modify them. You can access the attachments, links and other notes associated with the references from within via the Citar menu (citar-open), C-c w b o. Entries with a note are indicated with an N in the third column.

Open existing bibliographic notes

There are two entry points to find notes that relate to literature, either as a reference or as a citation.

Use citar-denote-open-note (C-c w b n) to open the Citar menu with only entries that have one or more associated notes. Select your target and hit ENTER.

Citar provides a list of resources for the selected entry: attachments, existing notes, links and an option to create an additional note. Select the note you seek, hit ENTER again and select the Denote file you want to open.

Screenshot of Citar and options for one of the entries.

The previous function shows all literature that has one or more bibliographic note(s). The citar-denote-find-citation function (C-c w c f) lists all bibliographic entries cited inside your Denote collection, from where you can open the relevant note. Some of these will also have their own bibliographic note, indicated in the Citar menu sidebar.

Searching through all your notes for citations can take a moment, depending on the size of your digital garden.

Open attachments, links and notes

The citar-denote-dwim function (C-c w b d) provides access to the Citar menu from where you can open attachments, other notes, and links related to the citation references associated with the current Denote buffer.

Select the required bibliographic item when there is more than one reference. You can then select the attachment, link or note you like to access and hit ENTER, after which you select your link, note or attachment. Alternatively, you can also create a new note

Open bibliographic entry

The citar-denote-open-reference-entry function (C-c w b e) opens the bibliographic entry (BibTeX, BibLaTeX or CSL file) for a selected reference from where you can edit the bibliographic data.

Convert existing notes to bibliographic notes

The citar-denote-add-citekey function (C-c w b k) adds citation keys or converts an existing Denote file to a bibliographic note. When converting a regular Denote file, the function adds the bib keyword to the front matter and renames the file accordingly.

This function opens the Citar selection menu and adds the selected citation keys to the front matter. Use the TAB key to select multiple entries.

Remove references from bibliographic notes

You remove citation references with the citar-denote-remove-citekey command (C-c w b K). If more than one piece of literature is referenced, select the unwanted item in the minibuffer first. The bibliography keyword is also removed, and the file is renamed when removing the only reference.

Citation management

What is the point of building a bibliography without using each entry as a citation or a reference in a bibliographic note? The last two functions let you cite literature or create a new bibliographic note for any item not used in your Denote collection.

The citar-denote-nocite (C-c w b x) function opens the Citar menu and shows all items in your bibliography that are neither cited nor referenced. From there you can create a new bibliographic note, follow a link or read the file.

Configuration

The Emacs Writing Studio configuration builds on the C-c w prefix keys. Pressing C-c w b provides access to bibliographic commands using the keystroke completion menu provided by the Which Key package .

This configuration also sets a variable from the Citar package that allows you to create multiple notes for a bibliographic entry.

  (use-package citar-denote
    :custom
    (citar-open-always-create-notes t)
    :init
    (citar-denote-mode)
    :bind
    (("C-c w b c" . citar-create-note)
     ("C-c w b n" . citar-denote-open-note)
     ("C-c w b x" . citar-denote-nocite)
     ("C-c w b k" . citar-denote-add-citekey)
     ("C-c w b K" . citar-denote-remove-citekey)
     ("C-c w b d" . citar-denote-dwim)
     ("C-c w b e" . citar-denote-open-reference-entry)))

The source code for Citar-Denote is available on GitHub and the package is available through MELPA.

This page is a previous version of the manual. You can read the recent manual with C-h R citar-denote or evaluate (info "citar-denote")

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The next article in this series describes how to use Emacs Org mode for distraction-free writing.

Emacs Org Mode for Distraction-Free Writing

Peter Prevos — Wed, 05 May 2021 00:00:00 +0000

The world is full of distractions, especially when focusing on writing. Most writing software is littered with icons and options to change the document’s design. Distraction-free writing tools remove distractions from the screen, so they become more like electronic typewriters that let the author focus on content over form. This article shows how to use Emacs Or Mode for Distraction-Free writing, using the built-in Org mode package.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Getting started with Org mode

Org mode is a powerful major mode that comes with Emacs by default. This software was initially developed in 2003 by Carsten Dominik, a professor of astronomy at the University of Amsterdam. Since then, countless other developers have continued to advance Org-mode.

You can use Org mode to publish websites, articles and books, keep a diary, keep research notes, manage your actions, and much more. And on top of all that, it is intuitive to use. This section shows you the basics of writing prose in Org mode with a screencast by Jake B. that briefly summarises the major options within Org mode.

Emacs Org Mode Demo by Jake B.

Creating new Org buffers

To start an Org file, create a file with a .org extension and start writing, for example, C-x C-f test.org <Enter>.

Each Org document starts with a header that contains metadata and settings relevant to the buffer. The Org mode metadata and settings start with #+ followed by a keyword and a colon. The document header can also contain metadata such as a subtitle or a date. Emacs packages can use this information when publishing the text and other functionality.

  #+title:   Lorem Ipsum
  #+author:  Nomen Nescio
  #+date:    [2021-05-08 Sat]

Org mode recognises several other types of front matter items to control exporting and other behaviours, some of which are mentioned below.

Your chosen Emacs theme will set the size and font for your document. The only purpose of these decorations is to help you navigate the document. The article about publishing articles and books with Org mode shows how to control the output format.

Document structure

Books have chapters, sections and paragraphs; articles have headings; poems have verses; and so on. Almost all forms of writing have a hierarchy. An excellent method to start writing is to work out your hierarchical structure and fill it with content.

Org mode is an outlining tool that helps you develop a document structure. Write an asterisk as the first character to start a new header, followed by a space. To create deeper levels, add more stars:

  * Heading 1
  ** Heading 2
  *** Heading 3

When you press M-enter, the following line becomes a new heading. With C-enter, the next line is added after the text in the current section.

Org mode makes it easy to move and promote or demote headings and associated text (which in Org mode is a subtree) with the ALT and arrow keys.

You can collapse or expand a heading with the TAB key. Pressing S-TAB will collapse the document, showing only the level one headings. Pressing S-TAB once again will show the headings, and repeating it for a second time reveals the entire buffer. You can keep cycling through these modes with the S-TAB key.

TAB / S-TAB: (un)fold headings
M-up / M-down: move a headline up or down
M-left / M-right: promote or demote a headline
M-RET: insert a new headline

Add numbers to the headers with the num option in the startup line: #+startup: num. Whether this numbering appears in any published output depends on your configuration.

Rich text

To change how Org Mode displays text, you surround it with special characters: /italic/, *bold*, _underline_, +striketrough+ and =verbatim=. You will see these markers in the Org mode text, but they disappear when you export the buffer.

Lists

Writing lists in Org mode could not be easier. Simply start a line with a dash and complete the entry with M-ENTER to create the next entry. Using the TAB and left or right arrow keys changes the depth of the item and, with the up and down arrows, moves the line up or down in the hierarchy. Change the list prefix with the SHIFT and left/right arrow keys.

  - Item
    + next item
      1. The following
      2. And another
         a. Down, down, deeper and down

The following configuration line enables alphabetically ordered lists as well as numerals.

  (use-package org
    :custom
    (org-list-allow-alphabetical t))

Tables

Every cell in an Org mode table is flanked by the pipe | symbol. You don't have to worry about aligning the text, as the TAB key does that for you. You create a horizontal line when you start a line with |- and hit TAB.

| Column 1 | Column 2 |
|----------+----------|
| Sator    |       12 |
| Arepo    |       26 |
| Tenet    |      878 |
| Opera    |       89 |
| Rotas    |       89 |

You navigate forward through the cells with the TAB or arrow up/down keys. Using s-TAB moves you back one cell. Combining Alt and Shift with the arrow keys adds and removes columns and rows.

Org mode has extensive capabilities to edit tables, including formulas to create spreadsheet functionality.

Images

Although Emacs is a text editor, it can also manage images. In Org mode, an image is a link to an image file. You can preview the file in your buffer or export it to your preferred format. You can also add captions and set display or export properties for each image.

  #+caption: Image caption.
  #+attr_html: :alt Alternative text
  [[file:path/to/image]]

To view the preview images in your document or remove the previews, you press C-c C-x C-v (org-toggle-inline-images). You can also add #+startup: inlineimages to view images by default for the buffer.

To add an image, press C-c C-l (org-insert-link), which opens the link menu. Org mode understands many types of links. We want the file type, so type file:. Press enter and select the image filename in the mini buffer. You can skip the file part with the C-u C-c C-l shortcut (adding C-u before the standard shortcut.)

After adding the image, you preview it with the org-redisplay-inline-images function or C-c C-x C-M-v.

You open a link in Org mode with a mouse click or by pressing C-c C-o (org-open-at-point) with your cursor on the link text. Emacs has some facilities to manage image libraries through the image-dired package.

The images in the buffer are all shown at the same width, which you can configure to meet your needs. The articles about writing articles and books with Emacs and the article about creating websites with Emacs explains how to control image size and other image properties.

Emacs Writing Studio provides a convenience function to insert images from screenshots, which is bound to the Control-PrintScreen key. This function asks for a filename (and forces a PNG extension). The user then selects a partial screenshot and a caption. The function then inserts the image link and caption.

  (defun ews-org-insert-screenshot ()
    "Take a screenshot with ImageMagick and insert as an Org mode link."
    (interactive)
    (let ((filename (read-file-name "Enter filename for screenshot: " default-directory)))
      (unless (string-equal "png" (file-name-extension filename))
        (setq filename (concat (file-name-sans-extension filename) ".png")))
      (call-process-shell-command (format "import %s" filename))
      (insert (format "#+caption: %s\n" (read-from-minibuffer "Caption: ")))
      (insert (format "[[file:%s]]" filename))
      (org-redisplay-inline-images)))

  (with-eval-after-load "org"
    (define-key org-mode-map (kbd "C-<print>") #'ews-org-insert-screenshot))

Chapter numbering

Org mode will number every heading when the #+startup: num option is set. The numbering will appear in the Org file. How numbering is treated in the export depends on the file format you use.

In some publications, not all headers are numbered, such as in the front matter of a book. You can exclude individual sections from being numbered by setting the :unnumbered: property as shown below. To add this property, drawer, type C-c C-x p, select the relevant property and enter the value.

:PROPERTIES:
:UNNUMBERED: t
:END:

Citation Management

Citations are the engine of academic writing. Org mode has a built-in citation management tool. You will first need to create a bibliography. You can either create one yourself, or dynamically link a file from a bibliography management tool such as Zotero.

Bibliographies can be global or local. The global bibliography is accessible from any place in Emacs, while the local bibliography file is only available within your Org mode file.

The global bibliography is set in your configuration (org-cite-global-bibliography variable) and can consist of one or more BibTeX / CSL files.

The local bibliography is linked to an Org mode file with the #+bibliography: "bibfile.bib" in your document header.

You can insert citations with org-cite-insert (C-c C-x @). Within Emacs Writing Studio, this command opens the Citar menu from where you can select one or more publications. A citation will look something like this: [cite:@einstein_1905;@newton_1728].

You can access the resources related to a citation with org-open-at-point (C-c C-o), which opens the Citar menu for the citation under point. If the citation has no entry in your local or global bibliography, then Org mode will warn that it could not find a match and will ask you to create a new heading, which of course is not the case.

Citar indicates wether an entry is cited in your project with a C in the list of publications.

Org mode has detailed methods to determine how citations are rendered in exported documents, which is discussed in another article.

Managing Your Writing Project

A writing project is about more than just smashing words into a document. Some functionality is available to add notes to your project, manage word counts and overall progress of your writing.

Writing Tools in Emacs.

Large Projects

Writing a book in a single Org mode file can be laborious because you need to navigate a large file. The built-in narrowing tool can help in keeping your focus. Narrowing in Emacs means that the buffer will only show a selected part of your text so you don't get distracted by the rest of the document. The hidden text is still available, just not visible on the screen.

To narrow your buffer to only show the subtree (heading and associated subheadings) you are currently working in use org-narrow-to-subtree (C-x n s). This commands reduces the visible text to the section under consideration. To go back to the full document evaluate the widen command (C-x n w).

Sometimes it might be a good idea to split larger projects over multiple files. Org mode has an inclusion function that creates a link between documents.

The #+include: "chapter-02.org" line includes a file named Chapter 02 inside the main document. You can visit this child document with C-c ' (org-edit-special). Org mode has some additional options to determine exactly which part of the child document is included.

This method allows you to work on a book or dissertation and store each chapter in a separate file. When you export the main file to a PDF, or ePUB file, all included files are added to the export.

Adding notes

Sometimes, adding notes to your text that should be separate from the final version is worthwhile. You should add a checklist of what needs to be done, links to websites or notes in your Denote digital garden, or anything else of importance during the process of writing.

Any line in Org mode that starts with a hashtag or pound symbol is a comment, which Emacs ignores when exporting the text to the final output.

  # This is a comment

Org mode also has handy drawers, which are bits of collapsible text that can store notes. You can keep multiple lines of text in a drawer, for example, if you need to cut out a bit of text and would like to save it for future reference. The beauty of Org mode drawers is that they are collapsible, so their contents do not distract you from the primary task. You can close or open a drawer with the TAB key when the cursor is on the drawer.

  ** This is a headline
  :DRAWER_NAME:
  This is inside the drawer.
  :END:
  
  After the drawer.

You can interactively insert a drawer at point by calling the org-insert-drawer, bound to C-c C-x d. The selected text will be placed inside the drawer when a region is active. This function will add the name for your drawer, usually written in full caps. The drawer is inserted at the current location of the cursor.

An Emacs Writing Studio function (ews-org-insert-notes-drawer) generates notes drawers, bound to the C-c C-x n keyboard shortcut. This function jumps just below the heading of the section you are writing and generates a notes drawer. If a drawer exists for this section, a new line is created at the end of the current note. After you finish writing the note, C-u C-SPACE will take you back to your original position in the text.

  ;; Notes drawers
  (defun ews-org-insert-notes-drawer ()
    "Generate or open a NOTES drawer under the current heading."
    (interactive)
    (push-mark)
    (org-previous-visible-heading 1)
    (next-line)
    (org-beginning-of-line)
    (if (looking-at-p "^[ \t]*:NOTES:")
        (progn
          (org-fold-hide-drawer-toggle 'off)
          (re-search-forward "^[ \t]*:END:" nil t)
          (previous-line)
          (org-end-of-line)
          (org-return))
      (org-insert-drawer nil "NOTES"))
    (org-unlogged-message "Press <C-u C-SPACE> to return to the previous position."))

  (with-eval-after-load "org"
    (define-key org-mode-map (kbd "C-c C-x n") #'ews-org-insert-notes-drawer))

If you need to take any notes unrelated to the document at hand, then a fleeting note stored in your central inbox is a better choice. The Org mode capture mechanism is the best solution for these situations.

Counting words

Emacs has a built-in functionality to count words in a region using the count-words-region function (M ==). To find out how many words each chapter or section of your work has, you will need to run this for each part of your document. Within Org mode, there is an easy way to automate this behaviour and get an almost instant overview of your word count for each document part. This method will also let you add word count targets for each section so you can monitor progress or balance your writing.

Org mode can add properties to each heading, which we can leverage to add word counts. To add a property, use C-c C-x p and select one from the list or create a new one. The ews-org-count-words function cycles through all headings in your open Org buffer and adds a word count for each heading, including the subheadings. To add a target word count, add a new property called target and add the relevant number.

  (defun ews-org-count-words ()
    "Add word count to each heading property drawer in an Org mode buffer."
    (interactive)
    (org-map-entries
     (lambda ()
       (let* ((start (point))
              (end (save-excursion (org-end-of-subtree)))
              (word-count (count-words start end)))
         (org-set-property "wordcount" (number-to-string word-count))
         (unless (org-entry-get nil "target")
           (org-set-property "target" "0"))))))

A collapsible property drawer shows a word count and your manually added target. When no target property exists yet, this function will add one with zero as a default. This property drawer works the same way as the notes drawer discussed previously.

  :PROPERTIES:
  :wordcount: 305
  :target: 300
  :END:

You can view the word count for each heading in Org mode's property column view C-c C-x c (org-columns). Ensure you evaluate this function at the highest level (beginning of the document). Org mode only shows the properties for the subtree you are currently in. This view creates an overlay. By cylcing through the document with S-TAB, a table will appear. To include the word counts and targets, you will need to add this line at the start of the document, which defines which properties to see in each column :

  #+columns: %40item %10todo %10wordcount %10target

The columns are an overlay and not part of the text, so nothing you wrote is modified. You can keep editing your document in column view. Place the cursor on a column overlay to remove the overlay and press q.

Status tracking

As you are working on various parts of your writing project it might be godo to know what the status of each chapter is.

Org mode also has an extensive system to manage projects, which you can also deploy to keep track of progress in your document.

Each heading in Org mode can have a status token, such as TODO, DONE or EDITED, or whatever workflow you like to use. You add a status token with the shift and left/right arrow keys when the cursor is on a heading. You can also use the C-c C-t shortcut (org-todo).

By default, the system only uses the TODO and DONE status. You can add additional workflow states by defining them in the document header. The example below will cycle through these four status tokens. The tokens before the vertical line are in progress and usually marked in red. The vertical line are completed and marked in green. Org mode will add a timestamp when a section gets to the FINAL status in this example.

  #+todo: WRITE DRAFT EDIT | FINAL

Org mode has extensive facilities to fine tune how these status tokens behave, including adding notes and timestamps.

This article only covered the tip of the iceberg of the writing and editing functionality in Emacs. Many specialised packages are available to enhance your writing experience. I advise keeping it simple for now and spend your tie writing instead of configuring Emacs. When you discover a new need, a few Google searches will often give you exactly what you need.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The screen still looks a bit plain, so in the next article we look at some techniques to improve the look-and-feel or Emacs and Org mode.

Ricing Org Mode: A Beautiful Writing Environment

Peter Prevos — Sun, 08 Aug 2021 00:00:00 +0000

A plain text editor helps you focus on content over form. Having said this, the default Emacs experience can be harsh on the eyes. This article describes how to improve Emacs by ricing Org mode for a more pleasant writing experience.

Ricing is a term often used by hackers as the activity to make a system look nicer. The Urban Dictionary defines it as: "To rice, or to soup up a crappy car with the mistaken idea that […] performance yellow paint makes it go faster."

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Org mode as a Text Processor

The main difference between a text editor and a Word processor is that in Emacs, the design of the text (font, colour and so on) is based on meaning and not the writer's choice. The purpose of the buffer design is to help you navigate the document. This will not be the way it looks when you export the text. The articles about publishing articles and books and the article about creating websites show how to export buffers.

The default design of an Org mode buffer is a bit bland, shown in the image below. This article will show you how to step-by-step improve the look-and-feel. These improvements are not beautification for the sake of aesthetics. Removing leading stars, indentation, converting $\LaTeX$ and line spacing facilitates the writing process by giving your eyes focus on content instead of form.

Before ricing Org mode.

All the configuration explained in this series of articles is available as an init file on GitHub.

Org mode Tweaks

Org mode needs some tweaks to remove clutter from the screen. We can change the etxt alignment, change the way fonts of rich text and special characters are displayed and preview images by default.

    ;; Improve org mode looks
    (setq-default org-startup-indented t
                  org-pretty-entities t
                  org-use-sub-superscripts "{}"
                  org-hide-emphasis-markers t
                  org-startup-with-inline-images t
                  org-image-actual-width '(300))

Indentation

Org mode aligns text to the left of the buffer and all the stars of headings are visible. You can also remove leading stars and align subheadings and text below its respective heading to create a more pleasant layout. To globally turn on Org Indent mode for all files, set the variable org-startup-indented to t, which removes the leading stars for a nice clean view.

Rich Text and Special Characters

The following three lines in the configuration instruct Org mode to display special characters and improve how we view rich text.

Pretty entities in Org mode relate to special symbols, such as superscript and subscript (x^{2} or x_{2}) and other LaTeX special characters, such as \pi, which display as x², x₂ and π.

You can toggle this behaviour with the C-c C-x \ keystroke (org-toggle-pretty-entities). For more complex mathematical expressions, see below.

By default, Org mode does not require curly braces for sub- and superscripts. But this can cause confusion if you like to write something using snake_case. This configuration limits applying sub- and superscripts to characters between curly braces by setting the org-use-sub-superscripts variable to {}.

Org mode uses various markers to indicate rich text. Emacs will adapt the text display, but the markers remain visible, which adds to the clutter on your screen. You can remove these markers by setting the org-hide-emphasis-markers variable.

The only problem with hiding emphasis markers is that rich text becomes hard to edit because it is unclear whether your cursor is on the marker or the first or last character. The org-appear package helps by displaying the markers while the cursor is on a rich text word.

  (use-package org-appear
    :hook
    (org-mode . org-appear-mode))

Preview Images

Org mode includes images as file links, but it can also preview the image itself. The penultimate line in this configuration instructs Org mode to preview all images in a buffer. You can toggle this with C-c C-x C-v (org-toggle-inline-images) for each open buffer.

You need to restrict the size of previews because they can get unwieldy when working with high-resolution images. You can set the default width of a preview in pixels, which in this example is 300. If you like to increase the size of a specific image, then place the cursor on it and use the plus and minus keys.

The code above shows the number of pixels in a list '(300) instead of just the number. This method instructs Org mode to override this setting for individual images with the #+attr_org: :width set. In this example, the image is displayed at a width of 100 pixels.

  #+caption: Image caption.
  #+attr_org: :width 100
  [[file:path/to/image.png]]

Prettify Special Symbols

When using Org mode or similar approaches, there a lot of characters that are not content but indicate semantics, such as the asterisks at the start of a header. These symbols are practical but look a bit rough. The previous configuration already removed the leading stars to reduce clutter. Now we will replace these symbols with some nicer looking bullets.

Emacs can change the way a text file is displayed by either hiding or replacing characters. The Org-Modern package implements a configurable package to rice your Org mode buffers. The package can style headline indicators, keywords, tables and source blocks. This configuration disables some of the functionality because hiding indicators is not always a good idea.

  (use-package org-modern
    :hook
    (org-mode . global-org-modern-mode)
    :custom
    (org-modern-keyword nil)
    (org-modern-checkbox nil)
    (org-modern-table nil))

LaTeX previews

If you have a working LaTeX installation, then Org Mode can preview LaTeX fragments, such as formulas, as images. To preview the fragment under the cursor, press C-c C-x C-l (org-latex-preview) to, for example, convert $\sum_1^\infty \frac{1}{x}$ to $\sum_1^\infty \frac{1}{x}$. Org mode created a subdirectory named

The org-fragtog package provides some nice functionality that toggles between the source and the preview of the formulas, which means you don't have to use the org-latex-preview function repeatedly. The package is loaded after the Org package has loaded.

  ;; LaTeX previews
  (use-package org-fragtog
    :after org
    :custom
    (org-startup-with-latex-preview t)
    :hook
    (org-mode . org-fragtog-mode)
    :custom
    (org-format-latex-options
     (plist-put org-format-latex-options :scale 2)
     (plist-put org-format-latex-options :foreground 'auto)
     (plist-put org-format-latex-options :background 'auto)))

The org-format-latex-options variable controls the way the Emacs presents fragments. By default, the text is a bit small. This variable is a list with properties such as colours and size. The plist-put function lets you change one of these options in the list. The foreground and background are set to take the same colour as your text. If you change from dark to light mode or vice versa, you need to evaluate the org-latex-preview function to change the way formulas are displayed.

Line Spacing

When the text is a bit too compact for your liking, then change the line-spacing variable. Setting it to nil results in a cramped reading space, so best to add some whitespace between lines. Space between paragraphs is formed by adding an empty line.

If the value is an integer, it indicates the number of pixels below each line. A decimal number is a scaling factor relative to the current window's default line height. The setq-default function sets this for all buffers. Otherwise, it only applies to the current open buffer. Try the value that best suits your preferences.

  ;; Increase line spacing
  (setq-default line-spacing 2)

A Clean Writing Interface

Writing takes total concentration to produce creative prose. Distractions are the natural enemy of concentration. While your computer is your most important writing tool, it can also be a source of distractions.

Distraction-free writing means that your computer screen is free of clutter and, just like an old typewriter, only shows the text you are working on. In the article about configuring Emacs, we already removed the menu bar and other on-screen distractions.

Olivetti is a simple Emacs minor mode that facilities distraction-free writing. The name Olivetti derives from the famous typewriter brand.

You activate Olivetti mode with M-x olivetti-mode. This minor mode reduces the width of the text to seventy characters and centres the text in the middle of the window. The width of the text is changeable with the M-x olivetti-set-with command or C-c \.

I have written a new function that makes Olivetti mode a bit easier to use. This code stores your window configuration when you hit F9 (or whatever key you prefer) and activates Olivetti mode. It also increases the text by two steps to create a nice focussed screen. Finally, typing F9 again restores the previous window settings.

This snippet of code is a simple example of the flexibility of Emacs and how relatively easy it is to change its behaviour to suit your preferences. The ews-distraction-free function checks whether Olivetti mode is currently active. If this is not the case, then it saves the current window configuration to a register and activates Olivetti mode with enlarged font size. If Olivetti mode is active, then the text is sized to its default and olivetti mode disabled. If the original window configuration had more than two open windows, then this is restored so you are back where you left off.

As a side note, all functions specifically written for the Emacs Writing Studio start with ews-, which is in line with best-practice naming conventions in Emacs Lisp where all functions start with their package name.

  ;; Distraction-free writing
  (defun ews-distraction-free ()
    "Distraction-free writing environment using Olivetti package."
    (interactive)
    (if (equal olivetti-mode nil)
        (progn
          (window-configuration-to-register 1)
          (delete-other-windows)
          (text-scale-set 2)
          (olivetti-mode t))
      (progn
        (if (eq (length (window-list)) 1)
            (jump-to-register 1))
        (olivetti-mode 0)
        (text-scale-set 0))))

  (use-package olivetti
    :demand t
    :bind
    (("<f9>" . ews-distraction-free)))

Ricing Org Mode

By ricing Org mode we have moved somewhere between pure plain text and a graphical interface. There is no functional need to do so other than convenience. While it might seem that we have moved away from the What You See is What You Mean approach, that is not quite the case. The way the screen is presented in Emacs is different from the way it looks in your final output.

For example, if you create a website, any Org mode text surrounded by a forward slash (/example/) is translated as <i>example</i>/. For LaTeX output, it becomes \emph{example}. The HTML style sheet or LaTeX document class determines the way this code looks in the final output (the "what you get" in WYSIWYG). This example is italic by default, but it could be something different.

The results of our efforts have made a difference, as shown by the image below. Your chosen theme will change the background and text colours.

Enhanced Org mode screen.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

Now that you have written a nice article or a new book, it is time to publish your work as an ebook or a physical book.

Publishing Articles and Books with Org Mode Export

Peter Prevos — Wed, 05 May 2021 00:00:00 +0000

Emacs Org mode is an ideal tool for writing without distractions because working with plain text allows you to focus on content instead of design. While writing articles and books in plain text is pleasant and productive, it is not ideal for sharing your work. The Org mode exports your work to PDF leveraging LaTeX, to a website, to a word processor document, an ebook and many more formats. This article explains how to export your work as a journal article, a print-ready book PDF, or an ebook.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Org mode export principles

The basic principle of exporting Org mode files to the desired format is that the system converts the text to the relevant format and then connects it to a document class, CSS style sheet or other type of template. Your text file can also link to a local or global bibliography to manage citations. Org mode has built-in export facilities for LaTeX (PDF), HTML and ODT (LibreOffice) and some other formats. Using external packages you can also export to other formats, such as ePub.

Org mode explort principles

In principle you don't need to know HTML or LaTeX, but it will certainly help with fine tuning the output.

The Org mode export function

Org mode includes a powerful export module to convert your files to various formats. Start the export module with the org-export-dispatch function, which you can run with the default C-c C-e keyboard shortcut. The dispatch provides a range of options. The first letter you type determines the export format, and subsequent letters the specific options. Using q exists the export dispatcher.

The Org mode manual describes a myriad of configuration options to fine-tune the look and feel of the exported file. This article describes the basics of the export functionality for PDF files, word processor formats and ebooks.

Export filename

By default, Org mode saves the result of the export process in the same folder as the source document. The exported file has the same name as the source file but with a different extension.

Exporting Denote Org mode files can create a filename conflict because you are effectively creating two files with the same identifier, so ideally you should change the name of the exported file.

You can change this behaviour by adding #+export_file_name: <filename> to the header. The filename can be any string without file extension. You can save the exported file in another folder, but this might cause errors for certain formats.

Basic settings

Before delving into the specific export formats, some housekeeping sets the defaults on how a document is exported. There are various settings to control how Org mode exports your document, which you add to the front matter of your file, or set them as global variable for all exports. Some generic export variables are:

#+title: Document title.
#+author: Author name (defaults to user-full-name).
#+date: Date of publication (You can set the format of the export with the org-export-date-timestamp-format variable).

The snippet below ignores drawers (which can be used for writing notes and other functions) and todo keywords in the export. This configuration also ignores broken links, does not produce a table of contents and turns straight quotation marks into more aesthetic curly ones.

  ;; Org Export Settings
  (use-package org
    :custom
    (org-export-with-drawers nil)
    (org-export-with-todo-keywords nil)
    (org-export-with-broken-links t)
    (org-export-with-toc nil)
    (org-export-with-smart-quotes t)
    (org-export-date-timestamp-format "%d %B %Y"))

You can override these settings for specific documents with header arguments, as shown below. The d is for drawers and the mystical looking \\=' for smart quotes. The line below shows drawers and todo markers in the export. This configuration marks broken links, shows two levels of the table of contents and quotation marks are not converted.

  #+options: d:t todo:t broken-links:mark toc:2 \\=':nil

Section 13.2 of the Org mode manual explains the various export settings in more detail. Each option can be set as a variable in your configuration or as a line in your Org mode header to fine tune export settings for individual files.

Excluding content from export

Not everything you write should be exported, as suggested in the export defaults in the previous section. The article on writing with Org mode explains how to add comments and notes drawers into your file. Comments with # and are not exported by default. The ews-org-insert-notes-drawer command inserts or appends a drawer in the current document section. The Emacs Writing Studio configuration does not export drawers, so your notes remain private.

Furthermore, can also exclude a section of your writing from export by adding the :noexport: tag. You add a tag to a heading by keying C-c C-q when the cursor is on or inside the heading. You enter the tag in the minibuffer and you can use completion to find existing tags. The tag appears on the right side of the heading name between colon markers.

Tags in Org mode provide flexible functionality to manage large files or categorise action lists. Org mode has extensive capabilities to use tags, which are not further used in Emacs Writing Studio.

Bibliographic references

Org mode has a built-in citation manager that can use BibTeX / BibLaTeX or CSL files to reference bibliographic items such as articles and books.

The article on managing BibTeX files explains how to create a bibliography. You can create a default bibliography, that is accessible to all Org mode files and specify specific bibliography files for each Org mode file.

When exporting to LaTeX, citations are managed by the external software. When exporting to any other format you need to specify how Org mode manages citations. The default settings in Org mode provide basic citation support.

This website by Tecosaur provides an in-depth description of how to manage citations in Org mode, much better than I can explain it.

Introducing Org mode exports.

Word Processor Export

The export function in Org Mode can export to the ODT format, which is compatible with MS Word, out of the box. The ODT export back-end relies on the zip program to create the final output

If you have LibreOffice installed, you can also create a docx file to make it easier for MS Word users to share in the joy of reading your writing. When you set this option, the export process will result in both an odt and a docx file. This behaviour is not the default in Emacs Writing Studio, so you must add this yourself if you prefer exporting in the Microsoft format. Alternatively, you can also use this variable to export to PDF.

  ;; Not included in EWS
  ;; Export ODT to MS-Word
  (setq-default org-odt-preferred-output-format "docx")
  ;; Export ODT to PDF
  (setq-default org-odt-preferred-output-format "pdf")

It is possible to set this option specific to the file you are exporting by adding these three lines somewhere in your Org mode file:

  # local variables:
  # org-odt-preferred-output-format: "pdf"
  # end:

When Org mode exports the file, these lines are evaluated but not exported as they are treated as comments.

You can use specific export settings in the front matter:

#+subtitle: The document subtitle.
#+description: and #+keywords are added to the exported file(s) metadata.
#+odt_styles_file: Add the path to an LibreOffice style file (ott format)

Creating a LibreOffice style file is straightforward:

Create a LibreOffice file to your liking using the Styles menu (press F11),
Modify each style to your liking.
Test the document.
Save as ott file.

A LibreOffice file is essentially a zip file with an embedded set of xml files. Org mode extracts the styles.xml file embedded in your template file. You cannot use this method for templates (pre-configured content).

You can fine-tune how Org exports to OpenOffice to a great extent. For a detailed discussion on odt Export, read the online Org mode manual.

LaTeX export

LaTeX is a powerful typesetting system (pronounced “LAY-tek” or “LAH-tek”), especially for writing scientific and technical documents. LaTeX can convert your text into a beautifully designed PDF file for publishing an article, ebook or physical book. Many publishers of technical literature have LaTeX templates to comply with their style guide.

You don’t necessarily need any knowledge of LaTeX to export to PDF. Still, it will certainly help if you like to fine-tune the design of your document. The basic syntax of LaTeX is easy to explain. Let's assume you have a straightforward Org mode file that looks like this:

  #+title:  Example document
  #+author: Peter Prevos
  #+latex_class: book

  Minimum example for a LaTeX document.

The Latex equivalent of this example is:

  \documentclass{article}
  \title{Example document}
  \author{Peter Prevos}
  \begin{document}
  \maketitle
  Minimum example for a LaTeX document.
  \end{document}

The LaTeX software can convert this document to a wonderfully formatted article due to the document class, which defines the typography and layout.

Writing documents directly in LaTeX can be confusing because you need to know its markup language, and your text is littered with backslashes, curly braces, and other syntactical distractions. Being productive as a writer requires focusing on the text’s content instead of how it looks. Org mode is the perfect LaTeX editor because it frees you from distractions and integrates perfectly with LaTeX.

To enable exporting Org mode files to PDF, you need to have LaTeX installed on your system. How to install LaTeX depends on your operating system, and your favourite search engine will point you in the right direction.

Keying C-c C-e to open the export dispatch, then l p creates and opens a PDF file. Other options are available to export the buffer to LaTeX or to save a PDF file without opening it.

To make the magic work, Org Mode converts your file to a tex file, after which the LaTeX software converts it to PDF. The system works out of the box without any configuration. With some configuration you can produce PDF files that are perfect for producing printed books.

Text elements

Org mode converts headers to relevant LaTeX headers, and text becomes a paragraph. The relationship between the heading level in your Org file and LaTeX depends on the configuration, explained below.

Tables and images

Org mode converts images and tables to LaTeX floats. You can add specific attributes to these floats by using #+attr_latex:, as shown in the image example below:

  #+caption: This is an example image caption.
  #+attr_latex: :width 5cm :options angle=90 :placement h
  [[directory/filename.png]]

Various parameters are available to determine how your table or image looks in the final output. The Org mode manual provides a detailed overview in section 3.10.

LaTeX snippets

You can write simple LaTeX commands directly into your org file. For example, \newpage will add a page break. You can also place equations using dollar signs, for instance $e^{i\pi} + 1 = 0$ results in $e^{i\pi} + 1 = 0$.

The org-latex-preview function (C-c C-x C-l) shows a preview of any LaTeX equations within the open buffer. The chapter on Ricing Org mode introduces the org-fragtog package to automatically toggle between the plain text and the preview.

To create front and back matter, use the \frontmatter and \backmatter LaTeX commands in your Org file at the appropriate locations.

For more complex snippets, you need to use a structure template. Press C-c C-, l to insert a LaTeX source block. This LaTeX example creates an image using the picture environment.

  ,#+begin_export latex
  \setlength{\unitlength}{1cm}
  \begin{picture}(10,10)(-5,5)
  \linethickness{1pt}
  \put(-2.5,0){\vector(1,0){5}}
  \put(0,-2.5){\vector(0,1){5}}
  \put(0,0){\circle{3}}
  \end{picture}
  ,#+end_export

The image will not appear in Org mode as it is generated by LaTeX and will only appear when exporting.

Example of LaTeX picture environment result.

LaTeX Packages and Classes

By default, Org mode uses the article class and a set of default packages to export documents. Org mode provides three mechanisms to use LaTeX packages in your export:

In the header, using #+latex_header:
Configuring the org-latex-packages-alist variable.
Configure the org-latex-classes variable.

The Org mode header can do a lot of the work. The example below specifies the book document class with A4 paper size. This example also specifies the Times fonts package. The last line tells Org mode to omit the table of contents from the export.

  #+latex_class: book
  #+latex_class_options: [a4paper]
  #+latex_header: \usepackage{times}
  #+options: toc:nil

The org-latex-packages-alist variable defines the default packages that are used for every LaTeX export.

You can define more complex header configurations by changing the org-latex-classes association list.

The example below adds the template for the American Psychological Association (APA) journals. This list's documentation provides all the details you need to configure packages and classes for your exports. The with-eval-after-load function ensures that this variable is only set once the LaTeX export function is loaded by Emacs. Note the double backslash instead of the single one in regular LaTeX syntax for compatibility with Emacs Lisp.

  (with-eval-after-load 'ox-latex
    (add-to-list 'org-latex-classes
                 '("apa6"
                   "\\documentclass[a4paper, jou, 11pt]{apa6}
                    \\usepackage[nodoi]{apacite}
                    \\usepackage{graphicx}
                    \\usepackage[T1]{fontenc}
                    \\usepackage{times}"
                   ("\\section{%s}" . "\\section*{%s}")
                   ("\\subsection{%s}" . "\\subsection*{%s}"))))

The last part of this association list specifies the mapping between Org mode and LaTeX headers. In this case, the highest level is associated with the section header, the second level with a subsection, and so on.

You can call this particular class by adding #+latex_class: apa6 to your file header. This mechanism empowers you to define bespoke LaTeX classes to create a library of export options.

These three mechanisms allow fine-grained control over how your Org mode document is exported to LaTeX and PDF. The Emacs Writing Studio configuration only uses the default settings for these variables because the possible use cases are too numerous to define a sensible default other than what is already available.

PDF export configuration

The Emacs Writing Studio configuration for PDF export defines the export process to ensure that bibliographies work appropriately. This configuration also cleans all temporary files that LaTeX creates so that only the exported file remains.

  ;; LaTeX PDF Export settings
  (use-package ox-latex
    :ensure nil
    :demand t
    :custom
    ;; Multiple LaTeX passes for bibliographies
    (org-latex-pdf-process
     '("pdflatex -interaction nonstopmode -output-directory %o %f"
       "bibtex %b"
       "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"
       "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"))
    ;; Clean temporary files after export
    (org-latex-logfiles-extensions
     (quote ("lof" "lot" "tex~" "aux" "idx" "log" "out"
             "toc" "nav" "snm" "vrb" "dvi" "fdb_latexmk"
             "blg" "brf" "fls" "entoc" "ps" "spl" "bbl"
             "tex" "bcf"))))

Publishing Ebooks

Most ebook publishers use the ePub format for distribution. This file type is a ZIP file with your book stored as a website optimised for an e-reader. The ox-epub package adds this functionality to the Org export dispatcher. This package uses the built-in Org to HTML export to create the ebook, so you can use any of its features to fine-tune the output.

  ;; epub export
  (use-package ox-epub
    :demand t)

There are some required export options that need to be set in your header:

#+title: the document title.
#+uid: a unique ID of the document, otherwise known as URI, could be a website or ISBN number.
#+date: the date of the document.
#+author: the document author or editor, the creator in the EPUB spec

The default settings will add a postamble to the bottom of the last page with a timestamp, author and a HTML validation service. Adding #+options: html-postamble:nil toepub the Org file header removes these from your ebook.

Any LaTeX fragments are ignored in the export, which means it is not possible to export the same source file to both PDF and ePub wit the same output.

However, LaTeX math formulas are possible with the tex:dvipng option added to the Org header. This option converts any LaTeX formula to a PNG image, which is not ideal but readable.

The ox-epub package does not convert Org mode timestamps to a date format that complies with the ePub standard. You can correct this by removing the square brackets and the day and time from the timestamp.

Lastly, please note to only use open image formats such as .png as some ebook readers cannot display JPG files and other proprietary formats. The ebook will look alright on your computer but might not pass any checks by a publisher.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The next article dives into using Org mode and Hugo to create websites.

Create Websites with Emacs: Blogging with Org mode and Hugo

Peter Prevos — Fri, 07 May 2021 00:00:00 +0000

When I developed my first website in 1997, I wrote HTML in a plain text editor. When discovering WordPress when it was first released, I thought it was the best thing since sliced bread because it allowed me to create websites without managing individual HTML and CSS pages.

Unfortunately, WordPress has become bloated, and the online editor is inconvenient. Last year, I bid farewell to WordPress and now use Emacs Org mode and Hugo to develop websites.

A significant disadvantage of WordPress is that you write your text in the browser, and your content resides inside a database. You can, as such, only use WordPress to work on your website. The advantage of using Emacs and Org mode to maintain a website is that your content consists of plain text files. There are no databases or access limitations, and you get to use the incredible editing powers of Emacs.

Another reason to ditch WordPress is that static websites are faster and safer because there is no database to hack. A static website generator creates pages in plain HTML, which render faster than a database-driven website, improving the user experience and your SEO. Static websites have at least three advantages over database-driven systems, such as WordPress:

Faster website due to static content (no need to access a database)
Writing web pages offline with your favourite editor
Your content remains in plain text
No dependence on third-party paid plugins

This website is entirely created with Emacs Org mode and Hugo, and this post explains how you can do the same.

This article is part of the Emacs Writing Studio, which explains how to use Emacs to undertake research and write and publish articles, books, and websites. You can find the most recent configuration files on GitHub:

Org mode

Org mode is a markdown language in plain text. This approach helps you to focus more on your writing instead of its design because it emphasises content over formatting. Text editors are not like What-You-See-Is-What-You-Get systems. You write the article in plain text, and when you publish the website, the content is merged with a style sheet that determines the design. The image below shows this article as I write it and what it looks like in the browser.

The WYSIWYM approach: Left, an article in Emacs. Right: The exported result.

Emacs can create simple websites with the default Org mode export function. While this function has a powerful option to use CSS templates, it is not a fully-fledged static website generator. That is why I use Hugo to generate HTML files.

Hugo static website generator

The Hugo static website generator converts a Markdown or Org mode file collection to a website using the Go language. You can use Hugo with Emacs out-of-the-box. The go-org library by Niklas Fasching forms part of Hugo, so in principle you do not need to update your init file.

You will need to install the Hugo software, create an empty site and download a theme before starting. The Quick Start guide for Hugo provides detailed instructions. When you get to step four, come back to this article to find out how to use Org Mode to add some content.

The video series by Mick Dane embedded below provides a comprehensive introduction to using Hugo.

Introduction to Hugo by Mick Dane.

All you need to do now is start writing pages in Org mode or Markdown in the content folder. Emacs can also help you with Markdown, but that is outside the scope of this article.

The theme for this website is optimised for working directly with Org mode files in Hugo. You can download the theme from GitHub. The Hugo website provides access to an extensive collection of themes. Changing the look and feel of your website is as easy as changing the theme.

Using Org Mode and Hugo

The Hugo website does not provide much information about using Org mode. The Go-Org website has a detailed overview of Org mode syntax and how Hugo renders the HTML.

The Org mode module in Hugo perfectly parses your org files, but there are some special considerations and limitations.

Front matter

Each Org mode file starts with the front matter containing your blog post's title, author, date, and other metadata. You can add non-standard variables, which you can use in your Hugo shortcodes or partials, for example:

  #+Title:        Create Websites with Emacs: Blogging with Org mode and Hugo
  #+date:         [2021-05-07 Fri]
  #+lastmod:      [2023-09-09 Sat]
  #+categories[]: Productivity
  #+tags[]:       Emacs Hugo
  #+images[]:     /images/emacs/emacs-hugo.jpg
  #+weight:       62
  #+example:      Hello world!

Any line that does not start with #+ ends the front matter.

Keyword values can be either strings (#+key: value), or a whitespace-separated list of strings (#+key[]: value_1 value_2). If you need to add a value with two words, you will need to use a hyphen.

Add single keywords to your template with, for example with {{- .Params.example =-}}, which renders as "Hello world!"

When working with a list, spaces act as a delimiter, so you cannot use tags with two words without a workaround, for example hyphenation ("Hello-World").

If you are handy with Hugo templates, then you can easily remove the hyphens. Modify your Hugo template files that call the relevant list variables to replace any hyphens with spaces:

{{- range .Params.tags }}
{{ replace . "-" " " }}
{{- end }}

You can instruct Emacs to automatically add a modification timestamp by adding the following code to your init file:

  ;; Update Org files with last modified date when #+lastmod:
  (setq time-stamp-active t
        time-stamp-start "#\\+lastmod:[ \t]*"
        time-stamp-end "$"
        time-stamp-format "[%04Y-%02m-%02d %a]")
  (add-hook 'before-save-hook 'time-stamp nil)

This hook will look for #+lastmod: in your file and add the current date. Hugo will recognise the lastmod variable, and if your template supports it, it will insert this date onto your page.

Internal Links

Org mode includes the ability to link to other pages. Unfortunately, you cannot use standard Org mode or Denote links in Hugo for two reasons.

Firstly, in Hugo, the slug of a post is not necessarily the same as its relative filename. When using the #+slug: variable in the front matter, the URL will not match the filename.

Secondly, the file's location on your drive might be in a different folder than it appears on the website.

Hugo uses the ref shortcode to manage these issues. This shortcode inserts the URL of the post into the link. When you move a post to another folder, Hugo finds it and creates the appropriate hyperlink. Hugo will throw an error when it cannot find your file during rendering.

You can create links between your pages using standard org mode syntax and the ref Hugo shortcode, which looks like this:

{{< ref filename.org >}}

Use the C-c C-l shortcut (org-insert-link) and enter the shortcode and a description. The source code of the link will look like this:

[[{{< ref filename.org >}}][Description]]

To find the filename of the post you want to link to, open the Dired file manager with C-x d and find your post. Press w to copy the filename and paste it into the link.

When the target is an index page, then you need to add the sub-folders starting with a forward slash and remove the _index.org filename. Please note that paths with a non-alphanumeric symbol, such as a directory separator /, need to be quoted.

[[{{< ref "/tags/emacs" >}}][Index Page]]

Org mode has flexible options to create special types of links. We can leverage this functionality to simplify creating internal Hugo links. Creating a new Org mode link type with some Lisp makes life slightly easier. Add the code below to your Emacs configuration.

This capability consists of three bespoke functions. The ews-get-hugo-directory and ews-hugo-list-content functions check whether the current open buffer is part of a Hugo project by locating the configuration file. If this is the case, then it presents a list of all articles and pages in your website. The ews-hugo-link-complete function lets you select a page or post of your website to link to. This function also manages the specific requirements for index pages.

Press C-l to create a link, and type hugo: and enter. Pick a file, hit enter, provide a label and hit enter again.

  ;; Create Hugo links
  (defun ews-get-hugo-directory ()
    "Lists the directory of the current Hugo website or nil."
    (when (string-match "\\(.*\\)content" default-directory)
      (match-string 1 default-directory)))

  (defun ews-hugo-list-content ()
    "List the content of the Hugo website of the current buffer.
  Return an error message when not in an apparent Hugo directory."
    (if-let* ((hugo-dir (ews-get-hugo-directory))
              (hugo-p (directory-files hugo-dir nil "^config\\..*"))
              (content-dir (concat hugo-dir "content/")))
        (let ((org-files (directory-files-recursively content-dir "\\.org\\'"))
              (md-files (directory-files-recursively content-dir "\\.md\\'")))
          (append org-files md-files))
      (user-error "Not in a Hugo buffer")))

  (defun ews-hugo-link-complete ()
    "Complete a Hugo weblink through the `org-insert-link' and hugo: hyperlink type."
    (let* ((posts (ews-hugo-list-content))
           (titles (mapcar (lambda (post)
                             (string-remove-prefix
                              (concat (ews-get-hugo-directory)
                                      "content/") post)) posts))
           (selection (completing-read "Choose page:" titles))
           (target (concat "/"
                           (replace-regexp-in-string
                            "_index.*" "" selection))))
      (when titles
        (concat "{{< ref \"" target "\" >}}"))))

  ;; New link type for Org-Hugo internal links
  (org-link-set-parameters
   "hugo"
   :complete #'ews-hugo-link-complete)

This configuration only defines how to create a link. Unfortunately, this approach does not allow you to follow these links as Org mode can only recognise links that start with a string and a colon. e.g. hugo:, while Hugo links start with curly braces.

Images

The standard image syntax in Org mode works fine, but you must link the image to its published destination, not the current one. Hugo keeps images in the static folder, so all your image paths assume this folder as the root. Hugo will convert these links to the full URL.

Unfortunately, you cannot preview images in Org mode because you are linking to the website folder, not the Org mode folder. An image stored in <hugo-directory>/static/images/picture.emacs.png will be listed as /images/picture.emacs.png.

Hugo will parse the usual metadata, such as captions, alternative text, title and width. The example below shows how to style an image with alt and title tags and set its width.

  #+attr_html: alt: Alternative text title: Image Title :width 800
  #+caption: Caption text.
  [[/images/file-path.png]]

Items and drawers

When writing in Org mode it can be useful to track progress and add notes inside drawers. The standard Org mode export function can be configured to ignore TODO tags and drawers in the export. Because working with Hugo bypasses the Org Mode export functionality, TODO tags will appear in your HTML files. Hugo ignores property drawers but includes the content of any other types of drawers.

Limitations

Using Hugo directly with Org mode has some limitations, mentioned above. The main limitation is that it bypasses the Org mode export engine which means that some functionality is unavailable. Org mode macros cannot be used, but you can Hugo shortcodes to insert parametrised bits of text or HTML in your website. For example: {{< youtube w7Ft2ymGmfc >}} embeds a YouTube video with the relevant ID in your page.

Bypassing the export engine also means that any Org mode export variables are ignored, for example to exclude TODO tags or drawers from your website.

To insert images you either use the figure shortcode or the method shown above. Unfortunately you cannot preview these images as you need to enter the image location on your website and not on your computer.

Lastly, links need to use the internal linking shortcode. The configuration shown above helps you insert links, but you cannot follow them.

These limitations have not stopped me to use Hugo and Org mode as described in this article to develop this and other websites.

Using ox-hugo

The ox-hugo package in Emacs can act as an intermediary between Org mode and Hugo. This package converts the Org files in your content folder into Markdown files, which Hugo will render.

This package has some advantages over using the plain Org mode method because it relies on the standard Org mode export functionality.

However, I don’t use ox-hugo because I like the flexibility of the plain Org Mode approach. The minor limitations of the native Org mode rendering in Hugo have not prevented me from maintaining my websites. But Emacs is a matter of choice, so feel free to use ox-hugo.

Deploying your website

After adding some content, you can continue following the Hugo Quick Start Guide instructions to run the internal webserver or generate the entire website.

You don’t need to leave Emacs to run these console commands. When you execute the shell function (M-x shell <enter>), you can run Hugo from within Emacs. You will need to execute these files from within the folder that contains your website.

hugo server -D # Run internal web server, showing draft pages

All buffers in the content folder must be saved to disk; otherwise, Hugo will throw an error.

When using the server option in the Hugo command line, the website is available in your local browser under https://localhost:1313.

Exporting your WordPress website to Org mode

Moving a WordPress website to Hugo can be an enormous task that can be semi-automated. You can export the content from WordPress to a CSV file to convert the content to either Org mode or Markdown. I have written some scripts to export my former WordPress websites.

Emacs Writing Studio

Emacs Writing Studio consists of a series articles and a configuration to help you publish articles, books, movie and theatre scripts and websites. You can find the most recent configuration files on GitHub:

Emacs is a malleable system, so everybody will have their personal preferences of how to undertake a task. Any article on how to be productive with Emacs is thus opinionated. If you have a different way of doing things, please share your views and leave a comment below, or complete the contact form to send me an email.

The next article in this series discusses how to manage your projects and get things done with Org mode.

Getting Things Done with Emacs: Manage your life with Org Mode

Peter Prevos — Sat, 08 May 2021 00:00:00 +0000

The market for productivity applications is saturated with apps and other complex systems to manage your actions. While these systems provide great functionality, the main problem is that you have to use them the way the designer intended instead of how you like to work. This article shows how to use Org mode to get things done with Emacs and configure a bespoke productivity system.

The approach in the article roughly follows the concepts of David Allen's Getting Things Done (GTD) system. This is, however, not the only method to manage your affairs. Whatever your preferred workflow looks like, you can configure Emacs to help you with that.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

The productivity quest

The quest to become more productive has become a popular past time. Millions of people around the world struggle with the demands of life in the twenty-first century and they constantly look for ways to cram more activities into their day. Self-proclaimed productivity gurus have published piles of popular books about methods to get more done in your limited available time.

All these methods boil down to three basic phases: set a goal, define the actions to achieve that goal and undertake these actions. These principles might sound simplistic, but it is the basic truth. The plethora of methods merely discuss different ways on how to manage these three steps effectively and efficiently. A central theme of these methods are means to process the enormous amount of information that people are exposed to every day.

In the slipstream of the productivity gurus, software developers have released many apps to manage the information stream of our daily lives. The internet contains many stories about people who moved from one application to the next, in search of the perfect way to manage their projects and tasks. I was one of those people, until I started using Emacs.

Org mode is an ideal system to help you getting things done. The Org mode package not only allows you to write prose, it is also helps you to manage actions and projects. As with any other Emacs package, it provides virtually unlimited freedom to implement your favourite method to get stuff done.

This chapter shows how to use Org Mode to manage your projects and tasks, loosely based on David Allen's Getting Things Done (GTD) method. David Allen describes iterative five steps to become more productive:

Capture: Empty your mind
Clarify: Describe what it all means
Organise Place it where it belongs
Reflect: Reflect on your progress
Engage: Take action

Capture: empty your head

One of the reasons we are often not as productive as we like is because our minds are filled with issues. When our brain is full of thoughts about what we should be doing, we start to worry about how busy we are instead of doing the thing we need to do. David Allen says that "Our minds are for having ideas, not for holding them".

The first step to getting things done is to empty your mind. This is not a Buddhism-inspired quest for enlightenment, but a simple technique to help you focus. A brain only has a limited capacity as we can only have one thought at a time. When your brain is full of thoughts about what you should be doing, creative and productive thoughts are suppressed.

The other problem with keeping ideas in your head is the risk that they will disappear in the fog of your brain. I am sure you all recognise the experience of having the most wonderful idea when enjoying your morning shower, but half an hour later you are unable to recall your gem.

Open a new Org Mode file with the find-file command (C-x c-f). You can call it something like inbox.org, or whatever you fancy. For the next fifteen minutes, write down everything that is floating around in your head. Dump the contents of your brain into this virgin Org Mode file. Try not to multitask when you empty your mind, but fully focus your attention on the list.

Start every new idea with an asterisk so that they become headlines. Your list will contain a jumble of random things. From minor household tasks to big future projects you want to do one day. Don't filter your thoughts. Just write them down.

For the next twenty-five minutes, focus only on this task and write down everything that is in your head. Don't multitask and give this activity your undivided attention. Multitasking is the enemy of productivity because our brains can only focus on one intellectual activity at a time. The fact that magicians can so easily fool people illustrates why multitasking is a fool's errand. Perhaps listen to some music with the Emacs EMMS package to keep you focused.

Don't spend any time thinking about these tasks. Don't worry about when you'll need to do it or in what order they need to be done just yet. Ensure that your mind is empty by the time you complete your list. If the list only has about a dozen items, then your list is not complete. For most people, fifty actionable items, projects and fuzzy goals are not unusual. If you are overwhelmed by the list then don't shoot the messenger as it merely reflects your life. If it takes you longer than twenty five minutes to empty your mind, then take a five minute break and start again.

You now have a long list of all the stuff you need to get done. But that is only the starting point. Org mode also has functions that can capture tasks as they come up in random moments of the day.

* Mow the lawn
* Clean up the backyard
* Improve my job skills
* Learn how to use Emacs
* Write an ebook about ... (fill in your speciality)
* Empty your e-mail inbox
* Prepare presentation for the quarterly meeting next week
* And so on, and so on ...

Clarify: Describe what it all means

Did you notice that most things on the list above, apart from items 6 and 7, are more work than just one action?

In GTD-speak, they are projects, which are desired outcomes that take more than one action to complete.

Your list will be a mess of tasks, projects, goals and vague ideas. Your next task is to organise them. Firstly you can add some order and hierarchy to the list with the ALT and arrow keys. M-up and M-down will move a heading up or down, while M-right and M-left will promote or demote your entry.

With these four keystrokes, you can organise your list so that projects and tasks go together and create some order in the chaos that came from your mind. So the garden becomes a project with two tasks:

* Gardening
** Clean up the backyard
** Mow the lawn

If you like a bit more clarity on your list, you can collapse a heading with the TAB key. Using S-TAB will collapse the whole document to only the first-level headlines. Repeatedly using S-TAB will cycle through the full content, only the first headlines and all headlines.

You can also add notes or link images below any heading to provide some context to the task by hitting enter at the end of the headline and typing.

Your list is starting to take shape now. The following steps will add more context to your tasks. If you like do some more finessing, then read my article on writing prose with Org mode.

Organise: Place it where it belongs

After a week or so, your inbox will start filling with stuff. Your inbox is not just one location, but a collection of places where you gather information. Emacs Writing Studio uses the Org mode capture mechanism as an inbox, but it can also be a physical inbox to collect papers. A digital notebook on your phone or a physical diary to collect notes are also viable options.

My personal inbox consists of my Org mode inbox file, my email inbox, a cloud-based notes application in my phone a physical inbox and my physical diary. As part of my weekly review I promise myself to empty these locations and process them into my system using this flowchart, which is based on the classic GTD model.

Example of a productivity workflow using Org mode.

During your regular review, you assess each bit of information you collected. If it is not actionable, the you either ditch it, otherwise capture it in Denote as a permanent note.

If the new item is actionable and it only takes a few minutes to do, then just go ahead and do it; don't waste your time formally registering the action.

Lastly, if the inbox item is actionable, bur requires some time, you can add it as an action to your agenda file. The status of each action depends on external circumstances.

Action Status

The problem with most todo-lists is that you get overwhelmed by the amount of stuff to be done. But in reality, most actions don't need or can't be progressed. You can be more precise in your records and mark items as the next action to be undertaken, or mark the ones where you are waiting for somebody else.

Next: The next action to be taken.
Todo: Something to be done in the future (either scheduled or as yet undetermined)
Waiting: If you are waiting for somebody to do something, then mark it as such.

Org mode can associate each headline in a document with a workflow state. By default, there are only two states, TODO or DONE. You can change the state of a heading with the shift and arrow keys. Org mode will cycle between the two states. The keywords are commonly written in all caps, but that is not necessary.

You can define workflow states any way you like. Adding the following Lisp lines to your init file will set your workflow states. The states after the vertical bar will be marked as completed, usually coloured green.

If you have many states, cycling through them can be tedious. The C-c C-t command provides a popup menu to quickly select your option. You add the menu letter between parentheses after the keyword like this:

  (setq-defualt org-todo-keywords
                '((sequence "TODO(t)" "NEXT(n)" "WAITING(w)" "|" "DONE(d)" "CANCELLED(c)")))

Emacs Writing Studio roughly follows the GTD approach and have a status for next actions, waiting for others or future actions. When an action is completed, it is either done or cancelled. There are, in principle, no limitations to how you use this functionality.

The basic principle of the GTD approach is not to have massive lists of items that you like to do one day but that you define the next action that needs to be done to achieve your goal, or schedule activities in your diary. This method ensures that only a subset of activities is in your consciousness and you don't get overwhelmed by your inactivity as action lists tend to be long list of promises to our future selves.

You can set different keywords can for each Org file. When, for example, you are writing a book, you could set specific keywords for each heading with the following line in your header:

#+TODO: TODO(t) DRAFT(d) | DONE(c) EDITED(e)

Adding dates and deadlines

Now that your list is nicely organised in the stuff you need to do, add dates to some of the tasks. Scheduling a task to a specific date is to commit your recalcitrant future self to the action.

You can add the date that the task is scheduled or a deadline by which is should be completed. A scheduled date indicates the date or period you plan to work on the task. A deadline is the time the task needs to be completed.

To add a scheduled date, use C-c C-s when on a headline. Emacs will pop up a calendar that you can use to select a date. The shift and arrow buttons move the timestamp by day or week. The < and > symbols move you a month in time. Press enter when done, and the date appears below the headline. You can add a deadline in a similar way but with the C-c C-a keystroke.

Timestamps use the ISO 8601 format: year, month, day. This format avoids any confusion between American formats and the rest of the world. Editing a timestamp is easy. Place your cursor on either the year, month or date and use the arrow keys to move it up or down.

Scheduled dates can also set a regular schedule. For example, suppose you add, for instance, +7d at the end of the date. In that case, Org mode recalculates the date every time you complete the task and resets the status to TODO. You can also use the letters w, m and y for scheduling a job weekly, monthly or yearly. Some actions have to be undertaken weekly on the same day, but in some instances it is better to restart the clock every time it is completed.

In the example below, I have planned to complete the Org mode article on 8 May. My tax return has a deadline of 30 September and appears annually on that date. It is a deadline instead of a schedule because the date is enforced on me externally. The last action is my weekly review of the inboxes. In this case, the seven days are recalculated every time I complete the action. So if I complete the review on 13 May instead of 11 May, the new date will become 20 May.

  * TODO Complete Org-Mode article
  SCHEDULED: <2021-05-08 Sat>

  * TODO Submit tax return
  DEADLINE: <2021-09-30 Mon +1y>
  
  * TODO Weekly review
  SCHEDULED: <2021-05-11 Sun ++7d>

Only add a scheduled date if this is the time that you plan to do the action. Try not to add too many self-imposed schedules because you will over commit your day. A deadline is only helpful if there is an external expectation that you need to complete something by a specific day, for example, get travel insurance before your flight leaves.

Checklists

Not every action needs to be a headline todo item. Org mode allows you to add checkbox items anywhere in your document by adding [ ] after the list indicator. Using M-S enter after a tick box item will create a new box. You could use this functionality as a checklist for a larger action.

The snippet below is an example of a todo item with a deadline, some notes and a checklist.

  * TODO Submit tax return
  DEADLINE: <2021-09-30 Mon +1y>
  Tax accountant: 0407 555 283
  - [X] Collect records
  - [ ] Prepare overview
  - [ ] Set appointment with accountant

Ticking and unticking any items is as simple as hitting C-c C-c. You can convert a plain list item to a todo item, or vice versa, with C-u C-c c.

You can also convert list items to headings and back again. The org-ctrl-c-star function (C-c C-*) converts a paragraph to a heading. If the line contains a tick box, it becomes a TODO item. Using C-c C-- (org-ctrl-c-minus) converts a paragraph to a list item.

Reflect: Review your progress

The key of any productivity workflow is to regularly review your list of actions, your priorities and goals. The central tool in Org mode to look at your list of registered actions is the agenda.

The agenda provides a summary of all your headings with an action status. Firstly, it is common practice to bind C-c a to the agenda menu. You can assign one or more Org mode files that contain your actions. Org mode will read these files and present the todo items as an agenda.

The Org files used to construct the agenda are stored in org-agenda-files. You can figure this variable with some keystrokes.To add a file with actions to the agenda use org-agenda-file-to-front (C-c [) and remove it with C-c ] (org-remove-file).

When you evaluate the org-agenda function with C-c a, Org mode reads all agenda files and constructs an agenda, listing actions by date.

You can change the status of items from within the agenda or jump to the file that contains the action, so you can ready any context or maintain checklist items.

The agenda starts with a self-explanatory menu screen which you can explore. Org mode has extensive capabilities to configure how the agenda is presented, which are outside the scope of this article (See chapter 11 of the Org mode manual).

Engage: Take action

Unfortunately, Emacs cannot help you with mowing the lawn, go shopping or other physical tasks on your lists. Emacs can help you with any task that involves creating written content, but it can't mow the lawn for you.

In the end, there is no productivity system in the world that does the tasks for you, no matter what the gurus promise. However, bringing order into your tasks keeps you focussed on your goals.

Org mode configuration

This code snippet provides a basic configuration for managing actions and projects in Org mode. You can of course modify the sequence of your action statuses to whatever you please.

  (use-package org
    :custom
    (org-todo-keywords
     '((sequence "TODO(t)" "NEXT(n)" "WAITING(w)" "|" "DONE(d)" "CANCELLED(c)")))
    :bind
    (("C-c a" . org-agenda)))

Learning More

Getting Things Done is just one of many methods to manage your busy life. This article shows how to implement the principles of David Allan's method with Org mode. The beauty of Emacs is that you have the freedom to implement whatever method you prefer.

You can find the Org manual on TODO items with C-h R org RET g todo RET.

Bavarian Org mode guru Rainer König has developed a comprehensive series of YouTube videos that explain using Org for managing actions and projects in great detail. I also recommend following his Getting Yourself Organized with Org mode course on Udemy.

Org mode Tutorial by Rainer König.

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

The next article discusses how to manage your files with Emacs dired.

Manage Files with Emacs: Organise your drive with Dired

Peter Prevos — Sun, 25 Apr 2021 00:00:00 +0000

Working with Emacs means that you will need to access, create files and manage files on your drives. Emacs comes shipped with dired (pronounced dir-ed), a potent file manager. This article explains the basic principles of using dired and shows how to manage files with Emacs to organise your hard drive.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Accessing directories and files with dired

Dired is short for “directory editor”, but it can do much more than just that. This software has an illustrious history. The first version of dired was a stand-alone program written circa 1974, so its origins are even further back in time than Emacs. The software comes packaged with Emacs and it provides a front end to various GNU core utilities.

You activate the file manager with the dired function or the C-x C-d shortcut. When you hit enter, Emacs will create a buffer with the content of the directory of your open buffer. You can also enter another directory in the minibuffer before you hit the enter key. Another useful function is dired-jump (C-x C-j) which opens dired and jumps to the file linked to the current buffer. The q button closes the dired window but does not kill (remove) it.

You can navigate the content with the arrow keys and press j to jump to a specific file by entering part of its name.

The enter key opens the respective file or directory. To open a file or directory in another window, press o (Using C-o open the file in another window, but the cursor stays in Dired). The carat key ^ takes you to the parent folder.

Note that every time you open a new directory, Emacs will open a new dired buffer. After a while, you can litter your Emacs session with unused dired buffers. Pressing the a key opens a directory in the same buffer. This functionality is disabled by default because the Emacs developers strangely believe that new users find it confusing. Emacs Writing Studio configures this behaviour by default.

Copying and Moving Files

To copy a file, press the C button. Dired will ask for a new name and location in the minibuffer. To move a file, you press R because moving a file is the same as renaming it with a new directory.

There is no need to close the buffer before you rename an open file. Emacs will link the open buffer to the new filename.

If you have two open dired buffers, Emacs Writing Studio copies and moves from the folder in the active window to the other dired buffer.

Renaming files

Renaming the file is the same as moving it. So press R and type a new filename.

It is sometimes useful to copy the name of a file to the kill ring with the w key, so you can use it to rename the file. So to rename a file, copy the name with w, rename the file with R and paste the existing name with C-y and edit the name to your new version.

Removing files

You can select and deselect files for deletion (killed) with the d and u buttons. After you selected the files you like to delete, press x to execute the deletion. Press capital D if you like to remove a single file.

By default, Emacs permanently removes files. The Emacs Writing Studio is configured so that files are moved to the recycle bin.

When you delete or trash a currently open file, Emacs will also ask you to close the appropriate buffer.

Working on Multiple Files

You can select multiple files to work on at the same time by marking them. The m button marks a file, and the u removes the mark. The capital U removes all marks in the buffer. The t key reverses your markings, which is helpful when you want to select everything but one or two files.

This method requires you to manually select each file. You can also use regular expressions to select files. Regular expressions are a powerful set of logical rules to select sections of text. Simply put, regex is wildcards on steroids—press % m to open the regular expression selection prompt.

For example, ^2023.*_journal* selects all Denote files that start with the 2023 and that have the journal file tag. Now press t to invert the selection and k to remove the selected files from view. This sequence is a useful method to find related files.

After you selected multiple files in this manner, you can use all file commands to act on the selected targets, for example moving all 2023 files with the _journal tag to another folder.

Opening a file with another program

Emacs is a Swiss-Army chainsaw, but it cannot do everything. Sometimes you might like to open a file in other software, such as your image editor or video player.

You can open files with external software by pressing & after which dired will ask for the appropriate software. You need to type the name of the executable file of the software you like to use, e.g. gimp.

Manage files with Emacs dired

If your head is buzzing with all the different key bindings, the table lists the functionality described in this article. These are only a small snapshot of the functions of the directory editor in Emacs. You can press the h key while in a dired buffer to view all functionality and related keybindings.

Key	Function	Action
`a`	`dired-find-alternate-file`	Open folder in same buffer
`C`	`dired-do-copy`	Copy a file
`j`	`dired-goto-file`	Jump to a file
`g`	`revert-buffer`	Refresh the dired buffer
`m`	`dired-mark`	Mark file under cursor
`% m`	`dired-mark-files=regexp`	Mark by regular expression
`o`	`dired-find-file-other-window`	Open file in other window
`q`	`quit-window`	Close the buffer
`R`	`dired-do-rename`	Rename (move) a file
`t`	`dired-toggle-marks`	Inverse marked files
`u`	`dired-unmark`	Unmark file under cursor
`U`	`dired-unmark-all-marks`	Unmark all files
`&`	`dired-do-async-shell-command`	Open file with other program
`enter`	`dired-find-file`	Open file

Dired key bindings.

The dired package is a convenient and powerful tool to keep your drives organised and access your information. Developers have published an extensive collection of extensions to dired to add functionality, which you can find in the package manager.

Dired lists files and directories in alphabetical order. I prefer a different view, which shows directories on top and files below them. The parameters determine the order of the entries in the folder.

Removing hidden files from view

Your Dired view might be littered with hidden files and folders that start with a dot. You can remove these by marking them with a regular expression and removing these lines from view, but that is a bit awkward to do every time you open a Dired buffer. The Dired-X package provides functionality to hide files that meet a certain condition. The Emacs Writing Studio configuration for Dired omits all files staring with a dot.

  ;; Hide hidden files
  (use-package dired-hide-dotfiles
    :hook
    (dired-mode . dired-hide-dotfiles-mode)
    :bind
    (:map dired-mode-map ("." . dired-hide-dotfiles-mode)))

Dired Configuration

Emacs Writing Studio uses the following configuration for Dired:

  (use-package dired
    :ensure
    nil
    :commands
    (dired dired-jump)
    :init
    (with-eval-after-load 'dired (require 'dired-x))
    :config
    (put 'dired-find-alternate-file 'disabled nil)
    :custom
    (dired-listing-switches
     "-goah --group-directories-first --time-style=long-iso")
    (dired-dwim-target t)
    (delete-by-moving-to-trash t)
    (dired-omit-files (rx (seq bol "."))) ;; Omit dot files
    :bind
    (:map dired-mode-map ("." . dired-omit-mode))
    :hook
    (dired-mode . dired-omit-mode))

Backup files

This last bit of configuration code defines how Emacs deals with automated backups. The default setting is that the system stores these files in the folder where the original files lives, clutterin/g your drive with copies of your stuff. The setting below modifies the backup-directory-alist variable so that Emacs saves all backups in your configuration folder. This configuration also eliminates lock files, which are only useful when working in shared folders.

  ;; Backup files
  (setq-default backup-directory-alist
                `(("." . ,(expand-file-name "backups/" user-emacs-directory)))
                create-lockfiles nil)  ; No lock files

Alternatively, you could instruct Emacs to not save backups at all with (setq-default make-backup-files nil). I prefer keeping backups as they have saved my bacon a few times in the past.

Recent Files and Bookmarks

Whenever you return to Emacs you might want to open a file you were working on recently. The recent files minor mode (recentf-mode) provides a transient list of the files you most recently opened.

This minor mode saves the most recent opened files when you exit Emacs to a file in your configuration folder. However, it might be more useful to save the recent files regularly to ensure it is saved. The run-at-time function runs a function at a regular interval, in this case every five minutes. The recentf-edit-list function opens the file with your recent acquisitions and lets you delete selected files.

By default, the recent files mode stores the last twenty opened files, which you can change by modifying the recentf-max-saved-items variable.

  ;; Recent files
  (use-package recentf
    :config
    (recentf-mode t)
    (run-at-time nil (* 5 60) 'recentf-save-list)
    :custom
    (recentf-max-saved-items 50))

Recent files are transient as they are continuously updated as you open new files. For a more permanent list of files you like to open, use bookmarks.

You can store a file as a bookmark with C-x r m (bookmark-set). The bookmark will also store the location of the cursor, so you can maintain multiple bookmarks for a file. The default name for the bookmark is the name of the file. You can also enter a bespoke name in the minibuffer before hitting ENTER.

To view a list of all bookmarks in the minibuffer and select the one you like to open, use C-x r b (bookmark-jump).

Bookmarks are saved in the bookmarks file in your configuration folder every time a new bookmark is created. The bookmark-save-flag is set to one so that the bookmarks file is saved every time you add a new one. The default value only saves it when you exit Emacs, which mean you could loose bookmarks in the unlikely event of an Emacs or system crash.

If you like to remove bookmark no longer required then use the bookmark-delete function, which has no default keybinding but is bound to C-x r D in the Emacs Writing Studio configuration.

  ;; Bookmarks
  (use-package bookmark
    :custom
    (bookmark-save-flag 1)
    :bind
    ("C-x r D" . bookmark-delete))

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

Emacs is a malleable system, so everybody will have their personal preferences to undertake a task. Any article on how to be productive with Emacs is thus opinionated. If you have a different way of doing things, please share your views and leave a comment below, or complete the contact form to send me an email.

The next article in this series discusses how to use dired to view images.

Viewing Images with Emacs and the image-dired package

Peter Prevos — Sun, 25 Apr 2021 00:00:00 +0000

Emacs is a powerful text editor that lets you manage your life and be creative and productive with text. This focus on text does mean that Emacs cannot help you managing and editing images. This article shows how to view and manage images with Emacs using the image-mode and the built-in image-dired package by Mathias Dahl and Stefan Kangas.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Prerequisites

Image-mode and the image-dired package are bundled with Emacs, so there is no need to install packages, but you might need some additional software.

You can view images without external software, but you cannot manipulate them. The ImageMagick software suite provides functionality for editing and manipulating images.

Emacs Writing Studio checks whether the required software is available when it starts. Check the *Messages* buffer to see of there are any warnings.

Viewing images

Emacs can display various popular image formats out-of-the-box with image mode. You can open an image file directly with find-file or through the directory editor (dired). You can also open an image from within an Org file with C-c C-o (org-open-at-point) with the cursor on the image. Emacs automatically scales the image to snugly fit inside the display window.

A range of keyboard shortcuts are available to view images. The n and p keys (next and previous) or the left and right arrows flick though the images in the current directory.

Image mode also provides several commands to change the display size of images (I am unsure why the prefix key is s for some commands, but i for others, but alas):

s o: Show image at original size (when it doesn't fit in the window, scroll through the image with the arrow keys).
s w: Fit the current image to the height and width of the window.
i +: Increase the image size by 20%
i -: Decrease the image size by 20%.

Furthermore, image mode can manipulate images:

i r: Rotate the image by 90 degrees clockwise.
i h: Flip the image horizontally.
i v: Flip the image vertically.
i c: Crop the image.
i x: Cut a rectangle from the image and replace with black.

The crop and cut commands display a rectangular frame superimposed on the image. Use the mouse to move and resize the frame. Type m to move the frame instead of resizing it and type s to convert the frame to a square. When you are satisfied with the result, type enter to crop or cut the image. You can exit the crop and cutting menu with q without changing the source file. Please note that these commands are only available when ImageMagick is installed.

If you like to retain the result of the transformation, press i o to save the image under a new name.

When you are done with watching images, use q to quit the image buffer, or k to kill the image buffer altogether.

The image-dired package

Viewing images individually is great, but wouldn't it be nice if you cold see thumbnails before delving into your collection? The image-dired package provides a thumbnail buffer to view and maintain images from within a dired buffer using thumbnails.

Evaluate the image-dired function (bound to C-c w I in Emacs Writing Studio) and select the directory you like to use. Emacs splits the screen and presents a thumbnail screen (of up to a thousand entries) to explore your collection. Emacs stores the thumbnails in the configuration directory for future reference.

Viewing some travel photos in Emacs with image-dired.

Alternatively, when you are inside a Dired buffer that contains images, mark the images you like to view and generate the thumbnails with C-t d (image-dired-display-thumbs). If you don't mark any files, the program uses the image under the cursor.

Yet another method of previewing your images is by marking the ones you need and generate inline thumbnails inside the Dired buffer with C-t C-t (image-dired-dired-toggle-marked-thumbs). The same shortcut also removes the thumbnails.

The active image is marked with a flashing border around the thumbnail and its filename is displayed on the top of the thumbnail window.

You can navigate the thumbnails with the arrow keys. The < and > keys take you to the start or end of the collection. You can remove a thumbnail from the collection with C-d. If you have selected more images than thumbnails that can fit on page, then image-dired tracks your movement, so as your cursor moves up or down, the thumbnails refresh.

To view an image, hit enter when the thumbnail is marked. You cycle through the marked images in your collection with the space and backspace buttons, or C-<left> / C-<right>.

But why manually flick through your album if you can let Emacs do this for you? The S key starts a slideshow with each image shown five seconds by default. You can configure the delay with the image-dired-slideshow-delay variable, or drive the slideshow manually with the space and backspace keys, or C-<left> / C-<right>.

The main image display is in image mode, so all the actions described in the previous section apply.

As usual, q quits the image or thumbnail window.

Tagging images

The image-dired software can also create a plain text database of images with searchable tags and thumbnail descriptions.

You tag images directly from a dired buffer. The C-t t keystrokes lets you tag the selected files. You can retrieve the tags in a future session using C-t f and view the image thumbnails with C-t d, as described above. You can tag images from within a thumbnail buffer with the t t shortcut. You can also add a comment to the image by pressing c in the thumbnail viewer.

The file name, tags and comments show in the mini buffer as you move through the image thumbnails: directory: filename (tags): comment.

Emacs stores the metadata in the image-dired directory in your configuration folder in the .image-dired_db file. This is a plain text file that lists each file name and the tags an comment.

Retrieving tagged files only selects images with tags in the directory of the current dired buffer. It does not work across multiple folders.

This tagging system is nice, but it creates an integrity vulnerability in that it separates tags and files. When you accidentally remove the database, all metadata is lost. You will also loose access to the metadata when you rename an image.

An alternative method to tag files is to use the Denote file naming convention, which encodes metadata into the name of the file.

Using Denote to manage images

Denote is a package to produce and maintain plain text notes, but it can also be used to maintain a collection of binary (non-text) files. When using the Denote file naming convention for your images, you can mark parts of you collection using regular expressions. The Denote file naming convention encodes four bits of metadata about a file into the filename. Only the timestamp is compulsory and serves as a unique identifier:

Timestamp in ISO 8601 format
An alphanumeric signature (starting with double equality sign (==)
The title in kebab-case (starting with double dash ==)
Keywords in snake_case (starting with double underscore __)

For example, one photo in my collection is: 19930825T132000--forbidden-city-throne-palace-of-heavenly-purity__china.jpg. So this photo was taken on 25 August 1993, the title describes the topic of the photograph and

I can now go into Dired and mark files with regular expressions, searching for each of these fields. For example, using %m _china marks photos taken in China, or %m -city all images with the word "city" in their title. Using the Dired convention for naming files is not only great for notes, it also helps you manage your photo collection.

The denote-rename-file function (C-c w R) lets you construct a Denote-compliant file name for existing images or other binary files. This function uses the last modified date as the identifier by default. But when you are sorting old collections you might want to add a date that is different to the last modified one. Adding the universal argument (C-u) will also ask you to provide a date and time, i.e. (C-u C-c w R).

Editing Images

Emacs is a powerful piece of software but it is mostly limited to editing text. To meaningfully work with images, you will need another package. Image-dired lets you open images in external viewers or editors by setting one variable.

Pressing C-enter opens the file in an external viewer or editor. You can tell Emacs which external viewer to user by configuring the image-dired-external-viewer variable. I linked it to the GIMP (GNU Image Manipulation Program). The content of this variable obviously depends on your system and preferred viewer.

When you are in an image-dired buffer, you open the external viewer with C-enter. Confusingly, when you are in a normal dired buffer, this function is bound to C-t x. Emacs Writing Studio remaps the keys so that you can use C-enter in dired and in image-dired to open an image in your favourite external editor.

Configuration

Emacs image-mode and image-dired work out-of-the-box, but some configuration helps to add some convenience.

This configuration adds keyboard shortcuts to kill an image buffer, so your Emacs session is not littered with open pictures. It also lets you view your image gallery with the arrow keys instead of the non-intuitive space and backspace combination.

If you like reading more details about image-mode and its configuration options, type C-h r to open the Emacs manual, followed by g image-mode return.

  ;; Image viewer
  (use-package emacs
    :bind
    ((:map image-mode-map
                ("k" . image-kill-buffer)
                ("<right>" . image-next-file)
                ("<left>"  . image-previous-file))
     (:map dired-mode-map
      ("C-<return>" . image-dired-dired-display-external))))

The configuration for the image-dired package defines the external viewer and defines some convenience keybindings to flick through your albums.

To find out about further customisation options, type C-h r to open the Emacs manual, followed by g Image-Dired return.

  (use-package image-dired
    :custom
    (image-dired-external-viewer "gimp")
    (image-dired-thumb-margin 10)
    :bind
    (("C-c w I" . image-dired))
     (:map image-dired-thumbnail-mode-map
      ("C-<right>" . image-dired-display-next)
      ("C-<left>" . image-dired-display-previous)))

Emacs Writing Studio

If you like to support my work, then please purchase the Emacs Writing Studio book.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

You can find the source files for the book and the latest configuration files on GitHub:

This is the last article in the Emacs Writing Studio series.

Using Emacs on a Chromebook: An Installation Guide

Peter Prevos — Sat, 29 Aug 2020 00:00:00 +0000

One of the main advantages of free software is that it runs on almost any platform. Google's Chromebooks are somewhat of an exception because of its restricted application access. Emacs is not available as an application within the app store. However, the foundations of Google's Chromebook are Linux, which makes it an ideal platform to use Emacs. Although Emacs does not run natively on a Chromebook, there is an elegant solution to this problem.

Installing Emacs on a Chromebook is like a three-course meal. You start with a Crouton or some Crostini as an appetiser, install Emacs as the main course, and some configuration as the desert. This article explains how to install Emacs on your Chromebook laptop.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Linux on a Chromebook

Several open-source projects developed over the years enable running Linux on a Chromebook. Crouton (not to be confused with the bread sticks) installs a GNU/Linux operating system, such as Ubuntu, on your Chromebook. Linux will live side-by-side with Chrome OS and you can switch between the two with a keyboard shortcut without rebooting.

More recently, Google introduced a new Linux subsystem called Crostini, for its Chrome OS so that you can use all the great software it offers, including the venerable Emacs. Crostini is a play on words as it is a fancy version of a crouton. With Crostini, your Linux terminal is an application within the Chrome OS.

Click on the clock and select the settings icon. Scroll down to the Linux section and follow the prompts. Not all Chromebooks support this method yet. If your Chromebook does not include Crostini, then you'll have to use Crouton instead. Crostini is still beta software and some functionality, such as USB, might not be fully functional.

Install Emacs

Once you have access to either Crouton or Crostini, you can install Emacs in the usual way. Crostini uses a stable version of Debian, with Emacs 25 as the most recent version. Start the Linux terminal and type:

sudo apt-get install emacs25

Emacs will now be accessible from Chrome OS, without needing the terminal. You can start it just like any other Chrome software.

Writing this acticle with Emacs on the Chromebook.

Using Emacs with Crouton

The Linux system is a virtual machine within Chrome and has a separate file system. Files within the Linux container are available by default from the file browser. The user's Linux home folder is a folder within the Chromebook's file system.

If you want to access the Chrome file system within Emacs, you will need to share the Chrome folder with Linux. Right-click on any folder in the Chrome file browser and select "Share with Linux". The shared folders are available in /mnt/chromeos/MyFiles/ in the Linux container.

Chromebook keybindings

The keyboard on Chromebooks is a bit different from the normal PC or Apple. The top-row arrow keys are by default bound to next-buffer and previous-buffer functions. The ALT and up/down arrow keys map to page-up and page-down, which is annoying when you use Org mode.

The code below restores the behaviour of M-up and M-down in Org mode. When you use this keybinding, you will have to use M-v and C-v for scrolling while in Org mode.

  ;; Chromebook shortcuts
  (define-key org-mode-map (kbd "<prior>") 'org-metaup)
  (define-key org-mode-map (kbd "<next>") 'org-metadown)

Edit in 2026: For a more recent view of installing Emacs on a Chromebook, watch this video by Timothy W. Ukert.

How to Install Emacs on a Chromebook in 2025.

Chromebook Emacs as a writing companion

Emacs is the ideal software for a Chromebook. They are cheap lightweight computers that are perfect for writing on the go. Emacs provides the perfect distraction-free writing environment for anyone who writes as a professional activity or creative outlet.

You can download a configuration file focused on content creators from my GitHub page.

Emacs is a malleable system, so everybody will have their personal preferences. Any article on how to be productive with Emacs is thus opinionated. If you have a different way of doing things, please share your views and leave a comment below, or complete the contact form to send me an email.

Using Emacs on Windows 11: An Installation Guide

Peter Prevos — Fri, 20 Aug 2021 00:00:00 +0000

This article describes how to install Emacs on Windows 10, including supporting packages. This article is part of the Emacs Writing Studio which explains how to use Emacs to undertake research and write and publish articles, books, and websites.

Emacs Writing Studio

Payhip e-readers Paperback LeanPub

The source files of the book and EWS configuration are also freely available on GitHub.

Installing Emacs on Windows 11

The first step is to download Emacs for Windows directly from its makers on the GNU website. You need the emacs-$VERSION-x86_64-installer.exe, where version is currently 29.4. This file contains a 64-bit build of Emacs with dependencies as an installer package.

Download this file and run the program to install Emacs on your system. You can now run Emacs like any other Windows software.

To upgrade Emacs in future, simply repeat this process.

But wait there is more. Emacs is neatly integrated with Linux systems, but to make it function properly on Windows some additional configuration steps and additional software is required.

Directories

Home Directory

The next step is to set the default folder for Emacs by setting the HOME variable. This variable instructs Emacs to use the C:\users\username\ folder as your home location. Open Powershell in administrator mode and enter this line:

  setx HOME %USERPROFILE%

You can also set this variable through the Windows Control Panel. Search for "environment" and click Edit System Variables and select Environment variables. Create a new variable called HOME and select your home folder and click OK.

In Emacs, your home folder is indicated with a tilde symbol. So lets say that your home folder is C:\Users\peter\ and you need the Downloads folder, then in Emacs refers to this location as ~/Downloads.

Emacs configuration folder

The location of the configuration folder depends on your operating system and Emacs version. If you are unsure where to store the init.el file, then you can use the Emacs help functionality. In most cases it will be something like ~/.emacs.d/. The dot at the start of the folder name means that it is hidden from view. To view the content of this folder in Windows you need to enable "Show hidden items" in the file explorer.

Type C-h v for help on variables and type user-emacs-directory and enter. The help buffer that now appears provides the correct folder name.

Read my article on configuring Emacs for more information.

External Software

Emacs is not a text editor, it is a computing platform with a built-in text editor. Emacs can also act as an interface to other Open Source free software. Most of these packages are included in common Linux distributions. Windows users need to use a bit more effort to install these.

Reading documents and ebooks

Emacs is great at displaying plain text but needs some assistance with reading other formats. The basic emacs package to view documents other than plain text is Doc-View. The image below explains the external packages required to read ebooks and articles.

For PDF documents you need to install Ghostscript or MuPDF. The pdftotext program from the Poppler software helps with finding and copying text in a PDF.

To read and export to office documents (MS Office and LibreOffice), Emacs needs to have acces to the LibreOffice software. Emacs uses this software to convert office documents to PDF and display them with the DocView package. LibreOffice can also assist with exporting Org mode files to office formats or to PDF.

To view EPub files you need a version of the zip and unzip programs. You can find these programs, and many others, in the GnuWin32 project.

You also need to tell Emacs where these tools live, so you need to customise the exec-path variable.

Document conversion in Doc-View.

For more information on reading non-text files in Emacs see my article on reading ebooks.

Spellcheck

The Flyspell package performs spell checking on Emacs. Read the distraction-free writing for more information on how to configure spell-checking.

The ezwinports project maintains a large collection of useful open source software compiled for Windows computers, including Hunspell.

To install Hunspell on Windows, download the software from ezwinports. Extract the files in a folder on your drive, for example: C:\ProgramData\ezwinports\.

You need to add the bin folder to your path environment variable. A simple workaround is to add the full path of the Hunspell variable to your configuration, as such:

(setq-default ispell-program-name "C:/ProgramData/ezwinports/bin/hunspell")

Note that Emacs uses a forward slash /, not the Windows backslash \.

The binary distribution includes dictionaries for US English and UK English. You can find the dictionaries for other languages on the Internet; install them into share/hunspell directory in your Hunspell installation directory.

Using Emacs on Windows 11

This article only discussed how to install Emacs on Windows 11. The other articles on this website show you how to configure Emacs and how to use Emacs to research, write and publish articles, books and websites.

The EWS GitHub repository contains the latest information about the required external software.

Taking Notes with Emacs Org Mode and Org-Roam

Peter Prevos — Sat, 08 May 2021 00:00:00 +0000

Taking notes is an important activity for all creative work. Recording your thoughts and ideas or somebody else’s ideas is 2a foundational activity whether you write scientific articles, novels or whatever else. Emacs Org mode has some useful functionality to take notes, which can be further extended with the Org-Roam package. This article explains how to take notes in Emacs and configure the Org-roam note-taking system for Emacs.

Org Roam is a popular note-taking package that uses Org mode as its central repository. The disadvantage of this package is that it relies on a database and can only manage Org mode files. This page assumes that you are using the latest version of Emacs, which at the time of writing is 29.1, which has a built-in database. This page is no longer maintained as Emacs Writing Studio uses Denote for notetaking.

Single File or Personal Wiki

Taking notes in Org mode can be as easy as maintaining one or more files with many headers. Org mode has some functionality to tag each heading to help you categorise notes.

While this is a great start, keeping topical notes in separate files is a more productive method because you can easily link Org mode files to each other, creating a personal Wiki. To create a hyperlink between two files, press C-c C-l (org-insert-link) and type file: (Or add the universal argument C-u C-c C-l to choose file links by default). You can then select the file with your completion system and type a description, which creates a hyperlink. You follow these links by pressing C-c C-o.

This method allows you to navigate from one file to the next, but Org mode does not know which other files link to your current buffer. There is no database in vanilla Emacs that allows you to review the structure of your linked files.

The Org-Roam package extends the Org mode functionality by saving all the links between files and headings in a database. This method allows you to also view the backlinks. The database stores a list of nodes, which are files or registered headings. This additional functionality allows you to navigate through your network of notes like you would in a personal wiki.

Zettelkasten: Structured collection of notes

Org-Roam implements the principles of the Zettelkasten method as a digital slip box full of cards. German sociologist Niklas Luhmann maintained a collection of tens of thousands notes, each written on an index card. These cards were linked through a series of keywords. This method allowed Luhmann to connect disparate ideas and writes his extensive oeuvre of books and articles.

Zettelkasten Note-Taking Method: Simply Explained.

The Org-Roam package in Emacs creates a network of links between your notes and manages your digital knowledge garden. Basically, the package extends Org mode functionality by storing all nodes (files and registered headings) in a database, including which between nodes. Furthermore, Org-roam provides each node in your knowledge network with a unique identifier consisting of a long string of numbers and letters, e.g. 711a7cac-e46b-4e64-b1e6-2940fa431e55.

The Org-Roam database provides excellent opportunities to analyse your network and discover patterns in your research that were perhaps previously invisible. You can examine this database and visualise its structure. The graph below is created with the igraph package in R.

My personal Org-roam notes network.

The package provides a lot of freedom to structure your personal Wiki, and this article describes the basic functionality. However, before we can get started, we need to configure the system.

Configuring Org-roam

Install database

The Org-roam package stores a summary of all nodes in your network in a database. In Emacs version 29.1, this functionality comes out of the box and no further configuration is required.

If you are using an older version of Emacs, then Org-Roam depends on the EmacSQL package to connect with the database. During package installation EmacSQL needs to compile a custom native binary for communicating with the SQLite database.

To enable this task, you will need to install a compiler on your system. On GNU/Linux systems, including Chromebook, you need to instal the GCC compiler through your favourite package manager.

On Windows you need to install Chocolatey. Open PowerShell as administrator and follow he instructions on the Chocolatey website. After Chocolatey is available, you can install the compiler in Powershell as administrator with:

choco install mingw

When Emacs configures the Org-Roam package, it will try to compile the database connector. This might take a little while, but is only done once.

Configure Org-Roam

The Org-roam package requires some configuration to get it to work. At a minimum, we need to activate the package, bind some keys, define the folder where your notes and database will live and set a template for new notes.

You will also need to install the use-package package, which the configure Emacs article discusses. Alternatively, you can download the complete init file for this series of articles from GitHub and store it in your configuration folder.

You will also need to install use-package, which is explained article on how to configure Emacs.

  ;; Org-Roam basic configuration
  (setq org-directory (concat (getenv "HOME") "/Documents/org-roam/"))

  (use-package org-roam
    :after org
    :custom
    (org-roam-directory (file-truename org-directory))
    :init
    (org-roam-setup)
    :bind (("C-c n f" . org-roam-node-find)
           ("C-c n r" . org-roam-node-random)		    
           (:map org-mode-map
                 (("C-c n i" . org-roam-node-insert)
                  ("C-c n o" . org-id-get-create)
                  ("C-c n t" . org-roam-tag-add)
                  ("C-c n a" . org-roam-alias-add)
                  ("C-c n l" . org-roam-buffer-toggle)))))

The first line defines the default file for Org mode files. I store my Org-Roam notes in /Documents/notes/ in my home folder. You need to change the name of the folder to your personal preference. The (getenv ““HOME””) part of this configuration is your home folder, so you only need to modify the relative path between quotation marks.

The org-roam-setup function initiates the database, which can take a while the first time round if you have many existing notes.

Lastly, the configuration defines the keyboard shortcuts. The first three shortcuts are added to the overall keyboard map, so you can activate it from anywhere in Emacs. The last five lines are only valid when you are in an open Org mode buffer. Note that all shortcuts start with C-c n to provide consistency.

Creating and Taking Notes

We are finally ready to write notes. You add a new note with the org-roam-node-find function (C-c n f). You can type the name of an existing note or of a new note. The minibuffer completion system, which we configured earlier, shows a list of available nodes and filters options as you type.

When you create a new node, it will open in an Org mode capture screen. You complete the node with C-c C-c and cancel it with C-c C-k. You can read these instructions in the top of the popup screen. When you complete the node, Org-Roam will construct a filename and save it to disk. The filename starts with a timestamp, followed by the name of the note, separated by dashes. There is need to worry too much about the filename because you will be retrieving nodes through their title and tags, rather than a filename.

An Org-Roam file (called a node) is just another Org mode file with some minor differences. The file starts with a properties drawer that defines the file ID. Using an ID instead of a file name makes it easier to maintain link integrity when you change file names or titles. A node can also contain one or more aliases, so you can search on multiple names.

The remainder of the file contains the header and content. Note the #+filetags: property. A file tag is a way to add tags to files so you can easily group entries. The tags are displayed after the title or alias when you find a new or existing file.

When you first create a new file, a capture screen appears where you can write your note. To save the note to your system, press C-c C-c, to cancel creating a new note, press C-c C-k.

  :PROPERTIES:
  :ID: f1bdb099-2184-46cf-a952-4808baa00b09
  :ROAM_ALIASES: "Note taking with Org-roam"
  :END:
  #+title: Zettelkasten method with Org-roam
  #+filetags: :notes::emacs:

  * First headline

If this file exists in your database, you will be able to find it through its title, alias or tag. An alias is useful for registering synonyms of words or definitions of abbreviations (AI / Artificial Intelligence). File tags help you cluster notes by topic.

Your note starts below your header. If you are new to Org-mode, then read the article on distraction-free writing.

Org-Roam Nodes

In Org-roam terminology, a file or a registered heading is a node in a network. You can promote a heading inside a file to a node by pressing C-c n o (org-id-get-create). This will create a property drawer below the heading with an id tag. Use the TAB key to open or close the drawer. Drawers are an Org mode functionality to store metadata about files or headings.

Org-roam indexes every node (file or heading with an id) that resides in its designated folder and subfolders in a database. This database contains metadata about the node, such as its title and any links from and to the node.

Tags and Aliases

To make it easier to group and find nodes, you can give it one or more alias titles with the C-c n a shortcut (org-roam-alias-add). The main and the alias titles are searchable when finding a node. The name of the alias stored in the drawer at the start of the file.

To add one or more tags to a Org-Roam file, use org-roam-tag-add (C-c n t). Note that tags have completion using the TAB key. Tags are also searchable when visiting a node. Tags are a way to add structure to your nodes by adding them to a group. Adding a tag to a heading is as easy as hitting C-c C-c, which is a standard Org mode function.

Linking Nodes

The second method to add structure is by linking nodes. You can start writing your note as per usual. Links to existing files are easy to insert with the C-c n i shortcut (org-roam-node-insert). Search for the file you want to link to and hit enter.

As your nodes grow in number, you slowly build a personal knowledge network. Org-Roam has graphing capabilities, provided you have the Graphviz package installed, which is outside the scope of this article. The org-roam-ui package provides an interactive graphical interface to your node network through your browser.

Discovering Nodes

If you like to know which files link to the buffer you are currently editing, then use org-rosm-buffer-toggle (C-c n l). A new window appears with a list of all files that link to the current buffer. Move to the window with org-open-at-point (C-c o), select your node, and hit ENTER to open the note. To close the backlinks window, use C-c n l again.

My favourite function is to open a random note. Pressing C-c n r (org-roam-node-random) opens a note and invite you to take a random walk through your knowledge garden.

Note Templates

The default new node only has a property drawer and a title and the filename starts with a timestamp. You can of course change this default behaviour by setting the org-roam-capture-templates variable.

For example, this configuration creates a new default node with a filename and a template head (file+head).

The file name is based on the title slug (${slug}.org), which is basically the node title in lower case, with spaces replaced by dashes.

The first lines of the head of this node contain the title, the creation date and a #+lastmod: entry, which you can use to store the date it was last saved. To make this happen, you also set the time-stamp-start variable.

The immediate-finish option bypasses the capture system and creates a new file without .

  (setq org-roam-capture-templates
        '(("d" "default" plain "%?"
           :if-new
           (file+head "${slug}.org"
                      "#+title: ${title}\n#+date: %u\n#+lastmod: \n\n")
           :immediate-finish t))
        time-stamp-start "#\\+lastmod: [\t]*")

The template functionality in Org-Roam is extensive, so I will leave it at this example. You can add additional lines to your node template after the title or lastmod entry in this example. The \n indicator denotes a new line.!

Bibliographic notes with Org-Roam

Managing a large collection of literature, electronic files and associated notes is one thing, but if you can’t quickly find them then you might as well not have them at all.

The Helm completion system provides a useful window to your bibliographies. The Helm BibTeX menu lets you search entries and act on the results. The C-z key provides a menu for further actions, such as viewing pdf files or adding and editing notes.

Emacs will open any associated PDF files. When you select to take notes, Emacs will create a separate Org mode file for the selected reference. When you also use Org-Roam for taking notes in Emacs, then you have a complete research management system that creates hyperlinked and tagged bibliographical notes.

The setup below provides a minimal configuration that uses the folder names defined above. Use C-c n B to start Helm BibTeX.

  (use-package helm-bibtex
    :config
    (setq bibtex-completion-bibliography bib-files-directory
          bibtex-completion-library-path pdf-files-directory
          bibtex-completion-pdf-field "File"
          bibtex-completion-notes-path org-directory
          bibtex-completion-additional-search-fields '(keywords))
    :bind
    (("C-c n B" . helm-bibtex)))

The paths in the file field in a BibTeX file can contain one or more filenames, separated by a semi-colon. When you start the filename with a colon, you can use a relative path, e.g.: file = {:/path/to/article.pdf} is the same as ~/Documents/bibliography/pdf/path/to/article.pdf, i.e. the colon symbol stands for the content of the pdf-files-directory variable we defined in the previous section.

The bibtex-completion-additional-search-fields includes the keyword as a search field, which makes it a bit easier to find stuff.

The first time you start Helm BibTeX might take a few seconds as it parses all entries in the collection of files, depending on the size of your bibliography. Just like all Helm applications, the interface uses incremental search, so just start typing and the bibliography will narrow.

The helm-bibtex website provides detailed information about further functionality.

Org-Roam BibTex Package

The org Roam database can also connect to a bibliography in the form of a collection of BibTeX files. This last configuration connects your bibliography to the Org-Roam database. The Org-Roam BibTeX package allows you to create notes directly from your bibliography. The only difference between a regular note and a bibliographic one is that the properties drawer contains a citation link in the ROAM_REFS property, as shown in the example below.

:PROPERTIES:
:ID:       c037d9f7-1aad-4774-8f8e-d5974de23b8c
:ROAM_REFS: cite:harris_customer_2001
:END:
#+title: Customer participation in retail service: Lessons from Brecht

You can view the associated PDF file from anywhere in the document with the C-c n b shortcut, which will open a menu with options.

  (use-package org-roam-bibtex
    :after (org-roam helm-bibtex)
    :bind (:map org-mode-map ("C-c n b" . orb-note-actions))
    :config
    (require 'org-ref))
    (org-roam-bibtex-mode)

Taking Notes with Org Roam

This is a short introduction to getting you started with Org-roam. There are lots of opinions on how to structure your personal knowledge network. My approach is organic, and I have imported all text files stored in various places in the Org-roam folder. I now have my third brain hemisphere available at my fingertips, thanks to the great work of the Emacs, Org mode, Org-Roam and Deft developers.

The following article in this series outlines how to manage bibliographies in Emacs and link these to Org-roam.

All the configuration explained in this series of articles is available as an init file on GitHub.

Emacs is a malleable system, so everybody will have their personal preferences for undertaking a task. Any article on how to be productive with Emacs is thus opinionated. If you have a different way of doing things, please share your views and leave a comment below, or complete the contact form to send me an email.

Simulating Text Files with R to Test the Emacs Denote Package

Peter Prevos — Thu, 28 Jul 2022 00:00:00 +0000

Emacs is the most user friendly piece of software ever invented by humanity. I use it for 90% of my computing tasks, including keeping my digital knowledge garden with notes. Several notes packages exist in the Emacs ecosystem, with Org Roam as the most popular and fully-featured. I have used this package for a while now, but it relies on a database and has grown a feature set far beyond my needs.

Protesilaos (Prot) Stavrou developed the Denote package that goes back to the basics of Emacs. The defining feature of this package is a file-naming convention that acts as metadata to find your notes. The basic structure is:

YYYMMDDTHHMMSS==signature--file-name__keyword1_keyword2.extension.

The filename starts with a timestamp at one second resolution to ensure unique file names (unless you create more than one per second). This timestamp also acts as the unique identifier to link notes.

The timestamp is followed by two dashes and the sluggified file name. Two underscores after the file name indicate the start of the keywords, separated by one underscore. Users can also add a signature, which is denoted with two equals signs.

This convention provides a convenient heuristics to find notes based on dates, title and keywords. Denote supports either Org mode, plain text or Markdown files.

The simplicity of Denote allows for it to be easily integrated with other Emacs packages and it can be easily extended with some Emacs Lisp code. My Denote Explore package is an example of a set of auxiliary functions to help find notes.

I considered moving away from Org Roam to the monastic simplicity of Denote. But before converting my existing knowledge base, I wanted to see how it behaves with thousands of files in a single folder. Rather then converting my existing files, I decided to generate some random files to see how it performs.

Generating Random Text Files for Emacs Denote

My coding chops in R are much better than Emacs Lisp, so I decided to write some R code to generate random text files and take Denote through its paces.

This code uses the Collins Scrabble Word list to generate random file names and keywords. Download this file to your working directory before using this code. The code reads the file and generates a set of 50 keywords. Random timestamps are set somewhere in the distant future. Each file has a template for the front matter.

  ## Simulate n files in denote folder

  ## Initiation
  library(stringr)

  n <- 10000
  k <- 50

  wordlist <- readLines("collins-scrabble-words-2019.txt")
  wordlist <- tolower(words)
  tag_words <- sample(words[nchar(wordlist) <= 5], k)
  timestamps <- Sys.time() + sample(600E6:666E6, n)
  template <- c("#+title:      ",
                "#+date:       ",
                "#+filetags:   ",
                "#+identifier: ")
  denote_directory <- "~/denote-sim/"
  dir.create(denote_directory)

This next code snippet generates $n$ Org mode files in the denote_directory folder. Titles are extracted by sampling the word list and the tags (keywords) are sampled from the $k$ defined tags. The front matter includes the tile, the creation date, keywords (called filetags in Org mode) and the identifier.

The Lorem Ipsum generator in the stringr package generates some paragraphs of text. The last part of the code generates some links to random posts.

    ## Generate n random posts

    for(i in 1:n) {
      title <- paste(sample(wordlist, sample(2:5, 1)), collapse = "-")
      tags <- paste(sample(tag_words, sample(4, 1)), collapse = "_")
      identifier <- format(timestamps[i], "%Y%m%dT%H%M%S")
      front_matter <- c(paste0(template[1],
                               str_to_title(str_replace_all(title,
                                                            "-", " "))),
                        paste0(template[2],
                               paste0("[", format(timestamps[i],
                                                  "%F %a %H:%M"), "]")),
                        paste0(template[3],
                               paste0(":", str_replace_all(tags,
                                                           "_", ":"), ":")),
                        paste0(template[4], identifier))
      links_list <- vector()
      for (j in 1:(sample(1:5, 1))) {
        links_list[j] <- paste0("- ", "[[denote:",
                                sample(format(timestamps, "%Y%m%dT%H%M%S"), 1), "]]")
      }

      content <- c(front_matter,
                   "",
                   stringi::stri_rand_lipsum(1),
                   "",
                   paste("*", str_to_title(paste(sample(wordlist,
                                                        sample(1:3, 1)),
                                                 collapse = " "))),
                   links_list)
      filename <- paste0(denote_directory, identifier, "--",
                         title, "__", tags, ".org")
    
      writeLines(content, filename)
    }

Generating thousands of files will take a few minutes …

This codes generates ten thousand notes to test the Denote package to review it if works at a large scale. This tests shows that Prot's approach is perfectly capable of working with thousands of notes. Just for kicks, I also synchronised these files with an Org Roam setup. My laptop struggled with the computational load. I was unable to properly access the files as with the large number of files and struggled accessing files. So case, closed - I am moving to Denote and teach myself more Emacs Lisp to build my ideal Zettelkasten.

Postscriptum

Since I wrote this post a few months ago I have migrated all my Org Roam notes to Denote. I migrated my 2,000 notes manually as I wanted to reread all my material and immerse myself in my old ideas to generate new ones. Paraphrasing Scocrates: gaining knowledge is the art of remembering.

Denote plays a central role in my Emacs Writing Studio configuration. This starter kit is specially designed for authors who have no need to develop software.

If you don't feel like spending the time to migrate manually, then other Emacs enthusiasts have developed automated methods:

Jeremy Friesen, Exploring the Denote Emacs Package
Jeremy Friesen, Migration Plan for Org-Roam Notes to Denote
Charanjit Singh, Notes Migrator

Plain Text Visualisation: The Emacs User Survey Results

Peter Prevos — Sat, 14 Jan 2023 00:00:00 +0000

The Emacs User Survey was held in 2020 and 2022. It is a bit ironic that the data was neither collected nor analysed with Emacs. This article shows how to visualise the Emacs user survey results with plain text inside of Emacs. First, we download the CSV file and then parse it into an association list. With some bespoke functions and the quirky chart package that forms part of core Emacs.

Read the cleaned CSV file

Before visualising the data, let's declare some required packages and download the 2020 Emacs user survey data.

  (require 'csv)
  (require 'f)
  (require 'dash)
  (require 'chart)

  (url-copy-file
   "https://emacs-survey.netlify.app/2020/Emacs-User-Survey-2020-clean.csv"
   "emacs-survey.csv" 1)

Define some functions

The CSV package by Ulf jasper provides functionality to parse a CSV file buffer into a list of alists. Unfortunately, this package has no functionality to read CSV files or extract data, so I wrote a few additional functions. Thanks to u/deaddyfreddy for helping with wrangling association lists. I initially tried chatGPT to write some elisp code, but that was a disaster. Glad to see that human intelligence still trumps the silicon variant.

The csv-parse-file function parses a CSV file into a list of alists. The csv-extract-column-* functions extract the values from a column by number or header. The result of these functions is a simple list that can be visualised.

  (defun csv-parse-file (file)
    "Read CSV FILE and parse its contents."
    (with-temp-buffer
      (insert-file-contents file)
      (csv-parse-buffer t)))

  (defun csv-extract-column-number (n csv)
    "Extract values in column N from parsed CSV file into an alist."
    (mapcar #'cdr
            (seq-map (lambda (list) (nth (- n 1) list)) csv)))

  (defun csv-extract-column-name (name csv)
    "Extract the values in a column with NAME from parsed CSV into a list."
    (mapcar #'cdr
            (seq-map (apply-partially #'assoc name) csv)))

Visualise the Emacs User Survey Results

Visualising data with in plain text is a lost art. The famous WOW signal is our best candidate for a signal from an extraterrestrial civilisation. No fancy visualisations like we are used from radio astronomy, but a simple plain text data stream printed on chain paper. Computer users were creative with plain text and there is still a cult following of ASCII art, images that only use plain text characters.

Wow Signal (North American Astrophysical Observatory).

We are now spoilt with graphical displays, but there are still some packages that can visualise data with plain text, including Eric Ludlam's chart package. This code can draw coloured bar charts in plain text. The chart-bar-quicky function takes several arguments, most notably the x and y variables for the chart. The n variable declares the number of bars to draw. We need some more helper functions to visualise the CSV data.

We want the bar height to count each unique item, so we also need a function to create a ordered frequency table. The visualose-frequency-table mashes everything together to create a bar chart.

  (defun create-frequency-table (data)
    "Generate an ordered frequency table from DATA."
    (sort (-frequencies data)
          (lambda (a b) (> (cdr a) (cdr b)))))

  (defun visualise-frequency-table (table n var title)
    "Create a bar chart from a frequency TABLE with top N entries.
  VAR and TITLE used for display."
    (chart-bar-quickie
     'horizontal
     title
     (mapcar #'car table) var
     (mapcar #'cdr table) "Frequency" n))

Now we are ready to visualise the data. Parsing the data can take a few moments as it contains thousands of results. The bar chart will pop up in a new buffer.

  (setq emacs-survey
        (csv-parse-file "emacs-survey.csv"))

  (visualise-frequency-table
   (create-frequency-table
    (csv-extract-column-number 6 emacs-survey))
   10
   "Responses" 
   "Which version of Emacs do you primarily use?")

Results of question six of the 2020 Emacs user Survey.

Conclusion

Visualising data with plain text is less aesthetic than using graphics, but it does the job of telling the story, which is the purpose of visualisation. There are other methods to create graphics inside an Emacs buffer. For example, one could use a literate programming approach and write an Org Mode file and run R, Python, MATLAB or GNUPLot code to create graphics. The latter also has an option to create graphics in plain text.

Let's hope the following Emacs user survey will be more Emacs-centric in its's data collection and presenting the results.

Emacs LISP exercises in Structure and Interpretation of Computer Programs: Section 1.1

Peter Prevos — Tue, 12 Jan 2021 00:00:00 +0000

This article contains proposed solutions to the exercises in section 1.1 of Structure and Interpretation of Computer Programs (SICP) by Abelson, Sussman and Sussman. MIT hosts video lectures to accompany the book on MIT OpenCourseWare.

The textbook uses the Scheme dialect of Lisp. I have written these solutions to help me learn Emacs Lisp (Elisp). Some of the solutions are derived from the work by Andres Raba. Most of the solutions include links to the online version of the GNU Emacs Lisp Reference Manual.

Chapter 1: The Elements of Programming

The first chapter of the textbook introduces the elements of the Lisp language. Harold Abelson starts the lecture with the interesting statement that computer science is neither a science, nor is it about computers, just like physics is not about particle accelerators.

Lecture 1A: Overview and Introduction to Lisp.

The first 28 minutes are an introduction to computer programming, after which the LISP lecture starts.

Exercise 1.1

Below is a sequence of expressions. What is the result printed by the interpreter in response to each expression? Assume that the sequence is to be evaluated in the order in which it is presented.

Answer

In Emacs, you can type this code in the scratch buffer and evaluate an expression with the eval-last-sexp function or the C-x C-e keyboard shortcut, placing the cursor after the end of the expression. The eval-last-sexp function evaluates the last expression before the cursor (point). The results will appear in the minibuffer at the bottom of the screen.

Emacs also has a REPL (Read-Eval-Print Loop), which you can access with M-x ielm. In REPL mode, the results appear below the expression after you hit enter.

Emacs provides the result in decimal numbers. The output between parenthesis is the result in octal (#o) and hexadecimal (#x) numeral system, e.g.: 1969 (#o3661, #x7b1). For values below 128, the output also includes an associated character, e.g. 26 (#o32 #x1a ?\C-z), where z is the character 26 in ASCII.

The code below shows the solutions in the comments after the expressions. The first lines demonstrate Arithmetic Operations in Emacs Lisp. Lisp uses Polish notation to resolve mathematical expressions. In this system, the operatore precede the operands, e.g. + 3 4 is the same as $3+4$ in infix notation.

The second set of expressions demonstrates setting variable values. The setq function sets universal variables. The q relates to the fact that the variable name is quoted. Without quoting the variable (the set function), Emacs will evaluate the name of the variable as a function and cause an error. You can also add a quotation mark to prevent Emacs from evaluating an expression. The following two expressions have the same result:

  ;; Equivalent expressions
  (setq a 10)
  (set 'a 10)

Emacs is a self-documenting editor and you can document your programming on the fly. The defvar function allows you to define a new variable plus a documentation string. This function is helpful when the variable will be available to other users. The help string that you define is available through the help function with the C-h v shortcut.

  ;; Arithmetic operations
  10 ; => 10
  (+ 5 3 4) ; => 12
  (- 9 1) ; => 8
  (/ 6 2) ; => 3
  (+ (* 2 4) (- 4 6)) ; => 10

  ;; Seting variable names
  (setq a 3)
  (setq b (+ a 1))

  (setq a 3
        b (+ a 1))

  (defvar example (* a b)
    "An example of a documented variable.")

  (+ a b (* a b)) ; => 19
  (= a b) ; => nil

These last lines of code look provide an example of basic conditionals. The first expression is a an if-then-else statement. The second expression is a conditional. The last clause is executed when none of the other conditions is true. The t parameter ensures that this expression is always evaluated.

    ;; Conditionals
    (if (and (> b a) (< b (* a b)))
        b
      a) ;; => 4

    (cond ((= a 4) 6)
          ((= b 4) (+ 6 7 a))
          (t 25)) ;;=> 16

    (+ 2 (if (> b a) b a)) ; => 6

    (* (cond ((> a b) a)
             ((< a b) b)
             (t -1))
       (+ a 1)) ;; => 16

Exercise 1.2

Translate the following expression into prefix form:

$$\frac{5 + 4 + (2 - (3 - (6 + \frac{5}{4})))}{3(6 - 2)(2 - 7)}$$

Answer

When Emacs Lisp divides two integers, the outcome will also be integer, e.g. (/ 22 7) results in 3 instead of 3.142… To force a floating point output, add a decimal to one of the parameters: (/ 22 7.0).

Elisp can evaluate this expression in a single line, but it becomes hard to read. This example demonstrates the multitude of parenthesis in Lisp expressions. Some people jokingly suggest that Lisp means “Lots of Irritating Superfluous Parentheses”. The smartparens package can help you write code.

Prettyprinting is a way to format source code to make it easier for human readers to understand. In Emacs, you can convert a convoluted expression to prettyprinting with the pp-macroexpand-last-sexp function. Run this function at the end of an expression, and the pp package, the pretty printer for Emacs Lisp, will provide a better formatted version.

  (/ (+ 5 4 (- 2 (- 3 (+ 6 (/ 4.0 5))))) (* 3 (- 6 2 ) (- 2 7)))

  ;; Pretty printing (pp-macroexpand-last-sexp)
  (/
   (+ 5 4
      (- 2
         (- 3
            (+ 6
               (/ 4.0 5)))))
   (* 3
      (- 6 2 )
      (- 2 7)))

Exercise 1.3

Define a procedure that takes three numbers as arguments and returns the sum of the squares of the two larger numbers.

Answer

defun declares a new function in Emacs Lisp. The name of the function appears after defun and its parameters are listed between parentheses. The quoted string on the next line is the documentation. The doc string will be accessible via the Emacs help functions.

The square function returns the square of a number and the sum-of-squares helper function produces the sum of squares. The main function determines the two largest numbers of three and provides the sum of their squares..

  ;; Exercise 1.3
  (defun square (a)
    "Squaring a number"
    (* a a))

  (defun sum-of-squares (a b) 
    "Sum the squares of two numbers"
    (+
     (square a)
     (square b)))

  (defun largest-sum-of-squares (a b c)
    "Find the largest sum of squares for two out of three numbers"
    (cond ((and (< a b) (< a c)) (sum-of-squares b c))
          ((and (< b a) (< b c)) (sum-of-squares a c))
          (t (sum-of-squares a b))))

  (largest-sum-of-squares 2 3 5)
  (largest-sum-of-squares 3 2 5)
  (largest-sum-of-squares 5 2 3)

The three functions are bow available through the whole Emacs session. The variables are only available within each function (scoping rules).

Exercise 1.4

Observe that our model of evaluation allows for combinations whose operators are compound expressions. Use this observation to describe the behavior of the following procedure:

  (defun a-plus-abs-b (a b)
    (funcall
     (if (> b 0)
         '+
       '-)
     a b))

  (a-plus-abs-b 10 -20)

Answer

The function evaluates as follows:

If b is positive, the function evaluates (+ a b)
If b is negative, the function evaluates (- a b)

funcall evaluates the expression (if (> b 0) '+ '-)) and uses the result (either + or -) as a function. The quote is needed to ensure the + and - are not evaluated in the conditional.

The original Scheme implementation used in the book and lectures did not use a function call. Scheme is a Lisp-1 and Emacs Lisp is Lisp-2. In scheme names and procedures are located in the same namespace, so it is possible to return a procedure name from a condition and evaluate it. In Lisp-2, procedures have their own namespace, so in order to call procedure from name you have to use funcal (Source: /u/andreyorst).

Exercise 1.5

Ben Bitdiddle has invented a test to determine whether the interpreter he is faced with is using applicative-order evaluation or normal-order evaluation. He defines the following two procedures:

(defun p () (p))

(defun test (x y) 
  (if (= x 0) 
      0 
      y))

Then he evaluates the expression:

(test 0 (p))

What behavior will Ben observe with an interpreter that uses applicative-order evaluation? What behavior will he observe with an interpreter that uses normal-order evaluation? Explain your answer. (Assume that the evaluation rule for the special form if is the same whether the interpreter is using normal or applicative order: The predicate expression is evaluated first, and the result determines whether to evaluate the consequent or the alternative expression.)

Answer

Applicative order: The function call results in an infinite loop, as the function p calls itself (Emacs' behaviour).
Normal order: The function never reaches p because it exits with 0 as a result.

Exercise 1.6

Alyssa P. Hacker doesn't see why if needs to be provided as a special form. “Why can’t I just define it as an ordinary procedure in terms of cond?” she asks. Alyssa’s friend Eva Lu Ator claims this can indeed be done, and she defines a new version of if:

  (defun new-if (predicate then-clause else-clause)
    (cond (predicate then-clause)
          (t else-clause)))

Eva demonstrates the program for Alyssa:

(new-if (= 2 3) 0 5)

(new-if (= 1 1) 0 5)

Delighted, Alyssa uses new-if to rewrite the square-root program:

(defun sqrt-iter (guess x)
  (new-if (good-enough? guess x)
          guess
          (sqrt-iter (improve guess x) x)))

What happens when Alyssa attempts to use this to compute square roots? Explain.

Answer

  (defun sqrt (x)
    (defun square (x)
      (* x x))
    (defun good-enough? (guess)
      (< (abs (- (square guess) x)) 0.001))
    (defun average (a b)
      (/ (+ a b) 2.0))
    (defun improve (guess)
      (average guess (/ x guess)))
    (defun sqrt-iter (guess)
      (new-if (good-enough? guess)
              guess
              (sqrt-iter (improve guess))))
    (sqrt-iter 1.0))

  (sqrt 2)

(average 12 13)

The version with new-if function crashes because of a nesting error (Lisp nesting exceeds ‘max-lisp-eval-depth). The sqrt function works fine with the regular if function.

Exercise 1.7

The good-enough? test used in computing square roots will not be effective for finding the square roots of very small numbers. Also, in real computers, arithmetic operations are almost always performed with limited precision. This makes our test inadequate for huge numbers.

Explain these statements, with examples showing how the test fails for small and large numbers. An alternative strategy for implementing good-enough?,is to watch how guess changes from one iteration to the next and to stop when the change is a very small fraction of the guess.

Design a square-root procedure that uses this kind of end test. Does this work better for small and large numbers?

Answer

The float function forces floating point answers. See Elisp manual about Numeric Conversions.

  ;; Exercise 1.7
  (defun sqrt-heron (x)
    "Determine a square root using Heron’s method."
    (defun avg (x y) (/ (+ (float x) y) 2))
    (defun abs (x) (if (< x 0) (- x) x ))
    (defun improve (guess)
      (avg guess (/ x guess)))
    (defun good-enough? (guess next-guess)
      (< (abs (- guess next-guess)) 1e-20))
    (defun try (guess)
      (if (good-enough? guess (improve guess))
          guess
        (try (improve guess))))
    (try 1.0))

  (sqrt-heron 1234567890)
  (* (sqrt-heron 1e-5) (sqrt-heron 1e-5))

Exercise 1.8

Newton’s method for cube roots is based on the fact that if $y$ is an approximation to the cube root of $x$, then a better approximation is given by the value:

$$\frac{x/y^2 +2y}{3}$$

Use this formula to implement a cube-root procedure analogous to the square-root procedure.

Answer

  ;; Exercise 1.8
  (defun cube-root-newton (x)
    "Newton’s method for cube roots"
      (defun abs (x) (if (< x 0) (- x) x ))
      (defun improve (guess)
          (/ (+ (/ x (* guess guess)) (* 2 guess)) 3.0))
      (defun good-enough? (guess next-guess)
          (< (abs (- guess next-guess)) 1e-20))
      (defun try (guess)
          (if (good-enough? guess (improve guess))
              guess
              (try (improve guess))))
      (try 1.0))

  (cube-root-newton (* 4 4 4))
  (cube-root-newton 1e-5)

Visualise Org-Roam Networks With igraph and R

Peter Prevos — Sat, 14 Aug 2021 00:00:00 +0000

The Emacs package Org-Roam provides a powerful tool to take notes following the idea of the Zettelkasten method. You can write notes with all the power that Emacs provides while linking your thoughts to each other and with your bibliography. This article shows how to analyse and visualise Org-Roam knowledge networks with the iGraph package and the R language.

The image below shows the current state of my personal knowledge network as visualised by iGraph. The Org-Roam community is working hard to create a visual user interface for Org-Roam with network diagrams. While this user interface is a great tool to explore your network, Emacs users tend to prefer writing code instead of clicking buttons. This article adds to this concept by showing how to apply some mathematical analysis and delve deep into your Zettelkasten. The beauty of iGraph is that it is available in R, Python, Mathematica and C/C++.

My personal Org-roam notes network.

You can download the code in this article from GitHub.

Loading the Database

Org-Roam stores information about the notes in your system in an SQLite database. This database holds information about the nodes (registered files and headings) and links. The DBI package enables connecting to these databases.

The first lines open the connection. You will have to change the path of the database location to your preferences for this to work. The two queries extract the unique ID and the title for all nodes. The second query extracts the links between nodes. The Org-Roam database also stores external links, which are excluded from this analysis.

The next part cleans the titles by removing quotation marks and joins the tables. I could have done this in the query, but I know Tidyverse better than SQL.

  ## Visualise Org-Roam databse

  ## Connect to database and extract nodes and links
  library(DBI)
  roam <- dbConnect(RSQLite::SQLite(), "~/.config/emacs/org-roam.db")
  nodes <- dbGetQuery(roam, "SELECT id, title FROM nodes")
  links <- dbGetQuery(roam, "SELECT source, dest FROM links WHERE type = '\"id\"'")
  dbDisconnect(roam)

  ## Clean node names and create network table
  library(tidyverse)
  nodes <- nodes %>% 
    mutate(title = str_remove_all(title, "\""))

  network <- left_join(links, nodes, by = c("source" = "id")) %>% 
    left_join(nodes, by = c("dest" ="id")) %>% 
    select(from = title.x, to = title.y)

The result of this code is a data frame with two names of the nodes at each vertex in the network. Any unconnected nodes are ignored due to the left join. The next step is to convert this data to iGraph format and start visualising and analysing.

Create iGraph network

The iGraph package uses a specific data format. The graph.data.frame() function converts a data frame to an iGraph network. The simplify() function removes any multiple links and self-references.

  ## Create network diagram
  library(igraph)
  g <- graph.data.frame(network)
  g <- simplify(g)

Analyse the Network

The iGraph package has an extensive library of functions to analyse networks. In this section, we look at a measure for centrality and clustering.

Centrality

Not all nodes in the network are of equal importance. Centrality is a concept in graph theory that describes the importance of a node in a network. The most straightforward method of calculating the centrality of a node is to count the number of vertices (connections) that it has with other nodes. The example below shows a simple undirected graph and the degree of each node.

Example of the degree centrality of an undirected graph.

In a directed graph, like with Org-Roam, we can calculate the in-degree and out-degree, the number of vertices point to the node, or the number of vertices pointing from the node (the number of backlinks and the number of links). The degree() function of the iGraph package calculates degree centrality. The code below generates a table with the top ten most connected nodes.

  ## Centrality
  centrality <- tibble(Node = names(degree(g)),
                       Degree = degree(g, mode = "total"),
                       Links = degree(g, mode = "out"),
                       Backlinks = degree(g, mode = "in")) 

  centrality %>% 
    top_n(n = 10) %>% 
    arrange(desc(Degree))

Communities

While some nodes might be more connected than others, it would also be good to know which nodes are closest to each other. In other words, can we find clusters of nodes (communities) to find meta-structure in the network? A community is a sub-network with densely connected nodes, but less so with nodes outside the community. Expressed probabilistically, two random nodes are more likely to be associated when they form part of the same community than when they don't. Thus, community detection increases the parsimony of the network by identifying those groups of nodes that are most closely related to each other.

Using the example network, two communities can be visually distinguished: nodes 1–4 and nodes 5–7. Texts A, B and C belong to the first community, while texts C and D belong to the second community. Text C (nodes 2 and 5) spans the two communities of discourse. This solution is valid because each node has more connections to nodes within its own community than nodes outside it.

Example of a network with two communities.

The examples' communities are easily detected visually, but community detection becomes more difficult as the network grows. Community detection is a mathematical process to cluster nodes into cohesive sub-networks.

Several algorithms for community detection have been developed based on a range of mathematical principles. However, the numerical validation of community structure has not yet been satisfactorily solved, and no agreed single method exists to assess the quality of communities. In my research into knowledge networks, the Spinglass algorithm provided the most interpretable results.

  ## Community-detection
  c <- cluster_spinglass(g)

Note that clustering algorithms only work with complete graph, which means that all nodes are connected to each other by one or more steps. If you have multiple clusters and one or more single nodes, you can decompose the graph into its connected components and analyse each of these.

  g.comp <- decompose(g)
  largest <- which.max(sapply(g.comp, diameter))
  g <- g.comp[[largest]]
  c <- cluster_spinglass(g)

This algorithm results in a named vector that assigns a number of each community to each node. Making sense of community structure requires some human interpretation because the algorithm does not analyse the texts, only the connections between them. Please note that clustering algorithms are computationally intensive and can take a while.

We can assign a name to each community by using the node's name with the most backlinks within each community. The code below creates a table of the community memberships and joins it with the centrality table. The last part assigns a name to each community by selecting the node's name with the most backlinks, counts the number of members and arranges the table by community size.

  communities <- tibble(Node = c$names,
                        Community = c$membership) %>% 
    left_join(centrality) %>% 
    group_by(Community) %>% 
    summarise(Name = Node[which.max(Backlinks)],
              Nodes = n()) %>% 
    arrange(desc(Nodes))

Visualise Org-Roam

Now that we have analysed the graph, it is time to visualise the results. iGraph has extensive plotting functionality that gives full control over every aspect of the visualisation. To create the graph shown at the start of this article, use the following code:

  ## Visualise graph
  par(mar = c(0, 0, 0, 0))
  plot(g,
       layout = layout.fruchterman.reingold,
       mark.border = NA,
       vertex.color = c$membership,
       vertex.label = NA,
       vertex.size = sqrt(degree(g, mode = "in") + 1),
       vertex.frame.color = NA,
       edge.arrow.size = 0,
       edge.color = "lightgrey")

The layout of graphs in this example follows the Fruchterman-Reingold method. This method tries to position the nodes so that the ones with the highest centrality are in the centre. Several other methods are available to compute an optimal layout.

The colour of each node relates to its community. The node's size is the square root of the number of backlinks (indegree) plus one so that larger nodes don't overpower the picture and small nodes vanish.

The remainder of the options improve the readability of the graph. This image is aesthetically pleasing, but it is not very informative. Adding text to the nodes would make it unreadable.

Subnetworks

When a network is fairly large it might be more informative to visualise only a section. The code below determines the number of the relevant node. The V() function holds the properties of al vertices (the nodes) in the network. The make_ego_graph() subsets a network from a given node (ego) and the order. This function creates a list of networks as you can enter more than one node.

  ## Subnetwork
  node <- which(names(V(g)) == "Topology")
  g1 <- make_ego_graph(g, order= 2, nodes = node)
  c1 <- cluster_walktrap(g1[[1]])

  plot(g1[[1]], layout = layout.kamada.kawai,
       mark.border = NA,
       vertex.size = sqrt(degree(g1[[1]])) + 2,
       vertex.color = c1$membership,
       vertex.frame.color = NA,
       vertex.label.family = "Sanserif",
       vertex.label.color = "black",
       edge.arrow.size = .3)

Subnetwork example.

Interactive visualisation

The networkD3 package can create interactive graphs, as implemented in this code snippet. The first step converts the iGraph object to a D3 object. Then, the htmlwidgets package lets you save this object as an interactive HTML file.

The code below adds the number of backlinks to the node table. Then, it defines the legend using the community names described above.

  ## Interactive visualisation
  library(networkD3)
  library(htmlwidgets)

  ## Convert to object suitable for networkD3
  d3 <- igraph_to_networkD3(g, group = communities$Name[membership(c)])
  d3$nodes$backlinks <- degree(g, mode = "in")

  ## Create force directed network plot
  p <- forceNetwork(Links = d3$links, Nodes = d3$nodes, 
                    Source = "source", Target = "target", 
                    NodeID = "name", Group = "group", 
                    Nodesize = "backlinks",
                    zoom = TRUE, legend = TRUE)

  saveWidget(p, file=paste0( getwd(), "/org-roam.html"))

Further Analysis

Graph theory is a broad field with lots of approaches to analyse graphs. This article only scathes the surface of what you can find out about your personal knowledge garden. Graph theory is a vital tool in many applications, including the fight against the Coronavirus.

Graph theory lectures by Anthony Bosman.

Export WordPress to Hugo Markdown or Org Mode with R

Peter Prevos — Sat, 18 Jul 2020 00:00:00 +0000

I started my first website in 1996 with hand-written HTML. That became a bit of a chore, so for about fifteen years, WordPress became my friend. WordPress has been great to me, but it is slowly becoming a pain to keep updating plugins, security issues, slow performance and the annoying block editor. I am also always looking for additional activities I can do with Emacs. Hugo takes a lot of the pain of managing site away as you can focus on the content and Emacs provides me with powerful editing functionality.

I recently returned to a static website using Hugo. This article explains how to export a WordPress blog to Hugo and customise it with R code. The only reason I used R is because it is the only programming language I know well enough.

You will also need to install the mighty Pandoc software to convert the content to Org mode and the WP All Export WordPress plugin to export your website to a CSV file.

Convert the content to Markdown or Org Mode

The first step is to export the WordPress posts database to a CSV file. Several plugins are available that help you with this task. I have used the WP All Export plugin to export the data. You need to download the ZIP file and install this plugin manually in your WordPress setup. Follow the steps in the All Export plugin and create a CSV file from your posts with at least these fields:

Title
Slug
Date
Content
Categories
Tags

Alternatively, you can link directly to the WordPress database and extract the data with the RMySQL package.

The content files for Hugo are either Markdown or Org Mode. I prefer to use Org Mode as it provides me with access to the extensive functionality that Emacs has to offer, including writing and evaluating R code.

The Content field in the exported CSV file contains HTML code of the article. The code below reads the CSV file and saves each content field as an HTML file, using the post's slug as the filename. The mighty Pandoc software converts this file to Org mode. Any draft posts or pages in the export file will have NA as the file name and are as such skipped.

Now that we have some content, we need to add the Org mode front matter so that Hugo can build a site. The last part of the code generates the front matter for each post, prepends it to the exported Org mode file and cleans some entries.

Copy the code below and save it as wp2org.R. You need to change the filename in the line that starts with file to the name of your export file. The script also creates two subdirectories to store the HTML and Org files.

You run this code with Rscript wp2org.R from the same directory where the CSV file is stored. The result will be a collection of Org mode files.

This new site will not be perfect just yet. To show the images, you need to download your wp-content folder and move it to the static/images folder in Hugo.

The internal links in your blogs will be hard-coded, which means that you need to configure Hugo to ensure your slugs stay the same.

There will be other bits and pieces that might not have adequately converted, so do check your pages.

All you have to do now is to add a theme to your website, and your blog is fully converted. The Hugo website has a great Quick Start page that will get you going.

You can create new posts and edit your content with your favourite text editor. I use Org mode in Emacs to develop this website.

Summary

In summary, you need to take the following steps:

Install pandoc software and WP All Export WordPress plugin.
Download your website as a CSV file with the WordPress plugin.
Copy the R script in a file called wp2org.R and save it in the same location as the CSV file.
Open your console and move to the folder with the script and CSV file
Run Rscript wp2org.R
Review the Org mode files and clean-up any issues

Script

  ## Export WP to Hugo

  ## Read exported WP content
  library(tibble)
  library(readr)
  library(dplyr)
  library(stringr)

  ## Replace the filename with the exported file
  posts <- read.csv("filename", skipNul = TRUE)

  ## Create subdirectories
  if (!dir.exists("tmp")) dir.create("tmp")
  if (!dir.exists("org")) dir.create("org")

  ## Read posts
  for (i in 1:nrow(posts)) {
      ## Save content as temporary html file
      filename <- paste0(posts$Slug[i], ".html")
      writeLines(posts$Content[i], paste0("tmp/", filename))
      ## Convert to Org mode with Pandoc
      pandoc <- paste0("pandoc -o ", paste0("org/", posts$Slug[i],
                                            ".org ", paste0("tmp/", filename)))
      system(pandoc)
  }

  ## Create front matter for all posts
  fm <- tibble(title = paste("#+title:", posts$Title),
               date = paste0("#+date: [", as.POSIXct(posts$Date, origin = "1970-01-01"), "]"),
               lastmod = paste0("#+lastmod: [", Sys.Date(), "]"),
               categories = paste("#+categories[]:", str_replace_all(posts$Categories, " ", "-")),
               tags = paste("#+tags[]:", str_replace_all(posts$Tags, " ", "-")),
               draft = "#+draft: true") %>%
      mutate(categories = str_replace_all(categories, "\\|", " "),
             tags = str_replace_all(tags, "\\|", " "))

  ## Load Hugo files an prepend front matter
  for (f in 1:nrow(posts)) {
      filename <- paste0("org/", posts$Slug[f], ".org")
      post <- c(paste(fm[f, ]), "", readLines(filename))
      ## Repoint images
      post <- str_replace_all(post, paste0("http.*wp-content"), "/images")
      ## Cleanup LaTeX
      post <- str_replace_all(post, "\\$latex ", "$")
      ## Remove remaining Wordpress artefacts
      post <- str_remove_all(post, ':::|\\{.wp.*|.*\\"\\}')
      ## Write to disk
      writeLines(post, filename)
  }

Emacs on Lucid Manager

Why Use Emacs

What is Emacs?

Why use Emacs?

Redefining user-friendliness

What You See is What You Mean

Principles of writing plain text

The Emacs learning curve

Advantages of using Emacs

Limitations of Emacs

Emacs Writing Studio

Getting Started with Emacs

Installing Emacs

Getting Started with Emacs

Working with the keyboard

Modes

Frames and Windows

Buffers

Files

Exiting Emacs

Learn more

Emacs Writing Studio

Configure Emacs Writing Studio

Configuring Emacs

The Initialisation File

Emacs Lisp

The Emacs Writing Studio Configuration

Checking external software requirements

Loading Packages

Configuring Packages

Keeping your packages up-to-date

Changing keyboard shortcuts

Emacs Writing Studio configuration

Emacs Writing Studio variables

Is Emacs a Productivity Sink?

Emacs Writing Studio

Writing Prose with Emacs

Writing modes

Basic principles

Moving around in a buffer

Making a mark

Focusing

Copying and pasting text the Emacs way

Selecting text

Copy and paste text

Search and replace

Correcting mistakes

Undo the Emacs way

Configure text modes

Checking spelling

Counting words

Text size

Text Completion

Emacs Writing Studio

Improving the Emacs User Interface

Improving the Emacs user interface

Themes

Other Themes

Setting the fonts

Tooltips

Emacs Writing Studio

Enhancing Minibuffer Completion

Emacs Completion Systems

Minibuffer completion

Vertico

Orderless

Marginalia

Help with keyboard shortcuts

Emacs Writing Studio

Emacs Workflow for Authors

The Emacs Workflow

Inspiration

Ideation

Fleeting notes

Permanent notes

Bibliographic notes

Journals

Production

Publication

PDF format