Trying out various emacs configurations, such as doom emacs and radix I’ve realized that I don’t like the fact that Org mode configuration is so tightly integrated with the emacs configuration. I think one step for Org mode to transcend emacs is to allow it to be customized on it’s own. To not make life all too difficult for me initially, I don’t think going away from elisp is the way. Maybe further down the road, with a yaml- or json-syntax for the config to make it less implementation specific. But for now my idea still revolves around elisp.
I would like to allow for folder specific configuration files, maybe
named .org-config or .org-collection. The idea is that such
configuration file would inject itself into the emacs initialization
based on a variable specifying if it should or should not be. Possibly
an emacs customization in the org-collections namespace to turn it
on, and then org-collections (as discussed above) would point out
the location of where to look for that .org-collection file.
The configuration file itself would allow any Org mode customization
to be set, but only Org mode customization. No other emacs
customization. I suppose the namespace should allow for ol- and
ox- prefixes as well since they sort of are a part of the org
namespace.
The configuration should take precedence over any global configuration
done in emacs customization. File local variables should probably
take precedence and possibly also directory local variable. I should
make sure that this .org-collection file doesn’t require safe
variables to be used.
Either a function is provided to switch between active collections or
it is done based on default-directory.
A customization is needed to signal to emacs if org-collection is to
be used or not. The added overhead will be that emacs needs to look
for the .org-collection file. How often is determined by how
switching collections is triggered…
The configuration file should probably mimic directory variable files pretty closely. I’m now thinking that the file should contain any number of alists mapping variables to values where emacs proabably only should set variables that it finds in the customization system. It should probably allow regular elisp comments in the file as well.
Something like:
;; Main org collection customization
((org-directory . "./")
(org-use-speed-commands . t)
(org-todo-keywords . '((sequence "TODO" "DONE"))))
;; Agenda customizations
((org-agenda-files . "agenda.org"))Some configurations, such as org-directory, should probably be set
by default if .org-collection exists in a folder that either is
visited or configured as the current default collection.
I have the sense that Org mode needs to be split up! It is really time to separate Org mode into the specification, the data format and it’s customizations and the customizations of Emacs to work with that specification and data format. Probably a feat to big to be done, but it’s really bothering me that it’s not done.
Org currently consist of:
- A Specification of a file typ with a particular syntax.
- Editor configurations and functions for workin with those files.
- Configurations and functions for exporting those files.
- Auxiliary modes to filter and present information in Org files.
Org currently lacks:
- A concept of grouping files together into collections in a
composable way.
The specification for grouping of files is important for certain features to work better when thinking outside of the scope of a single file.
How would it be split up if it were to be more sensible?
Take inspiration from, and make use of, the project functionality of Emacs! That should really be the guiding star in this work. Try to emulate how EDE looks like and works.
That means some designs are already decided:
- A project is simply a directory root.
default-directorydetermines the current project.- A customization for list of project directories,
org-project-directories. - Project files in the root of projects on a format similar to
(example from EDE):
(ede-proj-project "EDE testproject" :file "Project.ede" :name "EDE testproject" :targets nil :object-name "EDE testproject" :makefile-type Makefile\.am)
- The hook
project-find-functionsneeds to be used for determining if the file belongs to a project or not. Similar to:(add-hook 'project-find-functions #'project-try-ede)
I should then make the switch, or at least the extension for things in Org to be based on this project definition to work. Notably:
- Org agenda
- Org id
- Org export
I have to be pragmatic here. Org mode relies on global configurations quite a lot, and before Org collection is integrated well within Org mode things will need to be set globally. Maybe it’s not bad in itself… But I would like to have as a principle to try to keep things as local as possible. If possible. But not to bend my back over to make it work.
The module will create a global minor mode that can be used to enable org collection functionality.
Determining if org-collection customizations are active is done primarily based on two variables:
- org-collection-global
- org-collection-local (buffer local)
When these are both set org-collection events shall not run any code (except checking if those variables are set…).
Since this minor mode will modify some global Org mode variables only
one collection can be “active” at any time. Switching between
collections is automatic and based on default-directory.
The minor mode should ideally be able to consider the filesystem
hierarchy and enable the collection also for subfolders within a
collection. I.e partial matches on default-directory that finds the
special collection file are also valid. The “longeset” match on
default-directory is preferred.
- When global minor mode is on:
- Activation of collection is event-driven by a ‘maybe try collection activation’ event when a file is opened or buffer activated.
- Org collection can be refreshed with special function
org-collection-refresh. This refresh shall reset variableorg-collection-localto nil for all buffers where it’s set and re-trigger the ‘maybe try collection activation event’ on active buffer. Other buffers will be activated again by the event.
- When global minor mode if off:
- Nothing
Determining if org-collection customizations are active is done primarily based on these variables:
- org-collection-global
- org-collection-local (buffer local)
- org-collection-cached (buffer local)
These variables also act as a cache of sort. When set, org-collection events shall not run any code (except checking if those variables are set…). Org-collection-cached is used in all visited buffers as a signal to the hook for it to know to not scan it again for a collection.
Which hooks to use then?
It seems buffer-list-update-hook is the only option if I want this
to be as intrusive as running each time a buffer is selected.
The buffer list is resorted each time a switch between buffers is
made, so that the selected buffer is highest in the list of buffers.
This is not perfect as the name of the hook isn’t explicitly about
triggering when a buffer is selected. But it seems what Emacs has come
up with so far.
Using this hook should mean no other hook is needed. When a file is
opened it will create a buffer that should trigger the
buffer-list-update-hook mentioned above.
- Enables events (by hooks)
- Looks in current buffer and enables a collection as if a ‘maybe try collection activation’ event was triggered.
- Removes events (hooks)
- Removes org-collection customization globally and in all open buffers
Switching between collections should be done when a buffer is visited that has a collection attached to it that is different than the global collection.
The switch is really only related to switching the parameters that requires global configuration. Thus it should be enough to check that the global and local configs are different.
Note that this should be done even if the buffer-cache says the buffer has already been visited! The same event that deals with everything else needs to deal also with this.
- If the buffer is cached then it’s already been loaded and an
equality-check between
oc-globalandoc-localneeds to be made to determine if a global switch is needed or not. - If the buffer isn’t cached then the local settings needs to apply anyhow, and maybe it’s easier to also set globals at that point, even if it’s already set? Can revise later if needed.
…might be needed, since a refresh of org config when triggered here might be after Org mode was set for the buffer…
…there is the function org-mode-restart. Sounds not to bad.
Probably not that performant though, but this should only run when a
buffer either initializes a collection or is opened the first time.
Removed the following:
(interactive (list (org-completing-read "Set collection: " org-collection-list)))
May be useful in separate function later on.
It seems the buffer-list-update-hook triggers ALOT! Not good. And my
algorithms aren’t good or fast enough to deal with it. It feels
sluggish. The code is too hungry. Need to fix. I still want to think
of things as local first, global second. I might have to revise that
soon. But for now I’ll keep on going.
Hungry things:
- Org-mode refresh.
I have to minimize the refreshing. Only do it if it’s really (really!)
neccessary. One way is to put more checks in
org-collection-check-buffer-function. And maybe only set the local config
if the buffer has major mode set to Org-mode.
Also, I think I have a bug now in that org-agenda-files doens’t switch between the global global and the collection global value correctly… Need to investigate more!
Scenario:
- Start minor mode in Org mode file in collection
- Open agenda (sticky)
- Open Org mode file outside collection
Set temp variables at each step (in the opened buffer!) and debug as best possible.
(setq step1 `(list :oaf ,org-agenda-files
:ocg ,org-collection-global
:ocgdp ,org-collection-global-defaults-plist))
(setq step2 `(list :oaf ,org-agenda-files
:ocg ,org-collection-global
:ocgdp ,org-collection-global-defaults-plist))
(setq step3 `(list :oaf ,org-agenda-files
:ocg ,org-collection-global
:ocgdp ,org-collection-global-defaults-plist))Maybe window-selection-change-functions hook can be used instead.
Only available from Emacs 27.1 though. But what the hell, if you want
nice things - upgrade (sorry).
I did some crude visualization of which collection is active on the mode line. But it’s not using best practices or anything. Let’s have a look at improving that.
See: info:elisp#Mode Line Format and take inspiration from how vc-mode
works. See vc-hooks.el and for example variable vc-mode and
function vc-mode-line.
When setting up a test configuration I’m finding some issues.
- Requiring packages that aren’t loaded by default. I’m getting
issues with org-brain and org-agenda currently.
Solution? Trying to introduce a block for requiring those packages
- Paths that needs to be set relative to current directory. That’s not possible for some variables as I can see. In particular Org-brain-path. Ideally it would be based on Org-directory, but it’s not.
Capture templates aren’t happy when I’m forcefully reloading org mode in the buffer. The preparation done of the capture buffer basically gets ruined.
The remedy (at least for now) seems to be to let Org collection be even more aggressive in terms of events! If I check every new file that is loaded, as soon as I can, to see if org collection should be activated, then the mode doesn’t enforce a reload when org capture creates the indirect buffer for the capture. All is well again.
Update in the evening; Getting into even more similar issues puts more fule on the fire. Reloading Org mode really doesn’t integrate well with other things. Most of the issues (maybe all?) I’m having are with indirect buffers or similar. Maybe if I could make sure to always do the org collection check before Org mode is actually loaded. That way I would basically never need to reload Org mode unless switching between already opened buffers in different collections. According to the following stackoverflow-questions/answers to similar issues:
- https://stackoverflow.com/questions/19280851/how-to-keep-dir-local-variables-when-switching-major-modes/
- https://emacs.stackexchange.com/questions/20753/call-a-function-before-a-specific-major-mode-starts
Org collection should probably use an advice on Org mode! Emacs manual writes that it’s not recommended. Especially for Emacs built in code. So if Org collection at any point in time makes it into Org mode, this advice will have to go. If it’s inside Org mode, then Org mode itself could make sure to run the Org collection check when needed though, so in that case everything should work out just fine anyhow. Swell times.
A small but useful function to help with switching between known
collections is introduced, called org-collection-goto.
Also..