b7f84bd introduced a nasty regression that caused an infinite loop and
runaway memory usage on some pgtk+native-comp builds of Emacs when it
attempted to perform deferred native compilation of your packages. This
would make Emacs unusable and, if left alone, could even crash your
system.
The only Emacs builds I'm certain are affected are derived from
flatwhatson/emacs (like emacs-pgtk-native-comp on Guix and Arch Linux in
particular). 28.1 stable and master (on emacs-mirror/emacs@e13509468b)
are unaffected.
It appears that some, earlier pgtk builds stack idle timers differently.
I'm not entirely sure how, because it doesn't manifest in more recent
builds of Emacs, and I'm already burnt out on debugging this, but here's
how Doom encountered it:
Doom has an incremental package loader; it loads packages, piecemeal,
after Emacs has been idle for 2s, then again every 0.75s until it
finishes or the user sends input (then it waits another 2s before
starting again). However, if at any time the iloader detected that
native-compilation is in progress, it waits 2s before trying
again (repeat, until native-comp is done). But here's the catch, given
the following:
(run-with-idle-timer
2 nil (lambda ()
(run-with-idle-timer 1 nil (lambda () (message "hi")))))
I had assumed "hi" would be emitted after 3 seconds (once idle), but
instead it is emitted after 2. Like this, Doom's iloader would elapse
one idle timer directly into another, ad infinitum, until Emacs was
forcibly killed.
By switching to run-at-time and employing my own rudimentary idle timer,
I avoid this issue. Also, the iloader no longer needs to be considerate
of native-comp, because the latter does its own rate-limiting controlled
by native-comp-async-jobs-number.
Amend: b7f84bdd01
And emit more informative errors if they fail.
This eval-when-compile approach is used in preparation for v3, where
Doom's core libraries will be byte-compiled.
Rather than impose a 10-45min compilation step on users, I've disabled
ahead-of-time compilation for deferred compilation. In exchange, it will
eat up some CPU time the first time each uncompiled package is loaded,
but as this happens asynchronously (and are then quietly loaded in the
background), I think this is acceptable.
An --aot switch (or similar) will be added to `doom sync` and `doom
build` in the future, in case folks prefer the old behavior.
startup-redirect-eln-cache adds the new directory and removes
$EMACSDIR/eln-cache from native-comp-eln-load-path, but it's not
available in 28.1, so we'll have to wait until Doom drops 28.1 support
to use it.
doom-etc-dir will be renamed to doom-data-dir, to better reflect its
purpose, and align it with XDG_DATA_HOME (where it will be moved to in
v3, where Doom will begin to obey XDG directory conventions more
closely).
- Deprecates the doom-private-dir variable in favor of doom-user-dir.
- Renames the pseudo category for the user's module: :private -> :user.
- Renames the doom-private-error error type to doom-user-error.
Emacs uses the term "user" to refer to the "things" in user space (e.g.
user-init-file, user-emacs-directory, user-mail-address, xdg-user-dirs,
package-user-dir, etc), and I'd like to be consistent with that. It also
has the nice side-effect of being slightly shorter. I also hope
'doom-user-error' will be less obtuse to beginners than
'doom-private-error'.
To reduce redundancy, remove the maintenance hassle that version
constants would impose later on, and rely on built-in
facilities (featurep) more over global variables or doomisms, these
global constants have been deprecated in favor of Emacs "features":
- EMACS28+ -- replace with (> emacs-major-version 27)
- EMACS29+ -- replace with (> emacs-major-version 28)
- NATIVECOMP -- replace with (featurep 'native-compile)
- MODULES -- replace with (featurep 'dynamic-modules)
(These constants will be formally removed when v3 is released. The IS-*
constants are likely next, but I haven't decided on their substitutes
yet)
I also decided to follow native-compile's example and provide features
for Emacs' system features (since system-configuration-features' docs
outs itself as a poor method to detect features):
- dynamic-modules
- jansson
- native-compile -- this one already exists, but will instead be removed
if it's non-functional; i.e. (native-comp-available-p) returns nil.
These are now detectable using featurep, which is fast and built-in.
BREAKING CHANGE: This restructures the project in preparation for Doom
to be split into two repos. Users that have reconfigured Doom's CLI
stand a good chance of seeing breakage, especially if they've referred
to any core-* feature, e.g.
(after! core-cli-ci ...)
To fix it, simply s/core-/doom-/, i.e.
(after! doom-cli-ci ...)
What this commit specifically changes is:
- Renames all core features from core-* to doom-*
- Moves core/core-* -> lisp/doom-*
- Moves core/autoloads/* -> lisp/lib/*
- Moves core/templates -> templates/
Ref: #4273
