In this commit I start using doom-profile-*-dir vars, though it will
make no difference to the default Doom install (only to profiles, and in
the future, in v3, where we'll drop $EMACSDIR/.local entirely).
The sudden spike in CPU and memory utilization alarms people, so I've
reduced how many cores native-comp will use. In non-interactive
sessions, it will use all of them, however (that is, when I later
introduce an AOT switch).
You can still override this by setting native-comp-async-jobs-number or
comp-num-cpus yourself.
I use advice instead of setting comp-num-cpus so that users to avoid
trampling on default behavior, or attempts by the user to change them.
BREAKING CHANGE: This commit makes three breaking changes:
- Doom now fully and dynamically generates (and byte-compiles) your
profile and its init files, which includes your autoloads, loading
your init files and modules, and then some. This replaces
doom-initialize-modules, doom-initialize-core-modules, and
doom-module-loader, which have been removed. This has also improved
startup time by a bit, but if you use these functions in your CLIs,
for instance, this will be a breaking change.
- `doom sync` is now required for Doom to see your profiles (and must be
run whenever you change them, or when you up/downgrade Emacs across
major versions).
- $DOOMDIR/init.el is now read much earlier than it used to be. Before
any of doom-{ui,keybinds,editor,projects}, before any autoloads are
loaded, and before your load-path has been populated with your
packages. It now runs in the context of early-init.el, giving users
freer range over what they can affect, but a more minimalistic
environment to do it in.
If you must have some logic run when all that is set up, add it to one
of the module hooks added in e08f68b or 283308a.
This also poses a significant change to Doom's load order (see the
commentary change in lib/doom.el), along with the following (non
breaking) changes:
1. Adds a new `doom profiles sync` command. This will forcibly resync
your profiles, while `doom sync` will only do so if your profiles
have changed.
2. Doom now fully and dynamically generates (and byte-compiles) your
user-init-file, which includes loading all your init files, modules,
and custom-file. This replaces the job of doom-initialize-modules,
doom-initialize-core-modules, and doom-module-loader, which have been
removed. This has also improved startup time by a bit.
3. Defines new doom-state-dir variable, though not used yet (saving that
and the other breaking changes for the 3.0 release).
4. Redesigns profile directory variables (doom-profile-*-dir) to prepare
for future XDG-compliance.
5. Removed unused/unimportant profile variables in doom.el.
6. Added lisp/doom-profiles.el. It's hardly feature complete, but it's
enough to power the system as it is now.
7. Updates the "load order" commentary in doom.el to reflect these
changes.
Emacs' version library (e.g. version-to-list) understands a number of
suffixes (see version-regexp-alist), but -dev is not one of them. Rather
than break compatibility (or impose a new, non-portable value onto
version-regexp-alist), I think it's best we adopt -pre instead. I
could've chosen -rc, -alpha, -beta, or -git, but I don't think any of
these accurately represent Doom's current state yet (and I don't want to
lock its versioning to git).
doom-before-init-hook runs before $DOOMDIR/init.el is loaded.
doom-after-init-hook runs at the *very* end of the Emacs startup
process (after window-setup-hook).
BREAKING CHANGE: For consistency and correctness, I've renamed the
module init/config hooks, and added new ones:
- Adds doom-before-modules-config-hook
- Adds doom-after-modules-config-hook (replaced doom-before-init-modules-hook)
- Adds doom-before-modules-init-hook
- Adds doom-after-modules-init-hook (replaced doom-init-modules-hook)
- Removed doom-after-init-modules-hook (replaced w/ after-init-hook)
The old naming (and timing) was counterintuitive. Now, it's named after
the loaded file group (init.el vs config.el), and I added before/after
variants. Altogether, this should make them less ambiguous.
I've also moved some functions in various modules to more correct hooks.
Load order before this change:
- $EMACSDIR/early-init.el
- $EMACSDIR/lisp/doom.el
- $EMACSDIR/lisp/doom-start.el
- $DOOMDIR/init.el
- {$DOOMDIR,~/.emacs.d}/modules/*/*/init.el
- `doom-before-init-modules-hook'
- {$DOOMDIR,~/.emacs.d}/modules/*/*/config.el
- `doom-init-modules-hook'
- $DOOMDIR/config.el
- `doom-after-init-modules-hook'
- `after-init-hook'
- `emacs-startup-hook'
- `window-setup-hook'
Load order after this change:
- $EMACSDIR/early-init.el
- $EMACSDIR/lisp/doom.el
- $EMACSDIR/lisp/doom-start.el
- $DOOMDIR/init.el
- `doom-before-modules-init-hook'
- {$DOOMDIR,~/.emacs.d}/modules/*/*/init.el
- `doom-after-modules-init-hook'
- `doom-before-modules-config-hook'
- {$DOOMDIR,~/.emacs.d}/modules/*/*/config.el
- `doom-after-modules-config-hook'
- $DOOMDIR/config.el
- `after-init-hook'
- `emacs-startup-hook'
- `window-setup-hook'
I move our hackiest and least offensive startup optimizations to core,
so they're easy for me to keep track of (they'll likely change often,
between major Emacs releases), to keep them from affecting non-Doom
profiles, and make it easy for readers to use as a reference.
These only benefit interactive sessions, and doom-start's responsibility
is to configure interactive sessions; it doesn't make sense to keep
these in core.
b7f84bd introduced a nasty regression that caused an infinite loop and
runaway memory usage on some pgtk+native-comp builds of Emacs when it
attempted to perform deferred native compilation of your packages. This
would make Emacs unusable and, if left alone, could even crash your
system.
The only Emacs builds I'm certain are affected are derived from
flatwhatson/emacs (like emacs-pgtk-native-comp on Guix and Arch Linux in
particular). 28.1 stable and master (on emacs-mirror/emacs@e13509468b)
are unaffected.
It appears that some, earlier pgtk builds stack idle timers differently.
I'm not entirely sure how, because it doesn't manifest in more recent
builds of Emacs, and I'm already burnt out on debugging this, but here's
how Doom encountered it:
Doom has an incremental package loader; it loads packages, piecemeal,
after Emacs has been idle for 2s, then again every 0.75s until it
finishes or the user sends input (then it waits another 2s before
starting again). However, if at any time the iloader detected that
native-compilation is in progress, it waits 2s before trying
again (repeat, until native-comp is done). But here's the catch, given
the following:
(run-with-idle-timer
2 nil (lambda ()
(run-with-idle-timer 1 nil (lambda () (message "hi")))))
I had assumed "hi" would be emitted after 3 seconds (once idle), but
instead it is emitted after 2. Like this, Doom's iloader would elapse
one idle timer directly into another, ad infinitum, until Emacs was
forcibly killed.
By switching to run-at-time and employing my own rudimentary idle timer,
I avoid this issue. Also, the iloader no longer needs to be considerate
of native-comp, because the latter does its own rate-limiting controlled
by native-comp-async-jobs-number.
Amend: b7f84bdd01
And emit more informative errors if they fail.
This eval-when-compile approach is used in preparation for v3, where
Doom's core libraries will be byte-compiled.
Rather than impose a 10-45min compilation step on users, I've disabled
ahead-of-time compilation for deferred compilation. In exchange, it will
eat up some CPU time the first time each uncompiled package is loaded,
but as this happens asynchronously (and are then quietly loaded in the
background), I think this is acceptable.
An --aot switch (or similar) will be added to `doom sync` and `doom
build` in the future, in case folks prefer the old behavior.
startup-redirect-eln-cache adds the new directory and removes
$EMACSDIR/eln-cache from native-comp-eln-load-path, but it's not
available in 28.1, so we'll have to wait until Doom drops 28.1 support
to use it.
doom-etc-dir will be renamed to doom-data-dir, to better reflect its
purpose, and align it with XDG_DATA_HOME (where it will be moved to in
v3, where Doom will begin to obey XDG directory conventions more
closely).
- Deprecates the doom-private-dir variable in favor of doom-user-dir.
- Renames the pseudo category for the user's module: :private -> :user.
- Renames the doom-private-error error type to doom-user-error.
Emacs uses the term "user" to refer to the "things" in user space (e.g.
user-init-file, user-emacs-directory, user-mail-address, xdg-user-dirs,
package-user-dir, etc), and I'd like to be consistent with that. It also
has the nice side-effect of being slightly shorter. I also hope
'doom-user-error' will be less obtuse to beginners than
'doom-private-error'.
To reduce redundancy, remove the maintenance hassle that version
constants would impose later on, and rely on built-in
facilities (featurep) more over global variables or doomisms, these
global constants have been deprecated in favor of Emacs "features":
- EMACS28+ -- replace with (> emacs-major-version 27)
- EMACS29+ -- replace with (> emacs-major-version 28)
- NATIVECOMP -- replace with (featurep 'native-compile)
- MODULES -- replace with (featurep 'dynamic-modules)
(These constants will be formally removed when v3 is released. The IS-*
constants are likely next, but I haven't decided on their substitutes
yet)
I also decided to follow native-compile's example and provide features
for Emacs' system features (since system-configuration-features' docs
outs itself as a poor method to detect features):
- dynamic-modules
- jansson
- native-compile -- this one already exists, but will instead be removed
if it's non-functional; i.e. (native-comp-available-p) returns nil.
These are now detectable using featurep, which is fast and built-in.
BREAKING CHANGE: This restructures the project in preparation for Doom
to be split into two repos. Users that have reconfigured Doom's CLI
stand a good chance of seeing breakage, especially if they've referred
to any core-* feature, e.g.
(after! core-cli-ci ...)
To fix it, simply s/core-/doom-/, i.e.
(after! doom-cli-ci ...)
What this commit specifically changes is:
- Renames all core features from core-* to doom-*
- Moves core/core-* -> lisp/doom-*
- Moves core/autoloads/* -> lisp/lib/*
- Moves core/templates -> templates/
Ref: #4273
