defcustom does some initialization work to accommodate the possibility
that the user has set the variable before it was defined. This work is
unneeded so early at startup, so I disable it (temporarily).
In the future, Doom will use defcustom more, as it's a helpful
indication to readers what variables I intended for configuration (and
helps with discovery of options through `M-x doom/help-custom-variable`
or `M-x customize`). As that transition occurs, the benefit of this
optimization will begin to show, but for now its effect on startup time
is negligible.
* lisp/doom.el (warning-suppress-types): set this immediately. Since its
default value is nil and this happens so early at startup, we don't
have to be considerate of defaults. Plus, this custom-dont-initialize
optimization can cause breakage if a warning is thrown *before* before
this setting is changed.
Introduces a system to announce what execution contexts are active, so I
can react appropriately, emit more helpful logs/warnings in the case of
issues, and throw more meaningful errors.
* bin/doom: load module CLIs in the 'modules' context.
* lisp/cli/doctor.el: load package files in 'packages' context.
* lisp/doom-cli.el:
- (doom-before-init-hook, doom-after-init-hook): trigger hooks at the
correct time. This may increase startup load time, as the benchmark
now times more of the startup process.
- (doom-cli-execute, doom-cli-context-execute,
doom-cli-context-restore, doom-cli-context-parse,
doom-cli--output-benchmark-h, doom-cli-call, doom-cli--restart,
doom-cli-load, run!): remove redundant context prefix in debug logs,
it's now redundant with doom-context, which doom-log now prefixes
them with.
* lisp/doom-lib.el (doom-log): prefix doom-context to doom-log output,
unless it starts with :.
* lisp/doom-packages.el (package!, doom-packages--read): throw error if
not used in a packages.el file or in the context of our package
manager.
* lisp/doom-profiles.el (doom-profile--generate-init-vars,
doom-profile--generate-load-modules): use modules doom-context instead
of doom-init-time to detect startup.
* lisp/doom-start.el (doom-load-packages-incrementally-h): move function
closer to end of doom-after-init-hook.
* lisp/doom.el:
- (doom-before-init-hook, doom--set-initial-values-h,
doom--begin-init-h): rename doom--set-initial-values-h to
doom--begin-init-h and ensure it runs as late in
doom-before-init-hook as possible, as that is the point where Doom's
"initialization" formally begins.
- (doom-after-init-hook): don't trigger at the end of command-line-1
in non-interactive sessions. This will be triggered manually in
doom-cli.el's run!.
* lisp/lib/config.el (doom/reload, doom/reload-autoloads,
doom/reload-env): use 'reload' context for reload commands.
* modules/lang/emacs-lisp/autoload.el (+emacs-lisp-eval): use 'eval'
context.
* modules/lang/org/config.el: remove doom-reloading-p; check for
'reload' doom context instead.
* lisp/doom-cli.el:
- reference backport source commit.
- doom-cli--restart: a type check is all we need here. This is a
programmer error, not a user error.
* lisp/doom-editor.el (recentf): mention recentf-show-abbreviated (added in
emacs-mirror/emacs@32906819ad)
* lisp/doom-keybinds.el (doom-init-leader-keys-h): move to
doom-after-init-hook, in case the user customizes leader variables in
a previous hook (like emacs-startup-hook or after-init-hook).
* lisp/doom-start.el: use eval-when! to compile out the section on
non-macOS systems (when Doom gets around to compiling its core files,
later).
* modules/config/literate/autoload.el (+literate-config-file): use
file-name-concat instead of string concat. This relaxes the
requirement that doom-user-dir end in a /; a requirement I intend to
fully phase out.
* modules/lang/emacs-lisp/autoload.el (+emacs-lisp-non-package): remove
empty map! macro in flycheck-emacs-lisp-check-form. The macro already
no-ops at compile-time/in noninteractive sessions since b480ed51a3.
* modules/ui/hl-todo/config.el (hl-todo-keyword-faces): revise
commentary for default hl-todo keywords.
Ref: emacs-mirror/emacs@32906819ad
Ref: b480ed51a3
This introduces a depth field for modules so that they may dictate their
load order explicitly, it also treats depths <= -100 or >= 100 as
special depths, which will be loaded early, before their respective
doom-{before,after}-module-{init,config}-hook. This permits psuedo
modules like :core and :user modules to be treated as normal modules
without too many special cases.
This also fixes a module load order issue on Emacs 29 (#6813), caused by
emacs-mirror/emacs@4311bd0bd7, which changed the return value order of
hash-table-{keys,values} causing modules to be loaded in reverse order;
resulting in the loss of evil keybinds, among other things.
Other notable changes:
- Changes the data structure for module data caches from a list to a
vector. Uses less memory and permits faster lookups. Also adds two
depth fields to the front of it.
- Changes the signature of doom-module-list and doom-package-list.
- Renames doom--read-packages -> doom-packages--read for consistency
with naming convention.
- Add doom-module-depth function.
- Adds a temporary doom-core-dir/init.el file, which is responsible for
loading doom-*.el.
Fix: #6813
Ref: emacs-mirror/emacs@4311bd0bd7
In this commit I start using doom-profile-*-dir vars, though it will
make no difference to the default Doom install (only to profiles, and in
the future, in v3, where we'll drop $EMACSDIR/.local entirely).
The sudden spike in CPU and memory utilization alarms people, so I've
reduced how many cores native-comp will use. In non-interactive
sessions, it will use all of them, however (that is, when I later
introduce an AOT switch).
You can still override this by setting native-comp-async-jobs-number or
comp-num-cpus yourself.
I use advice instead of setting comp-num-cpus so that users to avoid
trampling on default behavior, or attempts by the user to change them.
BREAKING CHANGE: This commit makes three breaking changes:
- Doom now fully and dynamically generates (and byte-compiles) your
profile and its init files, which includes your autoloads, loading
your init files and modules, and then some. This replaces
doom-initialize-modules, doom-initialize-core-modules, and
doom-module-loader, which have been removed. This has also improved
startup time by a bit, but if you use these functions in your CLIs,
for instance, this will be a breaking change.
- `doom sync` is now required for Doom to see your profiles (and must be
run whenever you change them, or when you up/downgrade Emacs across
major versions).
- $DOOMDIR/init.el is now read much earlier than it used to be. Before
any of doom-{ui,keybinds,editor,projects}, before any autoloads are
loaded, and before your load-path has been populated with your
packages. It now runs in the context of early-init.el, giving users
freer range over what they can affect, but a more minimalistic
environment to do it in.
If you must have some logic run when all that is set up, add it to one
of the module hooks added in e08f68b or 283308a.
This also poses a significant change to Doom's load order (see the
commentary change in lib/doom.el), along with the following (non
breaking) changes:
1. Adds a new `doom profiles sync` command. This will forcibly resync
your profiles, while `doom sync` will only do so if your profiles
have changed.
2. Doom now fully and dynamically generates (and byte-compiles) your
user-init-file, which includes loading all your init files, modules,
and custom-file. This replaces the job of doom-initialize-modules,
doom-initialize-core-modules, and doom-module-loader, which have been
removed. This has also improved startup time by a bit.
3. Defines new doom-state-dir variable, though not used yet (saving that
and the other breaking changes for the 3.0 release).
4. Redesigns profile directory variables (doom-profile-*-dir) to prepare
for future XDG-compliance.
5. Removed unused/unimportant profile variables in doom.el.
6. Added lisp/doom-profiles.el. It's hardly feature complete, but it's
enough to power the system as it is now.
7. Updates the "load order" commentary in doom.el to reflect these
changes.
Emacs' version library (e.g. version-to-list) understands a number of
suffixes (see version-regexp-alist), but -dev is not one of them. Rather
than break compatibility (or impose a new, non-portable value onto
version-regexp-alist), I think it's best we adopt -pre instead. I
could've chosen -rc, -alpha, -beta, or -git, but I don't think any of
these accurately represent Doom's current state yet (and I don't want to
lock its versioning to git).
doom-before-init-hook runs before $DOOMDIR/init.el is loaded.
doom-after-init-hook runs at the *very* end of the Emacs startup
process (after window-setup-hook).
BREAKING CHANGE: For consistency and correctness, I've renamed the
module init/config hooks, and added new ones:
- Adds doom-before-modules-config-hook
- Adds doom-after-modules-config-hook (replaced doom-before-init-modules-hook)
- Adds doom-before-modules-init-hook
- Adds doom-after-modules-init-hook (replaced doom-init-modules-hook)
- Removed doom-after-init-modules-hook (replaced w/ after-init-hook)
The old naming (and timing) was counterintuitive. Now, it's named after
the loaded file group (init.el vs config.el), and I added before/after
variants. Altogether, this should make them less ambiguous.
I've also moved some functions in various modules to more correct hooks.
Load order before this change:
- $EMACSDIR/early-init.el
- $EMACSDIR/lisp/doom.el
- $EMACSDIR/lisp/doom-start.el
- $DOOMDIR/init.el
- {$DOOMDIR,~/.emacs.d}/modules/*/*/init.el
- `doom-before-init-modules-hook'
- {$DOOMDIR,~/.emacs.d}/modules/*/*/config.el
- `doom-init-modules-hook'
- $DOOMDIR/config.el
- `doom-after-init-modules-hook'
- `after-init-hook'
- `emacs-startup-hook'
- `window-setup-hook'
Load order after this change:
- $EMACSDIR/early-init.el
- $EMACSDIR/lisp/doom.el
- $EMACSDIR/lisp/doom-start.el
- $DOOMDIR/init.el
- `doom-before-modules-init-hook'
- {$DOOMDIR,~/.emacs.d}/modules/*/*/init.el
- `doom-after-modules-init-hook'
- `doom-before-modules-config-hook'
- {$DOOMDIR,~/.emacs.d}/modules/*/*/config.el
- `doom-after-modules-config-hook'
- $DOOMDIR/config.el
- `after-init-hook'
- `emacs-startup-hook'
- `window-setup-hook'
I move our hackiest and least offensive startup optimizations to core,
so they're easy for me to keep track of (they'll likely change often,
between major Emacs releases), to keep them from affecting non-Doom
profiles, and make it easy for readers to use as a reference.
These only benefit interactive sessions, and doom-start's responsibility
is to configure interactive sessions; it doesn't make sense to keep
these in core.
b7f84bd introduced a nasty regression that caused an infinite loop and
runaway memory usage on some pgtk+native-comp builds of Emacs when it
attempted to perform deferred native compilation of your packages. This
would make Emacs unusable and, if left alone, could even crash your
system.
The only Emacs builds I'm certain are affected are derived from
flatwhatson/emacs (like emacs-pgtk-native-comp on Guix and Arch Linux in
particular). 28.1 stable and master (on emacs-mirror/emacs@e13509468b)
are unaffected.
It appears that some, earlier pgtk builds stack idle timers differently.
I'm not entirely sure how, because it doesn't manifest in more recent
builds of Emacs, and I'm already burnt out on debugging this, but here's
how Doom encountered it:
Doom has an incremental package loader; it loads packages, piecemeal,
after Emacs has been idle for 2s, then again every 0.75s until it
finishes or the user sends input (then it waits another 2s before
starting again). However, if at any time the iloader detected that
native-compilation is in progress, it waits 2s before trying
again (repeat, until native-comp is done). But here's the catch, given
the following:
(run-with-idle-timer
2 nil (lambda ()
(run-with-idle-timer 1 nil (lambda () (message "hi")))))
I had assumed "hi" would be emitted after 3 seconds (once idle), but
instead it is emitted after 2. Like this, Doom's iloader would elapse
one idle timer directly into another, ad infinitum, until Emacs was
forcibly killed.
By switching to run-at-time and employing my own rudimentary idle timer,
I avoid this issue. Also, the iloader no longer needs to be considerate
of native-comp, because the latter does its own rate-limiting controlled
by native-comp-async-jobs-number.
Amend: b7f84bdd01
And emit more informative errors if they fail.
This eval-when-compile approach is used in preparation for v3, where
Doom's core libraries will be byte-compiled.
Rather than impose a 10-45min compilation step on users, I've disabled
ahead-of-time compilation for deferred compilation. In exchange, it will
eat up some CPU time the first time each uncompiled package is loaded,
but as this happens asynchronously (and are then quietly loaded in the
background), I think this is acceptable.
An --aot switch (or similar) will be added to `doom sync` and `doom
build` in the future, in case folks prefer the old behavior.
startup-redirect-eln-cache adds the new directory and removes
$EMACSDIR/eln-cache from native-comp-eln-load-path, but it's not
available in 28.1, so we'll have to wait until Doom drops 28.1 support
to use it.
doom-etc-dir will be renamed to doom-data-dir, to better reflect its
purpose, and align it with XDG_DATA_HOME (where it will be moved to in
v3, where Doom will begin to obey XDG directory conventions more
closely).
- Deprecates the doom-private-dir variable in favor of doom-user-dir.
- Renames the pseudo category for the user's module: :private -> :user.
- Renames the doom-private-error error type to doom-user-error.
Emacs uses the term "user" to refer to the "things" in user space (e.g.
user-init-file, user-emacs-directory, user-mail-address, xdg-user-dirs,
package-user-dir, etc), and I'd like to be consistent with that. It also
has the nice side-effect of being slightly shorter. I also hope
'doom-user-error' will be less obtuse to beginners than
'doom-private-error'.
To reduce redundancy, remove the maintenance hassle that version
constants would impose later on, and rely on built-in
facilities (featurep) more over global variables or doomisms, these
global constants have been deprecated in favor of Emacs "features":
- EMACS28+ -- replace with (> emacs-major-version 27)
- EMACS29+ -- replace with (> emacs-major-version 28)
- NATIVECOMP -- replace with (featurep 'native-compile)
- MODULES -- replace with (featurep 'dynamic-modules)
(These constants will be formally removed when v3 is released. The IS-*
constants are likely next, but I haven't decided on their substitutes
yet)
I also decided to follow native-compile's example and provide features
for Emacs' system features (since system-configuration-features' docs
outs itself as a poor method to detect features):
- dynamic-modules
- jansson
- native-compile -- this one already exists, but will instead be removed
if it's non-functional; i.e. (native-comp-available-p) returns nil.
These are now detectable using featurep, which is fast and built-in.
BREAKING CHANGE: This restructures the project in preparation for Doom
to be split into two repos. Users that have reconfigured Doom's CLI
stand a good chance of seeing breakage, especially if they've referred
to any core-* feature, e.g.
(after! core-cli-ci ...)
To fix it, simply s/core-/doom-/, i.e.
(after! doom-cli-ci ...)
What this commit specifically changes is:
- Renames all core features from core-* to doom-*
- Moves core/core-* -> lisp/doom-*
- Moves core/autoloads/* -> lisp/lib/*
- Moves core/templates -> templates/
Ref: #4273
