This introduces a depth field for modules so that they may dictate their
load order explicitly, it also treats depths <= -100 or >= 100 as
special depths, which will be loaded early, before their respective
doom-{before,after}-module-{init,config}-hook. This permits psuedo
modules like :core and :user modules to be treated as normal modules
without too many special cases.
This also fixes a module load order issue on Emacs 29 (#6813), caused by
emacs-mirror/emacs@4311bd0bd7, which changed the return value order of
hash-table-{keys,values} causing modules to be loaded in reverse order;
resulting in the loss of evil keybinds, among other things.
Other notable changes:
- Changes the data structure for module data caches from a list to a
vector. Uses less memory and permits faster lookups. Also adds two
depth fields to the front of it.
- Changes the signature of doom-module-list and doom-package-list.
- Renames doom--read-packages -> doom-packages--read for consistency
with naming convention.
- Add doom-module-depth function.
- Adds a temporary doom-core-dir/init.el file, which is responsible for
loading doom-*.el.
Fix: #6813
Ref: emacs-mirror/emacs@4311bd0bd7
An unintended change snuck into 2c14eff. The :core and :user virtual
modules are no longer stripped from the module list before iterating
through (and loading) them. This meant that Doom would load these two
like regular modules (and first, since these two are always at the start
of the list).
This is harmless for :core, because it has no init.el or config.el, but
:user does! This means $DOOMDIR/{init,config}.el would be loaded
twice (once before all other modules and again afterwards), causing load
order issues (like #6818).
Fix: #6818
Amend: 2c14eff7f1
- Batch more variables in Doom's autoloads files.
- Remove all the register-definition-prefixes calls generated in
autoloads files (for both modules' and packages' autoloads). These
don't serve much purpose, and only incur added cost growing a large
hash table.
Any buffers created before after-init-hook could trigger these hooks,
which may house expensive functions, but never anything that is
important at startup time.
However, it must not occur later than after-init-hook (which triggers
before file arguments are processed and file buffers are created).
To understand this issue, you have to understand these two things:
1. Doom builds an init file which combines all its autoloads (for
packages and modules), and Doom's bootstrapper (which loads modules,
$DOOMDIR, etc). This init file is byte-compiled.
2. When Emacs byte-compiles elisp, docstrings are lazy-loaded (by
embedding them in the elc as commented text to be retrieved later).
This is generally done to save on memory.
Now the issue: when these lazy-loaded docstrings are retrieved, Emacs
may evaluate the whole file to find it, including Doom's bootstrap
process, reloading all its files, the user's config files, and running
all its startup hooks. Not only is this terribly expensive, reloading
these files may have disastrous effects.
One such effect is compounded by Marginalia, which invokes this
docstring fetch process (by calling the `documentation` function in
`marginalia--function-doc`) for *each* symbol in the `M-x` or `C-h
{v,f}` completion lists, which means Doom re-bootstraps multiple times
and rapidly, causing Emacs to totally lock up.
The solution is to simply gate the expensive part of the initfile so it
doesn't run more than once, at startup, and when `doom/reload` is
called. The rest of the file loads instantly.
Still, this is a bit flimsy. I'll think of a more elegant solution
later.
- Since its arguments aren't used, make the advice n-arity, to future
proof the advice.
- Add commentary on load's side-effect on user-init-file.
- Add NOSUFFIX arg to load call, to spare Emacs the file IO of searching
for init.%d.elc{.{so{,.gz},elc{,.gz},el{,.gz},,gz}}.
Due to $DOOMPROFILE being set to an empty string when persisting Doom
CLI sessions, which would affect any case where a CLI command restarts
the session (e.g. when the :config literate module tangles a config or
'doom --debug ...' restarts to set DEBUG=1).
What used to be a `byte-compiled-config` trait, displayed in your `M-x
doom/info`, is now `compiled-user-config`, `compiled-core`, and
`compiled-modules`, for more helpful granularity for debugging possible
byte-code issues.
Due to a technical limitation of Emacs <=28, launching Emacs out of a
non-standard location is non-trivial, and `doom run` tries to promise
that it can do so on demand. Emacs 29 does introduce a --init-directory
switch that would make this easy, but it'll be some time before we can
rely on it.
So 'doom run' creates a fake $HOME in /tmp/doom.run/ and writes a
bootloader there to load your Doom config remotely. But there's a
problem: in this fake $HOME, none of the user's config, cache, data, or
binscript directories are available, so I symlink them there. This
should at least resolve the most trivial incompatibilities (like the
lack of all-the-icons fonts, which typically get installed to
$HOME/.local/share/fonts/ -- see #6807), but there may be yet more edge
cases. Still, this is a good enough compromise for now.
Fix: #6807
In 4a25375, it seemed that only setting the variables to nil early
enough would be sufficient, but this turned out not to be the case.
There's no avoiding calling the mode to disable it.
Ref: 58c0de6841
Amend: 4a253757cb
- Adds $DOOMPROFILELOADFILE: Controls where to read and write the
profile loader. Changing this may be helpful for users on nix/guix,
who have deployed Doom to a read-only location. This sets
`doom-profile-load-file`.
- Changed profile load file's default location (used to be
$EMACSDIR/profiles/init.el, is now $EMACSDIR/profiles/load.el). The
gitignore was updated to reflect this.
- Adds $DOOMPROFILELOADPATH: A colon-delimited list of profile config
files and directories (semi-colon on Windows) which dictate what Doom
reads in order to discover your profiles. Config files are required to
have an *.el extension. This sets `doom-profile-load-path`.
- Changes the nomenclature around this loader script. I used to refer to
it as the profile bootstrapper. I'll now refer to it as the profile
load file, and I've renamed `doom-profiles-bootstrap-file` to
`doom-profile-load-file` to reflect this.
- The variables `doom-profile-dirs` and `doom-profile-config-files` were
merged into doom-profile-load-path.
- Both envvars have also been documented in `doom help` (and
$DOOMPROFILE's has been updated).
Ref: #6794
Particularly DOOMPROFILE, without which the --profile switch wasn't
actually doing anything, and profile sessions would (silently) use the
default user-emacs-directory and doom-user-dir.
- Swap out the funcall+alist lookup for a pcase (which is expanded to a
cond, which is is faster and easier to read).
- Wrap bootstrap file to $EMACSDIR/profiles/init.el, but byte-compile it
to $EMACSDIR/profiles/init.X.el where X is emacs-major-version.
- Make doom-profiles-save's second argument optional (defaults to
doom-profiles-bootstrap-file).
- Make doom-profiles-save throw a error if byte-compilation fails for
some reason.
- Rename the tempvars to include 'doom' in their name, so debuggers know
where they originate.
In this commit I start using doom-profile-*-dir vars, though it will
make no difference to the default Doom install (only to profiles, and in
the future, in v3, where we'll drop $EMACSDIR/.local entirely).
The profile bootstrap file's first form is the doom-version it was
generated with. If this has changed, it should be considered outdated,
even if the user's profiles haven't changed.
If Doom doesn't live in ~/.emacs.d or ~/.config/emacs, then it cannot
play the role of bootloader, so opt out of generating the profile
bootstrappper in this case.
That said, don't disable the profile system entirely; it can still be
useful for internal, noninteractive, and sandbox use.
So that the resulting file expands to less code and doesn't apply too
magic to file paths (it may be unwanted).
And don't try to unintern a lexical binding.
Fixes two issues with implicit profiles:
1. Where user-emacs-directory (and sometimes doom-user-dir) would be
unexpanded in the profile init file (generated by 'doom profiles
sync'), which would be ineffective at runtime.
2. Where an implicit profile with a .doomprofile that lacks a
user-emacs-directory setting would not have any user-emacs-directory
set for it at all. Instead, it should fall back to that profile's
location.
The sudden spike in CPU and memory utilization alarms people, so I've
reduced how many cores native-comp will use. In non-interactive
sessions, it will use all of them, however (that is, when I later
introduce an AOT switch).
You can still override this by setting native-comp-async-jobs-number or
comp-num-cpus yourself.
I use advice instead of setting comp-num-cpus so that users to avoid
trampling on default behavior, or attempts by the user to change them.
It really isn't important if this function succeeds or not, but it seems
its stability is highly variable, dependent on your specific build and
version of Emacs. Since there are a lot of 29 Doom users, best to be
more permissive and simply fall back to byte-compilation if native-comp
fails.
Causing the envvar file to be generated to wrong place, and thus never
be updated/properly loaded at runtime.
This new setting is for later, where I'll integrate the envvar generate
into the profile generator proper.
