refactor!: complete profile gen and init systems

BREAKING CHANGE: This commit makes three breaking changes:

- Doom now fully and dynamically generates (and byte-compiles) your
  profile and its init files, which includes your autoloads, loading
  your init files and modules, and then some. This replaces
  doom-initialize-modules, doom-initialize-core-modules, and
  doom-module-loader, which have been removed. This has also improved
  startup time by a bit, but if you use these functions in your CLIs,
  for instance, this will be a breaking change.
- `doom sync` is now required for Doom to see your profiles (and must be
  run whenever you change them, or when you up/downgrade Emacs across
  major versions).
- $DOOMDIR/init.el is now read much earlier than it used to be. Before
  any of doom-{ui,keybinds,editor,projects}, before any autoloads are
  loaded, and before your load-path has been populated with your
  packages. It now runs in the context of early-init.el, giving users
  freer range over what they can affect, but a more minimalistic
  environment to do it in.

  If you must have some logic run when all that is set up, add it to one
  of the module hooks added in e08f68b or 283308a.

This also poses a significant change to Doom's load order (see the
commentary change in lib/doom.el), along with the following (non
breaking) changes:

1. Adds a new `doom profiles sync` command. This will forcibly resync
   your profiles, while `doom sync` will only do so if your profiles
   have changed.
2. Doom now fully and dynamically generates (and byte-compiles) your
   user-init-file, which includes loading all your init files, modules,
   and custom-file. This replaces the job of doom-initialize-modules,
   doom-initialize-core-modules, and doom-module-loader, which have been
   removed. This has also improved startup time by a bit.
3. Defines new doom-state-dir variable, though not used yet (saving that
   and the other breaking changes for the 3.0 release).
4. Redesigns profile directory variables (doom-profile-*-dir) to prepare
   for future XDG-compliance.
5. Removed unused/unimportant profile variables in doom.el.
6. Added lisp/doom-profiles.el. It's hardly feature complete, but it's
   enough to power the system as it is now.
7. Updates the "load order" commentary in doom.el to reflect these
   changes.
This commit is contained in:
Henrik Lissner 2022-09-15 18:53:06 +02:00
parent 3d6e0311b9
commit b914830403
No known key found for this signature in database
GPG key ID: B60957CA074D39A3
15 changed files with 981 additions and 402 deletions

View file

@ -1,17 +1,154 @@
#+title: Doom's profile directory
This directory houses Doom's profiles (both generated or static), which in turn
will contain all "local" data for that profile, including packages, caches,
server files, and so on. It's also where generated files (like autoloads) are
written to.
* Introduction
In order to power Doom's soon-to-be generational package manager, I wrote a
profile system. This system can effectively replace [[https://github.com/plexus/chemacs2][Chemacs]]; permitting you to
switch between multiple Emacs configs on-demand (and those configs don't have to
be Doom configs).
This directory may serve as an alternative to =$EMACSDIR/profiles.el= for
[[https://github.com/doomemacs/doomemacs/commit/5b6b204bcbcf69d541c49ca55a2d5c3604f04dad][declaring profiles]]: each directory here is an implicit profile, so assuming
=$EMACSDIR/profiles/foo/init.el= exists, then ~emacs --profile foo~ will be
equivalent to ~emacs --init-directory $EMACSDIR/profiles/foo~.
While I work on the formal documentation for this system, I've created this
brief guide to walk users through their use. *However, for this to work, Doom
must live in =~/.emacs.d= or =~/.config/emacs=.* I'll refer to this as
=$EMACSDIR= (and your private Doom config, in =~/.doom.d= or =~/.config/doom=,
as =$DOOMDIR=).
#+begin_quote
*Warning:* Generated (or static) profiles will follow the =X@Y= naming
convention. To avoid conflicts, avoid naming any profile you put in here the
same way. For example: =default@latest=, =test@942=, =safe-mode@static=,.
#+end_quote
* How use profiles
1. Declare all your profiles in either:
- One or multiple profile files at:
- =$DOOMDIR/profiles.el=
- =$EMACSDIR/profiles.el=
- =~/.config/doom-profiles.el=
- =~/.doom-profiles.el=
[[id:f9bce7da-d155-4727-9b6f-b566b5b8d824][Example profiles.el file]].
- Or an implicit profile, which are inferred from the sub-directories of:
- =$DOOMDIR/profiles/=
- =$EMACSDIR/profiles/=
Implicit profiles may have a =.doomprofile= file to apply additional
settings. [[id:ac37ac6f-6082-4c34-b98c-962bc1e528c9][Example .doomprofile]].
2. To run ~$ doom sync~ whenever you change the above, to regenerate Doom's
cached profile loader (generated at =$EMACSDIR/profiles/init.X.elc=, where X
is your major Emacs version).
3. To launch them:
- Launch the profile you want: ~$ emacs --profile FOO~
- Use ~bin/doom~ on the profile you want: ~$ doom sync --profile FOO~
* Auto-generated profiles
Doom v3's sandbox and transactional package manager are capable of generating
profiles on-the-fly. The former for rapid, isolated testing, and the latter for
rollback/snapshot traversal for disaster recovery purposes.
These auto-generated profiles will be stored and versioned in:
=$XDG_DATA_HOME/doom/$PROFILE_NAME/@/$PROFILE_VERSION/=
* Fallback profile
Unlike Chemacs, Doom's profiles has no notion of a "default"/fallback profile.
The fallback profile is the Doom installation doing the bootloading. This
"global" profile is unique in that it won't respect a =.doomprofile= -- in other
words, it's not treated as a normal profile.
It is this way so that the profiles system imposes no overhead on users that
aren't interested in the profile system (or prefer to use Chemacs).
However, you can emulate this behavior by registering the "global" profile as a
profile, and setting ~$DOOMPROFILE~ or aliasing ~emacs~, like so:
#+begin_src emacs-lisp
;; in a profiles.el file
((default)
...)
#+end_src
#+begin_src bash
# in .zshrc or .bash_profile
export DOOMPROFILE=default
# Or
alias emacs='emacs --profile default'
#+end_src
* Gotchas
There are two caveats with this profile system:
- It requires that Doom live in =~/.config/emacs= or =~/.emacs.d=. A
non-standard install location won't work, unless you use Emacs 29's new
=--init-directory DIR= switch and launch Emacs with ~emacs --init-directory
~/nonstandard/emacs.d --profile NAME~. =bin/doom= is fine with it, though.
- The profile system can be storage-inefficient. A barebones Doom config
averages at ~1mb without installed packages and ~3.75mb /with/ (straight alone
is 2.6m). A fully-fledged Doom config can average 500mb-1.4gb; the majority of
which are packages, but include server binaries, elisp+native bytecode, and
caches add up too.
To mitigate this, Doom dedups packages across snapshots of a single profile
(e.g. =profile@23= -> =profile@24=), but it cannot (yet) do this across
profiles (e.g. if =profile1= and =profile2= both install =org=). Even then,
packages whose recipes change (either locally or upstream) may dodge this
deduplication and get cloned anew (to ensure historical integrity) -- though
this shouldn't happen often, but can build up over time.
So v3 will introduce a ~doom gc~ command, which offers a couple nix.gc-esque
switches to control it. E.g.
- Acts on the "global" profile:
- ~doom gc --older-than 21d~
- ~doom gc --keep 10~
- Act on a specific profile:
- ~doom gc --profile foo ...~
- Act on all known profiles
- ~doom gc --profiles '*' ...~
Users can change defaults from their =init.el= or =cli.el=, or configure ~doom
sync~ to auto-GC by whatever rules they like. And the good doctor will warn
you if you haven't GCed in a while, or you're in excess of some threshold
(which I haven't decided yet).
* How to switch from Chemacs
1. Delete [[https://github.com/plexus/chemacs2][Chemacs]] from =$EMACSDIR=.
2. Install Doom there: ~$ git clone https://github.com/doomemacs/doomemacs
~/.config/emacs~
3. Move =~/.emacs-profiles.el= to =~/.config/doom/profiles.el= and transform the
string keys to symbols and adapt =env= entries like so:
#+begin_src emacs-lisp
;; ~/.emacs-profiles.el
(("default" (user-emacs-directory . "~/.emacs.default")
(env ("DOOMDIR" . "~/.doom.private")))
("spacemacs" (user-emacs-directory . "~/spacemacs"))
("prelude" (user-emacs-directory . "~/prelude")))
;; ~/.config/emacs/profiles.el
((default (user-emacs-directory . "~/.emacs.default")
("DOOMDIR" . "~/.doom.private"))
(spacemacs (user-emacs-directory . "~/spacemacs"))
(prelude (user-emacs-directory . "~/prelude")))
#+end_src
A comprehensive example of Doom's profiles.el file can be found
[[id:f9bce7da-d155-4727-9b6f-b566b5b8d824][in docs/examples.org]].
*Differences with Chemacs profiles:*
- Keys are symbols, not strings.
- Doom's profiles.el has a syntax for evaluating code, expanding paths, and
appending/prepending to variables (with deferral). See the examples.org
link above.
- Doom's profile system won't install [[https://github.com/raxod502/straight.el][Straight.el]] for you.
- Doom does not have a special "default" profile. If you don't specify a
--profile, it will simply start up the Doom config living in
=~/.config/emacs=. See the "Fallback profile" section below for a
workaround.
4. Then launch a profile. E.g. ~$ emacs --profile prelude~.
* But Doom is kinda heavy to be a bootloader...
I agree! To remedy that, I'll soon split Doom up into three projects: its core
(where its bootloader lives), its official modules, and its community
contributed modules. At that point, Doom will be much lighter!