checkers/spell: replace flyspell with spell-fu

Spell-fu is significantly faster, but does produce more false
positives (more faces must be added to spell-fu-faces-exclude to reduce
these).

Unfortunately, there is no fancy "correction" interface for spell-fu
yet, so we'll have to resort to ispell-word (on z=) for now.

modules/checkers/spell/README.org

@@ -11,76 +11,79 @@
 - [[#prerequisites][Prerequisites]]
 - [[#features][Features]]
 - [[#configuration][Configuration]]
+  - [[#changing-how-quickly-spell-fu-spellchecks-after-changes][Changing how quickly spell-fu spellchecks after changes]]
+  - [[#reducing-false-positives-by-disabling-spelling-on-certain-faces][Reducing false positives by disabling spelling on certain faces]]
+  - [[#adding-or-removing-words-to-your-personal-dictionary][Adding or removing words to your personal dictionary]]
 - [[#troubleshooting][Troubleshooting]]
 
 * Description
 This modules provides spellchecking powered by =aspell= or =hunspell=.
 
-Spellcheck is automatically loaded on the following modes:
+Spellcheck is automatically loaded in all ~text-mode~ derivatives, which
-+ org
+includes ~org-mode~, ~markdown-mode~, the Git Commit buffer (from magit),
-+ markdown
+~mu4e-compose-mode~, and others.
-+ TeX
-+ rst
-+ mu4e-compose
-+ message
-+ git-commit
 
 ** Maintainers
 This module has no dedicated maintainers.
 
 ** Module Flags
-+ =+aspell= Use =aspell= as a backend for spellchecking.
++ =+aspell= Use =aspell= as a backend for correcting words.
-+ =+hunspell= Use =hunspell= as a backend for spellchecking.
++ =+hunspell= Use =hunspell= as a backend for correcting words.
-+ =+everywhere= Use spellcheck in every mode.
++ =+everywhere= Use spellcheck in prog-mode and conf-mode derivatives as well as
+  text-mode. Basically, enable =spell-fu-mode= everywhere.
 
 ** Plugins
-+ [[https://github.com/d12frosted/flyspell-correct][flyspell-correct]]
++ [[https://gitlab.com/ideasman42/emacs-spell-fu][spell-fu]]
-+ [[https://github.com/d12frosted/flyspell-correct#flyspell-correct-ivy-interface][flyspell-correct-ivy]] (=completion/ivy=)
-+ [[https://github.com/d12frosted/flyspell-correct#flyspell-correct-helm-interface][flyspell-correct-helm]] (=completion/helm=)
-+ [[https://github.com/d12frosted/flyspell-correct#flyspell-correct-popup-interface][flyspell-correct-popup]] (if *neither* =completion/ivy= or =completion/helm=)
-+ [[https://github.com/rolandwalker/flyspell-lazy][flyspell-lazy]]
 
 * Prerequisites
-This module requires either =aspell= or =hunspell= as backend. It will
+This module requires =aspell= to be installed, whether or not you intend to use
-automatically pick =aspell= if both are installed.
+=hunspell= as the word correction backend. =spell-fu= does not yet support
+generating the word list with a custom command or =hunspell=, but =hunspell= can
+still be used to provide correction suggestions when invoking ~ispell-word~.
 
-You can specify the backend with the =+aspell= or =+hunspell= flag.
+You will need =hunspell= installed (via your OS package manager) to use it as a
+correction backend.
 
 * Features
-+ Spellchecking and suggestions based on =aspell= or =hunspell=.
++ Spellchecking based on =aspell=.
-+ Choosing suggestions using completion interfaces (=ivy= or =helm=).
++ Spell correction using =aspell= or =hunspell= (through ~M-x ispell-word~).
-+ Lazily spellchecking recent changes only when idle.
++ Ignores source code inside org or markdown files.
-+ Ignores source code inside org documents.
 
-When using =+everywhere=, =flyspell-prog-mode= will be automatically loaded for
+When using =+everywhere=, ~spell-fu-mode~ is activated for as many major modes
-the following modes:
+as possible, and not only ~text-mode~ derivatives. =spell-fu= will only spell
-+ yaml-mode-hook
+check comments in programming major modes.
-+ conf-mode-hook
-+ prog-mode-hook
-
-=flyspell-prog-mode= will only spellcheck comments.
 
 * Configuration
 Dictionary is set by =ispell-dictionary= variable. Can be changed locally with
 the function =ispell-change-dictionary=.
 
-Lazy spellcheck is provided by =flyspell-lazy= package.
+** Changing how quickly spell-fu spellchecks after changes
-
+Adjust ~spell-fu-idle-delay~ to change how long spell-fu waits to spellcheck
-=flyspell-lazy-idle-seconds= sets how many idle seconds until spellchecking
+after recent changes (its default value as ~0.25~).
-recent changes (default as 1), while =flyspell-lazy-window-idle-seconds= sets
-how many seconds until the whole window is spellchecked (default as 3).
-
-If you want to add =flyspell-mode= or =flyspell-prog-mode= to a specific mode,
-use =add-hook!=. To remove from a mode, use =remove-hook!=:
 
 #+BEGIN_SRC elisp
-(add-hook! '(org-mode-hook markdown-mode-hook
+(after! spell-fu
-             git-commit-mode-hook) #'flyspell-mode)
+  (setq spell-fu-idle-delay 0.5))
 #+END_SRC
 
+** Reducing false positives by disabling spelling on certain faces
+Exclude what faces to not preform spellchecking on by setting
+~spell-fu-faces-exclude~ in a buffer-local hook:
+
 #+BEGIN_SRC elisp
-(remove-hook! '(markdown-mode-hook git-commit-mode-hook)
+(setq-hook! 'markdown-mode-hook
-#'flyspell-mode)
+  spell-fu-faces-exclude '(markdown-code-face
+                           markdown-reference-face
+                           markdown-link-face
+                           markdown-url-face
+                           markdown-markup-face
+                           markdown-html-attr-value-face
+                           markdown-html-attr-name-face
+                           markdown-html-tag-name-face))
 #+END_SRC
 
-* Troubleshooting
+** Adding or removing words to your personal dictionary
+Use ~M-x spell-fu-word-add~ and ~M-x spell-fu-word-remove~ to whitelist words
+that you know are not misspellings.
+
+* TODO Troubleshooting

modules/checkers/spell/autoload.el

@@ -1,27 +1,99 @@
-;;; checkers/spell/autoload.el -*- lexical-binding: t; -*-
+;;; checkers/spell/autoload/ivy.el -*- lexical-binding: t; -*-
 
-;;;###autodef
+(defun +spell--correct (replace poss word orig-pt start end)
-(defalias 'flyspell-mode! #'flyspell-mode)
+  (cond ((eq replace 'ignore)
+         (goto-char orig-pt)
+         nil)
+        ((eq replace 'save)
+         (goto-char orig-pt)
+         (ispell-send-string (concat "*" word "\n"))
+         (ispell-send-string "#\n")
+         (setq ispell-pdict-modified-p '(t)))
+        ((or (eq replace 'buffer) (eq replace 'session))
+         (ispell-send-string (concat "@" word "\n"))
+         (add-to-list 'ispell-buffer-session-localwords word)
+         (or ispell-buffer-local-name ; session localwords might conflict
+             (setq ispell-buffer-local-name (buffer-name)))
+         (if (null ispell-pdict-modified-p)
+             (setq ispell-pdict-modified-p
+                   (list ispell-pdict-modified-p)))
+         (goto-char orig-pt)
+         (if (eq replace 'buffer)
+             (ispell-add-per-file-word-list word)))
+        (replace
+         (let ((new-word (if (atom replace)
+                             replace
+                           (car replace)))
+               (orig-pt (+ (- (length word) (- end start))
+                           orig-pt)))
+           (unless (equal new-word (car poss))
+             (delete-region start end)
+             (goto-char start)
+             (insert new-word))))
+        ((goto-char orig-pt)
+         nil)))
 
-(defvar +spell--flyspell-predicate-alist nil
+(defun +spell-correct-ivy-fn (candidates word)
-  "TODO")
+  (ivy-read (format "Corrections for %S: " word) candidates))
 
-;;;###autodef
+(defun +spell-correct-helm-fn (candidates word)
-(defun set-flyspell-predicate! (modes predicate)
+  (helm :sources (helm-build-sync-source
-  "TODO"
+                  "Ispell"
-  (declare (indent defun))
+                  :candidates candidates)
-  (dolist (mode (doom-enlist modes) +spell--flyspell-predicate-alist)
+        :prompt (format "Corrections for %S: " word)))
-    (add-to-list '+spell--flyspell-predicate-alist (cons mode predicate))))
+
+(defun +spell-correct-generic-fn (candidates word)
+  (completing-read (format "Corrections for %S: " word) candidates))
 
 ;;;###autoload
-(defun +spell-init-flyspell-predicate-h ()
+(defun +spell/correct ()
-  "TODO"
+  "Correct spelling of word at point."
-  (when-let (pred (assq major-mode +spell--flyspell-predicate-alist))
+  (interactive)
-    (setq-local flyspell-generic-check-word-predicate (cdr pred))))
+  (if (not (or (featurep! :completion ivy)
-
+               (featurep! :completion helm)))
-;;;###autoload
+      (call-interactively #'ispell-word)
-(defun +spell-correction-at-point-p (&optional point)
+    (cl-destructuring-bind (start . end)
-  "TODO"
+        (bounds-of-thing-at-point 'word)
-  (cl-loop for ov in (overlays-at (or point (point)))
+      (let ((word (thing-at-point 'word t))
-           if (overlay-get ov 'flyspell-overlay)
+            (orig-pt (point))
-           return t))
+            poss ispell-filter)
+        (save-current-buffer
+          (ispell-accept-buffer-local-defs))
+        (ispell-send-string "%\n")
+        (ispell-send-string (concat "^" word "\n"))
+        (while (progn (accept-process-output ispell-process)
+                      (not (string= "" (car ispell-filter)))))
+        ;; Remove leading empty element
+        (setq ispell-filter (cdr ispell-filter))
+        ;; ispell process should return something after word is sent. Tag word as
+        ;; valid (i.e., skip) otherwise
+        (unless ispell-filter
+          (setq ispell-filter '(*)))
+        (when (consp ispell-filter)
+          (setq poss (ispell-parse-output (car ispell-filter))))
+        (cond
+         ((or (eq poss t) (stringp poss))
+          ;; don't correct word
+          (message "%s is correct" (funcall ispell-format-word-function word))
+          t)
+         ((null poss)
+          ;; ispell error
+          (error "Ispell: error in Ispell process"))
+         (t
+          ;; The word is incorrect, we have to propose a replacement.
+          (setq res (funcall +spell-correct-interface (nth 2 poss) word))
+          ;; Some interfaces actually eat 'C-g' so it's impossible to stop rapid
+          ;; mode. So when interface returns nil we treat it as a stop.
+          (unless res (setq res (cons 'break word)))
+          (cond
+           ((stringp res)
+            (+spell--correct res poss word orig-pt start end))
+           ((let ((cmd (car res))
+                  (wrd (cdr res)))
+              (unless (or (eq cmd 'skip)
+                          (eq cmd 'break)
+                          (eq cmd 'stop))
+                (+spell--correct cmd poss wrd orig-pt start end)
+                (unless (string-equal wrd word)
+                  (+spell--correct wrd poss word orig-pt start end))))))
+          (ispell-pdict-save t)))))))

modules/checkers/spell/config.el

@@ -1,7 +1,54 @@
 ;;; checkers/spell/config.el -*- lexical-binding: t; -*-
 
-(defvar ispell-dictionary "en_US")
+(defvar +spell-correct-interface
+  (cond ((featurep! :completion ivy)
+         #'+spell-correct-ivy-fn)
+        ((featurep! :completion helm)
+         #'+spell-correct-helm-fn)
+        (#'+spell-correct-generic-fn))
+  "Function to use to display corrections.")
 
+(defvar +spell-excluded-faces-alist
+  '((markdown-mode
+     . (markdown-code-face
+        markdown-html-attr-name-face
+        markdown-html-attr-value-face
+        markdown-html-tag-name-face
+        markdown-link-face
+        markdown-markup-face
+        markdown-reference-face
+        markdown-url-face))
+    (org-mode
+     . (org-block
+        org-block-begin-line
+        org-block-end-line
+        org-code
+        org-date
+        org-formula
+        org-latex-and-related
+        org-link
+        org-meta-line
+        org-property-value
+        org-ref-cite-face
+        org-special-keyword
+        org-tag
+        org-todo
+        org-todo-keyword-done
+        org-todo-keyword-habt
+        org-todo-keyword-kill
+        org-todo-keyword-outd
+        org-todo-keyword-todo
+        org-todo-keyword-wait
+        org-verbatim)))
+  "Faces in certain major modes that spell-fu will not spellcheck.")
+
+
+;;
+;;; Packages
+
+(global-set-key [remap ispell-word] #'+spell/correct)
+
+(defvar ispell-dictionary "en_US")
 (after! ispell
   ;; Don't spellcheck org blocks
   (pushnew! ispell-skip-region-alist
@@ -39,74 +86,21 @@
     (_ (doom-log "Spell checker not found. Either install `aspell' or `hunspell'"))))
 
 
-(use-package! flyspell ; built-in
+(use-package! spell-fu
-  :defer t
+  :hook (text-mode . spell-fu-mode)
-  :preface
-  ;; `flyspell' is loaded at startup. In order to lazy load its config we need
-  ;; to pretend it isn't loaded.
-  (defer-feature! flyspell flyspell-mode flyspell-prog-mode)
   :init
-  (add-hook! '(org-mode-hook
+  (setq spell-fu-directory (concat doom-etc-dir "spell-fu"))
-               markdown-mode-hook
-               TeX-mode-hook
-               rst-mode-hook
-               mu4e-compose-mode-hook
-               message-mode-hook
-               git-commit-mode-hook)
-             #'flyspell-mode)
-
   (when (featurep! +everywhere)
     (add-hook! '(yaml-mode-hook
                  conf-mode-hook
                  prog-mode-hook)
-               #'flyspell-prog-mode))
+               #'spell-fu-mode))
-
   :config
-  (setq flyspell-issue-welcome-flag nil
+  (add-hook! 'spell-fu-mode-hook
-        ;; Significantly speeds up flyspell, which would otherwise print
+    (defun +spell-init-excluded-faces-h ()
-        ;; messages for every word when checking the entire buffer
+      (when-let (excluded (alist-get major-mode +spell-excluded-faces-alist))
-        flyspell-issue-message-flag nil)
+        (setq-local spell-fu-faces-exclude excluded))))
 
-  (add-hook! 'flyspell-mode-hook
+  ;; TODO custom `spell-fu-check-range' function to exclude URLs from links or
-    (defun +spell-inhibit-duplicate-detection-maybe-h ()
+  ;;      org-src blocks more intelligently.
-      "Don't mark duplicates when style/grammar linters are present.
+  )
-e.g. proselint and langtool."
-      (and (or (and (bound-and-true-p flycheck-mode)
-                    (executable-find "proselint"))
-               (featurep 'langtool))
-           (setq-local flyspell-mark-duplications-flag nil))))
-
-  ;; Ensure mode-local predicates declared with `set-flyspell-predicate!' are
-  ;; used in their respective major modes.
-  (add-hook 'flyspell-mode-hook #'+spell-init-flyspell-predicate-h)
-
-  (let ((flyspell-correct
-         (general-predicate-dispatch nil
-           (and (not (or mark-active (ignore-errors (evil-insert-state-p))))
-                (memq 'flyspell-incorrect (face-at-point nil t)))
-           #'flyspell-correct-at-point)))
-    (map! :map flyspell-mouse-map
-          "RET"    flyspell-correct
-          [return] flyspell-correct
-          [mouse-1] #'flyspell-correct-at-point)))
-
-
-(use-package! flyspell-correct
-  :commands flyspell-correct-previous
-  :general ([remap ispell-word] #'flyspell-correct-at-point)
-  :config
-  (cond ((and (featurep! :completion helm)
-              (require 'flyspell-correct-helm nil t)))
-        ((and (featurep! :completion ivy)
-              (require 'flyspell-correct-ivy nil t)))
-        ((require 'flyspell-correct-popup nil t)
-         (setq flyspell-popup-correct-delay 0.8)
-         (define-key popup-menu-keymap [escape] #'keyboard-quit))))
-
-
-(use-package! flyspell-lazy
-  :after flyspell
-  :config
-  (setq flyspell-lazy-idle-seconds 1
-        flyspell-lazy-window-idle-seconds 3)
-  (flyspell-lazy-mode +1))

modules/checkers/spell/packages.el

@@ -1,11 +1,4 @@
 ;; -*- no-byte-compile: t; -*-
 ;;; checkers/spell/packages.el
 
-(package! flyspell-correct :pin "dea1290a371c540dde7b8d0eef7a12d92f7a0b83")
+(package! spell-fu :pin "e94d01cdc822e02968971cde09276047a5d55772")
-(cond ((featurep! :completion ivy)
-       (package! flyspell-correct-ivy))
-      ((featurep! :completion helm)
-       (package! flyspell-correct-helm))
-      ((package! flyspell-correct-popup)))
-
-(package! flyspell-lazy :pin "3ebf68cc9eb10c972a2de8d7861cbabbbce69570")

modules/config/default/+emacs-bindings.el

@@ -280,7 +280,7 @@
         :desc "org-tree-slide mode"        "p" #'org-tree-slide-mode)
        :desc "Read-only mode"               "r" #'read-only-mode
        (:when (featurep! :checkers spell)
-        :desc "Flyspell"                   "s" #'flyspell-mode)
+        :desc "Spell checker"              "s" #'spell-fu-mode)
        (:when (featurep! :lang org +pomodoro)
         :desc "Pomodoro timer"             "t" #'org-pomodoro)
        (:when (featurep! :ui zen)

modules/config/default/+evil-bindings.el

@@ -662,7 +662,7 @@
         :desc "org-tree-slide mode"        "p" #'org-tree-slide-mode)
        :desc "Read-only mode"               "r" #'read-only-mode
        (:when (featurep! :checkers spell)
-        :desc "Flyspell"                   "s" #'flyspell-mode)
+        :desc "Spell checker"              "s" #'spell-fu-mode)
        (:when (featurep! :lang org +pomodoro)
         :desc "Pomodoro timer"             "t" #'org-pomodoro)
        :desc "Soft line wrapping"           "w" #'visual-line-mode

modules/lang/markdown/config.el

@@ -53,8 +53,6 @@ capture, the end position, and the output buffer.")
     (add-to-list 'org-src-lang-modes '("md" . markdown)))
 
   :config
-  (set-flyspell-predicate! '(markdown-mode gfm-mode)
-    #'+markdown-flyspell-word-p)
   (set-lookup-handlers! '(markdown-mode gfm-mode)
     ;; `markdown-follow-thing-at-point' may open an external program or a
     ;; buffer. No good way to tell, so pretend it's async.