Headless Configuration Again

Desktop
,
1

Can’t Start GNOME Display Manager for Remote Access, What Am I Doing Wrong?

I’m a retired mainly programmer with some small, badly outdated experience in *x system administration and not remotely (Ha!) competent in X11/GNOME ecosystems.

There’s a project I want to do with a brother, and so I’ve been trying to set up a headless Ubuntu 24.04 LTS server with an NVIDIA GPU for remote GNOME sessions, keeping the console as a TTY on a Matrox video adapter. I expect to have him connect via VPN so that’ll handle security.

I gave up out-of-the-box xrdp on my Ubuntu distribution. Upon advice from an xrdp developer[1] I’ve tried the GNOME ecosystem. Despite extensive troubleshooting, I keep hitting errors, particularly around gdm-headless.service failing to start due to authentication issues and session management problems. I evidently need help to build a reliable setup.

I was very excited to find this but it seems it’s not meant for noobs: I don’t understand D-Bus, systemd, PipeWire, RDP, GNOME, and who knows what else anywhere near well enough to pull it off and if I ever did manage it, I’d forget what I did in a couple days. The system integrator and programmer in me likes it though. Kudos to Joan Torres López.

X2Go has been suggested as an alternative. Never heard of it before. Wisdom appreciated.

Questions for GNOME and GNOME Remote Desktop Developers

  1. What am I doing wrong with the headless GDM3 setup? Everything seems to revolve around interprocess communication and authentication errors. It was the same on xrdp FWIW.
  2. Should I start with a “headful” GNOME setup? I’ve only tried a headless configuration. Would setting up GNOME to work locally on the NVIDIA GPU first, then adding remote sessions, be more reliable? At least I’d be using the Ubuntu installers. Are dummy plugs on the NVIDIA card’s DVI ports necessary for headless operation? This card has two independent monitor ports for dual monitor operation. That’s not reflected in /etc/X11/xorg.conf.d/10-gpu.conf
  3. Is there an authoritative guide? Is there an official or recommended guide for setting up headless GNOME remote desktop with GDM3 and gnome-remote-desktop on Ubuntu 24.04 using stable GNOME 46 components?
  4. Best practices for reliability? What configuration would a professional sysadmin use to ensure a robust, maintainable headless RDP setup with GNOME, particularly for session management and PipeWire integration?

Thanks so much,
t

Below is hardware, software environment, goal, steps taken, and configuration changes

Goal

I want to configure Ubuntu 24.04 LTS server with:

  • Headless Operation: No local graphical display, using an NVIDIA GPU (GeForce GTX 1060 3Gb) for remote GUI sessions (and compute tasks)
  • TTY Console: Local console on a Matrox G200eH2 video adapter, kept as a TTY.
  • Remote GUI Sessions: Stable, reliable remote desktop access. I am not married to any particular desktop. I have preferred RDC because Microsoft Remote Connection is ubiquitous and RDP is reputed to be better for higher-latency WAN operation vs. VNC. But I’m not married to that either. This attempt has been GNOME so that’s what I’ll talk about.
  • Sysadmin Best Practices: A configuration that a professional sysadmin would implement for maximum reliability and maintainability, using stable GNOME ecosystem components.
  • Never Tried “Headful”: I’ve only pursued a headless configuration, no local graphical session on the NVIDIA GPU
    • Should I first make GNOME work “headfully” on the GPU and then add remote sessions?
    • Would dummy plugs on the NVIDIA card’s monitor ports (both HDMI/DVI) be advisable or necessary?
    • Is there an official or authoritative guide for this setup using stable GNOME 46 components?

Hardware Environment

  • Server: HP ProLiant ML350e Gen8
  • CPU: Intel Xeon E5-2430L v2
  • RAM: 96 GB
  • Video Adapters:
    • Matrox G200eH2: Integrated, used for TTY console (PCI 1@0:0:1, device ID 102b:0533).
    • NVIDIA GeForce GTX 1060 3GB: GPU for remote sessions (PCI 13@0:0:0, device ID 10de:1c02, driver nvidia-driver-560).
  • Monitors (for remote sessions): Dell U2713HM (2560x1440), Dell U2414H (1920x1080), IBM 9491-HB2 (1280x1024, VGA/SVGA) - this is the system console but has a DVI input and could be used temporarily to provide a head for the NVIDIA card.

Software Environment

  • OS: Ubuntu 24.04 LTS (with thorough cleanup after failed xrdp/Xfce attempt, see xrdp GitHub discussion).
  • Kernel: 6.8.0-58-generic
  • Display Manager: GDM3 (gdm3 package, version 46.0-2ubuntu1).
  • Desktop Environment: GNOME 46 (gnome-core, ubuntu-gnome-desktop).
  • Remote Desktop: gnome-remote-desktop (version 46.3-0ubuntu1).
  • X11 Configuration:
    • Xorg server (xserver-xorg, version 1:7.7+23ubuntu3).
    • dummy driver for headless operation.
    • /etc/X11/xorg.conf.d/10-gpu.conf for resolution support.
  • NVIDIA Driver: nvidia-driver-560 (installed via apt, confirmed with dpkg -l | grep nvidia).
  • Kernel Parameters: rd.driver.blacklist=nouveau nvidia-drm.modeset=1 nvidia-drm.fbdev=1 (in /etc/default/grub).
  • Installed Packages (partial list, full output available on request):
    • gdm3, gnome-session, gnome-shell, gnome-remote-desktop, xserver-xorg, xserver-xorg-video-dummy, nvidia-driver-560, pipewire, wireplumber, fuse (for fusermount).
  • Shift from Wayland to X11: Initially tried Wayland (default in Ubuntu 24.04), but switched to X11 due to better compatibility with gnome-remote-desktop for headless RDP.

List of Steps Tried

Initial Setup

  • Action: Ubuntu 24.04 LTS, cleaned xrdp/Xcfe updated packages:
  sudo apt update
  sudo apt upgrade --verbose-versions
  • Why: Avoid conflicts from a failed xrdp/Xfce setup.
  • Outcome: Clean system, but no graphical session by default.
  • Reference: Ubuntu 24.04 Server Guide [3].

Installed GNOME and GDM3

  • Action: Installed GNOME and related packages:
  sudo apt install ubuntu-gnome-desktop gdm3 gnome-remote-desktop
  • Why: To set up GNOME for remote desktop access via gnome-remote-desktop.
  • Outcome: GDM3 started but failed to initialize a headless session; logs showed dconf permission errors.
  • Reference: GNOME 46 Release Notes [4], Ubuntu Desktop Guide [5].

Switched to X11

  • Action: Edited /etc/gdm3/custom.conf to disable Wayland.
  • Why: Wayland was incompatible with headless RDP; X11 offered better support.
  • Outcome: GDM3 switched to X11, but remote sessions still failed.
  • Reference: Arch Wiki: GDM [6], GNOME Remote Desktop README [7].

Created Custom Headless Service

  • Action: Created /etc/systemd/system/gdm-headless.service and /usr/local/bin/gdm-headless.sh to start a headless X11 session:
  sudo vi /etc/systemd/system/gdm-headless.service
  sudo vi /usr/local/bin/gdm-headless.sh
  sudo chmod +x /usr/local/bin/gdm-headless.sh
  • Why: Standard GDM3 expected a physical display; a custom service was needed for headless operation with the dummy driver.
  • Outcome: Service started but failed due to dconf permission errors and .gvfs mount issues.
  • Reference: Arch Wiki: Remote Desktop [8], Ubuntu GDM3 Source [9].

Fixed dconf Permissions

  • Action: Modified permissions for /var/lib/gdm3/.cache and /var/lib/gdm3/.cache/dconf:
  sudo mkdir -pv /var/lib/gdm3/.cache/dconf
  sudo chown -v gdm:gdm /var/lib/gdm3/.cache /var/lib/gdm3/.cache/dconf
  sudo chmod -v 755 /var/lib/gdm3/.cache /var/lib/gdm3/.cache/dconf
  • Why: Logs showed dconf access errors for the gdm user, preventing session initialization.
  • Outcome: Resolved dconf errors, but .gvfs mount issues persisted.
  • Reference: GNOME dconf Docs [10], Arch Wiki: GDM [6].

Fixed .gvfs Mount Issues

  • Action: Unmounted and removed /var/lib/gdm3/.gvfs:
  sudo fusermount -u /var/lib/gdm3/.gvfs
  sudo rm -rfv /var/lib/gdm3/.gvfs
  • Why: Logs showed /var/lib/gdm3/.gvfs as a stale mount point with d?????????, causing permission errors.
  • Outcome: Removed .gvfs, but RDP still failed with “Failed to connect” due to session and PipeWire issues.
  • Reference: GNOME GVFS Docs [11], Arch Wiki: GNOME [12].

Configured XDG_RUNTIME_DIR

  • Action: Manually created /run/user/125 and updated gdm-headless.sh and gdm-headless.service:
  sudo mkdir -pv /run/user/125
  sudo chown -v gdm:gdm /run/user/125
  sudo chmod -v 700 /run/user/125
  • Why: Logs showed XDG_RUNTIME_DIR is invalid or not set, breaking gnome-shell and PipeWire.
  • Outcome: Reduced some errors, but systemd --user and PipeWire still failed due to no logind session.
  • Reference: Arch Wiki: XDG [13], systemd Docs [14].

Added PipeWire Support

  • Action: Added PIPEWIRE_RUNTIME_DIR=/run/user/125 to gdm-headless.sh and attempted to start PipeWire:
  sudo -u gdm XDG_RUNTIME_DIR=/run/user/125 dbus-run-session systemctl --user start 
  pipewire pipewire-pulse
  • Why: gnome-remote-desktop requires PipeWire for screencasting; logs showed PipeWire connection failures.
  • Outcome: Failed with No medium found, indicating no D-Bus session bus due to missing logind session.
  • Reference: PipeWire Docs [15], GNOME Remote Desktop README [7].

Attempted logind Session with loginctl

  • Action: Modified gdm-headless.service to include ExecStartPre=/usr/bin/loginctl activate gdm.
  • Why: To create a logind session for gdm to enable systemd --user and PipeWire.
  • Outcome: No session created, as loginctl activate requires an existing session.
  • Reference: systemd loginctl Docs [16].

Added pam_systemd to PAM Configuration

  • Action: Edited /etc/pam.d/gdm-launch-environment to include pam_systemd.so.
  • Why: To ensure pam_systemd creates a logind session and /run/user/125 for gdm.
  • Outcome: Session creation started but failed due to subsequent authentication issues.
  • Reference: systemd PAM Docs [14].

Used su for Session Creation

  • Action: Modified gdm-headless.service to use ExecStart=/usr/bin/su - gdm -c ....
  • Why: To trigger a logind session via pam_systemd for gdm.
  • Outcome: Failed due to su authentication error (auth could not identify password for [gdm]), as gdm has no password.
  • Reference: systemd Docs [14], Arch Wiki: GDM [6].

Decision

At this point I decided I was chasing rabbits and needed help.

Final Configuration Files

Below are the final configuration files with comments explaining the path, modifications, problems targeted, rationale, and references.

# /etc/gdm3/custom.conf
[daemon]
# Problem: Wayland lacked headless support
# Added WaylandEnable=false to use X11
# Why: X11 recommended for gnome-remote-desktop in headless setups
# Reference: `https://wiki.archlinux.org/title/GDM`
WaylandEnable=false

# /etc/pam.d/gdm-launch-environment
#%PAM-1.0
auth    requisite       pam_nologin.so
auth    required        pam_permit.so
@include common-account
session optional        pam_keyinit.so force revoke
# Problem: gdm-headless.service failed to create XDG_RUNTIME_DIR and logind session
# Added pam_systemd.so to create a logind session for gdm user
# Why: pam_systemd ensures session creation, including /run/user/125
# Reference: `https://www.freedesktop.org/software/systemd/man/pam_systemd.html`
session optional        pam_systemd.so
session required        pam_limits.so
session required        pam_env.so readenv=1
session required        pam_env.so readenv=1 user_readenv=1 envfile=/etc/default/locale
@include common-session
@include common-password

# /etc/systemd/system/gdm-headless.service
[Unit]
Description=Headless GDM3 for Remote Desktop
After=network.target

[Service]
# Problem: No logind session for gdm, causing XDG_RUNTIME_DIR and PipeWire failures
# Changed from direct ExecStart to su to trigger PAM session
# Why: su with gdm-launch-environment PAM stack should create session
# Reference: `https://www.freedesktop.org/software/systemd/man/pam_systemd.html`
ExecStart=/usr/bin/su - gdm -c "/usr/local/bin/gdm-headless.sh"
Restart=always
# Problem: XDG_RUNTIME_DIR and session type not set
# Added environment variables for session setup
# Why: Explicitly set to ensure gnome-shell and PipeWire work
# Reference: `https://wiki.archlinux.org/title/XDG#XDG_RUNTIME_DIR`
Environment=DISPLAY=:10
Environment=XDG_SESSION_TYPE=x11
Environment=XAUTHORITY=/var/lib/gdm3/.Xauthority
User=gdm
Group=gdm
RestartSec=5
PAMName=gdm-launch-environment

[Install]
WantedBy=graphical.target

# /usr/local/bin/gdm-headless.sh
#!/bin/bash
# Problem: Headless session needed explicit configuration
# Set display and session type for X11
# Why: Ensures X11 session on :10 display
# Reference: `https://wiki.archlinux.org/title/Remote_desktop`
export DISPLAY=:10
export XDG_SESSION_TYPE=x11
export XAUTHORITY=/var/lib/gdm3/.Xauthority
# Problem: PipeWire failed to connect
# Added PipeWire runtime directory
# Why: PipeWire needs XDG_RUNTIME_DIR for sockets
# Reference: `https://pipewire.org/`
export PIPEWIRE_RUNTIME_DIR=/run/user/125
unset SSH_AUTH_SOCK
/usr/bin/xauth -f /var/lib/gdm3/.Xauthority add :10 . $(xxd -l 16 -p /dev/urandom)
/usr/bin/X -nolisten tcp -noreset +extension GLX +extension RANDR +extension RENDER -config /etc/X11/xorg.conf.d/10-gpu.conf -logfile /var/lib/gdm3/Xorg.10.log :10 &
sleep 5
# Problem: No D-Bus session bus, causing PipeWire and systemd --user failures
# Added dbus-run-session for D-Bus bus
# Why: Ensures gnome-session runs in a D-Bus session
# Reference: `https://www.freedesktop.org/software/systemd/man/pam_systemd.html`
dbus-run-session /usr/libexec/gnome-session-binary --session=gnome

# /etc/X11/xorg.conf.d/10-gpu.conf
# Problem: Complex NVIDIA driver setup hindered headless operation
# Changed from nvidia driver to dummy driver for headless GUI
# Why: Dummy driver simplifies headless X11 session, avoiding hardware dependencies like monitor detection
# Reference: `https://wiki.archlinux.org/title/Xorg#Dummy_driver`
Section "ServerLayout"
    Identifier "Layout0"
    Screen 0 "Screen0"
EndSection

Section "Device"
    Identifier "Dummy"
    Driver "dummy"
    VideoRam 256000
    Option "NoAccel" "true"
    Option "IgnoreEDID" "true"
EndSection

Section "Monitor"
    Identifier "Monitor0"
    HorizSync 28.0-90.0
    VertRefresh 48.0-76.0
    ModeLine "2560x1440" 241.50 2560 2720 2968 3408 1440 1441 1444 1481
    ModeLine "2048x1152" 198.00 2048 2184 2400 2752 1152 1153 1156 1189
    ModeLine "1920x1080" 172.80 1920 2040 2248 2576 1080 1081 1084 1118
    ModeLine "1280x1024" 108.00 1280 1328 1440 1688 1024 1025 1028 1066 +hsync +vsync
    ModeLine "1024x768" 65.00 1024 1048 1184 1344 768 771 777 806 -hsync -vsync
    ModeLine "800x600" 40.00 800 840 968 1056 600 601 605 628 +hsync +vsync
    ModeLine "640x480" 25.18 640 656 752 800 480 490 492 525 -hsync -vsync
EndSection

Section "Screen"
    Identifier "Screen0"
    Device "Dummy"
    Monitor "Monitor0"
    DefaultDepth 24
    SubSection "Display"
        Depth 24
        Modes "2560x1440" "2048x1152" "1920x1080" "1280x1024" "1024x768" "800x600" "640x480"
    EndSubSection
EndSection

Outcome of Final Experiment

The final configuration with su in gdm-headless.service failed due to authentication errors:

Apr 30 10:49:04 homeserver su[37904]: pam_unix(su-l:auth): auth could not identify password for [gdm]
Apr 30 10:49:04 homeserver su[37904]: FAILED SU (to gdm) gdm on none
Apr 30 10:49:04 homeserver su[37904]: Password: su: Authentication failure

This prevented gdm-headless.service from starting, so gnome-shell, PipeWire, and gnome-remote-desktop didn’t initialize, resulting in the persistent “Failed to connect” error. Temporary logind sessions (e.g., session-c21.scope) were created but terminated immediately due to the service failure.

References

[1] “xrdp GitHub Discussions.” GitHub. https://github.com/neutrinolabs/xrdp/discussions/3491.

[2] Torres López, Joan. “Headless Remote Sessions in GNOME, Part 1.” SUSE. https://www.suse.com/c/headless-remote-sessions-in-gnome-part-1/.

[3] “Ubuntu Server Guide.” Ubuntu. https://ubuntu.com/server/docs.

[4] “GNOME 46 Release Notes.” GNOME. https://release.gnome.org/46/.

[5] “Ubuntu Desktop Guide.” Ubuntu. https://help.ubuntu.com/stable/ubuntu-help/.

[6] “GDM.” Arch Wiki. https://wiki.archlinux.org/title/GDM.

[7] “GNOME Remote Desktop README.” GNOME. https://gitlab.gnome.org/GNOME/gnome-remote-desktop/-/blob/main/README.md.

[8] “Remote Desktop.” Arch Wiki. https://wiki.archlinux.org/title/Remote_desktop.

[9] “Ubuntu GDM3 Source.” Launchpad. https://git.launchpad.net/ubuntu/+source/gdm3.

[10] “GNOME dconf Docs.” GNOME. https://gitlab.gnome.org/GNOME/dconf.

[11] “GNOME GVFS Docs.” GNOME. https://gitlab.gnome.org/GNOME/gvfs.

[12] “GNOME.” Arch Wiki. https://wiki.archlinux.org/title/GNOME#GVFS.

[13] “XDG.” Arch Wiki. https://wiki.archlinux.org/title/XDG#XDG_RUNTIME_DIR.

[14] “pam_systemd.” systemd. https://www.freedesktop.org/software/systemd/man/pam_systemd.html.

[15] “PipeWire Docs.” PipeWire. https://pipewire.org/.

[16] “loginctl.” systemd. https://www.freedesktop.org/software/systemd/man/loginctl.html.