Help creating a simple reporting app

Applications
1

I’m currently in the process of creating a GNOME user survey. The goal is to generate data that will help us to make design decisions. The survey covers various things including what kind of hardware the user has, which apps they have installed, how they have their desktop configured, and so on.

Some of the questions are essentially a workaround for a lack of automated data collection, and in some cases the survey literally asks the respondent to run a gsettings command and then paste the result into the questionnaire.

So I’m wondering: does anyone fancy making an app which would produces that data itself? The idea is that, as part of the survey, the user would install the app, and copy/paste its output into the questionnaire. That would allow us to cut out a chunk of the survey and would make the overall process more reliable.

The challenge here is that the app would need to be available cross-distro. Some of the items of data we want to collect might be slightly tricky to get from a technical perspective, too, but we can cross that bridge when we come to it.

Anyone interested?!

1 Like
2

Is there a list already of the data you want the application to be able to collect?

1 Like
3

This is the current list of the survey questions which could potentially be automated. It would be great if we could just get the simple ones covered.

  • Distro
  • Distro version
  • Dual/multi boot?
  • Computer form factor
  • Computer manufacturer
  • Computer model
  • Resolutions and sizes of connected displays
  • Enabled repositories
  • Default browser
  • Number of users on the system
  • List of installed apps
  • List of pending firmware updates
  • Is a weather location configured in the shell?
  • Are any world clocks configured?
  • Value of org.gnome.shell.overrides workspaces-only-on-primary (and other supported keys for this setting)
  • Value of org.gnome.shell.overrides dynamic-workspaces
  • List of enabled extensions(gnome-extensions list --enabled)
  • List of enabled online accounts
  • Value of org/gnome-shell/favorite-apps
  • Number/list of currently running apps
  • Number of currently open windows
4

Those settings have not been used since GNOME 3.30, you’ll want the keys from org.gnome.mutter instead.

5

This sounds interesting to me. But there’s some “Danger Will Robinson!” triggers going off. I don’t know how much data you’re looking for, but to get as many people as possible to run it, there’s some things you may want to consider.

Some of the data will be easy to gather as the user, such as the various gsettings values, default browser, distro version. Other data likely will need more access, via sudo or root access, which may make some users a bit twitchy.

If you make this a graphical desktop application, you make it harder for people to get and run, and the maintenance of it becomes much larger, especially if you want it to be run across many distros. If (for example) you leverage GNOME 40 to build it, and expect to only get responses from those users, you limit yourself to a tiny proportion of the Linux desktop userbase.

I’d suggest making this a simple command line tool, perhaps a Go or Rust binary, which has minimal additional external requirements. That way it can be easily installed (via whatever packaging system, but also via curl or whatever) and executed.

There’s some prior art here, in Ubuntu we run ubuntu-report post-install. It doesn’t gather all the data you’re after, but it does grab a fair amount of it. It’s GPL’ed and available on GitHub. Here’s what it looks like if you run it in a terminal, which I just ran on my personal desktop:

alan@robot:~$ ubuntu-report -f
This is the result of hardware and optional installer/upgrader that we collected:
{
  "Version": "21.04",
  "OEM": {
    "Vendor": "Intel Corporation",
    "Product": "NUC8i7HVK",
    "Family": "Intel NUC"
  },
  "BIOS": {
    "Vendor": "Intel Corp.",
    "Version": "HNKBLi70.86A.0053.2018.1217.1739"
  },
  "CPU": {
    "OpMode": "32-bit, 64-bit",
    "CPUs": "8",
    "Threads": "2",
    "Cores": "4",
    "Sockets": "1",
    "Vendor": "GenuineIntel",
    "Family": "6",
    "Model": "158",
    "Stepping": "9",
    "Name": "Intel(R) Core(TM) i7-8809G CPU @ 3.10GHz",
    "Virtualization": "VT-x"
  },
  "Arch": "amd64",
  "GPU": [
    {
      "Vendor": "1002",
      "Model": "694c"
    }
  ],
  "RAM": 32.8,
  "Disks": [
    4000.8,
    1000.2
  ],
  "Partitions": [
    490.2,
    1,
    983.5,
    3844.6,
    976.3
  ],
  "Screens": [
    {
      "Size": "531mmx299mm",
      "Resolution": "1920x1080",
      "Frequency": "60.00"
    },
    {
      "Size": "531mmx299mm",
      "Resolution": "1920x1080",
      "Frequency": "60.00"
    }
  ],
  "Autologin": false,
  "LivePatch": false,
  "Session": {
    "DE": "KDE",
    "Name": "KDE",
    "Type": "x11"
  },
  "Language": "en_GB",
  "Timezone": "Europe/London",
  "Install": {
    "Media": "Ubuntu 20.04.1 LTS \"Focal Fossa\" - Release amd64 (20200731)",
    "Type": "GTK",
    "OEM": true,
    "PartitionMethod": "manual",
    "DownloadUpdates": false,
    "Language": "en",
    "Minimal": false,
    "RestrictedAddons": false,
    "Stages": {
      "0": "language",
      "12": "console_setup",
      "14": "console_setup",
      "16": "wireless",
      "23": "prepare",
      "45": "partman",
      "53": "partman",
      "54": "partman",
      "55": "partman",
      "67": "partman",
      "74": "partman",
      "131": "partman",
      "153": "partman",
      "154": "partman",
      "158": "partman",
      "168": "partman",
      "191": "partman",
      "202": "partman",
      "215": "partman",
      "216": "partman",
      "225": "partman",
      "229": "start_install",
      "230": "timezone",
      "237": "usersetup",
      "254": "user_done",
      "466": "done"
    }
  },
  "Upgrade": {
    "From": "20.10",
    "InstallMedia": "Ubuntu 20.04.1 LTS \"Focal Fossa\" - Release amd64 (20200731)",
    "Type": "Text",
    "ThirdPartySources": false,
    "Stages": {
      "0": "start",
      "2": "PREPARE",
      "10": "MODIFY_SOURCES",
      "127": "FETCH",
      "314": "INSTALL",
      "316": "INSTALL",
      "357": "INSTALL",
      "1161": "POSTUPGRADE",
      "1165": "CLEANUP",
      "5934": "REBOOT"
    }
  }
}
Do you agree to report this? [y (send metrics)/n (send opt out message)/Q (quit)] y

Easy to use and run, transparently showing what is gathered, and offers the option to send or not. You can see some of the anonymised and aggregated gathered statistics at https://ubuntu.com/desktop/statistics

We took the (contentious) decision to run this (optionally) on install for all new users. Your use case seems different, but you’re after similar information. Worth a look. It’s helped us to understand what systems people install on. Your scope appears a lot wider, which would need a lot of care not to reveal personal information, but I’m sure you’re aware of that. :slight_smile:

2 Likes
6

This perfectly illustrates the point I made above. There’s almost certainly more people running GNOME <3.30 than GNOME 40 right now, by some margin. Any tool gathering this kind of data, might want to set a minimum version that you care about, and as such, how many users you’re willing to not care about the data from.

7

I’m not entirely sure I understand the point about doing this CLI only… How does that make it easier to distribute? Does it change the need to elevate privileges?

One advantage I can think of with a graphical app is that you can have a copy button, which will eliminate errors. I’d also be happy if we could get to the point where someone doesn’t need to use the terminal to fill in the survey.

8

If you want the absolutely widest audience for your survey, you need the friction as low as possible. I suggested command line because you can get almost any Linux user to grab a single binary and run it and move on, if it’s for the greater good. Getting them to install a graphical application, no matter what distro they’re on, is more difficult. Also, building one that can gather all that data in a distro agnostic way is hard too. I’m not saying impossible, just hard.

Not depending on specific GNOME, Python, Glib and a plethora of other libraries makes it easier. If, for example, you made a GNOME GTK application, based on GNOME 40, you could make a flatpak of it, chuck it in flathub. Elephant in the room - not everyone has flatpak enabled. Some will flat (lulz) out refuse to add it, especially when there’s nothing “in it” for them. Sure, people enable it, and will add a flatpak for spotify, slack or chromium - things they need. But this isn’t their need, it’s yours.

So let’s say you make it a flatpak, and a snap, and an appimage, that’s then tons of overhead. Plus the confinement of snap will make it very difficult to gather all the data you want.

Which leads me to “just make it a simple command line binary”. That way, anyone, on any distro, for any release, can probably just grab the binary, run it, review the output for privacy reasons and then paste the output into your survey/website/whatever. They then delete the binary and get on with their day, knowing they did a Good Thing for the GNOME project, and the application isn’t going to continue “spying” on them.

The clipboard issue you mention can be solved. Command line applications can inject their output into the paste buffer.

To be clear. I hope I am not coming across as negative. I think this is a great idea. I am fully in support of tools like ubuntu-report when designed and used well. I also believe it’s a good thing to gather data, where appropriate. We (the Linux community) are a fun bunch though, and this is an area fraught with dangerous pot-holes. I wish the best for this project :slight_smile:

9

Based on this list, forking ubuntu-report to include some additional data like the output of some gsettings seems fairly straightforward. The items that stand out to me as possibly invasive is List of enabled online accounts. We would need to be explicit that we are only collecting only the type of online account, not the PII.

Also, Number/list of currently running apps and Number of currently open windows will be trickier, depending on whether the session is Wayland or Xorg.

List of installed apps

Would this include Flatpaks, snaps, and classic packages? Or just a query of desktop-entry files? Are you looking for a dump of packages, like rpm -qa?

1 Like
10

I was imagining that, if we had a Flatpak and a Snap of the app, that would cover most of the common cases, and would sidestep version compatibility issues.

Maybe I need to refresh my memory of which distros support what…

11

The app definitely doesn’t need to cover everything in the list. The list of online accounts is fairly low in terms of priority, in that it’s fairly easy for someone to open the settings and fill in the information manually.

The list of installed apps is a tricky matter and is much more important. We need to get this, whether we automate it in the app, or get the user to run a command themselves. And yes, it needs to cover every potential format.

12

Alternatively the org.gnome.Shell.Introspect API has GetRunningApplications() and GetWindows() methods. That API is limited to the xdg-desktop portal by default, but there’s a GSetting to enable it for all callers that we could ask users to toggle before gathering information.

13

Speaking of making a CLI binary, why not make a GUI binary itself ?
If we are going to distribute it via package manager for CLI, the same will happen for GUI too. Like we can use language like Rust or C and ship it as a single binary that can be executed and then deleted.
Similar to how the binaries of Firefox, Godot etc. work.

Or am I missing something ?

14

The list I provided above was too long. In the first instance if we could get the data for a bunch of the relatively simple settings, that would be good:

  • Is a weather location configured in the shell?
  • Are any world clocks configured?
  • workspaces-only-on-primary setting
  • dynamic-workspaces setting
  • List of enabled extensions (gnome-extensions list --enabled)
  • favorite-apps in the dash
  • Default browser

After that, I would probably prioritise those pieces of information which are likely to produce errors if the user manually enters them themself:

  • Distro version
  • Computer model
  • Resolutions and sizes of displays
  • Enabled repositories
1 Like
15

If this is specifically for people using GNOME, would it be a good idea to use the Settings app? I imagine they’d want a daemon which would talk to the setting (which could be granular in a similar way to how KDE does it).

I think this would solve three problems that I’ve seen in this thread :

  • We want it to be accessible for all GNOME users
  • We want an underlying program which could run on non-GNOME systems (although if this is aimed at people using GNOME, this might not make sense)
  • We need privilege to get some of the information (the daemon would have its own user with the relevant permissions)

Relevant Art from KDE :

image
image1154×859 120 KB

1 Like
16

Yes we could build this into the system at some point in the future. That would obviously have a fairly long lead time before we could expect it to be in the hands of the majority of GNOME users, though, which is why I was hoping for a standalone tool in the first instance.

17

I don’t know how else GNOME would be able to distribute to the majority of people using GNOME to be honest.

Like, sure, GNOME can try to distribute it but I doubt more than 20% of people will even realize that this is a thing. The communication between GNOME and the people using it is quite low as far as I know. There would need to be a relatively big marketing for it and I don’t think that’s plausible. Maybe I’m a pessimist.

18

The idea is that we’d link to the app as part of the survey.

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.