Performance problem related to bad input

Hello, tracker people. I’m here to discuss a problem that I have encountered which made my system unresponsive, requiring a hard reset to get out of, and which occurred every time I subsequently logged into my machine until I finally diagnosed it (in two minute chunks, with a reboot in between each opportunity for debugging).

Essentially, at some point shortly before the issue occurred, I dropped my keyboard onto my chest, or my cat did anyway (I am disabled and use my machine from bed with a rolling table that’s above me). When the keyboard fell, it hit the windows-logo button (I think that’s referred to as “Super”? but I’m not sure so sorry for the microsoft reference) and also the ~/` key, resulting in the Ubuntu launcher overlay popping up and then input to the search function that looked like this:

```````````````````````````````````````````````````````````````````

In other words, just a long string of back-tics probably hundreds of characters long by the time I was done petting the cat and picked the keyboard back up. I thought nothing of it and closed the overlay.

This results in some tracker process consuming the entirety of a 6-core cpu. The cpu usage doesn’t begin until the first time something causes the tracker background processes to begin, and then slowly ramps up, so at the beginning it’s difficult to notice, but after a minute or two the entire desktop is unusable, and shortly after that it’s impossible to even type characters in the Ctrl-Alt-F2 terminal.

Issuing a “tracker reset --hard” command and answering yes at the “are you sure?” prompt resolves the issue.

I would raise an issue via git, but I’m not completely certain whether this would be considered a bug by most people, and discourse looks like a nicer place to hold that discussion if needed than in git bug report.

So, I just want to bullet point a few thoughts, and then let people who know tracker better than me (I only discovered it exists via the issue above) figure out what to do about it, whether to treat it as a bug or an enhancement request, what relative priority to assign, etc.

  • Is it possible to effectively limit the total cpu usage by the various tracker processes, in case a similar bug occurs even after this one gets fixed?

  • Could this be a security issue? For example, does tracker index any sources that are world-writable by default, or that include locations that may have been made world-writable by the user? Or read from the network? Etc.

  • Even if not, could this be considered a security issue from the perspective of remote users / server processes on a machine that also has a console user, who is essentially performing a local denial of service attack against the rest of the machine?

  • When fixing this bug, I assume mostly by ensuring that inputs are checked for sanity, what other sanity checks should be performed? It’ll be easy to add any checks needed at the same time, and I’m sure that if this input causes issues, there are going to be other classes of input that also cause issues.

  • When checking input specifically in the application search menu, should the input be checked as each character is typed, or when the user is done typing?

  • Is any other fix possible, in a “belt and suspenders” or “don’t put all your eggs in one basket” spirit, beyond input checking?

I would appreciate feedback on the correctness of my approach with this message, as I am new to discourse.gnome.org and may have erred in some way. Anything correctable will be cheerfully corrected.

Likewise, it seems the transition from the tracker mailing list is in progress, so I would appreciate any response by a tracker developer to let me know that this message has reached its target audience.

  1. No.

  2. You can configure which directories to index. But no gui. it was removed. So use dconf.

  3. There is always a way to attack. Nothing is 100% secure. But why someone attack only you ?

4,5,6. It is not as simple as checking the inputs. It’s more complex than that.

Tracker is somewhat buggy. If possible avoid using it. Some other gnome desktop like Budgie uses index searcher applet which uses zeitgeist backend.

Thanks @sunnysigara for the reply, I appreciate the information about indexing configuration. I’m not so much concerned about my own usage of tracker, as I now know about the issue and how to fix it. I’m more focused on seeing that the tracker developers get whatever information they need to appropriately prioritize and eventually fix this issue for future users.

Hi! I can’t seem to reproduce this on Debian, so I would suggest that opening a bug in the Ubuntu Launchpad is probably the best way to raise it.

2 Likes

Thanks @nmcgovern, for the attempt at reproduction and suggestion! I will look into doing that. :smiley:

1 Like

Hi @strask,

Tracker developer here, we got the message!

I think the two events you mentioned (search entry being filled with ````, and cpu usage) are unrelated. Even though some shell search providers use Tracker underneath, they most normally access the Tracker database directly, all CPU a query might consume would be seen in top as coming from those processes (eg. nautilus).

Instead, the Tracker processes are only spawned/active when there is something left to index. If you say a tracker reset --hard helped, this is probably yet another duplicate of https://gitlab.gnome.org/GNOME/tracker/issues/32. FTR, Tracker 2.3 detects this situation and performs the reset without further user interaction.

Regarding your questions:

  1. This already happens, but not the way you expect. Tracker processes do lower their CPU and I/O scheduling priorities, so the CPU share will be given up under system pressure. There’s however a number of dconf settings in /org/freedesktop/tracker/miner/files to influence various timeouts.

  2. Tracker indexes your homedir non recursively, and other XDG directories recursively. So, no by default, but being software it of course can be made to happen. Tracker already sandboxes its extractors if that could ever be a problem.

  3. Honestly seems convoluted when you can just do a fork bomb :slight_smile: . Still requires write access to the indexed locations as another user.

  4. to 6. As stated above, I don’t think bad input was the problem here :slight_smile: . Small proof of that is that tracker reset --hard fixed the issue.

1 Like

This is not correct, It has always been in gnome-control-center search panel. In earlier versions in the gear menu, in 3.34 the option is right in the window bar.

You are free to not use it. Recommending others to avoid it means you’re actively set to impede it getting better. Doing so at Gnome Discourse is a rather poor trolling attempt.

  • Is it possible to effectively limit the total cpu usage by the various tracker processes, in case a similar bug occurs even after this one gets fixed?

Tracker already attempts to signal to the kernel that it’s low priority via nice(), and it even has a ‘throttle’ feature in miner-fs that causes it to spend more time in poll() than it normally would, but this isn’t always enough to convince Linux that it should prioritize the gnome-shell process over tracker processes.

I have high hopes that the work on systemd user sessions will help here as distros will be able to set hard limits on Tracker resource usage using cgroups, and that will hopefully be more effective.

Ideally Tracker would never overuse resources, of course, but the nature of a filesystem indexer means that there’s always some risk of choking on impractically large inputs.

Absolutely NOT. Just because I am advising doesn’t mean I am against it getting better. Govt transport service in my village is very poor. I advise anyone to take a cab. But that doesn’t mean I don’t want the service to improve at all. I experienced it myself and now I am advising against it. That’s what “advise” actually mean. It’s a personal opinion. I think you are talking my comments too personally irrespective of your bias as maintainer (I don’t like sharp comments about my app either but I accept it any way)

Ubuntu stayed away from tracker for long long time until recently in Ubuntu 18.04 LTS. Debian’s Martin Pitt vehemently expressed his disliking about battery draining and here I quote

This isn’t true. This will use a lot more CPU and disk cycles, thus decreasing
battery life. Or, if you don’t index while on battery, it’s going to be even
less reliable.

That was true back then and it was true even now. There are several issues filed, looked into closed and reopened. Some other major issues like 94, 95 89 present almost every releases and encountered by multiple people. So I do not speak out of context.

I tell you this, I would the happiest person in the word if there issues are finally get fixed in next release. We don’t ship tracker in our unity based remix, Umix, but that can change. I may even create tracker-plugin for our dash.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.