I want to hear some crazy and wild ideas for the future of search in GNOME

Hello
All the work on Tracker 3.0 has made me curious about what the future holds for GNOME in terms of search and recommendations.

I was inspired to make a post on /r/gnome asking for ideas related to search and recommendations and I want to ask the same thing here.

I’ve seen posts in different places about custom tags, automatic feature extraction and a linguistic (voice) interface, and while I have no plans to push forwards on any of these right now, it would be nice to at least collect all the ideas in one place :slight_smile:

This might not be wild or crazy, but it’d be nice if there were an easy way to add a search provider. Like, really easy. There’s a great GNOME shell extension called Argos that allows you to write new panel widgets with just a shell script that echoes something. It’d be great if there were something so simple for search.

That would be interesting!
Writing a search provider in Python is already possible, you need to implement the relevant DBus interfaces. It wouldn’t be far from there to having the python script execute a shell program to generate results.

I once wrote a commandline tool that queries the desktop search providers which is a useful testing tool for search provider authors.

I have an idea @sthursfield! Tracker records a lot of things and it seems like you could create a model of user behavior. How can we use things like tracker to build personas? If you could throw the kind of things that people do using gnome-search provider and others and throw it at tensorflow or something where would that lead? What would the possibilities - what would be the privacy concerns? How can we make a better desktop experience using these kind of things if the desktop is able to learn what you do.

You could extrapolate from when active files are created, right?

sorry just throwing it out there - it might be completely nonsensical but you did say crazy and wild ideas. :slight_smile:

I think from a higher level, how can we take advantage of all these other technologies that other communities are developing and integrate them in a way that makes sense. The purpose does a number of things:

  • New and interesting directions to explore, it keeps working on this stuff fresh
  • Exposing our community to new communities and vice versa
  • Using these technologies to create new experiences in the stack

eg how can tracker / gnome-search influence UX designs - what kind of data can we derive from search? Google has been doing this for a long awhile…but you know we did this even before as well.

We had some interesting concepts around ‘presence’ in the past. Anyways… more thoughts. :slight_smile:

I did ask for crazy ideas, so thanks for this :slight_smile:
Despite the name, Tracker doesn’t actually track user behaviour. It’s an indexer for contents and information about the user’s files. Once upon a time we had Zeitgiest and the GNOME Activity Journal to record and display user activity in quite a detailed way but those projects have been quiet for a long time.
I’m also curious what uses we might have for profiles built from user activity. The main usecase I’m aware of is to sell targetted advertising and make billions of dollars, something often known as “spyware” – GNOME doesn’t have a manifesto but if we did, I think we’d take an anti-spyware position!

1 Like

yeah, I was just exploring the space… it’s not that I want spyware, but looking to see how we can leverage tools we have to improve the platform.
You are right that I am thinking of GNOME Activity Journal or Zeitgeist which I feel kind of sad that we didn’t find a way to take advantage of them.

My wierd idea was to see if we could make a learning desktop, based on what tracker finds are being written to the filesystem. But it seems that I’ve only caused only head scratching :smiley: I’m sure you’ll never look at me the same again! Too bad, I would have enjoyed having a beer about it in person at GUADEC to sort of expand on my idea. :smiley:

Me too … :mask:. Although, one small upside to the Mexico edition being postponed is that there’s more chance I could attend in 2021, so hold that thought.

I do think the ideas of Zeitgiest & Activity Journal have potential, every desktop already has the concept of ‘Recent files’ (often implemented in a pretty annoying way), every browser has the “Awesome bar” concept, etc. The design aspect is very tricky because you’re making things dynamic rather than static. Testing a UI is a lot of work (ask Clarissa for example!). Now imagine the UI is different for every user, and different every time they look at it, suddenly the testing matrix has become practically infinite.

I don’t say that to dismiss the ideas, more to say that we are probably correct to have taken a conservative attitude up til now. There are definitely some cool research projects to be done in this space, I can imagine that research on voice assistants (see MyCroft and Almond) will drive some of that research, as a voice assistant basically has to personalize itself to user preferences.

Yes, exactly… I feel like looking at some of the conceptual things people are doing with machine learning and how we could incorporate that with the desktop. Today, everyone is ignoring desktops and focusing on mobile devices for all their machine learning/AI things. I looked at tracker as a ‘data generator’ which might be wierd for a lot of people, but it does tell a story of sorts if you see what kind of files are being generated. I know it does sound kind of crazy and yes there are definitely privacy implications but an opt-in group of people doing it for a specific purpose on a special build of GNOME might be interesting I guess.
Anyways, I like exploring new ideas based on my exposure to other ecosystems and vice versa.
I would like to see Zeitgeist resurrected.