Tracker reports "database disk image is malformed"

sorrel · September 23, 2023, 12:26am

Hello! My gnome-tracker install appears to have broken itself slightly: tracker3 status reports “Database corrupt: database disk image is malformed” for hundreds of my music files. I can’t honestly tell if this is having any effect on anything I use Tracker for; I originally noticed this because Music was listing only a subset of my albums, so I started investigating, but it seems like maybe Tracker was just taking its time to reindex everything, for some reason? Music now appears to display something more like my full collection, at least.

I figure I can fix any lingering effects by tracker3 reset, but before I do that, is there anything useful I can do to help figure out what caused the corruption in the first place?

garnacho · September 23, 2023, 7:58am

What version are you using? The errors should fix themselves away automatically if you update to >= 3.5.0.

sorrel · September 23, 2023, 8:52am

$ tracker3 --version
Tracker 3.5.3

rvl · October 3, 2023, 6:39am

I noticed this error after removing folders from the Search Locations list in GNOME Settings. This is with Tracker 3.5.3.

For example, I switched off the Home folder from indexing, then found that file:///home/rodney was still listed in the results of tracker3 search but tracker3 status and tracker3 info file:///home/rodney would say “database disk image is malformed”.

Logs enclosed...

$ tracker3 search rodney --limit=1                                     
Results:
  file:///home/rodney
  rodney


$ tracker3 index | grep '/home/rodney[^/]'
$ tracker3 info file:///home/rodney                                    
Querying information for entity: 'file:///home/rodney'
  'file:///home/rodney'
Results:
  'nrl:added' = '2023-05-04T08:45:56Z'
  'nrl:modified' = '22905'
  'nie:interpretedAs' = 'urn:fileid:9fb57146-158e-4fc2-abb8-bd08effdce73:257:257'
  'rdf:type' = 'http://www.w3.org/2000/01/rdf-schema#Resource'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nie#DataObject'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nfo#FileDataObject'
  'nfo:fileLastAccessed' = '2020-01-20T22:15:57Z'
  'nfo:fileCreated' = '2020-01-20T22:15:57Z'
  'nfo:fileSize' = '2516'
  'nfo:fileName' = 'rodney'
  'nfo:fileLastModified' = '2023-10-03T06:03:49Z'
  'tracker:extractorHash' = '51e376f9f68b3bf903f23741101490fe9060de1278691ab8a442883d9c950616'
  'nie:url' = 'file:///home/rodney'
  'nie:byteSize' = '2516'
  'nie:created' = '2020-01-20T22:15:57Z'
  'nie:dataSource' = 'urn:fileid:9fb57146-158e-4fc2-abb8-bd08effdce73:257:257'
  'http://purl.org/dc/elements/1.1/source' = 'urn:fileid:9fb57146-158e-4fc2-abb8-bd08effdce73:257:257'
  'http://purl.org/dc/elements/1.1/date' = '2020-01-20T22:15:57Z'
  'http://purl.org/dc/elements/1.1/date' = '2023-10-03T06:03:49Z'
  'tracker:available' = 'true'
  'nie:mimeType' = 'inode/directory'
  'nie:rootElementOf' = 'urn:fileid:9fb57146-158e-4fc2-abb8-bd08effdce73:257:257'
  'nrl:added' = '2023-09-21T00:35:40Z'
  'nrl:modified' = '22905'
  'nie:isStoredAs' = 'file:///home/rodney'
  'rdf:type' = 'http://www.w3.org/2000/01/rdf-schema#Resource'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nie#DataSource'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nie#InformationElement'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nfo#DataContainer'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nfo#Folder'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/tracker#IndexedFolder'

Extraction failed for this file. Some metadata will be missing.
Error message: Database corrupt: database disk image is malformed
SPARQL: # Inputs: uri
DELETE {
  GRAPH tracker:FileSystem {
    ?f a rdfs:Resource .
  } .
  GRAPH ?g {
    ?f a rdfs:Resource .
    ?ie a rdfs:Resource .
  }
} WHERE {
  GRAPH tracker:FileSystem {
    ?f a rdfs:Resource ;
      nie:url ~uri
    GRAPH ?g {
      ?f a rdfs:Resource .
      OPTIONAL {
        ?ie nie:isStoredAs ?f
      }
    }
  }
}

              
$ tracker3 index --add /home/rodney                                    
$ tracker3 index | grep '/home/rodney[^/]'                             
/home/rodney                                                 -
$ tracker3 info file:///home/rodney                                    
Querying information for entity: 'file:///home/rodney'
  'file:///home/rodney'
Results:
  'nrl:added' = '2023-05-04T08:45:56Z'
  'nrl:modified' = '22910'
  'nie:interpretedAs' = 'urn:fileid:9fb57146-158e-4fc2-abb8-bd08effdce73:257:257'
  'rdf:type' = 'http://www.w3.org/2000/01/rdf-schema#Resource'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nie#DataObject'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nfo#FileDataObject'
  'nfo:fileLastAccessed' = '2020-01-20T22:15:57Z'
  'nfo:fileCreated' = '2020-01-20T22:15:57Z'
  'nfo:fileSize' = '2516'
  'nfo:fileName' = 'rodney'
  'nfo:fileLastModified' = '2023-10-03T06:24:54Z'
  'tracker:extractorHash' = '51e376f9f68b3bf903f23741101490fe9060de1278691ab8a442883d9c950616'
  'nie:url' = 'file:///home/rodney'
  'nie:byteSize' = '2516'
  'nie:created' = '2020-01-20T22:15:57Z'
  'nie:dataSource' = 'urn:fileid:9fb57146-158e-4fc2-abb8-bd08effdce73:257:257'
  'http://purl.org/dc/elements/1.1/source' = 'urn:fileid:9fb57146-158e-4fc2-abb8-bd08effdce73:257:257'
  'http://purl.org/dc/elements/1.1/date' = '2020-01-20T22:15:57Z'
  'http://purl.org/dc/elements/1.1/date' = '2023-10-03T06:24:54Z'
  'tracker:available' = 'true'
  'nie:mimeType' = 'inode/directory'
  'nie:rootElementOf' = 'urn:fileid:9fb57146-158e-4fc2-abb8-bd08effdce73:257:257'
  'nrl:added' = '2023-09-21T00:35:40Z'
  'nrl:modified' = '22910'
  'nie:isStoredAs' = 'file:///home/rodney'
  'rdf:type' = 'http://www.w3.org/2000/01/rdf-schema#Resource'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nie#DataSource'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nie#InformationElement'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nfo#DataContainer'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/nfo#Folder'
  'rdf:type' = 'http://tracker.api.gnome.org/ontology/v3/tracker#IndexedFolder'

garnacho · October 3, 2023, 9:06am

Thanks for the info! I’ll try to reproduce the issue. Since you are still reproducing these database “corruptions” on >=3.5.x, it would be good to file a new issue to tracker, so we can request some more info there if needed.

FTR, this is just the fulltext-search index being corrupted, it is a correctable situation by rebuilding the FTS index, and that’s something that happens ootb with e.g. locale changes. I thought those were gone for good, but perhaps not…

garnacho · October 3, 2023, 12:35pm

Unfortunately I was unable to reproduce on 3.6/git, with or without TRACKER_DEBUG=fts-integrity. Since I am unclear if the issue was really fixed meanwhile, better let’s move this to a gitlab issue.

rvl · October 17, 2023, 2:25am

OK, fair enough, thanks @garnacho.

How would I rebuild the FTS index to check this?

I haven’t yet learnt about the tracker SQLite schemas, and how the sparql store and FTS index fit together.

I was going to open an issue, but I have this problem with GNOME GitLab where my account keeps getting deleted! I created an account again, and will open an issue if I can reproduce this error after rebuilding the FTS index.

system · December 1, 2023, 2:25am

This topic was automatically closed 45 days after the last reply. New replies are no longer allowed.