Balooctl: Add checkDb command
Review Request #125424 - Created Sept. 27, 2015 and submitted
Balooctl: Add checkDb commandThis command can be used by developers to check if their PostingDb and DocumentUrlDB is in a valid state. We have a few bugs caused because all the databases are not correctly updated. This will help us diagnose this better.
I am trying to diagnose the bug revealed by review 125362. Still not entirely sure.
First note: it would be useful if it would show not only IDs of documents, but also full paths/names.
Second note: please also add 'reversed' check for postingDB. Right now it checks only in each pair "Document ID -> Term" has corresponding pair "Term -> Document ID" in PostingDB. But it should also check that each "Term -> Document ID" link in PostingDB has corresponding term in DocumentDB.
I noted this because checkDb shows some corrupted data in DB, but it doesn't show files I expected (that has weird type 'directory' being just PDFs). That's because I can search it both by type "Document" and type "Folder":# baloosearch --type directory 'result.pdf' /store/documents/[...]/results.pdf # baloosearch --type document 'result.pdf' /store/documents/[...]/results.pdf
Reversed check would reveal this kind of corruption.
- I didn't land up adding document urls since once can always fetch them via 'balooshow', but may I still should. It makes the output harder to read.
- Whoa! This revealed a lot of easily reproducible corruptions
Revision 2 (+157)
Oh, I didn't know that you can feed an ID to balooshow. Yeah, in that case I agree with you, URL is not necessary.
Full check now takes some time... It would be useful if it would also report progress, since we know the total amount of terms. For example I would suggest reporting each 5% or 10% of terms processed with a message like "Progress: 30% (elapsed time: 10 min, ETA: 25 min)". But I guess it's needed only in the last check you added, it's the slowest one.
Good news: after applying patch from https://git.reviewboard.kde.org/r/125362/, it does not report any invalid entry :)