Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup database #518

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 49 additions & 1 deletion modules/database.js
Original file line number Diff line number Diff line change
Expand Up @@ -935,8 +935,56 @@ export let Database = {

return tree;
},

async cleanupEntries() {
//TODO: Do not delete entries that are in the feed now.
let query = this.query({
deleted: 'deleted'
});
let ids = await query.getIds();
for(const id of ids.values()) {
let tx = this.db().transaction(['entries', 'revisions'], 'readwrite');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you benchmarked this? I'm afraid that the initial cleanup will take a lot of time in this mode. I think this code should process entries to be deleted in batches of 1000 or so (possibly reusing Migrator.BATCH_SIZE)

let request = tx.objectStore('entries').delete(id);
DbUtil.requestPromise(request);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just for error reporting?


let request2 = tx.objectStore('revisions').delete(id);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't assume that entry IDs and revision IDs match (and in general that there's just one revision per entry). Yes, that's the case at the moment, but I'd prefer being able to eventually change that.

DbUtil.requestPromise(request2);
}
},

async cleanupHiddenFeeds() {

var transaction = this._db.transaction(['feeds'], 'readonly');
var objectStore = transaction.objectStore('feeds');

const request = objectStore.getAll();
request.onsuccess = async ()=> {
const feeds = request.result;

for (let feed of feeds) {
if (feed.hidden == 0) {
continue;
}
let query = new Query({
feeds: [feed.feedID],
});
let ids = await query.getIds();
for(const id of ids.values()) {
let tx = this.db().transaction(['entries', 'revisions'], 'readwrite');
let request = tx.objectStore('entries').delete(id);
DbUtil.requestPromise(request);

let request2 = tx.objectStore('revisions').delete(id);
DbUtil.requestPromise(request2);
}

let tx = this.db().transaction(['feeds'], 'readwrite');
let request = tx.objectStore('feeds').delete(feed.feedID);
DbUtil.requestPromise(request);
}
}
}
};
//TODO: database cleanup
//TODO: bookmark to starred sync

function Query(filters) {
Expand Down
2 changes: 2 additions & 0 deletions ui/feedlist.js
Original file line number Diff line number Diff line change
Expand Up @@ -1163,6 +1163,8 @@ export let Commands = {

emptyTrash: function cmd_emptyTrash() {
ViewList.db.query(ViewList.getQueryForView('trash-folder')).markDeleted('deleted');
ViewList.db.cleanupHiddenFeeds();
ViewList.db.cleanupEntries();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that markDeleted is async and I'm not really sure whether you'll have any guarantees on the order of these operations. Also you may want to hook into entry expiration too for the sake of users who don't empty their trash manually.

},

toggleSelectedEntryRead: function cmd_toggleSelectedEntryRead() {
Expand Down