Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(sync): Reduce db operations for accounts without CONDSTORE #9235

Merged
merged 1 commit into from
Feb 6, 2024

Conversation

JohannesGGE
Copy link
Contributor

Fix: #6341


Queries (only related) with open mail app for 40min (approx. 170 mails in the mailbox)

before:

1367 	 primary UPDATE `oc_mail_messages` SET `flag_answered` = :flag_answered, `flag_deleted` = :flag_deleted, `flag_draft` = :flag_draft, `flag_flagged` = :flag_flagged, `flag_seen` = :flag_seen, `flag_forwarded` = :flag_forwarded, `flag_junk` = :flag_junk, `flag_notjunk` = :flag_notjunk, `flag_mdnsent` = :flag_mdnsent, `flag_important` = :flag_important, `updated_at` = :dcValue1 WHERE (`uid` = :uid) AND (`mailbox_id` = :mailbox_id)

after:
(So usually there are 2 queries per flag (138) (set true and set false). seen and notjunk are the only ones with doubled queries because for the other flags, either all mails or no mail have the flag in this specific mailbox.)

    138 	 primary UPDATE `oc_mail_messages` SET `flag_seen` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)
    138 	 primary UPDATE `oc_mail_messages` SET `flag_notjunk` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)
     69 	 primary UPDATE `oc_mail_messages` SET `flag_mdnsent` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)
     69 	 primary UPDATE `oc_mail_messages` SET `flag_junk` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)
     69 	 primary UPDATE `oc_mail_messages` SET `flag_important` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)
     69 	 primary UPDATE `oc_mail_messages` SET `flag_forwarded` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)
     69 	 primary UPDATE `oc_mail_messages` SET `flag_flagged` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)
     69 	 primary UPDATE `oc_mail_messages` SET `flag_draft` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)
     69 	 primary UPDATE `oc_mail_messages` SET `flag_deleted` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)
     69 	 primary UPDATE `oc_mail_messages` SET `flag_answered` = :dcValue1, `updated_at` = :dcValue2 WHERE (`uid` IN (:uids)) AND (`mailbox_id` = :dcValue3)

Copy link
Member

@ChristophWurst ChristophWurst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

->set('updated_at', $queryFalse->createNamedParameter($this->timeFactory->getTime(), IQueryBuilder::PARAM_INT))
->where($queryFalse->expr()->andX(
$queryFalse->expr()->in('uid', $queryFalse->createParameter('uids')),
$queryFalse->expr()->eq('mailbox_id', $queryFalse->createNamedParameter($mailboxId, IQueryBuilder::PARAM_INT))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to experiment with: AND flag=1

to lower the number of accessed rows and hopefully the transaction size (for rollbacks and for replication)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JohannesGGE JohannesGGE force-pushed the perf/6341/updateBulk branch 2 times, most recently from 2a391b4 to 2dddfae Compare February 5, 2024 19:17
Copy link
Member

@ChristophWurst ChristophWurst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use modern array syntax

🚀

lib/Db/MessageMapper.php Outdated Show resolved Hide resolved
lib/Db/MessageMapper.php Outdated Show resolved Hide resolved
@ChristophWurst ChristophWurst merged commit a4a0af9 into main Feb 6, 2024
37 checks passed
@ChristophWurst ChristophWurst deleted the perf/6341/updateBulk branch February 6, 2024 12:08
Copy link

github-actions bot commented Apr 4, 2024

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Restructure \OCA\Mail\Db\MessageMapper::updateBulk to run in O(1)
2 participants