- If you select Razorpay, you can pay using Credit Card, Debit Card, Net
- Banking, UPI, Wallets, etc. If you are using Net Banking, it may take upto
- 5 days for balance to reflect.
-
- We are attempting to charge your card with
- {{ formattedMicroChargeAmount }} to make sure the
- card works. This amount will be refunded back to your
- account.
-
diff --git a/dashboard/src2/pages/saas/VerifyEmail.vue b/dashboard/src2/pages/saas/VerifyEmail.vue
index 527d4634cf2..2393aa9957e 100644
--- a/dashboard/src2/pages/saas/VerifyEmail.vue
+++ b/dashboard/src2/pages/saas/VerifyEmail.vue
@@ -51,7 +51,7 @@
:loading="$resources.resendOTP?.loading"
@click="$resources.resendOTP.submit()"
>
- Email didn't arrive? Resend
+ Didn't get email? Resend
From 656e9faeea795929b613a5aa210df3f55d67792c Mon Sep 17 00:00:00 2001
From: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
Date: Thu, 21 Nov 2024 16:16:37 +0000
Subject: [PATCH 48/93] docs(saas): added documentation for new saas flow
---
press/saas/README.md | 62 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 62 insertions(+)
diff --git a/press/saas/README.md b/press/saas/README.md
index e69de29bb2d..2da0948ee35 100644
--- a/press/saas/README.md
+++ b/press/saas/README.md
@@ -0,0 +1,62 @@
+### New SaaS Flow (Product Trial)
+
+It has 2 doctypes.
+
+1. **Product Trial** - Hold the configuration for a specific product.
+2. **Product Trial Request** - This holds the records of request for a specific product from a user.
+
+#### Configure a new Product Trial
+- Create a new record in `Product Trial` doctype
+- **Details Tab**
+ - **Name** - should be a unique one and will be used as a id in signup/login flows. e.g. For `Frappe CRM` it could be `crm`
+ - **Published**, **Title**, **Logo**, **Domain**, **Release Group**, **Trial Duration (days)**, **Trial Plan** - as the name implies, all fields are mandatory.
+ - **Apps** - List of apps those will be installed on the site. First app should be `Frappe` in the list.
+- **Pooling Tab**
+ - **Enable Pooling** - Checkbox to enable/disable pooling. If you enable pooling, you will have standby sites and will be quick to provision sites.
+ - **Standby Pool Size** - The total number of sites that will be maintained in the pool.
+ - **Standby Queue Size** - Number of standby sites that will be queued at a time.
+- **Sign-up Details Tab**
+ - **Sign-up Fields** - If you need some information from user at the time of sign-up, you can configure this. Check the field description of this field in doctype.
+ - **E-mail Account** - If you want to use some specific e-mail account for the saas sign-up, you can configure it here
+ - **E-mail Full Logo** - This logo will be sent in verification e-mails.
+ - **E-mail Subject** - Subject of verification e-mail. You can put `{otp}` to insert the value in subject. Example - `{otp} - OTP for CRM Registration`
+ - **E-mail Header Content** - Header part of e-mail.
+ ```html
+
You're almost done!
+
Just one quick step left to get you started with Frappe CRM!
+ ```
+- **Setup Wizard Tab**-
+ - **Setup Wizard Completion Mode** -
+ - **auto** - setup wizard of site will be completed in background and after signup + setup, user will get direct access to desk or portal of app
+ - **manual** - after signup, user will be logged in to the site and user need to complete the setup wizard of framework
+ - **Setup Wizard Payload Generator Script** [only for **auto** mode] - Check the field description in doctype.
+
+ Sample Payload Script -
+ ```python
+ payload = {
+ "language":"English",
+ "country": team.country,
+ "timezone":"Asia/Kolkata",
+ "currency": team.currency,
+ "full_name": team.user.full_name,
+ "email": team.user.email,
+ "password": decrypt_password(signup_details.login_password)
+ }
+ ```
+ - **Create Additional System User** [only for **manual** mode] - If this is checked, we will add an additional system user with the team's information after creating a new site.
+
+#### FC Dashboard
+- UI/UX - The pages are available in https://github.com/frappe/press/tree/master/dashboard/src2/pages/saas
+- The required apis for these pages are available in https://github.com/frappe/press/blob/master/press/api/product_trial.py
+
+#### Billing APIs for Integration in Framework
+
+> [!CAUTION]
+> Changes in any of these APIs can cause disruption in on-site billing system.
+
+- All the required APIs for billing in site is available in https://github.com/frappe/press/tree/master/press/saas/api
+- These APIs use a different type of authentication mechanism. Check this readme for more info https://github.com/frappe/press/blob/master/press/saas/api/readme.md
+- Reference of integration in framework
+ - https://github.com/frappe/frappe/tree/develop/billing
+ - https://github.com/frappe/frappe/blob/develop/frappe/integrations/frappe_providers/frappecloud_billing.py
+
From 40647f941d7bdc5ffc7f6c5a760332f8c3682db9 Mon Sep 17 00:00:00 2001
From: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
Date: Thu, 21 Nov 2024 22:40:29 +0530
Subject: [PATCH 49/93] feat(saas): support to configure the redirection url
for site after login/signup (#2302)
---
dashboard/src2/pages/saas/LoginToSite.vue | 5 ++++-
press/saas/README.md | 1 +
press/saas/doctype/product_trial/product_trial.json | 12 ++++++++++--
press/saas/doctype/product_trial/product_trial.py | 5 +++++
4 files changed, 20 insertions(+), 3 deletions(-)
diff --git a/dashboard/src2/pages/saas/LoginToSite.vue b/dashboard/src2/pages/saas/LoginToSite.vue
index b097d4fb2bc..bbf2ed77ed5 100644
--- a/dashboard/src2/pages/saas/LoginToSite.vue
+++ b/dashboard/src2/pages/saas/LoginToSite.vue
@@ -152,7 +152,10 @@ export default {
method: 'get_login_sid',
onSuccess(data) {
let sid = data;
- let loginURL = `https://${this.$resources.siteRequest.doc.site}/desk?sid=${sid}`;
+ let redirectRoute =
+ this.$resources?.saasProduct?.doc?.redirect_to_after_login ??
+ '/desk';
+ let loginURL = `https://${this.$resources.siteRequest.doc.site}${redirectRoute}?sid=${sid}`;
this.isRedirectingToSite = true;
window.open(loginURL, '_self');
}
diff --git a/press/saas/README.md b/press/saas/README.md
index 2da0948ee35..470555f37ee 100644
--- a/press/saas/README.md
+++ b/press/saas/README.md
@@ -44,6 +44,7 @@ It has 2 doctypes.
}
```
- **Create Additional System User** [only for **manual** mode] - If this is checked, we will add an additional system user with the team's information after creating a new site.
+ - **Redirect To After Login** - After SaaS signup/login, user is directly logged-in to his site. By default, we redirect the user to desk of site. With this option, we can configure the redirect path. For example, for gameplan the path would be `/g`
#### FC Dashboard
- UI/UX - The pages are available in https://github.com/frappe/press/tree/master/dashboard/src2/pages/saas
diff --git a/press/saas/doctype/product_trial/product_trial.json b/press/saas/doctype/product_trial/product_trial.json
index 6064ce17317..4139b4f9cb6 100644
--- a/press/saas/doctype/product_trial/product_trial.json
+++ b/press/saas/doctype/product_trial/product_trial.json
@@ -36,8 +36,9 @@
"email_header_content",
"setup_wizard_tab",
"setup_wizard_completion_mode",
+ "setup_wizard_payload_generator_script",
"create_additional_system_user",
- "setup_wizard_payload_generator_script"
+ "redirect_to_after_login"
],
"fields": [
{
@@ -217,6 +218,13 @@
"fieldname": "create_additional_system_user",
"fieldtype": "Check",
"label": "Create Additional System User"
+ },
+ {
+ "default": "/desk",
+ "fieldname": "redirect_to_after_login",
+ "fieldtype": "Data",
+ "label": "Redirect To After Login",
+ "reqd": 1
}
],
"image_field": "logo",
@@ -231,7 +239,7 @@
"link_fieldname": "product_trial"
}
],
- "modified": "2024-11-19 16:12:29.471858",
+ "modified": "2024-11-21 22:13:33.724250",
"modified_by": "Administrator",
"module": "SaaS",
"name": "Product Trial",
diff --git a/press/saas/doctype/product_trial/product_trial.py b/press/saas/doctype/product_trial/product_trial.py
index c8553caab08..b54b0907bee 100644
--- a/press/saas/doctype/product_trial/product_trial.py
+++ b/press/saas/doctype/product_trial/product_trial.py
@@ -39,6 +39,7 @@ class ProductTrial(Document):
enable_pooling: DF.Check
logo: DF.AttachImage | None
published: DF.Check
+ redirect_to_after_login: DF.Data
release_group: DF.Link
setup_wizard_completion_mode: DF.Literal["manual", "auto"]
setup_wizard_payload_generator_script: DF.Code | None
@@ -56,6 +57,7 @@ class ProductTrial(Document):
"domain",
"trial_days",
"trial_plan",
+ "redirect_to_after_login",
)
USER_LOGIN_PASSWORD_FIELD = "user_login_password"
@@ -100,6 +102,9 @@ def validate(self):
if field.fieldtype != "Password":
frappe.throw(f"{self.USER_LOGIN_PASSWORD_FIELD} field should be of type Password")
+ if not self.redirect_to_after_login.startswith("/"):
+ frappe.throw("Redirection route after login should start with /")
+
def setup_trial_site(self, team, plan, cluster=None, account_request=None):
from press.press.doctype.site.site import get_plan_config
From 685d38048db83e94eed83b8292e6afff56c53dbd Mon Sep 17 00:00:00 2001
From: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
Date: Thu, 21 Nov 2024 23:01:08 +0530
Subject: [PATCH 50/93] chore(saas): update the body of login e-mail
---
press/saas/doctype/product_trial/product_trial.py | 12 ++++--------
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/press/saas/doctype/product_trial/product_trial.py b/press/saas/doctype/product_trial/product_trial.py
index b54b0907bee..2bb72292e89 100644
--- a/press/saas/doctype/product_trial/product_trial.py
+++ b/press/saas/doctype/product_trial/product_trial.py
@@ -350,15 +350,11 @@ def send_verification_mail_for_login(email: str, product: str, code: str):
print(f"Code : {code}")
print()
return
- product_trial = frappe.get_doc("Product Trial", product)
+ product_trial: ProductTrial = frappe.get_doc("Product Trial", product)
sender = ""
- subject = (
- product_trial.email_subject.format(otp=code)
- if product_trial.email_subject
- else "Verify your email for Frappe"
- )
+ subject = f"{code} - Verification Code for {product_trial.title} Login"
args = {
- "header_content": product_trial.email_header_content or "",
+ "header_content": f"
You have requested a verification code to login to your {product_trial.title} site. The code is valid for 5 minutes.
",
"otp": code,
}
if product_trial.email_full_logo:
@@ -370,7 +366,7 @@ def send_verification_mail_for_login(email: str, product: str, code: str):
sender=sender,
recipients=email,
subject=subject,
- template="saas_verify_account",
+ template="product_trial_verify_account",
args=args,
now=True,
)
From ef2b1ff3e55a9a637db26f432b55d462ec196180 Mon Sep 17 00:00:00 2001
From: Shadrak Gurupnor <30501401+shadrak98@users.noreply.github.com>
Date: Fri, 22 Nov 2024 09:50:52 +0530
Subject: [PATCH 51/93] fix: update next attempt date if exists
---
press/press/doctype/stripe_webhook_log/stripe_webhook_log.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/press/press/doctype/stripe_webhook_log/stripe_webhook_log.py b/press/press/doctype/stripe_webhook_log/stripe_webhook_log.py
index cf3991032ed..39ae096bbec 100644
--- a/press/press/doctype/stripe_webhook_log/stripe_webhook_log.py
+++ b/press/press/doctype/stripe_webhook_log/stripe_webhook_log.py
@@ -61,7 +61,7 @@ def before_insert(self):
"name",
)
- if self.event_type == "invoice.payment_failed" and self.invoice:
+ if self.event_type == "invoice.payment_failed" and self.invoice and payload.get("data", {}).get("object", {}).get("next_payment_attempt"):
next_payment_attempt_date = datetime.fromtimestamp(
payload.get("data", {}).get("object", {}).get("next_payment_attempt")
).strftime("%Y-%m-%d")
From 85c9f246f0aa84dd14abd07419b03390f7be1ec6 Mon Sep 17 00:00:00 2001
From: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
Date: Fri, 22 Nov 2024 10:10:01 +0530
Subject: [PATCH 52/93] chore(saas): update docs
---
press/saas/README.md | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/press/saas/README.md b/press/saas/README.md
index 470555f37ee..3764f5624a7 100644
--- a/press/saas/README.md
+++ b/press/saas/README.md
@@ -5,6 +5,11 @@ It has 2 doctypes.
1. **Product Trial** - Hold the configuration for a specific product.
2. **Product Trial Request** - This holds the records of request for a specific product from a user.
+#### How to know, which site is available for allocation to user ?
+
+In **Site** doctype, there will be a field `standby_for_product`, this field should have the link to the product trial (e.g. erpnext, crm)
+If `is_standby` field is checked, that site can be allocated to a user.
+
#### Configure a new Product Trial
- Create a new record in `Product Trial` doctype
- **Details Tab**
From d285e33e2078804d6d59a37aee1658e5dfa205f0 Mon Sep 17 00:00:00 2001
From: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
Date: Fri, 22 Nov 2024 12:03:06 +0530
Subject: [PATCH 53/93] fix(site-api): if app source is none, don't include it
- Sentry - PRESS-7WP
- While fetching app sources with repository_owner, branch filter, some apps can be in the installed_bench_apps list but may not available in app_sources due to filter. That raises the error.
---
press/api/site.py | 16 +++++++++-------
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/press/api/site.py b/press/api/site.py
index 0440b9c7878..0e7620af78c 100644
--- a/press/api/site.py
+++ b/press/api/site.py
@@ -1296,6 +1296,8 @@ def get_installed_apps(site, query_filters: dict | None = None):
installed_apps = []
for app in installed_bench_apps:
app_source = find(sources, lambda x: x.name == app.source)
+ if not app_source:
+ continue
app_source.hash = app.hash
app_source.commit_message = frappe.db.get_value("App Release", {"hash": app_source.hash}, "message")
app_tags = frappe.db.get_value(
@@ -1341,16 +1343,16 @@ def get_installed_apps(site, query_filters: dict | None = None):
app_source.app_title = marketplace_app_info.title
app_source.app_image = marketplace_app_info.image
- app_source.plan_info = frappe.db.get_value(
- "Marketplace App Plan",
- subscription.plan,
- ["price_usd", "price_inr", "name", "plan"],
- as_dict=True,
- )
+ # app_source.plan_info = frappe.db.get_value(
+ # "Marketplace App Plan",
+ # subscription.plan,
+ # ["price_usd", "price_inr", "name", "plan"],
+ # as_dict=True,
+ # )
app_source.plans = get_plans_for_app(app.app)
- app_source.is_free = app_source.plan_info.price_usd <= 0
+ # app_source.is_free = app_source.plan_info.price_usd <= 0
else:
app_source.subscription = {}
From 46bddaa314151a30c5f68887c8b74797604b0d69 Mon Sep 17 00:00:00 2001
From: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
Date: Fri, 22 Nov 2024 12:04:28 +0530
Subject: [PATCH 54/93] chore(site-api): revert the changes of local
environment
---
press/api/site.py | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/press/api/site.py b/press/api/site.py
index 0e7620af78c..ca5397a78d8 100644
--- a/press/api/site.py
+++ b/press/api/site.py
@@ -1343,16 +1343,16 @@ def get_installed_apps(site, query_filters: dict | None = None):
app_source.app_title = marketplace_app_info.title
app_source.app_image = marketplace_app_info.image
- # app_source.plan_info = frappe.db.get_value(
- # "Marketplace App Plan",
- # subscription.plan,
- # ["price_usd", "price_inr", "name", "plan"],
- # as_dict=True,
- # )
+ app_source.plan_info = frappe.db.get_value(
+ "Marketplace App Plan",
+ subscription.plan,
+ ["price_usd", "price_inr", "name", "plan"],
+ as_dict=True,
+ )
app_source.plans = get_plans_for_app(app.app)
- # app_source.is_free = app_source.plan_info.price_usd <= 0
+ app_source.is_free = app_source.plan_info.price_usd <= 0
else:
app_source.subscription = {}
From a74f3d74e94e6fd69965537202ae88fcfbcbc338 Mon Sep 17 00:00:00 2001
From: Balamurali M
Date: Thu, 21 Nov 2024 11:02:54 +0530
Subject: [PATCH 55/93] chore(SiteMigration): Correct docstring for plan
adjustment
---
press/press/doctype/site_migration/site_migration.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/press/press/doctype/site_migration/site_migration.py b/press/press/doctype/site_migration/site_migration.py
index 5c2f4a4cf18..8415331fb70 100644
--- a/press/press/doctype/site_migration/site_migration.py
+++ b/press/press/doctype/site_migration/site_migration.py
@@ -604,7 +604,7 @@ def downgrade_plan(self, site: "Site", dest_server: Server):
return None
def adjust_plan_if_required(self):
- """Change Plan to Unlimited if Migrated to Dedicated Server"""
+ """Update site plan from/to Unlimited"""
site: "Site" = frappe.get_doc("Site", self.site)
dest_server: Server = frappe.get_doc("Server", self.destination_server)
plan_change = None
From a08b7c775ae0ac7945ce5d1a295702864f66004b Mon Sep 17 00:00:00 2001
From: Balamurali M
Date: Fri, 22 Nov 2024 12:34:58 +0530
Subject: [PATCH 56/93] fix(DatabaseServer): Set myisam-recover-options to
FORCE
This ensures no .BAK files are created for when Error Log and other
myisam tables crash. We don't use them anyway. The stray .BAK files
prevent archive jobs from completing.
Also use newer `myisam-recover-options` instead of `myisam-recover`
https://mariadb.com/kb/en/myisam-system-variables/#myisam_recover_options
---
press/playbooks/roles/convert/templates/mariadb.cnf | 2 +-
press/playbooks/roles/mariadb/templates/mariadb.cnf | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/press/playbooks/roles/convert/templates/mariadb.cnf b/press/playbooks/roles/convert/templates/mariadb.cnf
index 307d3e9d472..dcf6d39931e 100644
--- a/press/playbooks/roles/convert/templates/mariadb.cnf
+++ b/press/playbooks/roles/convert/templates/mariadb.cnf
@@ -5,7 +5,7 @@ default-storage-engine = InnoDB
# MyISAM #
key-buffer-size = 32M
-myisam-recover = FORCE,BACKUP
+myisam-recover-options = FORCE
# SAFETY #
max-allowed-packet = 256M
diff --git a/press/playbooks/roles/mariadb/templates/mariadb.cnf b/press/playbooks/roles/mariadb/templates/mariadb.cnf
index 67cf36ee740..ea1d1d45182 100644
--- a/press/playbooks/roles/mariadb/templates/mariadb.cnf
+++ b/press/playbooks/roles/mariadb/templates/mariadb.cnf
@@ -5,7 +5,7 @@ default-storage-engine = InnoDB
# MyISAM #
key-buffer-size = 32M
-myisam-recover = FORCE,BACKUP
+myisam-recover-options = FORCE
# SAFETY #
max-allowed-packet = 256M
From 6769938f1933fc5283374b381a6b612ea9a58ca3 Mon Sep 17 00:00:00 2001
From: Balamurali M
Date: Fri, 22 Nov 2024 12:44:08 +0530
Subject: [PATCH 57/93] chore(MariaDBVariable): Add myisam_recover_options
This overrides the (previously) set default myisam_recover variable when both are set in
the config
---
press/fixtures/mariadb_variable.json | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/press/fixtures/mariadb_variable.json b/press/fixtures/mariadb_variable.json
index 10822cc71bc..eb538590221 100644
--- a/press/fixtures/mariadb_variable.json
+++ b/press/fixtures/mariadb_variable.json
@@ -370,5 +370,17 @@
"name": "net_write_timeout",
"set_on_new_servers": 0,
"skippable": 0
+ },
+ {
+ "datatype": "Str",
+ "default_value": "FORCE",
+ "doc_section": "server",
+ "docstatus": 0,
+ "doctype": "MariaDB Variable",
+ "dynamic": 0,
+ "modified": "2024-11-22 12:31:31.593757",
+ "name": "myisam_recover_options",
+ "set_on_new_servers": 0,
+ "skippable": 0
}
]
\ No newline at end of file
From d747e563bfc6fa933d66834dbb4f2aa3d3d41c2b Mon Sep 17 00:00:00 2001
From: Balamurali M
Date: Fri, 22 Nov 2024 12:53:02 +0530
Subject: [PATCH 58/93] refactor(MariaDBVariable): Remove set_on_new_servers
check from fixture
Cloud-init will update from config anyway
- Update convert playbook conf with same vars from mariadb.cnf
---
press/fixtures/mariadb_variable.json | 28 +++++++++----------
.../roles/convert/templates/mariadb.cnf | 10 +++++--
.../roles/mariadb/templates/mariadb.cnf | 7 +++--
3 files changed, 25 insertions(+), 20 deletions(-)
diff --git a/press/fixtures/mariadb_variable.json b/press/fixtures/mariadb_variable.json
index eb538590221..5d1b0e7d963 100644
--- a/press/fixtures/mariadb_variable.json
+++ b/press/fixtures/mariadb_variable.json
@@ -138,9 +138,9 @@
"docstatus": 0,
"doctype": "MariaDB Variable",
"dynamic": 1,
- "modified": "2023-09-22 08:38:16.807229",
+ "modified": "2024-11-22 12:51:29.101315",
"name": "tmp_disk_table_size",
- "set_on_new_servers": 1,
+ "set_on_new_servers": 0,
"skippable": 0
},
{
@@ -150,9 +150,9 @@
"docstatus": 0,
"doctype": "MariaDB Variable",
"dynamic": 1,
- "modified": "2024-03-05 12:55:47.689706",
+ "modified": "2024-11-22 12:52:00.473677",
"name": "extra_max_connections",
- "set_on_new_servers": 1,
+ "set_on_new_servers": 0,
"skippable": 0
},
{
@@ -162,9 +162,9 @@
"docstatus": 0,
"doctype": "MariaDB Variable",
"dynamic": 0,
- "modified": "2024-03-05 12:55:54.950776",
+ "modified": "2024-11-22 12:52:35.958089",
"name": "extra_port",
- "set_on_new_servers": 1,
+ "set_on_new_servers": 0,
"skippable": 0
},
{
@@ -174,9 +174,9 @@
"docstatus": 0,
"doctype": "MariaDB Variable",
"dynamic": 1,
- "modified": "2024-07-29 13:26:57.153685",
+ "modified": "2024-11-22 12:50:54.084797",
"name": "max_connections",
- "set_on_new_servers": 1,
+ "set_on_new_servers": 0,
"skippable": 0
},
{
@@ -270,9 +270,9 @@
"docstatus": 0,
"doctype": "MariaDB Variable",
"dynamic": 1,
- "modified": "2023-09-22 14:02:04.157025",
+ "modified": "2024-11-22 12:50:46.547631",
"name": "innodb_old_blocks_time",
- "set_on_new_servers": 1,
+ "set_on_new_servers": 0,
"skippable": 0
},
{
@@ -282,9 +282,9 @@
"docstatus": 0,
"doctype": "MariaDB Variable",
"dynamic": 1,
- "modified": "2023-09-22 08:38:25.704420",
+ "modified": "2024-11-22 12:51:34.994866",
"name": "max_allowed_packet",
- "set_on_new_servers": 1,
+ "set_on_new_servers": 0,
"skippable": 0
},
{
@@ -294,9 +294,9 @@
"docstatus": 0,
"doctype": "MariaDB Variable",
"dynamic": 1,
- "modified": "2024-03-28 15:23:30.635417",
+ "modified": "2024-11-22 12:50:58.514076",
"name": "max_statement_time",
- "set_on_new_servers": 1,
+ "set_on_new_servers": 0,
"skippable": 0
},
{
diff --git a/press/playbooks/roles/convert/templates/mariadb.cnf b/press/playbooks/roles/convert/templates/mariadb.cnf
index dcf6d39931e..e8ce58ee685 100644
--- a/press/playbooks/roles/convert/templates/mariadb.cnf
+++ b/press/playbooks/roles/convert/templates/mariadb.cnf
@@ -8,7 +8,6 @@ key-buffer-size = 32M
myisam-recover-options = FORCE
# SAFETY #
-max-allowed-packet = 256M
max-connect-errors = 1000000
innodb = FORCE
@@ -27,11 +26,15 @@ tmp-table-size = 32M
max-heap-table-size = 32M
query-cache-type = 0
query-cache-size = 0
-max-connections = 500
+max-connections = 200
thread-cache-size = 50
open-files-limit = 65535
table-definition-cache = 4096
table-open-cache = 10240
+tmp-disk-table-size = 5120M
+max-statement-time = 3600
+extra_port = 3307
+extra_max_connections = 5
# INNODB #
innodb-flush-method = O_DIRECT
@@ -39,9 +42,10 @@ innodb-log-files-in-group = 2
innodb-log-file-size = 512M
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
-innodb-buffer-pool-size = {{ (ansible_memtotal_mb * 0.685)|round|int }}M
+innodb-buffer-pool-size = {{ (ansible_memtotal_mb * 0.6)|round|int }}M
innodb-file-format = barracuda
innodb-large-prefix = 1
+innodb-old-blocks-time = 5000
collation-server = utf8mb4_unicode_ci
character-set-server = utf8mb4
character-set-client-handshake = FALSE
diff --git a/press/playbooks/roles/mariadb/templates/mariadb.cnf b/press/playbooks/roles/mariadb/templates/mariadb.cnf
index ea1d1d45182..9086a4228eb 100644
--- a/press/playbooks/roles/mariadb/templates/mariadb.cnf
+++ b/press/playbooks/roles/mariadb/templates/mariadb.cnf
@@ -8,7 +8,6 @@ key-buffer-size = 32M
myisam-recover-options = FORCE
# SAFETY #
-max-allowed-packet = 256M
max-connect-errors = 1000000
innodb = FORCE
@@ -29,13 +28,15 @@ tmp-table-size = 32M
max-heap-table-size = 32M
query-cache-type = 0
query-cache-size = 0
-max-connections = 500
+max-connections = 200
thread-cache-size = 50
open-files-limit = 65535
table-definition-cache = 4096
table-open-cache = 10240
tmp-disk-table-size = 5120M
-max-statement-time = 10800
+max-statement-time = 3600
+extra_port = 3307
+extra_max_connections = 5
# INNODB #
innodb-flush-method = O_DIRECT
From 820918ff44ad97addc26379720c3bd25d06e68ad Mon Sep 17 00:00:00 2001
From: Shadrak Gurupnor <30501401+shadrak98@users.noreply.github.com>
Date: Fri, 22 Nov 2024 16:42:24 +0530
Subject: [PATCH 59/93] fix: remove additional storage check
---
press/press/doctype/subscription/subscription.py | 3 ---
1 file changed, 3 deletions(-)
diff --git a/press/press/doctype/subscription/subscription.py b/press/press/doctype/subscription/subscription.py
index 371cddd6c96..e85364a2be4 100644
--- a/press/press/doctype/subscription/subscription.py
+++ b/press/press/doctype/subscription/subscription.py
@@ -137,9 +137,6 @@ def create_usage_record(self):
team = frappe.get_cached_doc("Team", self.team)
- if self.additional_storage:
- return None
-
if team.parent_team:
team = frappe.get_cached_doc("Team", team.parent_team)
From bbeb77664e4f3568f2c7c2b574932622779abc01 Mon Sep 17 00:00:00 2001
From: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
Date: Fri, 22 Nov 2024 17:46:13 +0530
Subject: [PATCH 60/93] chore(saas): add notes for important saas related apis
---
press/api/developer/saas.py | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/press/api/developer/saas.py b/press/api/developer/saas.py
index 5730f4074e8..60d73d0edd7 100644
--- a/press/api/developer/saas.py
+++ b/press/api/developer/saas.py
@@ -112,6 +112,17 @@ def get_trial_expiry(secret_key):
return api_handler.get_trial_expiry()
+"""
+NOTE: These mentioned apis are used for all type of saas sites to allow login to frappe cloud
+- request_login_to_fc
+- validate_login_to_fc
+- login_to_fc
+
+Don't change the file name or the method names
+It can potentially break the integrations.
+"""
+
+
@frappe.whitelist(allow_guest=True, methods=["POST"])
@rate_limit(limit=5, seconds=60)
def request_login_to_fc(domain: str):
From 9806b0fc970880fdda1b2bef672f1259379dc9e4 Mon Sep 17 00:00:00 2001
From: s-aga-r
Date: Sat, 23 Nov 2024 15:46:22 +0530
Subject: [PATCH 61/93] fix(spamd): Request Entity Too Large (#2276)
Co-authored-by: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
---
press/api/email.py | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/press/api/email.py b/press/api/email.py
index 4c56fd6bf8f..2bfd4a8dbed 100644
--- a/press/api/email.py
+++ b/press/api/email.py
@@ -145,10 +145,10 @@ def validate_plan(secret_key):
)
-def check_spam(message: str):
+def check_spam(message: bytes):
resp = requests.post(
"https://server.frappemail.com/spamd/score",
- {"message": message},
+ files={"message": message},
)
if resp.status_code == 200:
data = resp.json()
@@ -158,7 +158,7 @@ def check_spam(message: str):
SpamDetectionError,
)
else:
- log_error("Spam Detection: Error", data=resp.text, message=message)
+ log_error("Spam Detection: Error", data=resp.text, message=message.decode("utf-8"))
@frappe.whitelist(allow_guest=True)
@@ -173,8 +173,8 @@ def send_mime_mail(**data):
api_key, domain = frappe.db.get_value("Press Settings", None, ["mailgun_api_key", "root_domain"])
- message = files["mime"].read()
- check_spam(message.decode("utf-8"))
+ message: bytes = files["mime"].read()
+ check_spam(message)
resp = requests.post(
f"https://api.mailgun.net/v3/{domain}/messages.mime",
From 21a3f767afd01682ced368ce360f3ce39c604486 Mon Sep 17 00:00:00 2001
From: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
Date: Sat, 23 Nov 2024 16:15:53 +0530
Subject: [PATCH 62/93] chore(email): ignore spam check errors in case of mail
server update
---
press/api/email.py | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/press/api/email.py b/press/api/email.py
index 2bfd4a8dbed..e7789e5ca9e 100644
--- a/press/api/email.py
+++ b/press/api/email.py
@@ -146,19 +146,22 @@ def validate_plan(secret_key):
def check_spam(message: bytes):
- resp = requests.post(
- "https://server.frappemail.com/spamd/score",
- files={"message": message},
- )
- if resp.status_code == 200:
+ try:
+ resp = requests.post(
+ "https://server.frappemail.com/spamd/score",
+ files={"message": message},
+ )
+ resp.raise_for_status()
data = resp.json()
if data["message"] > 3.5:
frappe.throw(
"This email was blocked as it was flagged as spam by our system. Please review the contents and try again.",
SpamDetectionError,
)
- else:
- log_error("Spam Detection: Error", data=resp.text, message=message.decode("utf-8"))
+ except requests.exceptions.HTTPError as e:
+ # Ignore error, if server.frappemail.com is being updated.
+ if e.response.status_code != 503:
+ log_error("Spam Detection : Error", data=e)
@frappe.whitelist(allow_guest=True)
From 04ee6c577f4acf66bf5fb191d4a73ad658236ca6 Mon Sep 17 00:00:00 2001
From: Tanmoy Sarkar <57363826+tanmoysrt@users.noreply.github.com>
Date: Mon, 25 Nov 2024 12:13:27 +0530
Subject: [PATCH 63/93] feat: database user and permission manager (#2245)
---
dashboard/src/components/global/Badge.vue | 1 +
dashboard/src2/components/SiteActionCell.vue | 2 +-
.../components/SiteDatabaseAccessDialog.vue | 327 +++++++++------
.../SiteDatabaseAddEditUserDialog.vue | 393 ++++++++++++++++++
.../SiteDatabaseColumnsSelector.vue | 92 ++++
.../SiteDatabaseUserCredentialDialog.vue | 101 +++++
press/agent.py | 71 +++-
press/api/client.py | 1 +
press/fixtures/agent_job_type.json | 54 +++
press/hooks.py | 2 +
press/patches.txt | 1 +
...db_access_users_to_site_db_perm_manager.py | 33 ++
press/press/doctype/agent_job/agent_job.py | 17 +-
.../agent_job/agent_job_notifications.py | 3 +
press/press/doctype/site/site.json | 7 +-
press/press/doctype/site/site.py | 13 +-
.../__init__.py | 0
.../site_database_table_permission.json | 72 ++++
.../site_database_table_permission.py | 26 ++
.../doctype/site_database_user/__init__.py | 0
.../site_database_user/site_database_user.js | 70 ++++
.../site_database_user.json | 184 ++++++++
.../site_database_user/site_database_user.py | 299 +++++++++++++
.../test_site_database_user.py | 20 +
24 files changed, 1641 insertions(+), 148 deletions(-)
create mode 100644 dashboard/src2/components/site_database_user/SiteDatabaseAddEditUserDialog.vue
create mode 100644 dashboard/src2/components/site_database_user/SiteDatabaseColumnsSelector.vue
create mode 100644 dashboard/src2/components/site_database_user/SiteDatabaseUserCredentialDialog.vue
create mode 100644 press/patches/v0_7_0/move_site_db_access_users_to_site_db_perm_manager.py
create mode 100644 press/press/doctype/site_database_table_permission/__init__.py
create mode 100644 press/press/doctype/site_database_table_permission/site_database_table_permission.json
create mode 100644 press/press/doctype/site_database_table_permission/site_database_table_permission.py
create mode 100644 press/press/doctype/site_database_user/__init__.py
create mode 100644 press/press/doctype/site_database_user/site_database_user.js
create mode 100644 press/press/doctype/site_database_user/site_database_user.json
create mode 100644 press/press/doctype/site_database_user/site_database_user.py
create mode 100644 press/press/doctype/site_database_user/test_site_database_user.py
diff --git a/dashboard/src/components/global/Badge.vue b/dashboard/src/components/global/Badge.vue
index 4cfdf13db5a..193a2b303c3 100644
--- a/dashboard/src/components/global/Badge.vue
+++ b/dashboard/src/components/global/Badge.vue
@@ -29,6 +29,7 @@ export default {
Running: 'blue',
Pending: 'orange',
Failure: 'red',
+ Failed: 'red',
'Update Available': 'blue',
Enabled: 'blue',
'Awaiting Approval': 'orange',
diff --git a/dashboard/src2/components/SiteActionCell.vue b/dashboard/src2/components/SiteActionCell.vue
index 2963d572d11..3c2af195942 100644
--- a/dashboard/src2/components/SiteActionCell.vue
+++ b/dashboard/src2/components/SiteActionCell.vue
@@ -48,7 +48,7 @@ function getSiteActionHandler(action) {
'Restore from an existing site': defineAsyncComponent(() =>
import('./site/SiteDatabaseRestoreFromURLDialog.vue')
),
- 'Access site database': defineAsyncComponent(() =>
+ 'Manage database users': defineAsyncComponent(() =>
import('./SiteDatabaseAccessDialog.vue')
),
'Version upgrade': defineAsyncComponent(() =>
diff --git a/dashboard/src2/components/SiteDatabaseAccessDialog.vue b/dashboard/src2/components/SiteDatabaseAccessDialog.vue
index 73563742611..6676cb4a1eb 100644
--- a/dashboard/src2/components/SiteDatabaseAccessDialog.vue
+++ b/dashboard/src2/components/SiteDatabaseAccessDialog.vue
@@ -1,5 +1,8 @@
-
-
-
-
- Using an Analytics or Business Intelligence Tool
-
-
- Use following credentials with your analytics or business
- intelligence tool
-