Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add dedicated consensus RPC and Nova RPC for squids #326

Merged
merged 3 commits into from
Jul 30, 2024

Conversation

DaMandal0rian
Copy link
Member

@DaMandal0rian DaMandal0rian commented Jul 18, 2024

PR Type

enhancement, configuration changes


Description

  • Introduced rpc-squid-node-config and nova-squid-node-config configurations in main.tf.
  • Added new variables and outputs for rpc-squid and nova-squid nodes.
  • Updated DNS records and provisioner scripts to support new node types.
  • Removed configurations and scripts related to full_node.

Changes walkthrough 📝

Relevant files
Configuration changes
13 files
main.tf
Add RPC Squid and Nova Squid Node Configurations                 

resources/gemini-3h/main.tf

  • Added rpc-squid-node-config and domain-node-config configurations.
  • Introduced nova-squid-node-config with relevant parameters.
  • Updated instance types and counts for new configurations.
  • +22/-3   
    outputs.tf
    Update Outputs for RPC Squid and Nova Squid Nodes               

    resources/gemini-3h/outputs.tf

  • Added outputs for rpc-squid-node and nova-squid-node configurations.
  • Removed outputs for full-node.
  • +8/-3     
    variables.tf
    Add Variables for RPC Squid and Nova Squid Nodes                 

    resources/gemini-3h/variables.tf

  • Added variables for rpc-squid and nova-squid instance types and
    counts.
  • +4/-2     
    dns.tf
    Add DNS Records for RPC Squid and Nova Squid Nodes             

    templates/terraform/network-primitives/dns.tf

  • Added DNS records for rpc-squid and nova-squid-rpc.
  • Removed commented-out system domain resource.
  • +9/-18   
    domain_node_provisioner.tf
    Update Domain Node Provisioner Configuration                         

    templates/terraform/network-primitives/domain_node_provisioner.tf

  • Removed lines related to RELAYER_SYSTEM_ID and RELAYER_DOMAIN_ID.
  • +0/-2     
    instances.tf
    Add AWS Instances for RPC Squid and Nova Squid Nodes         

    templates/terraform/network-primitives/instances.tf

  • Added aws_instance resources for rpc_squid_node and nova_squid_node.
  • Removed aws_instance resource for full_node.
  • +70/-8   
    nova_squid_node_provisioner.tf
    Add Provisioner for Nova Squid Nodes                                         

    templates/terraform/network-primitives/nova_squid_node_provisioner.tf

    • Added provisioner configurations for nova_squid_node.
    +191/-0 
    outputs.tf
    Update Outputs for RPC Squid and Nova Squid Nodes               

    templates/terraform/network-primitives/outputs.tf

  • Added outputs for rpc_squid_node and nova_squid_node.
  • Removed outputs for full_node.
  • +26/-9   
    rpc_squid_node_provisioner.tf
    Add Provisioner for RPC Squid Nodes                                           

    templates/terraform/network-primitives/rpc_squid_node_provisioner.tf

  • Added provisioner configurations for rpc_squid_node.
  • Removed provisioner configurations for full_node.
  • +49/-29 
    variables.tf
    Add Variables for RPC Squid and Nova Squid Nodes                 

    templates/terraform/network-primitives/variables.tf

  • Added variables for rpc-squid-node-config and nova-squid-node-config.
  • +43/-3   
    create_domain_node_compose_file.sh
    Update Domain Node Compose File Script                                     

    templates/scripts/create_domain_node_compose_file.sh

    • Updated NRIA_DISPLAY_NAME to include DOMAIN_PREFIX.
    +1/-1     
    create_full_node_compose_file.sh
    Remove Full Node Compose File Script                                         

    templates/scripts/create_full_node_compose_file.sh

    • Removed script for creating full node compose file.
    +0/-106 
    create_rpc_node_compose_file.sh
    Update RPC Node Compose File Script                                           

    templates/scripts/create_rpc_node_compose_file.sh

    • Updated NRIA_DISPLAY_NAME to include DOMAIN_PREFIX.
    +1/-1     
    Documentation
    1 files
    README.md
    Update README for RPC Squid Node                                                 

    resources/README.md

    • Updated references from full_node to rpc-squid_node.
    +3/-3     

    💡 PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    Copy link

    PR Reviewer Guide 🔍

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Key issues to review

    Configuration Duplication
    The configuration blocks for rpc-squid-node-config and domain-node-config seem to have repeated parameters such as docker-org, docker-tag, disk-volume-size, and disk-volume-type. Consider refactoring these into a shared configuration or using a base configuration to reduce duplication and potential errors in updates.

    Security Group Reference
    The security group reference aws_security_group.network_sg.id is commented out in the dependency list for aws_instance.nova_squid_node. Ensure that this is intentional and that appropriate security groups are assigned.

    Copy link

    github-actions bot commented Jul 18, 2024

    PR Code Suggestions ✨

    CategorySuggestion                                                                                                                                    Score
    Best practice
    Correct the variable naming to use underscores for consistency and to avoid parsing errors

    Ensure that the variable rpc-squid-node-config.prune is properly named with
    underscores instead of hyphens to follow Terraform's naming conventions and avoid
    potential issues in parsing.

    templates/terraform/network-primitives/rpc_squid_node_provisioner.tf [67]

    -count = var.rpc-squid-node-config.prune ? length(local.rpc_squid_nodes_ip_v4) : 0
    +count = var.rpc_squid_node_config.prune ? length(local.rpc_squid_nodes_ip_v4) : 0
     
    Suggestion importance[1-10]: 10

    Why: Following Terraform's naming conventions by using underscores instead of hyphens in variable names prevents potential parsing issues and maintains consistency across the codebase.

    10
    Improve the predictability and simplicity of list handling by using tolist

    Replace the use of flatten function with tolist to ensure that the output is always
    a list type, which is more predictable in handling outputs from resources in
    Terraform.

    templates/terraform/network-primitives/rpc_squid_node_provisioner.tf [2-4]

    -rpc_squid_nodes_ip_v4 = flatten([
    -  [aws_instance.rpc_squid_node.*.public_ip]
    -])
    +rpc_squid_nodes_ip_v4 = tolist(aws_instance.rpc_squid_node.*.public_ip)
     
    Suggestion importance[1-10]: 9

    Why: Using tolist instead of flatten ensures that the output is always a list type, which is more predictable and aligns better with Terraform's handling of resource outputs. This change improves code reliability and readability.

    9
    Update the deployment version to be dynamic or incremented to improve configuration management

    It is recommended to set deployment-version dynamically or increment it to reflect
    changes or updates in deployment configurations. Using a static value like 0 can
    lead to issues in environments where version tracking or rollback functionalities
    are required.

    resources/gemini-3h/main.tf [41]

    -deployment-version = 0
    +deployment-version = var.deployment_version
     
    Suggestion importance[1-10]: 8

    Why: Making the deployment version dynamic or incremented is a best practice for configuration management, allowing for better version tracking and rollback capabilities. This suggestion addresses a significant improvement in maintainability and deployment management.

    8
    Enhance the robustness of script execution by handling potential errors

    Add error handling for the remote-exec provisioner to ensure that the script
    execution errors are caught and handled properly.

    templates/terraform/network-primitives/rpc_squid_node_provisioner.tf [60-61]

     inline = [
    +  "set -e",
       "sudo bash /home/${var.ssh_user}/subspace/installer.sh",
       "bash /home/${var.ssh_user}/subspace/acme.sh",
    +  "set +e",
     ]
     
    Suggestion importance[1-10]: 8

    Why: Adding error handling to the remote-exec provisioner improves the robustness of the script execution, ensuring that any errors are caught and handled properly, which is a best practice for reliable deployments.

    8
    Security
    Improve security by handling sensitive data more securely in script operations

    Use a more secure approach for handling sensitive data by avoiding the direct echo
    of sensitive keys into environment files.

    templates/terraform/network-primitives/rpc_squid_node_provisioner.tf [167]

    -"echo NODE_KEY=$(sed -nr 's/NODE_${count.index}_KEY=//p' /home/${var.ssh_user}/subspace/node_keys.txt) >> /home/${var.ssh_user}/subspace/.env",
    +"NODE_KEY=$(sed -nr 's/NODE_${count.index}_KEY=//p' /home/${var.ssh_user}/subspace/node_keys.txt)",
    +"echo NODE_KEY=${NODE_KEY} >> /home/${var.ssh_user}/subspace/.env",
     
    Suggestion importance[1-10]: 10

    Why: Handling sensitive data more securely by avoiding direct echo commands reduces the risk of exposing sensitive information, enhancing the overall security of the deployment process.

    10
    Maintainability
    Make the pruning of nova-squid nodes configurable through a variable

    The prune configuration in the null_resource for prune-nova-squid-nodes should be
    conditional based on a variable, allowing it to be enabled or disabled without
    modifying the code directly.

    templates/terraform/network-primitives/nova_squid_node_provisioner.tf [68]

    -count      = var.nova-squid-node-config.prune ? length(local.nova_squid_nodes_ip_v4) : 0
    +count      = var.enable_prune ? length(local.nova_squid_nodes_ip_v4) : 0
     
    Suggestion importance[1-10]: 9

    Why: Making the prune configuration conditional based on a variable improves maintainability and flexibility, allowing for easier adjustments without code changes. This is a significant improvement for operational efficiency.

    9
    Scalability
    Modify DNS record naming to handle scaling of nova-squid nodes effectively

    Ensure that the DNS records for nova-squid nodes are correctly configured to handle
    potential scaling. The current configuration may not correctly handle multiple
    instances due to the static nature of the count.index usage.

    templates/terraform/network-primitives/dns.tf [21-27]

     resource "cloudflare_record" "nova-squid-rpc" {
       count   = length(local.nova_squid_nodes_ip_v4)
       zone_id = data.cloudflare_zone.cloudflare_zone.id
    -  name    = "${var.nova-squid-node-config.domain-prefix}-${count.index}.${var.network_name}"
    +  name    = "${var.nova-squid-node-config.domain-prefix}-${format("%03d", count.index)}.${var.network_name}"
       value   = local.nova_squid_nodes_ip_v4[count.index]
       type    = "A"
     }
     
    Suggestion importance[1-10]: 8

    Why: Using a formatted index for DNS record names ensures that the naming convention can handle scaling and multiple instances effectively, which is crucial for scalability and avoiding potential conflicts.

    8
    Performance
    Adjust the IOPS and throughput settings based on the instance type and load expectations

    To improve the security and manageability of the instance, consider using a more
    granular approach for the ebs_block_device configuration. Specifically, adjust the
    iops and throughput based on the instance type and expected load.

    templates/terraform/network-primitives/instances.tf [144-148]

     ebs_block_device {
       device_name = "/dev/sda1"
    -  volume_size = var.rpc-squid-node-config.disk-volume-size
    -  volume_type = var.rpc-squid-node-config.disk-volume-type
    -  iops        = 3000
    -  throughput  = 250
    +  volume_size = var.rpc-squid-node-config.disk-volume_size
    +  volume_type = var.rpc-squid-node-config.disk-volume_type
    +  iops        = var.custom_iops
    +  throughput  = var.custom_throughput
     }
     
    Suggestion importance[1-10]: 7

    Why: Customizing IOPS and throughput based on instance type and expected load can enhance performance and resource utilization. This suggestion is beneficial but not critical, as the current settings may still function adequately for some use cases.

    7

    @DaMandal0rian DaMandal0rian requested a review from vedhavyas July 18, 2024 19:55
    @DaMandal0rian
    Copy link
    Member Author

    @vedhavyas can i get a review here please, ty! 🙏

    @DaMandal0rian DaMandal0rian merged commit 0c87d88 into main Jul 30, 2024
    1 check passed
    @DaMandal0rian DaMandal0rian deleted the rpc-squids-tf branch July 30, 2024 13:00
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    1 participant